Thursday, March 3, 2011

Flapjax - Functional Reactive Programming in Javascript

[This preview is posted here during the transition phase; once the transition is complete I will stop posting here] 

What if programming can be as simple as writing spreadsheet formulas?
Now - there are some complex spreadsheets out there for sure, but the power of spreadsheet comes from that the values are self-updating, i.e. changing value in one cell will automatically cause all its dependent cells to reflect the change.  If spreadsheets cannot do the auto-propagation, even the simplest spredsheet will be much more complex.
Most programming languages do not have the auto-propagating capability, but the ones that do are called reactive programming languages.  And if the language is also functional, then it's called a functional reactive programming language.
Since most of the mainstream languages did not have reactive features built-in, the feature need to be "bolted on", as a library or a transformer.  The one for Javascript is called Flapjax.

Tuesday, March 1, 2011

Administrivia: Moving of Weblambda to New Blog

I am consolidating my blogging over to my personal site,, in the near future.

The main site will contain more than just scheme development - if that interests you, that's great.  You can grab the whole feed from

In case you just want to have scheme-related posts - use (RSS feed at

There will be time to make the switch - I will be linking from here to the new site for the next few posts, before I fully make the move.


Sunday, July 18, 2010

BZLIB/SHP Web API & Converters

The previous post describes the basics on how to use the web API.  This post will focus on integrating your module with the web API.

As shown before, you can create a web API by creating an SHP script as follows:

;; /api/add2 
(:api-args (a number?) (b number?)) 
(+ a b) 
Where both a and b are validated as number?.  It would be nice if we can validate any type of scheme values, as long as the value can be created via the request input. 

For example - let's say that you have a struct with the following definition:
(define-struct foo (bar baz)) 
We want to do the following:
(:api-args (foo foo?)) ;; takes in the foo struct 
And let web API handle the rest.   This is achieved via converters.


The mappings between the request and the api args are done via converters, which maps the parameter key against the type's test function (such as number?).   And when you want to use the converter in the api-args expression, you specify the <test?> function in the parameter position in one of the following forms:

Thursday, July 8, 2010

BZLIB/SHP.plt 0.4 now available - Web API

A new version of SHP.plt is now available via planet.  This is a major rewrite of SHP and provides two main upgrades:
  • a "web API" interface - your web script can now be exposed as an "API" (think XMLRPC/JSON), and it automatically works with either XMLRPC or JSON (details below)
  • general performance enhancement - the scripts are now compiled and cached to reduce disk IO.  If the scripts are updated then they are automatically recompiled
As usual, the code is released under LGPL.


(require (planet bzlib/shp:1:3)) 

SHP requires some newer dependencies (bzlib/base:1:6, bzlib/date:1:3, bzlib/parseq:1:3, bzlib/xml:1:3, bzlib/mime:1:0), and the current versions of PLT Scheme and Racket have issues with version dependencies (the link: module mismatch bug), so you might have to clear out the planet cache and recompile them again.

As usual, SHP comes with a small example site that you can play with under the example sub directory - cd to the example directory and run (require "") will start the example site.  The example site is still just trivial code right now - it will eventually be enhanced and separated into its own package.

Cached Compiled Script

All of the scripts are now compiled and cached.  This has some potential performance benefit, since we will only access the file content when the file timestamp changes (meaning the file has been touched and/or modified).  As this is a non-visible feature, we won't spend much time discussing it, except to note that the change is not just done for performance reasons - it is also done to enable and simplify the design of web api, which is discussed below.


Under the example site you can find the script shp/api/add2, which contains the following:

;; -*- scheme -*- -p 
(:api-args (a number?) (b number?)) 
(+ a b)

This is the new *web api* - it takes in 2 numbers, a and b, and return the added result. To write an api script, you must use the :api-args expression, and then supply the arguments inside. The arguments can be specified in the following forms:

(:api-args a b) ;; both a & b are non-validating and you get what's passed in

(:api-args (a number?) (b string?)) ;; a expects a number, and b expects a string 

(:api-args (a number? 3) (b number? 5)) ;; a & b both expect numbers, and both have default values if they are not passed in (a defaults to 3, and b defaults to 5). 

When you run the example site you can access the api via the following http call:

GET /api/add2 HTTP/1.0 

When running the above in browser you should get back an XMLRPC response:

Content-Type: text/xml; charset=utf-8 

<string>required: a</string>

XMLRPC is the default response mode for web APIs.  What it returns by default as shown above is an error message, because neither a or b is passed in.

To pass in the values - you just need to specify them in the query string as following:

Monday, January 18, 2010

BZLIB/PLANET.plt - A Local PLANET Proxy/Repository Server

PLT's planet system is great - if you want to install a planet module, you just declare it in the code, and it will automatically manage the download and the install for you, without you having to separately run another module installation programs like Perl's CPAN, Ruby's GEM, PHP's PEAR, etc.

However, planet can still be improved. Specifically, as there is currently only a single central repository, you might experience some inconvenient server outage from time to time. And it is difficult to take advantage of the planet's automatic code distribution power unless you plan on releasing the code for public consumption.

That is - until today.

BZLIB/PLANET.plt is designed to solve exactly the issue of a single planet repository. Going forward, you can install bzlib/planet and run a local planet proxy and repository. bzlib/planet is, of course as usual, available via planet under LGPL ;)


bzlib/planet contains a proxy server that you can setup and run. The first thing to do is to install it via require:

(require (planet bzlib/planet/proxy)) 

Friday, January 8, 2010

BZLIB/PARSEQ.plt - (4) Token-Based Parsers API

Previously we have looked at fundamental parsers and the combinators API, as well as common parsers for character, number, and string, now it is time to look at token-based parsers provided by bzlib/parseq.

A huge class of parsing involves tokenizing the streams by skipping over whitespaces. For example, if we want to parse for a list of 3 integer, separated by comma, we currently have to write:

(define three-int-by-comma 
  (seq whitespaces 
       i1 <- integer 
       i2 <- integer 
       i3 <- integer 
       (return (list i1 i2 i3)))) 
The code above looks messy, and it would be nice if we do not have to explicitly specify the parsing of whitespaces. token allows us to abstract away the parsing of whitespaces:

(define (token parser (delim whitespaces)) 
  (seq delim 
       t <- parser 
       (return t))) 
The above code can now be rewritten as:

(define three-int-by-comma2 
  (seq i1 <- (token integer) 
       (token #\,) 
       i2 <- (token integer) 
       (token #\,) 
       i3 <- (token integer) 
       (return (list i1 i2 i3)))) 
Which looks a lot better. But given tokenizing is such a common parsing task, we have a shorthand for the above called tokens:

(define-macro (tokens . exps) 
  (define (body exps) 
    (match exps 
      ((list exp) (list exp)) 
      ((list-rest v '<- exp rest) 
       `(,v <- (token ,exp) . ,(body rest)))
      ((list-rest exp rest) 
       `((token ,exp) . ,(body rest)))))
  `(seq . ,(body exps)))
Which will reduce the above parsing to the following:

(define three-int-by-comma3
  (tokens i1 <- integer 
          i2 <- integer 
          i3 <- integer 
          (return (list i1 i2 i3)))) 
There is a case insensitive version of tokens called tokens-ci that allows the character and string token to be parsed in case insensitive fashion.

Besides tokenizing, another common need in token-based parsing is to handle delimited sets. In the above example, the 3 integers are delimited by commas. delimited generalize the pattern:

(define (delimited parser delim) 
  (tokens v <- parser 
          v2 <- (zero-many (tokens v3 <- delim
                                   v4 <- parser
                                   (return v4)))
          (return (cons v v2))))
The following parses a list of comma-delimited integers:

(delimited integer #\,) 
Another common pattern is to parse for brackets that surrounds the value that you need. Just about all programming languages have such constructs. And bracket handles such parses:

(define (bracket open parser close) 
  (tokens open
          v <- parser 
          (return v))) 
And bracket/delimited combines the case where you need to parse a bracketed delimited values:

(define (bracket/delimited open parser delim close) 
  (tokens open ;; even the parser is optional...  
          v <- (zero-one (delimited parser delim) '()) 
          (return v)))

That's it for the bzlib/parseq API. If you find anything missing, please let me know.


BZLIB/PARSEQ.plt - (3) Common Parsers API

Previously we have looked at fundamental parsers and the combinators API, now it is time to look at some common parsers provided by bzlib/parseq.

In this case, since we are constructing these parsers on top of the fundamental parsers and combinators, we will show the definitions accordingly.

Character Category Parsers

digit is a character between #\0 and #\9.

(define digit (char-between #\0 #\9)) 
not-digit is a character not between #\0 and #\9.

(define not-digit (char-not-between #\0 #\9))
lower-case is a character beween #\a and #\z.

(define lower-case (char-between #\a #\z)) 
upper-case is a character between #\A and #\Z.

(define upper-case (char-between #\A #\Z))
alpha is either an lower-case or upper-case character.

(define alpha (choice lower-case upper-case)) 
alphanumeric is either an alpha character or a digit character.

(define alphanumeric (choice alpha digit)) 
whitespace is either a space, return, newline, tab, or vertical tab.

(define whitespace (char-in '(#\space #\return #\newline #\tab #\vtab)))
not-whitespace is a character that is not a whitespace.

(define not-whitespace (char-not-in '(#\space #\return #\newline #\tab #\vtab)))
whitespaces parses for zero or more whitespace characters:

(define whitespaces (zero-many whitespace))
ascii is a charater bewteen 0 to 127:

(define ascii (char-between (integer->char 0) (integer->char 127)))
word is either an alphanumeric or an underscore:

(define word (choice alphanumeric (char= #\_)))
not-word is a character that is not a word:

(define not-word (char-when (lambda (c) 
                              (not (or (char<=? #\a c #\z)
                                       (char<=? #\A c #\Z)
                                       (char<=? #\0 c #\9) 
                                       (char=? c #\_))))))
Finally, newline parses for either CR, LF, or CRLF:

(define newline 
  (choice (seq r <- (char= #\return) 
               n <- (char= #\newline)
               (return (list r n)))
          (char= #\return)
          (char= #\newline)))

Number Parsers

sign parses for either + or -, and defaults to +.

(define sign (zero-one (char= #\-) #\+))
natural parses for 1+ digits:

(define natural (one-many digit)) 
decimal parses for a number with decimal points:

(define decimal (seq number <- (zero-many digit)
                     point <- (char= #\.)
                     decimals <- natural 
                     (return (append number (cons point decimals)))))
positive parses for either natural or decimal. Note decimal needs to be placed first since natural will succeed when parsing a decimal:

(define positive (choice decimal natural)) 
The above parsers returns the characters that represents the positive numbers. To get it to return numbers, as well as parsing for both positive and negative numbers, we have a couple of helpers:

;; make-signed will parse for the sign and the number.
(define (make-signed parser)
  (seq +/- <- sign
       number <- parser 
       (return (cons +/- number)))) 

;; make-number will convert the parsed digits into number. 
(define (make-number parser)
  (seq n <- parser 
       (return (string->number (list->string n)))))
Then natural-number parses and returns a natural number:

(define natural-number (make-number natural))
integer will parse and returns an integer (signed):

(define integer (make-number (make-signed natural))) 
positive-number will parse and return a positive number (integer or real):

(define positive-number (make-number positive)) 
real-number will parse and return a signed number, integer or real:

(define positive-number (make-number positive)) 

String Parsers

The following parsers parses for quoted string and returns the inner content as a string.

escaped-char parses for characters that were part of an escaped sequence. This exists for characters such as \n (which should return a #\newline), and character such as \" (which should return just "):

(define (escaped-char escape char (as #f)) 
  (seq (char= escape) 
       c <- (if (char? char) (char= char) char)
       (return (if as as c)))) 

;; e-newline 
(define e-newline (escaped-char #\\ #\n #\newline)) 

;; e-return 
(define e-return (escaped-char #\\ #\r #\return)) 

;; e-tab 
(define e-tab (escaped-char #\\ #\t #\tab)) 

;; e-backslash 
(define e-backslash (escaped-char #\\ #\\))
quoted parses for the quoted string pattern (including escapes):

;; quoted 
;; a specific string-based bracket parser 
(define (quoted open close escape)
  (seq (char= open) 
       atoms <- (zero-many (choice e-newline 
                                   (escaped-char escape close) 
                                   (char-not-in  (list close #\\)))) 
       (char= close)
       (return atoms)))
make-quoted-string abstracts the use of quoted.

(define (make-quoted-string open (close #f) (escape #\\)) 
  (seq v <- (quoted open (if close close open) escape)
       (return (list->string v))))
Then single-quoted-string and double-quoted-string look like the following:

(define single-quoted-string (make-quoted-string #\'))

(define double-quoted-string (make-quoted-string #\"))
Finally, quoted-string will parse both single-quoted-string and double-quoted-string:

(define quoted-string 
  (choice single-quoted-string double-quoted-string))

That is it for now - we will talk about parsing tokens next. Enjoy.