Managing your Javascripts through SHP

If you are doing extensive AJAX development (which many people are these days), you probably are writing quite a bit of OOP Javascript code, along with using libraries such as Prototype, jQuery, Dojo, etc.

The trouble with javascript is that it lacks modern module management facilities that we have come to expect from programming languages, especially given its importance in the browser world. You either develop all of your code in one large script, or you risk the browser needing to download numerous small scripts, which is inefficient. Furthermore, if you plan on using javascript compressors such as the YUI compressor, if you make updates to the source script you would have to recopmress the code manually.

It would be nice to have a tool that can help with the above issues:

• allows you to code in multiple small scripts, but automatically combine the smaller scripts into a single script as you specify
  
• automatically helps you recompress the code if you make changes to them, so you just need to worry about the source (which will not be served directly unless specified)
  

Let's look at what we need to accomplish the goal:

• ability to load and combine multiple files into a single response
  
• ability to detect timestamp of the source and the compressed scripts, and re-compress if the source script's timestamp is newer than the compressed scripts

You might have come across "script builders" such as jQuery UI, and Mootools - you can use them as a mental model for the goal (for your own scripts!).

Compressor Choice

Currently the best choice of javascript compressor appears to be YUI compressor (if you are not trying to obfuscate the scripts too much) as it provides the best combination of compression ratio as well as the decompression speed (obfuscators such as Packer loses speed during the unpack phase), so it is our choice for compressor. It is licensed under BSD license, so we can redistribute the binary without issues. In the future we might add support for other compressors, but for now it's out of scope.

Prerequisite

YUI compressor requires Java (version >= 1.4), which you'll have to install separately and make it available in the system path.

First Attempt - a Single Javascript File
Let's see if we can get a simple version of this up and running - let's first just serve out a javascript file sitting somewhere in our system. No compressions. It is quite straight forward.

(define (make-loader root-path)
(lambda (path)
  (open-input-file (build-path root-path path))))

The basic signature will be make-loader, which will take a path and returns a procedure we can then use to load the file. If the file does not exist an appropriate error will be thrown (which we'll have to handle once integrating into servlets). We'll also have to handle setting the content-type during integration.

Second Attempt - Multiple Javascript Files

Now let's see if we can serve multiple javascript files. PLT Scheme makes this easy with input-port-append:

(require scheme/port)
(define (make-loader root-path)
(lambda (path . paths)
  (define (helper path)
    (open-input-file (build-path root-path path)))
  (apply input-port-append #t (map helper (cons path paths)))))

All of the input-port are now appended together, so when we exhausted the data from the first port, we'll then retrieve the data from the second port, so on.

Adding Compression

So far, so good. Let's now try to see if we can add compression in here.

Let's just say that the YUI compressor live in the same directory as the module, and Java is in the path. And all of the compressed script's path is the same as the source path, with the addition of ".min":

(require scheme/system scheme/string)
(define (make-loader root-path) 
  (lambda (path . paths) 
    (define (min-path-helper full-path)
      (string->path (string-append (path->string full-path) ".min")))
    (define (path-helper path)
      (build-path root-path path))
    (define (helper path)
      (let* ((path (path-helper path))
             (min-path (min-path-helper path)))
        (when (or (not (file-exists? min-path))
                  (< (file-or-directory-modify-seconds min-path)
                     (file-or-directory-modify-seconds path)))
          (system (string-join (list "java" 
                                     "-jar" 
                                     (path->string (build-path (this-expression-source-directory)
                                                               "yuicompressor.jar"))
                                     "-o" (path->string min-path) (path->string path))
                               " ")))
        (open-input-file min-path)))
    (apply input-port-append #t (map helper (cons path paths)))))

The only thing we need to make sure to handle is the error that could result from the system - it returns #f when the call failed, so we need something a bit more verbose (including error messages) in order to throw error about the failed compilation (it is best to have such error messages show up during your development, rather than during production).

Allright - the next step would be for us to integrate this into SHP. Stay tuned.