Including other files like PHP

If we want to develop complex scripts we will need to be able to modularize the pages so we are not completely repeating ourselves, so we want to be able to include other scripts!

We should have something like the following:

(include! "/path/to/script/file")

The include would then be embeddable within the output xexprs, or just for the side effects (for example, the include file contains the common required modules).

Although relative path seem interesting - we want include! to work as similar to the script dispatcher as possible, so we will only support the full path.

Getting the Root Path

Currently the root path is held in the closure of make-shp-handler so is inaccessible outside its scope. We can pass store the value in a parameter, but as previously noted, if we started to increase the number of configuration values it's probably best to store the whole value into a configuration object. It's time to create that configuration object.

In a way - the configuration object basically contains the settings of the server, so we'll call this the $server object. And since we are in schemeland, there is no reason this object should just hold configuration values - we can have it hold the server (well - the shp handler) itself, so it's full invokable.

(define $server (make-parameter #f))

The simplest way to do so is to convert shp-handler into a struct, the swiss army knife of PLT Scheme:

(define-struct shp-handler (path default not-found)
  #:property prop:procedure
  (lambda ($struct request)
    (parameterize (($pathinfo ($pathinfo))
                   ($request request)
                   ($server $struct))
      (let ((proc (evaluate-terms 
                   (file->values 
                    (url->shp-path (request-uri request))))))
        (make-response (proc request))))))

With the above definition we now can make a shp-handler by calling make-shp-handler and use it as a procedure (we also need to remove the previous make-shp-handler to avoid name conflict). And since we parameterize $server to $struct itself, we now have access to the shp-handler structure within the scope of the request.

Since we now can access the config values through accessors such as shp-handler-path, shp-handler-default, and shp-handler-notfound within the scope, we no longer have to pass them around through function parameters, so the following functions have their signatures changed.

;; partially matching the path from the beginning of the path
(define (normalize-partial-path segments)
  (define (helper rest path default not-found)
    ;; if we did not find any match return not-found 
    (cond ((null? path) (build-path path not-found)) 
          ;; then we test to see if this is a directory path & whether the default file exists for the directory
          ((and (no-extension? (car rest))
                (file-exists? (build-path path (car rest) default)))
           ($pathinfo (cdr rest)) ;; updating the pathinfo so it can be accessed.
           (build-path path (car rest) default))
          ((file-exists? (build-path path (car rest))) ;; otherwise the segment is a file and see if it exists... 
           ($pathinfo (cdr rest)) ;; udpdating the pathinfo so it can be accessed.
           (build-path path (car rest)))
          (else
           (helper (cdr rest) (build-path path (car rest)) default not-found))))
  (helper segments (shp-handler-path ($server)) (shp-handler-default ($server)) (shp-handler-not-found ($server))))

(define (segments->path segments (partial? #t))
  ;; test for full path first and then partial path. 
  (let ((script (apply build-path (shp-handler-path ($server)) segments)))
    (cond ((file-exists? script) script)
          ((and (directory-exists? script)
                (file-exists? (build-path script default)))
           (build-path script default))
          (partial? 
           (normalize-partial-path segments)) 
          (else 
           (build-path script (shp-handler-not-found ($server)))))))

(define (url->shp-path url)
    (segments->path (url->path-segments url) #t))

For those who love function purity it looks bad - but scheme is a practical language, and SHP will not enforce the concept of pure functional programming.

And include! now looks like this:

(define (path->segments path)
  (filter (lambda (path) 
            (not (equal? path "")))
          (regexp-split #px"\\/" path)))

(define (include! path) 
  (let ((proc (evaluate-terms
               (file->values
                (segments->path (path->segments path) #f)))))
    (proc ($request))))

Pure Side Effect Scripts


For scripts that exists primarily for side effects (such as a common require module list), because we need to generate a procedure (even if it's a dummy procedure), we'll insert a dummy return value (i.e. an empty string) to satisfy the procedure creation rules if it does not exists after the filtering.

;; terms->exps 
(define (terms->exps terms)
  (let ((exps (filter (compose not require-exp?) terms)))
    (if (null? exps) 
        '("") ;; ensure there is at least one exp in the lambda.
        exps)))

;; abstract the eval process
(define (evaluate-terms terms)
  (require-modules! terms) ;; first register the required modules 
  ;; then we filter out the required statement and evaluate the rest of the terms as a proc. 
  (eval `(lambda (request) 
           . ,(terms->exps terms))
        handler-namespace))

With this we can have a script that only contains side effects without having to worry about it returning any value, because the platform takes care of it for us.

Bug - Evaluation Order


The code as written above has one subtle bug, and that is the evaluation order of the include file. Basically - if you include a common required module script, the first time the script gets evaluated is when the containing script gets compiled, but because the definition resides inside the inner include, it does not get fully evaluated until the compilation failed. So the first time around when you are executing the script you will get:

reference to an identifier before its definition: ...

We'll talk about how to solve this problem in the following post, stay tuned.