Include Other Files - Solve the Evaluation Order Bug

If we want the evaluation order to be done correctly, what we need is to compile the inner functions before we compile the outer functions, so we can make sure that the compilation is done correctly.

If we are solving this in the "general" sense, it can get quite hairy, because we'll need to invert the evaluation process by doing the following:

1. walk through the expressions to pull out included scripts
   
2. parse the included scripts to look for their required statements
3. evaluate those first, and throw away the rest of the statements
   
4. repeat 2 & 3 recursively
   
5. then finally go back and eval the rest of the expressions, which will then run through the included scripts and this time create the procedures and return the results
   

So we would need to create a separate compilation phase for evaluating the require statements in the included scripts, and then finally redo the eval phase for generating the output from the them.

While the above solves the problem in a general sense, it is neither elegant nor the only approach to solve the problem, not to mention the problem's impact is actually "small" (the exception is only thrown on the very first time the scripts are eval'd before the require modules are loaded), and hence we would get very little return solving it that way.

Sometimes the best way to solve the problem is to redefine the problem, and this is one of such case.

20% Effort for 80% Gain

The reason we want to have require statements inside includes is to keep them in one place (since the require statements have global scope anyways) so we can save on typings. So if we look at it from that perspective, even having to include them in all of the outermost scripts is too much work. It's probably easier if we can simply have a specialized scripts to contain all of the required statements.

This is very similar in concept to the "notfound.shp", which gets invoked when we cannot find the right script to dispatch. With such a specialized script, we simply need to make sure that it is evaluated whenever it changes.

The first step is to add the required path to the shp-handler struct

(define-struct shp-handler (path default not-found required) ..)

The next step is to check to see if the script exists and if it has changed, if so, evaluate it. In order to do so we'll need to keep track of the timestamp of the script. In order to do so we either add another field to shp-handler, or we will have to something else besides the path to the required field.

(define-struct script (path (timestamp #:mutable))) ;; for comparing to new timestamp

Then we can create the struct by check to

(define (normalize-path path (base (shp-handler-path ($server))))
  (apply build-path base (path->segments path)))

(define (init-script path (base (shp-handler-path ($server))))
  (let ((path (normalize-path path base)))
    (make-script path 0))) ;; 0 is epoch time.

We do not want to call init-script every time, so we will need a wrapper for make-shp-handler:

(define (*make-shp-handler path ;; will be renamed when provided out
                           (default "index.shp")
                           (not-found "notfound.shp")
                           (required "required.shp"))
  (make-shp-handler path default not-found (init-script required path)))

Then we just need to make sure this file is tested to see if it's changed, and evaluated if necessary.

(define (eval-script-if-changed! script)
  (unless (not (file-exists? (script-path script)))
    (let ((timestamp (file-or-directory-modify-seconds (script-path script))))
    (when (> timestamp (script-timestamp script))
      (set-script-timestamp! script timestamp)
      (let ((proc
             (evaluate-terms
              (file->values (script-path script)))))
        (proc ($request)))))))

and finally ensure that it is called before dispatching the request:

(define-struct shp-handler (path default not-found required)
  #:property prop:procedure
  (lambda ($struct request)
    ;; evaluate if
    (parameterize (($pathinfo ($pathinfo))
                   ($request request)
                   ($server $struct))
      (eval-script-if-changed! (shp-handler-required ($server)))
      (let ((proc (evaluate-terms
                   (file->values
                    (url->shp-path (request-uri request))))))
        (make-response (proc request))))))

Allright - with this change we now have a common require script in place. If it does not exist, nothing will happen, and we only evaluate it if the file actually has changed. And if you like to add the require statement to the individual scripts - that still works! Of course - this means that evaluating the inner include require statements before the outer function gets compiled will not be supported.

Problem solved.