Continuing of HTTP Call & Proxy Integration (2)

In the last post we basically have a working rudimentary HTTP client. It should not be a lot of work to create a rudimentary proxy out of the HTTP client.

Basically - we want to more or less pass the request to the destination web service, with as minimal fuss as possible.

By default, the proxy call should look something like the following:

http://host:port/http://target/path
http://host:port/https://target2/path2

But of course we also want to allow for hardcoded path to possibly map directly to another url, for example:

http://host:port/google => http://www.google.com/

In order for the above to work, we would want to have scripts called http: and https: located at the SHP root path with the following syntax:

;; http: or https:
(proxy! (pathinfo->url))

But unfortunately http: and https: are not valid file names in Windows, so the above technique cannot be used with Windows (but should be usable with Linux). and we'll settle for the following:

http://host:port/proxy/http://target/path
http://host:port/proxy/https://target2/path2

Then the code for proxy should be:

;; proxy
(proxy! (path->info->url (cdr ($pathinfo))))

Custom Headers

Many web services (such as Twitter, Del.icio.us, and Blogger) do not take all of the client headers with their APIs, since they are not expecting the browser making direct connections and hence do not need all of the headers issued from browsers.

What this means is that we should filter out most of the headers except needed ones, a simple way to determine what's needed is via custom headers. We'll choose to create our custom headers via prefix of bzl-.

So, any headers started with bzl- shall be passed onto the target, and other ones (except mandatory headers such as Content-Type, Content-Length) should be filtered out. This way it would allow fine grained control by us.

Since all of the headers are already available in $headers parameter, the conversion should be automatic.

Authentication & Passing User Credentials Securely

A special header is the Authorization header, which contains the user's login and password. HTTP Basic is basically insecure, and HTTP Digest will require a challenge and a response - for now we'll only handle passing HTTP Basic.

There are a couple of ways of passing user credentials - one is via the above mentioned custom header mechanism (i.e. bzl-Authorization), and the other is via the user field of the URL value, i.e.

http://user:password@host/path...

Then within our proxy it looks like:

http://host:port/proxy/http://user:password@host/path...

Since either way is insecure without a SSL, it does not really matter which way is preferred. But what we need to make sure is not to use the regular Authorization header, since that should be used for potentially authenticating against the proxy itself.

Review of Requirements

Let's reiterate the requirements:

• a pass through of request to the proxied target, and a pass through of responses
• take the URL from the pathinfo of the request
  
• very simple usage within SHP (i.e. no fancy setup of anything)
• a clean way of passing through headers
  
• a clean (& mostly secure) way of passing credentials for the 3rd party web service
  

Let's see how we can develop such a proxy to match the requirements. To be continued...