JSONP

JSON with Padding: JSONP

What is JSONP?

JSONP is a mechanism for requesting and loading JSON data from a server other than the one the containing HTML page was loaded from. It makes use of a dynamically created HTML "<script>" tag, which references in its source attribute the URL of the resource to be accessed (’src="URL-of-resource"’). In normal usage this would most commonly be a JavaScript file, but it can be anything, which is how the technique works.

It used to be seen as a distinctly dubious aproach, but JSONP is increasingly entering the mainstream, providing the basis of such prestigious things as many of Google’s JavaScript APIs.

The Problem

Ajax applications are, as the name - Asynchronous JavaScript and XML might suggest - built on a number of disparate technologies, although not always the ones mentioned in it!

One core element is the XMLHttpRequest (AKA "XHR") object made available by web browsers to developers as part of their API.  This is the component that facilitates the dynamic communication between web pages (or JavaScript applications, depending on your perspective) and their back-end servers which make Ajax pages so much more responsive and interactive.  It was designed, as the name suggests, to transfer XML data over HTTP, but it is just as capable dealing with other kinds of data, such as the very popular JSON format, or CSV data (compact, but structurally limited), or HTML snippets, or even JavaScript code.

The XHR has a (deliberately) built-in limitation however. It can only access resources from the same web site as the owning page was originally loaded from (with a bit of jiggery-pokery this can be broadened to any resource within the same domain: any "host" within "unicorninterglobal.com" for instance).  This restriction was intended as a defence against cross-site scripting (XSS) attacks, however on the one hand, given the overall weakness of the browser security model, this is a bit like blocking one hole in a sieve, while on the other hand this policy, formulated around 2000 for a web very much more primitive and limited than the one we experience today, is now much more of a hindrance than a help.

Why would you want to do that anyway?

Fundamentally you would want to use this technique any time a client-side web application needed to access data from more than one source.  In today's web this is becoming ever more common as all sorts of interesting services are provided by various organisations.  For instance our own "Customer Map" example uses customer data from our web server (which is no problem: it can be retreived using the standard XHR mechanism) which it then combines with map data from Google Maps to produce its results, however that map data is coming from one of Google's servers, not Unicorn's web server.

Mashups - web applications combining data and presentation from multiple sources - are a "hot" topic in current web development, and without this kind of mechanism they would simply not be possible.  Any web application you build which you would like to have incorporate data from another source will need to use this approach, or an equivalent one.

The server-based way around the limitation

One approach to getting around this limitation is to use the web server as a proxy.  The JavaScript application makes a call (through XHR) to its web server, telling it that it wants whatever the resource is, and the web server (which is not restricted in what it can do) goes and gets it, then returns it to the JavaScript application.

This might seem like a good, safe approach, but it is flawed in (at least) three ways:

  1. It is more complex and hence more fragile: three sets of code have to work, instead of just two; two network requests have to succeed instead of just one.
  2. It involves two network round trips - one from the browser to the web server and another from the web server to the resource provider - rather than just one.  If the size of the data involved is large (think of a screen-full of Google map images for instance) that doubling of the network load - and of the network lag -  can be an unacceptable overhead.
  3. If the resource in question really is a security threat, you are now exposing not just the client web browser to it, but also the web server itself, which in turn potentially exposes all users of that server to whatever exploits those evil black-hats might have up their sleeves!

Maybe not such a great idea then.

JSONP

The JSONP approach is to use a dynamically created (and also usually dynamically destroyed again) script tag to access the resource directly from the browser.  The script tag has been around longer than the XHR and is not beset by the same security restrictions.  The "src" attribute of a script tag can point to anything that can be accessed through a URL.

Of course for this to work, the resource in question has to return its response in the right format - there is no point just dumping a big load of data raw into your HTML page.  There are generally two ways of doing this, both involving having the data returned as part of a JavaScript code snippet.  One is for the data to be returned as part of a JavaScript assignment statement:

   scriptData = { whatever-data-is-required-in-JSON-format };

The other is for it to be passed as the argument to a function call:

   scriptDataHandler({ whatever-data-is-required-in-JSON-format });

The second approach is usually the preferred one because something happens (effectively an "event") when the data is returned (here the function scriptDataHandler is called) allowing the receiving code to take whatever action (within, or called from within, the receiving function) is required to deal with it.

Implemention

In the JavaScript, a fairly generic implementation might look something like this:

   function getJsonP (url, func, params) {
      var script = document.createElement("script");
      var src = url + "?func=" + func";
      if (params) {
         src += "&json=" + encodeURIComponent(JSON.stringify(params));
      }
      script.setAttribute("id", "tempScriptTag");
      script.setAttribute("src", src);
      document.body.appendChild(script);
   }
   
   function deleteTempScriptTag () {
      var script = document.getElementById("tempScriptTag");
      if (script) {
         document.body.removeChild(script);
      }
   }
   
   //  elsewhere...
   
   function getOrder (ordNum) {
      getJsonP(
         "http://orderserver.otherdomain.com/orderService/getOrder",
         "gotOrderData",
         { orderNumber: ordNum }
      );
      
   }
   
   function gotOrderData (data) {
      deleteTempScriptTag();
      
      // Code dealing with the returned data here...
      
   }

When the "getOrder" function is called, being passed an order number to retrieve, it just calls the utility function "getJsonP", passing it the URL of the remote service that provides order data, the name of the function to be called with the returned data (as a string: "gotOrderData" in this case) and a JavaScript object containing the parameters to the call, in this case only a single parameter: the order number.

In "getJsonP" it first creates a script tag using the DOM method "document.createElement", then it initialises its "src" variable with the passed URL, appending the value of its "func" argument in the query string ("?func=" + func).

If the "params" argument (which is expected to be a JavaScript object with the actual parameters as data members, or properties) has been passed then it appends (the JavaScript "+=" operator) to that string another query string parameter ("&json=" + ...) the value for which is composed of the "params" object, first serialized into a JSON string using the "JSON.stringify" function (from Douglas Crockford's json2.js script, or the browser's native version of the method if it implements that), then URI-encoded (to escape any characters, such as spaces, which are forbidden in URLs) using the "encodeURIComponent" browser function.

(The conditional addition of the "json" parameter is to cater for calling operations which don't take any parameters - they can thus be invoked by calling "getJsonP" with only its first two arguments: "params" will be "undefined" - so false - and hence the "json" parameter will be omitted from the URL.)

The newly-created script element will have its "src" attribute set to the value of the "src" variable (via the DOM "setAttribute" function) and its "id" attribute set to a known unique value (here "tempScriptTag") so that it can be cleaned up later.

Finally the script element is appended to the HTML body ("document.body.appendChild"), although it could be appended elsewhere (we prefer the body).  This sets things in motion, as the browser will immediately evaluate the new element and, since it is a script tag, will attempt to retrieve the resource identified in its "src" element via an HTTP "GET" request, which will look like this:

   http://orderserver.otherdomain.com/orderService/getOrder?
                        func=gotOrderData&json={"orderNumber":%20nnnnnn}
(but without the line-break obviously)

The back-end function on the remote server ("getOrder" in this case) will thus receive two parameters: the function-call to return its data "wrapped" in and the JSON string containing its parameters (just the order number in this case).

It will do whatever it does to fulfill the request and will then encode its response as a JSON object-string (which can be as complex as required), but will then "wrap" it in the form:

   gotOrderData( ... JSON-formatted response data ... );
for returning to the client.

Back on the client, the script, once received, will be immediately compiled and executed, causing the function "gotOrderData" to be invoked with the JSON string, now effectively a JavaScript object literal, being passed as an argument.

In "gotOrderData" there will be no need to deserialise the "data" argument - it will already be a normal JavaScript object with data members containing whatever data the server has returned for the order.

Note that the very first thing that "gotOrderData" does is to call "deleteTempScriptTag" to remove the script tag and stop it cluttering up our HTML - not a big deal if this is something you will only do a few times, but crucial if there might be thousands of JSONP calls during the course of an application session.

Example

We now have an example of JSONP in action.