Unmasking JSONP

Why

This post is about JSONP, or in jQuery you might know it as $.getJSON(). I was trying to find a good tutorial or just a small function to allow me to do JSONP requests without jQuery because I was writing a JavaScript library of my own for a client. After numerous Google and StackOverflow searches I discovered every result was a jQuery $.getJSON() tutorial. I figured I can't be the only one using JSONP outside of jQuery. So, in this tutorial I'll explain what it is, why it works, and writing your own JSONP library to handle it.

What is JSONP

JSONP is "JSON with padding". What you need to know is that JSONP is a security hack. It was solution thought up to get around "same origin policy" in browsers. In short, the same origin policy in browsers prevents developers and, more importantly, hackers from inserting JavaScript into their site that they can use to hack your site.

In technical terms, JSONP is simply when you insert a script into your site that has a source of another website just like what most of you have done with <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js"></script> which inserts jQuery from Google's servers. The only difference is that rather than executing a script it actually just returns JSON in a function that you call to get the raw JSON / JavaScript object.

How it Works

So what does that look like? Here is JSONP request to Twitter for my timeline: http://api.twitter.com/1/statuses/user_timeline.json?screen_name=oscargodson&callback=JSONP123

Notice the first word (name of the function) in that link is JSONP123 and that it matches the callback= value in the URL. This matches a function you defined on your side as var JSONP123 = function(json){ console.log(json) } so that when that Twitter <script> loads, it triggers your function and does what you want with the JSON inside. In that sample code, when the Twitter script loads, it will show the JSON returned from Twitter in your console.

The Code

With that knowledge lets make a super simple call to Twitter, and display your avatar in your <body>. We'll need to first create a function to handle the returned result then a script tag to grab the JSON.

The code in it's simplest, non-dynamic, terms:

//Var name is used in the callback=X in the URL of the generated script
var JSONP = function(json){
  //"json" will the JSON returned from Twitter, so
  //json[0].user.profile_image_url will be the URL to the avatar image
  document.getElementById('pic').innerHTML = '';
}

//Here we create a script element and save it to the scriptTag variable
var scriptTag = document.createElement('script');

//After it's created, we set the src attribute to the Twitter timeline we
//want. Change the screen_name for yours if you want just make sure to
//match the callback=JSONP to match the function above.
scriptTag.setAttribute("src", "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=oscargodson&callback=JSONP");

//The very last step, simply add it to our  for processing.
document.getElementsByTagName("head")[0].appendChild(scriptTag);

Here is a live sample. You can edit the source as well.

This is great if you are going to be using JSONP once, but you're going to use JSONP more and more throughout the years as a developer. Let's build ourself a tiny JSONP library you can keep on reusing. Here's the library we are going to build:

(function(window, undefined) {
  var JSONP = function(url, method, callback){
    url = url || '';
    method = method || '';
    callback = callback || function(){};

    if(typeof method == 'function'){
      callback = method;
      method = 'callback';
    }

    var generatedFunction = 'jsonp'+Math.round(Math.random()*1000001);

    window[generatedFunction] = function(json){
      callback(json);
      delete window[generatedFunction];
    };

    if(url.indexOf('?') === -1){ url = url+'?'; }
    else{ url = url+'&'; }

    var jsonpScript = document.createElement('script');
    jsonpScript.setAttribute("src", url+method+'='+generatedFunction);
    document.getElementsByTagName("head")[0].appendChild(jsonpScript);
  }
  window.JSONP = JSONP;
})(window);

I'll step through it and hopefully give you a good understanding of what each piece does.

The first step in building any JavaScript library is the skeleton. The skeleton looks like this:

(function(window, undefined){
  var JSONP = function(url, method, callback){
    url = url || '';
    method = method || '';
    callback = callback || function(){};
  }
})(window);

The first line:

(function(window, undefined){

This sets up the anonymous function. It's considered anonymous because it has no name. If it had a name it'd look like the normal function foo(){} or var foo = function(){}, but instead this is (function(){}). Since it has no name you call it by attaching a () at the end like (function(){})() just like you'd do function foo(){}; foo(), but it has no name so you just do (). We do this because we don't want our variables to collide with the other variables the website we're working on might have. If you don't understand what I'm talking about, that's fine. JavaScript variable scope is confusing, but out of the scope (no pun intended) of this article. For more info on JavaScript scopes check out this article by Skilldrick. The window and undefined are in there for the same reasons as jQuery. Explanation of that on this great StackOverflow thread.

The next line:

var JSONP = function(url, method, callback){

This is simply the name of the library we want with the parameters you want to allow. You'd use the library like:

JSONP('http://site.com',function(json){ console.log(json) });

The method parameter is optional. We'll talk more about that later, but for now I just wanted you to see what this line is all about.

Next we have this chunk of code:

url = url || '';
method = method || '';
callback = callback || function(){};

If you've never seen the foo = foo || 'bar'; syntax before, it's simply saying "url = url if it's set, otherwise use '' if it's not". This is great shorthand for if(!url){url = '';}; For the callback parameter we set it as a blank function that does nothing so if it's not set it doesn't break anything expecting a function rather than a string or integer for example.

The next step to building our library is making any optional parameters optional. The only one we really need to catch is method. Again, I'll explain what this is exactly further down, but to make it optional we'll do a check to see if it's a function. If it's a function we know the developer wanted to omit the method string and instead he wrote the callback function in it's place like JSONP('http://somesite.com',function(){}) rather than JSONP('http://somesite.com','jsoncallback',function(){}).

if(typeof method == 'function'){ //If method wasn't a string, but a function.
  callback = method; //Since the "method" is actually the callback, we switch it.
  method = 'callback'; //Next, we set a default method ourself
}

Because this library is all about being able to have lots of JSONP requests on a single page at any given time, we need to make sure every callback function we define for the API's JSONP service we're using (like Twitter) is completely random. You can do this lots of ways but I do it like this:

var generatedFunction = 'jsonp'+Math.round(Math.random()*1000001)

That'll output something like jsonp204309. Math.random() gives back a value such as 0.2043090183287859, but then we multiply that by 1000001 and we get 204309.222637804228786. Since we are going to use this as a function name, we need to remove the . in the value, so we use Math.round() which then gives us (finally) a nice 204309.

After we have the function name we need to give this function to the browser somehow. Then, so we don't clutter up everything, we need to delete it when we are done. Again, there are a few ways to do this, but I find the most elegant way is like this:

window[generatedFunction] = function(json){
  callback(json);
  delete window[generatedFunction];
};

This gives the window (your website) the function we created and we make it into a function with one parameter, json. We also delete the function after it's run. Clean and simple. This will run once our script tag is generated and inserted into the DOM.

Before we create the script tag and execute everything, we need to do a tiny test on the URL. We need to check if the URL provided does or doesn't have a ? in it already. The reason being is that if we give it a URL like http://mysite.com/api/ as the JSONP API source, it'll work fine because we'll modify the URL to be http://mysite.com/api/?callback=jsonp204309, however, many times API's have other parameters to set such as the format or anything else and the URL could end up being like http://mysite.com/api/?format=json and we don't want the generated URL to be http://mysite.com/api/?format=json?callback=jsonp204309 which wont work. We need to switch that for a &. The fix is surprisingly simple. We can just do a check if a ? exists in the url, and if not, add the ? otherwise if it does add a & instead.

if(url.indexOf('?') === -1){ url = url+'?'; }
else{ url = url+'&'; }

Now it's time for the magic. We create the <script> tag dynamically with JavaScript by using document.createElement() then we set the 'src' attribute to our url + method + the random function name (jsonp204309()). The we grab the <head> and append our newly created <script> tag. It'd end up looking something like <script src="http://mysite.com/api/?callback=jsonp204309"></script>.

var jsonpScript = document.createElement('script');
jsonpScript.setAttribute("src", url+method+'='+generatedFunction);
document.getElementsByTagName("head")[0].appendChild(jsonpScript);

Finally I'll explain what the method parameter is for. The method is for APIs that don't use the standard callback= format in the URL. For example, Flickr's API requires jsoncallback=. If you do callback= it'll break the API and it won't work. If you go to the jQuery $.getJSON() API documentation page the example is using the Flickr API and you'll notice the word jsoncallback=? in the URL they are using as an example.

The final step is to expose this to the DOM. As you read earlier we wrapped this in an anonymous function like (function(){})(). Well, somehow we need to give our JSONP() function/var to the DOM and we can do it by writing (like jQuery does) window.JSONP = JSONP;. This gives our JSONP function to the window (your website) so whenever you or any other developer writes JSONP() in their scripts it'll run the code hidden in this anonymous function. Cool aye?

Conclusion

So what does this all look like completed? I've uploaded it to GitHub (feel free to fork it and send pull requests) and here is a live sample and the code calling my last tweet and a photo from my Flickr stream.

While this script is actually fairly small (less than 1kb compressed), it's pretty tricky and thats why people just use the jQuery version. The jQuery version does have some other cool features like passing a JS object like {name:'oscar',foo:'bar'} and it modifies the URL to be http://somesite.com?name=oscar&foo=bar&callback=jsonp123456, but that's a whole 'nother post and it's not really needed.