This post is more technical and detailed than what I
usually
write on fettig.net. I’m going to talk about a technique I’ve
been
working on to work around a limitation in the XmlHttpRequest
object
used in Ajax applications. Web geeks, please read on.
Updates to this post
See this post for an updated version of this techinique that works (hack-free) in Firefox 1.5.
The problem
XmlHttpRequest objects are bound by the
same
origin security policy
of browsers, which prevents a page from accessing
data
from another server. This has put a serious limitation on
Ajax
developers: you can use XmlHttpRequests to make background
calls to a
server, but it has to be the same server that served up the
current page.
Known workarounds for this limitation involve either
server-side
reverse proxying
or
bypassing XmlHttpRequest
entirely.
In my case neither of these approaches was
going to work. I
wanted to use
LivePage,
the live-update framework
developed by Donovan Preston and other brilliant
hackers at
Divmod. LivePage
works
because it uses Twisted,
which is good at handling lots of long-lasting
network connections at
the same time. Since Apache isn’t good at handling
lots of long-lasting
connections, putting an Apache reverse proxy in front of
the Twisted
server would put a major hurt on performance and scalability.
And
since LivePage is bound to XmlHttpRequest, I couldn’t use
a
non-XmlHttpRequest alternative.
Our approach with JotSpot Live has been to let a Twisted
server
handle all page requests and XmlHttpRequest calls. That’s fine
as
long as JotSpot Live is a standalone, dedicated web site. But what
our
customers are asking for (and what we want) is the ability to
have
Live-style realtime updates in ordinary xxx.jot.com sites. And we
don’t want
to put a Twisted server in front of every site in our
domain. So I’ve been
trying to find a way to let any page in our domain
communicate with
live.jot.com through XmlHttpRequests. As it turns out, it is
possible, but
you have to jump through some hoops.
A couple of notes on the examples here:
- I’m not really using two different servers. In my examples,
a
page on http://fettig.net/ is attempting to make an XmlHttpRequest call
to http://www.fettig.net/. The fact that my Apache setup maps these to
the same virtual host makes no difference to a browser, though: they’re
not the same hostname, and they’re treated no differently than if they
were two different sites with different content. The results would be
the same if I was trying to make XmlHttpRequest calls to an
ajax.fettig.net server with its own IP address. - I’m using a simple XmlHttpRequest wrapper library that provides a
getUrl
function. getUrl takes a URL and a callback function, opens an
XmlHttpRequest connection to that URL, and calls the function with the
results. If you’re interested you can view the full code here. - The service to which I’m making XmlHttpRequest calls is a
simple
PHP page, ajaxdata.php, that prints out the current UNIX time on the
server: Not particularly useful in real life, but good enough for testing purposes.
First attempt: the naive approach
Here’s the first thing I tried: a standard
XmlHttpRequest with
the full URL of a page in a different subdomain. I
was pretty sure this
wouldn’t work, but I wanted to verify it for
myself:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function getTime(){ getUrl(AJAX_URL, gotTime); } function gotTime(status, headers, result) { document.getElementById(’time’).innerHTML = result; setTimeout(getTime, 1000) } window.onload = getTime; </script> </head> <body> <div id=”time”></div> </body></html>
This page is served from fettig.net, and tries to make
an
XmlHttpRequest call to www.fettig.net. In real life, these are the
same
server, but the browser doesn’t know that. Unsurprisingly, this
doesn’t
work in any browser. You get a security error such as this one
(from
Firefox):
Error: uncaught exception: Permission denied to call method XMLHttpRequest.open
You can try this for yourself here.
Second attempt: using an iframe and document.domain
On my last trip to the JotSpot office Alex pointed me toward iframes and the
document.domain
property. iframes are similar to XmlHttpRequests in
that they load data
into the current page from another page. But iframes
aren’t limited to
pulling pages from same web server as their containing page
- they can load any URL. To prevent cross-site security
problems, browsers
enforce the same origin policy in the javascript
object model: scripts
running in one frame can’t access any objects
inside another iframe, unless
both pages came from the same server.
There’s an exception to this rule, however. If both pages come
from the same parent domain, and
both
of them set the property document.domain to the same parent
domain,
scripts running in either frame will be allowed to talk to each
other.
For example, say the page http://www.example.com/ loads the
page
http://ajax.example.com/ in an iframe. Since both pages are in
the
domain example.com, if both set document.domain to “example.com”
they
will be be given the ability to programatically access each
other’s
data. While there are a few differences
So, can you use an iframe with document.domain
to make
XmlHttpRequest connections? Yes, with two restrictions:
- The iframe must be served from the server to which you’ll be making XmlHttpRequest calls.
- You have to open the XmlHttpRequest connection before you set document.domain.
Here’s the code I used. First, the page test2.html:
<html> <head> <script type=”text/javascript”> document.domain=”fettig.net”; function gotTime(result) { document.getElementById(’time’).innerHTML = “Server timestamp: ” result; } </script> </head> <body> Single XmlHttpRequest. Works in all modern browsers. <div id=”time”></div> <iframe src=”http://www.fettig.net/playground/ajax-subdomain/test2-iframe.html”> </iframe> </body></html>
This script in this page sets document.domain, and defines the function gotTime
to handle the results of the XmlHttpRequest call. All the
XmlHttpRequest
stuff happens in an iframe, though. Here’s the code in
test2-iframe.html:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function gotResult(status, headers, result) { document.domain = “fettig.net”; // set d.d before talking to parent frame window.parent.gotTime(result); } getUrl(AJAX_URL, gotResult); </script> </head></html>
test2-iframe.html makes the XmlHttpRequest call
to
www.fettig.net. It’s allowed to do this because it was served
from
www.fettig.net, so it’s just talking to its originating server. Once
it
gets the response back, it sets document.domain to fettig.net,
to match
the document.domain value of the parent frame. Now it’s
allowed to talk to
the parent frame using javascript, so it’s able to
call
window.parent.gotTime.
The key is to do this in the right order. Once you
set
document.domain (to anything other then the actual host the page
was
served from) you lose the ability to make XmlHttpRequest calls.
So
you’ve got to take care of your XmlHttpRequest business before you
set document.domain and to enable communicating with the parent frame.
Follow those rules, though, and you’re good to
go. This
technique seems to work in all modern browsers (I tested in IE
6,
Firefox
1.0.7, Safari 1.3, Opera 8.5, and Konqueror 3.4), and it doesn’t
contain anything I’d consider a hack. You can try it yourself here. So far
so good!
Third attempt: repeated XmlHttpRequests
I was happy to have figured out a way to make
a
single
XmlHttpRequest to another server in our domain. But as presented
so
far, this technique has a pretty severe limitation: You can only
make
XmlHttpRequest calls up to the point where you set
document.domain.
Once you do that, you gain the ability to communicate with
the parent
frame, but you lose the ability to make future XmlHttpRequest
calls.
This wasn’t good enough for LivePage, which needs the ability
to
continuously make XmlHttpRequest calls to the server and handle
the
results. I needed a way to let the iframe communicate with both
the
server (using XmlHttpRequests) and the parent frame (which it can
only
do by setting document.domain).
This raised the question: is setting the
document.domain
property one-time-only, or is it possible to switch it on the
fly? If
it can be changed back and forth, it might be possible to change
it
back and forth as needed. Here’s test3-iframe.html, a modified version of
the iframe page:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function gotResult(status, headers, result) { var oldDomain = document.domain; document.domain = “fettig.net”; window.parent.gotTime(result); document.domain = oldDomain; setTimeout(getTime, 1000); } function getTime(){ getUrl(AJAX_URL, gotResult); } getTime(); </script> </head></html>
Like in test2-iframe.html, the gotResult
function in
test3-iframe.html sets document.domain to fettig.net (to
enable access
to the parent frame) and then calls window.parent.gotTime. But
first it
stores the current value of document.domain, which defaults to the
host
the page was served from. After it’s finished working with the
parent
frame, it sets document.domain back to the original value,
which
restores the ability to make XmlHttpRequests. It then uses
setTimeout
to set up another XmlHttpRequest call. The result should be a
continually updated timestamp on the main page.
You can see this in action at
http://fettig.net/playground/ajax-subdomain/test3.html
.
This
page makes repeated XmlHttpRequests, with the iframe calling a
function in
the parent page each time it gets a result from the server.
Unfortunately,
while this technique works great in IE, Safari, and Konqueror, it doesn’t work
in Mozilla and Opera. The
first XmlHttpRequest will work, but when you try to
change
document.domain back to the original value you get an error. From
Firefox:
Error: [Exception... "Illegal document.domain value" code: “1009″ nsresult: “0×805303f1 (NS_ERROR_DOM_BAD_DOCUMENT_DOMAIN)” location: “http://www.fettig.net/playground/ajax-subdomain/test3-iframe.html Line: 13″]
Mozilla and Opera are more strict about the value
of
document.domain - it can only be set to the current value or
a
higher-level domain. For example, if the host is aaa.bbb.example.com,
I
could set document.domain to bbb.example.com. At that point I
could
change it again, to example.com, but I couldn’t change it back
to
aaa.bbb.example.com. Once you move to a higher-level domain,
you’re
stuck there.
Fourth attempt: Making it work in Mozilla
I could live without Opera, but not Mozilla. So I set out to find a way - any
way - to work around this problem. After thinking about it for a while
I
started wondering: just how strict are security limitations on
communications
between frames? We know that browsers don’t let frames
access each other’s
objects unless they have the same document.domain.
So setting a child frame’s
document.domain to something different from
the parent’s and then trying to
look at window.parent.foo isn’t going
to work. But what if you could set an
attribute in the child frame that
pointed to a function in the parent frame,
and then changed the document domain? Would the child frame still be able
to call that function?
The answer is yes, and that was the break I needed to get this thing working in Mozilla. The trick is to use two frames:
parent window bridge iframe child iframe
The bridge iframe and child iframe are both served from the
server
to which you’ll be making XmlHttpRequest calls. The parent page code
is
the same as in the previous examples: it defines a function to
handle
the XmlHttpRequest result, and loads an iframe, this time
containing
the bridge code, which looks like this:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> function gotTime(result) { window.parent.gotTime(result); // pass result up to the parent } window.onload = function(){ var subframe = document.createElement(’iframe’); document.body.appendChild(subframe); subframe.src = “test4-iframe.html”; subframe.contentWindow.bridgeGotTime = gotTime; document.domain = “fettig.net”; } </script> </head> <body></body></html>
When this frame loads, it loads another frame inside
itself.
Since the child frame is loading from the same server as the
bridge
frame, the bridge frame can access the child frame’s objects
through
script. The bridge frame takes advantage of this to set
subframe.contentWindow.bridgeGotTime to its own gotTime
function. Next, the bridge frame changes its document.domain to match
that
of the parent window. At this point the bridge frame loses the
ability to
talk directly to the child frame, since their
document.domain properties no
longer match. But the child frame retains
the ability to communicate with the
bridge frame through the
bridgeGotTime function, which was set up first. And
since the bridge
frame’s document.domain now matches the parent frame’s
document.domain,
the bridge and parent can communicate freely.
In my limited testing, this techinique only seems to work
on
Mozilla-based browsers (I haven’t found a solution for Opera yet, but
I
haven’t spent much time on it). My final example uses a hybrid
approach
that tries to use document.domain switching, and then falls back on
the
bridge-iframe approach if the browser raises an error when you try
to
reset document.domain. Here’s the main page code:
<html> <head> <script type=”text/javascript”> document.domain=”fettig.net”; function gotTime(result) { document.getElementById(’time’).innerHTML = “Server timestamp: ” result; } </script> </head> <body> <div id=”time”></div> <iframe src=”http://www.fettig.net/playground/ajax-subdomain/test4-iframe.html”></iframe> </body></html>
Same as everything else so far except that the iframe is loaded from test4-iframe.html. Here’s the code in that page:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function getTime(){ getUrl(AJAX_URL, gotTime); } function gotTime(status, headers, result) { var oldDomain = document.domain; if (window.bridgeGotTime) { window.bridgeGotTime(result); } else { document.domain = “fettig.net”; window.parent.gotTime(result); } try { document.domain = oldDomain; setTimeout(getTime, 1000); } catch(e) { // denied access to switching the domain, use bridge instead document.location.replace(”test4-bridge.html”); } } getTime(); </script> </head></html>
This code tries to switch document.domain back and forth as needed
to
communicate with both the XmlHttpRequest server and parent frame.
If
this fails, however, it loads the bridge iframe in its place. To
keep
from having two different pages doing the same thing, this page
also acts as the child of the bridge frame. If it sees a
window.bridgeGotTime attribute it knows it’s running under the bridge
iframe, so it calls window.bridgeGotTime instead of trying to talk
directly to window.parent.
You can see this in action in test4.html. It works in all the latest browsers except Opera. Update:use test5.html instead. See this post for details.
End result: it’s possible to make XmlHttpRequest calls to
another
server in your domain. You can make repeated calls too, if
you’re
willing to deal with a little setup overhead. This has a number
of
possible benefits:
- You could have a dedicated Twisted server handling
LivePage
requests, while the main content of pages if served by Apache or some
other web server. - You could have a dedicated server for handling web services, including XmlHttpRequests
- You can use DNS to free XmlHttpRequest from
the
two-max-connections limit of browsers. Browsers don’t let you have more
then two open connections to a server at a time - this gums things up
if you’re using an app like JotSpot Live and try to have more then one
tab going at a time. But if the XmlHttpRequests don’t have to be going
to the same server, you can use wildcard DNS and set each page to make
XmlHttpRequest calls something like randomhash.livepage.yourdomain.com.
If you’re read this far, thanks for hanging in there! I
hope
this is a useful tool in your bag of Ajax tricks. I’m sure there’s
more to be
explored here, but having gotten this
far I wanted to write this post and put
it up for comments. If you
think I’ve overlooked something, that there’s a
better way, or that all
of this is an ugly hack, please tell me about it in
the comments.
