Sun 14 Dec 2008

Dojo.reload Reloaded: reload app without reloading browser

Posted by liucougar under dojo
[9] Comments 

After introducing dojo.reload in my previous post, I talked with kriszyp in IRC, and he asked whether dojo.reload can actually modify already created objects from dojo.declare-d classes: in most web 2.0 apps, after the app is loaded, there are lots of instances of classes already created, and using dojo.reload to reload their definition won’t actually update any of these existing instances.

As I do not have an idea to actually implement that, I do come up with another “workaround” (if you like): rather than propagate changes to existing objects on the page, we can actually try to ask the app to reload itself without reloading the whole page. We don’t want to add in any special code in any frontend apps to handle this sort of “self reload” mechanism just to speed up development. Instead, a generic way should be available without any extra logic support in the application layer.

With the help of dojo and with one assumption, I did come up such an implementation to support that. The one assumption is that, all existing objects you care about in your applications are all dijits. If this assumption can be met, what we really need to do in order to reload an application is to:

  1. call onunload handlers (in case any are registered with dojo)
  2. destroy all dijits in the page
  3. reset the document.body.innerHTML to what it looks like when the page is first loaded (before any js actually modifys it)
  4. call all onload hanlders registered with dojo (such as dojo.parser etc.)

Item 3 can be achieved via a xhr call to retrieve current page from server and figure out the content within <body> tag. Item 4 requires a bit of extra work, because once all the onload handlers are called by dojo, they are removed. So in order to actually rerun them, we have to remember the original list of handlers.

Of course, as you may already figured, the method mentioned above won’t be able to run any modified onload handlers (even after they are dojo.reload-ed), however, it does cover most of other cases.

The one assumption mentioned above is not actually true in most web 2.0 applications using dojo, in such cases, any non-dijit objects have to be destroyed somehow so that they can be recreated, but that logic has to be application specific. On the other hand, in order to prevent memory leaks in browsers (espcially IE), it is always a good idea to have a destroy function for your objects to clean itself up. So if you are not worried about any thing other than dijits, then the method mentioned above should work reasonably well, while if you do want to re-create other non-dijits stuffs, you can call some application specific cleanup routines (which as I mentioned above would be good to have in any case).

You can find the code in dojoc.util.loader, the function is actually called dojo.reloadPage(). The normal use case for this is:

  1. dojo.reload any modules you modified and want to test;
  2. dojo.reloadPage to actually test them without waiting for reloading all js files/reloading your browser window.

Note: dojo.reloadPage only works if it is included in the initial page load, if you dojo.require dojoc.util.loader after the page is loaded, dojo.reloadPage won’t work (see comment at the beginning of the file to know why)

dojo.reload now also resides in dojoc.util.loader, and compared to the previous version, two additional features are add:

  • Partial module name match: now you just need to type in a partial name of the module you want to reload, if it only matches one module already loaded, that module will be reloaded. If there are multiple matches, it will list all possible modules and error out.
  • Automatically remove cache of dijit template: if a module references widget template files (using templatePath), these cached templates will be cleared automatically, so next time you create such a dijit, the new template will be loaded from your sever directly.

A new function to reload css files is also included in dojoc.util.loader, dojo.reloadCss, which also supports partial name match. If it is given no argument, it will reload all css files in the page (with a href attribute, of course), similar to ReCSS feature in the firebug lite shipped with dojo core.

Warning: use at your own risk.

Sat 6 Dec 2008

Reverse proxy cherrypy with Apache/Nginx

Posted by liucougar under Linux , dojo
[2] Comments 

For the past several months, I am developing frontend against a bare cherrypy server which also serves all static files, like javascripts/images etc. Without building frontend code (so there are lots of small js files to load), it takes more than 20 seconds to reload the unbuilt frontend application I am working on (which is the main driver behind my hacking on dojo.reload).

Use Apache as proxy server

Today I decided to shield an apache proxy server in front of the cherrypy server to off load all static files serving duties from the latter, in the hope of speeding up reloading speed of the app.

The apache proxy should be setup so that it directly serves any files under /debug and /release, all other requests are dynamic and should be handled by cherrypy server. In addition, our backend app sometimes use HTTP redirects to direct client to a new page. In Apache configuration file, all this can be achieved by:

ProxyPassMatch ^/(?:debug|release)/.* !
ProxyPass / http://127.0.0.1:8000/
 
ProxyPassReverse / http://127.0.0.1:8000/
 
Alias /debug /var/www/htdocs
Alias /release /var/www/htdocs/release

Note: the /debug directory is the unbuilt frontend code, while the /release points to the built frontend code (by default, dojo will put the built version under release directly as peer of dojo dir).

With the above settings, cherrypy is nicely sitting behind the Apache server without worrying about any static files, and the reloading time of the unbuilt frontend code now reduces to about 4 seconds, which is a dramatic improvement.

Try nginx instead

nginx is normally considered to be a faster reverse proxy server than apache, so I want to give it a try.

In nginx configuration file, proxy_pass is used to pass the request to a backend server, while it does not use proxy_pass_reverse, instead the equivalent in nginx is proxy_redirect directive. As long as your host domain name is properly set up, the following nginx directive is equavalent to the above apache directive:

location /debug/ {
    alias /var/www/htdocs;
}
location /release/ {
    alias /var/www/htdocs/release;
}
location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_redirect default;
}

More info on proxy_redirect can be found in official documentation.

Impression of Nginx compared to Apache

While I don’t want to do any thorough comparsion of the two reverse proxy servers, I just tried each of them several times and monitors the net panel output in firebug. It seems, nginx delievers more consistent performance: for the same page, apache sometimes deliver it in 1 second, sometimes in 4 seconds, while nginx always delivers it in 1 second. Thus I guess I will just keep using nginx at least for now.

Tue 2 Dec 2008

Reload dojo modules without reloading browser

Posted by liucougar under dojo
[2] Comments 

When working with FAT client web 2.0 application, it is normally quite slow whenever you need to reload your application because you are developing against unbuilt code (the source files are changed frequently, so you don’t want to build your frontend code every time a modification happens). As a frontend developer, you are using Firefox + Firebug, with console/script tabs enabled (sometimes net tab in firebug as well), which makes the loading time of a page with lots of javascript files even more slower.

While we are used to ajax ways of updating part of a page without reloading the entire webpage, no one seems to apply the same methodology to reloading javascript source files.

Dojo has the powerful dojo.require mechanism which can pull in javascript source files on the fly, and it is smart enough to cache any source files which are already downloaded. However, it does not provide any way to force reloading of a js file.

After digging into dojo.require, I came up with a simple function to overcome this very issue: basically, it will unset any memory dojo has about that downloaded source file (or module, as called in dojo.require), and dojo.require that module again.

Here is the source code for this function, which is called dojo.reload for now:

dojo.reload=function(d){
	//remove the module namespace from its parent namespace
	var sym=d.split('.');
	if(sym.length>1){
		var pname=sym.slice(0,-1).join('.');
		var prop=sym[sym.length-1];
		var parent=dojo.getObject(pname,false);
		if(parent){
			console.log('delete property',prop,'on object',pname,':',delete parent[prop]);
		}
	}
 
	//remove _loadedModules entry so that dojo.require will re-load it
	delete dojo._loadedModules[d];
 
	//tidy up _loadedUrls so that dojo._getText will reload it
	sym=dojo._getModuleSymbols(d);
	var relpath = sym.join("/") + '.js';
	//copied from dojo._loadUri
	var uri = ((relpath.charAt(0) == '/' || relpath.match(/^\w+:/)) ? "" : dojo.baseUrl) + relpath;
 
	delete dojo._loadedUrls[uri];
	dojo._loadedUrls.splice(dojo.indexOf(dojo._loadedUrls,uri),1);
 
	//set cacheBust to make sure we load latest file
	dojo.config.cacheBust=+new Date;
 
	dojo._loadreloaded=d;
	if(!dojo._reloadhotkey){
		dojo._reloadhotkey=dojo.connect(document.documentElement,'onkeydown',function(e){
			if(e.keyCode===dojo.keys.F6){
				dojo.reload(dojo._loadreloaded);
				dojo.stopEvent(e);
			}
		});
	}
	console.log('reloading module',d);
	dojo.require(d);
};

Just copy and paste this function somewhere to your frontend code base, and when you want to reload a particular module, just type dojo.reload('mymodule.myfile') in Firebug console, and the latest version of your module will be downloaded and ready to be used/tested on immediately. In order to make sure the browser does not use one from its cache, cacheBust will be set to current timestamp to make sure it is loading from server directly. Another fine touch in the dojo.reload function is that, if you just want to reload the module you last reloaded, simply hit F6 which saves you some type.

WARNING: this is only meant for development, don’t put this in your deployed code base. This is only tested in firefox 3, but it should work in other browsers as well. I think this will only work with the default loader and it will fail if you want to use it in xdomain case. At last, the famous phrase: use at your own risk.

Edit: fixed mangled greater than symbol (thanks slackorama). changed all this->dojo so that no matter what this function is called, it can always correctly unset any “dojo memory”. – December 2, 2008 at 7:51 am

Fri 22 Feb 2008

share dojo among iframes

Posted by liucougar under dojo
[5] Comments 

As pointed out by Jon’s blog post, simply use the dojo/dijit objects on the top window within an iframe won’t work for dojo APIs in most cases, because “the top level dojo object has no Out of the Box knowledge of the iframes or their content”.

Actually, dojo has two APIs in the core to support running code in the context of an iframe: dojo.withGlobal and dojo.withDoc. However, they do not have much attention, as most of their usage is only seen in the dijit.Editor code. Actually, they were introduced for the sole purpose of dojo.widget.Editor (which is the dojo 0.4 Editor widget) for dojo 0.4 in the first place.

Both of them have identical signature:

dojo.withGlobal(/*Object*/globalObject, /*Function*/callback,/*Object?*/thisObject, /*Array?*/cbArguments)

In order to use them, you can wrap your code in a function first, like this:

function iframefunc(){
    dojo.byId(‘foo’);
    dijit.byId(‘foo’);
    dojo.query(‘a.foo’);
}

and call the function as this in an iframe:

dojo.withGlobal(window,iframefunc)

You can also call dojo APIs directly use dojo.withGlobal. Say you want to run dojo.byId(‘foo’) in an iframe, you can do:

dojo.withGlobal(window,'byId',dojo,['foo'])

Note that, the above code can also run correctly in the context of the top window.

The differences between dojo.withGlobal and dojo.withDoc is the first argument: if you want to only change document of a function call operates on, you can use dojo.withDoc instead and pass it with the document of an iframe.

Thu 21 Feb 2008

dojo.{declare,extend} support for Komodo

Posted by liucougar under dojo
No Comments 

As an open source editor, komodo editor is targeted at dynamic programming language editing, which supports javascript, php, python and perl, among others.

One of the features for those supported dynamic languages is auto-generation of a list of available functions defined in a script. However, when editing dojo based javascript files, komodo can not pick up any classes declared using dojo.declare and it does not support dojo.extend either.

(more…)

Wed 6 Feb 2008

Our first child is arrived

Posted by liucougar under Life
[5] Comments 

Our first child, a little boy, nick named haohao (full name still needs to be decided) was just borned naturally today in York Hospital. As 6/Feb/2008 is the chinese new year eve, so haohao is still born in the year of pig, rater than the year of rat.

I will upload pictures of our little baby to the online gallery soon.

Thu 24 Jan 2008

latest rsync incompatible with EC2 kernel

Posted by liucougar under Linux
1 Comment 

I just noticed that re-bundling in EC2 instance is not working. Google told me that it is due to the incompatibility between latest rsync and relatively old kernel used by EC2 (2.6.16).

In the thread about this issue, 5 possible workarounds are presented. However, none of them are easy under gentoo without messing up with ebuild/emerge. I found another way, and you don’t need to regenerate the configure file for rsync at all, all you need is to comment the following line in the config.h file (which is generated after you run the configure script):

#define HAVE_LUTIMES 1

then make and make install, all should be fine now.

Sat 22 Dec 2007

Dynamic dns setup for gentoo EC2 AMI

Posted by liucougar under Linux
No Comments 

If you are using a dns service provider with dynamic dns support, and using a gentoo based EC2 AMI, then you may find this script to be useful. It can update the IP address for a specified domain name whenver the gentoo AMI is booted.

To use this script, just extract the file, and place ec2.conf under /etc/conf.d and rename it as ec2, then move file ec2 to /etc/init.d

Before you can use it, you have to set some parameters in file /etc/conf.d/ec2, please consult the comments in that file.

In order to register IP address whenever the AMI is getting booted, a final step is needed:

rc-update add ec2 default

Wed 21 Nov 2007

text.rb:292:in `normalize’: private method `gsub’ called for 35:Fixnum (NoMethodError)

Posted by liucougar under Linux
1 Comment 

Just before generating the manifest file, ec2-bundle-vol under gentoo, throws this error at me:

Googling around, it turns out that, it’s due to my “too recent” version of ruby (I only use stable packages in gentoo, but we all know gentoo is always cutting edge, even its stable branch, isn’t it?), which has a backward incompability. The fix is trivial:

— rexml/text.rb.orig 2007-10-22 08:00:04.000000000 +0100
+++ rexml/text.rb 2007-10-22 08:00:33.000000000 +0100
@@ -286,7 +286,7 @@
EREFERENCE = /&(?!#{Entity::NAME};)/
# Escapes all possible entities
def Text::normalize( input, doctype=nil, entity_filter=nil )
- copy = input
+ copy = input.to_s
# Doing it like this rather than in a loop improves the speed
#copy = copy.gsub( EREFERENCE, ‘&’ )
copy = copy.gsub( “&”, “&” )

— rexml/document.rb.orig 2007-10-22 08:02:36.000000000 +0100
+++ rexml/document.rb 2007-10-22 08:03:01.000000000 +0100
@@ -183,7 +183,7 @@
output = Output.new( output, xml_decl.encoding )
end
formatter = if indent > -1
- if transitive
+ if trans
REXML::Formatters::Transitive.new( indent, ie_hack )
else
REXML::Formatters::Pretty.new( indent, ie_hack )

Wed 21 Nov 2007

Install Amazon EC2 AMI Tools in gentoo host

Posted by liucougar under Linux
[2] Comments 

Amazon EC2 AIM tools are necessary if you want to create your own images (by either doing it from scratch or modifying others AIM) to use on Amazon EC2 service. That tool is only supporting linux, so they provide a RPM (maybe primarily targeted for FC?).

If you are a gentooer, then you have to do it manually. The following is what I have done to make it work

rpm2targz ec2-ami-tools.noarch.rpm #if you don't have rpm2targz, run emerge rpm2targz first
tar xvfz ec2-ami-tools.noarch.tar.gz
mv usr/local/* /usr/local/
mv usr/lib/site_ruby/ /usr/lib/
mv etc/aes/ /etc/
ln -s /usr/lib/site_ruby/aes /usr/lib/ruby/site_ruby/1.8/i686-linux/aes

After that, you should be able to run ec2-bundle-vol and others just fine.

Another tip: I have this in my ~/.bash_profile (~/.bashrc will do too):

#for Amazon EC2 AMI bundle
EC2_BUNDLE="-k /path/to/your/pk/pem -c /path/to/your/cert/.pem -u <user_id>"
alias ec2-bundle-vol="ec2-bundle-vol $EC2_BUNDLE"
alias ec2-bundle-image="ec2-bundle-image $EC2_BUNDLE"

So that you don’t need to specify all those mandatory options every time you invoke these commands.

« Previous Page — Next Page »