A case study to demonstrate how Unicorn developed an Electos Web Service to improve the performance of an existing website for a public sector organisation.
The Business Requirement
A public sector organisation had a customised Electos Content Management System (CMS) web site, which, in addition to the usual navigational menu system, also featured an A-Z index of pages.
Functionally the site worked well, although there were some minor display issues in browsers other than Internet Explorer, but since the entire user community was standardised on IE, that was not of itself a problem.
It worked well… but it didn’t work quickly. In fact it was so slow as to be unusable from any practical point of view. It took from 7 to 10 seconds to navigate from one page to the next!
The Challenges
The first thing to do was to understand why the site was behaving so slowly. We took a copy of the site and set it up on an in-house server and observed its behaviour.
Identifying the Root of the Problem
The agonisingly slow page-to-page navigation was immediately apparent. Looking at the process in Firebug’s (a plug-in debugging environment for the Mozilla Firefox web browser) “Net” tab, which breaks down how long each network request (and there can be many for a single web page) takes to complete, quickly identified the culprit to be the basic page generation itself.
Investigating further involved placing some debugging and timing code in the ASP files, which the site used to generate its content using VBScript calls to the Electos web application. This code measured the time various aspects of the page took to generate and output its results as HTML comments within the web page, where they could be viewed using “View Page Source” or similar (such as the “HTML” tab in Firebug).
This approach rapidly highlighted that the major culprits were the menu system (taking around 2 seconds on each page generation) and the A-Z index mechanism (taking 5 to 7 seconds on each page generation).
Looking more deeply into the code for the biggest offender – the A-Z index – we calculated that for the roughly 130 pages that were being indexed, the ASP code was making well in excess of 700 calls to the Electos web application. Not only was this a bad situation, but one that would only get worse as more and more pages were, as was anticipated, added to the site.
Our Proposals
Following a thorough assessment, we outlined the problems to the customer.
One aspect was the length of time it was taking to extract the index and menu content from the Electos web application through VBScript calls from the ASP code.
Another was the very design of the CMS system itself, which by its nature served up complete HTML pages to the user’s browser on every page navigation, which in turn meant that the menu and index content was being regenerated every time the user requested a page.
We suggested three possible solutions:
- Do without the A-Z index. This would not make the site exactly quick, but it would at least make it usable, which it wasn’t as it stood. Was the index crucial?
- Create the page without the A-Z index content, or the menu content, but then provide a mechanism which would load that content after the page had been displayed to the user.
- Radically change the structural design of the site so that the offending menu and index was only generated once and remained static throughout a user’s session, with only the internal page content being served up on page-to-page navigation.
The customer opted for the second choice. The index was too important to do without, while complete architectural redesign was both too large, and too risky, an undertaking to contemplate in an era of constrained budgets and time-scales.
The Unicorn Solution
Unicorn used a copy of the source code of the Electos system (available as the “Electos Developer Edition“). This allowed us to work out exactly what each of those hundreds of VBScript calls on the ASP pages (which, after all, had been producing the technically correct results) was doing.
This in turn allowed us to create a parallel web service (in a different workspace, and hence in a different virtual directory on the web server, but accessing the same Electos data) which would be able to serve up the same content as was currently being written into the pages by the ASP code.
We built the new web service and also built JavaScript code into the web pages to invoke the web service, first for the menu content and then for the index content. We had the web service invocation use the “JSON” protocol option (supported in Visual DataFlex since version 15), both for more compact data transmission “on the wire” and for the vastly greater ease of dealing with JSON (as opposed to XML/SOAP) data in JavaScript. That JavaScript code then took the returned content and inserted it into the page to provide the menu system and A-Z index, formatted exactly as it had been by the original ASP code.
This might seem as if we had simply moved the problem from one place to another: from the initial page generation on the server to a post-page-load process over the network, but this was not the case.
While the same work – in retrieving the content from the database – was still being done, it was now being done within the scope of a single call (actually two calls: one for the menu content and one for the index content) rather than many hundreds, and in a single process within the high-performance Visual DataFlex web application providing the web service.
Not only was a user clicking on a link immediately presented with the target page’s content, but the subsequent building of the menu and index content became very much faster. On our venerable server in the office the loading of the menu content through the web service takes around 0.2 a second, while that of the index takes around 0.5 of a second. Contrast that with the 2 seconds and 5-7 seconds in the original site and you can see that, in addition to the user getting a sub-one-second display of the page content, the loading of the menu and index has been speeded up by a factor of roughly 10. We are told by the customer that in their production environment this is even faster.
In addition, because the two JavaScript calls to the web service are asynchronous (what the “A” in “Ajax” stands for after all), they are processed in parallel, rather than sequentially, meaning that those 0.2 and 0.5 seconds are not cumulative, but overlap, giving a total time for everything to load of under 1.5 seconds.
Naturally the customer was delighted!