Optimising Websites

 All, Technology  Comments Off on Optimising Websites
Oct 072014

RapidFMS is not your typical website. From the outset and more than hinted by the name, it is supposed to be fast. To achieve the speed we do, many techniques were researched but also other’s recommendations were adopted. One site that contains a long list of recommendations for website performance as well as a tool to enable evaluation of you site from within your browser is Yahoo (See Yahoo’s performance recommendations here https://developer.yahoo.com/performance/rules.html).

For RapidFMS we actually looked at each and every one of them and this along with PageSpeed from Google (https://developers.google.com/speed/pagespeed/) and the awesome web tool GTMetrix http://gtmetrix.com/ we were able to double the very fast performance we had.

This is how we managed to reduce the loading time of RapidFMS (Sydney to Melbourne) from about 2s (non-cached login form) / 10s (non-cached application home) to 0.5s (non-cached login form) / 2s (non-cached application home) and how we reduced the size of the payloads to about 1/3rd. Note: the cached versions were already sub-1s to begin with but are now even faster than they were.

(We’ll go into some of our own additional findings after we go over Yahoo’s recommendations.)

Yahoo’s recommendations:

– Minimize HTTP Requests

Before we optimised we had about 114 JS files to download in our first payload. Partly this was due to our code modularisation of code but also a couple of the 3rd party components we used were modularised.

At the end of the day we have to download the content of the JS files, but the user’s perception is what matters. For the login form we only required about 80 of them, the rest were required only after having logged in.

Reducing the JS files from 114 to 80 gave a very noticeable speed increase in page loading. 80 is still a lot of JS files, so we looked at bundling them into different payloads as follows which worked very well and is what we have stuck with at this moment. The structure of the payload is somewhat important to ensure that what is needed early on is delivered, but also re-used later on. We put the minimum we required to bootstrap the system and load the login form into a bootstrap payload (payload 1). We put the 3rd party components that could be minified together in another (payload 2) and 3rd party components that wouldn’t work minified into the remaining (payload 3). Our system still caters for optional lazy loading of components, optional pre-fetching of non-bundled components to cater for 3rd party component oddities where required.

– Use a Content Delivery Network

Using a CDN is just not plain black and white any more. There are issues if you have an SSL site with non-SSL components with browsers poping up warnings and then there are Australia’s new privacy rules. To make things simpler for the latter, we decided not to use a CDN hosted overseas. In fact, we chose not to use a dedicated CDN all-together pretty much for the following reasons (http://wonko.com/post/javascript-ssl-cdn). Having said that, we believe our performance is still pretty awesome without but will consider our options here in future if required.

– Add an Expires or a Cache-Control Header

For most part we actually started with smart caching from the start – that is one of the main ways we managed to get such great performance from the outset. We didn’t have much to do here but we also cater for dumber (or older) browsers which don’t honour cache control mechanisms properly.

– Gzip Components

As a server-side setting, we always had the luxury of this enabled for our TEST / DEMO environments, but we actually had it disabled on our slower DEV environments. This does noticeably reduce the actually transmitted payload size by about half.

– Put Stylesheets at the Top

This is an HTML requirement, we always did this.

– Put Scripts at the Bottom

This is something that is quite easy to do for code you write, however it isn’t always possible for the 3rd party components you may use. For most part this really is worthwhile. Moving all the scripts we could was a pretty trivial exercise.

– Avoid CSS Expressions
– Make JavaScript and CSS External

Not only for performance reasons, but even for modularity and debugging ability, everything is just easier if your JS in particular are external. Nothing to do here.

– Reduce DNS Lookups

Nothing to do here.

– Minify JavaScript and CSS

We had already chosen to minify our JS files so we didn’t have anything to do here. However, we chose to not use a minifier but rather Google’s closure compiler (https://developers.google.com/closure/compiler/). Although technically it isn’t a minifier, it does an admiral job of having the same effect. In fact, in many many situations it can reduce code size to be smaller than a plain minifier (see my other blog here clould.com.au/?p=161) – and you can actually minify the closure compiled code for even greater impact – but… bundling of multiple minified files proved to be an issue, but bundling of multiple closure compiled wasn’t. So forgoing an additional saving of less than 1% we only closure compile.

– Avoid Redirects

For most part we don’t have a need for redirects as we essentially have a single-page Web App.

– Remove Duplicate Scripts

Nothing to do here.

– Configure ETags

At present we aren’t using ETags, something to play with in future.

– Make Ajax Cacheable

Making Ajax cachable is an interesting exercise. It makes sense for some things (lazy loaded JS files) but not others (dynamic data). We already did have fine control over what we required in this area so nothing to do here.

– Flush the Buffer Early

Likely an easy thing to try in future but our pages are very very small given the lazy loading nature so we won’t have much to send after a flush.

– Use GET for AJAX Requests

We already have the ability to use both GET or POST but for the time being we have left it configured to use POST. We didn’t see much noticeable different in performance here.

– Post-load Components

This is likely one of the single most important factors in making your website faster. Not only does a large application only then need to fetch (and cache) what a user wants to use, the individual payloads are so tiny the user doesn’t necessarily realise what was happening behind the scenes. Lazy loading forms is very easy as they are loaded and cached on demand.

– Preload Components

Like Yahoo says, this sounds like the opposite of Post-load, but actually what we call pre-fetch you can do in an intelligent way behind the scenes, in parallel to whatever the user happens to be doing.

– Reduce the Number of DOM Elements

Yahoo gives a simplistic explanation of this. In our research performance of a page wasn’t that great a difference with a few or a lot of DOM elements – what was more important was the structuring or ‘scoping’ of DOM searches. We like and use JQuery and the way the selectors work, but what most people don’t realise is that selectors have an optional scope – this scope is extremely important to use and use correctly to ensure you are not traversing the entire DOM tree whenever you need to select something.

– Split Components Across Domains

For similar reasons to using a CDN, we’ve chosen not to do anything here yet.

– Minimize the Number of iframes

None of the reasons not to use iframes suggested by Yahoo affect RapidFMS. Nothing to do here.

– No 404s

Nothing to do here.

– Reduce Cookie Size

We also don’t like cookies (unless they are edible) but we need a cookie. Because a cookie in our case is required, we have one and only one.

– Use Cookie-free Domains for Components

For similar reasons to using a CDN, we’ve chosen not to do anything here yet.

– Minimize DOM Access

See my comments above in the section Reduce the Number of DOM Elements.

– Develop Smart Event Handlers

There is a lot of good and bad that can be done with event handlers. We have actually expanded on the idea of how JQuery provides event handler’s to minimise the overheads required where large numbers of handlers may normally have been required. Having said that, we typically don’t require large numbers of event handlers.

– Choose <link> over @import

Although YSlow states we are using @import, we actually don’t have any in our or 3rd party components. Maybe there is a bug in YSlow that mis-identifies !important for @import? Haven’t investigated.

– Avoid Filters

Minor and we have decided not to consider this for the time being.

– Optimize Images

Agreed on this. Nothing to do here.

– Optimize CSS Sprites

We really only debated this for 5 of the images we use but the issue with this, is those images are only used in cases where we have a dumb browser (which may not cater for CSS Sprites). We could consider just removing the 5 images for text equivalents.

– Don’t Scale Images in HTML

Nothing to do here.

– Make favicon.ico Small and Cacheable

Nothing to do here.

– Keep Components under 25K

It’s worth doing your own research here and also reading articles such as this (http://www.guypo.com/uncategorized/mobile-browser-cache-sizes-round-2/). Not everything is as it seams and deferring or lazy loading components will help make your situation much much easier if you have to satisfy tight cache requirements.

– Pack Components into a Multipart Document

For future investigation.

– Avoid Empty Image src

This is easy to do and causes no issues. Nothing to do here.


Our additional recommendations:

– Client side over Server side

Make as much of your web site or web app client side. This with all caching considerations considered allows much of your site to be cached. This means the only payloads to/from the server should be data. A re-visit of RapidFMS to get to the home page has a payload of around 12kb.

– Lazy load lazy load

Where-ever you have the ability to do so in an un-noticeable manner, lazy load.

– Cache

Be sure to cache what you can in the most appropriate way. This can include caching in the file system or even caching in DOM for the session. The latter can be useful for devices that may not cache anything, but then watch your memory usage also.

– JQuery

JQuery is an awesome library, use it well. Learning selectors and scoping – and doing it right will make your code faster and more re-usable.

Here really only touches the surface of optimisations.  Perhaps one day we’ll do a followup articule.




Device Pairing coming to RapidOS

 All, Technology  Comments Off on Device Pairing coming to RapidOS
Oct 072014

We have been working on device pairing to bring a truely unique Web Experience unlike any other to the RapidOS framework. Any number of devices locally or remote using any mixture of Windows, OSX, Linux, WebOS, IOS, Android, Blackberry, Windows Phone and more can be paired together to seamlessly work as one, allowing a user to view and input things in a number of interesting ways. For example, using a set of tablet’s and a phone in place of a Desktop PC, allowing a touch screen device to be used for input, allowing tablets to be additional displays, even GoogleTV’s as additional displays not to mention we’ve retained the ability to print to any printer World Wide from all of these devices.


Monty Hall Problem Simulation

 All, Technology  Comments Off on Monty Hall Problem Simulation
Jun 162014

We often talk about maths, science and other interesting and often paradoxical problems.  One problem we had a good discussion of is the Monty Hall Problem.

To prove it, I created a simulation that can be run or forked from here: http://jsfiddle.net/zhulien/L654v/

There are a lot of discussions on the net and on youtube as to why it is the way it is, but not too many simulations to see it working in action.

How the simulation works:

Out of a defined set of rounds, I chose 10,000 as an arbitrary number, we allocate a control (random number between 1 and 2) as well as choose a door where the car is placed and where the person chooses.

We then evaluate the results of the allocations.  In one case, the always switch case, we count how many times we are NOT moving away from the car.  Given that Monty Hall has always chosen one of the goats, then we will also be moving away from a goat in this situation.  In the other case, the 50/50 case, we are using our existing allocated control number to decide whether we would switch or not.  This is a 50/50 decision as to whether we would be moving away from the car or to the car – makes little difference which way you take it.  Our control skew which is very small is our inaccuracy.  Surprisingly, even though at the start we knew 100% of the time we would have one of our negative choices eliminated by Monty Hall, we still have 2/3rd chance of winning the car if we switch 100% of the time vs 50% if we treat the decision as a new one between 2 doors.

Interestingly… I played with the idea of putting the car always behind a fixed door (let’s say 1).  And it made zero impact on the outcome.  I was wondering if ignoring human behaviour and leaving it purely to chance, if randomizing the door the car was behind would impact the outcome – it didn’t.  It’s all about the chance of moving ‘away’ from the car.


Cut the crap out of freight

 All, Business, Technology  Comments Off on Cut the crap out of freight
May 152014

Cut the crap out of freight. Subscribe to the RNS standard of barcode numbering, a one off cost will give you an infinite sized range of Consignment numbers, Manifest Numbers and Item Numbers to last your entire business’s lifetime – and you can easily exchange data with other businesses using the same RNS standard and they can all co-exist in the one system making tracking a breeze.

If you want you can personalise your numbering scheme today. How would you love to have SANTA as your exclusive numbering system prefix. At a special introductory price we have a SANTA prefix just for you. Don’t let someone else get your preferred number prefix and contact us today!

We’ll even show you how you can allocate these same allocated numbers to all of your senders, receivers and 3rd parties… best of all, you will never run out!


RapidFMS Compatability as of April 2014

 All, Business, Technology  Comments Off on RapidFMS Compatability as of April 2014
Apr 152014

RapidFMS Compatability (Clients)

  • Various Android based phones 2.2 and later
  • Various Android based Media players
  • iPad, iPad 2, iPad 3, iPhone 3G, iPhone 4
  • Google Chromebook
  • Windows Phone 8
  • Blackberry Playbook
  • Windows PC (Windows XP, Windows 7, Windows 8)
  • Mac with OSX (Leopard, Snow Leopard and Lion)
  • Linux Ubuntu (and other flavours)
  • Google TV
  • Blackberry Bold 9780 (the browser sucks, it works but not really nicely)
  • SONY Playstation 3
  • Nintendo Wii (works suprisingly well if not a little slow – likely due to lack of RAM)

RapidFMS Compatability (Servers)

  • Various Android based phones 2.2 and later
  • Various Android based Media players
  • Windows PC (Windows XP, Windows 7, Windows 8)
  • Mac with OSX (Leopard, Snow Leopard and Lion)
  • Linux Ubuntu (and other flavours)



 All, Business, Technology  Comments Off on RapidFMS
Mar 242014

RapidFMS is the ideal platform to build your custom Freight Software Solution. We pride ourselves on developing the most up to date solutions with years of investment in R&D to provide our customers with a solid platform to build upon now and for the foreseeable future.

Why Choose Us?

Our core group has over 40 years experience in the transport industry and has recognised the many changes in the way businesses need to dispatch and label their freight for acceptance by their customers while at the same time being compliant with requirements.

We believe RapidFMS to be one of the most advanced high performance freight systems available today with features such as the ability to work on multiple consignments at the same time, sign on glass or the ability to print remotely and directly to thermal or laser printers anywhere across the World even from a tablet or mobile phone.

RapidFMS is compatible with Windows, Linux, MacOS desktop systems as well as iOS, Android, Blackberry and Windows Phone devices and more including in-vehicle devices.  Also compatible with the most popular Web Browsers, RapidFMS can be used from anywhere around the World.

The Freight Management System itself has many features that can be tailored to your needs. The system is suitable for a whole range of clients ranging from Freight Brokers, Carriers, Wholesalers / Large Retail chains to Small Businesses.

  • Leveraging off proven development libraries, standards and techniques, we are able to provide customisations to our software in favourable timeframes at highly competitive rates.
  • Users whether they are clients, staff or management can work more efficiently by working with multiple forms and documents at the same time. This multitasking capability enables a continuous flow of work with less interruption.
  • Your important data is secured from exploitation giving you peace of mind. Security was part of the system’s design from the start and will remain important for the life of the system.
  • Users will wait less. RapidFMS is faster and smarter! Intelligent caching and lower network traffic than your traditional Web App enables high performance even over a 3G / 4G mobile network. This is particularly important when using mobile or in-vehicle devices.
  • Your users expect software to run on their device. With out-of-the-box support for Windows, OSX, Linux, Android and IOS you will have catered for almost everyone.
  • Most business applications have the need to print, be it reports, labels or charts. RapidFMS has inbuilt printing services including the ability to print to locally attached printers, over a network, over the internet and even from mobile devices using Android and IOS.
  • We found charting was so desirable to our clients we built charting services into RapidFMS. You can display multiple charts of multiple styles, report on them or just watch them update in near real-time.
  • Branding is your business’ identity and it’s important for client-facing and even sometimes internal-facing Apps that they are branded for your business.
  • With its modular design, RapidFMS is easily extensible allowing additional functionality to be added easily as your business grows.
  • RapidFMS has inbuilt document repositories for document or images storage and retrieval.
  • B2B Internet Services as well as in-house software integration facilities are already built into RapidFMS allowing a greater choice in how you can provide your staff and customer services.
  • An important factor of any project often overlooked is the planned lifespan of a project when making technology choices on which to base it. RapidFMS uses current mainstream marketplace technologies, so you should be able to look back in a decade’s time with a smile.
  • Because of the mainstream technology choices for which RapidFMS is based upon it is easy for you to find those additional development resources as you require.
  • RapidFMS gives you a choice second to none of hosting options. Host it on a server or locally on any of the following – Linux, Windows, OSX or even on Android.
  • With the investment we have already put into RapidFMS, we continually research and invest back into the product providing our customers with regular enhancements, bug fixes and if they ever arise, timely security patches.

Want to know more about RapidFMS


Just a new RapidFMS preview

 All, Technology  Comments Off on Just a new RapidFMS preview
Feb 172014

Here is a new screen shot of RapidFMS showing the Calendar, the File Format editing form that lets you create new file formats, the Sign on Glass form as well as the new Script Editor form which allows things such as workflow to be scripted visually.



Choosing your Web Technologies

 All, Technology  Comments Off on Choosing your Web Technologies
Feb 052014

RapidOS is a relatively new web technology based framework / platform for developing advanced desktop like web apps quickly.

We had quite a lot of requirements, but among the highest priority were:

1 – must be developed in a commonly used well established programming language.
2 – have maximum hosting options available including on mobile devices themselves.
3 – have few application architecture restrictions.
4 – have a low learning curve for current and future developers.
5 – must be relatively quick and easy to implement user requirements.

When we sat down to create RapidOS, we first looked at available programming languages as well as experimenting with many of their existing frameworks.  We soon realised that although some of the frameworks were promising, they generally were not well suited to our requirements.

We had a good idea what the most commonly used technologies were, and a quick investigation of hundreds of popular websites / web apps proved what we already knew. w3techs.com is one of the leading web technology usage stats publishing websites – their stats also reflect somewhat our findings. Let’s look at the top 3 most popular techs in each of the categories below (stats taken from w3techs.com):

Server-side Languages: PHP 81.7%, ASP.NET 18%, Java 2.7%

Interesting these stats. Although we knew these were the top 3 platforms, we never reaslised how little used Java was as a server-side language. Java isn’t a bad choice, nor is ASP.NET, but of these only PHP satisfied our number 2 priority. In fact we have done POCs using ASP.NET and Java also should we require to develop a server in these techs one day, but at this time not a priority.  We like the idea of JavaScript on the server but it isn’t even a blip in the stats nor does it satisfy our number 2 priority – over time this may change. We chose to code our current server in 100% PHP.

Client-side Languages: JavaScript 88.1%, Flash 15.3%, Silverlight 0.2%

We don’t mind what can be achieved with Flash, but it was never a consideration. Neither was Silverlight. In fact, we considered JavaScript, Java and PHP only for this – but if we chose PHP, our client wouldn’t be ‘as rich’.

A RapidOS App has a client component and a server component – both could be hosted in a browser but both could be hosted on a PC or Server. In the end we chose to use both JavaScript and PHP for the client (98% JavaScript, 1% PHP and 1% Java).

JavaScript Libraries: JQuery 57.5%, Mootools 4.9%, Modernizr 4.6%

As we had chosen JavaScript as our client-side language, we looked at whether we wanted to use any util libraries. We liked the way the JQuery selectors worked and the way it hid some of the browser idiosyncrasies and the fact that there is a vast array of plugins and 3rd party components that also use JQuery, so we chose to use JQuery. In fact we use a few other less common libraries too which are simple to redevelop should we ever need.

Markup Languages: XHTML 50.5%, HTML 50.2%

We use XHTML for most part, because the World is moving that way, but that doesn’t really matter so much for now. Our templates are 100% XHTML/HTML with zero logic or embedded funny extensions.

Site Elements: CSS 90%, Compression 54.2%, Cookies 46.5%

Cool, interesting to know that chances are, whatever we do in CSS (sans HTML5 extensions) are usable by most of the World.

Web Servers: Apache 62.7%, Nginx 18%, Microsoft IIS 14.4%

Related to our language choices, we mostly host on Apache, but our client and server also works on Microsoft IIS. That gives us at least 77.1% target hosting compatibility based on those stats.

Content Languages: English 55.5%, Russian 6.1%, German 6%

We only speak English, but our software is easy to cater for multiple languages.  We have made specific effort to ensure that the core foundation of our software is Asian language capable due to one of our requirements.