Experimenting With HTML5's Cache Manifest For Offline Web Applications
Posted June 21, 2010 at 10:06 AM
After playing with Safari's SQLite support for use with creating client-side databases, I thought I would try looking into HTML5's "Cache Manifest" for creating offline web applications. The Cache Manifest is a text file that lists out all of the application resources that need to be cached in order for the given application to work without an internet connection. The files listed in the cache manifest get stored in the "Application Cache". The Application cache is like the browser's standard disk cache, but much more robust. Once a resource is stored in the application cache, the browser will never request it from the web until the cached item is de-cached or the cache manifest is invalidated.
| | | | | |
| | | |||
| | | |
When it comes to using the cache manifest, I'm less interested in the "offline" aspect of it and more interested in the beasty caching mechanism that comes with it. Once the browsers sees the cache manifest file and gets your approval to create an offline cache, the browser then proceeds to progressively download all cachable items in the background. As such, you can hit your homepage (for example) and end up getting the entire application cached. And, once the application resources are cached, navigating around the application essential becomes a local operation, not a remote one.
To experiment with the HTML5 Cache Manifest, I created a very simple photo gallery page that lists out a few links to view higher resolution photos. Each of the "view" pages is a ColdFusion page that dynamically loads a given photo based on the photo ID. The HTML for the list page is rather straightforward:
- <!---
- Use an HTML5 doc-type. This is required for some browsers to
- work properly with the offline-cache.
- --->
- <!DOCTYPE HTML>
-
- <!---
- Add a file to the cache manifest. NOTE: Any file that calls
- a cache manifest file will be automatically cached, even if
- it is in the WHITELIST portion of the cache manifest.
- --->
- <html manifest="./cache_manifest.cfm">
- <head>
- <title>Cache Manifest For Offline Pages</title>
-
- <!---
- These are the two graphics (logo and startup screen) use
- for "app mode" on the iPhone (when the web page is put on
- the main screen as its own logo).
-
- NOTE: This part has nothing to do with the cache manifest
- stuff - this is strictly related to runnin a website in
- "App Mode" on an iPhone. Feel free to remove this entirely
- from the example.
- --->
- <link rel="apple-touch-icon" href="icon.png"/>
- <link rel="apple-touch-startup-image" href="home.png" />
- </head>
- <body>
-
- <h1>
- Cache Manifest For Offline Pages
- </h1>
-
- <h3>
- Pictures of Alisha Morrow
- </h3>
-
- <ul>
- <cfoutput>
-
- <!---
- Notice that we are dynamically generating this
- page. It will only be generated on the first
- request. After that, this page will be coming out
- of the cache.
- --->
- <cfloop
- index="pictureIndex"
- from="1"
- to="5"
- step="1">
-
- <li>
- <a href="view.cfm?id=#pictureIndex#"
- >View Image #pictureIndex#</a>
- </li>
-
- </cfloop>
-
- </cfoutput>
- </ul>
-
- <p>
- Now:
- <cfoutput>
- #timeFormat( now() , "hh:mm:ss TT" )#
- </cfoutput>
- </p>
-
- </body>
- </html>
When it comes to the cache manifest, there are two crucial things going on here. First, because cache manifest is part of HTML5, we need to use the HTML5 doctype:
- <!DOCTYPE HTML>
While this is not required for all browsers (supposedly), it is recommended for cross-browser comparability. The second, and perhaps more important thing that we are doing is including a link to the actual cache manifest file. This link is defined in the HTML tag:
- <html manifest="./cache_manifest.cfm">
This file, cache_manifest.cfm, is the file that lists out all of the cachable resources in our application:
Cache Manifest (cache_manifest.cfm)
- <!---
- Define the Cache Manifest content. I'm doing it this way since
- the "CACHE MANIFEST" line needs to be the first line in the file
- and storing it in a buffer allows us to TRIM later without having
- ugly line breaks.
- --->
- <cfsavecontent variable="cacheManifest">
-
- <!---
- NOTE: Cache Manifest must be the very first thing in this
- manifest file.
- --->
- CACHE MANIFEST
-
- <!---
- When a cache manifest is reviewed by the browser, it uses a
- complete byte-wise comparison. As such, we can use COMMENTS
- to defunk a previously used cache manifest. In this way, we
- can use a version-comment to indicate change even when the
- file list has not changed.
-
- NOTE: If ANY part of this file is different from the previous
- cache manifest, ALL of the files are re-downloaded.
- --->
- # Cache Manifest Version: 1.3
-
- <!---
- Let's list the file that get cached. The URLs to these files
- are relative to the cache manifest file (or absolute).
- --->
- # Core files.
- ./index.cfm
-
- # iPhone App files.
- ./icon.png
- ./home.png
-
- # View Pages. Notice that these can be dynamic pages as long
- # as they are valid URLs.
- ./view.cfm?id=1
- ./view.cfm?id=2
- ./view.cfm?id=3
- ./view.cfm?id=4
- ./view.cfm?id=5
-
- # Images that get viewed.
- ./images/1.jpg
- ./images/2.jpg
- ./images/3.jpg
- ./images/4.jpg
- ./images/5.jpg
-
- </cfsavecontent>
-
-
- <!--- ----------------------------------------------------- --->
- <!--- ----------------------------------------------------- --->
-
-
- <!---
- Let's reset the output and set the appropriate content type.
- It is critical that the manifest file be served up as a type
- "text/cache-manifest" mime-type.
-
- NOTE: We need to be careful about the whitespace here since
- the very first line of the file must contain the phrase,
- "CACHE MANIFEST". As such, we must TRIM() the content.
- --->
- <cfcontent
- type="text/cache-manifest"
- variable="#toBinary( toBase64( trim( cacheManifest ) ) )#"
- />
When it comes to the cache manifest, there are few essential points to understand. One, the phrase, "CACHE MANIFEST", must be the very first piece of content in the cache manifest file. Two, the file must be served up as the mime-type, "text/cache-manifest". And three, the entire cache manifest file is versioned by byte. This last point means that if any part of the cache manifest changes, including any of the comments (#), that cache is invalidated. As this point - when the cache is invalidated - the entire list of cachable resources must be downloaded again.
Just as in a CSS file, the resource paths listed in the cache manifest can be full URLs or URLs relative to the location of the cache manifest file. Because we are caching dynamic pages, each "dynamic version" of the page needs to be individually listed out in the cache manifest. There are other ways to do this, but for my experiment, this was the lowest-hanging-fruit solution.
The "view" page that I am linking to does nothing more than take the given ID and dynamically generate an IMG tag.
view.cfm
- <!--- Param the image ID that we are going to view. --->
- <cfparam name="url.id" type="numeric" />
-
-
- <!---
- Use an HTML5 doc-type. This is required for some browsers to
- work properly with the offline-cache.
- --->
- <!DOCTYPE HTML>
-
- <!---
- Add a file to the cache manifest. NOTE: Any file that calls
- a cache manifest file will be automatically cached, even if
- it is in the WHITELIST portion of the cache manifest.
- --->
- <html manifest="./cache_manifest.cfm">
- <head>
- <title>View Image</title>
- </head>
- <body>
-
- <h1>
- View Image
- </h1>
-
- <p>
- « <a href="./index.cfm">Back to List</a>
- </p>
-
- <p>
- <cfoutput>
- <img src="./images/#url.id#.jpg" width="300" />
- </cfoutput>
- </p>
-
- </body>
- </html>
I am including the cache manifest link in the view page as well; although, I don't believe it is required. As long as a user hits the list page first, the cache manifest will be downloaded and the view pages should be cached automatically. I simply kept it here for good measure.
In the list page, you may have noticed a few bizarre LINK tags. These LINK tags are specific to the Apple iPhone and are used to define the Icon and Startup screen graphics to be used when the web application is running in "App Mode". This is the full-screen mode that can be achieved when the given URL is added to the home screen of the iPhone. These link tags are not relevant to the cache manifest; but, they are indicative of where I want to go with this type of HTML5 exploration - building native-esque web applications for the iPhone.
The cache manifest seems like a very cool feature of HTML5. I was able to get this example running in FireFox, Safari, and on my iPhone (didn't test any other browsers). I didn't do too much with my desktop Safari research, but it was mighty awesome to see this working in Offline mode in FireFox and in Airplane mode on the iPhone. As I have stated before, where I'd like to go with this research ultimately, is building my Dig Deep Fitness web application to run as if it were a native application on the iPhone.
NOTE: When I first started playing around with the cache manifest, I was having the hardest time getting the offline cache to be recognized. For two days I kept checking my spelling and my linking to no avail. Then, I restarted my FireFox and blam - it starts working. So, if this doesn't work right off the bat, you might have to restart your browser.
Reader Comments
The Manifest is really nice as you can tell the browser to download objects when it does reach the listed content of a website, I am still unsure if this is more of a benefit to those websites that don't turn on headers through IIS/Apache for setting a content expires header.
@Josh,
I think one of the nicest benefits of it is that it will download the given files in the background. As such, you can download and cache files that you have not even been to yet, which is very different than disk cache (which requires a visit first). Also, from what I have read, the SIZE of the cache is very different. Especially on the iPhone, the disk cache is very small, where as the offline cache is much larger.
Speaking of headers, though, what I have seen one person do is actually set an expires header on the cache manifest itself. In this way, the browser doesn't keep trying to check the stored manifest against the life one (as it is explicitly told that the cache manifest won't expire for a given amount of time). So, in that case, it's kind of the best of all worlds.
Thanks Ben. I LOVE what HTML5 and CSS3 are doing to make life easier. Kinda makes the callouses on the fingertips, and forehead bruises a little futile now.. :-)
@Billy,
Ha ha, yeah the CSS3 stuff is especially cool (once you bail on IE6).
This morning, I took this Cache Manifest stuff a bit further and used it to create native-like web applications on the iPhone:
A comment about this comment:
Add a file to the cache manifest. NOTE: Any file that calls
a cache manifest file will be automatically cached, even if
it is in the WHITELIST portion of the cache manifest.
That seems problematic. If I want to cache images for example, but not the HTML of a page, it sounds like the second I link to the cache file from index.cfm, then index.cfm ITSELF is cached. Is that right?
@Raymond,
Yes, that's my experience with it. I had a ColdFusion page a while back (when I first looked into this) that needs to do just that - cache the images for a photo library. Unfortunately, the second I started using the cache manifest, the ColdFusion page stopped being dynamic :) So, it kind of defeated the purpose.
Hmm. So forgive me if this is covered - I'm mainly just talking this out to myself.
If I only wanted to cache images - I'd have to at least cache the index page too. And if the index itself was dynamic, I'd want to use a Cache Manifest Version that changed whenever the images changed.
@Raymond,
This is exactly what I was trying to battle. I tried including an IFrame in the index page that linked to a page that did nothing more than include the Manifest directive. I can't remember if that worked or not (part of me feels that it did not).
This raises a great question though, one for clarification. Can a non-cached page make use of any cached items? In other words, does the calling page have to include the cache manifest for it to be able to make use of the application cache?
I guess where I'm going with this is - how does the URL to cached items work? It it *strictly* based on resource URL? Or, is it a combination of resource URL and the calling page?
I'll have to play around with that a bit to see what kind of functionality I can get in offline mode in FireFox. Good food for thought!
I would have assumed that if you are on
n.html
and n.html points to a cache file that says urls A+B+C are cached, that n.html need not be cached. But if you click a link to A within n.html, it would use the cached version if offline.
Just guessing.
@Raymond,
I just tried playing around with this non-cached / cached interaction and found some interesting results. At least in "offline" work mode in FireFox, it appears that non-cached pages cannot take advantage of cached images:
FYI, darn good article on the cache:
http://www.html5rocks.com/tutorials/offline/takingappoffline/
@Raymond,
Oh good link, thanks. I saw that the HTML5Rocks web site was just launched like Monday or something on Twitter but have not had a chance to look through it yet. I am sure it is chock full of goodness :)
Curious about something, Ben: Why the toBinary(toBase64())?
When I saw that, I thought that maybe I had missed something. So I looked through http://developer.apple.com/safari/library/documentation/iphone/conceptual/safarijsdatabaseguide/SafariJSDatabaseGuide.pdf again, and there isn't any mention about the cache manifest being anything but text. Also no mention of making the manifest binary at the http://www.html5rocks.com/tutorials/offline/takingappoffline/ URL that Raymond mentioned. And also nothing at the official http://www.w3.org/TR/offline-webapps/ document.
When you think about it, text/cache-manifest has the primary mime type of text, which would argue that you should not make it binary.
So now I'm more confused than ever.
I have no doubt that you read something somewhere about it that made you want to add toBinary(toBase64()). And from your success with BuffBabes, it apparently works fine. But I'd like to read more about the binary conversion rationale if you have some URLs to quote.
Thanks.
Hi Ben,
Just want to say really enjoying this series on HTML5 features, I'm using them at the same time and you're insights are really helping.
I'm using a dynamic cache.manifest.php file (to add entries for all subfolders/images/css) and I've found it a bit hit and miss as to whether the app displays the new content after updating this file.
Usually it takes at least 30 seconds after I've edited the manifest before I can run the app (from Home Screen icon) and have it display the new content.
Not sure where this arbitrary 30 seconds comes in but as long as it works!
@Steve,
The toBinary() thing has nothing to do with the cache manifest itself. It has to do with the CFContent tag in ColdFusion. It's just my personal preference / style when streaming content back to the client. The CFContent tag has a "Variable" attribute that allows you to pass in a binary data value that will become the response data. As such, I need to convert my respond output to a byte array before I can use it with CFContent.
Again, just my personal preference. And in this case, where the "CACHE MANIFEST" directive needs to be the very first thing in the response, this technique, in combination with trim(), really makes things easier.
@Richard,
I have also experienced the delay in how the updates to the cache manifest affect the update. When testing in FireFox, I typically just clear out the offline store and refresh. In Safari on the iPhone, however, I was finding that it did have a noticeable delay in when the changes were acknowledged.
@Ben, thanks. I'm an Adobe Certified Advanced CF developer, but the binary aspect of the cfcontent variable attribute slipped under my radar. Good to know that you have to call toBase64 too to get it debinarized back to text.
FWIW, I use enablecfoutputonly for the same grabbage strippage. (Sorry. Just got back from Fogo de Chao. Post-huge-meal anemic torpor has me playing fast and loose with whether or not something is really a word.)
@Steve,
Mmmm, Fogo :) It's so good it hurts.
Yeah, the Variable attribute of CFContent I believe is new in CF8. I think I'm pretty much the only person who loves it. I very rarely ever see anyone else use it. I just find it makes tailoring the output much easier since you don't have to think about settings and line breaks.
But of course, to each their own. I happen to love it; but not everyone thinks it's all that.
I think the delay is just to do with how the update works. It updates in the background first, and does not automatically replace the version you're looking it. I've written a bit of code which allows the user to choose to update (this works for the data-entry app I have been working on).
So the update prompt appears if the manifest has changed, and roughty 1 minute or however long the assets take to download. Then they are applied and the page is refreshed.
@Richard,
Cool stuff. I've seen the Javascript hooks in the Cache Manifest documentation but have not looked into them at all. From your code, it looks pretty straightforward. I'll have to play around with it. Thanks for sharing the link.
hi ben,
nice tutorial but i found difficulty in update data with javascript
i found this
window.applicationCache.swapCache();
I try this but still cache data is not updating
do you have any idea?
@Kuldeep,
I haven't actually tried any of the Javascript-based ways of interacting with the offline application cache. When I get to that, I'll see if I can shed more light on the issue.
Hi Ben, having checked all articles on Html5's appCache, is there a solution to just update newer files, using the manifest file? I am looking into using application cache to actually have an offline website running on USB stick which is capable to update itself. But only download the changed or added files?
It seems that AppCache lacks support with this particular matter.



