Sticky CDN - A Content Delivery Network (CDN) For Your Local Development Environment
So, last week, I had an itch to create a content delivery network (CDN) for my local development environment. I was able to get something working in a few hours; but it looked junky and I've since spent the last few mornings refactoring it into something that feels a little bit more elegant. I present Sticky CDN - a super simple CDN built in ColdFusion.
View this project on my GitHub account.
The concept behind Sticky CDN is simple - you make requests to it (Sticky CDN) and it turns around and makes the same request to the origin server mapped in your configuration file. A single Sticky CDN instance can work with any number of incoming domains and outgoing origin servers.
If Sticky CDN can pull the requested file from the origin server, it caches it locally. On subsequent requests, it then pulls the file from the local cache instead of going back to the origin server. I try my best to adhere to expiration dates defined in the "Expires" and "Cache-Control" headers; but, if I can't find (or parse) those headers, I default to caching the file for 7 days (which can be overridden in the configuration file).
At this point, you're probably wondering why I didn't just use something like Squid or Varnish? Great question. For starters, building something is just more fun, isn't it? Plus, I find that tinkering helps me build a stronger mental model for how these types of things actually work. And, of course, my primary development language is ColdFusion, so having a small CDN built in ColdFusion means a much more effort-free installation process.
Anyway, this is really just for my own personal use; but, I thought I would share.
Reader Comments
Local Storage = file system, right? Are you leveraging Ehcache? (I didn't see it referenced when I browsed the source.) Knowing that browsers may have up to 8 parallel connections, how many simultaneous requests can it handle?
One pitfall is Tomcat's permanent caching of DNS. If you move any resources, StickyCDN won't timeout and DNS won't refresh until the CF service is restarted. (The settings are only "always check" or "never", right?)
We started doing similar with IIS, but only using static JS, CSS and image resources and long expires. We gained better performance when we switched back to local hosting and added IISpeed (since it automatically minifies/concats & hashes filenames to force refreshes when resources are updated.)
Regarding proxying resources, have you looked at the options of Apache Traffic Server? I initially considered installing it locally, but I am now considering working with a 3rd party to ensure it's configured correctly.
https://docs.trafficserver.apache.org/en/latest/reference/configuration/index.en.html
@James,
Re "local storage", correct, it's just reading/writing to the filesystem. Essentially, it makes an HTTP request to the origin server; then, saves that file to the local file system:
sticky_cdn/site/cache/
Then, on subsequent requests, it looks at the local file system first, and if it exists, just pipes it into the CFContent tag for a response:
<cfcontent file="#response.filepath#" />
It will lock the requests based on the unique URL of the request. So, it will single-thread concurrent requests for the same file. But, for different files, with unique URLs, it should handle as much traffic as ColdFusion can handle... of course, disk I/O would be a point of contention.
But, just as a focusing perspective, this was intended only to be used for my local development environment - for when my production uses a "real" CDN, and I want to be able to mimic some of that behavior locally, during development.
As far as DNS caching, I can't say. Each browser seems to handle that differently. When I change my local DNS (/etc/hosts), Firefox seems to pick it up right away while Chrome seems to sometimes get cranky about it.
I've heard of, but I haven't looked at Apache Traffic Server yet. Part of the reason is that we're still running Windows in production mostly; but I do develop on Mac (Linux) with Apache, so I will definitely dig into it.