Using Hosted Graphite With ColdFusion And GraphiteD
I want to measure all the things! Unfortunately, I'm not so great at it. So, I want to really start digging into things like StatsD and Graphite. For me, I learn best (sometimes) by building. So, instead of installing StatsD and Graphite (which I hear is a pain to do), I built GraphiteD.cfc - a ColdFusion component that aggregates StatsD-inspired metrics and feeds them into the remote Graphite SaaS (Software as a Service) - HostedGraphite.com.
| |
|
|
||
| |
 |
|
||
| |
|
|
View my GraphiteD.cfc project on GitHub.
I actually found HostedGraphite.com first, and then started building GraphiteD.cfc as a means to feed data into the remotely hosted Graphite service. This was a fun experiment because it helped me to learn more about using UDP (User Datagram Protocol) in ColdFusion, the philosophy of StatsD, and the way in which Graphite stores and represents data.
GraphiteD.cfc aggregates three different kinds of metrics: Counters, Timers, and Gauges. Each of these metrics is defined by a dot-delimited name and a numeric value:
- GraphiteD.recordCounterMetric( name, value ) :: void
- GraphiteD.recordGaugeMetric( name, value ) :: void
- GraphiteD.recordTimerMetric( name, value ) :: void
The way in which the value is consumed depends on the type of metric being recorded. Counters add the value to a running total; Timers apply the value to a running set of calculations, including min, max, average, count, and total; and Gauges simply record the given value, as is. These metrics were inspired by the StatsD Node.js daemon, but are not quite as intelligent and do not present any of the configuration options presented by StatsD.
I'm sure that I could have squeezed all of the GraphiteD.cfc code into a single ColdFusion component; but, as with all of these kinds of experiments, I am trying to learn how to better structure my code. As such, I tried to break the code up into a set of focused components that have a small set of responsibilities:
- GraphiteD.cfc - The main ColdFusion component that exposes the metric-recording behavior.
- Interval.cfc - The collection of all the metrics being recorded.
- Metric.cfc - The base component for metric behavior (meant to be sub-classed).
- Counter.cfc - The representation of a counter metric.
- Timer.cfc - The representation of a timer metric.
- Gauge.cfc - The representation of a gauge metric.
- UDPTransport.cfc - The component that compiles and sends the metrics to HostedGraphite.com over UDP.
- HTTPTransport.cfc - The component that compiles and sends the metrics to HostedGraphite.com over HTTP.
I don't currently have any tests in place; but, breaking up the logic into smaller components should, I hope, lend well to unit testing.
When possible, GraphiteD.cfc attempts to send metrics over UDP inside of an asynchronous CFThread. It does, however, gracefully fallback to synchronous delivery as well as falling back to HTTP delivery if the message payload is too large.
Anyway, not too much more to say about it. It was a fun experiment and I'm definitely jacked up to learn more about how to record and then make sense of web metrics. I also see that HostedGraphite.com offers StatsD as a service (by request only); so, I'll see if I can check that out as well.
Reader Comments
Excellent! I'll take a look at your library when I get a sec. As mentioned on the UDP thread, I created a simple StatsD client for CF that we've been using in production for a couple years:
https://github.com/MWers/cfstatsd
It's worked quite well for us but it has a major downside: It doesn't buffer and aggregate metrics but instead spews them as UDP packets as soon as a metric event occurs.
@Matt,
We've actually been using CFStatsD as well :) Though, we've run into a problem locally with file-descriptors running out since our app "rebuilds" on every request locally. At some point, we hit a "too many open files" error and have to restart ColdFusion. I'm currently working on a small wrapper that doesn't hold open the socket... but not sure how well it will work :D
@All,
This is a follow-up project, to create a modular StatsD client for ColdFusion:
www.bennadel.com/blog/2849-statsdgateway-cfc-a-coldfusion-module-for-communicating-with-statsd-servers.htm