Instrumenting ColdFusion And Lucee Code With New Relic's Java Agent
As I mentioned two weeks ago, I am currently in the process of migrating the explicit instrumentation of my ColdFusion JVMs using Datadog and DogStatsD over to the implicit Agent-based instrumentation of my JVMs using New Relic's Java Agent. The Java Agent covers a lot of ground right out of the box when it comes to Database transactions and overall request processing. But, it doesn't really know how to introspect the ColdFusion application internals. As such, the New Relic Agent still requires some explicit instrumentation on behalf of the developer. And, to do that, I've been cultivating a ColdFusion proxy component that makes interacting with the Java Agent a bit easier.
Ultimate credit for this ColdFusion component should go to Andrew Dixon who published his Lucee New Relic Component repo on GitHub 4-years ago. I basically just took his core concept and extended it to fit my own needs.
The first method that I created gave me the ability to name the Transactions running through my ColdFusion application:
setTransationName( name [, category ] )
Out of the box, it looked like New Relic was taking care of this automatically. It appeared to be taking the cgi.script_name and cgi.path_info values and using them to identify and group Transactions. However, upon further investigation, I noticed that a lot of requests weren't showing up at all in my Transaction list. New Relic was also grouping requests in a way that obfuscated some of the request distinction.
So, I decided to add code at the beginning of all of my request processing that explicitly names the Transaction. At work, we use Framework One (FW/1); so, our ColdFusion application is organized by "subsystems", "sections", and "items". I use these components, along with the HTTP Method to identify each request:
// Setup the New Relic transaction breakdown.
javaAgentHelper.setTransationName( "#cgi.request_method#/#fw.getSubsystemSectionAndItem()#" );
javaAgentHelper.addCustomParameter( "resourceUri", cgi.path_info );
Using the fw.getSubsystemSectionAndItem() method, we end up producing Transactions with names like:
/GET/d:login.default/GET/common:main.default/POST/api:pusher.authenticate
... which map directly back the code-organization of the ColdFusion application.
As you can see, part of my Transaction identification also adds a custom parameter. My New Relic ColdFusion component has two methods for adding custom parameters:
addCustomParameter( name, value )addCustomParameterAsNumber( name, value )
These both add a key-value pair to the Transaction, which can be queried later on in the NRQL (New Relic Query Language); but, the latter adds the value as a double whereas the former adds the value as a string. This is an important distinction because the NRQL engine won't cast "numeric strings" to numbers. As such, if you want to use aggregation in your NRQL, such as sum(*), you have to make sure the value being aggregated is stored as a number.
For example, when recording the payment processing Webhook in my ColdFusion application, I include the user ID as a string and the dollar amount as a number:
javaAgentHelper.addCustomParameter( "user.id", user.id );
javaAgentHelper.addCustomParameter( "paymentProcessing.webhook", "success" );
javaAgentHelper.addCustomParameterAsNumber( "paymentProcessing.webhookAmount", amountInDollars );
Then, with the dollar-amount as a number, I can aggregate it as a sum() in a NRQL query:
SELECT
sum( paymentProcessing.webhookAmount ) AS 'Dolla-Dolla Bills!'
FROM
Transaction
WHERE
appName = 'my_coldfusion_app_name'
AND
paymentProcessing.webhook = 'success'
ASIDE: A cool feature of the New Relic transaction parameters is that they can be stored as structured objects. So, when I add a parameter called
user.id, that stores it as anidproperty on auserobject. This provides a nice way to group related parameters and avoid name-collisions across your ColdFusion application.
In addition to identifying Transactions and associating custom parameters, I also want to be able to time various portions of the ColdFusion application. For this, I created several Segment-related methods:
timeSegment( name, callback )startSegment( name )endSegment( segment )
The timeSegment() method uses the startSegment() and endSegment() methods under the hood, wrapping them around the invocation of the given callback. It just makes the management of the Segment a bit easier:
javaAgentHelper.timeSegment(
"security.authenticate.getUser",
function() {
rc.user = userService.authenticateCurrentSession();
}
);
This code executes the given callback synchronously and then adds a security.authenticate.getUser segment to the Trace details of the request.
Now, some requests are important from a ColdFusion application standpoint; but, are known to be really long running. Things like generating Zip files or running scheduled tasks take a long time and negatively impact some of the New Relic graphs and Apdex scores. As such, I added methods that allow me to side-step the instrumentation of long-running requests:
ignoreApdex()ignoreTransaction()
To be honest, there is probably a much better way to handle this. Telling the New Relic agent to ignore a request means that we no longer see it in the New Relic dashboard at all. This obviously has drawbacks when it comes to monitoring the application. But, it's the best way I have - so far - of curating dashboards that more directly reflect the user-facing experience.
And, of course, there's a method for monitoring errors:
noticeError( error [, errorParams [, expected ] ] )
I've only just begun to experiment with this method. The New Relic Java agent requires a "Throwable" object under the hood. And, I'm having trouble accessing the "Throwable" from a given ColdFusion error object. As such, I'm explicitly creating a java.lang.Throwable behind the scenes and then populating its Stacktrace using the TagContext property of the ColdFusion error. It's not a perfect mapping; but, it might be enough to facilitate debugging.
With that said, here's the code for my JavaAgentHelper ColdFusion component that proxies the underlying New Relic Java Agent. You'll notice that each method has an if statement internally that quietly becomes a no-op (No Operation) if thew New Relic Agent isn't installed in the current environment.
NOTE: I think, at first, I didn't realize that you could have the New Relic agent installed but not "activated". As such, we don't have it configured on the JVM in all environments (such as Developer environments). In retrospect, we should have installed it in all environments and then only "activated" it in Production. This would keep the code-execution paths uniform in all environments (which is the ideal configuration).
component
output = false
hint = "I help interoperate with the Java Agent that is instrumenting the ColdFusion application (which is provided by New Relic)."
{
// I initialize the java agent helper.
public any function init() {
// The New Relic Agent is not available in all contexts. As such, we have to be
// careful about trying to load the Java Class; and then, be cautious of its
// existence when we try to consume it. The TYPE OF THIS VARIABLE will be used
// when determining whether or not the New Relic API should be consumed. This
// approach allows us to use the same code in the calling context without having
// to worry if the NewRelic agent is installed.
try {
variables.NewRelicClass = createObject( "java", "com.newrelic.api.agent.NewRelic" );
} catch ( any error ) {
variables.NewRelicClass = "";
}
}
// ---
// PUBLIC METHODS.
// ---
/**
* I associate the given custom property with the current request transaction.
*
* @name I am the name of the custom property.
* @value I am the value of the custom property.
* @output false
*/
public void function addCustomParameter(
required string name,
required string value
) {
if ( shouldUseNewRelicApi() ) {
NewRelicClass.addCustomParameter(
javaCast( "string", name ),
javaCast( "string", value )
);
}
}
/**
* I associate the given custom property as a DOUBLE with the current request
* transaction.
*
* NOTE: Numeric parameters can only be consumed AS NUMBERS in the NRQL queries if
* they have been recorded as actual numbers (unlike ColdFusion, which can seamlessly
* cast between strings and numbers). So, for example, you can't use "sum(*)" in NRQL
* on numbers that have been added using the sibling method, "addCustomParameter()".
*
* @name I am the name of the custom property.
* @value I am the value of the custom property.
* @output false
*/
public void function addCustomParameterAsNumber(
required string name,
required numeric value
) {
if ( shouldUseNewRelicApi() ) {
NewRelicClass.addCustomParameter(
javaCast( "string", name ),
javaCast( "double", value )
);
}
}
/**
* I end the segment and associate the resultant timing with the current request
* transaction.
*
* NOTE: If you fail to end a segment, it will eventually timeout based on the
* "segment_timeout" property, which has a default value of 600-seconds.
*
* @segment I am the segment being ended and timed.
* @output false
*/
public void function endSegment( required any segment ) {
// In the case where the segment is not available (because the NewRelic agent
// has not been installed), it will be represented as an empty string. In such
// cases, just ignore the request.
if ( isSimpleValue( segment ) ) {
return;
}
if ( shouldUseNewRelicApi() ) {
segment.end();
}
}
/**
* I include the current request in Transaction Tracing, but do not include it when
* calculating the Apdex score.
*
* @output false
*/
public void function ignoreApdex() {
if ( shouldUseNewRelicApi() ) {
NewRelicClass.ignoreApdex();
}
}
/**
* I exclude the current request from Transaction Tracing and Apdex calculations.
*
* @output false
*/
public void function ignoreTransaction() {
if ( shouldUseNewRelicApi() ) {
NewRelicClass.ignoreTransaction();
}
}
/**
* I report an error to NewRelic.
*
* CAUTION: Since the ColdFusion error object is not an inherently compatible type for
* the NewRelic agent, it is transformed into a Java Throwable under the hood so that
* the Stacktrace can be recorded. All properties on the ColdFusion error object are
* attached to the Transaction as custom properties (on the errorParams object) using
* the "cfError" prefix (ex, "error.ExtendedInfo" becomes "cfErrorExtendedInfo").
* These can then be queried using NRQL.
*
* NOTE: These errors MAY NOT SHOW UP in New Relic if they are being ignored based on
* the "newrelic.yml" file (see options, "ignore_errors" and "ignore_status_codes").
*
* @error I am the error object or message (string) being reported.
* @errorParams I am the key-value pairs to associate with the transaction.
* @expected I determine if the error was expected or not.
* @output false
*/
public void function noticeError(
required any error,
struct errorParams = {},
boolean expected = false
) {
if ( shouldUseNewRelicApi() ) {
if ( isSimpleValue( error ) ) {
NewRelicClass.noticeError(
javaCast( "string", error ),
errorParams,
javaCast( "boolean", expected )
);
} else {
NewRelicClass.noticeError(
createThrowable( error ),
augmentErrorParams( error, errorParams ),
javaCast( "boolean", expected )
);
}
}
}
/**
* I attempt to set the name of the current transaction (which is used to separate
* requests within the New Relic dashboard).
*
* @name I am the name of the transaction. It will be used as-is.
* @category I am the optional category of metrics to use (uses default by default).
* @output false
*/
public void function setTransationName(
required string name,
string category
) {
if ( shouldUseNewRelicApi() ) {
if ( structKeyExists( arguments, "category" ) ) {
NewRelicClass.setTransactionName(
javaCast( "string", category ),
javaCast( "string", name )
);
} else {
NewRelicClass.setTransactionName(
javaCast( "null", "" ),
javaCast( "string", name )
);
}
}
}
/**
* I start and return a new Segment to be associated with the current request
* transaction. The returned Segment should be considered an OPAQUE TOKEN and should
* not be consumed directly. Instead, it should be passed to the .endSegment() method.
* Segments will show up in the Transaction Breakdown table.
*
* NOTE: If you fail to end a segment, it will eventually timeout based on the
* "segment_timeout" property, which has a default value of 600-seconds.
*
* @name I am the name of the segment being started.
* @output false
*/
public any function startSegment( required string name ) {
if ( shouldUseNewRelicApi() ) {
return( NewRelicClass.getAgent().getTransaction().startSegment( javaCast( "string", name ) ) );
}
// If the New Relic API feature is not enabled, or it errors, we still need to
// return something as the OPAQUE SEGMENT TOKEN so that the calling logic can
// be handled uniformly.
return( "" );
}
/**
* I wrap a new Segment with the given name around the execution of the given
* callback. Segments will show up in the Transaction Breakdown table. I pass-through
* the return value of the callback invocation.
*
* @name I am the name of the segment being started.
* @callback I am the callback being executed and timed.
* @output false
*/
public any function timeSegment(
required string name,
required function callback
) {
var segmentToken = startSegment( name );
try {
return( callback() );
} finally {
endSegment( segmentToken );
}
}
// ---
// PRIVATE METHODS.
// ---
/**
* I augment the given errorParams object with properties plucked from the given error
* object. All keys are prefixed with "cfError" so as not to collide with any existing
* keys on the errorParams object.
*
* @error I am the error object being inspected.
* @errorParams I am the params object being augmented.
* @output false
*/
private struct function augmentErrorParams(
required any error,
required struct errorParams
) {
var errorKeys = [
// Standard error keys.
"Type",
"Message",
"Detail",
"ExtendedInfo",
"ErrorCode",
// Non-standard error keys.
"NativeErrorCode",
"SqlState",
"Sql",
"QueryError",
"Where",
"ErrNumber",
"MissingFileName",
"LockName",
"LockOperation"
];
for ( var key in errorKeys ) {
// Only include the property if it is a meaningful value.
if (
structKeyExists( error, key ) &&
isSimpleValue( error[ key ] ) &&
len( error[ key ] )
) {
errorParams[ "cfError#key#" ] = error[ key ];
}
}
return( errorParams );
}
/**
* I crate a native Java Throwable instance based on information in the given error
* object. The NewRelic agent needs a Throwable for proper logging.
*
* @error I am the ColdFusion error object being emulated.
* @output false
*/
private any function createThrowable( required any error ) {
// When constructing a Throwable object, we need to provide a Message. However,
// we may not have an actual message to provide, depending on how the root error
// was throw. In cases where no message is available, NewRelic shows the error
// as "java.lang.Throwable" in the Error Dashboard. To make this a bit more
// human-consumable, I'm going to use the "Type" as the "Message" in cases where
// no message is available. While technically inaccurate, this just makes the
// data more readable.
// If there is a "message" available, use that.
if (
structKeyExists( error, "message" ) &&
isSimpleValue( error.message ) &&
len( error.message )
) {
var message = error.message;
// If there is no "message", but there is a "type" available, use that.
} else if (
structKeyExists( error, "type" ) &&
isSimpleValue( error.type ) &&
len( error.type )
) {
var message = error.type;
} else {
var message = "Unexpected error";
}
var throwable = createObject( "java", "java.lang.Throwable" )
.init( javaCast( "string", message ) )
;
// If the TagContext is available on the error object, let's use it to generate
// a Stacktrace collection. This Stacktrace will be presented in NewRelic UI.
if (
structKeyExists( error, "tagContext" ) &&
isArray( error.tagContext )
) {
var stackTraceElements = [];
for ( var item in error.tagContext ) {
var normalizedItem = normalizeTagContextItem( item );
arrayAppend(
stackTraceElements,
createObject( "java", "java.lang.StackTraceElement" ).init(
javaCast( "string", normalizedItem.className ),
javaCast( "string", normalizedItem.methodName ),
javaCast( "string", normalizedItem.fileName ),
javaCast( "int", normalizedItem.lineNumber )
)
);
}
throwable.setStackTrace(
javaCast( "java.lang.StackTraceElement[]", stackTraceElements )
);
}
return( throwable );
}
/**
* I transform the given tagContext item into something that is predictable.
*
* @tagContextItem I am the tagContext item being transformed.
* @output false
*/
private struct function normalizeTagContextItem( required any tagContextItem ) {
var normalizedItem = {
className: "",
methodName: "",
fileName: "unknown",
lineNumber: -1
};
// The TagContext details are actually quite different between the Adobe and
// Lucee ColdFusion engines. As such, we're just going to concentrate on plucking
// out the right template and line-number. These are sufficient for debugging.
if (
structKeyExists( tagContextItem, "template" ) &&
isSimpleValue( tagContextItem.template )
) {
normalizedItem.fileName = tagContextItem.template;
}
if (
structKeyExists( tagContextItem, "line" ) &&
isNumeric( tagContextItem.line )
) {
normalizedItem.lineNumber = tagContextItem.line;
}
return( normalizedItem );
}
/**
* I check to see if this machine should consume the New Relic static API as part of
* the Java Agent Helper class (this is to allow the methods to exist in the calling
* context without a lot of conditional consumption logic).
*
* @output false
*/
private boolean function shouldUseNewRelicApi() {
// If we were UNABLE TO LOAD THE NEWRELIC CLASS, there's no API to consume.
if ( isSimpleValue( NewRelicClass ) ) {
return( false );
}
return( true );
}
}
Explicitly instrumenting my ColdFusion / Lucee application with the New Relic Java agent is still an ongoing experiment (and learning experience) for me. As such, I am sure this component will be augmented over time. I'm still trying to figure out how to best express all of the application information and metrics in New Relic, whether it be as Transaction parameters or custom events (the latter of which I haven't even tried yet). That said, I hope this has been helpful to others that may be looking to add New Relic to their ColdFusion applications.
Want to use code from this post? Check out the license.
Reader Comments
Ben, I have been wanting to look into this for so long! Thanks for doing it! We use New Relic for our Scala Play apps and I wanted to see how it would work with CF but haven't had the time.
@Tim,
My pleasure, good sir. I'm just starting to get more into the nitty-gritty of New Relic. We've had it for years, and I've literally only used it to look at the average response times (since we have historically done most of our metrics in Datadog). But, now that we're migrating from Datadog to New Relic across the whole org, I need to start to figure out how to use this beast.
@All,
It would probably be helpful if I actually linked to some of the Java Agent Docs :D
Java Agent Overview: https://docs.newrelic.com/docs/agents/java-agent/api-guides/guide-using-java-agent-api
Java Docs on GitHub: http://newrelic.github.io/java-agent-api/javadoc/index.html?com/newrelic/api/agent/NewRelic.html
@All,
On a related note, I've recently started to track my feature flag state as custom parameters on my New Relic request transactions:
www.bennadel.com/blog/3731-tracking-feature-flags-in-new-relic-and-nrql-using-the-java-agent-in-lucee-cfml-5-3-3-62.htm
This allows me to pick apart my request performance on a granular level using the New Relic Insights NRQL (New Relic Query Language).