Skip to main content
Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.

Annotating An Image Point Using GraphicsMagick And Lucee CFML 5.2.9.31

By Ben Nadel on
Tags: ColdFusion

Continuing my exploration of GraphicsMagick in Lucee CFML, this morning, I wanted to see how I might go about annotating an image. That is, labeling a point on the image with a marker and an annotation index. I suspect that this can be achieved in several ways; however, the first approach that jumps to mind involves drawing Circle and Text primitives using GraphicsMagick and Lucee CFML 5.2.9.31.

CAUTION: I am very much a novice when it comes to GraphicsMagick. As such, please take the following as an exploration, not an explanation.

In a previous demo, I looked at how to get the RGB value of a given pixel using GraphicsMagick. In that exploration, the user would click on the image in the browser; and then, I would send the {X,Y} coordinates of the click to the ColdFusion server, which would extract the RGB color value at the given location of the target image. For this demo, I want to use a similar approach; only, instead of reading a pixel, I'm going to annotate the image at the given offset.

And, to make things a bit more interesting, I want the user to be able to apply more than one annotation to the image. To do this, I'm going to use Lucee's special URL syntax that allows a collection of values to be spread over multiple query-string parameters:

?points[]=x,y&points[]=x,y&points[]=x,y

By suffixing the points query-string entry with array notation, [], it indicates to the ColdFusion server that all of the points[] query-string parameters should be collected into a single Array within the url scope. As such, making a request to the URL:

?points[]=1,1&points[]=99,99

... results in the following url.points value on the server:

[ "1,1", "99,99" ]

To leverage this behavior, every time the user clicks on the image within the browser, I'm just going to refresh the page with an additional points[] value. Then, on the server, I'm going to start with a base image and apply an individual annotation for each of the collected point coordinates.

Each annotation will be implemented using two -draw options: one for the circle and one for the label. Example:

  • -draw 'circle 50,50 50,60'
  • -draw 'text 50,50 "Hello"'

Here's what I came up with:

<cfscript>

	param name="url.points" type="array" default=[];

	startedAt = getTickCount();

	inputFilename = "demo.jpg";
	inputFilepath = expandPath( "./#inputFilename#" );

	// Copy the fresh image into the current directory - we always start with the fresh
	// image even if we have multiple annotations to add. This way, we're not constantly
	// adding additional compression artifacts to intermediary images.
	fileCopy( "../images/face-square.jpg", "./demo.jpg" );

	// If the user has provided a collection of points, annotate the image.
	if ( url.points.len() ) {

		// All of the image utilities are provided through the GraphicsMagick binary.
		command = "gm";

		// We're going to use the Convert utility to draw primitive shapes.
		utility = "convert";

		// We want to be EXPLICIT about which input reader GraphicsMagick should use.
		// If we leave it up to "automatic detection", a malicious actor could fake
		// file-type and potentially exploit a weakness in a given reader.
		// --
		// READ MORE: http://www.graphicsmagick.org/security.html
		utilityReader = "jpg";

		// Setup the initial options for the Convert utility.
		commandOptions = [
			utility,

			// Provide the source image to be read with the an explicit reader.
			( utilityReader & ":" & inputFilepath ),
		];

		// Add each point annotation to the Convert command.
		// --
		// NOTE: Some of configurations in the following Draw commands could be factored
		// out to reduce repetition. However, in order to make the code a bit more
		// simple, I'm just repeating configurations as needed such that all of the code
		// is collocated.
		url.points.each(
			( point, i ) => {

				var x = point.listGetAt( 1 );
				var y = point.listGetAt( 2 );
				var radius = 19;

				commandOptions.append( "-fill '##ff3366'" );
				commandOptions.append( "-stroke '##ffffff'" );
				commandOptions.append( "-strokewidth 2.5" );
				commandOptions.append( "-draw 'circle #x#,#y# #( x + radius )#,#y#'" );

				// We have to manually center the text within the circle.
				var textX = ( x - 4 );
				var textY = ( y + 6 );

				// If the stringified annotation index has more than one digit, nudge the
				// text a little further to the left to try and keep it centered within
				// the circle.
				if ( len( i ) > 1 ) {

					textX -= 5;

				}

				commandOptions.append( "-font 'Helvetica-Bold'" );
				commandOptions.append( "-pointsize 17" );
				commandOptions.append( "-fill '##ffffff'" );
				commandOptions.append( "-stroke 'transparent'" );
				commandOptions.append( "-draw 'text #textX#,#textY# ""#i#""'" );

			}
		);

		commandOptions.append( "-quality 90" );
		commandOptions.append( inputFilepath );

		// Execute GraphicsMagick on the command-line.
		execute
			name = command
			arguments = commandOptions.toList( " " )
			variable = "successResult"
			errorVariable = "errorResult"
			timeout = 5
		;

		// If the error variable has been populated, it means the CFExecute tag ran into
		// an error - let's dump-it-out and halt processing.
		if ( variables.keyExists( "errorVariable" ) && errorVariable.len() ) {

			dump( errorVariable );
			abort;

		}

	}

	duration = ( getTickCount() - startedAt );

</cfscript>
<cfoutput>

	<p>
		<!--
			This is our annotated image - as the user clicks on it, we will reload the
			page to add additional annotations.
		-->
		<img
			id="image"
			src="./#inputFilename#"
			width="600"
			height="584"
			style="cursor: pointer ;"
		/>
	</p>

	<p>
		<a href="./index.cfm">Reset image</a>
	</p>

	<p>
		Duration: #numberFormat( duration )# ms
	</p>

	<script type="text/javascript">

		// When the user clicks anywhere on the demo image, we want to translate the
		// viewport coordinates to image-local coordinates and then refresh the page
		// with the additional offset.
		document.getElementById( "image" ).addEventListener(
			"click",
			( event ) => {

				// Get viewport coordinate data.
				var viewportX = event.clientX;
				var viewportY = event.clientY;
				var imageRect = event.target.getBoundingClientRect();

				// Translate viewport coordinates to image-local coordinates.
				var localX = ( viewportX - imageRect.left );
				var localY = ( viewportY - imageRect.top );

				// We're going to be passing each point as an additional entry in the
				// points collection on the query-string.
				var nextHref = ( window.location.search )
					? ( window.location.href + "&" )
					: ( window.location.href + "?" )
				;

				// NOTE: Using special array-notation for a single query-string parameter
				// that is spread across multiple URL entries.
				nextHref += `points[]=${ Math.round( localX ) },${ Math.round( localY ) }`;

				window.location.href = nextHref;

			}
		);

	</script>

</cfoutput>

In order to keep the demo as simple as possible, I'm repeating all of the configuration options in each iteration of my points.each() loop. In reality, I could have factored-out some of the options, like -font and -pointsize; however, for the sake of readability, I liked having all of the configuration objects collocated within the code.

As you can see from the annotation logic, I'm manually centering the text within the annotation circle. This is unfortunate; however, I believe it's the only way to do this with my selected approach. In a subsequent exploration, I might try writing each annotation to an intermediary image using the -gravity center option; and then, overlay that annotation image onto the base image. But, I haven't actually tried that yet - one exploration at a time.

That said, if we run this Lucee CFML code and click around on the image, we get the following output:

An image of Lucy being annotated with GraphicsMagick and Lucee CFML 5.2.9.31

Getting the right configurations for the various -draw options required a lot of trial-and-error. But, in the end, I think I came up with a workable solution. I think there are other ways to annotate images using GraphicsMagick. But, this it the first approach I wanted to try.



Reader Comments

@All,

I should add that in order to find a usable font, Helvetica-Bold, I had to use the following command:

gm convert -list font

This gave me the following output:

root@ed0ea7ed1423:/app# gm convert -list font
Path: /usr/lib/GraphicsMagick-1.3.28/config/type-ghostscript.mgk

Name                             Family                  Style   Stretch  Weight
--------------------------------------------------------------------------------
AvantGarde-Book                  AvantGarde              normal  normal    400
AvantGarde-BookOblique           AvantGarde              oblique normal    400
AvantGarde-Demi                  AvantGarde              normal  normal    600
AvantGarde-DemiOblique           AvantGarde              oblique normal    600
Bookman-Demi                     Bookman                 normal  normal    600
Bookman-DemiItalic               Bookman                 italic  normal    600
Bookman-Light                    Bookman                 normal  normal    300
Bookman-LightItalic              Bookman                 italic  normal    300
Courier                          Courier                 normal  normal    400
Courier-Bold                     Courier                 normal  normal    700
Courier-Oblique                  Courier                 oblique normal    400
Courier-BoldOblique              Courier                 oblique normal    700
Helvetica                        Helvetica               normal  normal    400
Helvetica-Bold                   Helvetica               normal  normal    700
Helvetica-Oblique                Helvetica               italic  normal    400
Helvetica-BoldOblique            Helvetica               italic  normal    700
Helvetica-Narrow                 Helvetica Narrow        normal  condensed 400
Helvetica-Narrow-Oblique         Helvetica Narrow        oblique condensed 400
Helvetica-Narrow-Bold            Helvetica Narrow        normal  condensed 700
Helvetica-Narrow-BoldOblique     Helvetica Narrow        oblique condensed 700
NewCenturySchlbk-Roman           NewCenturySchlbk        normal  normal    400
NewCenturySchlbk-Italic          NewCenturySchlbk        italic  normal    400
NewCenturySchlbk-Bold            NewCenturySchlbk        normal  normal    700
NewCenturySchlbk-BoldItalic      NewCenturySchlbk        italic  normal    700
Palatino-Roman                   Palatino                normal  normal    400
Palatino-Italic                  Palatino                italic  normal    400
Palatino-Bold                    Palatino                normal  normal    700
Palatino-BoldItalic              Palatino                italic  normal    700
Times-Roman                      Times                   normal  normal    400
Times-Bold                       Times                   normal  normal    700
Times-Italic                     Times                   italic  normal    400
Times-BoldItalic                 Times                   italic  normal    700
Symbol                           Symbol                  normal  normal    400

I assume it's possible to install other fonts. I don't know how to do that; so, I just tried a few of the fonts in the given list until I found one that worked well-enough.

Reply to this Comment

where/how do you find these hidden (to me) gems such as the special "points[]" notation? Very cool.

Also, my instinct would have been to create a new image with the same dimensions as the original (transparent png more than likely) and add all the points to that, then absolutely position it over the top of the original. This way, you could easily toggle the points on/off.

Reply to this Comment

@Chris,

Ha ha, they hid it away in the "hidden gems" portion of the docs:

https://docs.lucee.org/guides/cookbooks/Hidden_gems.html

I think the secondary image makes sense. Tomorrow morning, I'm going to try writing the markers themselves to tiny individual files, and then gluing them altogether. Similar idea, but ultimately I end up with a single file. It will be interesting cause I'll need the markers to be in PNG (so they have alpha-transparency).

Reply to this Comment

@All,

As I think I mentioned somewhere in here, I had planned to do a follow-up exploration where I generate the annotations as separate files in an attempt to use the "gravity" to center the text better. This ended up having two issues:

One -- The annotation images were going to have a transparent canvas. But, it seems that drawing primitives on a transparent canvas causes issues wherein the partially-transparent pixels end-up generating a "halo" effect. This is touched on here:

There is some discussion on getting around this; but, most of the technical details around alpha-channels and matte options went way over my head.

Two -- it turns out that using "gravity center" really didn't center the text all that much better than me just eye-balling it and adjusting based on the number of digits in the annotation.

Three -- it's way slower creating intermediary images.

Altogether, I'm going to abandon that thought-experiment. It didn't seem like it was going to be a value-add in the long run.

Reply to this Comment

Post A Comment

You — Get Out Of My Dreams, Get Into My Blog
Live in the Now
Oops!
NEW: Some basic markdown formatting is now supported: bold, italic, blockquotes, lists, fenced code-blocks. Read more about markdown syntax »
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.