Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
Ben Nadel at CFUNITED 2009 (Lansdowne, VA) with: Aaron West
Ben Nadel at CFUNITED 2009 (Lansdowne, VA) with: Aaron West@awest )

ColdFusion Client For The Word-Finding Datamuse API

By Ben Nadel on
Tags: ColdFusion

As a side-project to learn more about Docker and Redis, I've been working on an application that will help me write poetry. A big part of writing poetry is understanding rhyme schemes, syllable counts, and words that have similar meanings. My application hopes to bring all of these tools together in one nice, neat user interface (UI). And, it will be doing so with the help of the Datamuse API. The Datamuse API is a super flexible word-finding API that allows you to search for words based on sounds, meanings, rhymes, synonyms, consonants, lengths, prefixes, suffixes, and a host of other constraints. In order to leverage this API in my application, I've created a thin ColdFusion wrapper about the Datamuse HTTP schema.

View my ColdFusion Datamuse project on GitHub.

At first, I wanted to build a ColdFusion library that added some semantic abstractions over the Datamuse API. But, when I started to dig into the API, I realized that it was so flexible that any attempt at add a semantic abstraction would be a limitation rather than an facilitation. As such, I ended up writing a ColdFusion component that does little more than make the HTTP request a bit easier to execute.

At this time, my DatamuseClient.cfc provides two methods, each of which correspond to an HTTP end-point in the Datamuse API. The arguments for these methods mirror the arguments defined in the Datamuse API - again, I'm not trying to create a new abstraction, I'm just trying to make the API easier to consume. And, to be honest, I'm not even sure what all the arguments really mean.

getWords( [all arguments optional] ) - Returns words that match constraints.

  • ml - Means like term.
  • sl - Sounds like term.
  • sp - Spelled like term.
  • rel_jja
  • rel_jjb
  • rel_syn - Synonyms for term.
  • rel_trg
  • rel_ant - Antonyms for term.
  • rel_spc
  • rel_gen
  • rel_com
  • rel_par
  • rel_bga
  • rel_bgb
  • rel_rhy - Perfect rhymes for term.
  • rel_nry - Near rhymes for term.
  • rel_hom
  • rel_cns
  • v
  • topics
  • lc
  • rc
  • max - Max number of results (defaults to 10).
  • md - Metadata for term.
  • qe - Query echo for term.

getSuggestions( s [,v [, max]] ) - Returns words that are suggestions for a prefix.

  • s - The term being searched.
  • v
  • max - Max number of results (defaults to 10).

To see the DatamuseClient.cfc in action, I've put together a few examples:

  • <cfscript>
  • // Get words that start with the string "st" and are 5 letters long.
  • words = new lib.DatamuseClient().getWords(
  • sp = "st???"
  • );
  •  
  • // Returns:
  • // state, stick, stand, stock, style, stone, store, study, steel, story
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get words that start with "s" and end with "y".
  • words = new lib.DatamuseClient().getWords(
  • sp = "s*y"
  • );
  •  
  • // Returns:
  • // strategy, say, study, serendipity, story, security, society, savvy, synergy, survey
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get words that start with "s" and rhyme with "bad".
  • words = new lib.DatamuseClient().getWords(
  • rel_rhy = "bad",
  • sp = "s*"
  • );
  •  
  • // Returns:
  • // sad, scad, shad, strad, sociedad, sinbad, soledad, stalingrad, scantily clad, sketch pad
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get words that mean something similar to "awesome".
  • words = new lib.DatamuseClient().getWords(
  • ml = "awesome"
  • );
  •  
  • // Returns:
  • // amazing, awful, impressive, awing, awe-inspiring, unbelievable, fantastic, incredible, wonderful, terrific
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get words that are synonyms to "insane" and are 4-letters long.
  • words = new lib.DatamuseClient().getWords(
  • rel_syn = "insane",
  • sp = "????"
  • );
  •  
  • // Returns:
  • // wild, sick, daft, nuts, amok, loco, bats
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get the number of syllables in "countenance".
  • // --
  • // NOTE: There is no "end point" for syllable count. So, in order to get it, we're
  • // going to use the Query Echo parameter, which will echo the value in the given
  • // parameter (sp) as the first item in the result. We can then use the metadata
  • // parameter (md) to tell Datamuse to return the (s) syllable count in the result.
  • words = new lib.DatamuseClient().getWords(
  • sp = "countenance",
  • qe = "sp",
  • md = "s",
  • max = 1
  • );
  •  
  • // Returns:
  • // countenance with syllable count metadata, 3.
  •  
  • // ------------------------------------------------------------------------------- //
  • // ------------------------------------------------------------------------------- //
  •  
  • // Get word suggesions for "bla".
  • words = new lib.DatamuseClient().getSuggestions(
  • s = "bla"
  • );
  •  
  • // Returns:
  • // black, blancard, blanford, blanket, blast, blazen, blatant, blade, blaze, blank
  •  
  • </cfscript>

In the above commenting, I've only listed the actual words that get returned; but, someone of the actions return additional metadata, like syllable count. Plus, you can include the "md" argument to return explicit metadata like syllable count, pronunciation, and definitions.

The Datamuse API is free for up to 100,000 requests a day. If you plan to use more than that, you can request an API Key for your application. I've requested an API Key, but have not yet heard back. As such, my DatamuseClient.cfc accepts an API Key value, but does not apply it to the underlying HTTP request. Once I hear back from Datamuse - and can see how to apply the API Key - I will up date the ColdFusion code. Unfortunately, this approach is not documented.

Here's the code for my initial release of the ColdFusion client:

  • component
  • output = false
  • hint = "I provide a low-level HTTP client for the Datamuse API."
  • {
  •  
  • /**
  • * I initialize the Datamuse API client with the given settings.
  • *
  • * @apiKey I am the API key used to authenticate the HTTP requests.
  • * @timeout I am the timeout (in seconds) used to manage the HTTP requests.
  • * @output false
  • */
  • public any function init(
  • string apiKey = "",
  • numeric timeout = 5
  • ) {
  •  
  • setApiKey( apiKey );
  • setTimeout( timeout );
  •  
  • }
  •  
  • // ---
  • // PUBLIC METHODS.
  • // ---
  •  
  • /**
  • * I make a request to the /sug end-point in Datamuse.
  • *
  • * For documentation on parameters, see: https://www.datamuse.com/api/
  • * @output false
  • */
  • public array function getSuggestions(
  • required string s,
  • string v = "",
  • numeric max = 10
  • ) {
  •  
  • return( makeRequest( "sug", filterOutEmptyValues( arguments ) ) );
  •  
  • }
  •  
  •  
  • /**
  • * I make a request to the /words end-point in Datamuse.
  • *
  • * For documentation on parameters, see: https://www.datamuse.com/api/
  • * @output false
  • */
  • public array function getWords(
  • string ml = "",
  • string sl = "",
  • string sp = "",
  • string rel_jja = "",
  • string rel_jjb = "",
  • string rel_syn = "",
  • string rel_trg = "",
  • string rel_ant = "",
  • string rel_spc = "",
  • string rel_gen = "",
  • string rel_com = "",
  • string rel_par = "",
  • string rel_bga = "",
  • string rel_bgb = "",
  • string rel_rhy = "",
  • string rel_nry = "",
  • string rel_hom = "",
  • string rel_cns = "",
  • string v = "",
  • string topics = "",
  • string lc = "",
  • string rc = "",
  • numeric max = 10,
  • string md = "",
  • string qe = ""
  • ) {
  •  
  • return( makeRequest( "words", filterOutEmptyValues( arguments ) ) );
  •  
  • }
  •  
  •  
  • /**
  • * I set the Datamuse API key to be sent with each request.
  • *
  • * CAUTION: At this time, the API key does not get applied (I am waiting on a response
  • * from Datamuse on how to actually implement the API key).
  • *
  • * @newApiKey I am the Datamuse API key.
  • * @output false
  • */
  • public void function setApiKey( required string newApiKey ) {
  •  
  • apiKey = newApiKey;
  •  
  • }
  •  
  •  
  • /**
  • * I set the timeout (in seconds) for the HTTP request.
  • *
  • * @newTimeout I am the timeout (in seconds) to be applied to the HTTP request.
  • * @output false
  • */
  • public void function setTimeout( required numeric newTimeout ) {
  •  
  • timeout = newTimeout;
  •  
  • }
  •  
  • // ---
  • // PRIVATE METHODS.
  • // ---
  •  
  • /**
  • * I return a collection that contains only the defined, non-empty, simple values
  • * contained within the given data structure.
  • *
  • * @data I am the collection being filtered.
  • * @output false
  • */
  • private struct function filterOutEmptyValues( required struct data ) {
  •  
  • var filtered = {};
  •  
  • for ( var key in data ) {
  •  
  • // Skip if null.
  • if ( ! structKeyExists( data, key ) ) {
  •  
  • continue;
  •  
  • }
  •  
  • var value = data[ key ];
  •  
  • // Skip if complex or empty.
  • if ( ! isSimpleValue( value ) || ! len( value ) ) {
  •  
  • continue;
  •  
  • }
  •  
  • filtered[ key ] = value;
  •  
  • }
  •  
  • return( filtered );
  •  
  • }
  •  
  •  
  • /**
  • * I make the HTTP request to the Datamuse API.
  • *
  • * @resource I am the end-point being accessed.
  • * @urlParams I am the collection of key-value pairs to add to the search-string.
  • * @output false
  • */
  • private array function makeRequest(
  • required string resource,
  • required struct urlParams
  • ) {
  •  
  • var apiRequest = new Http(
  • method = "GET",
  • url = "https://api.datamuse.com/#resource#",
  • getAsBinary = "yes",
  • timeout = timeout
  • );
  •  
  • for ( var key in urlParams ) {
  •  
  • apiRequest.addParam(
  • type = "url",
  • name = lcase( key ),
  • value = urlParams[ key ]
  • );
  •  
  • }
  •  
  • var apiResponse = apiRequest.send().getPrefix();
  •  
  • // NOTE: Even though we are using "getAsBinary" in the HTTP request, the
  • // fileContent will only be binary if the request is successful. If, for example,
  • // the request fails to connect, the fileContent may contain a plain string with
  • // something like, "Connection Failure," in it.
  • var fileContent = isSimpleValue( apiResponse.fileContent )
  • ? apiResponse.fileContent
  • : charsetEncode( apiResponse.fileContent, "utf-8" )
  • ;
  •  
  • if ( ! reFind( "200", apiResponse.statusCode ) ) {
  •  
  • throw(
  • type = "Datamuse.HttpError",
  • message = "HTTP request returned with a non-200 status code.",
  • detail = "HTTP Status Code: #apiResponse.statusCode#"
  • extendedInfo = "HTTP File Content: #fileContent#"
  • );
  •  
  • }
  •  
  • try {
  •  
  • return( deserializeJson( fileContent ) );
  •  
  • } catch ( any error ) {
  •  
  • throw(
  • type = "Datamuse.ContentError",
  • message = "Content could not be deserialized.",
  • detail = "HTTP File Content: #fileContent#",
  • extendedInfo = "Root Error Message: #error.message#"
  • );
  •  
  • }
  •  
  • }
  •  
  • }

As you can see, it does little more than make the HTTP request a bit easier to construct and consume. All semantic abstractions probably make more sense in the calling context, not in the client itself.

The Datamuse API is flexible, powerful, and seems to be blazing fast. I'm looking forward to consuming it in my poetry application!



Looking For A New Job?

Ooops, there are no jobs. Post one now for only $29 and own this real estate!

100% of job board revenue is donated to Kiva. Loans that change livesFind out more »

Reader Comments

@Chris,

I'll be using it to provide information in the context of a poetry app:

  • Identify number of syllables per line of the poem.
  • Identify words that rhyme (for example, with trailing word).
  • Identify words that mean similar things.
Reply to this Comment

Need to get your awesome API on ForgeBox to make it easier for others to install, update, and use. Would you like me to send you a pull to add the box.json to your repo? Then all you'll need to do is:

forgebox login
bump --major

:)

Reply to this Comment

This is really great. Just out of interest, in CFScript, is it necessary to prefix variables with the 'variables' scope. I rarely use CFSCRIPT, but I am about to start a new job and the client only uses CFSCRIPT.

Reply to this Comment

And one other thing:

new lib.DatamuseClient()

Is this a new way of connecting to a CFC?
I can see what you are doing, but I haven't seen this syntax before?

Reply to this Comment

Charles, all the same variable scoping rules apply to cfscript the same as tags. Really no difference there.

The new keyword is the same as doing
createObject( 'component', 'lib.DatamuseClient' ).init()

Reply to this Comment

No, the "new" keyword is nothing more than a shortcut for creating CFC instances. There is a generic way to call any tag in script, but that's a different thing entirely.

Reply to this Comment

OK. I am with you. It's just that I noticed this pattern with TAGs that have a children tags, but I didn't realise they were CFCs themselves. Interesting.

Reply to this Comment

@Brad,

Yeah, I really gotta get on ForgeBox. I've looked through it, and I've read most of the documentation on the Box CLI; I just haven't taken that next step. But, it is something I want to be better about doing. There's other stuff that I think could go in there as well.

Reply to this Comment

@Charles, @Brad,

Well, you are somewhat right in that recent versions (since 9??) do have ColdFusion components for several of the built-in tags. Like Mail.cfc, Http.cfc, and Query.cfc. But, as Brad was saying, there is now syntax that allows you to invoke any tag in CFScript.

As far as the variables scope, it is the "implied" scope. So, most of the time, you can assume that these two statements are the same:

  • variables.foo = "bar";
  • foo = "bar";

In the latter case, variables. is implied.

That said, there are huge caveats to that statement. For example, if you do that inside a CFThread, it will save to the thread-local scope. And, if the variable is already assigned to a scope (such as with var foo, then it will state in that scope. And, I believe in Lucee, you can actually configure the behavior of an unscoped assignment inside a Function body.

But, allllll those caveats aside, the variables. scope is the "page scope".

Reply to this Comment

Thanks for that. I actually understand all of the stuff about the 'variables' scope, I just wondered whether these same rules applied in CFSCRIPT. So, thank you for giving me a definitive 'yes'. My understanding of CFSCRIPT is very limited, but I have been reading Adam Cameron's 'github' repo on the subject and it looks fairly straightforward:

https://github.com/tonyjunkes/cfml-tags-to-cfscript/blob/master/README.md

And also Brad has given me some excellent reading material...

I also see that in CF11, it is possible to use the following pattern:

cfparenttag(attributes1=,attributes2=){
   cfchildtag(attributes1=,attributes2=){
   }
}
Reply to this Comment

And also your explanation of scope helped to validate my current understanding. The CFTHREAD stuff was enlightening. I usually just use 'attributes' & 'caller', but I have always wondered how to create a thread local scope. So thanks again.

Reply to this Comment

@Charles,

CFThread is a funny beast. Technically, under the hood, it's actually just a Function that gets called asynchronously. So, like a Function, it has an arguments scope, which is not really documented. And, like a Function, it also as a local scope as well. Which is why you can var variables inside of a CFThread body and they are local to the thread. And then, it also has the thread scope which is bridge between the thread and the parent context (ie, if you want to pass values "back" from the thread to the parent context).

The biggest hurdle with CFThread is that if you want to set a variables-scoped value from inside a thread, you have to explicitly provide the variables. prefix; otherwise, the value will get saved into the thread-local scope.

function myFunction() {
    foo = "hello"; // Variables scope.

    thread ... {
        // CAUTION - this will get saved to the THREAD-local scope.
        foo = "bar";

        // To set to variables INSIDE a thread, you need to explicitly provide scope.
        variables.foo = "bar";
    }

}

Other than that, threads basically "just work" :D

Reply to this Comment

Oh crap, that line:

foo = "hello"; // Variables scope.

... was supposed to be:

variables.foo = "hello"; // Variables scope.

Reply to this Comment

So, am I write in assuming:

function myFunction() {
		var bar = "foo";
    variables.foo = "hello"; // Variables scope.

    thread ... {
        // CAUTION - this will get saved to the THREAD-local scope.
        bar = "bar";

        // To set to variables INSIDE a thread, you need to explicitly provide scope.
        variables.foo = "bar";
    }
    
    writeOutput(variables.foo);
    // bar
    writeOutput(bar);
    // foo

}
Reply to this Comment

@Charles,

Yes, 100%. In fact, there's no way (that I know of) to alter a Function-local variable from inside a CFThread body. So, the first var bar = "foo"; you have in the Function body, cannot be touched by the thread-body.

Reply to this Comment

Wow. This gives me so many different options now. I feel like for the first time, I truly understand CFTHREAD scoping. But, I am like a dog with a bone. I now want to know more about the CFTHREAD ARGUMENTS scope. I am intrigued?

Reply to this Comment

@Charles,

Ha ha, honestly, don't even think about the "arguments" scope for threads. It's not really even documented. It's just there because it's technically a function under the hood.

That said, if you want to have to some fun, run this:

<cfscript>

	thread name="mythread" a="1" b="2" {
		var c = "3";

		thread.args = arguments;
		thread.local = local;
		thread.funcName = getFunctionCalledName();
	}

	thread name="mythread" action="join";

	writeDump( cfthread.mythread );

</cfscript>

In the output, you will be able to see that the name of the Function that represents the thread. For example, when I run this, the output of cfthread.mythread.funcName is _cffunccfthread_cftest2ecfm13918505741.

Reply to this Comment
thread name="mythread" a="1" b="2" 

These look like attributes to me?
But, what you are saying makes perfect sense, because if CFTHREAD is a function, then it must have ARGUMENTS.

I thought CFTHREAD was like CFMODULE, and that you could pass attributes in.
So are you saying that the ARGUMENTS scope is synonymous for the ARGUMENTS scope. Or, am I just going mad, and there is no ATTRIBUTES scope?

Reply to this Comment

@Charles,

You are not mad. If you run that code, you will see that the attributes scope, in the context of CFThread is actually an argument to the "hidden" function. So, when you reference:

attributes.a

... you are implicitly referencing:

arguments.attributes.a.

It's best not to think about it too much :D

Reply to this Comment

Post A Comment

You — Get Out Of My Dreams, Get Into My Comments
Live in the Now
Oops!
NEW: Some basic markdown formatting is now supported: bold, italic, blockquotes, lists, fenced code-blocks. Read more about markdown syntax »
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.