Skip to main content
Ben Nadel at the jQuery Conference 2011 (Cambridge, MA) with: Niall Kader
Ben Nadel at the jQuery Conference 2011 (Cambridge, MA) with: Niall Kader

Does Order Of Twitter API Parameters Make A Difference?

By on
Tags:

I've been having a hell of a time trying to clean up KinkyTwits - my ColdFusion powered Twitter client - for release. One big problem that I've been having is that the API seems to suddenly stop working. It will be chugging along fine and then all of a sudden, I get nothing but the following errors:

Twitter / Over capacity - Too many tweets! Please wait a moment and try again.

This error will keep coming back for hours. Then, completely randomly, it will work for a moment, and then stop again for hours.

Now, I know it can't be that the API is over capacity because when this starts to happen, I'll jump on TweetDeck and it's fine. I've been banging my head against the wall for many hours on this problem and I think I might have finally gotten to the bottom of the issue. I don't see this mentioned anywhere in the API documentation, but it seems as if the order of the Parameters passed in the GET request makes a difference. To see what I'm saying, take a look at this test code:

<!---
	Make the first API request putting the SINCE_ID paramter
	followed by the COUNT parameter.
--->
<cfhttp
	method="get"
	url="http://www.twitter.com/statuses/friends_timeline.json"
	useragent="KinkyTwitsTwitterBot"
	username="bennadel"
	password="123xyz"
	result="objTwitterRequest">

	<cfhttpparam
		type="url"
		name="since_id"
		value="1158700580"
		/>

	<cfhttpparam
		type="url"
		name="count"
		value="200"
		/>

</cfhttp>

<!--- Output response. --->
<cfdump
	var="#ToString( objTwitterRequest.FileContent )#"
	label="Request With SINCE_ID First."
	/>


<!---
	Make the second API request putting the COUNT paramter
	followed by the SINCE_ID parameter (the rest of the code
	is exactly the same as the first API request).
--->
<cfhttp
	method="get"
	url="http://www.twitter.com/statuses/friends_timeline.json"
	useragent="KinkyTwitsTwitterBot"
	username="bennadel"
	password="123xyz"
	result="objTwitterRequest">

	<cfhttpparam
		type="url"
		name="count"
		value="200"
		/>

	<cfhttpparam
		type="url"
		name="since_id"
		value="1158700580"
		/>

</cfhttp>

<!--- Output response. --->
<cfdump
	var="#ToString( objTwitterRequest.FileContent )#"
	label="Request With COUNT First."
	/>

Notice that in the first request, I am sending the SINCE_ID followed by the COUNT parameter. This is the same order as it is listed in the API documentation (although no mention of order specification could be found). In the second request, I am then just switching the order of the parameters. Now, when I run this, very consistently, the first request returns JSON-formatted statuses and the second request returns the "Too many tweets! Please wait a moment and try again" error.

If you are thinking that this is happening because I am following one API request with another immediately after it, don't worry - I thought of that. If I run the code above, with the CFHttp tags reversed, I get the same, consistent outcome but in the reverse order (first request fails, second one succeeds).

Could it be that the order of the URL parameters in the Twitter API requests actually makes a difference? Is that possible? I can't see how that could be true in any sort of name-keyed system. Has anyone else had any experience with this?

So, why does this work sometimes for a while and then start failing all of a sudden? I assume it's because I am passing my Parameters collection through to my Twitter API execution method as a ColdFusion struct. I am then looping over that struct and creating CFHttpParam tags:

<!--- Make URL parameters for GET method. --->
<cfloop
	item="LOCAL.Key"
	collection="#ARGUMENTS.Parameters#">

	<cfhttpparam
		type="url"
		name="#LCase( LOCAL.Key )#"
		value="#ARGUMENTS.Parameters[ LOCAL.Key ]#"
		/>

</cfloop>

Because ColdFusion has no inherent order for structs, the order of the CFHttpParams is generated randomly. Of course, from CFDump experience, we know that this order is not totally random - it does get stuck in a rut (or perhaps comes out alphabetical). Either way, if we need ordering to be specific, we simply cannot rely on ColdFusion structs. Looks like I'm gonna have to go back and start passing my parameters through using an array of name-value pairs.

Want to use code from this post? Check out the license.

Reader Comments

15,666 Comments

@BigMadKev,

I thought that, but if I reverse the order of the CFHTTP tests above, they fail in reverse order. If it was an API request limit, they would both have to fail on repeated requests. But, the once with "since_id" first always works, regardless of test placement.

15,666 Comments

@BigMadKev,

Plus and API limit issue would return with a 400 status code - these are returning with a 200 OK status code.

2 Comments

In that case then it's time to blame the technology behind twitter ;)

As it's not the developers fault is it :)

Same as MySpace.com was the Technologies fault not the devleopers.

:p hehe

15,666 Comments

@BigMadKev,

I just switched from using Structs to using Arrays for my Parameters collection and its working :) Crazy stuff!

25 Comments

The only thing I can see is that they're using index-based arguments, rather than the actual argument key.

ie: count = arguments[ 1 ].

Then again, if that's whay they're doing the order would impact the actual values assigned.

42 Comments

Ben,

I am using the twitter api cfc from Ria Forge and I just looked at the internals and here is what I found-

<cfhttp
url="http://twitter.com/#arguments.resource#.#returnFormat#?#queryString#"
method="#arguments.method#"
username="#variables.username#"
password="#variables.password#"
useragent="ColdFusion/8.0">

<cfif arguments.method eq "post">
<cfloop collection="#arguments.params#" item="key">
<!--- Deal with nulls so passing the arguments scope works properly. --->
<cfif structKeyExists(arguments.params,key)>
<cfhttpparam name="#lCase(key)#" value="#arguments.params[key]#" type="formfield">
</cfif>
</cfloop>
</cfif>
</cfhttp>

That appears to be doing about the same thing that you are doing. I haven't had any problems whatsoever.

15,666 Comments

@Brandon,

I have no idea :) Like I said, mine was working for a long time and then stopped. Maybe ColdFusion started sorting the params differently? Now that I have switched to an array, its all good. Depending on what params you have, CF might just be ordering them correctly.

48 Comments

I'm throwing a hail mary here, but could it be that they're using some sort of hash for the requests you are sending as a key to throttle you and the order of the parameters results in a different hash?

48 Comments

Check this:

<cfset a = createObject("java", "java.util.HashMap") />
<cfset a.put("keyOne", "valOne") />
<cfset a.put("keyTwo", "valTwo") />

<cfset b = createObject("java", "java.util.HashMap") />
<cfset a.put("keyTwo", "valTwo") />
<cfset a.put("keyOne", "valOne") />

<cfset c = createObject("java", "java.util.HashMap") />
<cfset c.put("keyOne", "valOne") />
<cfset c.put("keyTwo", "valTwo") />

<cfdump var="#a#">
<cfdump var="#b#">

A = B?<br />
<cfdump var="#a.equals(b)#">
<br />
A = C?<br />
<cfdump var="#a.equals(c)#">

48 Comments

DAMN! I got all excited thinking that was correct (that A != B but A=C), until I saw my copy paste error. Here is the correct code:

<cfset a = createObject("java", "java.util.HashMap") />
<cfset a.put("keyOne", "valOne") />
<cfset a.put("keyTwo", "valTwo") />

<cfset b = createObject("java", "java.util.HashMap") />
<cfset b.put("keyTwo", "valTwo") />
<cfset b.put("keyOne", "valOne") />

<cfset c = createObject("java", "java.util.HashMap") />
<cfset c.put("keyOne", "valOne") />
<cfset c.put("keyTwo", "valTwo") />

<cfdump var="#a.hashCode()#"><br />
<cfdump var="#b.hashCode()#"><br />
<cfdump var="#c.hashCode()#"><br />

A = B?<br />
<cfdump var="#a.equals(b)#">
<br />
A = C?<br />
<cfdump var="#a.equals(c)#">

A == B == C

So, at least in Java the order of the keys in a map does not result in a different hashcode for a hashmap. But do you see where I was going with it?

15,666 Comments

@Todd,

I am not sure I follow?? We are not passing the hash map to Twitter. The Struct is merely used as a collection over which we iterate to write out the CFHttpParam tags. So, collection-aside, the values are passed individually.

48 Comments

Right, but, my skewed thinking was that somehow on Twitter's side of the API they are taking the arguments that you are passing in and creating a hash of that argument structure as a key to identify your requests. My thought was that they use that key to throttle you if you send too many requests within a specified time period. If the order in which you passed the args (or, the order in which they received the args) resulted in a different hash code then theoretically your requests would appear to be from a different source if you passed them in a different order.

Crazy theory - just disregard me ;)

I believe in love. I believe in compassion. I believe in human rights. I believe that we can afford to give more of these gifts to the world around us because it costs us nothing to be decent and kind and understanding. And, I want you to know that when you land on this site, you are accepted for who you are, no matter how you identify, what truths you live, or whatever kind of goofy shit makes you feel alive! Rock on with your bad self!
Ben Nadel