ColdFusion CFPOP - My First Look
Posted November 27, 2007 at 9:52 AM
I have never done anything mail related with ColdFusion other than sending mail using ColdFusion's CFMail tag. After years of this, I think it's finally time to start making this a two way street - healthy relationships are all about give AND take. I want to start looking into ColdFusion's CFPOP tag to see how I can get mail off a server and thereby, open up an automated bidirectional line of communication using the pop mail server as the go-between.
Before I could even start looking into ColdFusion CFPop, I had to get a mail server that could handle non-SSL POP3 support. I can't use my gMail accounts because apparently they use SSL which ColdFusion cannot yet handle (as of CF8). So, I did a quick Google search for "Free Pop mail" and came across the site Lavabit. This site seems perfect for testing. Their FREE, basic account offers:
- Storage: 128 MB
- Advertising: No
- Virus Protection: Yes
- Statistical Spam Filter: No
- Incoming Message Limit: 1,024 messages/day
- Outgoing Message Limit: 256 messages/day
- Message Size Limit: 32 MB
This should be more than enough to provide the perfect ColdFusion CFPop testing environment. I would say the setup took about 2 minutes... and did I mention that the basic account is FREE :)
Once I got the mail server setup done, I had to start building my ColdFusion testing environment. I am going to keep this ultra simple while I get my bearings on how all the CFPop stuff works. I set up a simple ColdFusion Application.cfm template:
Launch code in new window » Download code as text file »
- <!--- Set page settings. --->
- <cfsetting
- showdebugoutput="false"
- requesttimeout="20"
- />
-
-
- <!---
- Create the POP server login information. Lavabit's
- UNSECURED POP3 port is 110.
- --->
- <cfset REQUEST.Pop = StructNew() />
- <cfset REQUEST.Pop.Server = "lavabit.com" />
- <cfset REQUEST.Pop.Username = "bennadel" />
- <cfset REQUEST.Pop.Password = "XXXXXXXX" />
- <cfset REQUEST.Pop.Port = "110" />
Now, all of my testing code can use the centralized REQUEST.Pop configuration information.
Ok, so let's start taking a look at how ColdFusion's CFPop tag works. The CFPop tag has three actions:
- GetHeaderOnly
- GetAll
- Delete
GetHeaderOnly and GetAll both receive information from the POP mail server. The Delete action deletes a given message (or list of messages) from the mail server. The reason there are two "get" actions is for performance optimization. The GetHeaderOnly action gets information about the messages but leaves out the attachments and content information that can take more processing and transfer time:
Launch code in new window » Download code as text file »
- <!--- Gather the email headers. --->
- <cfpop
- action="getheaderonly"
- name="qHeader"
-
- <!--- Server configuration. --->
- server="#REQUEST.Pop.Server#"
- port="#REQUEST.Pop.Port#"
- username="#REQUEST.Pop.Username#"
- password="#REQUEST.Pop.Password#"
- />
The email header information that comes back from the server is stored in a standard ColdFusion query object with the following columns:
- To
- CC
- From
- ReplyTo
- Subject
- Date
- Header
- MessageID
- MessageNumber
- UID
The To, CC, From, ReplyTo, Subject, and Date fields are pretty self-explanatory. The Header field contains the email header information that seems to have a lot of routing and "behind the scenes" information. The MessageID is some sort of unique identifier that is parsed out of the Header data. The MessageNumber seems to be the sequential "order ID" of the message as it sits in the mail box. The UID seems to be a unique identifier for that message that can be used to filter all of the CFPop actions (GetHeaderOnly, GetAll, and Delete); apparently it is the header's X-UID field, but this field is not present in the Header data.
Note that I am saying "seems to be" on several of these values above because really, I am not 100% where these values come from. This is the first time I am looking into this CFPop stuff, so please don't take this with any sort of authority.
When you use the GetAll action:
Launch code in new window » Download code as text file »
- <!--- Gather the entire email messages. --->
- <cfpop
- action="getall"
- name="qMessage"
-
- <!--- Server configuration. --->
- server="#REQUEST.Pop.Server#"
- port="#REQUEST.Pop.Port#"
- username="#REQUEST.Pop.Username#"
- password="#REQUEST.Pop.Password#"
- />
... ColdFusion retrieves all of the above columns plus the following additional content and attachment columns:
- Body
- TextBody
- HTMLBody
- AttachmentFiles
- Attachments
- CIDs
The TextBody and the HTMLBody fields contain data from the different parts of the message if different Content Types were supplied (think CFMailPart [type = text/plain or text/html]). Needless to say, Text/Plain goes into the TextBody field and the Text/HTML goes into the HTMLBody field. The Body field always contains the first content part found in the email message (in top-down order). The attachments is a tab delimited list of attached file names and the AttachmentFiles is a tab delimited list of file path corresponding to the file names (the files are downloaded to the ColdFusion server as part of the CFPop action when AttachmentPath is supplied in the CFPop tag). CIDs is a structure that contains the key-value pair data for the embedded images in an HTML email. I have not seen this in action yet, but from the looks of it, the key is the name of the image file and the value is the CID that it corresponds to.
Once we get the email messages from the POP server, we can delete them using the Delete action:
Launch code in new window » Download code as text file »
- <!--- Delete the given email message. --->
- <cfpop
- action="delete"
- uid="82379091"
-
- <!--- Server configuration. --->
- server="#REQUEST.Pop.Server#"
- port="#REQUEST.Pop.Port#"
- username="#REQUEST.Pop.Username#"
- password="#REQUEST.Pop.Password#"
- />
As you can see here, I am filtering the delete action using the UID attribute. This UID attribute can contain a single value or a comma delimited list of values that correspond to the UID fields in the GetAll and GetHeaderOnly CFPop actions. If you do not supply some sort of filtering UID (exclude the UID attribute in the CFPop tag), then all of the emails on the server will be deleted.
Ok, so that's like a birds eye view of the ColdFusion CFPop tag. Please don't look at this as a tutorial; this was mostly a gathering of notes for myself. There is a lot that can be done here, but I think the premise is fairly simple. Now, I just need to come up with a really cool idea that will help me learn how to fully utilize this functionality. My first instinct is to integrate this with some SMS text messaging through the cellular email gateways... but that's another post, and another adventure.
Links for my future study:
Download Code Snippet ZIP File
Post Comment | Ask Ben | Permalink | Other Searches | Print Page
Newer Post
Using CFPOP And CFMAIL For Bidirectional SMS Text Messaging In ColdFusion
Older Post
Final Thoughts On CFParam And Data Type Validation
Reader Comments
The fun part comes when you are trying to render the html body. for instance, Outlook likes to throw a lot of Outlook specific tags in its message body. CID's are sent as attachments. The fun part comes in when you are trying to render the HTMLBody. You get to go in and parse out the CID tags and link it back to the attachment.
I have a sick sense of fun. ;)
Nicely done. CFPOP can be a real pain some days - depending on the server you are dealing with. Having an email account you know works with CFPOP is greatly helpful in debugging other email providers.
@Shane,
Good challenges are fun indeed. Nothing sick or twisted about it. I am sure I will build up to doing something cool like that.
@WTL,
I would imagine so. Luckily, this LavaBit.com place seems pretty good. From the testing I have done, it seems snappy enough and has not shown any problems with my action. I haven't messed around with any attachments yet, but so far, so good.
Ben - as always great explanation. I'd read your blog even if you were writing about CFPOOP and instead of CFPOP.
@Bruce,
Ha ha ha :) Thanks a lot. I hope I can do some cool stuff with CFPOP. It seems like there is all this potential here that I can run wild in.
Hi Ben,
you can integrate with gmail probably using IMAP...
read here
http://webtrenches.com/post.cfm/connecting-to-gmail-using-coldfusion
I really look forward to your postings on SMS integration. I've been meaning to set aside some time to research that myself, so if you beat me to it you will save me some time :).
Hey Ben, I was just researching CFPOP and SSL tonight when I came across your post. It turns out there is a way to support CFPOP over SSL: just go into the JVM properties and set it to use SSL over the specified port. I learned of this technique by looking at the code in a UDF at
http://cf_sslpop.riaforge.org/
I've found nothing yet to indicate whether CFPOP can be made to work with TLS (although you can make it work with SSL over port 995, although I have no idea whether that's technically using TLS or not).
@Tom,
This is interesting. Do you have any idea if it changes the JVM settings for just my page request? Or does that SSL setting stick across pages. I just hesitate to mess with settings that may or may not affect other applications running on the same box.
@Anuj,
It looks like far more code goes into using IMAP over POP. I will look into IMAP stuff after I get a little more comfortable with the POP world. This is my first foray into the world of "receiving mail" so I want to take it slow.
@Ben,
Regarding whether changing the javaService's pop3 properties affects other applications or other sessions, the answer is that I think it does, but not in a bad way. After changing the javaService properties to enable SSL in one session (for a connection to a Yahoo! mailbox over port 995), I dumped the same properties in another session (for a connection to my own QMail server over port 110). Both sessions showed the same, new settings with SSL enabled. But cfpop continued to work correctly for both sessions, even though my own server doesn't support SSL.
So let's put it this way-- I think the changes to the java properties do affect other sessions and applications, but my guess is that they only enable the use of SSL. The changes don't seem to make all cfpop calls require SSL.
That being said, I am definitely a noob when it comes to the underlying Java platform. Do share a little uneasiness about this technique, and I would love to hear the opinions of someone who's more of an expert. I'll contact the UDF's author to see if he can shed any light on the discussion.
@Tom,
Good point as to the minimal impact. Like you, my only uneasiness is that I don't know much about the underlying Java platform (only some of its classes). I suppose enabling SSL can never really harm other projects.
You can also use Stunnel. I've been successfully using it for a long time now and was hoping to quit using it in CF8 (with new CFMail SSL/TLS support) but nope... still is needed as CFPOP was left behind.
More here:
http://www.harelmalka.com/?p=64
and here:
http://www.stunnel.org/
@Harel,
Thanks for the tip.
This is a great introduction to the use of CFPOP. Recently I was asked to write an app that would go through about 8,000 emails, filter the bouncebacks, and retrieve all the bouncing email addresses that came back from a recent newsletter we sent out. I immediately thought of CFPOP even though I had never used it before. The second link from my Google search took me here.
Initially, I couldn't get CFPOP to work because of the SSL requirement our mail server has. Luckily, I found a workaround for that on livedocs.adobe.com. Once I got it to connect, all I had to do was loop through each email and filter out the addresses with some regular expressions. This tag made it so much easier than having to go through each message manually!
@Jose,
That's awesome my man! Glad to know that I could help out a bit. I've also been getting back into CFPOP and found a great link with SSL and GMail via POP:
http://techfeed.net/blog/index.cfm/2008/3/28/No-SSL-With-ColdFusions-CFPOP
Anyway, good luck with your endeavors!
Nice cfpop intro. I have just started using it and have a problem that no one so far has responded to on other sites so I thought I would see if you have any suggestions. My uids all have commas in them, so it basically makes them useless as far as I can tell. What should the uids look like? Running CF8. Here are a few examples of the uids I am getting:
1242171504.5738.mail16,S=7799
1256158656.31258.mail41,S=4187
1256160932.4879.mail4,S=5539
Since the uid attribute is a comma-separated list or single value, none of these uids work.
Thanks for any suggestions.
@Cheryl,
I think it's probably ok that the UUID has a comma in it. UUIDs are not numeric values - they usually contain alpha-numeric characters and dashes. As long as the mail client can use it to look up an item, it should be fine.
Are you getting an error when trying to use them?
Thanks for responding Ben. The problem is that the cfpop tag uid attribute is looking for a single value or a comma separated list of ids. Since the uid has a comma in it, the tag things it is handling multiple ids when really it is just one. So, the cfpop query does not find the messages by uid because the ids that cfpop is using are incorrect as it is breaking the single id into two ids because of the comma. I have not found a way to specify a different list delimiter. I also tried using pieces of the uid to see if that would work, but the message are not found. I found one other person reporting the problem (they posted a message to the CF8 documentation page for cfpop as did I), but no one else has commented. I have also opened a thread in the Adobe CF forum and posted messages to my local user group, but no one has responded with a solution. For now, while it is not ideal, I moved forward using the message id, but just seems that there should be some solution to this problem. The mail account is hosted on Network Solutions, and the site is currently running CF7 (not CF8).
Let me know if you have any suggestions. Your site is great by the way:)
Thank you very much,
Cheryl
@Cheryl,
Ahhh, very interesting. I didn't even think of that. I can't say that I have any advice to offer up - I'm sorry. I haven't played with this too much and I certainly never had to deal with that problem.
Ben - just thought I would give you an update (or in case anyone else is looking for a fix to this problem). Here is a reply back I received from Adobe from their bug reporter project:
"UIDs are generated by mail server. Probably there might be some setting in mailserver to generate in a particular format."
So, back to Network Solutions I go though I doubt I will get far. I worked around the problem by using the message ID, but Adobe should realize that the UID does not always work and include something in the docs.
Anyway, thought you might be interested.
@Cheryl,
Ok, that's good to know (that it might be a setting with the mail provider). I am eager to hear what you figure out with your hosting provider.
