Monday, October 13, 2008    
Home My Books Blog ColdFusion About Me Back    

Calendar
<< May 2007 >>
S M T W T F S
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    

Search

Categories
 • Acrobat (2) [RSS]
 • Adobe (68) [RSS]
 • AdobeMAX06 (45) [RSS]
 • AdobeMAX07 (59) [RSS]
 • AdobeMAX08 (25) [RSS]
 • AIR (134) [RSS]
 • Appearances (122) [RSS]
 • Books (69) [RSS]
 • CFEclipse (14) [RSS]
 • ColdFusion (1154) [RSS]
 • Data Services (13) [RSS]
 • Fish Tank (2) [RSS]
 • Flash (106) [RSS]
 • Flex (373) [RSS]
 • Home Automation (3) [RSS]
 • Jobs (97) [RSS]
 • JRun (13) [RSS]
 • Labs (27) [RSS]
 • LiveCycle (22) [RSS]
 • MAX (160) [RSS]
 • Regular Expressions (13) [RSS]
 • RIA (12) [RSS]
 • SQL (38) [RSS]
 • Stuff (505) [RSS]
 • Tips (CF Studio) (80) [RSS]
 • Tips (CF) (795) [RSS]
 • Tips (Dreamweaver) (91) [RSS]
 • Tips (Flex Builder) (2) [RSS]
 • Using CF (137) [RSS]
 • Wireless (99) [RSS]

Other BLOGs
 • Charlie Arehart
 • Lee Brimelow
 • Ray Camden
 • Christophe Coenraets
 • Sean Corfield
 • Mihai Corlan
 • Cornel Creanga
 • John Dowdell
 • Danny Dura
 • Enrique Duvos
 • Steven Erat
 • Kevin Hoyt
 • Serge Jespers
 • Adam Lehman
 • Duane Nickull
 • Miti Pricope
 • Andrew Shorten
 • Ryan Stewart
 • James Ward
 • Greg Wilson
 • Full As A Goog

RSS Feeds
 • Feed
 • Subscribe

Join my mailing list and find out about new books and other topics of interest.

Thoughts, ideas, tips, musings, and pontifications (not necessarily in that order) by Ben Forta ...
NOTE: This is my personal blog, and the opinions and statements voiced here are my own.

Viewing By Entry / Main
May 11, 2007

Scorpio File I/O Enhancements

At one of the usergroup sessions this week someone asked if there was a way to get file information (size, date time, etc.) easily using a function. I said they should use <CFDIRECTORY>, but afterwards remembered that we did indeed add a new function to Scorpio called GetFileInfo() which returns a structure containing:

  • canread
  • canwrite
  • ishidden
  • lastmodified
  • name
  • parent
  • path
  • size
  • type

Which makes this a good opportunity to review some of the file i/o changes coming in Scorpio.

For starters, if you have ever had to work with large text files in ColdFusion (maybe parsing a large CSV file) you'll know that doing so is very inefficient. You probably use code like this:

<!--- Read entire file --->
<cffile action="read"
   file="#fileName#"
   variable="myFile">

<!--- Loop through file variable one line at a time --->
<cfloop list="#myFile#"
   index="line"
   delimiters="#chr(10)##chr(13)#">

   <!--- Do stuff with line here --->
   ...
</cfloop>

This is slow for two reasons. Not only does ColdFusion read the entire file into memory in a variable all at once, but also looping through the file requires treating it as a list which involves lots of parsing which can also be resource intensive.

Well, inefficient no more. In ColdFusion Scorpio you'll be able to replace the above code block with this:

<!--- Loop through file one line at a time --->
<cfloop file="#fileName#"
   index="line">

   <!--- Do stuff with line here --->
   ...
</cfloop>

This code block open the file, reads one line at a time, and closes it when done. I actually used this myself recently in a ColdFusion code snippet that had to parse a massive tab delimited file, turning each line into a query row. Replacing the old <CFFILE> <CFLOOP> with a new <CFFILE FILE=> cut down processing time from several minutes to under 10 seconds.

Oh, and although reading files line by line is the more common use case, you can also read by n characters at a time, like this:

<!--- Loop through file 100 characters at a time --->
<cfloop file="#fileName#"
   index="chars"
   characters="100">

   <!--- Do stuff with line here --->
   ...
</cfloop>

In addition to the <CFLOOP> enhancements, we've also added lots of new file i/o functions that you can use to access and manipulate files directly. The new functions include:

  • FileClose()
  • FileCopy()
  • FileDelete()
  • FileIsEOF()
  • FileMove()
  • FileOpen()
  • FileRead()
  • FileReadBinary()
  • FileReadLine()
  • FileSetAccessMode()
  • FileSetAttribute()
  • FileSetLastModified()
  • FileWrite()
  • GetFileInfo()
  • IsImageFile()
  • IsPDFFile()

TrackBacks
Faster file parsing in Scorpio
Finally! File parsing will get faster in Coldfusion, as Ben Forta informs us via his blog.
Tracked by Build It Smarter | Tracked on 5/14/07 12:33 PM

No trackback URL. Trackbacks are only allowed via interactive form.

Comments
Am I safe in assuming that the file loop releases each line from memory when it's processing completes? Otherwise the memory issues will persist, but you didn't mention it explicitly so I thought I'd verify.
# Posted By Rob Wilkerson | 5/11/07 1:12 PM
Rob, yes, it reads a line at a time, it does not keep previously read lines in memory, and so it does indeed solve that memory issue.

--- Ben
# Posted By Ben Forta | 5/11/07 1:16 PM
Ben, this is awesome. I especially like being able to do file/directory operations in cfscript without having a bunch of UDFs!
# Posted By Rick Root | 5/11/07 1:19 PM
you have just made my day! thanks for the info ben.
# Posted By tony of the weeg clan | 5/11/07 3:49 PM
I cant wait to see the performace increase when dealing with CSV style files
# Posted By Damien Jorgensen | 5/12/07 3:10 AM
Arrgh. So just release the @#$%@$#%^ thing already! :-)
Seriously, looks like we're getting some very nice robust enhancements.
Some which I can use like now! :-0

Anyways, I do look forward to the release and your teaser points are nice to read. It's comforting to know the there are advantages to an owned and fiscally supported product - ie: features, and that Adobe's first full release under their own name (I think the current version has one of Adobe's longest official products names, something like "Macromedia ColdFusion MX 7 Standard Edition By Adobe", or something like that) should be a substantial re-launch for the product.
# Posted By Stephen Cassady | 5/12/07 4:42 AM
In the words or Dr Who, "Fantastic!"
# Posted By James Holmes | 5/12/07 6:24 AM
Oh boy, this is going to solve so many problems I've had in the past using cffile/cfloop.
Fantastic new feature.
# Posted By Marko | 5/13/07 9:38 AM
I am wondering though if there is any functions for directory manipulation to complement then file I/O functions? That would be great also.
# Posted By Shuns | 5/13/07 9:45 PM
Hello Ben
This is just great work. Thanks for all you did.
# Posted By Abhi | 5/16/07 5:44 PM
This is an old thread but I am parsing a large CSV file with data that might contain commas in each quoted string. For instance, one record might look like this: "454454545","Last Name, First Name","City, State" It's the comma INSIDE the quotes that is throwing everything off. How can I allow for this?
# Posted By Downs | 3/10/08 6:35 PM

  © Copyright 1997-2008 Ben Forta, All Rights Reserved