RSS feed for InstantSpot site cfZen

Top 10 Architecture Scalability Mistakes Made in ColdFusion

Sat, 08 Aug 2009 05:56:00 GMT

This is a focus on architecture design mistakes I've seen made too often over the years by CF developers building ColdFusion architectures. Often times, the codebase and system started small and grew too fast, such that the original developer/architect didn't have the time or experience to scale the system up with the demands of high-load applications.

Not dedicating enough time and resources on optimizing the database:

Too often, there are smart, but unqualified programmers/developers designing and maintaining the database. A qualified/certified DBA, even part-time, is well worth the investment. You need one who can profile your database, tune all the server and individual database settings, recomming index changes, be responsible for DB design and integrity, regular maintenance, set up replication/failover/clustering, and will proactively monitor your database to let you know what problems need attention in relationship to your applications.
Not dedicating enough time and resources on optimizing queries:

Again, developers tend to throw themselves into the role of writing all the queries, but don't necessarily think about the performance/scalability issues of writing non-optimal queries. Often times a DBA who is qualified for all the Database Server stuff (see #1 above) isn't the same type of DBA who can optimize your queries. You need someone who can run the queries with explain plans, and then can make recommendations on improvements. These improvements pay off immensely!
Coupling all parts of the system into one mega-application, instead of separate applications

Usually they start out as fine: clever little applications that "do the job". They may have started in the CF 4 or 5 days, and grow and grow over the years until they're a mass of spaghetti code. It may never occur to the developers that as the app gets bigger that not everything belongs together any more. It will take some work, but it's important for long-term stability and scalability that you de-couple big chunks into smaller applications that run independent from the original beast. Scheduled Tasks, Web Services, APIs, User Admin Areas, Reporting, file/image services, Search services, etc. should no longer be coupled to the main application. Each of those parts should be broken out, so that each can be tuned optimally. Do you want your whole system to start bottlenecking because Search or Web Services become slow or go down? If they're decoupled, you don't have that problem.
Not using a good "shared" file system scalable for high numbers of file writes

When there are multiple physical servers in your cluster (if applicable), you may want a single location to save files. No-no's here are mapped drives/mounts to a non-dedicated server (such as a drive on your database server, some external drive, etc.). A good NAS and/or SAN solution with failover scales better as file writes become higher over time. Cloud Computing solutions are becoming big now, too, but I haven't tested enough in this area to make sound recommendations. I'm sure other with more experience can tell us. Comments?
Not tuning the JVM

Many articles have already been written on this subject, so I won't go into it. If you're having performance problems and you don't know how to tune your JVM, get help from one of the many articles out there, hire an expert, or get Adobe Platinum support for CF.
Not tuning Request settings in the ColdFusion Administrator

See #5 above.
Not making good use of Multiple Instances

When you de-couple areas of your application (see #3), often it's wise to put your new applications into separate CF instances. For instance, all heavy tasks (Scheduled Tasks and other "batch" type processes) should run on a separate instance. You don't want to know how many times I've seen a Scheduled Task bring the entire infrastructure to it's knees.
Not using CF Monitoring tools, such as SeeFusion, CF8 Server Monitor, and/or FusiontReactor

Get familiar with at least 2 of these, and hire an expert to help you get up to speed on them more quickly. These save your butt!
Trying to recreate the "holy grail" reporting system from scratch

First, any reporting system should be a prime candidate for de-coupling. And reports that come from the database should not be running against your "write" (master) database, but rather against your "read-only" (slave) database, or else you'll have a lot of bottlenecks on your hands as users create massive reports reading data while other users are trying to simultaneously write data via the rest of the application.

But secondly, look for reporting solutions that are already available. Most of the time users don't really need *real-time* reports, and off-line/archived data lends itself better to the more complex reports that users start to request over time (allowing Excel exports makes a lot of people happy, cuz they know how to generate reports "the way they like them" in familiar Excel. Also, don't write a Report UI that allows users to request *too many* records - your milage will vary, but it's almost never a good idea to allow reports that have no filters, such that they can just *get everything* all at once. Few web apps are built for handling that much data, so find other solutions.
Not taking advantage of various caching capabilities:

We all know (hopefully) to cache queries where possible, but... do you use the Application/Server scope where appropriate, and to use cfcache (or other 3rd party CF cache tags/CFCs/frameworks) in areas that rarely change? Do you come up with caching strategies for data, so that it can persist in the Application/Server scope without going back to the dB unnecessarily? Could other data/files be cached to disk? Can you use Verity (or Lucene/Solr) for searches instead of pulling directly from the database all the time?

OPTIMIZE all MySQL database tables via ColdFusion

Mon, 29 Jun 2009 20:46:00 GMT

Ever want to run a script that does an OPTIMIZE of all tables in your database? First off, you can do this with a shell script or cron job with something like: "mysqlcheck -u root -p --auto-repair --check --optimize --all-databases"

But what if you, say, don't have a full-time DBA on staff, and want to run these on a schedule which you can easily control with ColdFusion? That was our situation, and so here's what I came up with. We ran it last weekend during our scheduled "maintenance window" and it took about 90 minutes with no downtime on a MySQL 5.1 18G database

(disclaimer: the speed of this process may vary, depending on numerous variables such as hardware, memory, version of MySQL, etc.)

<!--- set these according to your database size: 14400 and 1800 were used after extensive testing  on a system with 8G ram and quad core in 2009 for an 18G database; you can easily try different  values here by trying them in the URL  (ex: /OPTIMIZE_ALL_DB_tables.cfm?queryTimeout=5&numOfTables=5) ---> <cfparam name="URL.RequestTimeout" default="14400"> <cfparam name="URL.queryTimeout" default="1800"> <cfparam name="URL.myDatasource" default="my_database"> <cfparam name="URL.numOfTables" default="1000000">  <!--- set u/p to one that has permissions to run OPTIMIZE & FLUSH on your tables ---> <cfset Variables.myUsername = "db_user"> <cfset Variables.myPass = "my_password"> <cfset Variables.toAddresses = "webmaster@mydomain.com;aaron@mydomain.com"> <cfset Variables.fromAddress = "webmaster@mydomain.com">  <cfsetting requesttimeout="#URL.RequestTimeout#">   <!--- get host and instance if they don't already exist ---> <cfif NOT IsDefined("Server.hostName")>   <cftry>   <cfset local.oInetAddress = CreateObject("java", "java.net.InetAddress")>   <cfset Server.hostName = local.oInetAddress.getLocalHost().getHostName()>    <cfcatch type="application">    <cfset Server.hostName = "Could not determine with java.net.InetAddress">   </cfcatch>  </cftry>  </cfif>  <cfif NOT IsDefined("Server.instanceName")>   <cftry>   <cfset local.oJRun = CreateObject("Java", "jrunx.kernel.JRun")>   <cfset Server.instanceName = local.oJRun.getServerName()>    <cfcatch type="application">    <cfset Server.instanceName = "Could not determine with jrunx.kernel.JRun">   </cfcatch>  </cftry>  </cfif>    <cfquery name="rsAllTables"   datasource="#URL.myDatasource#"   username="#Variables.myUsername#"   password="#Variables.myPass#">  SHOW TABLES IN peoplexs_test; </cfquery>  <!--- <cfdump var="#rsAllTables#" format="text"> ---> <cfset processInfo = "">  <cfoutput query="rsAllTables" maxrows="#URL.numOfTables#">   <cfset dbErrorInfo = "">  <cfset anyErrorInfo = "">  <cfset beginProcessInfo = "">  <cfset endProcessInfo = "">   <cfsavecontent variable="beginProcessInfo">  --- <strong>"OPTIMIZE TABLE #rsAllTables.TABLES_IN_PEOPLEXS_TEST#" started</strong>... <br/>  </cfsavecontent>  <cfset processInfo &= beginProcessInfo>   #beginProcessInfo#  <cfflush>   <cftry>    <cfquery name="qOptimizeTable"     datasource="peoplexs_test"     username="#Variables.myUsername#"     password="#Variables.myPass#"     timeout="#URL.queryTimeout#">    OPTIMIZE TABLE #rsAllTables.TABLES_IN_PEOPLEXS_TEST#;   </cfquery>    <cfquery name="qFlushTable"     datasource="peoplexs_test"     username="#Variables.myUsername#"     password="#Variables.myPass#">    FLUSH TABLE #rsAllTables.TABLES_IN_PEOPLEXS_TEST#;   </cfquery>    <cfcatch type="database">    <cfsavecontent variable="dbErrorInfo">    <span style="color:red;">     Database ERROR: "#cfcatch.Cause.Message#" on     "#cfcatch.SQL#"    </span><br/>    </cfsavecontent>    <cfset processInfo &= dbErrorInfo>    #dbErrorInfo#   </cfcatch>    <cfcatch type="any">    <cfsavecontent variable="anyErrorInfo">    <span style="color:red;">    <cfdump var="#cfcatch#" format="text">    </span><br/>    </cfsavecontent>    <cfset processInfo &= anyErrorInfo>    #anyErrorInfo#   </cfcatch>   </cftry>   <cfsavecontent variable="endProcessInfo">  ------ done with #rsAllTables.TABLES_IN_PEOPLEXS_TEST#. <br/><br/>  </cfsavecontent>   <cfset processInfo &= endProcessInfo>  #endProcessInfo#  <cfflush>  </cfoutput>  <cfmail to="#Variables.toAddresses#"         from="#Variables.fromAddress#"         subject="OPTIMIZE_ALL_DB_tables.cfm on #Server.hostName#/#Server.instanceName# at #DateFormat(Now(), 'yyyy-mm-dd')#"         type="HTML"         server="localhost"> <cfoutput>#Trim(processInfo)#</cfoutput> </cfmail>

MultiCFAdmin Alpha 0.007 is on RIAForge.org now

Wed, 03 Jun 2009 05:51:00 GMT

Here's the link: http://multicfadmin.riaforge.org/

Description:

In this Alpha version 0.007, which allows connection to CF Admin API on local and remote CF instances, there are only a few features:

1) jQuery UI
2) ability to see basic stats of each CF8 Server Monitor API
3) ability to clear Template Cache for all instances
4) ability to toggle Trusted Cache for all instances
5) can configure multiple CF Admin URLs, passwords, and a sleep() time for how long MultiCFAdmin waits after Trusted Cache is changed

Requirements:

* ColdFusion 8.01 Enterprise or Developer, Multi-server
* multiple CF instances other than cfusion (default)
* access to jrun-web.xml configuration files on each instance

Multi-CF Admin Alpha 0.02

Fri, 08 May 2009 21:14:00 GMT

Okay, this is pretty cool (at least I think so). Here's the zip.

How many of us with multiple instances in a cluster have wanted to be able to do things like clear the template cache for every instance in the cluster with *one* button?

Well, I have.

Now, I have an early alpha that is not much more than a proof-of-concept, but does exactly that.

and in a few seconds, it's all cleared (see below)...

Here's the readme.txt:

Requirements:
* ColdFusion 8.01 Enterprise or Developer, Multi-server
* multiple CF instances other than cfusion (default)

Installation:
1.) put /MultiCFAdmin folder at www root
2.) open jrun-web.xml file at [jrun_root]\servers\[instance_name]\cfusion.ear\cfusion.war\WEB-INF (you may need to create it by copying over from cfusion instance if it doesn't exist)
3.) add a line within the XML block that points to where your WS files are, like this:

/Webservices
/opt/jrun4/servers/cfusion/cfusion-ear/cfusion-war/MultiCFAdmin/Webservices

4.) NOTE: don't attempt to connect to the cfusion instance because it causes errors with the Web Service calls to try and connect from the same instance
5.) set environment info for Application.Environment in config/Environment.cfm
6.) set instance info in config/CFInstances.cfm

Here's the zip again.

Send me a tweet if you're interested in helping with the code: http://twitter.com/aqlong

Verity "java.io.IOException: Read timed out" error?

Mon, 27 Apr 2009 19:33:00 GMT

An error occurred while performing an operation in the Search Engine library. Error switching collection offline.: com.verity.api.administration.ConfigurationException: java.io.IOException: Read timed out

I'm on twitter now: aqlong

Fri, 17 Apr 2009 16:01:00 GMT

If you care to follow me on twitter now, my account is aqlong

anyone use Nirvanix CloudNAS?

Thu, 09 Apr 2009 21:01:00 GMT

I'm looking for feedback, good or bad, on Nirvanix.

When looking at potential "cloud providers", I was amazed to read about the CloudNAS solution that Nirvanix has, which gives your servers a way to mount Nirvanix cloud storage as a drive... too cool!

It has the traditional advantages of "Global Virtualization, Unlimited Scalability, Continuous Availability, and a Usage Based Service Model." - http://www.nirvanix.com/resources.aspx#documentation.

Anyone used the CloudNAS or heard of it?

Anyone running Axis 1.4 on a CF 8 or otherwise?

Tue, 17 Mar 2009 14:48:00 GMT

For those of you who have published SOAP Web Services in ColdFusion (Adobe CF 8, OpenBD, Railo, or other), are any of you using Axis 1.4?

My research shows CF 8 and OpenDB use Axis 1.2.x., and I'm also trying to find out what Railo 3 uses. My problem is that we have a legacy system that runs some Web Services that many 3rd party partners use to interact with our system constantly. They have slow or non-existent development teams, so unfortunately forcing them to change their code to support a lower version of Axis is not an option.

I've had some discussions here with Tom Jordahl on trying to get 1.4 running on CF 8.01, but we haven't gotten anywhere yet. So, has anyone else got it working on any flavor of CF?

(fingers and toes crossed)

2 Web Developer job openings in the Netherlands

Tue, 03 Feb 2009 17:58:00 GMT

At PeopleXS.com, we have 2 positions open for developers, and you and I can work together on some exciting projects. See http://www.peoplexs.com/comp/jobs.html

We will consider applicants from all over the world. Heck, I'm an American who was living in Austin, TX six months ago. Here's the first job description:

ColdFusion Developer

We are currently looking for a ColdFusion developer to join our development team in Rijswijk ZH.

PeopleXS is an innovative high-tech Internet company specializing in on-demand recruitment systems. Although PeopleXS started up in 1999, we still have a strong start-up culture in which each initiative is encouraged. PeopleXS has a rapidly growing customer base in the Netherlands and internationally such as Capgemini, Randstad, Total, Abn Amro, Mexx and many others.

Responsibilities:

Code applications and new features according to specifications
Enhance functionality and/or performance of applications as necessary
Assist in resolving application problems and defects
Research, identify, analyze, and fulfill requirements of users as specified by Product Management
Develop and maintain technical documentation and guidelines
Unit test code
Participate in peer code reviews
Research & Development with newest CF 8 Enterprise features, and possibly AIR and Flex
Participate in development team meetings

Requirements:

Knowledge of ColdFusion or prepared to learn ColdFusion
Good knowledge of (X)HTML, CSS and JavaScript
Good knowledge of the SQL language and relational databases
Familiarity with Web standards
Good communication skills
Team player who can also work independently

It would be a plus if you have already had some experience in: Ajax, jQuery/ext-js, MySQL, SVN, XML, SOAP, Flex, Apache and Linux.

Benefits:

New, high-quality DELL Laptop
Salary up to 3600 Euro (based on experience level)
Pension
25 days off
Work from home
Flexible work hours
32-40 hours per week
Daily lunch provided
Professional training classes available
Entire development team is sent to Adobe MAX Europe every year!
We work hard, but encourage a culture of fun

You can develop yourself further in our company, like becaming Lead Developer or Lead Architect, but we also have opportunities in less technical areas such as Consultant or Product Manager.

And the second:

Junior web developer

We are currently looking for a Junior web developer to join our development team.

We are a young, high-tech and innovative company and although the company was founded already back in 1999 there is still a high start-up atmosphere in the team. We like it when things move fast and love to chase good ideas as a team. Just for the fun of it ! We have used (better: breathed) ColdFusion since version 4.0, really love it and value what it means for our company. Our idea of a good company-party would be to meet ColdFusion guru's and exchange ideas with fellow developers.

You will go through a trainee-program, called "Young professionals", where we will teach you all you need to know about ColdFusion.

Requirements:

good understanding of at least one Web scripting language, preferably ColdFusion,
basic knowledge of (D)HTML, CSS and JavaScript,
basic knowledge of the SQL language,
at least 1 year experience designing dynamic web sites,
familiarity with Web standards,
good communication skills
team player and also be able to work as an individual

It would be a plus if you already had some experience in: ColdFusion, Ajax, MySQL, SVN, XML, Linux, Apache and speak and write the Dutch language.

You can develop yourself further in our company, like becaming Lead Developer or Lead Architect, but we also have opportunities in a less technical direction like Consultant or Product Manager.

Zen Web Software Management

Fri, 19 Dec 2008 20:52:00 GMT

I thought I'd throw together a list of simple and basic (Zen) web software development "principles" that I try to keep in mind as I work on infrastructure, architecture, release process planning for my company, Peoplexs.com.

DISCLAIMER: These are pretty high-level, may be obvious to many and cliche to others, but are worth mentioning because they are so often missed or ignored by very smart people and organizations.

(in no particular order)

Communicate constantly, both upwards and downwards.
Spend the highest percentage of time on design, UI, and detailed specifications, rather than on development.
Under-promise, and over-deliver.
Commit to, and invest in, professional QA Testing.
The time to build automated tools (for testing, making builds, and releases) is well worth it in the end.
Document, practice, and update your Disaster Recovery Plan.
Lead Developers must manage expections for Product Managers at least as much as Product Managers manage the products.
Use quality, time-tested, software for integrating release management, defect tracking, and project management such as Jira or FogBugz.
Inform everyone as early as possible if project is running late.
Adding more developers late in a project only makes it more late.
Have Product Managers review, and then commit to, a final UI before development begins.
Developers want and need time to do cool and fun work. Giving them some such will help motivation and productivity.
Schedule in patch releases and archictural releases regularly, and at least 2 times per year if usage is growing quickly.
Listen fully to all suggestions and criticism, without a defensive attitude.
Be strong, and don't let a few clients dictate your long-term strategies.
Invest in Technical Conferences and Training for your team.
Empower every team member. Give each person at least one specialty that he is "in charge of".
Trust your veterans, even after they make mistakes, as they are probably more upset at themselves than you are at them.
Use something simple and flexible, like a Wiki, for internal documentation. And don't be too strict on *how* it's documented - you're lucky that there is *any*.
Lead, don't push or drag.
Celebrate, or at least acknowledge, significant accomplishments. It doesn't have to be a huge party. Something as simple as a short, yet specific, thank you on IM will do nicely... or saying to one of the Executives, "Check out the creative and helpful widget that [Developer X] whipped up last Friday!" in front of a room full of co-workers (unless Developer X is extremely shy... which sounds like a mythical person to me ;)
Keep your negative comments to private conversations as much as you can. Publicly positive people are much more likely to be followed and liked. Of course there's a fine line you must walk to keep from seeming like you're *two-faced*.
A fun, exciting work environment begins with you, from within.

I'll probably add to this list as I think of ramdom, appropriate items.

Peace.

ColdFusion Developer opening in Holland

Thu, 20 Nov 2008 16:09:00 GMT

I've been living and working in Holland for about 7 weeks now, and am loving every minute of it. Come work with me and other top-notch CF developers on exciting projects that will keep you interested for the long-term.

They have nice benefits, a fun working environment, and "perks" like getting to go to Adobe MAX Europe and to Luxembourg for a camping trip on company money.

Come join me: http://peoplexs.com/comp/ColdFusionDevelopers.htm

Who's gonna be a Adobe MAX Europe?

Thu, 20 Nov 2008 16:05:00 GMT

If so, post anything (i mean anything) below, and maybe I can buy the first round :D

I've only been in Europe (Holland) a month and a half, and don't know any CF'ers (other than 2 coworkers) in the area. I'm from Texas, so everything in Europe is "in the area" to me. Let me know.

how many bytes can a CF function be?

Wed, 19 Nov 2008 16:04:00 GMT

Apparently, the answer is 65535 bytes.

I had a huge laugh today when we got this error...

"500 cfReports2ecfc1683660545$funcRUN (Code of a method longer than 65535 bytes) cfReports2ecfc1683660545$funcRUN (Code of a method longer than 65535 bytes)"

... when trying to run a function I had just refactored for some inherited code. It had the dreaded problem that none of the function-local variables were scoped. So, I dutifully added cfset Var local = StructNew() to the top of the function and spent about 4 hours prepending all relevant variables with "local."

then, we got this error and noticed the huge, complex function has 2189 lines of CF code. I'm guessing CF server was able to compile the code, but just isn't allowed to run a function larger than 65535 byte.

Has anyone else ever seen this? (i did a short Google session, but didn't turn up anything)

Yes, I know this function needs to be rewritten badly to shorter functions, yadda, yadda... but I couldn't help sharing.

An American CF developer's adventure to Holland (part 2)

Sun, 16 Nov 2008 21:10:00 GMT

So, we made it to The Hague (Den Haag), The Netherlands and I have been loving it, both professionally and personally. Here's our personal blog: http://longnion.blogspot.com.

This is part 2 of my series on moving to, and working in, Holland. Did you miss part 1?

How difficult is it to move to Holland?

In short, quite difficult, but clearly not impossible. What do you need?

a job. I don't know how it would be possible without a job lined up, and coworkers who help a lot
a passport.
patience. The various government agencies make for lot's of red tape! ugh!
guts.
plane ticket.
place to live, preferrably fully-furnished, as many apartments are in Holland.
a willingness to ask a lot of questions.
The phrase, "Spreekt u Engels?", to ask if they speak English... over 70% of Dutch people do!
A GPS device - I liked the TomTom we used for the first month.
Internet - I used Ziggo, but it was quite difficult to set up since their website and phone system is only in Dutch. I had to bribe my coworkers to come over and help out a few times.
I willingness to walk and take public transportation. Between walking, trams, trains, and busses, I see no need for a car. And with all the walking, I already lost probably 15 pounds in a month and a half!
Tolerance of other cultures and of lot's of change. I work with Dutch and Danish people, and one guy from Madagascar. We are constantly learning new things about cultural, language, food, musical, and other differences - and it's open my eyes to worlds I never knew existed.
Warm clothes and walking shoes.

An American CF developer's adventure to Holland (part 1)

Wed, 27 Aug 2008 14:47:00 GMT

I've resigned my current position to move to Holland to work at PeopleXS.com as the Lead CF Developer. So, this is part 1 of the following blog series I plan to write.

How does one decide to completely uproot one's family, home, and job to dare move to a country on another continent?
How difficult is it to move to Holland?
What is it like to live in Holland as an American expat?
How is being a Web Developer different in Holland than in America?

Obviously, parts 2-4 will have to wait until I get to Holland. And if there's interest in the subject, I may write additional related entries.

Part 1:

How did I decide to completely uproot my family, home, and job to dare move to a country on another continent? Not easily!

While reading through blogs (I forget which one), I noticed a blog that described a Lead CF Developer position for PeopleXS.com. BTW - there's still some openings for additional developers. I emailed it to myself to look over later, as I had always been curious about what it would be like to live in Europe (especially a progressive, liberal country like the Netherlands). A week later, I finally got around to emailing my resume in.

PeopleXS was glad I had applied, was open to hiring developers from outside the country, and really piqued my interest. So, the next evening at the dinner table at home, not expecting a good reaction from my wife, I tried to casually mentioned that I had found an "interesting opportunity" in Holland. Without blinking she smiled and said, "let's do it!"

.... (back to earth) ....

I was completely floored by her reaction, but love her all the more for it. I hadn't even told her anything about it really, so obviously we had a lot more "figuring out" to do, but it gave me enough of a boost to forge onward.

Where? The Hague

When? October 1st

Who? me, my wife, and 2-year old son

What? Lead Developer of a small team of CF developers and a QA team that will expand to meet the needs of a fast-growing SaaS company.

How? Fly us to The Hague a few days early to get set up in temporary housing before my first day on the job.

Paid time off? 5 weeks, instead of 2 weeks here at most jobs in the US

Transportation? bikes and public transit (see video below)

How long? indefinitely - as long as I like the country and job

But what about everything I have to leave behind? Family? Job? House? Friends? Material things? Austin, the live music capital of the world, for God's sake?!!

Family and friends - this is obviously the most difficult, and it really hasn't "set in", yet. But I'm sure we'll get very emotional as it gets closer, and we say our last good-byes. And our families took the news much better than expected, and have been very supportive. Not to mention we'll use much of our vacation/holiday time to visit the US as much as we can.
House - in the process of being sold. (cross your fingers)
Job - also pretty scary, since I had a very solid, stable job in a fast growing company. I may look back in a few months and think, "what the hell did I do?!!!"
Austin - yes, it's where I grew up and I love it. But still it's in *Texas*. I don't want to look back at my life, when I'm barely able to lift my own head, and wonder why I spent my entire in one place. I'd like to think I'm a bit adventurous at heart. We'll see if I can handle a real adventure...
Material things - aside from a few personal items, gifts, and family stuff, I'm honestly not really *attached* to much that I own, man. My hippie parents passed this attribute to me, and I guess it's finally paying off. ;-) [er, oh wait - I can't really bare to give up my Wii and Wii Fit, though... and maybe it'll be a way to lure a few Dutch people to be my friends...

After over a month of interviewing, long strings of emails to PeopleXS, mulling it over and over with my wife, and rationalizing that this was "too big of an opportunity to pass up", we took the plunge and I accepted the position. Only patience, positivity (is that the right word?), hard work, and an open mind will keep us on the right track for this crazy ride. Let me know if you think I'm crazy, or if you have any questions?

What does "The Hague/Den Haag" look like? see this video

Experienced CF Developer needed

Tue, 26 Aug 2008 14:12:00 GMT

My job is up for grabs, and it's the best development job I've ever had (and I've had quite a few ;-).

I'm *only* leaving it because I'm moving to Holland (more on that in another post). I have no complaints, but want to make it clear that the job is for people who want to work in a small ".com style" team. The owners are so nice to work for, and it's truly been a pleasure to have the freedom to design the code creatively, use the new "Enterprise" CF 8 features, see my work directly increase revenue, and get things done quickly without corporate red tape and umpteen levels of middle-management. Below is the job description.

"We are looking hard for an experienced ColdFusion Developer opening we have to fill immediately. This would be for the lead programmer for the LandsofAmerica.com Network which is the nation's largest rural real estate website. We have many cutting-edge applications we are working on and implementing with the latest in ColdFusion 8 in exciting and new ways.

This position needs experience in high traffic websites to manage our 16 million page views across 300 websites including a new partner that has the potential to add another 30% to our traffic numbers in the next 3 months. We have started powering the Farm & Ranch sections on the Houston Chronicle and Texas Monthly as well as new Hearst properties soon. High traffic website management as well as web server/database server optimization experience is a must. Work experience with CF8 and SQL Server 2005 is a big plus.

We are a hard working, small team that gets things done quick and can see results fast. It's a lot of fun! We are growing at a phenomenal rate with our number of listings at 63,000 and just two years ago they were at 2,500.

If you are interested please send your resume and any work you've done in the past two years. A detailed description of the exact work done will be very beneficial in moving this process forward ASAP. If you know of anyone this opportunity might sound interesting to, please forward this on to them.

Great pay, full health coverage, and monthly bonuses. "

send resumes to allen@landsofamerica.com

RegEx to find SQL in code without CFQueryParam

Wed, 16 Jul 2008 20:35:00 GMT

I know it's far from perfect (that's partly why I'm posting - so you can show me the light), but here's a RegEx I'm using in Eclipse to find all the places in our code where we didn't use CFQueryParam around variables (shame, shame). Please let me know if you see ways to improve it.

\s+(where|set)\s+[a-z_\.]+\s+(like|\=)\s+'?%?#[a-z"_'\.\(\)\+\/]+#%?'?

And can someone help me figure out how to make this Regex include CF variables passed in to INSERT statements?

TIA

Paginating Records in CF with One SQL Server 2005 call (update 1)

Mon, 09 Jun 2008 16:18:00 GMT

I've been meaning to update folks on the excellent use I've been getting out of this way of paginating records with SQL Server 2005 via CF, and James Allen's comment today gave me an excuse to finally update folks:

Note that I've been using this method in production at www.LandsofAmerica.com - see some example search results, as well as all our sister sites: www.landsofTexas.com, www.landsofCalifornia.com, etc. Our combined network gets upwards of 15 ColdFusion requests per second, and this dynamic stored procedure has held up extremely well, despite the enormous load and wide variety of possible WHERE clauses in the search.
And now to answer James's comment, "I've been using your SP extensively around my new site and it's working great. However, I am now tring to get it to work with a table join and I just can't get it to work with a table or alias prefix (in the columns parameter)." - The "sample DAO CFC method" code below should give you what you need to fix the issue you refer to. :)

First, the Stored Procedure SQL again:

USE [propertyControlCenter] GO /****** Object:  StoredProcedure [dbo].[sproc_pagination]    Script Date: 06/09/2008 11:28:30 ******/ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO -- ==================================================  -- Author:        Aaron Longnion  -- Create date: 10/18/2007  -- Description:    A gereral-use way of dynamically  --        creating SQL to easily paginate the results;  --        also includes a total record count based on  --        the search criteria in the WHERE clause  -- ==================================================  CREATE PROCEDURE [dbo].[sproc_pagination]      -- Add the parameters for the stored procedure here      @SqlColumns VARCHAR(MAX),      @SqlFriendlyColumns VARCHAR(MAX),      @SqlTableClause VARCHAR(MAX),      @StartRow INT,      @EndRow INT,      @SqlWhere VARCHAR(MAX),      @SqlRowNumOrderBy VARCHAR(MAX),      @SqlOuterOrderBy VARCHAR(MAX)  AS  DECLARE @rsSQL NVARCHAR(MAX)  DECLARE @rcSQL NVARCHAR(MAX)  BEGIN      -- SET NOCOUNT ON added to prevent extra result sets from      -- interfering with SELECT statements.      SET NOCOUNT ON;      -- build pagination SQL, using StartRow and EndRow to determine      -- which results to output      SET @rsSQL = N' WITH tempTable AS ( ' +          N' SELECT ' +              @SqlColumns +          N' , ROW_NUMBER() OVER(ORDER BY ' +              @SqlRowNumOrderBy +          N' ) AS RowNumber ' +          N' FROM ' +              @SqlTableClause      IF @SqlWhere + '' <> ''          BEGIN              SET @rsSQL = @rsSQL +                                   N' WHERE ' +                      @SqlWhere          END      SET @rsSQL = @rsSQL +          N' ) SELECT ' +              @SqlFriendlyColumns +          N' FROM tempTable ' +          N' WHERE RowNumber >= ' +              CAST(@StartRow AS NVARCHAR(32)) +          N' AND RowNumber <= ' +              CAST(@EndRow AS NVARCHAR(32)) +          N' ORDER BY ' +              @SqlOuterOrderBy            -- uncomment PRINT to debug      --PRINT @rsSQL      EXEC sp_executesql @rsSQL      -- build second recordset simple for the count      SET @rcSQL =              N'SELECT COUNT(*) AS CountAll FROM ' +                  @SqlTableClause      IF @SqlWhere + '' <> ''          BEGIN              SET @rcSQL = @rcSQL +                                   N' WHERE ' +                      @SqlWhere          END            EXEC sp_executesql  @rcSQL          SET NOCOUNT OFF;  END   PRINT 'sproc_pagination created'

Second, some sample Gateway CFC method code:

 <cffunction name="searchInventory"     output="false"     access="public"     returntype="struct"     hint="Returns total count and recordset based on search criteria">    <cfargument name="SearchCrit"      required="true"      type="struct"      hint="Search criteria, usually passed in from the URL scope" />    <cfset var retStruct = StructNew() />    <cfparam name="Arguments.SearchCrit.SiteStateID" type="string" default="" />   <cfparam name="Arguments.SearchCrit.State_ID" type="string" default="" />   <cfparam name="Arguments.SearchCrit.Region_ID" type="string" default="" />   <cfparam name="Arguments.SearchCrit.County_ID" type="string" default="" />   <cfparam name="Arguments.SearchCrit.Style" type="string" default="" />   <cfparam name="Arguments.SearchCrit.Style_ID" type="string" default="" />   <cfparam name="Arguments.SearchCrit.SqFt" type="string" default="" />   <cfparam name="Arguments.SearchCrit.Price" type="string" default="" />   <cfparam name="Arguments.SearchCrit.AuctionDateStart" type="string" default="#DateFormat(Now(), 'yyyy-mm-dd')#" />   <cfparam name="Arguments.SearchCrit.AuctionDateEnd" type="string" default="" />   <cfparam name="Arguments.SearchCrit.ListingType" type="string" default="" />   <cfparam name="Arguments.SearchCrit.PageNum" type="numeric" default="1" />   <cfparam name="Arguments.SearchCrit.RecordsPerPage" type="numeric" default="15" />   <cfparam name="Arguments.SearchCrit.sortby" type="string" default="sqft" />   <cfparam name="Arguments.SearchCrit.sortbyDirection" type="string" default="DESC" />    <!--- derive start and end rows --->   <cfset Arguments.StartRow = ( (Arguments.SearchCrit.PageNum-1) * Arguments.SearchCrit.RecordsPerPage )+1 />   <cfset Arguments.EndRow = (Arguments.StartRow + Arguments.SearchCrit.RecordsPerPage)-1 />    <cfset retStruct = DAO.searchInventory( ArgumentCollection=Arguments ) />    <cfset retStruct.PagingData = Server.Utilities.Toolbox.getPagingLinks(                RecordsPerPage = Arguments.SearchCrit.RecordsPerPage                , RecordCount = retStruct.RC                , NumberOfLinks = 10                , RecordLimit = 1000000                , QSItemsToStrip = "messageCode,searchIt"                ) />    <cfreturn retStruct />   </cffunction>

Third, a sample DAO CFC method. This is the key: see how the "friendly" (no table aliases) names of the columns are in the "local.SqlFriendlyCols" section. Notice how the column names in the "local.SqlCols" clause contain references to table aliases in "local.SqlTableClause". The only caveat for this, which I haven't found a work-around yet, is that you'll get an "ambiguous column" SQL error if you have a JOIN and try to sort by a column that does not have a unique name :

 <cffunction name="searchInventory" access="package" output="false" returntype="struct">    <cfset Var local = StructNew()>    <!--- columns --->         <cfsavecontent variable="local.SqlCols">             I.listing_city,    ST.State_name,    ST.State_shortCode,    S.listing_type,    I.listing_price,    I.listing_photo,    I.listing_id,    I.listing_state_id,    I.listing_lot_size,    I.listing_county_id,    C.county_name         </cfsavecontent>    <!--- friendly cols (if no JOINs/aliased table names, inherit from SqlCols above) --->         <cfsavecontent variable="local.SqlFriendlyCols">             listing_city,    State_name,    State_shortCode,    listing_type,    listing_price,    listing_photo,    listing_id,    listing_state_id,    listing_lot_size,    listing_county_id,    county_name         </cfsavecontent>    <!--- tables --->   <cfsavecontent variable="local.SqlTableClause">     listing_(nolock) I    INNER JOIN     listing_types(NOLOCK) S     ON S.listing_type_ID = I.listing_type_ID    INNER JOIN     counties(nolock) C     ON C.County_ID = I.listing_County_ID    INNER JOIN     states(NOLOCK) ST     ON ST.State_ID = I.listing_State_ID      </cfsavecontent>    <!--- conditions --->         <cfoutput>             <cfsavecontent variable="local.SqlWhere">     I.listing_Active = 1     AND I.listing_Acct_Active = 1     AND I.listing_Status = 0     <!--- dynamic info --->     <cfif Len(Arguments.SearchCrit.siteStateID)>      AND I.listing_state_id = #Arguments.SearchCrit.siteStateID#     </cfif>     <cfif ListLen(Arguments.SearchCrit.State_ID)>      AND I.listing_state_id IN(#Arguments.SearchCrit.State_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.Region_ID)>      AND C.listing_Region_id IN(#Arguments.SearchCrit.Region_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.County_ID)>      AND I.listing_County_id IN(#Arguments.SearchCrit.County_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.listing_type)>      AND S.listing_type_id IN(#Arguments.SearchCrit.listing_type#)     <cfelseif ListLen(Arguments.SearchCrit.listing_type_ID)>      AND S.listing_type_id IN(#Arguments.SearchCrit.listing_type_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.SqFt)>      AND I.listing_size >= #Val(ListFirst(Arguments.SearchCrit.listing_size))#      AND I.listing_lot_size <= #Val(ListLast(Arguments.SearchCrit.listing_size))#     </cfif>          <cfif ListLen(Arguments.SearchCrit.listing_Price)>      AND I.listing_Price >= #Val(ListFirst(Arguments.SearchCrit.listing_Price))#      AND I.listing_Price <= #Val(ListLast(Arguments.SearchCrit.listing_Price))#     </cfif>             </cfsavecontent>         </cfoutput>    <!--- order --->   <cfoutput>          <cfsavecontent variable="local.SqlRowNumOrderBy">              #Arguments.SearchCrit.sortby# #Arguments.SearchCrit.sortbyDirection#          </cfsavecontent>    <!--- used for order column that has an alias prefix --->          <cfsavecontent variable="local.SqlOuterOrderBy">              #ListLast(local.SqlRowNumOrderBy, ".")#          </cfsavecontent>   </cfoutput>    <!--- dynamically get the data based on the SQL built above --->         <cfset local.returnStruct = Server.DAO.Common.getPaginatedData( SqlCols = local.SqlCols,                   SqlFriendlyCols = local.SqlFriendlyCols,                   SqlTableClause = local.SqlTableClause,                   StartRow = Arguments.StartRow,                   EndRow = Arguments.EndRow,                   SqlWhere = local.SqlWhere,                   SqlRowNumOrderBy = local.SqlRowNumOrderBy,                   SqlOuterOrderBy = local.SqlOuterOrderBy ) />    <cfreturn local.returnStruct />   </cffunction>

And finally, the getPaginatedData() method:

 <cffunction name="getPaginatedData" access="public" output="false" returntype="struct" hint="generic way to get pagination data from the dB">    <cfargument name="SqlCols"      type="string"      required="true"      hint="list of table columns ('col1, col2, col3', 'a.col1, b.col2, c.col3', or 'a.col1 AS acol, b.col2 AS bcol, (SELECT c.col3 FROM c_table) AS subCol3' are all acceptable" />    <cfargument name="SqlFriendlyCols"      type="string"      required="true"      hint="list of table columns ('col1, col2, col3' is acceptable" />    <cfargument name="SqlTableClause"      type="string"      required="true"      hint="list of table columns (most JOINs and sub-SELECTs are acceptable" />    <cfargument name="StartRow"      type="numeric"      required="true"      hint="beginning row number to display" />   <cfargument name="EndRow"      type="string"      required="true"      hint="last row number to display " />    <cfargument name="SqlWhere"      type="string"      required="true"      hint="WHERE clause filters" />    <cfargument name="SqlRowNumOrderBy"      type="string"      required="true"      hint="usually the PK of the primary table" />    <cfargument name="SqlOuterOrderBy"      type="string"      required="true"      hint="chose from the columns from SqlFriendlyCols" />    <cfset var returnStruct = StructNew() />    <cfstoredproc procedure="sproc_pagination" datasource="#Variables.DSNs.primary#">     <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlColumns" value="#Trim(Arguments.SqlCols)#">    <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlFriendlyColumns" value="#Trim(Arguments.SqlFriendlyCols)#">    <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlTableClause" value="#Trim(Arguments.SqlTableClause)#">    <cfprocparam type="In" cfsqltype="CF_SQL_INTEGER" dbvarname="SqlStartRow" value="#Arguments.StartRow#">    <cfprocparam type="In" cfsqltype="CF_SQL_INTEGER" dbvarname="EndRow" value="#Arguments.EndRow#">    <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlWhere" value="#Trim(Arguments.SqlWhere)#">    <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlRowNumOrderBy" value="#Trim(Arguments.SqlRowNumOrderBy)#">    <cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR" dbvarname="SqlOuterOrderBy" value="#Trim(Arguments.SqlOuterOrderBy)#">    <!--- return data + count --->    <cfprocresult name="rs" resultset="1">    <cfprocresult name="recordcount" resultset="2">          </cfstoredproc>    <cfset returnStruct.rs = rs />         <cfset returnStruct.rc = recordcount.countAll />    <cfreturn returnStruct />   </cffunction>

Let me know what you think.

 <cffunction name="searchInventory" access="package" output="false" returntype="struct">    <cfset Var local = StructNew()>    <!--- columns --->         <cfsavecontent variable="local.SqlCols">             I.listing_city,    ST.State_name,    ST.State_shortCode,    S.listing_type,    I.listing_price,    I.listing_photo,    I.listing_id,    I.listing_state_id,    I.listing_lot_size,    I.listing_county_id,    C.county_name         </cfsavecontent>    <!--- friendly cols (if no JOINs/aliased table names, inherit from SqlCols above) --->         <cfsavecontent variable="local.SqlFriendlyCols">             listing_city,    State_name,    State_shortCode,    listing_type,    listing_price,    listing_photo,    listing_id,    listing_state_id,    listing_lot_size,    listing_county_id,    county_name         </cfsavecontent>    <!--- tables --->   <cfsavecontent variable="local.SqlTableClause">     listing_(nolock) I    INNER JOIN     listing_types(NOLOCK) S     ON S.listing_type_ID = I.listing_type_ID    INNER JOIN     counties(nolock) C     ON C.County_ID = I.listing_County_ID    INNER JOIN     states(NOLOCK) ST     ON ST.State_ID = I.listing_State_ID      </cfsavecontent>    <!--- conditions --->         <cfoutput>             <cfsavecontent variable="local.SqlWhere">     I.listing_Active = 1     AND I.listing_Acct_Active = 1     AND I.listing_Status = 0     <!--- dynamic info --->     <cfif Len(Arguments.SearchCrit.siteStateID)>      AND I.listing_state_id = #Arguments.SearchCrit.siteStateID#     </cfif>     <cfif ListLen(Arguments.SearchCrit.State_ID)>      AND I.listing_state_id IN(#Arguments.SearchCrit.State_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.Region_ID)>      AND C.listing_Region_id IN(#Arguments.SearchCrit.Region_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.County_ID)>      AND I.listing_County_id IN(#Arguments.SearchCrit.County_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.listing_type)>      AND S.listing_type_id IN(#Arguments.SearchCrit.listing_type#)     <cfelseif ListLen(Arguments.SearchCrit.listing_type_ID)>      AND S.listing_type_id IN(#Arguments.SearchCrit.listing_type_ID#)     </cfif>     <cfif ListLen(Arguments.SearchCrit.SqFt)>      AND I.listing_size >= #Val(ListFirst(Arguments.SearchCrit.listing_size))#      AND I.listing_lot_size <= #Val(ListLast(Arguments.SearchCrit.listing_size))#     </cfif>          <cfif ListLen(Arguments.SearchCrit.listing_Price)>      AND I.listing_Price >= #Val(ListFirst(Arguments.SearchCrit.listing_Price))#      AND I.listing_Price <= #Val(ListLast(Arguments.SearchCrit.listing_Price))#     </cfif>             </cfsavecontent>         </cfoutput>    <!--- order --->   <cfoutput>          <cfsavecontent variable="local.SqlRowNumOrderBy">              #Arguments.SearchCrit.sortby# #Arguments.SearchCrit.sortbyDirection#          </cfsavecontent>    <!--- used for order column that has an alias prefix --->          <cfsavecontent variable="local.SqlOuterOrderBy">              #ListLast(local.SqlRowNumOrderBy, ".")#          </cfsavecontent>   </cfoutput>    <!--- dynamically get the data based on the SQL built above --->         <cfset local.returnStruct = Server.DAO.Common.getPaginatedData( SqlCols = local.SqlCols,                   SqlFriendlyCols = local.SqlFriendlyCols,                   SqlTableClause = local.SqlTableClause,                   StartRow = Arguments.StartRow,                   EndRow = Arguments.EndRow,                   SqlWhere = local.SqlWhere,                   SqlRowNumOrderBy = local.SqlRowNumOrderBy,                   SqlOuterOrderBy = local.SqlOuterOrderBy ) />    <cfreturn local.returnStruct />   </cffunction>

Does your email validation allow plus signs?

Wed, 30 Apr 2008 18:21:00 GMT

Go ahead and check. I'll wait.

I'm not kidding, go ahead. It'll be alright. You're not the only one whose says made_up_account_here+do_not_spam_me@gmail.com is an invalid address.

"er, are you sure a plus sign is valid?", you ask? Yep. I'm certain. All the time, I use my gmail accounts nifty feature that allows me to use, say, made_up_account_here+twitter@gmail.com to filter emails from twitter to a twitter "label". But some sites, especially blog sites, always tell me my email is invalid when I put in the plus sign. Well, your site is wrong, and I'd appreciate it if you'd fix it. ;-) (when you get time of course)

anyways, after you checked, what did you find?

Bad Day at the Office video

Tue, 15 Jan 2008 19:35:00 GMT

I usually don't post stuff like this, but since I had a good laugh (cuz can't we all relate?) and wanted to try the embed feature on the new InstantSpot 2.0 software.

glumbert - Bad Day at the Office