0

Top 10 Architecture Scalability Mistakes Made in ColdFusion

ColdFusion, Software, MySQL

This is a focus on architecture design mistakes I've seen made too often over the years by CF developers building ColdFusion architectures. Often times, the codebase and system started small and grew too fast, such that the original developer/architect didn't have the time or experience to scale the system up with the demands of high-load applications.

  1. Not dedicating enough time and resources on optimizing the database:

    Too often, there are smart, but unqualified programmers/developers designing and maintaining the database.  A qualified/certified DBA, even part-time, is well worth the investment. You need one who can profile your database, tune all the server and individual database settings, recomming index changes, be responsible for DB design and integrity, regular maintenance, set up replication/failover/clustering, and will proactively monitor your database to let you know what problems need attention in relationship to your applications.
  2. Not dedicating enough time and resources on optimizing queries:

    Again, developers tend to throw themselves into the role of writing all the queries, but don't necessarily think about the performance/scalability issues of writing non-optimal queries.  Often times a DBA who is qualified for all the Database Server stuff (see #1 above) isn't the same type of DBA who can optimize your queries.  You need someone who can run the queries with explain plans, and then can make recommendations on improvements.  These improvements pay off immensely!
  3. Coupling all parts of the system into one mega-application, instead of separate applications

    Usually they start out as fine: clever little applications that "do the job". They may have started in the CF 4 or 5 days, and grow and grow over the years until they're a mass of spaghetti code.  It may never occur to the developers that as the app gets bigger that not everything belongs together any more.  It will take some work, but it's important for long-term stability and scalability that you de-couple big chunks into smaller applications that run independent from the original beast.  Scheduled Tasks, Web Services, APIs, User Admin Areas, Reporting, file/image services, Search services, etc. should no longer be coupled to the main application.  Each of those parts should be broken out, so that each can be tuned optimally.  Do you want your whole system to start bottlenecking because Search or Web Services become slow or go down?  If they're decoupled, you don't have that problem.
  4. Not using a good "shared" file system scalable for high numbers of file writes

    When there are multiple physical servers in your cluster (if applicable), you may want a single location to save files.  No-no's here are mapped drives/mounts to a non-dedicated server (such as a drive on your database server, some external drive, etc.).  A good NAS and/or SAN solution with failover scales better as file writes become higher over time.  Cloud Computing solutions are becoming big now, too, but I haven't tested enough in this area to make sound recommendations. I'm sure other with more experience can tell us.  Comments?
  5. Not tuning the JVM

    Many articles have already been written on this subject, so I won't go into it.  If you're having performance problems and you don't know how to tune your JVM, get help from one of the many articles out there, hire an expert, or get Adobe Platinum support for CF.
  6. Not tuning Request settings in the ColdFusion Administrator

    See #5 above.
  7. Not making good use of Multiple Instances

    When you de-couple areas of your application (see #3), often it's wise to put your new applications into separate CF instances.  For instance, all heavy tasks (Scheduled Tasks and other "batch" type processes) should run on a separate instance.  You don't want to know how many times I've seen a Scheduled Task bring the entire infrastructure to it's knees.
  8. Not using CF Monitoring tools, such as SeeFusion, CF8 Server Monitor, and/or FusiontReactor

    Get familiar with at least 2 of these, and hire an expert to help you get up to speed on them more quickly.  These save your butt!
  9. Trying to recreate the "holy grail" reporting system from scratch

    First, any reporting system should be a prime candidate for de-coupling.  And reports that come from the database should not be running against your "write" (master) database, but rather against your "read-only" (slave) database, or else you'll have a lot of bottlenecks on your hands as users create massive reports reading data while other users are trying to simultaneously write data via the rest of the application.

    But secondly, look for reporting solutions that are already available.  Most of the time users don't really need *real-time* reports, and off-line/archived data lends itself better to the more complex reports that users start to request over time (allowing Excel exports makes a lot of people happy, cuz they know how to generate reports "the way they like them" in familiar Excel.  Also, don't write a Report UI that allows users to request *too many* records - your milage will vary, but it's almost never a good idea to allow reports that have no filters, such that they can just *get everything* all at once.  Few web apps are built for handling that much data, so find other solutions.
  10. Not taking advantage of various caching capabilities:

    We all know (hopefully) to cache queries where possible, but... do you use the Application/Server scope where appropriate, and to use cfcache (or other 3rd party CF cache tags/CFCs/frameworks) in areas that rarely change?  Do you come up with caching strategies for data, so that it can persist in the Application/Server scope without going back to the dB unnecessarily?  Could other data/files be cached to disk?  Can you use Verity (or Lucene/Solr) for searches instead of pulling directly from the database all the time?
Craig Kaminsky said:
 
Very nice post, Aaron. Some excellent tidbits in here and the post is especially well-timed for me, since I'm in charge of my first true CF enterprise application!
 
posted 68 days ago
Add Comment Reply to: this comment OR this thread
 
cfZen said:
 
@Craig - thanks! I am humbled... I posted it back in August, but nobody seemed to actually read it the first time :\

I'm interested in any feedback or questions you might have on these types of issues. And do you have more items to add to this list?
 
posted 68 days ago
Add Comment Reply to: this comment OR this thread
 
Craig Kaminsky said:
 
I'm actually keeping a running list of items I think my team and I need to address and have already added six of your ten to that list. As we move along, I promise to post feedback :)! Right now, the one I'm most concerned about, for our application, is locating pieces of the codebase to cache (be it a view or query or other).

Thanks, again! And, I'm really glad I found this post. I normally just read posts from your Posterous blog but am stoked to have stumbled across this one today!
 
posted 67 days ago
Add Comment Reply to: this comment OR this thread
 
Shaun McCran said:
 
Hi,
An interesting article, I especially liked the points about caching. Coldfusion has some very powerful caching frameworks/Tools and it is always suprising to see them go to waste.

I am always amazed by the performance increase when you shift out your SQL to stored procs, and tune your CFC's to be memory based.

I totally agree with the "One application to rule them all" point. It seems to be common practive to produce massive sprawling apps that soon spiral out of control. To few times do people look at where the application should actually exist before starting a project.
 
posted 68 days ago
Add Comment Reply to: this comment OR this thread
 
Shawn Holmes said:
 
Great list. I would also like to contribute:

"Inappropriately choosing OO development over Non-OO for the application at hand".

It seems like a constant issue in the CF community. One group is always pushing hard to use a full OO model for their app implementation. Another group always seems hell-bent on pushing back, citing OO as too complex or too performance-inhibiting to be of any value.

The answer is (and always should be) take a step back and look at what you are trying to build, as a whole. Is it a simple one or two template application that builds an RSS feed? You probably don't need a full-blown OO model for that. What about an enterprise level site involving hundreds of domain objects that are closely related and working together to adhere to the business rules of a particular company? Well, perhaps an OO model is more applicable in this case.

Know the difference, and use what's appropriate in each case.
 
posted 67 days ago
Add Comment Reply to: this comment OR this thread
 
Henry said:
 
separate applications? Then what technique do you use to do communication between the applications?

event driven? share database and producer write to the DB, and consumer read the DB every n seconds?

Thx
 
posted 67 days ago
Add Comment Reply to: this comment OR this thread
 
cfZen said:
 
@Henry - there are lot's of ways to skin that cat, and the appropriate design depends on the situation. So, could you give some specific examples?

In some cases, the database is all you need; file system could be appropriate; thinking of your separate apps as "Services" (Service Oriented Architecture) and using REST or SOAP based Web Services may come into play; and then you may want your apps to interact via AJAX using XML or JSON in other situations.

The main point is that, as your mega-app gets larger and more complex, for scalability you want to decouple to some degree so that you can fine-tune scalability and performance for each "Service" in your architecture. I hope that makes sense.
 
posted 67 days ago
Add Comment Reply to: this comment OR this thread
 

Search

Aaron  Longnion

The Hague, NL