Jul 18 2008

What Does Effective Load Testing Cost?

Posted by Mike Brunt at 7:41 AM
2 comments
- Categories: Default | ColdFusion | JRun-J2EE

We have three current clients all of who have Web 2.0 elements in their web applications, two are Flex based and the other is Ajax based.  These clients brought us in to perform load-testing as a precursor to capacity planning, as their applications scale.  We encountered some interesting behavioral characteristics relating to Web 2.0 applications and I will be blogging on what we found in greater detail, soon. As a comment, this sort of testing is crucial for all web applications whether Web 2.0 centric or not.  A full suite of testing including a very comprehensive report and delivery of load test scripts for ongoing use carries a cost of around $6,000.00, depending on he amount of on-site work required.  Our recommendation is to limit the on-site element as this will add to the cost and in reality does not produce significantly better results.


Hourly rates for good developers are typically on the rise and it does not take very long to burn $6,000.00 and much more when re factoring of inefficient design or code becomes necessary after an application has been deployed.  Load testing is a crucial step in the evolution of all web applications that are designed to be used by others.

Comments

Dan Wilson

Dan Wilson wrote on 07/18/08 10:18 AM

Well said.

6k is a minimal expense, especially considering the total cost of building a custom web application.

If context is taken in to account, an eCommerce site experiencing failure under load could result in an opportunity cost of much greater than 6k. Other applications may have different calculations to work through, lost productivity, additional troubleshooting by IT staff, general friction on the process... but I'd wager the costs would exceed 6k as well.

As the space matures, organizations are taking this more seriously and I think the general awareness of the benefits are increasing as well.

The fact is, this isn't 1999 and we aren't in the heyday of a nascent industry where speed to market is king. In this mature age of web software, applications have to meet a higher standard.

DW
Mike Brunt

Mike Brunt wrote on 07/18/08 3:04 PM

@Dan you make very good points and often what stops companies from load testing is the perceived cost or real cost. For instance a tool such as Mercury Load Runner can easily top $50,000 with meaningful licenses for vUsers. Then there is the learning curve adding potentially additional costs. We have been involved in load-testing web applications since 1999 so have figured out the more cost-effective tools and methodologies.

Write your comment



(it will not be displayed)