Nov 7 2009

Is The Database A Poor Step-Child?

Posted by Mike Brunt at 9:57 AM
21 comments
- Categories: Web Servers | .NET | DataBase | CloudComputing | ColdFusion | JRun-J2EE

Databases are so important to us, in many ways, they are the number one dependency we deal with in ColdFusion and yet, in many ways,  we treat them as ignored step-children and I fear the advent of Hibernate ORM in CF 9 will make matters even worse.  If client variables are in use, this becomes even more critical, tuning the database will always certainly yield more benefits than any amount of JVM tuning.  By tuning I mean looking at indexing, affinities, CPU allocation, hard-drive layout, etc.  All of theses things can yield immediate benefits or cause immediate pain if they are not right.

Typically, when we get new clients we try to persuade them to spend some time on our database review and report offering but most of the time they decline, wanting to focus all time/spend on the CF-web server, eventually we are typically able to persuade them to let us look at the database.  They would genuinely benefit greatly if we started out at the database.  Here are the main things we look at and report on...

  • Table Indexing
  • Back-up Redundancy Schema
  • Security Settings - Ports etc
  • Security Settings - User Accounts
  • Use of Views
  • Use of Stored Procedures
  • CPU Affinity
  • CPU Allocated to Database
  • Service Pack Level
  • Data File Location
  • Log Files Location
  • Compatibility Level (SQL Server Specific)
  • Heavy/bad processes such as Auto_Shrink
  • Data Type usage (looking for the most efficient data types)
  • Unnecessary Services
  • Long Running SQL

 

 

 

Comments

Bucky Schwarz

Bucky Schwarz wrote on 11/07/09 11:39 AM

I agree completely. Hell, even before you get to the tuning of the actual database, how much better would so many apps run if better SQL was written. It seems like instead of adopting the mindset that a good CF developer should write good CF as well as good SQL, we instead go in the opposite direction and say "don't worry about the SQL, ColdFusion 9 will handle that for you." I think the result is going to be the database becoming a sort of 'black box' where magic just kind of happens for you. What happens when the magic doesn't work? You end up digging around in the database and figuring how that works, THEN figuring out how to make the database work with Hibernate. What happens when you have to start writing complex queries with HQL? At that point, is it even worth it to use Hibernate?

But back to your original point, I think the database is absolutely the red-headed stepchild, and I'm not really sure why. The whole point of web apps is managing data and presenting it in a useful and meaningful way. It's kind of what web developers do: manage data and give it to the layman in a way that makes sense, so he doesn't have to go digging around in a database. A lot of people just hate dealing with databases, which should make you happy since I guess it gives you more business opportunities ;)
Mike Brunt

Mike Brunt wrote on 11/07/09 11:59 AM

@Bucky, thanks for taking the time to add just a great comment, you expressed it far better than I did. Sometimes I have people ask me why I spend so much time focused on the database as they thought I was JVM specialist, well I hope I am a performance specialist and of course as all data passes through ColdFusion, guess what takes the blame? I have two very good DBA colleagues one in the SQL Server world the other in Oracle and they do not like ORM centric work.
TJ Downes

TJ Downes wrote on 11/07/09 5:35 PM

I'd venture to say that the relational database is nearing the end of its life for many types of application development. OO development has made object storage more feasible and common, and applications will be better for it, in my opinion :)
Mike Brunt

Mike Brunt wrote on 11/08/09 10:13 AM

@TJ thanks for your comment and no doubt there are other mechanisms afoot, CouchDB for instance. From experience with clients, I still see mostly RDBM's and I think it will be some time, if ever they go away completely. What I do see happening relatively quickly is the adoption of caching mechanisms, such as memcached, to move data out of its slowest responding place; the hard drives.
tiffany jewelry

tiffany jewelry wrote on 11/09/09 12:20 AM

I'd venture to say that the relational database is nearing the end of its life for many types of application development.
mkv converter for mac

mkv converter for mac wrote on 11/09/09 4:57 AM

I was just now searching for about this when I came by your post
Mike Kaplan

Mike Kaplan wrote on 11/09/09 6:14 AM

Don't forget about defragging your indexes. We have a weekly job that checks all indexes and defrags those that surpass a certain level of logical scan fragmentation. DBCC SHOWCONTIG and DBCC INDEXDEFRAG are powerful weapons in making sure your indexes keep doing what they're supposed to be doing, particularly in a system with a lot of data being added and edited.
Peter Bell

Peter Bell wrote on 11/09/09 6:16 AM

I think this is one of those cases where people try to generalize rules that are applicable to specific situations. The purpose of an ORM is to speed development of applications. The purpose of database tuning is to optimize performance of applications.

The vast majority of apps written never get heavy load - of course Mike, you probably don't get called in to do performance consulting on the many apps which don't need to perform! For some apps cost/effort/time to market is less important than the ability to immediately handle high loads (especially when replacing legacy systems - green field apps seldom need to handle huge loads on day 1 unless there's a lot of marketing or pent up demand to drive traffic).

For a lot of apps an ORM allows you to get to market quicker, then if you have a performance problem, load testing will identify bottlenecks (usually in the DB) which you can remove one at a time until the app performs acceptably for the current load - sometimes through DB only changes, sometimes through replacing specific hibernate queries with HQL or even with direct SQL through JDBC (although that raises a host of other issues), and sometimes through better caching strategies in the app level.

I think it's important to realize that the DB is often the source of performance issues, but also to understand that there are plenty of apps where cost and time to market are more important than performance. If there weren't, we'd write all of our apps in Java - not ColdFusion, Groovy or Ruby which are nowhere near as performant for many operations.
Mike Brunt

Mike Brunt wrote on 11/09/09 2:55 PM

@ Mike, thanks for the pointers and tips I am sure they will help many.

@ Peter, thank you for your comments and insights. You are right of course that most of the time we are called in for large applications where performance has become an issue. My view on this is that there are cost-effective ways to engineer and test applications before they are released, irrespective of size and it is important to do so. Mainly because small applications can grow big rather quickly if they become compelling enough. Load testing should be part of all application deployments in my opinion.
high quality cellphone bargain

high quality cellphone bargain wrote on 11/10/09 3:56 AM

I like the topic. Great article and well done. This is a helpful websites.


<a href="http://www.superelecstore.com/computer-part-c-84.html/">useful and helpful PC parts</a>
<a href="http://www.superelecstore.com/flash-mediamemory-c-58.html/">flash memories bargain
</a>
<a href="http://www.superelecstore.com/digital-products-mp4-players-c-150_152.html/">mp4 player seller</a>
<a href="http://www.superelecstore.com/digital-photo-frames-c-57.html/">China digital photo frames</a>
<a href="http://www.superelecstore.com/ipod-accessories-c-62.html/">china ipod accessory</a>
<a href="http://www.superelecstore.com/cellphone-accessories-c-80.html/">cellphone accessory china</a>
<a href="http://www.superelecstore.com/games-c-140.html/">games accessories buy</a>
<a href="http://www.superelecstore.com/batteries-c-155.html/">PDA and mobile battery</a>
<a href="http://www.superelecstore.com/toy-c-160.html/">toys for boy</a>
<a href="http://www.superelecstore.com/light-c-161.html/">Flashlights Headlights Bikelights</a>
<a href="http://www.superelecstore.com/cell-phone-c-165.html/">Touch Screen Hiphone</a>
<a href="http://www.superelecstore.com/cell-phone-c-165.html/">cell phone watch style</a>
<a href="http://www.superelecstore.com/cell-phone-c-165.html/">high quality cellphone bargain</a>
<a href="http://www.superelecstore.com/digital-products-mp3-players-c-150_151.html/">cheap new mp3 player</a>
<a href="http://www.superelecstore.com/games-c-140.html/">buy psp ps2 ps3 accessories</a>
Plamen Ivanov

Plamen Ivanov wrote on 11/10/09 9:57 AM

Interesting..
Replica Watches

Replica Watches wrote on 11/10/09 9:52 PM

Our website have many goods,<A href="http://www.newstylerolex.com/"; title="Cheap Brand Name Watches">Rolex</A> -
<A href="http://www.newstylerolex.com/"; title="Replica Rolex">Replica Rolex</A> -
<A href="http://www.newstylerolex.com/"; title="Brand Name Watches">Replica Watches</A> -
<A href="http://www.newstylerolex.com/"; title="Brand Name Watches">Rolex Watches</A> -
<A href="http://www.sunglassvip.net"; title="Welcome to our Replica Watches website">Replica rolex</A> -
<A href="http://www.sunglassvip.net"; title="high quality timepieces">Rolex</A> -
<A href="http://www.2g-3g.com"; title="high quality timepieces">Rolex</A>
<A href="http://www.watches-life.com"; title="cheap price">Rolex</A>
<A href="http://www.watches-life.com"; title="cheap price">Replica Rolex</A>
<A href="http://www.rolex-hot.com"; title="cheaper price">Rolex</A>
<A href="http://www.rolex-hot.com"; title="cheaper price">Replica Rolex</A>
<A href="http://www.vertuexclusiveshop.com"; title="cheaper vertu">Vertu</A>
<A href="http://www.vertuexclusiveshop.com"; title="cheaper Replica vertu">Vertu Replica</A>,
<A href="http://www.e-lv.net"; title="Replica handbags">Replica handbags</A>,
<A href="http://www.enjoy-watches.com/"; title="high quality watches">Rolex</A>,
<A href="http://www.enjoy-watches.com/"; title="Brand Name Watches">Replica Watches</A>,
<A href="http://www.iphonebatteries.org/"; title="Batteries,Phone Batteries">Phone Batteries</A>,
<A href="http://www.lrolex.com/">Rolex</A>;,
<A href="http://www.newstylerolex.com/">Omega</A>,I belive that you will like them !050
new york hostels

new york hostels wrote on 11/11/09 10:30 AM

Databases may be replaced by other applications but I think they're the very basis and sometimes, somehow, we end up researching it.
Greg
service manual

service manual wrote on 11/11/09 5:48 PM

Hi, I just came to your blog today, I read a few of your other posts and really liked it.. will be back to check it out some more

http://manualfile.com
berita riau

berita riau wrote on 11/12/09 1:59 AM

great post sir,really interesting
girl ugg

girl ugg wrote on 11/12/09 7:19 AM

2)   You know that sort of completely exhausted screwed that only twenty year-olds get? The kind of screwed that leaves you limp useless and light headed for hours?
women ugg boots

women ugg boots wrote on 11/12/09 7:20 AM

The sort of screwed Cosomo magazine is always on about to middle aged women who have no chance of experiencing that sort of thing ever again?
replhandbags.com

replhandbags.com wrote on 11/17/09 1:52 AM

<p><a target="_blank" target="_blank" href="http://www.replhandbags.com">louis vuitton</a> Marc Jacobs in the end how to become so popular it? With his design in order? <a target="_blank" href="http://www.replhandbags.com">louis vuitton replica </a>The last several quarters, he tells us that clothes are not exactly.<a target="_blank" href="http://www.replhandbags.com">louis vuitton handbags</a> Now, it is certain that he will once again fade underwear on his fame has a direct relevance to the impact! After the lose of fat <a target="_blank" href="http://www.replhandbags.com">louis vuitton on sale</a> his success, Marc Jacobs is extremely<a target="_blank" href="http://www.replhandbags.com">; replica louis vuitton </a>enthusiastic about showing their bodies, and even worship of his own body.<br>

<a target="_blank" href="http://www.replhandbags.com/21-monogram-denim"; title="View of our online outlet store ,the Louis Vuitton replica Monogram Denim collection. it&rsquo;s made of Monogram Denim Canvas and Oxidizing Cowhide Leather Trimmings, combined all the symbol elements from Louis Vuitton, such as monogram pattern, classic denim canvas, honey colored leather trimmings. A must have collection for Louis Vuitton lovers.at other catagory&#039;s handbags,walllets,clutches,purses,with high qualty satisfied your too!">Louis Vuitton Monogram Denim</a> |
<a target="_blank" href="http://www.replhandbags.com/22-monogram-denim-cruise"; title="Louis Vuitton Monogram Denim Cruise Collection,a fabulous category in our online shop,you will find the high quality denim cruise handbags bags wallets purses and clutches.denim cruise bags features practical ecnomic and fashion,at there!">Louis Vuitton Monogram Denim Cruise</a> |
<a target="_blank" href="http://www.replhandbags.com/24-monogram-dentelle"; title="Look through our onlin shop Louis Vuitton or Lv Replica Monogram Dentelle Collection, A symbol of LV handbags, and Louis Vuitton Purses, classic design with this Monogram canvas, sometimes even a badge of fashion femininity and denotes a social class as well. At other category your can find the fabulous handbags,wallets,pourses and clutches too,access for your enjoy life!">Louis Vuitton Monogram Dentelle</a> |
<a target="_blank" href="http://www.replhandbags.com/28-monogram-mini-lin"; title="Louis Vuitton Monogram Mini Lin collection in our oneline shop,a shiny point of Louis Vuitton,because of Monogram Mini Lin handbags,bags,wallets,purses and clutches show a slight or relax feeling to people,working under the high press environment,while your after one work,accompanied with a Monogram Mini Lin handabag,bag,wallet,purse or clutches,you will feeling at easy.">Louis Vuitton Monogram Mini Lin </a> |
<a target="_blank" href="http://www.replhandbags.com/29-monogram-mirage"; title="Louis Vuitton Monogram Mirage Collection a very impotant component of louis vuitton,in our online shop.The mirage line has done impossibly by turning the boring monogram canvas into a true art! Monogram Mirage collection every product stood for its philosophy, inspired by bright and active art.">Louis Vuitton Monogram Mirage</a> | <a target="_blank" href="http://www.replhandbags.com/30-monogram-miroir"; title="Louis Vuitton Monogram Miroir collection exhibite in our online shop,the designer style and pratical handbags provided at a affordable price,the qulity can be guaranteed.the Monogram Miroir handbags make you charming while you take it under the sunshine.">Louis Vuitton Monogram Miroir</a> |
<a target="_blank" href="http://www.replhandbags.com/31-monogram-multicolor"; title="Monogram Multicolor collection be inspired the practical and graceful design, durable and lightweight. Look through Monogram Multicolor collection in our online shop,the replica Louis Vuitton collection of Monogram Multicolor, the varies style and colorfull. Monogram Multicolor collection new arts element combine the monogram.weleome to our online shop chose your favorite replica louis vuitton handbags,bags,wallets,purses and clutches.">Louis Vuitton Monogram Multicolor</a> |
<a target="_blank" href="http://www.replhandbags.com/32-monogram-perforation"; title="Louis vuitton replica Monogram Perforation collection in our online shop,the monogram perforation handbags,bags,wallets,purses and clutches,features it&rsquo;s perforation ,practial and designer style. Monogram Perforation collection members is the streem of fashion.">Louis Vuitton Monogram Perforation</a> |
<a target="_blank" href="http://www.replhandbags.com/34-monogram-rivets"; title="Louis Vuitton Monogram Rivets, with a kind of superiority of native simplicity. Look at the Replica Louis Vuitton Purses, While the important thing is to feel good about fashion pace of morden society , It is still not a bad idea to get more Replica products, Replica Hermes and Replica Bags, you can not helping to praise them while you get it.">Louis Vuitton Monogram Rivets</a> | <a target="_blank" href="http://www.replhandbags.com/35-monogram-rubis"; title="Louis Vuitton Monogram Shearling collection.">Louis Vuitton Monogram Rubis</a>
wedding desses

wedding desses wrote on 11/18/09 7:33 PM

For many women, a <a href="http://www.weddingnova.com/"><strong>wedding dresses</strong></a> will be the most elaborate and expensive thing they will ever wear -- the crowning glory of a day that will pass in a blur of adrenaline and joy.
And then what?
I have dutifully carted my silk Priscilla of <a href="http://www.weddingnova.com/aboutwedding/Attach-Gold-In-Face-Wedding.html">Attach Gold In Face Wedding</a>Boston gown through six moves and 18 years <a href="http://www.weddingnova.com/aboutwedding/Wedding-Photography.html">Wedding Photography</a> of marriage. I am not <a href="http://www.weddingnova.com/aboutwedding/Wise-Wedding-Location.html">Wise Wedding Location</a> alone; closets across America are <a href="http://www.weddingnova.com/aboutwedding/Wedding-Music.html">Wedding Music</a> hiding fancy gowns of silk, tulle and French lace amassed in the 30 years or so since the wedding industry became a juggernaut.
Now, the combined forces of the Web and the recession <a href="http://www.weddingnova.com/aboutwedding/Enjoyment-Wedding-Sunflower-Paradise.html">Enjoyment Wedding Sunflower Paradise</a> are bringing many of those dresses out again. As online venues for <a href="http://www.weddingnova.com/"><strong>selling dresses proliferate</strong></a>, wedding planners and other specialists say that they're seeing more brides selling their dresses.
View Full Image
Credit: Amanda Inscore for The Wall Street Journal
Kate Bulow bought a used <a href="http://www.weddingnova.com/aboutwedding/Disney-Wedding-Dress.html">Disney Wedding Dress</a> for $450 from a woman whose wedding had been called off. The style had cost $1,200 <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html">Wedding Dresses</a> at one bridal shop.
Josie Daga's Janell<a href="http://www.weddingnova.com/aboutwedding/Dove-Chocolate-Wedding.html">Dove Chocolate Wedding</a> Berté gown, for which she paid $3,000 in <a href="http://www.weddingnova.com/category/Prom-Dresses-812-1.html"><strong>Prom Dresses</strong></a> 1999, arrived home after the wedding, cleaned and preserved, in a three-foot-by-four-foot box. ;You can't fit it under the bed or in a closet, and I thought the reality of my daughter ever wanting to wear my dress is pretty slim,; says Ms. Daga, who is 38.
She listed the dress in 2004 on Craigslist, where she <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html">wedding gowns</a>received a few $200 bids and a lot of offers that she suspected were scams. <a href="http://www.weddingnova.com/category/Flower-Girl-Dresses-804-1.html">Flower Girl Dresses</a> On EBay, ditto. In consignment shops, nada. So she did what an enterprising young advertising executive does -- she <a href="http://www.weddingnova.com/category/Bridesmaid-Dresses-802-1.html">Bridesmaid Dresses</a> launched a Web site http://www.weddingnova.com/, and sold her dress for $800. A year <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html"><strong>Wedding Dresses</strong></a> later, she had 400 listings, with each seller paying a minimum $25 fee, and soon afterward, running the Web site became her full-time occupation. Today, PreOwnedWeddingDresses.com has about 2,360 gowns listed.
Boosting its business these days are brides on a budget. There are 2.2 million weddings each year in the U.S., according to Brides Magazine. While their cost on average in recessionary <a href="http://www.weddingnova.com/category/Bridesmaid-Dresses-802-1.html">Bridesmaid Dresses</a> 2008 was a still-high $22,000, that was down roughly 20% from 2007. Disc <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html"><strong>Wedding Dresses</strong></a> jockeys, photographers <a href="http://www.weddingnova.com/">wedding dresses</a> and videographers are being hit hard as brides opt for iPods and friends with cameras.
Emotionally, it can be hard to downsize a wedding-dress dream. Some women justify the purchase of an expensive <a href="http://www.weddingnova.com/category/Wedding-Accessories-814-1.html">Wedding Accessories</a> gown by planning to resell it. Sally Lorensen Conant, owner of Orange <a href="http://www.weddingnova.com/category/Prom-Dresses-812-1.html"><strong>Prom Dresses</strong></a> Restoration Labs in Orange, Conn., says fewer brides want to have their gowns preserved these days. I'm seeing more women who say, 'I <a href="http://www.weddingnova.com/category/Bridesmaid-Dresses-802-1.html">Bridesmaid Dresses</a> just want you to clean the dress. I'm going to try to sell it,' she says.
It's a risky maneuver. A gown isn't a financial <a href="http://www.weddingnova.com/aboutwedding/Wedding-Roles.html">Wedding Roles</a> investment. It can take months or even a year to sell a gown, <a href="http://www.weddingnova.com/aboutwedding/Lawn-Wedding.html">Lawn Wedding</a>and the returns are comparable to those of selling a used car. Transaction details must be negotiated. That includes not only payment but, if the dress isn't being mailed, where to meet and try <a href="http://www.weddingnova.com/aboutwedding/Wedding-Party.html">Wedding Party</a>on the dress. One seller <a href="http://www.weddingnova.com/aboutwedding/Top-Barriers-For-Groom.html">Top Barriers For Groom</a> I interviewed arranged to meet a potential <a href="http://www.weddingnova.com/aboutwedding/Expensive-Wedding-Dress.html">Expensive Wedding Dress</a> buyer at her dry cleaner, which felt safer than inviting a stranger into her home.
But when a deal is done, it can feel, for seller and <a href="http://www.weddingnova.com/category/Evening-Dresses-813-1.html">Evening Dresses</a> buyer alike, like serendipity. Jennifer Ramsey tore a photo of a Tomasina gown from a magazine when she was 19 and kept the magazine page until she became engaged 18 years later. The dress, miraculously, was still being manufactured, <a href="http://www.weddingnova.com/aboutwedding/Global-Wedding.html"><strong>Global Wedding</strong></a> but at $8,000, it was priced too high for her budget. I thought <a href="http://www.weddingnova.com/aboutwedding/Fashion-Wedding.html">Fashion Wedding</a> was the most stunning dress I'd ever seen, she says. So I <a href="http://www.weddingnova.com/aboutwedding/Lawn-Wedding.html">Lawn Wedding</a> googled Tomasina and the style code.
Suzanne Fargo's listing for the Tomasina dress was halfway down the Google <a href="http://www.weddingnova.com/aboutwedding/Wedding-Flowers.html"><strong>Wedding Flowers</strong></a> page that led to PreOwnedWeddingDresses.com, and the dress came with a matching bolero jacket, two veils and a <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html"><strong>Wedding Dresses</strong></a> tiara. When Ms. Fargo, a 41-year-old retail manager in Orange County, Calif., got married in 2001, I thought some <a href="http://www.weddingnova.com/category/Flower-Girl-Dresses-804-1.html">Flower Girl Dresses</a> day my daughter might want to wear it, she says. Now divorced, with no daughter, she says, I want to simplify my life.
Ms. Ramsey and her mother flew to California from her home in Jacksonville, Fla., to see the dress. When they cracked open the sealed box, they discovered Ms. Fargo's forgotten custom <a href="http://www.weddingnova.com/category/Prom-Dresses-812-1.html">Prom Dresses</a> Jimmy Choo shoes and a handbag. Ms.<a href="http://www.weddingnova.com/category/Bridesmaid-Dresses-802-1.html"><strong>Bridesmaid Dresses</strong></a> Fargo threw them in, too. Ms. Ramsey, who is getting married in <a href="http://www.weddingnova.com/category/Wedding-Dresses-806-1.html"><strong>Wedding Dresses</strong></a> September, says she paid $3,200 for a dream come true.
I've kept my dress so long because a friend <a href="http://www.weddingnova.com/aboutwedding/Wedding-Gifts.html">Wedding Gifts</a>once told me I should save it for my daughter. At seven, Saskia is enthusiastic about the dress, but I suspect that by the time she marries -- if she does -- she'll have less-sentimental ideas.
Termites

Termites wrote on 11/20/09 4:35 AM

The database is one of the main reasons why cms systems can be hacked
Trust deeds

Trust deeds wrote on 11/20/09 4:49 AM

The database is not really conducive for optimal performance and you are right there are so many things to look into.

Write your comment



(it will not be displayed)