A story in eWeek today showed just how bad a data center migration can get. Hostway, a large web hosting company with as many as 600,000 customers, was moving 3,700 servers (representing roughly 3,000 customer accounts) 270 miles from one facility to another. They prepared their customers, many of them small businesses, for 12 to 15 hours of downtime. This was on July 28; many of the customers were still down on July 30th, and a significant number are still down today. For a small business, even anticipated downtime can be costly in terms of lost business and communications. When the length of the outage goes from hours to days, the level of pain goes from bad to excruciating. What’s worse is that the company isn’t communicating with the affected customers, tech support lines are down, and frantic emails are going unanswered. Of course, the Hostway website is still up their pitch to new customers is beyond ironic in light of their ongoing service problems.
An interview with Hostway VP of marketing shows that these guys aren’t evil ;they truly expected the move to come off smoothly and feel terrible about the outage. But when you’re packing up and moving 3,000 servers, maybe, just maybe you should factor in the possibility that extraordinary problems might occur and have contingency plans in place to soften the impact. This is particularly important when other businesses are relying on your service for their livelihood. If this screw-up only disrupted a single company, that’s one thing, but it hurt thousands of companies and that’s what makes this situation, in my mind at least, disgraceful. Could be I’m a little sensitive to this because I run a small business and am intimately familiar with how much we depend on reliable hosting to make our company work.
So what could they have done differently? One thought immediately comes to mind.Did they absolutely have to move all 3,000 servers at once? A phased move would have limited the risk of a massive outage and also relieved some of the time pressure. They also could have reached out to their vendors for assistance in making the move. For large customers, vendors will often provide help above and beyond what they are required to do by contract, arranging loaner systems, helping get boxes up and running, etc. We couldn’t help but notice that Dell is featured prominently on the Hostway web page; perhaps not the best publicity, particularly since the Hostway spokesperson cites hardware failures as the chief cause of the outage.
A phased move and getting extra help would almost certainly have made the migration more expensive for Hostway. But how much will it cost them to repair the damage to their reputation, and how much of their existing business will they lose? As a business owner, I’m having a hard time imagining anything that Hostway could do to regain my trust and keep me on board as a customer.’