How Do You Evaluate a Data Center? 211
Posted
by
ScuttleMonkey
from the check-for-major-fault-lines dept.
from the check-for-major-fault-lines dept.
mpapet writes to ask about the ins and outs of datacenter evaluation. Beyond the simpler questions of physical access control, connectivity, and power redundancy/capacity and SLA review, what other questions are important to ask when evaluating a data center? What data centers have people been happy with? What horror stories have people lived through with those that didn't make the cut?
i ran a junky data center (Score:5, Informative)
I ran a data center long, long ago. My sales guy knew it wasn't going to pan out and threw me to the wolves. He asked me to start the tour, and then he took a long lunch to miss it.
The guys I gave the tour to seemed very intelligent. They only spent about 60 seconds on our data center. The instant they saw the carpet, their eyebrows were up. When I didn't lie to them that there was no diesel generator on the other side of the (secretly dead) batteries, they did exactly what they should have and stormed out without saying thanks.
Some important questions: (Score:3, Informative)
I'm assuming this is evaluating for co-location purposes. Here are some things I'd ask.
1) How quickly can I get a new server deployed into it? How do I do it?
2) Can I get a tour? Now? (Note that this not only lets you see the data centre, but also will give you an idea of security. Look for procedures on getting in, notice if they ask you to sign a release form, etc.)
3) How close to capacity are you? (The answer should include space, floor weight, power, cooling, and network. If it doesn't, why not?)
4) What are your racking/networking/cabling standards? (They should have some, at least where you connect to them, but they shouldn't be onerous).
5) How many people manage the data centre? You don't want to be one car accident away from loss of access or service.
6) How about power management? Is the centre on a UPS, redundant UPSes, or nothing? Can you get charts of the power going to the servers? Can you get DC for telecom servers, or only AC? Is it on a generator for long-term outages? (Note that you may not need this--in which case you shouldn't pay for it. Alternatively, if you need it, make sure it's there!)
7) Is it manned 24/7? (Ditto!)
If you can, ask them to pull a tile so you can see under the raised floor. Underfloor cabling (and suspended ceiling cabling for that matter) should be neat, tied, and labelled. Dead cables should be pulled, not left to rot. There has to be sufficient clearance for unrestricted airflow. Cages are better than lying on the floor.
Most of what makes a good data centre comes down to organization. If it's a rats nest, then even if there's one guy who knows "everything," it will be less reliable, less consistent, and less predictable. Procedures should be written down, printed, filed in labeled binders, and regularly updated. (Note: Online copies should be canonical, but also needs to be accessible offline when shit --> fan.)
Fire suppressant mechanisms (wet vs. dry, live pipes, etc.) need to be considered, as does emergency lighting. If the operators need to start digging around for a flashlight to read what they should be doing, then things aren't happening the way they should.
Be picky. If they're leasing space to you, then their data centre design and maintenance is their BUSINESS, and they had better get it right! Look for a neat, well-organized, well-documented, well-panned data centre. Also make sure that it fits your needs.
Do not jump in with both feet (Score:4, Informative)
Regardless of how well they are decked out, always start with a "pilot project". Start small for a short period to evaluate real world performance of both their equipment and their tech support. We currently have a pilot project in place to evaluate a datacentre for outsourcing our compute requirements. We have learned that while they have exceptionally good equipment in place, their responsiveness and ability to provision is highly questionable.
Re:Just off the top of my head (Score:5, Informative)
I noticed something when touring one datacenter. They had a neat conference room that overlooked the whole datacenter. You could see the heat rising off of one area (Google's room). They went on and on about the wonders of their cooling, and how they had so much capacity.
We later took the guided tour. The person I was with was talking to our guide, and I was paying careful attention to our environment. There were tremendous hotspots on the floor. We're not talking about 78 degrees. It was closer to the 90's. Other spots were downright cold. Why? Because they had all this capacity, and no real planning. The circulation was insufficient, even though the capacity was available. A well populated rack will always be hot at the back, but it's expected that they will draw the air off of that area rather quickly. I've even seen datacenters that enforce their hot/cold aisles, but then there isn't much of a reason for it. There is no air return on the hot side, and it's just blowing at another aisle's cold side.
Sometimes it's good to just walk the floor with a tech (not a salesman), and ask questions about the operation. What kind of fiber do you have coming in? How many providers? How good are your generators really? Do you test them on a regular basis? I've found a sales minion will say there are a dozen providers coming in, but it will turn out that only one has substantial fiber, and the others are sharing that. {sigh} Sometimes they will have generators, but they've never test fired them. Sometimes the tech is just frustrated at the nonsense at that datacenter, and that's indicative of how it's going to be to work with them.
Re:Just off the top of my head (Score:3, Informative)
you state "Raised Floor Height". What is good?
24" is good, 36" is better. I once had a place with 8'0".
Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors.
Hogwash.
Cooling Capacity" -- what's good and what is bad? How is this measured?
Capacity is measured in BTUs, or specifically tons (12,000 BTUs to a ton). What's most important is the relationship between BTUs and KW consumptions. In a nutshell, how much heat can you remove from the building vs how much are you putting in?
an outside air duct (Score:4, Informative)
This worked well until the outside temperature got down to about 15 degrees Fahrenheit, but then it failed miserably: the outdoor condensers no longer functioned, the AC shut down, and the entire IT department went into a panic.
The first time this happened, I (a lowly Help Desk tech) suggested to the CIO that he run a duct into the room from the outside: a simple fan would bring in enough sub-freezing air to cool the servers.
The second time it happened, the look on his face told me he hadn't taken my suggestion seriously enough.
The third time, he flipped a switch and the fan cooled his server room just fine.
Re:Just off the top of my head (Score:3, Informative)
Yeah, what do I know about the subject? I'm just quoting from a recent talk given by Subodh Bapat, Vice President, Energy Efficiency and Distinguished Engineer, Sun Microsystems.
Oh, and there are some articles about this [greentechmedia.com]
But please, continue to refute my statement with clear, unsupported, single-word denials. They carry so much weight in an argument.
Re:an outside air duct (Score:1, Informative)
I've also had to do this in a cooling emergency (which luckily happened in the middle of winter in Wisconsin) The important thing to remember is that you lose control of humidity, which is almost as important as temperature.
How to choose a colocation facility (Score:2, Informative)
I wrote an extensive article on choosing a datacenter/colocation facility several months back. The full post can be found on my blog, but I will paste it below for your Slashdot reading convenience:
http://www.bitplumber.net/2009/04/how-to-choose-a-colocation-facility/
How to choose a colocation facility
Choosing a colocation facility is one of the most important decisions an IT professional can make. It will have repercussions for years down the road, as there is generally a contract term associated, and it becomes difficult/costly to move. At the same time, unless you are a facilities professional, it is hard to tell the difference between the quality of one facility vs. that of another without knowing the right questions to ask. I have developed this list in the hopes that it will be a reference to folks evaluating datacenter options. This has been written using the assumption that you need a local datacenter rather than a DR facility (which can have very different needs), however, many of the same concepts will apply.
Location
When it comes right down to it, there are still certain things you have to do physically in person. You can’t run a network cable through SSH or RDP. Having a datacenter close by makes a huge difference, especially when you lose remote connectivity and must go push a button in an emergency (we all have done this once or twice). In general, the newer, more high-end, and redundant your equipment is, the less you should have to touch it in person. Things are getting much better with out of band remote access controllers, but sometimes being there is worth a lot. You can’t hear that fan making funny noises from your office.
Does the facility have good access to transportation such as freeways and airports? Are their hotels nearby if you will have out-of-town contractors visiting? How close to logistics depots are you for your vendor-of-choices parts, i.e. Cisco, Dell, HP, etc
Does the facility have adequate parking that is close to the building, does it cost money? Is it somewhere you want to leave your car in the middle of the night while you are inside working?
Do you have line-of-sight to the datacenter? If you can manage to get a wireless link to your datacenter this can be an extremely cost-effective option for high speed connectivity. There is something to be said for controlling your own destiny when it comes to your connectivity rather than being at the mercy of a telecom provider. Will the facility allow you to put a wireless antenna on the roof and how much will they charge?
Staffing
Do they have on-site staff 24×7 to respond to emergency situations, to secure the facility, and to provide access when you forget/loose your badge (or have to stop by on your way home from the gym).
If they do not have staff on site 24×7, what is their on-call policy? How long would it take them to respond to a power failure, a UPS exploding, a transformer catching fire in the parking lot, an Internet outage, an FM-200 fire suppression system going off, an HVAC system failing, or any other major malady (yes I have had all of these things happen to me in facilities I have worked in, and I am still waiting for the day a fire sprinkler goes off or there is a real fire in a datacenter).
What level of professional services can they provide? Basic remote hands (please press the power button)? More advanced troubleshooting (help diagnose a failed network switch)? Or even managed services (i.e. they take care of backups).
How competent are their NOC engineers, facilities folks, etc What quality of vendors do they use to do electrical work, HVAC maintenance, network cabling? This can be hard to tell, but there are lots of small clues you can pick up on.
Does their staff speak English fluently and without heavy accent? It is extremely difficult to communicate on the phone with someone in a loud datacenter environment about complex technical issues when both of you are having a hard time understanding each other. This dramatically slo
Working Conditions (Score:3, Informative)
Is there a good desk working area? Is there a landline/PBX for you to make calls from? Is there decent mobile phone reception in the work area and by your cabinet? Can you eat food or bring drinks into the work area or around your cabinet? Is it in a shady neighborhood, where you might feel a little intimidated bringing in tens of thousands of dollars of emergency IT equipment @ 3 AM? In the event that your credentials aren't working (i.e. hand scanner, ID card swipe), can they let you in remotely, or is it manned 24/7? Is it carrier neutral and are there other backbone providers that you can connect with? Do they charge for running cables between cabinets, especially in cases where the cabinets are not adjacent? What is the max amperage that they'll provide per cabinet? Do the rack cabinet doors remove easily? Are there chairs available, and damn it, are they comfortable?
Are you buying the DataCenter or Renting Rackspace (Score:2, Informative)
Your question is a little ambiguous. Are you looking to buy a data center of your own or are you renting rackspace?
If you are buying the Data Center
1.) Normal title , lien, Structural due diligence as for any RE purchase
2.) Is it on a flood plain
3.) Seismically active site?
4.) Serviced by multiple communication providers from multiple CO's
5.) Power available from two different substations.
6.) Physical security / susceptibility to civil unrest
7.) Physical access driveways, parking, loading docks, hallway widths elevators ramps
8.) Floor / raised floor design loads. I have seen more than one raised floor rippled by rolling overweight gear on it.
9.) On site power generation / fuel storage. Mech. condition, age, availability, reliability, repair-ability
10.) Sufficient Chiller Capacity
11.) Sufficient UPS / Power Conditioning
12.) Sufficient space both for current needs and growth for planned lifetime
13.) Sufficient office / command center space
Those should be adequate to get you started.
For rented rackspace
I would say you at least need to glance at items 2 through 11 above. Beyond that
1.) Per rack power limits
2.) Physical security
3.) If you are using "hands on" services it's skill set and response time.
4.) Whatever value add services you will be using.
Sorry it is late and a long day and this is all I can think of.
The things they won't tell you (Score:1, Informative)
All the normal stuff is important, but it is the things they won't tell you that is really important.
I visited a data center that was really easy to commute to. See, it was above a subway train. Further, it was below a parking garage. Nice.
I've seen data centers at the end of airplane runways. I've seen data centers in flood zones, hurricane areas, earthquake prone locations and with roof leaks. Nobody is going to point these things out to you.
I really like how one DC manager showed off the huge springs that the turbine generators were sitting on to prevent vibration translating to other parts of the building, but they were behind on their battery maintenance schedule. Doing the simple things, well and consistently are key.
In the end, you can't trust anyone and the safest answer is to spend your money in 2-3 locations. Don't make 1 a "primary." They all need to be "primary." Mix production, DR, Dev and Test apps across all locations, split them evenly, and keep them 500mi or more apart. Don't have a single supplier for the DCs either. Consistency is good, unless it isn't. Being different in each location and having different management is a hassle, but both probably teams won't make the same critical mistake that takes your systems down.