Inside the Google-Plex 130
tappytibbins writes "Baseline magazine has an in-depth story about how Google manages its own IT infrastructure. From the article: 'In general, Google has a split personality when it comes to questions about its back-end systems. To the media, its answer is, "Sorry, we don't talk about our infrastructure." Yet, Google engineers crack the door open wider when addressing computer science audiences, such as rooms full of graduate students whom it is interested in recruiting.'"
print friendly version (Score:5, Informative)
Elevator Pitch Version: (Score:4, Informative)
=== End Elevator Summary ===
Not many companies are willing to write their own application layers to deploy services. Most companies CAN'T. It's just not worth it. It's worth it to Google because developing and deploying world-wide information retrieval services is their business.
However, a standardized Application OE that can run and take advantages of the resources of many potentially unreliable computing resources would be very valuable to many businesses.
Grid technologies, web services, J2EE, and clustering technologies are just scratching the surface.
Forget the googleplex (Score:5, Funny)
I want inside the google party plane!!
Re:Forget the googleplex (Score:2)
Re:Forget the googleplex (Score:2)
Also... (Score:4, Informative)
I'm still waiting for pictures of the "party plane", though.
Re:Also... (Score:1)
PARTY!
Re:Also... (Score:1, Interesting)
1. Isn't PageRank based on a voting system? OK, in an intranet full of PDFs and Word docs, who votes? Yup, that's right, PageRank doesn't work in an enterprise context.
2. Infrastructure. Th
Re:Also... (Score:5, Insightful)
Re:Also... (Score:2)
Hope that helps.
Friendly (Score:3, Informative)
It still worries me that google will soon know everything about everyone. I hope they dont share that data with ANYONE.
Re:Friendly (Score:2)
And that is if it isn't already. Id love to do a poll finding out age versus perception of Google. I am willing to bet that those older, who have seen this story play out before, have seen corporations abuse power for a couple decades have a much more
Re:Friendly (Score:2, Informative)
Re:Friendly (Score:2)
450,000 servers? (Score:2, Funny)
Re:450,000 servers? (Score:1)
Sysasmin(S) (Score:2)
They still haven't been able to break that 225000 to 1 host to sysadmin ratio barrier yet. But they're working on it.
Re:Sysasmin(S) (Score:4, Interesting)
Re:Sysasmin(S) (Score:5, Interesting)
i'm guessing google are big enough to have thier own datacenters and thus not have space at such a premium as smaller operations. If space isn't at a premium replacing a machine in a rack probablly isn't worth it (it means you have a machine whose remaining usefull life is out of sync with the rest of the rack its in).
fast-food approach to hardware (Score:1)
Re:450,000 servers? (Score:2)
NDA time? (Score:2)
Re:NDA time? (Score:5, Funny)
This slip of the mind will prove invaluable in my Google-imitation plots.
Re:NDA time? (Score:2)
That makes perfect sense (Score:5, Insightful)
Re:That makes perfect sense (Score:1)
A Google Lecture Experience (Score:5, Interesting)
An alum of my university who works at Google recently visited and gave an informative lecture with a long Q&A session. I can vouch for the fact that we were told more than I've ever been able to read online about the way Google manages various issues, like their IT infrastucture. However there were still limitations to what he would/could tell us (sorry I won't go into specifics). It seemed (as you would expect) the better our questions, the better his answers, and if we asked questions that were too good, then it was likely that he did not feel liberated to answer.
Also, Google was cool enough to sponsor a Programming Contest and a Graduate Research Conference we held. Our alum attended our little conference and had great feedback and questions for our presenting students. With respect to knowledge, intelligence, and humor this guy was all I would imagine and/or hope for one of our alums working at Google.
On the otherhand, I was very unimpressed with certain issues concerning lack of professionalism in the lecture. As one example, though this is only an impression, it seemed that he felt he could just get away with wearing jeans and a Google t-shirt for the few days that he was with us because he worked at the ever prestigious Google. It seemed a bit arrogant. Also keep in mind that his position at google is higher than a solutions engineer.
Just thought I'd share.
Re:A Google Lecture Experience (Score:5, Insightful)
It's been my experience that the companies who worried most about what their IT staff was wearing were the worst to work for.
Re:A Google Lecture Experience (Score:2)
What no pants? (Score:2)
Re:A Google Lecture Experience (Score:2, Insightful)
Sure, it's all relative, but just because you're a programmer or even a manager of programmers doesn't mean you have a free pass to represent y
Comment removed (Score:5, Interesting)
Re:A Google Lecture Experience (Score:5, Interesting)
Please accept the above for the lovely second-hand urban myth that it is, one belonging to a CDC 6600 site where I was lucky enough to attend a few lectures.
Re:A Google Lecture Experience (Score:3, Funny)
The CEO walks into the elevator, and sees a man stood there in jeans smoking a cigarette. The company had a strict no-smoking policy (this is some years ago, before it was commonplace) and a strict dress code.
Incensed, the CEO demands how much this man is paid.
"$750 per month" (told you it was some years ago!)
"Here's $750. Get out and don't come back."
"OK."
Re:A Google Lecture Experience (Score:1)
Re:A Google Lecture Experience (Score:2)
Quite correct, and I apologise.
"Logic card" meant a circuit board that contained a single logic element, such as a half-adder or a register.
"CDC" meant Control Data Corporation, whose first deliveries of 6000-series supercomputers were the reason IBM announced the System 360 and OS one year before the first prototype was seen.
"Seymour", single name, used to mean only
Re:A Google Lecture Experience (Score:2)
I blame the fluff between the neurons that comes from having ideas in off-line storage too long. Some of dem bitz is getting negatized.
Re:A Google Lecture Experience (Score:2, Insightful)
Re:A Google Lecture Experience (Score:1)
It's not disrespect when someone doesn't wear a suit, it's common sense. The bloody things have got to be the most ridiculously impractical peices of clothing on the face of the Earth. The jacket is like a sweater; hot as hell in the summer yet not warm enough to replace a winter coat. The pants stain, wrink
Re:A Google Lecture Experience (Score:2)
...Three or four hundred interviews for tech staff to my name (and had to live with the result); truth is, you can generally tell if someone's the goods in the first two minutes of the interview, irrespective of wha
professional presentation for interview (Score:2, Interesting)
Re:A Google Lecture Experience (Score:2)
The 2 cultures: suits and jeans (Score:2, Informative)
At the business end of the spectrum, you have the MBAs and the like, and the more expensive the suit you wear, the more respect you get. Pick top-level executives from any company other than Google and a very few other technical companies, and you'll see what I mean. They will always wear expensive suits, and people who aspire to be in their position will d
Re:A Google Lecture Experience (Score:3, Insightful)
In this case, his primary purpose was recruiting. In particular, he wanted to recruit really smart people. Really smart people know that clothes are irrelevant to the job (unless the job is to model clothes or work at IBM). Thus, the appropriate professional attire is (drum roll please) jeans and a t-shirt. Those are the clothes most l
On the other hand... (Score:2)
I don't know what Google's dress code is, but I do know that when, say, Oracle sends a consultant out to help with setpup/problem resolution, etc. they usually show up in business-casual attire - khakis and a button-down shirt with their Logo.
Again, I don't know what Google's stance is on dress. They may be perfectly fine with Jeans and a T-Shirt.
Re: (Score:3, Funny)
Re:On the other hand... (Score:1)
Re:On the other hand... (Score:4, Interesting)
So is it neat to have a trendy office space? Sure. Is it neat to have communal centres scattered around the building, and be encouraged to stay afterhours to play games? I guess. But it's the kind of thing that gets old once you realize you've got a family and a life outside of work. Working for Google sounds like working in a basement with a bunch of friends, but that only really works if you don't have other things you want to be devoting time to. Once their workforce matures a bit, I'd guess their "kooky, trippy workspace" won't work quite so well. Don't forget, they're still basically a glorified startup. I'm sure Microsoft had a lot of the same feel back in '86.
Re:On the other hand... (Score:1)
But it's the kind of thing that gets old once you realize you've got a family and a life outside of work."
Funny you should say that - my project manager was wearing sandals today and I didn't think much of it... There's a guy who turns up in clogs and nobody thinks it especially out of the ordinary :)
(we're the London office of a large multinational media agency/software house)
But no, that's by the by: The thing I rea
Re:On the other hand... (Score:2)
Re:On the other hand... (Score:1)
For example, we have a flexible start time - I come in late quite a lot, not really being a 'morning person', but I leave later, enough to more than make up for it. Being a tester, I think this tends to actually benefit the project I'm working on, as the dev team will often put up a new build they've be
Re:A Google Lecture Experience (Score:2)
*has a quick sniff*
Excuse me, just a second please.
Re:A Google Lecture Experience (Score:2)
Re:A Google Lecture Experience (Score:1)
Your lecturer was RMS!?!
Re:A Google Lecture Experience (Score:2)
Re:A Google Lecture Experience (Score:5, Insightful)
Re:A Google Lecture Experience (Score:5, Insightful)
I'm a computer guy, I wear jeans to work every day, and I'm happy that way. If I'm being asked to speak in front of a large group of people I don't grab something out of the hamper, y'know?
Re:A Google Lecture Experience (Score:2)
All things considered... (Score:2)
Considering that Google has 450,000 servers, that would be one massive e-penis, from which I would gladly accept the gUrine.
Re:A Google Lecture Experience (Score:2)
Maybe in a presentation environment, but in a work environment Go
Re:A Google Lecture Experience (Score:2)
Congratulations, you are NOT qualified to be a Google employee. In some industries, wearing particular clothes is not the definition of professionalism. Google is in such an industry.
You want a job in accounting. Or at EDS.
Re:A Google Lecture Experience (Score:1)
Did anyone ask? (Score:2)
Funny, I didn't notice anyone ask.
Re:A Google Lecture Experience (Score:1)
Hold on, you're a student, and you're accusing someone of lack of professionalism?
Do you not just think he was trying to make everyone comfortable? Or were all you students in suits/ties etc and so he stood out?
Get over yourself, you're just another student. I'm sure if he had been presenting on Wall Street he'd have dressed in a suit.
Re:It's hard to believe... (Score:3, Funny)
I read that *many* of those machines have more than 640K of RAM!
Re:It's hard to believe... (Score:1)
Re:It's hard to believe... (Score:1)
this list is far from complete [google.com]
Re:It's hard to believe... (Score:2, Insightful)
Post-Beta (Score:4, Insightful)
If it is anything like their web-presence, half the stuff must have 'Beta' appended to it.
New GPayRoll-Beta!
If you read TFA, (Score:2)
They had a big fight switching to that, before that they were using (drum roll please), QUICKEN!!! (lol)
Although the HR apps are custom written. Exciting
Re:If you read TFA, (Score:1)
I once had an interest in Google (Score:2, Interesting)
So I tried to get another interview for a while, but no bites. Goog
Re:I once had an interest in Google (Score:4, Informative)
Perhaps you should have informed your recruiter about your background?
Re:I once had an interest in Google (Score:1)
You'd probably fail a basic grammar test also...
Re:I once had an interest in Google (Score:1)
Regards,
MBC1977,
(US Marine, College Student, and Good Guy!)
Note the context (Score:3)
So the question to ask is: Who is `we'? I could provide an answer. `We' here, is Google's official answer to such questions from the ever inquisitive press men. Those who speak on behalf of Google have been asked to memorize that answer if they do not wish to talk about a topic. It works well. This approach reminds me of the government's `We can neither confirm nor deny...' mantra.
What follows next could be interpreted as a Google engineer's answer to a question, which answer may simply represent one of many possible implementations and NOT one you could find supporting Googles infrastructure.
Guys, this is all about semantics and context. Good night!
In other news (Score:5, Funny)
Re:In other news (Score:2)
Re:In other news (Score:1)
Re:In other news (Score:1)
Re:In other news (Score:2)
Could be worse -- if Sony bought out Apple we'd have a new generation of Beta Macs
Re:In other news (Score:1)
Close (Score:2)
Little in the way of structuring data (Score:4, Interesting)
Re:Little in the way of structuring data (Score:1)
http://it.slashdot.org/comments.pl?sid=190798&cid
Re:Little in the way of structuring data (Score:2)
I would bet that they manage structured data within the filesystem itself - in fact, i think thats the ONLY way they could do it. I started going into detail in this post, but it started being
Re:Little in the way of structuring data (Score:1)
Just another Google Publicity Stunt (Score:1)
mb
go to the source (Score:5, Interesting)
Sawzall [google.com] (simplified scripting on top of MapReduce)
MapReduce [google.com] (Google's massively parallel system based on the concept found in functional programming. The system takes care of managing jobs, parallelism, and fault tolerance, allowing engineers to more quickly produce code.)
GFS [google.com] (Google's File System)
Google's Cluster [google.com] (An older paper describing how Google's search cluster works. The cluster described in this paper is a few generations out of date.)
BigTable [andrewhitchcock.org] (Google's semi-structured database. There haven't been any papers released, but this is my write up based on a talk given in October 2005.)
And here are some videos:
The Google Linux Cluster [washington.edu]. This is an older video where Urs Hoelzle talks about their system and focuses more on the hardware side of things.
Google: A Behind-the-scenes Look [washington.edu]. Jeff Dean gives an overview of most of the technologies mentioned in papers above. I thought the demonstration of Google's internal word clustering was interesting (and funny).
Perspectives on the Information Industry [washington.edu]. This is a technology-light (IIRC) talk given by Eric Schmidt.
BigTable: A Distributed Structured Storage System [washington.edu]. The talk from which I created my BigTables notes (above).
Andrew
Re:go to the source (Score:3, Funny)
Ah, I thought it would be something like this (Score:2)
Hey guys, watch it, it's about the women at Google!
I guess now we have yet another reason to go there.
My Google Recruitment experience (Score:5, Interesting)
This lead to my observation of part of their file storage system which is quite possibly the most tweaked NFS nightmare/genius/what-the-fuck I'd ever seen. My past experience with networked file system was, I admit, very limited compared to what they had going on. Now, again, I wasn't even going to have anything to do with this system or any sysadmin work at all, but it was obvious that they wanted you to at least have knowledge of the system on some level beyond the user. It also came across as a showing-off culture too. I am glad I didn't take the job for various reasons, but if you are a sysadmin freaker who loves dinking with shit, you'd fit in; especially if you like to show it off too. Just be prepared to have some middle manager there fuck with you for a hour or two on the phone before you get to the outer part of the inner sanctum.
GoogleFS and You (Score:1)
by leandrod (17766)
Everyone is talking about GoogleFS. But no one is talking about how they manage structured data. How do they do it? Some SQL stuff, some homegrow potion, or have they managed to create a sensible interface for structured data on top of GoogleFS?
--
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Linux
----------8<----------
Wild Guess® ?
Rob Pike?
http://herpolhode.com/rob/
http://plan9.bell-la
Re:GoogleFS and You (Score:1)
Everyone??? Oh, I get it, you mean everyone who didn't bother to RTFA. Makes sense now. You and that other guy taking his wild guesses at building a file system (with zero background) ought to compare notes. Ha ha ha. I mean, why have half an unsubstantiated 'conversation' when you could have the real, unsubstantiated, deal?
I know, I know, nobody is supposed to read the article. But, in th
Re:GoogleFS and You (Score:1)
http://justfuckinggoogleit.com/ [justfuckinggoogleit.com]
GPL? (Score:1)
Similarities with the Borg cube? (Score:2)
A whole lot of words about nothing? (Score:1)
Romance? (Score:2)
This sounds more like the beginning of some Harlequin romance novel than the first line of a serious piece of journalism. The sentence even manages to run-on and has a simile. I expect that by the end of the article his hairy chest will be bared by the protagonist and that they wi