![Google Google](http://a.fsdn.com/sd/topics/google_64.png)
![Businesses Businesses](http://a.fsdn.com/sd/topics/business_64.png)
![The Internet The Internet](http://a.fsdn.com/sd/topics/internet_64.png)
![IT IT](http://a.fsdn.com/sd/topics/it_64.png)
Google Launches Google Sitemaps 223
Ninwa writes "Google has launched Google Sitemaps. It seems to be a service that allows webmasters to define how often their sites' content is going to change, to give Google a better idea of what to index. It uses some basic XML as the method of submitting a sitemap. More information on the protocol is available in an FAQ. What's most interesting is that Google is licensing the idea under the Attribution/Share Alike Creative Commons license. According to the Google Blog, this is being done '...so that other search engines can do a better job as well. Eventually we hope this will be supported natively in webservers (e.g. Apache, Lotus Notes, IIS).' They even offer an open source client in Python."
great interview (Score:5, Informative)
http://blog.searchenginewatch.com/blog/050602-195
More unabashed Google loving... (Score:5, Funny)
I guess the rest of the world has a long way to go to catch up...
Search Engine (Score:3, Funny)
They had white pages
And hits by the score
All the people's queries
Waiting by the door
Ooooh, what a search engine it was
Ooooh, what a search engine it was
Many geeks and hackers
They made up its core
Everybody's dearest
A daily stop for more
Ooooh, what a search engine it was
Ooooh, what a search engine it was
It went to the market
Of the engines it was king
Of his honor and his glory
Slashdot would sing
Ooooh, what a search engine it was
Ooooh, what a search engine it was
A burst had found it
Re:Search Engine (Score:2)
Re:Search Engine (Score:2)
Especially since Google really does have a great idea here. I know Slashdotters on the whole love Google, and I know there's a bit of a backlash, but for the sake of the integrity of the argument, let's have that backlash be for some legitimate reason, not just because Google's too popular because, well, it really is great.
D
Re:Search Engine (Score:2)
Cool idea (Score:5, Interesting)
For example, a website we launched a couple months ago is primarily images. We played nice - all of the images have legitimate alt tags, and we tried to let the site degrade properly in older browsers (although you really wouldn't get much, in those instances).
But the biggest problem we had was trying to get the site spidered by Google. It would be, and it would appear in the index, but it would be listed far below sites that linked to it. I don't believe Google likes sites that are primarily images. We populated meta tags with descriptions, but they weren't included; we even tried using hidden text - legitimate, hidden text that would serve as the sites description, but not break the design - but you know how Google feels about those sorts of things. We had to walk a fine line. This'll be nicer.
Re:Cool idea (Score:2, Interesting)
Re:Cool idea (Score:4, Informative)
Quite right, a new site can be listed in the Google index pretty quickly -- it only took a few days for my latest site to be found by the Googlebot -- but it takes a while before any PageRank gets assigned to its pages, especially if there are no inbound links to the site. No PageRank, no top listing...
EricCurrently at #1 for adsense tips [google.com]
Re:Cool idea (Score:5, Informative)
Re:Cool idea (Score:2)
Maybe you had few competitors and those competitors (for the search result) were also new.
Re:Cool idea (Score:2)
Re:Cool idea (Score:2)
It depends on how many pages there are that match those keywords. If your title is unique enough, then sure, your site will show up first. But as soon as there's contention for the keywords, don't expect to stay up top.
EricView your HTTP headers here [ericgiguere.com]
Re:Cool idea (Score:4, Informative)
It's quite common to be high up for matching terms for about a week, then disappear for three months or so. This seems to be normal behaviour for new sites and is nicknamed the Google sandbox [google.com] and seems to have been confirmed by the patent application recently made public.
The sandbox is just an artificial lowering, so if you're a match for a rare term you can still be found quite easily.
Re:Cool idea (Score:2)
So you're the one who came up with "DISCREET ONLINE PHARMACY" ??
Seriously though, if there aren't a lot of other sites containing your title, that's easy. If you're one among a dozen or so, not so easy.
Re:Cool idea (Score:3, Informative)
This is how the system works. Google can index your site very quickly (within a couple of days), if you have an incoming link or submit to their crawler. If your site is well keyword optimized for a fairly rare keyword, it is entirely plausible that it would come up number one fairly quickly.
What takes a long time is for google to update their pagerank index. This is where your site will sit in the Go
Re:Cool idea (Score:2)
Re:Cool idea (Score:2)
Sorry.
D
Re:Cool idea (Score:2)
Re:Cool idea (they stole my idea?) (Score:2, Interesting)
This is a suggestion for the people who take care of indexing web sites.
Because Google is the first search engine of choice it has enough of influence to point noses into the same direction.
So, i propose a new element to be added to websites: a sitemap file. Similar to the favicon file, every site could have an (xml?) file containing information about the info and the info-topography on the site.
Google has already a 'si
fuckedgoogle.com anyone? (Score:2, Interesting)
http://www.fuckedgoogle.com/ [fuckedgoogle.com]
Re:fuckedgoogle.com anyone? (Score:2)
Sitemaps abuse? (Score:3, Insightful)
Re:Sitemaps abuse? (Score:2)
Re:Sitemaps abuse? (Score:3, Interesting)
Re:Sitemaps abuse? (Score:2)
I've not seen anything to suggest sitemaps will improve your ranking, just get you indexed more often.
If you claim pages update every day, but they don't, it will be pretty easy for the spider to tell. So you could stop the frequent scans if they aren't really needed, if after say a month the supposed daily updates never happened.
Re:Sitemaps abuse? (Score:2, Interesting)
Anyway, a brief look at the proposed format gives very little scope for abuse - you can specify location, change frequency, last modified and a priority, and that's it. The priority is specified as only applying to urls from the same site, so what you can do with it is fairly limited. Overall, it looks written as a set of additional hints to spiders crawling the site.
Re:Sitemaps abuse? (Score:4, Informative)
First, the priority is a relative priority, so if you want to set every page to 1.0 (defined as the highest priority) it'll mean nothing.
Second, if you lie about update frequency or the date of the last update they'll figure it out pretty quick.
These aren't commands, they're hints.
While they are at itmaybe new meta tags? (Score:2)
Re:While they are at itmaybe new meta tags? (Score:2)
I think the "coverage" tag would be probably what you're looking for.
Re:While they are at itmaybe new meta tags? (Score:2)
Most static sites do not use RSS.
Re:While they are at itmaybe new meta tags? (Score:2)
Sure. Lots of people use it. Does Google grok it? I dunno.
"Most static sites..."
So. Be a trend setter. I encourage you to not think in terms in static and dynamic but in terms of modern and outdated.
Do modern search engines even care about meta tags anymore? It seems that technorati-style tags may be a more modern equivalent, but they do tend to influence the way you display information on your site. There's a lot of discussion on whether or not these tags are primed to spiral
Re:While they are at itmaybe new meta tags? (Score:2)
I have to admit that I wonder how a location tag could be abused.
modern vs static?
Not every page needs to be dynamic. A page for a restaurant does not tend any dynamic content so RSS seems like over kill.
Reinventing the wheel? (Score:2)
Reminds me of blog pings - what's wrong with using the Referer header? Doing some checking and then fetching the referering page and checking for linkage?
Has the world gone XML crazy?
Has the world gone XML crazy? (Score:2)
Just think of this sort of thing as inter-linking web services sitting on top of the http protocol.
Justin.
Re:Has the world gone XML crazy? (Score:2)
I don't see how centralising all this header information serves webmasters better. Only Google.
Re:Reinventing the wheel? (Score:2)
Re:Reinventing the wheel? (Score:2)
I believe proxies cache the headers as well, unless must-revalidate is specified in which case it must do a If-Modifed-Since or similar request which will return fresh headers. How is it not Google's responsibility to remember when to crawl your page anyway? Thats exactly what they intend to do.
"They're designed for clients like web browsers, where you only care about whether there have been changes when the user is checking on the site"
Why is Google any d
Google is IT's Willy Wonka (Score:5, Funny)
Re:Google is IT's Willy Wonka (Score:2)
if you wander into the room where Duke Nukem 3D is being tested you'll be thrown out.
I think you mean Duke Nukem Forever.
Duke Nukem 3D is nearly ten years old, I remember playing it at high school on my Pentium-100 laptop.
Re:Google is IT's Willy Wonka (Score:3, Informative)
I can neither confirm nor deny the existence of any secret video game testing rooms.
-B
Re:Google is IT's Willy Wonka (Score:2)
Creative Commons Meme (Score:2, Informative)
http://www.realmeme.com/miner/preinflection.php?s
How is this a win-win? Here's how.... (Score:2)
Livejournal.com has had a number of problems with Google, and often just plain outright bans them from spidering the site. Part of the problem is that all the registered users have their journals at journalname.livejournal.com as well as livejournal.com\users\journalname. This means indexing the journals for resisted users doubles the load on their server farm!
With something like this, livejournal would be able to define exactly how often the indexing process occurs,
Re:How is this a win-win? Here's how.... (Score:2)
Re:How is this a win-win? Here's how.... (Score:2)
Re:How is this a win-win? Here's how.... (Score:2)
robots.txt (Score:2)
Re:robots.txt (Score:2)
Google wants this sitemap funtionality to make into the web server itself. So, it looks like they're opting for the long-term solution.
Google Evil Index (Score:5, Funny)
Lotus Notes? (Score:2)
(The submitter probably meant Lotus Domino, which is still a bad webserver, but not nearly as bad as Notes would be.)
Re:Lotus Notes? (Score:2)
great idea (Score:2)
Or maybe another hidden use... (Score:4, Insightful)
Marketplace of Ideas (Score:2)
And I'm willing to license my idea, "better search engines with better user interfaces", to Google, for a modest sum.
what's the basis of the license? (Score:2)
In any case, patented or not, the CC license that this falls under seems acceptable for an open standard, even if it is patented, because it is transferable and because its requirements are minimal. Contrast this with the Microsoft Office XML license, which is royalty-free (for now...), but non-transferable.
Darn it (Score:2)
Will wait until I get my new server.
Re:Darn it (Score:2)
What does Creative Commons mean here? (Score:2)
Re:What does Creative Commons mean here? (Score:2)
There was never a patent on GIF; Unisys had a patent on the LZW algorithm which could be used in GIFs. Uncompressed GIFs were not covered by the patent.
But if you look at a simple XML format like Google Sitemaps, there is no novel algorithm involved in reading or writing the format and thus no basis for patent.
More proof that Google isn't Netscape (Score:2)
The thing that seems so cool about this sort of thing is that it opens up the search service to the rest of us to help us make our content easier to find when it is updated. One thing that I have come to really respect about Google is that they don't rely on the government to beat Microsoft back down the way Netscape did. Google has managed to make a product that 47% of the US Internet users want to use, even though MSN is the default in IE. Remember Netscape 4? There's a reason that bloated POS failed, any
search forms (Score:2)
If you have a bunch of data in a MySQL database, ordinarily Google can't find it. You have to create a static link somewhere with a URL for the search you want to make googlable. Those take maintenance.
There may be some sites that want certain areas crawled, but not others, and those areas aren't maintained by the webmaster or only the top-level part should be h
Feeling the heat from google-watch and critics? (Score:2)
You do have to wonder how much of the 'do no evil' philosophy is cover for the "let us store and index all information about everything, including you" philosophy. Not that I'm going to stop using Google until their results become less usable than Yahoo's results...
google watch watch ? wow. (Score:2)
That's a thing of beauty. [google-watch-watch.org] Well, not really, it's a damn shame to waste a domain name on a nearly plain-text page, but it's still pretty funny. Does anyone really love google enough to host a page like that on their own? Wow, if so. I mean, I've always liked google, but would I rent out a domain to host a anti-anti-google website? I doubt it. Thanks for that, though. Definitely a +1 interesting from an AC.
502 Server Error! (Score:4, Funny)
Could be better (Score:2)
What about a robots.txt extension to define the
location of the sitemap index?
Why not just use rss/atom? (Score:3, Insightful)
Re:Why not just use rss/atom? (Score:3, Interesting)
What About Us Python-Free Zones? (Score:2)
insight into unlinked directories (Score:3, Informative)
essentially using "find" and "grep" alone, but this tool is much better,
faster and easy to configure. Cool.
Note that this tool will allow google to reach files which never would be
found by spidering a site, because the files are not linked. If you
include something like
<directory path="/var/www/html" url="http://www.example.com/"
in your config.xml and run "sitemap_gen.py" on it, you will give the world
access to a large amount of material
(like test versions of your website or source code you did not want to
make accessible). We might see lot more material material which had been
'hidden'.
Re:IIS? (Score:2)
Remember that MS doesn't have a monopoly on web servers, so they can't be dicks about it like they can with everything else.
Re:Off Topic, Yeah, But I Am So-o-o-o Googled Out (Score:3, Insightful)
Well, maybe if Google stop doing stuff for a while?
Lots of slashdotters seem interested in what Google does, either becuase it tends to be neat, or so they can worry about privacy and the info Google potentially has access to.
Re:Off Topic, Yeah, But I Am So-o-o-o Googled Out (Score:2)
Re:Off Topic, Yeah, But I Am So-o-o-o Googled Out (Score:2)
This is just a idea that allows a corporation to do its job better. It would be like if the Census Bureau asked everyone to mail a list of all of the members of their household to them, and update it when they have kids, so they didn't have to take the time to count door to door.
Re:Off Topic, Yeah, But I Am So-o-o-o Googled Out (Score:2)
Re:Still in Beta (Score:2)
Just about all of Google seems to be in beta. While it is nice to get the stuff early, "beta" is a pretty meaningless term as far as Google stuff is concerned.
Re:Still in Beta (Score:2)
Re:How does this benefit me? (Score:5, Insightful)
It benefits you because:
Also, you wouldn't necessarily have to maintain more than one sitemap. You could use XSLT to create the sitemap.html file for your site from the XML file you create for Google. In fact, wouldn't it be nice for Web authoring tools to do this automatically for you?
EricMake Easy Money with Google: The Blog [makeeasymo...google.com] (powered by blojsom [sf.net])
Eh? (Score:2)
Re:Eh? (Score:2)
You're right, but I bet a lot (most?) sites don't do it right and Google figures this is the next best way... Easier to ask people to put up sitemaps than tell them to fix their pages/servers.
EricWhy the Vioxx recall reduced spam [ericgiguere.com] (humor)
Re:How does this benefit me? (Score:3, Interesting)
Google immediately knows that the site exists, immediately knows how many pages there are, how often they are supposed to change, AND what priority I place on them, so out of my 150 pages, the 10 I want spidered first are labeled as higher priority.
This makes total sense to me.
Re:SHUT UP SHUT UP SHUT UP (Score:2)
Re:Google Bitching (Score:2)
You are right.
No other company has ever launched an Internet Search function.
No other company has ever launched web-based email.
No other company has ever provided online maps.
No other company has ever offerd the contents of usenet via the web.
No other company has ever offered navigable satalite photos of the planet.
No other company has ever offerd realtime webcaching and compression to "speed up" one's access.
No company has ever cached websites for access wh
Re:Google Bitching (Score:2)
Frequently the google implementation is vastly superior in some purely technological manner. In this way they are innovative.
Frequently, google is able to turn the superior technology into a superior user experience as well. In this way, too, they are innovative.
Frequently, google's cool creations are hyped beyond all belief. In this manner, they are... erm... the recipients of a geek love-affair, and not
Re:Google Bitching (Score:2)
This is such bullshit. Some of the stuff they put out is very cool and newsworthy like Google Maps, Gmail, etc... But so much crap that is either not ready yet, not unique, or just plain boring gets posted here. Its literally a direct feed of the Google blog half of the time.
It also annoys people like me how people on slashdot treat Google like its the second coming of Christ. If anyone says anything negative they get bombarded with posts saying
Re:Next thing you know... (Score:2)
Which is a pretty good clue that it is a troll.
but think about it. Isnt it possible?No, no it isn't possible.
Re:Next thing you know... (Score:2)
If Google starts doing crap like that, designers and devlopers aren't going to go along with it, at which point Google's usefulness drops dramatically, and all their users go to their competitors.
Re:Next thing you know... (Score:2)
It's possible, but then... (Score:2)
Then again, Google coming up with detailed design guidelines for their pages for public consumption would be incredibly useful for designers. They use a lot of cutting edge JS tricks lik
Re:Next thing you know... (Score:4, Informative)
As long as everyone can freely and voluntarily use these specs without having to pay anything, how is this a bad thing?
Re:Next thing you know... (Score:2)
And what is this about?:
Slashdot requires you to wait 2 minutes between each successful posting of a comment to allow everyone a fair chance at posti
Re:Next thing you know... (Score:2)
This might be marked as troll... but think about it. Isnt it possible?
No. Because Google needs our pages indexed by its robots more than we need Google to index us.
Sonny, let me tell you 'bout a time be
Re:Next thing you know... (Score:2)
Re:Next thing you know... (Score:5, Insightful)
Nice. Google proposes a way to help web site administrators have a bit more control over how their site is perceived by a search engine, releases this proposal under an open source license, and at least a few people on slashdot accuse them of (*pinky to corner of mouth*) taking over the internet.
Most of Google's recent actions have been good things -- sponsoring open source developers for the summer, proposing ways for site administrators to provide additional info about their site, and implementing a "nofollow" option to prevent spammers trying to increase their page ranking. However, if they constantly get criticized and second-guessed for doing good things, what incentive do they have to continue? If you give a charity $20 and they criticize you for not giving them $30, are ever going to give anything to that charity again?
Let's give Google the benefit of the doubt. Just like a person, they'll probably make some mistakes, but like a person I'll give them the benefit of the doubt until they prove me wrong. Some corporations do actually do good things and still manage to be successful, and in those cases they should be supported, not attacked.
Re:Cool idea: Browser utilization of this data! (Score:2)
Re:Google is mightier than slashdot (Score:2, Informative)
Re:Slashdotted? (Score:2)
Their blog is holding up at least.
Re:Information already available to Google? (Score:2)
-Get preferences from admins (priority, approximate update frequency).
-Get metadata (time of last update)
They can tell most of that just by downloading the page regularly but with 8 billion pages it's probably pretty hard to do every one of them with any frequency and most of them probably change a few times a year if that.
Now they can tell if a page has changed and whether it's likely to change in the future with a few kb of gzipped xml instead of megabytes of HTML.
They've op
Re:WE BROKE GOOGLE! Woohoo! (Score:2)
Python is Cool (Score:2)