Over 650 TB of Data Up For Grabs From Publicly Exposed MongoDB Database (csoonline.com) 96
itwbennett writes: A scan performed over the past few days by John Matherly, the creator of the Shodan search engine, has found that there are at least 35,000 publicly accessible and insecure MongoDB databases on the Internet, and their number appears to be growing. Combined they expose 684.8 terabytes of data to potential theft. Matherly originally sounded the alarm about this issue back in July, when he found nearly 30,000 unauthenticated MongoDB instances. He decided to revisit the issue after a security researcher named Chris Vickery recently found information exposed in such databases that was associated with 25 million user accounts from various apps and services, including 13 million users of the controversial OS X optimization program MacKeeper, as reported on Slashdot on Wednesday.
Re: (Score:1)
What firewall. There are even easy to use wrappers for this stuff people!
sudo apt-get install shorewall ... ?
Maybe they're not using a shit OS?
Windows has sudo apt-get?
NoSQL is amateur land. (Score:4, Insightful)
I'm not totally surprised. In my experience, those who use and advocate NoSQL "databases" tend to be on the amateurish side of the spectrum.
Except for a very small number of cases, if you have data to store you should be using one of the many existing relational database management systems.
It doesn't even matter which one you use. There are many you can pay for, and many free ones, too. PostgreSQL is the best free one. MySQL isn't very good, but it's a hell of a lot better than the NoSQL systems.
These databases can easily store key-value data. They can also easily store JSON, XML, and other non-relational formats.
Learn how to use indexes. It turns out that many of the performance problems these people use to justify switching to a NoSQL database only arise because they don't know how to index their relational database system. Some of these NoSQL supporters don't even know what an index is at all!
Learn how to use SQL. It ain't perfect, but it's a fuck of a lot better than writing queries in JavaScript, of all things!
Even the most basic book about relational databases that you can find at your local bookstore will clearly explain all of these concepts.
It's like most people who use NoSQL, especially when they're dealing with anything less than petabytes of data, try as hard as they can not to put in the small amount of effort needed to learn now to use relational databases properly.
Re:NoSQL is amateur land. (Score:5, Insightful)
The issue here really isn't SQL vs NoSQL. It's about securing the data and access. Lack of security is not inherent in NoSQL, it just occurs more often than SQL databases.
Re:NoSQL is amateur land. (Score:4, Informative)
The issue here really isn't SQL vs NoSQL. It's about securing the data and access. Lack of security is not inherent in NoSQL, it just occurs more often than SQL databases.
Well, I'd have to disagree here. If I install a MongoDB on some cloud VM using the default setup, I have an insecure database available on the internet. If I install Postgres, well... I can't even access the database remotely.
Much of the time NoSQL is a problem looking for a solution. I've had to deal with this a number of times before - and I'd concur what the AC above posted - the culture around NoSQL often seems to be "squeee! I have a NoSQL database.. look at my awesome speed" without investing time getting to know existing SQL properly. Or securing the systems.
Case in point: a while ago I replaced an Elasticsearch database with 40 lines of decent SQL. It wasn't rocket science (MySQL), but it was an order of magnitude faster and less complicated as it didn't require an additional server and the data pump between the SQL and Elasticsearch databases. Yes, I know Elasticsearch isn't a NoSQL db in the same sense as Mongo, but it was part of swag (4 in total) non-traditional databases in use papering over poor indexing / querying on the existing MySQL db. Oh.. and no security. That's the NoSQL culture I've experienced, and not just at one shop. All over the place.
We lost something when database administrators were given short shrift and we all became "full stack developers". So much knowledge has just gone out the window.
Re:NoSQL is amateur land. (Score:5, Insightful)
The problem is that NoSQL usually is set up by inexperienced people or amateurs. Relational databases tend to be the opposite. Regardless of whether NoSQL installs with total access and Postgress with no access doesn't matter. Both need to be configured to provide the access needed. I do agree that Postrgress' method is safer because nobody has access unless you grant them it. But, that doesn't change the fact that both need to be configured.
The reality is NoSQL is being implemented in cases where it is not the best solution. Same could be said for Relational databases. Often they are overkill for simple things. In the end, both are tools and selecting the right tool for the job is the best solutions.
Re:NoSQL is amateur land. (Score:5, Informative)
The issue here really isn't SQL vs NoSQL. It's about securing the data and access. Lack of security is not inherent in NoSQL, it just occurs more often than SQL databases.
Well, I'd have to disagree here. If I install a MongoDB on some cloud VM using the default setup, I have an insecure database available on the internet. If I install Postgres, well... I can't even access the database remotely.
Ummm, unless something has changed recently the respective "createdb" tools for both MySQL and Postgres make it very simple to start up a new DB with an open root account and listening on an IP.
I agree with your general view on NoSQL and who tends to use it, but it's the ones using it that are the problem here and not the software itself. The only argument against NoSQL/Mongo here is that it is shocking in this day and age for any server to allow un-authed access. That can be applied to many server software packages though including OSS SQL DBs.
What this is really a symptom of is the people with no real experience that are just about "getting the job done" rather than understanding the actual problem and the potential risks. I don't know how many times I've seen cases in my career where the first sign of communication issues causes people to start tearing down the security barriers. Most of the time the real problem was a minor config issue (going over the wrong interface, missing host to host ACL, listening on the wrong port, etc..), but the damage done to "just get it working" is rarely ever fixed.
Re:NoSQL is amateur land. (Score:4, Interesting)
I'm not sure about Postges, but the default install of MySQL will only listen on localhost. So you have to change your config file to even get it to respond from a remote computer. Then there's the account you're actually accessing it from. By default, the root account in MySQL is also only accessible from localhost. So that's 2 things you have to change simply to even allow a connection from a remote machine in MySQL.
I agree that there''s a bigger problem with people just wanting to "get the job done". Even those two steps only take about 20 minutes to figure out how to get around them and leave your root account accessible from the internet, even if you have no idea what you are doing. But shipping the database with such lack security is right up there with default passwords on home routers and other such problems. Stuff like this just shouldn't happen. Even if the security can be easily disabled by a determined user, it shouldn't be insecure by default. That way, if they have an insecure setup, at least it's because of something they actively went out and did themselves.
Re: (Score:1)
I'm not sure about Postges, but the default install of MySQL will only listen on localhost. So you have to change your config file to even get it to respond from a remote computer. Then there's the account you're actually accessing it from. By default, the root account in MySQL is also only accessible from localhost. So that's 2 things you have to change simply to even allow a connection from a remote machine in MySQL.
Wow. Two things? WOW!
And here I thought it would be simple to make sure that out of the box you don't immediately broadcast your enterprise/development/baby learning database to everyone ever like say the actual point of the article.
Re: (Score:2)
Elastic is not remotely similar to SQL solutions. Out of my 2000 table database find me all the denormalized records where column a in table b meets criteria c or columns d in table e (a 1/2 dozen relations away) and e in table f meet criteria f unless g in columns i,j,k in tables l,m,n apply. And then support dozens of similar queries. And remember those criteria are often searches against multi-megabyte clob fields.
NoSQL exists because no database can possibly offer durability of commits, horizontal
Re:NoSQL is amateur land. (Score:4, Insightful)
I think what AC is trying to say here is that there's a lot of woo surrounding NoSQL, and that people who buy into that woo don't have a solid grounding or understanding about how computers work, hence security isn't even an afterthought for them.
As far as I can tell, NoSQL is basically a buzzword that means either key-value pair or document storage. (I'm certain there must be some edge case where Mongo or others are the correct answer, at least I hope so.) In the case of key-value pairs, why not just use PostgreSQL? In the case of document storage, why not just use one of these things called a filesystem?
So, you're correct: lack of security is not inherent to NoSQL; lack of critical thinking, experience, and in-depth knowledge about tech is. This pretty much gets back to why our information infrastructure just sucks in general. Computers are sufficiently advanced technology and are thus indistinguishable from magic. Put another way, pretty much everybody, even some people in tech, thinks computers are magickal palantirs powered by waldos used to project some kind of digital self into the nether realm of cyberspace, a place not Sheol, which can only be tamed by wizards.
Their idea of a cyber-security is having a wizard on hand who can shout at the evil hackers, who have equally mystical powers drawn from the foul evils of the burning hells, "You cannot pass! I am a servant of the Secret Fire, wielder of the Flame of Anor. The dark fire will not avail you, Flame of Udun! Go back to the shadow. You shall not pass!"
Re:NoSQL is amateur land. (Score:5, Insightful)
I'm a huge fan of postgres. I introduced it to the organization I work in probably 15 years ago and we continue to use it today. But there are some things it just doesn't do as well, - like scale, at least not as easily as a database like Cassandra can. Don't get me wrong, you give up a lot in many cases by not choosing an SQL database and it's not always the right choice. However, lack of critical thinking isn't inherent in using a NoSQL database. Lack of critical thinking is assuming a certain type of database is always the right choice for any future project, just because it's worked well in previous projects.
Re: (Score:2)
Well if you want to emulate Mongo you would need a filesystem that is horizontally scalable where copies of the filesystem and directories (indexes) are being coordinated across dozens of machines. You would also want clustering so that read and write loads on the filesystem can be segmented. So either you are talking a SAN or a filesystem database.
Re: NoSQL is amateur land. (Score:2, Insightful)
No, lack of security is inherent in 20-something IT types who think that everything that's ever been dune in this profession is beneath them and not worth knowing. Those are the types who tend to use and recommend this garbage--then they set it up wrong because learning how stuff works is also beneath them.
Apologies to the 20-somethings who learn stuff and actually belong in this profession.
Re: (Score:3)
It's like most people who use NoSQL, especially when they're dealing with anything less than petabytes of data, try as hard as they can not to put in the small amount of effort needed to learn now to use relational databases properly.
I've had issues with getting people to even understand relational databases in the first place, including the sortakinda amusing case where under orders, I gave a co-worker one to work with and looking for a progress report a month later, he told us that the entire database didn't work and couldn't work and nothing could be done to make it work. Funny, because it had been in flawless use for years, and not funny because it put me a month behind.
It was also the last time my boss told me "Ol, you have to l
Re:NoSQL is amateur land. (Score:5, Interesting)
Re: (Score:3)
Re: (Score:3)
That question is about 5 levels above the Fizz Buzz test [c2.com], so I'm not surprised that you had so much trouble finding people who could pass your test. People who work with databases should definitely be able to answer that question, but it's also common enough to find people with computer science degrees who've never had to deal with a database in their life.
Most computer science degrees are only going to have a single course on databases, and it's probably easy enough that you can get through by kind of unde
Re: (Score:1)
I'm not totally surprised. In my experience, those who use and advocate NoSQL "databases" tend to be on the amateurish side of the spectrum.
Except for a very small number of cases, if you have data to store you should be using one of the many existing relational database management systems.
It doesn't even matter which one you use. There are many you can pay for, and many free ones, too. PostgreSQL is the best free one. MySQL isn't very good, but it's a hell of a lot better than the NoSQL systems.
These databases can easily store key-value data. They can also easily store JSON, XML, and other non-relational formats.
Learn how to use indexes. It turns out that many of the performance problems these people use to justify switching to a NoSQL database only arise because they don't know how to index their relational database system. Some of these NoSQL supporters don't even know what an index is at all!
Learn how to use SQL. It ain't perfect, but it's a fuck of a lot better than writing queries in JavaScript, of all things!
Even the most basic book about relational databases that you can find at your local bookstore will clearly explain all of these concepts.
It's like most people who use NoSQL, especially when they're dealing with anything less than petabytes of data, try as hard as they can not to put in the small amount of effort needed to learn now to use relational databases properly.
I concur with the PostgreSQL. I've stood up many instances, and they have been robust and rock-solid.
I steered clear of MySQL ever since Darth Ellison's empire "embraced" it.
Re: (Score:2)
I have to agree. It's crazy how far people go to avoid just having to deal with relational database in OO languages. Sure, things like Hibernate/JPA can be clunky, but there are better models out there.
I can't count how many times I've had to explain to people that because we are using .Net, it's really not hard to use relational databases, and we don't need all those things that NoSQL people thing are revolutionary and groundbreaking, because LINQ has been working fine for years, and when it comes down to
Web-scale breach (Score:2)
Impressive that 35,000 users forgot how to NAT and firewall. "Oops!"
Re: (Score:1)
I am behind TWO nat/routers/spi firewalls with DD-WRT client bridging.
This is probably more of an issue for cloud hosted software (data).
Locking down / securing is never easy for people who just want things to work. I blame the designers of the software and hostiing for this not the user.
Re: (Score:2, Informative)
Read the documentation for the software which even comes with reasonable guides for Linux and Windows?
https://docs.mongodb.org/v3.0/core/security-network/
For best results and to minimize overall exposure, ensure that only traffic from trusted sources can reach mongod and mongos instances and that the mongod and mongos instances can only connect to trusted outputs.
Sorry but anybody working in this space should know better.
Re: (Score:1)
Sorry but anybody working in this space should know better.
Yes, but untrained monkeys work so much cheaper!
Re: (Score:2)
It's an issue as old as this industry.
NoSQL needs a boost in adopters to gain momentum and beat-out a lot of entrenched relational databases?
Easy. Just make it easy. Make it mind numbingly easy to make the database perform useful work. Simple package installation, no need to configure anything away from defaults. Example project can connect and do CRUD-y things in minutes. Explain away security and good practice in the documents no one will read, (because the manual as they see it is the blog post on "
Re: (Score:1)
Locking down / securing is never easy for people who just want things to work. I blame the designers of the software and hostiing for this not the user.
This. Secure the default install. It's so simple. Turn off services until used. Generate a random password on startup, not a default one. Encrypt partitions automatically during install. Open ports only to the local machine not to the internet by default. Save attachments don't execute. Turn on SSL and not unencrypted connections. etc.etc.
OpenBSD etc have shown this works. You may lose a few users, but Apple, though far from perfect, has shown that you can do okay security whilst keeping a simple
Re:Web-scale breach (Score:5, Insightful)
Doesn't matter if they forgot or tried, but their defenses were flawed by a misspelling, or misunderstanding, or a semicolon that should have been a colon. Truth is that trying to secure information on computers connected to the Internet of Horrors is roughly equivalent to stashing your wealth in a huge paper tent guarded by an elderly german shepard who has been sedated and two winos who have provided themselves with a liberal supply of cheap booze.
Folks, this internet thing really does have enormous potential. And it will always be useful for broadcasting and reference work and cat videos. But it's way too complex to provide a reliable vehicle for financial information, personal data, or command and control of most infrastructure. No amount of frantic patching, blaming someone else, and trying to administer remote computers with unknown configurations is going to fix that. This sucker just can't do what "they" want it to do. At least not safely.
What's the answer? I haven't a clue. But acknowledging that there is a problem is probably a good first step.
Re: (Score:2)
Mongo on the other hand comes with access controlled turned off by default.
Perhaps the answer is start with proven technologies before adding immature tools like Mongo into the mix. And before any NoSQL fan
Re: (Score:3)
> Security seems to be almost an afterthought.
By the time you get done activating security high availability, and integrity checking, you've often lowered the performance to below that of a more mature, well-tuned SQL database. There is a very amusing video that covers some of these issues in passing:
http://www.mongodb-is-web-scal... [mongodb-is-web-scale.com]
Re: (Score:3)
For those without Flash, here's an another version: https://www.youtube.com/watch?... [youtube.com]
Re: (Score:3)
Re: (Score:2)
What's the answer?/p>
The Cloud, trickle down economics, and CoCo-Puffs.
Re: (Score:2)
Not NATing is sensible. Not firewalling your unpassworded database? Not so much.
In the (slight) defense of people running these servers, the article points out that MongoDB's default configuration used to be to accept connections from the internet. They've changed that, but upgrading uses your old config file so you won't get the new defaults automatically.
But still, this is something you should be checking for.
Clone available here (Score:5, Funny)
I've posted a clone of this data-cache on my BBS. Just make sure you have at least a 54.6 baud modem, so ensure a speedy download.*
Long-distance telephone charges may apply.
What the FUCK is Tb (SICK) Supposedly to BE? (Score:1)
Whom to measures in bits? Dum motha fux.
Re: (Score:1)
Yes, it should be To, I mean if they specified it in Bytes they'd have to specify the byte size too. 5 bit? 6 bit? 9bit? Who could tell?
Re: (Score:2)
Yeah, BUT.... (Score:5, Funny)
.....something something webscale.
If you're gonna leak data, go big or go home!
If only someone had told them.... (Score:2)
Re: (Score:1)
Unlike, say, Subversion, that stores your passwords in local cleartext by default? At least it announces now when it's doing so the first time. Or unlike, say, every SSH key generator on the planet, that also stores your private SSH keys with no passphrase by default? Or unlike, say, MySQL and Postgresql, that also both are installed with no password by default?
Gigabit fiber is arriving just in time! (Score:5, Funny)
Once AT&T hooks us up, I'll be able to download this volume of data in a mere two months.
Oh, wait. 1TB monthly data caps.
Okay, once AT&T hooks us up, I'll be able to download this volume of data in a mere 54 years.
Re: (Score:2)
Could you try again? Google Translate seems to have some trouble with your arithmetic.
Re: (Score:2)
650TB worth of 5TB drives -- external drives, so you don't need to worry about sleds and stuff -- would be under $20K today.
I expect getting a single drive (SSD of one form or another) with 650TB of capacity will be easy within 10 to 20 years. I'm a bit nervous extending the current cost-of-storage trend out much further than that, despite its solid track record over the past 30 or 40 years. By the time we're talking about petabyte capacity on a single small device, a lot of things will be very different.
Hipsters != DBA (Score:1)
This is what happens when you let hipster doofus script kiddies decide your database architecture.
Anyone who uses mongodb.... (Score:5, Interesting)
... deserves to lose their data as a lesson not top use amateur hour software.
Field data longer than 8kb? Ooh, can't index that and it won't get returned in a query using that index.
Shard gets corrupted? Oh bad luck, thats some of your data gone - unless you've used also replication in which case you'll have spent 2 months trying to set it all up.
Lots of concurrent writes? Yeah, well, with monogdbs single monolithic write lock - good luck with that.
Want a DB that uses encrypted network transfers between shards and replica sets? Sorry.
Want a DB that uses a sane query language - ie not one thats a nightmare mashup between pure javascript and parameter passing using javascript to an underpowered underlying query engine? Don't use mongo.
Etc , the list goes on.
Re: (Score:2)
Want a DB that uses a sane query language - ie not one thats a nightmare mashup between pure javascript and parameter passing using javascript to an underpowered underlying query engine? Don't use mongo.
Etc , the list goes on.
Don't forget that you have different dynamic typing rules to battle with depending on whether you're doing a regular json query (BSON type rules) or whether or not you're doing something javascript-ey (like collection.mapReduce, where you get Javascript type rules).
Example (code is probably wrong; I'm working off memory and haven't touched Mongo in nearly a year):
db.collection.insert({foobar: "1"});
//no results
db.collection.find({foobar: 1});
db.collection.mapReduce(function(){
if (this.fooba
Re: (Score:2)
Shard gets corrupted? Oh bad luck, thats some of your data gone - unless you've used also replication in which case you'll have spent 2 months trying to set it all up.
To be fair, I think all exposed MongoDBs mentioned in TFA are being replicated right now, so they've inadvertently got that side of things covered! :D
Are the users aware they use Mongodb? (Score:3)
There are so many people with very little internet skills own domains and basically hire someone to build their site. I recently registered a domain and I am being swamped by solicitations to "the new small business owner" about "making a great website to sell your product" with "search engine optimization" and "customer support data base" blah blah blah. Frankly even their email skills are fishy and I am sure they will jury-rig some solution. If they change the default installation to make it less secure, it definitely looks like someone managing something for someone else where convenience overrides security.
Re: (Score:2)
This.
I installed Debian on a laptop some time ago. Occasionally I'd experience some temporary performance issues. A quick review of 'top' showed mongodb to be at the top of the process list, hogging resources. I've looked at a number of apps to figure out what might be using it and found nothing, so I killed it. Nothing seems to have suffered, so I just disabled its startup.
Sharding (Score:1)
So, how does one scan for this? (Score:2)
How do you do a scan of your network to find unsecured NoSQL databases like this? I bet that I'd find a few of them on my work's intranet.
Re: (Score:1)
MacKeeper, LOL (Score:2)
MacKeeper's only reason for existence is people who don't know any better. It's malware, pure and simple. I really wish that decent sites would ban MacKeeper ads.
Re: (Score:2)
I think you're responding to the wrong article, however if you change "MacKeeper" with "MongoDB", your post still works.
Re: (Score:2)
Did you even read the submission? :)
Re: (Score:2)
*blink* I'm obviously missing something, but heck if I can tell what that is. :P I probably shouldn't be slashdotting before coffee.
650Tb is only 81TB (Score:2)
"684.8 terabytes of data"
So wouldn't that be "650TB", not "650Tb"?
Re: (Score:2)
Headline should be "5.5Pb."
Re: (Score:2)
The real problem (Score:2)
The real problem is that MongoDB is the Visual Basic of databases.
People have been flocking to MongoDB because they consider SQL databases "too difficult" and "require too much effort". They want something easy that they can just slap together and get up and running, and all other considerations be damned.
And this is the result. Databases are *not* hard, but they *do* require you to actually think things through. If you can't do that, you shouldn't be doing development to begin with.
NoSecuritySQL (Score:2)
I up your NoSQL databases with NoSecuritySQL. Not only is it webscale, but it's now it's even faster without security checks.
Send In Ming the Merciless (Score:1)
He should be the one taking care of Mongo's databases!