Tracking the IT Job Market with a Bot 166
atlantageek writes "Is the
IT job market improving? Is the growth in Unix or Windows? Should I
study Data Warehousing or E-Commerce? Identify the recent trends with
CJ Miner, a small tool I've written that has been monitoring the Computer Jobs website for the
last year."
Don't you mean (Score:3, Insightful)
Re:Don't you mean (Score:5, Informative)
That was also my interpretation of this project. I'm afraid that computerjobs.com wouldn't necessarily represent the entire IT market, but rather a very small percentage of it. The software would be limited to indicating various demographics at computerjobs.com, perhaps arguably and tentatively serving to indicate the "competency level" of their members. Without data from many sources, however, you couldn't hope to provide an accurate impression of the overall market.
Maybe the programmer should sell the collected data back to them.
Re:Don't you mean (Score:4, Interesting)
Some IT jobs make you the webmaster, network guy, database guy and janitor. Other jobs just leave you a single position and hit deep. How can the bot be intelligent enough to separate?!
Re:Don't you mean (Score:1)
or just use indeed.com (Score:3, Informative)
Or you can use indeed.com [indeed.com], which lets you search all jobs within the last 30 days from almost a thousand job sites (including computerjobs.com).
You don't even have to visit the site to check for new jobs -- it has RSS feeds and email alerts for new jobs that match your search criteria.
Or if you're really ambitious, use their free XML API [indeed.com] and do whatever you want with the data.
Quick... (Score:2, Funny)
Re:Quick... (Score:5, Funny)
Re:Quick... (Score:2)
Re:Quick... (Score:3, Funny)
Oh, no, you're right. You'd have to remove the advanced part =]
economics (Score:1)
More info needed (Score:3, Interesting)
2) What do these jobs mean in terms of disposable income?
Re:More info needed (Score:1)
As for income, couldn't say, but I do get the feeling that th
Re:More info needed (Score:2, Insightful)
Anyway, who the hell actually uses any sort of service or website to find a tech job above anything but entry level? That's what contacts and networking are for. You find yourself unemployed or looking for a new job and you put your feelers out to all your friends and colleagues who have moved to other c
Re:More info needed (Score:2)
Job agencies often post openings that don't really exist in the hope of bulking up their "potential" files.
Re:More info needed (Score:2)
Seen many dupes on CJ.. Slashdot doesn't have the market cornered and CJ's a close second.
IT Market Does Not Follow Economic Laws (Score:4, Insightful)
However, whenever a shortage of labor occurs in the IT market, the government consistently intervenes by importing H-1B workers to fix this shortage. As a result, the growth in wages is damaged. Working conditions (like working 60+ hours per week) do not improve.
Any perceived shortage in the market for IT labor is illusory. If this shortage were real, it would be short-lived, due to government intervention.
By the way, we see the same phenomenon in the market for unskilled labor: e.g. picking vegetables and fruits. The government fixes this shortage by allowing illegal aliens to flood this market for unskilled labor. As a result, wages (hovering around $5.00 per hour for fruit-picking in Southern California) never rise. Working conditions (like standing for more than 9 hours per day in the strawberry fields) never improve.
The rub is that politicians do not care about Washington's gross tampering in and bludgeoning of a (relatively) free market like the USA. Washington is eager to fix shortages of labor. However, Washington rarely fixes shortages of jobs by, for example, creating more government jobs. The interests of Washington are not aligned with the hopes and aspirations of middle America.
We should close the American market to (relatively) non-free markets like India, China, and Mexico. Further, the American market should be flung wide open to (relatively) free markets like Eastern/Western Europe, Canada, and Japan. Free trade is good -- only when we are trading with other societies that maintain (relatively) free markets.
Re:IT Market Does Not Follow Economic Laws (Score:2)
1) become an H-1b/L-1 IT worker
2) join the US military
3) become an a working illegal alien-and bide your time until the next amnesty.
Re:IT Market Does Not Follow Economic Laws (Score:2)
Foreign legion (Score:2, Informative)
Surely, you have to be a US citizen before you can join the military?
Nope. Non-citizens may enlist in the U.S. armed services [todaysmilitary.com]. Think about it: France has a foreign legion [foreignlegionlife.com]; why can't the USA?
Re:IT Market Does Not Follow Economic Laws (Score:3, Informative)
Re:IT Market Does Not Follow Economic Laws (Score:2)
When I was getting out of high school, I wanted to go to the Naval Academy. I had the grades, etc to get in, but not citizenship (non-citizens must have permission from their country of citizenship, and are not given US commissions upon graduation). Even tried to get sponsored for immediate US citizenship by my Congressman: no dice!
Considered just going to OCS when I graduated college, but I couldn't cause I wasn't a citizen yet. By the time I got my citizenship a y
Re:IT Market Does Not Follow Economic Laws (Score:2)
What's worse, is that non-resident aliens are signed up for Selective Service and could in theory be drafted to fight for the US without any real ties to it other than doing a degree there! I was positively shocked to find out t
Re:IT Market Does Not Follow Economic Laws (Score:2)
Also, social security is an especially good deal for someone that comes in and only works a few years in the US system.
Re:IT Market Does Not Follow Economic Laws (Score:1, Interesting)
If you talk of general industry - then lets pick Steel - a very very regulated markket in the US with many stipulations to close free trade with other more compet
Re:IT Market Does Not Follow Economic Laws (Score:1, Interesting)
Or Japan... where the average cow earns US$7.50 a day in subsidies?
Link to article [indianexpress.com]
Re:IT Market Does Not Follow Economic Laws (Score:2)
1) There's nothing that says that a shortage in IT workers should result in an improvement in working conditions (defined in your post as a reduction in working hours). In fact, if there is a shortage of workers, the expectations for hours worked individually should increase, not decrease. Nobody's going to say "gee, Jenkins, we're so happy to have one of the few available experts on our staff that we want you to go home at 4:00 every day."
2) Washington rarely fixes shortages
Re:IT Market Does Not Follow Economic Laws (Score:2, Insightful)
No, because if there really was a shortage, one could threaten to leave to work for a 40-hour company, and the biz would have to comply or start another difficult hunt.
(BTW, there is no "IT shortage". Biz lobbyists made i
Re:IT Market Does Not Follow Economic Laws (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2, Interesting)
Re:IT Market Does Not Follow Economic Laws (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2)
Israel and Switzerland have both come close (in their own respective and different ways), but I don't think I would like to live in either place (and for different reasons in each case).
Re:IT Market Does Not Follow Economic Laws (Score:2)
What's better? (Score:2)
And btw, "free market" is just an idealized abstraction [wikipedia.org] - with the risk of burning some karma I'll say it's pretty crude one, actually.
The notion of "free market" was developed during the Enlightenment or "Age or Reason", when Europe started to attribute everything to science and reason (after the Middle Ages attributed everything to the will of God). As a consequence, it is based on the fact that players in the market make their decisi
Re:What's better? (Score:2)
Wages won't go up, because there's suddenly more workers available. Thus, it won't lure new workers in from other industries where there may be a more abundance of workforce right here in the US.
And when times get tough, these H-1's, illegal immigrants, and other special practices are rarely scaled back, and so we end up with unemployed ci
Re:What's better? (Score:3, Insightful)
Re:What's better? (Score:2)
There's not many professional professions where this type of thing happens. Think doctors, lawyers, stamped surveyors, etc. Good programmers and IT professionals usual
Re:IT Market Does Not Follow Economic Laws (Score:2, Insightful)
Closing a market to supply (or demand) makes it "non-free", correct? We are talking about liberal (to neoclassical) economics right?
The U.S. market is purchasing the labor of India, China, and Mexico. Foll
Re:IT Market Does Not Follow Economic Laws (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2)
The first problem with your arguement is definition of what constitutes the "market." Just like in thermo when you can arbitrarily define what constitutes a system, in economics you can do the same when it comes to a market.
If you define the labor market as all the workers in the US then a
Re:IT Market Does Not Follow... _ MOD PARENT UP!!! (Score:2)
Goddammit MOD PARENT UP!!! It's been a while since I read something as true and as insightful on slashdot. And modders beware particularly of those who disagree with him in long posts; writing too much does *not* equal being insightful - I say this because I have just seen a long post of someone who disagreed with him modded insightful though on reading it I had the certain feeling the modder either was an idiot or did *not* bother read that reply and examine its content for vailidity.
And for anyone who
Re:IT Market Does Not Follow Economic Laws (Score:2)
"Free trade is good -- only when we are trading with other societies that maintain (relatively) free markets."
This is simply not true. Free trade is very good to the US for two reasons:
1. It allows us to trade g
Re:IT Market Does Not Follow Economic Laws (Score:5, Insightful)
Well, except for the fact that you have it all absolutely backwards! When the shortage appears, the government, which is already intervening elects to relax its interventions.
A limited number of foreign workers (fixed number of H1-B's) is itself per se an intervention. Without this intervention, the market would freely correct itself, through unrestricted immigration.
So what we have is a market where the government defacto creates shortages (through dejure immigration controls), but occasionally lets up on the shortage-creating phenonomenon, allowing normal market dynamics to function.
C//
H1B Myths (Score:2)
Re:H1B Myths (Score:2)
Re:IT Market Does Not Follow Economic Laws (Score:2)
citizens gravitate towards those that are.
"CJ" (Score:2)
Not many I'm afraid. Too many companies out there are "testing the waters" by posting false job postings and then tracking the number of resumes they receive.
The app's name, "CJ Miner", is strangely appropriate. In the adult web industry, CJ stands for "circle jerk", and refers to ad-laiden websites designed to fool visitors into believing that the site offers free content (pics and videos), when in fact nearly all links actually lead to other CJ sites.
Well, shit. (Score:2)
So how do I find people who are actually looking to hire?
--grendel drago
Re:Well, shit. (Score:2, Informative)
Dammit! (Score:5, Funny)
Re:Dammit! (Score:1)
With the Cylons.
Re:Dammit! (Score:2)
-- Message posted by Slashbot #1192392, activation date: 06/23/2004.
First step toward Slashdot domination (Score:3, Funny)
Script Kiddie Shortcut... (Score:2)
As for my experience looking for a job recently in Silicon Valley, I would say things are getting better for contract work. From what my friends are telling me, companies don't seem to be hiring for the long-term. I even got an ema
Re:Script Kiddie Shortcut... (Score:2, Insightful)
You don't understand. This is slashdot. We form burning urges to automate drudgery instead of live it. A true geek can only pull so many fake smiles and handshakes before going insane. When cornered, skunks spray; geeks code.
Colorblindness (Score:1)
Re:Colorblindness (Score:1)
Re:Colorblindness (Score:1)
Re:Colorblindness (Score:2)
Not that it isn't a stupid post anyway...
Very informative, I thought! (Score:4, Interesting)
If they really do get a pretty good number of I.T. related listings all collected up in one place, then yes - I think this is a pretty useful little graph/tool.
I've been out of work since the beginning of May, and living in the St. Louis area, it seems to me that there are currently very slim pickings. I keep hearing talk of the economic recovery, but at least around here - I'm not really seeing it.
According to your chart, that would be an accurate accessment too - since it clearly shows a sharp decline in I.T. jobs available in St. Louis since April of 2005. (And worse yet, I'm really mainly interested in the hardware side of things, but if you look at that specifically - you see that in my city, there were only a grand total of about 2 jobs fitting that category, at any given time!) In the whole U.S., it looked like I.T. hardware jobs only averaged around 1,200 *total*, for that matter. Not good... not good at all!
It looks like a scam to me. (Score:2)
Personally, this submission to Slashdot just looks like a cheap way of advertising a lame job board. If there was a real intent to track the IT market, all one would have to do is go to dice.com, and use the bots that you get for FREE there.
Dice.com is bigger, and nationwide. So you could really track how the IT market is doing over time.
Dice is also the only board which takes consultants seriously. That is, it offers specialized selections to make it easy
Re:It looks like a scam to me. (Score:2)
Re-inventing the wheel (Score:5, Interesting)
Should I study Data Warehousing or E-Commerce (Score:2)
You should study what you enjoy to study and work on. If this means a less trendy job, so be it. I'd rather eat broken glass than do web development, for example. Even if it was the hottest job on the market, I wouldn't do it anyway.
Re:Should I study Data Warehousing or E-Commerce (Score:1)
Jerks (Score:1, Interesting)
Read his resume, he's in Atlanta. I went to Georgia Tech and lived in Atlanta for a number of years, and this site was the first I would check when job hunting. It's fairly big there. They don't aggregrate other site's jobs, employers have to pay. With the exception of the head hunters, they were quality jobs. (But I've found that every head hunter posted job I
Re:Jerks (Score:2)
Thank intelligent design for the various projects and classes helped keep me sane, and 'kept my skil
Visa Sponsor (Score:1)
Whats that (Visa Sponsor)?
Re:Visa Sponsor (Score:2, Informative)
With my CCNA... (Score:1)
up-up-down-A-B-B-A-down dept. (Score:2)
software? (Score:2)
Man, I miss Echo (Score:1, Offtopic)
Re:Man, I miss Echo (Score:2, Interesting)
And yet if my stats are correct (I work for one of the largest IT resellers in Europe), the Insurance Industry are the #1 early adopters for virtualisation software. I think something like 18 of our top 20 VMware customers are all Insurance or Financial Servi
What to study? (Score:2, Insightful)
You should be studying Computer Science...
Ever wonder what happened to all those mainframe or COBOL folks? Knowing about E-commerce, Unix, Windows, Java, XML, or whatever the technology or trend du jour is might be impressive now, but in a few years, come the next thing, where will you be then? These things change at the blink of an eye.
On the other hand, algorithms, computability theory, formal languages, predicate logic, etc. don't.
A solid foundation of
Re:That's a myth (Score:2)
I would agree that it's good to have the theoretical background, but without the desire to learn new things, I'd say it's much less useful than following the trend du jour.
Re:That's a myth (Score:2, Informative)
That being said, a language is nothing more than a way to describe a concept. Ask the "trend du jour" people about programming concepts, and you'll probably get a bunch of software engineering babble in the reply. (an experiment of this would be interesting, given that the person wasn't mislead)
My basic belief in learning computer science is to learn how NOT to be a code-monkey. Any i
Re:That's a myth (Score:2)
Obviously (Score:4, Funny)
You should learn how to provide vertically integrated e-commerce solutions providing dynamic interaction to customers in synergistic markets.
Knowing how to work a sock puppet [tvacres.com] also helps.
Wanna do it with hardware? (Score:2)
check it out
http://www.thinkgeek.com/gadgets/electronic/732d/
Already been done (Score:1)
legacy systems? (Score:3, Insightful)
Rather Skewed Data (Score:2)
Re:Rather Skewed Data (Score:2)
I just talked to some recruiters yesterday in downtown LA, and
Re:Rather Skewed Data (Score:2)
I mean, even now, I'd guess you'd just start to see people with 3 years of experiance in Win2k in large amounts (as most places don't deploy stuff right away) yet Win2k is considered rather outdated...
I mean, sure I have 3 years of ex
Ted Shieh's Programming Job Comparison Graphs (Score:1)
Too bad -- getting a profile with the same measurement instrument over the entire bubble-bust cycle would have been very valuable for historians of technology and the politics of H-1b visas most particularly.
I suspect what the graphs would have shown was a far faster drop-off in the jobs for languages other than Java than for Java due primarily to the fact that the Indian CS diploma mills were set u
Employment analysis through VC investments (Score:2)
Other Examples... (Score:2)
He used to link to it in his /. sig but then, about half a year ago, he announced that he wanted to "move on with his life" and would sell his code, site and related goodwill.
I think part of the deal was that he was going to open the source but I never heard any more about it. Anyone know if that happened? Or of any FOSS projects working on graphing employment data?
Re:Other Examples -- Yes, it still operates (Score:2)
SkillsMarket [mshiltonj.com] is apparently still in business, and Hilton is still trying to sell it.
I used to search for an appartment like that (Score:2)
My script would poll the site every hour or so and notify me, if anything not seen before appeared.
Proper statistics from proper data (Score:2)
I'm sorry, but basing the "health of the IT employment market" on the number of job ads is flawed.
Employment consultants are somewhere just below Real estate agents and car salesmen, where their job is not just making an individual sale(read commission), but portraying a positive and bouyant market, while continuing the industry's success.
By glutting the market with jobs ads that many would not stand up to audit, achieves several things:
- A percieved shortage means that commissions remain higher.
Project management and Calif* (Score:3, Insightful)
Resume readers don't care if you're a Windows programmer, a UNIX programmer, a hardware designer, or a secretary. They want to see if you're a programmer, project lead, project manager, marketing manager, director, etc.
Things like Google, open source, wiki have leveled the playing field to where it doesn't matter if you study hardware, windows, AS/400, or UNIX. These things can all be learned by anyone at any time. In modern companies the skills at any given level of responsibility are being learned on demand as they're needed. Hardware designers one day are being used as UNIX programmers the next day.
Todays differentiation is in how much responsibility you're capable of having. Most resumes are being divided into management, sales and programming and as far as we can tell from the 36 checkboxes, management is the place to be.
Nothing new (Score:4, Informative)
Re:fp? (Score:2)
Re:Why is it ... (Score:2)
It's ok as a first pass, the 'raw' data is only useful to a point.
Re:Entry Level jobs (Score:1)
Re:The Problem With The Software... (Score:2)
Still, I liked his software. If nothing else, it's a nice start on what could become a bigger, more sophisticated project...
Re:The Problem With The Software... (Score:2)
You are right about the implications of future-looking trend prediction from these data, or any data for that matter. It's necessarily and always a backwards-looking exercise to try to make empirical p
Re:Obviously an American user interface student (Score:2)
Now if you had gone to an American college you would have prepared an executive sales pitch and sold your vaporware to the first sucker who came along. If you can't make it work, hire more sales people.