Exploring the Relationships Between Tech Skills (Visualization) 65
Nerval's Lobster writes: Simon Hughes, Dice's Chief Data Scientist, has put together an experimental visualization that explores how tech skills relate to one another. In the visualization, every circle or node represents a particular skill; colors designate communities that coalesce around skills. Try clicking "Java", for example, and notice how many other skills accompany it (a high-degree node, as graph theory would call it). As a popular skill, it appears to be present in many communities: Big Data, Oracle Database, System Administration, Automation/Testing, and (of course) Web and Software Development. You may or may not agree with some relationships, but keep in mind, it was all generated in an automatic way by computer code, untouched by a human. Building it started with Gephi, an open-source network analysis and visualization software package, by importing a pair-wise comma-separated list of skills and their similarity scores (as Simon describes in his article) and running a number of analyses: Force Atlas layout to draw a force-directed graph, Avg. Path Length to calculate the Betweenness Centrality that determines the size of a node, and finally Modularity to detect communities of skills (again, color-coded in the visualization). The graph was then exported as an XML graph file (GEXF) and converted to JSON format with two sets of elements: Nodes and Links. "We would love to hear your feedback and questions," Simon says.
Re: (Score:2)
Aside from the fucking annoying factor, it didn't even link anywhere or tell you where to buy them. Way to fail even at shilling, DiceDot!
No PHP? (Score:2)
I didn't find PHP anywhere.
Also, is there a way (or browser) to make the diagram larger?
Re: (Score:2)
Found the zoom. Still didn't find PHP. Curious.
Re:No PHP? (Score:5, Funny)
"I didn't find PHP anywhere."
If only...
Re: (Score:3)
Remember "You may or may not agree with some relationships, but keep in mind, it was all generated in an automatic way by computer code, untouched by a human."
How amazing computer code that has never been touched by a human. Organic software I guess, no possible way for human bias to creep in.
Re: (Score:1)
Re: (Score:1)
I didn't find PHP anywhere.
Well, the article *did* say it was limited to technical skills...
Also, is there a way (or browser) to make the diagram larger?
Case in point.
Re: (Score:2)
Re: (Score:2)
"Applicant must recognize a computer, and not attempt to eat the mouse. 2-4 years experience stuffing envelopes preferred, because we don't understand metered trifolds. Ideal candidate can tell Brioni from Armani by smell alone".
Dice supplying stuff to make a resume look nicer? (Score:5, Insightful)
However, I'm getting older and I learn another tech only if I actually need to use it for something I'm working on. So I'll be happily deficient in lots of languages I don't need. I'm certain I could be at least of average skill after about two weeks of most languages as that is my past experience with new languages. But don't tell HR. Today's software engineering world is so averse to training people it rarely considers searching for a veteran software engineer and letting him come up to speed on random techs.
If I spent a couple weeks on every tech I hear about for the sake of toying with it, I'd never get anything done.
Re: (Score:3)
Today's software engineering world is so averse to training people it rarely considers searching for a veteran software engineer and letting him come up to speed on random techs.
Not to put too fine a point on it but that's your own responsibility, not the company you work for.
If there is an aversion to companies training people. that' offset by the ease of learning any newer (or even older) technology, for free.
If you wait for the company to help you, you (and your career) will ossify. I have seen the res
Re: (Score:2, Informative)
Today's software engineering world is so averse to training people it rarely considers searching for a veteran software engineer and letting him come up to speed on random techs.
Not to put too fine a point on it but that's your own responsibility, not the company you work for.
Are you kidding?
Sure, that applies if you're a short-term contractor. But permanent staff ought to be getting a regular training budget from their companies. If you're not, then you're working for a bad employer.
If there is an aversion to companies training people. that' offset by the ease of learning any newer (or even older) technology, for free.
It may be free in the financial sense, but it still has a cost in time. A good employer will give you that time. And pay the costs if there are any. You may need to explicitly ask for it, but the bottom line is that they should be investing in their long-term staff.
If you wait for the company to help you, you (and your career) will ossify. I have seen the result when I was younger, the result is not good for your freedom to choose favorable working conditions.
My own experience: In the last yea
Re: (Score:2)
Today's software engineering world is so averse to training people it rarely considers searching for a veteran software engineer and letting him come up to speed on random techs.
Not to put too fine a point on it but that's your own responsibility, not the company you work for.
Only if you are a contractor. Otherwise, the answer is no.
Now that was cool! (Score:3)
Find and click on MongoDB. Then notice that "Database" is this tiny-assed little circle waaaaaaaaaaaaaaaaaaay off to one side.
Got a pretty good laugh out of it. :)
Re: (Score:3)
it's just analyzing the appearance of words in listed skills
actual database pros would not put "database" as an enumerated skill
maybe the kind of person who lists "windows" "internet explorer" and "microsoft word" as tech skills would, but such people would not show up in the data set analyzed here: resumes from serious professionals working in the tech sector
so it makes sense "database" would only be a tiny little distant circle
Re: (Score:2)
it's just analyzing the appearance of words in listed skills
so it makes sense "database" would only be a tiny little distant circle
So what you are saying is their chosen implementation for analyzing relationships between skills is incomplete?
Re: (Score:2)
are you saying there exists some implementation that analyzes every resume in existence perfectly? it's "incomplete" in the sense that any such effort is incomplete and imperfect by nature of the problem. your criticism is invalid, you don't understand the task if you expect completeness is possible
Re: (Score:2)
are you saying there exists some implementation that analyzes every resume in existence perfectly? it's "incomplete" in the sense that any such effort is incomplete and imperfect by nature of the problem. your criticism is invalid, you don't understand the task if you expect completeness is possible
The task is to determine which professional skills relate to each other (this goal comes from TFA, not just the summary). The use of resumes to accomplish this is an implementation decision. Any shortcomings of using resumes is not necessarily an inherent limitation to solving the task at hand. There are other ways to complete the task that do not rely on resumes at all, or that rely on them less.
For instance, you could use web search results to obtain data for your distributional semantics research. Websit
Re: (Score:2)
your alternative method is inferior as the specific request is tech *skills*, which you find on resumes, people speaking to their merits to get hired
not "tech appearing together on message boards," which indicated a whole host of relationships, relationship by skillset being far down the list
the simple fact is there is no perfect methodology so criticizing the methodology for being imperfect is without merit. and in articulating a yet even more inferior methodology in your latest comment i have to assume yo
Re: (Score:1)
Palo Alto's themselves are not that complex. The interface is an interesting attempt at being usable, and it's getting better. What I thought was interesting about the PA node was how many connections to Apache products it has. That makes me think that people are not happy with using Panorama to view/manage the logs and run reports against the logs.
I can sympathize.
Re: (Score:2)
RDBMS is one of the largest circles though. It is located in the orange area which seems like the traditional, popular, programming skills area. The RDBMS circle doesn't have as many links from it as I thought it would, but that would be how the cutoff is implemented. It's pretty nice.
a lot comments will probably nitpick (Score:4, Informative)
and i've seen my fair share of poor visualizations
but i think this is actually really well done, useful even
congratulations to Simon Hughes
Re: (Score:1)
As one who goofs off with some of the Graphviz tools (dot), how possiblewould it be to dowith this toolchain?
Re: (Score:2)
I agree that it is a cool visualization. I do think it's noteworthy to mention that the data was reverse-engineered from resumes, so the relationships all have the bias of what individuals _think_ will matter to hiring committee or want to be hired to do, vs what actual skills they possess.
Read the blog post again. (Score:2)
Read the blog post again. http://insights.dice.com/2015/... [dice.com]
"I think that’s pretty cool, given we’re generating that automatically from job descriptions posted on our site. We also tried using the resume dataset, but the results were of a lower quality, as the skills extracted from resumes can be from different jobs."
It was extracted from job-postings, which would only identify Schelling points in the hiring industry, not skill clusters common to people with certain desirable skill sets; in othe
Re: (Score:2)
Ah, you're right. Thank you for that correction.
No problemo (Score:2)
Ah, you're right. Thank you for that correction.
No problemo
I miss subtle stuff all the time. I rely on really strict semantics in place of "not trusting people", which I have a hard time doing (data is data).
Re: (Score:2)
but i think this is actually really well done, useful even
To the extent that I want to bookmark this, just for reference as to the tools he used ;-
Tool of the d (Score:1)
Quick! Hide it before they use it to generate H-1B targeting resumes having impossible combo's of skills to filter away citizens.
OT question - But dice related (Score:2)
When did the right panel get subsumed by ads? Had this been happening for long and I just hadn't noticed? I checked out the site in chrome to see the site without scripts, and holy smokes has it gotten bad.
I would not frequent this site if the normal experience was accompanied by a trojan condom video ad (among several others) on loop.
It's not very reliable data. (Score:4, Insightful)
It's not very reliable data.
They took the similarity vectors from the job postings, not from resumes, so rather than "what you're likely to know", they computed "what an employer is likely to want at the same time as wanting something else", and then declared that a similarity due to an already skewed cosine similarity metric. This happens because employers are more likely to copy other, similar job postings, or other job postings for companies in a similar business as them, or those of a company whose employees they wish to hire away.
They claimed that they tried using resumes, but that the resulting data was not as "clean"; uh... duh?
This visualization was not actually very useful, unless you are trying to design a resume to get yourself hired, regardless of your actual current capabilities.
Don't give anyone funny ideas (Score:2)
It's a polite way of saying "stop putting crap in your resumes". Don't encourage people to do more of it ;-)
In all seriousness though, have you ever tried to analyse unstructured text? It's hard. How would you realistically improve it? Do you start with a preconceived list of technology key words and count them in the resumes? People misspell words. Words have multiple meanings depending on context. Technologies change and you might miss key words. Or you could count recent Stack Exchange topic tags, but th
I've been writing code like this since 1985. (Score:2)
In all seriousness though, have you ever tried to analyse unstructured text? It's hard. How would you realistically improve it? Do you start with a preconceived list of technology key words and count them in the resumes? People misspell words. Words have multiple meanings depending on context.
I've been writing code like this since 1985. Then, it was in LISP.
It's actually trivial to me at this point. You end up with a meaning trie with differential probability vectors, and some of the roots wither away as you go down. Making a machine decision is harder, but not entirely impossible.
I get incredibly annoyed at people like Lazlo Bock who want to put everyone's resumes into a form that basically allows Google (Lazlo Bock works for Google) or other companies to magically allow you to come into a n
Dice guy runs software to generate graph from thei (Score:1)
This is news?
Go figure (Score:2)
C# is just as big as Linux? Whodathunk.
Buzzword association (Score:4, Interesting)
by importing a pair-wise comma-separated list of skills and their similarity scores ...
we’re generating that automatically from job descriptions posted on our site.
So what this really shows is how often the same two buzzwords appear together in a job description posted on Dice.
I found another comment in his report interesting:
We also tried using the resume dataset, but the results were of a lower quality,
I assume by "lower quality" he really means "people list every buzzword they can think of on the resumes posted on Dice".
Given the inputs I wouldn't expect any surprises in the results. But that said, it's an interesting project and they did a very nice job with the visualization.
All Oranges, no Apples (Score:3)
Re: (Score:2)
Bizare scaling (Score:2)
C has a nice big dot, but connects to only a handful of extremely broad-scoped nodes.
SQL, OTOH, has a tiny dot, and connects to just about everything on the chart.
Palo Alto?! (Score:1)
Since when Palo Alto is a tech skill?
Need to update my CV...
NoNerval Script (Score:2)
Electronic Engineering/Hardware Skills? (Score:2)
Where are the hardware skills?
'Tech skills' don't mean much without hardware.
Strat