95% of IT Projects Not Delivered On Time

An anonymous reader wrote " The Globe and Mail reports that 'A new report conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups are not delivering some number of projects on time or to the full satisfaction of the business executive.' The article goes on to discuss the reasons for this pervasive (perceived?) problem. The article mentions Info-Tech's reasons: unrealistic time frames, staff shortages, and poorly defined project scope. However, the article's author lays the blame with vendors."

Nah

[suso]

I'd say its actually closer to 100%.

Actually, it really depends on who they would ask in a company. Whether it be

the business executive (probably a higher estimate)
the IT middle manager (lower estimate)
the IT worker (who would think that they are on time)
or the customer (who sometimes have unrealistic expectations)

Re:Nah

[alexandreracine]

The other day I asked a programmer to bring me some cofee on the spot. The next day I had a new screen saver in java.
You may learn from this experience.

Re:Nah

[Qzukk]

The other day I asked a programmer to bring me some cofee on the spot. The next day I had a new screen saver in java.

Clearly a specification error on the customer's behalf. You should have requested 8 (or so) fluid ounces of liquid caffeine-bearing (I assume!) sustenance produced by passing hot water through the ground, blended beans of particular coffee tree species, while supported in a paper (or copper, or gold. Again, assumptions!) filter.

Re:Nah

[Anonymous Coward]

To be real-world, just as the server (ha!) brings the coffee to your desk, you should say that you changed your mind and want tea and a poppy seed bagel.

Re: Here's a relevant old joke

[nganju]

A man is flying in a hot air balloon and realizes he is lost. He reduces height and spots a man down below. He lowers the balloon further and shouts, "Excuse me, can you tell me where I am?"
The man below says, "Yes, you're in a hot air balloon, hovering 30 feet above this field. " "You must be an engineer", says the balloonist. "I am", replies the man. "How did you know?" "Well", says the balloonist, "everything you have told me is technically correct, but it's of no use to anyone."

The man below says, "You must be in management." "I am", replies the balloonist, "but how did you know?" "Well", says the man, "you don't know where you are, or where you're going, but you expect me to be able to help. You're in the same position you were before we met, but now it's my fault."

Re: Here's a relevant old joke

[CrazyWingman]

A man driving a backroad across country came upon a cowboy out driving cattle. He stopped, got out, and said to the cowboy, "If I can tell you how many cattle you have, would you give me your smallest cow?" and indicated the animal he desired.

"My smallest cow, you say? Well, why not, give me your estimate," replied the cowboy.

"Sir, you have exactly 400 head of cattle," the man said after some contemplation.

"Wow, that's exactly correct," said the cowboy surprised. So, the man walked over, picked up his prize and put it in his trunk. The cowboy, concerned for the animal, asked, "Now, if I can tell you your profession, would you let me win back the animal?"

The man, somewhat taken aback, agreed with a chuckle, "Sure."

"Sir, you are a consultant," said the cowboy without hesitation.

"Wow. That's pretty impressive. How did you know?"

"Well, you came out of nowhere telling me that you could give me an answer to a question I didn't ask for a price that was over the top," said the cowboy with a stern look. "Now give me back my dog."

--Wish I knew who to attribute this to :(

Re:Nah

[koa]

Ahh.. the old programmers plight:

Upon delivering the completed project, the end user simply states:

"Now hold on, this is exactly what I asked for.. But not what I wanted!"

Re:Nah

[bitchell]

In my experience when planning projects there is never ever enough testing and contigancy time.

Managers just seem to cut it out of plans because clients don't like paying for it.

Re:Nah

[Nos.]

As a developer I would agree that this is where most of the time lies. Allowing 1 week for testing andf fixes for an application with > 500,000 lines of code and interoperations with 4 or 5 different systems is not adequate. If the project took 3 or 4 months (at least) to build, don't expect it to be launch read a week later.

Re:Nah

[aoteoroa]

Many business people think building software is like building a house. When the framing is done, it's done. You usually don't spend weeks testing how the wall interacts with the drywall and foundations.

Non IT people just don't understand why code isn't written correctly the first time.

Re:Nah

[computational super]

I think that's exactly the problem with software expectations. They always assume that building software is like building a house, or a bridge, or a toaster. In other words, they always assume that building software is done by experienced people who've built nearly identical software systems before. And, no matter how many times we stress and repeat this, we can't get it through their thick skulls - if it's been implemented in software even one time in the past, it doesn't need to be implemented again. By definition, every single software project ever undertaken is a brand new set of problems to figure out. The more experienced we are, the better we know what to avoid, in general, but if there are no unknown problems, the program doesn't need to be written. This is true by definition. Designing and implementing software is more like proving/solving a mathematical theorem than it is like building a house - I doubt mathemeticians often get paid to figure out how to prove the pythagorean theorem.

Re:Nah

[Coryoth]

I think that's exactly the problem with software expectations. They always assume that building software is like building a house, or a bridge, or a toaster. In other words, they always assume that building software is done by experienced people who've built nearly identical software systems before.

When an engineer designs and builds a new bridge it is entirely possible that no bridge like it has ever been designed or built before. Sure, there are some base cases that just get churned out, but there are also big, new, creative designs that occur for bridges. How is that bridge engineers usually manage to not have their bridges falling down all the time? Well, for starters the designer doesn't run with a "build and test" mentality. There are formal methods for bridge design, and if you assume the properties of various basic components, there are methods to prove the stability and properties of the bridge. Did you know that there are formal methods for software design, and if you assume the properties of basic components (like hardware, OS , etc.) there are methods to prove the stability and properties of the software?

Yes, formal software methods are hard and time consuming compared to just building and testing. Formal bridge design is hard and time consuming compared to just building and testing.

For some reason we accept that software should be just thrown together rather than properly designed and proved. Yes, there are plenty of projects for which the level of formality I'm talking about simply isn't required - that's fine. My point is that there are plenty of projects for which the level of formality I'm talking about would be a damn good idea - yet it is never even contemplated let alone used. At worst you should be considering some level of formality for just those components of your system that are most critical.

Jedidiah.

Re:Nah

[Stevyn]

And that's why programmers aren't "Software Engineers". Engineers test their designs and then implement them. Programmers test their implementation because, to them, the code must be the design.

Re:Nah

[iabervon]

Actually, good civil engineers do have a "build and test" mentality, and the ones who don't have their bridges fall down. The thing is that their tests are performed on models and on the materials. The reason they don't build bridges and test them is simply that the cost and danger of having a bridge fall down is too great, so they do enough testing on smaller parts that when they actually build a bridge, they can be sure that it will work.

With software, the construction costs are negligable and the costs of a full-size model failing are also trivial. The correct engineering discipline in this case is to do the testing on the actual software.

In order to use a proper engineering discipline for software, you need to document what the code is supposed to do, all of the assumptions about the state of the system, and so forth. Then you need to test whether the actual code does what it is supposed to and maintains the constraints on the data. But you only do a little of this with theory and formal methods; most of it needs to be done by actually trying the code in various circumstances. Traditional engineering is done with a lot of testing of parts, testing of models you hope are similar, some simulation, some intuition, a bit of theory, and a big final inspection.

Now, it is true that a lot of the engineering practices are missing in the software world. But the things people actually do in the software world are a part of engineering; just not a complete method. And there is an unfortunate tendancy towards producing extra work which is not actually useful in the name of adding superficial similarity to different parts of the traditional engineering process. It is a bit like asking a civil engineer to produce a drawing of a new bridge not to scale without any information about the materials or the site.

Bridge not comparable to most business software

[GunFodder]

What are the design constraints of a bridge? It generally solves a simply stated problem - get X lanes of traffic from point A to point B. And failure is not acceptable. Due to these constraints millions of dollars and years of planning and construction are available. This might be comparable to the Shuttle software discussed a few weeks ago, but not any projects I have worked on.

Consider the construction of a house, or an addition to an existing domicile. Price is a significant factor, and the customer has many arbitrary constraints (call them "aesthetics"). In many cases the customer isn't sure what they want until they see what they don't want, which requires rework. There is no official certification process for most construction trades - only specialties like electrical wiring. So it is difficult to know how good a crew is until you work with them. Many (if not most) construction projects like this run over budget or over schedule.

I think writing business software is more like building a house. The constraints are unique and vague. The workers vary in their abilities. And the customers are cheap bastards. Projects in this environment have very little chance of coming in under budget and on time.

Re:Nah

[Daniel Dvorkin]

Not only do different people have different measures of success, some types of success are easier to measure than others. Success in software is relatively easy to measure: if the software has the features the customer expects, and it's usable and stable, then it's done. Success in "business" (i.e., what the executives do) is much harder to measure and subject to endless spin. I'd love to see a study of how often management fails to deliver up to expectations on time, but of course the people who pay for the studies are the same people who would have to be evaluated by some kind of objective standard ... and you can bet they're not going to want that.

Re:Nah

[Ghetto_D]

In other news, 95% of people in IT careers habitually read Slashdot.

The article says that 51%+ ARE on time (approx).

[khasim]

Well, more specifically ...

First, before anybody runs away with the idea that the failure of IT projects is rampant, it's necessary to look further into the study's findings. In a later section of the report, Info-Tech admits the majority of IT projects are in fact delivered on time, on budget and do meet expectations. So what's eating the executives?

And "majority", in this instance, would mean 51% or more.

So, 51% or more of the projects are delivered on time and on spec.

BUT if you've EVER missed a deadline or been off-spec, then you get counted as bad.

If you deliver 99 projects on time and on spec, but fail on 1 project, you're counted the same as if you failed on 100 projects.

Well, some projects inevitably fail to measure up, and getting good results most of the time isn't good enough, it seems. Failure is failure, and the infrequent missteps are tarnishing the reputation of IT groups in the eyes of business executives, the researchers say.

Right.... that tells me that the "answer" to this "problem" isn't technology. The "answer" is to have the IT managers take a few marketing classes and spread the bullshit to the "business executives".

"Only 5 per cent of enterprises told us they were always on time," the report states. "This indicates that 95 per cent of IT shops are not delivering some number of projects on time or to the full satisfaction of the business executive. This is a major contributor to a misalignment of business and IT."

Again, all it takes is 1 failure to be lumped in that group. No matter how many successes you've had.

"So, you've solved world hunger, the arms race, poverty, racism, nationalism and have single handedly established a viable human colony on Mars. But we're really concerned about that jay-walking ticket from last year. Let's focus on what you can do to prevent such a failure in the future, okay?"

Re:Nah

[mattspammail]

But Jobs would've scrapped the whole project if capacity was projected at 18. He'd have demanded more.

How many times have you seen a realistic bid lose out to a lower budget, quicker timeline bid that ends up late and over budget, often worse than the bid you had to pass on (but that was realistic from the get-go. In a bid situation, you're often times rewarded for your empty promises with an accepted bid.

Doesn't make it right. It just explains it.

Re:Nah

[2nd Post!]

Yes, and in this case if Jobs is the customer, then his programmers/developers have the duty of customer relations. They should have promised to him 16, given him 25, and then marketing can project 18.

Not on time again

[alex_guy_CA]

I was going to be the first post, but I could not get it in on time.

Understanding your art

[BWJones]

95 per cent of information technology groups "are not delivering some number of projects on time or to the full satisfaction of the business executive."

Could it be that marketing is always overselling the product? Seriously. I cannot count how many times I have heard (in the past now I am in science), "oh, yeah....well, you need to include feature X because we told customer Y we already had that feature". This is often followed up by the engineer muttering under his/her breath "Dumb jock. :-) I say that joking, but have seen discussions like this almost erupt into fist fights as the sales staff makes promises to customers that are either 1) blatantly false or 2) concepts under development and are nowhere near "production".

So, this is another example of why pre-announcing products is a baaaaad idea. Treat your customers with honesty and announce the product when it is ready and not before. Again, this is why vaporware only serves to irritate your customers and build expectation of a product that is not always delivered.

I also believe the fundamental problem is that managers these days (in many cases) no longer come from the ranks and are not engineers. So, they do not always understand what is involved in 1) building the codebase 2) testing code base 3) proper interface design 4) end user testing 5) documentation 6) making sure it does not suck.

The last point is where most executives seem to get hung up. More often than not in most companies, executives really have no idea of what makes good code and all too often, what makes a good product. Come on now, a good portion of executives can barely use their personal computers to answer email or browse the Internet. When you have companies run by executives and managers that have come up through the ranks, you are much more likely to get quality which often is much more important than meeting an arbitrary deadline.

VERY true

[JeanBaptiste]

"Could it be that marketing is always overselling the product?"

I work with OCR/ICR technologies. NO SALESPERSON should EVER be allowed to sell this without taking a month long training course about what it actually does.

I can't count the number of times customers were expecting 100% accuracy because thats what the salesman sold them.

Re:Understanding your art

[Sgt O]

"...they do not always understand what is involved."

- you hit the nail right on the head!

I'm working on a project right now (software is installing as I type this) where I'm supposed to migrate an existing web server to a new datacenter accross the country.

The PM's take on the whole thing was "All you have to do is:"

- Load the software on the new server.
- transfer the data
- ship the server to the new location
- have the server racked and powered up


I could tell he though I was just being difficult when I told him:

- While we're at it we should upgrade the app that's running on it since it's no longer supported.
- If we upgrade the server will be running a different web server and therefore will need a new ssl cert.
- Get the network guys engaged so they can punch what ever holes are needed in the firewall.
- We'll need to do some level of testing.
- Notify the end users that the look-n-feel will change/new applets will be downloaded, etc.

And now, as far as upper management is concerned, I'm the one that is behind schedule...

Re:Understanding your art

[Phrogman]

I recall when I was working Tech Support for a company, hearing a Sales dweeb asking one of the programmers "Do you remember that utility program you mentioned to me? I hope its available because I just sold it to a customer."

The utility program mentioned supported one of our products and was for inhouse use only. It had been whipped up quickly in the devs spare time, had no documentation, no time spent on QA, was not an official product of the company, and had a completely unfriendly UI because the dev had developed it piecemeal for his own use. Needless to say once it had been *sold* to a customer as a feature that attracted their interest enough to purchase our product, it became an official product and was quickly rewritten to be more presentable, but that developer told me he would *never* mention anything to a sales guy again because they couldn't be trusted.

I have seen sales people sell a product based on a feature that they assured the customer the product offered, then once the phone call was done and the sale completed, checked with Tech Support to see if it actually did offer the feature they sold it based on.

Re:Understanding your art

[Bellyflop]

You know, I used to think of sales as been the annoying bastards over there. But now I've been dating a salesperson for a couple of years and I think that I see what the problem is. Her managers give her unrealistic expectations. They don't care if there's a market for the product or if companies have already set their budgets or whether the buying market is just soft for some other reason. But then her job gets threatened unless she hits "her" numbers. They really aren't her numbers - they are her manager's number divided by the number of people he manages. Now it's not all his fault of course - he has a manager too who is doing the same thing to him. And so it goes up the chain.

Usually, it's because the board of directors has written a clause into a CEO contract that says that if he hits certain numbers, then it triggers a stock grant or a bonus. So of course, he's pushing for it. The board has the shareholders wanting big returns so they are pushing too - the board members have a financial stake in the company too of course. And then you have the analysts who give everyone in charge an incentive to say that growth is going to be high so that they can offer a "buy" rating. But none of these people actually have to do any of the sales calls. Nor do they have to build the product.

So from the developers point of view, it's the salesperson being a pain in the ass. But I think the problem is really that people in charge have the wrong incentives and don't have the balls to say "hey, those numbers are unrealistic."

Re:Understanding your art

[DoofusOfDeath]

And a good portion of the younger developers I've dealt with have no clue about marketing or financial pressures on getting a product out that's "good enough".

Don't get me wrong: I'm a developer, not an MBA-type. But I've run across primadonna coders that get so hung up on thinking of their coding as a form of aestheticly-oriented art, that left unchecked they'd bankrupt the company or totally miss the market window for the requested feature.

I also see this in computer science research, which is truly th

Re:Understanding your art

[ajs]

"

95 per cent of information technology groups "are not delivering some number of projects on time or to the full satisfaction of the business executive.""

Could it be that marketing is always overselling the product?

Why is everyone responding as if this were about software companies delivering product to external entities? First off, the article cites "IT groups" and "IT descision makers".

Ok, so that said, here is why it happens: IT departments in most large companies are operating in a grey area between d

7)

[elgatozorbas]

...1) building the codebase 2) testing code base 3) proper interface design 4) end user testing 5) documentation 6) making sure it does not suck

7) enter the market too late.
I have never worked in a company (only academic... sweet...), but can imagine 7) is why 1) to 6) are neglected.

Re:Understanding your art

[BWJones]

I don't think you fully grasp the nature of capitalism.

There is always a market for well engineered products that are designed and built with passion. These companies may not necessarily be McDonalds, but they can certainly be companies like Apple Computer, Porsche, Canon, Oakley, etc...etc...etc....

This just in.

[Anonymous Coward]

95% of the time, the business changes their mind about the project and/or doesn't know what they want, anyway.

Re:This just in. Project Bloat

[dfn5]

95% of the time, the business changes their mind about the project and/or doesn't know what they want, anyway.

I suppose this was modded Funny because it is true. I have not seen a project yet that hasn't resulted from project bloat. As the project progresses new deliverables are tacked onto the end. One can try and have the project plan set in stone at the beginning, but it never works because all the new stuff is "critical to the business".

So how can any project meet it's deadline under these ci

Phew!!!

[Skraut]

I'm still working on a project due this past Monday.

Thanks /. a copy of this story now sits on my boss' desk.

Re:Phew!!!

[EllisDees]

One of mine's due today. Ain't no way it's going to be done 'on time'!

No coincidence

[elgatozorbas]

Thanks /. a copy of this story now sits on my boss' desk.

Probably also thanks to /. your project is late in the first place...

misleading headline

[wankledot]

There's a huge difference between 95% of firms not delivering a project ontime, and 95% of all projects being late.

Re:misleading headline

[geminidomino]

Welcome to Slashdot. ;) Smarmy comment aside, I thought the exact same thing when I compared the headline to the blurb.

Re:misleading headline

[Pxtl]

ME TOO! /aol

Re:misleading headline

[14erCleaner]

True, but think what this implies: 5% of development groups never deliver a project that is late. Amazing...

Re:misleading headline

[Iamthefallen]

Canadian information technology groups can't seem to get IT right.

Not to mention, that 95% Canadian is only like 50% American.

Re:misleading headline

[Phat_Tony]

Yeah, that's the first thing I noticed. If there's a company that does 100 projects per year, and delivers 99 of those on time, then one project is late, so that company goes in the "delivers projects late" pile.

Why not have a headline "95% of IT employees will lose their jobs this year," based on an article showing that 95% of IT firms will terminate at least one employee this year.

While highly plausible, this 95% statistic is worthless, and the headline is not at all supported by the article. Actually

Merge it.

[mfh]

The Globe and Mail reports that 'A new report conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups are not delivering some number of projects on time or to the full satisfaction of the business executive.

The answer to this problem is change, and isn't change always the answer?

Consider if you will for a brief moment the vast difference between the average executive and the average programmer. Programmers are generally broad-picture thinkers who solve largely complicated problems that regular folks can't possibly wrap their heads around. The executive runs on a schedule and uses reports and correspondance to understand what is going on, because business folks have to judge their employees and projects.

These two groups are forced to work together, and we expect good results? We need someone to interpret between these two groups! The HR department can't regularly serve in the interpretive capacity, but perhaps they should.

Managers generally don't want to give the programmers the whole picture, because management often believes that they are superior in rank to programmers, placing the programmers on a need-to-know-basis, only. Huge mistake.

What programmers and managers need to do is realisticly approach their solutions together. They need to be honest with each other. They need to share each other's thoughts and feelings about the subject matter. It's not happening today.

The programmers need to come to the table and care about their customers a little more. The managers need to come to the table and care about their programmers a little more. The customers need to be more specific and realistic about how far their dollar can go. Then deadlines will be met and promises kept and successful solutions provided to customers.

I encourage a no-holds-barred approach to project management. The superior product is developed using the Agile [agilemanifesto.org] method.

Re:Merge it.

[millwall]

Programmers are generally broad-picture thinkers who solve largely complicated problems that regular folks can't possibly wrap their heads around.

Please...

I know it's Slashdot I'm reading, but I didn't know readers had that high thoughts about themselves.

Re:Merge it.

[Oloryn]

Managers generally don't want to give the programmers the whole picture, because management often believes that they are superior in rank to programmers, placing the programmers on a need-to-know-basis, only. Huge mistake.

I call this the myth (or culture) of managerial superiority. It tends to go "I'm your boss, therefore I must be smarter than you are." It's one of the reasons that some managers come to resent technical people, whose jobs require that they be smart, and who are not usually reluctant to show those smarts. Maintaining a culture of managerial superiority is difficult when your subordinates often demonstrate that they know more than you do. Too often, the result is either denigration of the subordinates knowledge (You were hired for your expertise in X, but your manager keeps on overriding you on matters relating to X, leading you to wonder "Why did they hire me if they won't pay attention to the knowledge I was hired for?"), or denigration of the importance of subordinates knowledge.

The fact is, management requires a different set of abilities, not necessarily a superior set of abilities. Acknowledge that, and you've at least opened the door for each side to recognize the others talents and use them together.

And in related news...

[Richard_at_work]

95% of IT project specifications are what the user WANTS rather than what the user NEEDS. When they get what they want, and discover its not what they need, of course they wont be satisfied.

Where's the Obvious Tag

[Shadow Wrought]

when you need it?

Oh, its late...

In other news

[kevin_conaway]

A study shows that 95% of clients don't know what they want.

Re:In other news

[leerpm]

Mod parent up. While I don't want to see a return to the excesses of the late 90's, I think some blame has to be layed at the feet of management and customers too. Ask for more features, with fewer resources to implement them and guess what slips?

There is a good saying: Fast, Cheap, and Good. Pick two. You can't expect to deliver great software on time with a lack of appropriate resources.

Re:In other news

[MarkGriz]

"Film at 11"

Actually, it won't be until 11:30
We're running late

Slashdot Already Proves This

[teiresias]

Anyone who reads this site knows that this site is (probably) the reason 95% of IT Projects not being delivered on time. My PM just lost 2 minutes to this post when I could have been writing Rose models like I'm suppose to!

95%...

[grub]

... and the other 5% never ship at all. (ie Duke Nukem Forever)

Re:95%...

[FidelCatsro]

People always confuse this one , you see the game is just called "Duke Nukem" again and Forever was the expected development time

Comment removed

[account_deleted]

Comment removed based on user account deletion

Up to 100% off

[CosmeticLobotamy]

95 per cent of information technology groups are not delivering some number of projects on time.

Zero's a number.

Title is misleading...

[mfender9]

TFA does not say "95% of IT projects not delivered on time". It says "95 per cent of information technology groups are not delivering some number of projects on time..." Big difference.

Not Suprising

[pianoman113]

I'm sure that 95% of IT shops have little to no software engineering. Until IT Depts. as a whole start to realize the value of things like defect tracking, estimation, time tracking, coding standards, and the like there will be no improvement in this area.
It is not necessary to impliment the full-blow SEI CMM to produce good software on-time. If developers would take the time to first make reasonable estimates of how long it will take to finish a project and then track how much time they actually put into

My current project..

[carpe_noctem]

..is almost done. Should be ready by the end of the week (crosses fingers)! I'm just about ready to implement the final touches on the UI. Right after I read slashdot, that is... and maybe check my email, and some of the other boards online. Shit, after that, it'll be lunchtime!

Yeah, sometime after lunch, it'll almost probably definitely maybe get done. I promise.

This is why

[nagora]

Us: We can do that for $x in 12 months.

Customer: But Joe Bloggs says his company can do it for $x/2 in 3 weeks!

Us: That's simply not possible.

Customer: Well, for that sort of savings we're going to give them a try.

11 months later and $x^2 later they're still waiting for Bloggs to finish but by then we're on the dole and Bloggs is laughing all the way to the bank.

TWW

Re:This is why

[Psychotext]

Couldn't agree more. We've had plenty of conversations like this:

Customer: Joe Bloggs says his company can do it for (stupidamounts) in (notime)

Us: OK, we've got no problem being competitive. Can we just check that their system is being designed to the following standards ?

Customer: They didn't mention it, but I'm sure it must be included.

Us: I see, and is our support agreement in line with what they are offering?

Customer: * Thumbs their document * - Don't see anything about support. But they're cheaper - You must be trying to have us over!

Us: ...

...and yes, I've been accused directly of trying to "Have one over" on a customer because our prices were higher than someone else. In those circumstances it's better just to walk away as you're never likely to have a successful relationship on your hands. :) It's the age old story for the customer, pay peanuts - get monkeys.

I remember on my first web dev

[Ohreally_factor]

I remember the first time I heard someone ask, "Is this deadline hard or soft?" I was just the video guy, so I kept my mouth shut and didn't laugh. The lead didn't even blink, and said, "Well, it's mostly firm, but a launch date hasn't been announced publicly, so we'll see at the end of the month." Good thing I didn't laugh.

Re:I remember on my first web dev

[MisanthropicProgram]

This reminds me of my PM class I had.
In a nutshell, the instructor said that the requirements, then the specs, then design, ..., then a project plan, then an end date is to be derived.
The guy next to me who was a PM just shook his head and said, "No, the end date comes first and then we figure out how to get it done." I have had the same experience in my decade+ of experience.
The instructor said that's why most projects fail.

Not projects

[Council]

Well, quite obviously, it's a problem with how deadlines are chosen, combined with peoples' natural work tendencies (where they work up to a deadline and get it done sometime soon after).

This isn't "oh no evidence of every company doing some particular thing wrong 95% of the time." It's a general property of this type of project.

To put it differently, if 95% of people are voted above 9 on hotornot, it means there are some parameters to the voting or the choices or the statistics. Not that the world is incredibly attractive.

If an IT department never, ever is late...

[TechnoWeenie]

Then they are either padding their project plans way too much, or are really not trying to do anything new.

Adding requirements

[Therlin]

It doesn't matter how much time I spend in the pre-planning stage, meeting with users, coming up with all the needs and feature requests. Once the project is in progress, and specially as you start demoing it, the end users will start adding requirements that had never come up before no matter how detailed you tried to be. And of course all the "wouldn't it be nice if"s.

Sometimes some of the new requirements or wants involve going back and rewriting a good chunk of code, or changing the DB design, etc, no matter how carefully you wrote your code and flexible the code may be.

Re:Adding requirements

[cmburns69]

Our mantra is:

"Customers don't know what they want to begin with. So just build something, and they'll tell you how to fix it."

This works very well here, especially since we're not afraid to tell them that each fix costs $$$

Well duhhhh

[03Cobra]

Thats cuz i spend all my time on /.

Salespeople are the root of all evil

[jerometremblay]

False claims by individual salespeople suggest a short-sighted approach aimed at getting the deal done, hitting the sales target and moving on to the next prospect.

Case closed.

Correction: 95% of Schedules are Wrong

[Flamesplash]

I'd be morelikely to conclude that this means the schedules are simply wrong. it's so difficult to plan a correct schedule, and asking developers how long they think XYZ will take doesn't really work well.

Have there been any advances in scheduling technology? Like profiling developers over the types of software they write, etc.. ?

Re:Correction: 95% of Schedules are Wrong

[serutan]

Exactly the point. I don't know why this isn't totally obvious. Once somebody makes an estimate and establishes a schedule, no matter how they arrived at it, everything that doesn't measure up is deemed a failure. In almost 30 years of programming I've rarely seen an incompetent project team goof off or in other ways screw up a project. But I can't think how many times everything is going just fine, it's just not going the same as somebody 6 months ago thought it would. The staff is put under tremendous pre

Re:Correction: 95% of Schedules are Wrong

[qwijibo]

One trend I've noticed with increasing frequency is for a suit to push for an "aggressive schedule" only to move on to another organization before the results of their actions can be felt.

I spent most of last year on a project like this. I personally spent almost 3200 hours on the project, and I know the rest of the staff was working 50-60 hour weeks the entire time. We managed to bring the project to a successful completion on time, but our Manager and his Director were both gone (to the same other part

Canadian I.T. projects?

[jacobcaz]

That 95% not-on-time rate is for Canadian I.T. projects. So once you factor in the conversion it's only 78% for US I.T. projects.

not just IT

[venicebeach]

I don't think this is specific to IT. I bet 95% of all human deadlines are missed....

In my experience

[Fnkmaster]

This is almost always because the scope of a project changes between when it's initially described and when it's delivered. A majority of projects I've been involved with fall into one of two categories:

1) requirements are agreed on, seem reasonable enough, but then detailed specifications are drawn up and client keeps pushing to add more things to specs until you have a 120 page document that will take 2 years to deliver. If, however, you tell the client it will now take 2 years vs. the 5 months you said when you were looking at a 2 page requirements document, they will cancel the project, and if they weren't paying for the requirements phase, forget about collecting any money for them (why you should always get paid for all phases of project planning). Since you can't do this, the client will eventually get upset, even though it's their own fault.

2) Project is delivered very early in prototype form, only to have the client say they want 50 more features that they forgot to describe in the requirements process, but they refuse to pay more, and refuse to acknowledge that the time frame must be pushed out to accomodate their new requests.

Yes, I've managed client relationships before and large (multimillion dollar) implementation and customization projects. I have reasonably good people skills, and still found these problems generally insurmountable when my client's company had a completely nontechnical person in the role of project sponsor and manager on their side.

The best predictor of success of a project in my experience are the lines of reporting and control in the client's company, and the existence of some technical knowledge in a position of responsibility and authority. If their CIO or President or whoever is the ultimate decision maker has a senior arhitect or tech VP that knows their shit AND functions as a trusted aid in the decision making process, these issues can usually be bypassed. If there is no senior source of technical knowledge, you can kiss the entire project's ass goodbye. Try to get as much money as possible from the client while it's going on, but forget about the project being "successful" in how its received by management.

Re:In my experience

[Psychotext]

When I started out on my own I made a very early decision to charge 50% upfront on all contracts. It's very hard to pull off but overall it's been very successful. Process goes a little like this:

1: Budgetary Quote.
2: Requirement Gathering (We assist)
3: Outline Specification (Huge number of meetings prior to this point).
4: 50% Non-refundable deposit.
5: Detailed system bible.
6: Changes to system bible (Chargeable).
7: Develop / Change / Rinse Repeat.
8: Finish Project (Final 50%)
9: Support

We have agreed to refund one deposit in the last two years (We screwed up their requirement gathering). We have had two clients pull out and lose their deposits. Everyone else has been happy, and through good communication hasn't had a problem when we have charged them for modifications to the system bible half way through the project.

Of course, the worst part of doing it this way is when some ass wastes weeks of your time and walks away with your outline spec (No doubt to give it to Joe Bloggs or use it in-house) having paid you nothing. It's probably worth noting that we don't always do the full specs if we don't trust the customer. :)

Oh, and one final bit of advice - GET TO THE TOP PERSON IN THE CHAIN! If someone is likely to override the decisions of the person you deal with in the company, start involving them. Even if it's just an email or a meeting to make sure they are happy with how things are going. Avoids some serious grief at the end of the project when you find you completely missed out on the features that the purse holder wanted!

Methodologies and the lack of it

[cOdEgUru]

First of all, the report focuses with Candian firms.

Second of all, rather than delving in to the varied array of processes and methodologies prevalent in the software development arena and reviewing advantages and disadvantages of each, the report goes in lenght talking about Vendor issues. I dont have a clue why.

Right from the Waterfall approach, or having no approach as well, we have RUP, XP and a mix of each playing itself out for the last few years. In my past projects, I have implemented each or a customized version of each and has varying levels of success. The biggest issue that I have so far seen is the lack of adequate knowledge in each methodology that someone who starts implementing any approach, either loses interest and resorts to a quick fix at which point the process starts to wither and die. More over, to some of the developers I have worked with, its not process, its documentation. CRC Cards is not a design tool, its documentation, its impediment to writing code. Much of it has to do with no academic background in best design or coding practices. They have heard of design patterns, and probably has used MVC to death, but when it comes to designing a system, its back to "lets start writing code rightaway and maybe the design will flesh out over time". The system gets built, but it suffers from Simplicity, it has very low quality.

I have seen a lot of firms talk about having a process, they love throwing RUP in the air, but nary a project which has successfully or much less adequately implemented any sort of process. They talk about Use Cases/ User Stories, but when the project gets kickstarted, they resort to SRS documents or less. And then they forget to adequately keep them up to date. Many even have Requirements management tools like Requisite Pro which hasnt seen daylight yet. Fuck the tools, atleast have a damn process. Many dont even define success at the outset of a project, no acceptance criteria either. They end up in a deathly downward spiral towards absolute failure dragging their clients with it.

I love XP, I love it for several reasons. Having a Client involved during all phases of the project, much less have a day to day interaction with the team is a phenomenal idea. I have had great success with this, but it becomes a bit of a problem, when the project is outsourced. No amount of communication (documents/mail/phone) can stand up to having a person next to you to tell you whats important and whats not. I love pair programming even though I could never fully implement it, when the client doesnt believe in it. I have tried pair design and have had great success. I have a few developers reviewing Test First Design and this has limited success as well. Limited, since the developers rebel, having low discipline and patience to write tests as they code. They are tutored and trained in the ways of "lets code first, maybe manually test it later and let the QA worry about it".

I would like to hear about other's experiences as well.

Re:Methodologies and the lack of it

[meburke]

Excellent point....

The first methodology that should be looked at is the scope of work. If you're building a house and the customer wants a change from the original plan, then the customer is responsible for any additional costs and delays. (But at least there IS a plan!) Too often, in my experience (and I will have been a programmer for 40 years in July), people don't take the time to actually build a plan for the project. My best argument for UML is the possible careful analysis UP FRONT, especially Use

Far To Many Variables

[EXTomar]

I don't see this as a human failure but more of an unrealistic expectation with limited resources. In short any project will encounter any number of problems along the way most of which were never dreamt of during the meeting room planning phase. The best planners try to learn from past experiences to plan for the semi-cyclic stuff but in the end there are too many variables that will interfere. From the mundane (a server blows up taking out internal development for a couple of days) to the rare (a devel

or

[FidelCatsro]

Could be seen as 95% of IT projects not given enough time for completion by marketing

Re:or

[broothal]

Yeah - marketing departments are the roots of all evil. I once was asked to estimate a project, and the marketing droid said "by the way - your estimate can't be longer than until October, because we already booked the TV commercials". And that's not from dilbert - that's a true story.

Optimism

[delta_avi_delta]

We had some very good project management classes in college. During one, the lecturer asked us to give answers to a pop-quiz, but we could give the answer as a range, for example, for "How long is the Danube" you could guess "1000-1500km" with no limits.

Even with the benefit of fixing our estimates as we liked, the entire class did very poorly. The morals of the story were

a) people are over optimistic in the accuracy of thier predictions

b) even, in our case, when we could have given zero-to-infinity ranges, we tied ourselves to restrictively narrow frames.

I thought it was fascinating, it's one of the few classes I remember vividly.

story

[justforaday]

Wasn't this story supposed to be run yesterday?

"SOME number" -- Useless statistic

[Enthrash]

They state:

"95% are not delivering some number of projects on time or to the full satisfaction of the business executive."

This could means that 99% of the projects attempted were delivered on time by all IT groups which is a more, or it could mean 99% of the projects were delivered late. By using the phrase "some number" this statistic is utterly general, and wholely useless.

Oddly enough, later in the same report they state "the majority of IT projects are in fact delivered on time" which really what counts.

The fact that IT groups do not deliver on time 100% of the time should be no surprise. The fact is that there simply aren't any professions which bat 100.

Botton line, stat is completely pointless.

Rich...

In my experience ...

[johnhennessy]

(Stolen shamelessly from someone else...)

A Product can be:

1) On time
2) On budget
3) Feature complete.

Pick any two.

The Three Failures of Engineering

[stlhawkeye]

In my experience, there's three major reasons why projects aren't delivered on time or to the satisfaction of the end user.

1. Failure to Understand Business Need

2. Gathering requirements is fine, but there's rarely a strong feedback loop between engineering and business. For example, I see requirements like this a lot:

   Each customer in the database will have a unique ID. This seems like a good requirement. Adding any more detail moves you into the realm of high level design, right? Well, maybe. In any case, engineering needs to ask important questions at this point. This was an actual requirement I got at a previous job. We assumed, erroneously, that this just meant that the data stream we received and processed would contain a unique ID for each customer and that it had to go into the database. The truth was that customers did not have unique IDs, the business was expecting us to engineer a technique by which to assign them one. For various reasons I won't go into, simple starting at 1 and assigning each user the next available number wasn't sufficient. This misunderstanding didn't come up until late in the project, and it took almost two weeks to understand what all had to go into the unique ID, and then engineer a process to calculate and assign the ID to each customer. It sounds like such a simple thing, but overlooking the simple things is what puts projects behind.

3. Trying to Solve Training/Documentation Issues via Engineering

4. There's a problem. We found a flaw in the program. If the user does X, then Y, then Z, then X again, and then Z twice, and then Y while holding down the shift key, the program behaves funny. Well, all of those functions are legitimate uses of the software, and this particular path through causes problems. Not crashes, not erroneous results, just unexpected results. Well, that's a documentation issue. Or a training issue. "What if the user goes in and manually hacks the URL and screws up our query string?" Well, then they get errors or bad results! Engineers often want to solve these problems ("take away the URL nav bar!" "But we have to support IE, Netscape 4.7, Mozilla, and 4 other browsers, plus their Macintosh and Linux versions! What a testing nightmare!") in code, but at some point it's best to accept the risk and just document the hazard. Every problem doesn't need to be solved by engineering.

5. Scope/Requirements Creep

6. "Johnson! Real quick, can you add a Print option to this right-click pop-up menu?" "Sure, no problem." Congratulations, you're part of the problem. Yeah it'll only take a few minutes to code it. And update documentation. And update test cases. And test it. But wait! If Print is on the pop-up menu THERE it ought to be available over HERE too! Changing code costs more time than just the few minutes you spent changing code. Pile up a dozen trivial requests and suddenly you've added hidden weeks of effort to the project.

Join me next week as I discuss the problem with dumbing down your architecture so that you can hire morons for less money to maintain it when all your best talent gets fed up with their 2% raises and quits.

www.dilbert.com

[rexguo]

^^^ enuff said.

But what about non-IT projects?

[DoofusOfDeath]

How many non-IT projects are done on time?

Let's see... Boston's Big Dig. Nope. Designing a new aircraft carrier... nope. Red Sox winning the World Series... badly delayed ;)

I wonder if in general it's creative projects or maybe highly complex projects that suffer lousy under-estimates for completion dates. Many software projects (i.e., MS Longhorn) are both.

On an unrelated note: I wonder how project planning estimate accuracy correlates with the experience level of the person making the estimate. Because if the IT industry tends to burn out the young people os that there are't many older IT people, that could contribute as well.

Re:But what about non-IT projects?

[Oloryn]

On an unrelated note: I wonder how project planning estimate accuracy correlates with the experience level of the person making the estimate. Because if the IT industry tends to burn out the young people os that there are't many older IT people, that could contribute as well.

For a good look at this, see Tom DeMarco's "Controlling Software Projects". One of Tom's points is that one reason we are so bad at estimation is that we so rarely do it. What we often oall estimation is often actually more like ne

Algorithm: why projects are not delivered on time.

[crazyphilman]

if(internal project){
if(doneByEmployees){
if(manager.clueless){
if(manager.schedule.isRidiculous()){
project.lateness.reason = "Employees came, they saw the schedule, they laughed, then they did the project in its natural timeframe";
} else {
project.lateness.reason = "Employees came, they saw the schedule, something went wrong, all hell broke loose, then they finished the project as fast as they could, considering";
}
} else if (manager.isEvil) {
project.lateness.reason = "Employees hate him anyway and ignored his sadistic schedule. General sentiment of 'fuck it, I'm on salary' prevails, manager crashes and burns, employees get reassigned, everybody sings 'ding dong, the witch is dead' and goes to Starbucks for coffee";
} else {
project.lateness.reason = "Unforseen problems arose, employees did their best to deal with them, stakeholders wouldn't budge on schedule, so the project was late.";
}
} else { // CONTRACTORS! HERE BE DRAGONS!
project.lateness.reason = "maximization of billable hours (duh)";
}
} else { // VENDORS! GOOD LORD, HIDE THE WOMEN, KIDS, AND FARM ANIMALS!
project.lateness.reason = "Incredible, absolutely amazing scope creep, maximization of billable hours, platform/system/vendor changes midstream, refusal to engage in technology transfer as extortion technique, total screw up of vendor, outsourcing to country without indoor plumbing (but assume they can handle high technology), etc, etc, etc";
}

Did I miss anything? ;)

Immature Industry

[awerg]

Duh! All Enterprise IT projects are like custom cars of the 20's and 30's. Each one is hand crafted by a small group of skilled craftsmen (coders, artists, project managers, etc.). No two implementations are the same and the users are just not that sophisticated. Every Enterprise Graphical User Interface I have seen is just an electronic version of "how we have always done it".

Outsiders (the clients upper management) will apply time measurements to IT projects as if they are building bridges or building cars. These are unrealistic because both those industries have been around for 50+ years. The IT industry is immature and still growing. Just think how many languages you have to know just to do your job? Or how many versions of compilers, or how many changes in OS, dll, registry, configs, /bin, /sbin, kernel, etc. that we have been through in the last 5 years. 10 years ago most Enterprse systems did not exist, were running windows 3.1, consoles or just purple ink memo's.

Come on, until there is standadization in tasks (what the client wants to do), interfaces (how the client wants to do it) and tools (how we make it so), all IT projects will be optimistically scheduled and all projects will be under time pressure from the beginning.

I won't even start on budgeting of projects...

Ok, I'm down off the soap box.

Obviously

[bigpat]

I blame Slashdot

Unrealistic management pressure

[SlashDread]

Get this:

- I scope a project
- I pitch it to mngmt
- Their response -always- is "be quicker"
- My response is, quicker means either: less qualiity, or more money, ypou pick.
- They say, you get neither.
- Ill say, sigh, Ill -try-
- It gets delivered on my original timescale
- They fuss about the project being "too late"

Do what -smart- project managers do: overestimate everything.

Re:Unrealistic management pressure

[Run4yourlives]

"Ill say, sigh, Ill -try-" Hence the cause of all your problems. If they aren't willing to commit, why do you accept the responsibility?

Yep.

[JustNiz]

I'm a software developer now resident in the USA for about 5 yrs. Preivious to that I've been a developer and consultant working all over europe.

In my experience, a much higher percentage of European projects are delivered on time than US ones. The simple reason is that in Europe, engineers are more respected and are usually tightly involved with the requirements gathering/planning phase.

Unfortunately in the US it usual practice to keep engineers away from clients and only involve them when everything is already agreed on paper. This means that the engineer gets a garbled requirement to work from, and the technical decisions have already been made/commited to by someone without any technical skills (i.e. sales or management).

The net result is that the engineer is expected to implement someone elses bad design that usually misses important aspects or doesn't address the actual problem, in a hopelessly optimistic timeline. Furthermore god help the engineer if the customer isn't kept happy.

There is No Solution: Godel Incompleteness

[SparafucileMan]

Look....

Programming is math. And math is hard. For starters, 99.9999% of all math is random and impossible to understand in the technical, math sense (proove) (extrapolation from Godel and Turing and Chaitin and Cantor...and yes, it really is 99.999....%).

When you sit out and design something to match a real-world process, you do fine. Then, you change something, you'll never know the impact of that change. You can't design for it in advance. A change of 1 character in the design could litterly require the entire code to be rewritten. You cannot prove how big your impact will be. Ever. That is why programmers get frustrated when customers change their mind "o but I just want it in this differnt order," "godamnit now i have rewrite half the loop..."

That is because good programming isn't just about being "smart", or about "planning"... most of the times, you are running against the fundamentals of understanding the algorithms in question, even with something as simple as lists or hashs or whatever.

The fact of the matter is is that programs are late because of bugs, and there are bugs because of the fundamentals limitations of math/the universe. You can't just smart them away cause you're some genius coder. All the genius coders write in their own designed language that best matches their thought processes and is easy to rewrite (i.e. they just gave up and went with LISP) with the assumption that they are going to rewrite most of the damn thing anyway at all stages of the game and the quicker they can rewrite it the better.

And that is why all these languages are getting closer and closer to LISP was 40 years ago...python, java, smalltalk, etc etc etc....automatic memory management and fast re-write cycle is the best way to write code for 100% of all projects (sans anything with 100,000+ intensive simultaneous users or an airplane code or something when you are bearing up against the engineering portion of things).

That's just my opinion. But I think it's fairly accurate. I've been programming from birth it feels like and I studied the math and I now I write for a large corporation where every schedule slips all the time and I have to deal interpreting customers and figuring out what they want and all I have say is:

There is no magic bullet and there NEVER WILL BE. Read Godel, Turing, Chaitin, etc. You'll be better for it. You're not going to fix your problems by using OSX instead of Windows or Oracle triggers instead of MySQL+PHP or Object Oriented instead of functional or procedural or XML web services instead of TCP/IP and binary. None of those things fucking matter, ok, fanbois? They're corporate games, that's it.

The best you're going to get is to find a language that fits how your brain thinks and that you can rewrite things in QUICKLY (i.e., don't even bother with the write-compile-link cycle... write it in LISP then have something covert the LISP to compiled code at some later day after you've profiled it)...

o, and a good text editor ;)

and if management gives you shit, tell em to jump off a cliff. it's their only job to schedule, and if they aren't smart enough to schedule things with the understanding that schedules change for things that have nothing to do with how smart or good someone is as a programmer, they're worse than worthless anyway and you'd best find a new boss quick.

You have to lie to get the project

[calstraycat]

As others have pointed out, all projects are "late". This phenomena is not unique to IT projects. I put "late" in quotes because the projects are usually delivered on time -- from a realistic standpoint -- but the manager has to lie about the cost and time to be assigned the project in the first place.

There are two type of managers. Let's call them "Honest Joe" and "Sleazy Bob". Both want to lead the project and must meet with the executive who can approve the project. This is how it goes.

Executive: "Hi Joe. Tell me you much this project will cost and how long it will take."

Honest Joe: "It's going to cost five million dollars and will take about eighteen months".

Executive: "Thanks, Joe. You're fired. Before you clean out your office, could you stop by Bob's office and tell him I want to talk to him?"

Bob walks in...

Sleazy Bob: "Wow, I really like your tie. You know, I saw that tee shot you made on number four yesterday. Absolutely amazing. Did you ever consider going pro?"

Executive: "Thanks, Bob. Now about this project. How much will it cost and how long will it take?"

Sleazy Bob: "Six months and a half a mil."

Executive: "Sounds great. Get on it".

Eighteen months and five million dollars later the project is complete and Bob gets promoted.

If I had a nickel for every time I've seen this scenario play out, I wouldn't need a job anymore.

If you had a nickel...

[don.g]

...for each time you saw this scenario:

Let's be generous, and assume you've seen this scenario four times per day: that's USD0.20/day.

Assuming you don't work on weekends, that's USD1.00/week.

If you don't take holidays, that's USD52/year.

I'd always assumed the cost of living in the US was a bit higher than that.

The wisdom of Scottie

[aztektum]

If you guys were truly nerds you'd rely on Scottie's advice. Tell your captain... er boss, that the project will take a year, not six months, and when you have it done in 8 months you'll be heroes.

If there is a budget, if it will realistically cost 10 grand, say 15 and go "Look we have an extra 5k for you!"

Unfortunately in the business world, when you succeed in this fashion they expect you to work miracles next time under truly unrealistic conditions. Sucks.

canada

[alexq]

has anyone clicked on the article? it seems to just be about canada (read the first paragraph)

Canadian information technology groups can't seem to get IT right.

A new report conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups "are not delivering some number of projects on time or to the full satisfaction of the business executive."

and why is the heading in slashdot "95% of IT projects" while the actual statistic is "95% of IT groups have SOME late projects"?

This seems pretty misleading to me!

Re:This could be because of Slashdot

[jdray]

Hey, waitaminute... I'm on that project too!

come on now...

[Run4yourlives]

MS Project dosen't plan projects, PM's do.

Garbage In, Garbage Out.