Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
Programming IT

Should Developers Do All Their Own QA? (itnews.com.au) 299

An anonymous reader quotes IT News: Fashion retailer The Iconic is no longer running quality assurance as a separate function within its software development process, having shifted QA responsibilities directly onto developers... "We decided: we've got all these [developers] who are [coding] every day, and they're testing their own work -- we don't need a second layer of advice on it," head of development Oliver Brennan told the New Relic FutureStack conference in Sydney last week. "It just makes people lazy..."

Such a move has the obvious potential to create problems should a developer drop the ball; to make sure the impact of any unforeseen issues is minimised for customers, The Iconic introduced feature toggles -- allowing developers to turn off troublesome functionality without having to deploy new code. Every new feature that goes into production must now sit behind one of these toggles, which dictates whether a user is served the new or old version of the feature in question. The error rates between the new and old versions are then monitored for any discrepancies... While Brennan is no fan of "people breaking things", he argues moving fast is more beneficial for customers.

"If our site is down now, people will generally come back later," Brennan adds, and the company has now moved all of its QA workers into engineering roles.

Should Developers Do All Their Own QA?

Comments Filter:
  • Fuck no (Score:5, Insightful)

    by Anonymous Coward on Saturday November 04, 2017 @10:45PM (#55491413)

    We're lazy as shit.

    • Re:Fuck no (Score:5, Insightful)

      by Decker-Mage ( 782424 ) <jack_of_shadows@yahoo.com> on Saturday November 04, 2017 @10:59PM (#55491479)
      More like using their customers for beta-testing, just as with every other shop it seems these days.
      • by thsths ( 31372 )

        Exactly.

        It is fine that every developer should be testing. But nobody should be testing their own code - it is just not effective.

        • And there are issues with confirmation bias when testing your own stuff.
        • by Altrag ( 195300 )

          Its still not really "fine," at least not on any sort of large scale. Programming and QA testing are actually quite different skill sets. Certainly there's some overlap (especially if you want/allow your testers to try and find the problem in the code rather than just reporting it) but they're definitely not the same role.

          Add on to that the fact that programmers generally command higher wages than testers on average, and you end up with paying more for people to do a job they don't want to do and aren't e

    • Rotate (Score:5, Interesting)

      by sycodon ( 149926 ) on Saturday November 04, 2017 @11:07PM (#55491521)

      EVERYONE on your team should be a developer.

      But QA should come from that pool for one project. Swap them out for the next, etc.

      Developers working at QA know where the mistakes are likely to be.

      Developers testing their own shit subconsciously avoid the shoddy parts.

      This also keeps the team as one.

      It's like a platoon of Marines taking turns burning the shit.

      • Re:Rotate (Score:5, Insightful)

        by Maxo-Texas ( 864189 ) on Saturday November 04, 2017 @11:18PM (#55491573)

        As a manager, my best developers were not my best testers. And my best QA people couldn't have coded their way out of a paper bag.

        Our top QA guy and head of the QA department was an ex marine sergeant. Didn't know how to code. But he was a top notch tester who caught anything that didn't match spec's or that changed from one release to the next.

        • Re:Rotate (Score:4, Interesting)

          by Anonymous Coward on Saturday November 04, 2017 @11:32PM (#55491619)

          That's not surprising, the two groups that are best for finding bugs are the power users and the completely incompetent. And the power users are likely not the same type of people that are doing the programming.

          I've personally got a talent for breaking software by doing innocuous things that should have been tested for.

          • by Bigbutt ( 65939 )

            Yea, I have a guy at work that I have check things. He always finds something. It’s annoying :)

            [John]

            • by thsths ( 31372 )

              Yes, and IT always tells me that they cannot fix it. They bounce the bug around between three levels of support, until I give up. :-(

          • by vux984 ( 928602 )

            That's not surprising, the two groups that are best for finding bugs are the power users and the completely incompetent.

            I'd say that there are at least 3 groups. Some developers are also really good testers. Some power users are really good testers. And the totally incompetent also are brilliant at finding bugs.

            And you really want your QA to include all 3 types, because they each find different kinds of problems.

        • by antdude ( 79039 )

          Is your employering hiring? ;)

        • by PPH ( 736903 )

          And my best QA people couldn't have coded their way out of a paper bag.

          Then who wrote your QA test scripts?

      • Re:Rotate (Score:5, Insightful)

        by ShanghaiBill ( 739463 ) on Saturday November 04, 2017 @11:35PM (#55491633)

        Developers testing their own shit subconsciously avoid the shoddy parts.

        They also tend to have preconceived ideas about how users will use the program that are very wrong.

        The first time I watched a videotape of someone using a GUI that I had coded, I kept screaming "NO, not THAT button!!!!", but it was a recording, so they couldn't hear. It was a traumatic and humbling experience.

        • by Xyrus ( 755017 )

          A programmer doing QA:
          *runs code*
          "Oh c'mon, no one would ever do that. I'm doing this instead."
          *marks as tested*

      • Re:Rotate (Score:4, Insightful)

        by TechyImmigrant ( 175943 ) on Saturday November 04, 2017 @11:40PM (#55491657) Homepage Journal

        Consider the plight of us semiconductor designers. You've got 10 billion transistors, all hooked up in designs by thousands of people and it all has to work together and if you get it wrong, you can't fix it. It has to be right. So hell yes, you test your own stuff before letting others see it. Then other people test it. Then you test their stuff with your stuff. Meanwhile there are teams of people putting it all together in lots of ways to try and break it.

        The value in a design is not the design. It's the level of trust that the thing will work when you put it in a chip.

        • Consider the plight of us semiconductor designers. You've got 10 billion transistors, all hooked up in designs by thousands of people and it all has to work together and if you get it wrong, you can't fix it. It has to be right. So hell yes, you test your own stuff before letting others see it. Then other people test it. Then you test their stuff with your stuff. Meanwhile there are teams of people putting it all together in lots of ways to try and break it.

          The value in a design is not the design. It's the level of trust that the thing will work when you put it in a chip.

          ...and then, after it goes into the field, there are STILL one or more Errata that follow, with nice things things like "Oh, the SPI Enable doesn't work when PD20-23 are configured as SPI2"...

      • Re:Rotate (Score:5, Insightful)

        by quantaman ( 517394 ) on Sunday November 05, 2017 @12:00AM (#55491709)

        Developers testing their own shit subconsciously avoid the shoddy parts.

        The trouble with developers testing their own code is their objective is to finish the feature and move onto the next issue, not to find bugs.

        A developer who spends 50% longer debugging their code is likely to get a reputation as a slow coder, if a bug shows up 6 months later it might not even get attributed to their change. And even if there is some potential blowback in 6 months time, it's hard to take that into account when your manager is breathing down your neck because the project is behind. I know I've taken heat in performance reviews for not finishing features quicker, I also know I've spent a lot of time fixing other people's bugs.

        The reason for a QA department isn't that they've better at finding bugs in new features, they aren't. Rather the QA department gets to focus on the testing that developers have no incentive to do. QA also tests the features that developers don't work on, because every once in a while changing number of wings on a butterfly triggers a null pointer on the other side of the source tree.

        • by PPH ( 736903 )

          their objective is to finish the feature and move onto the next issue

          Your feature isn't finished until it is bug free. If devs don't get credit for completion until the QA is signed off, they will make a better effort to get it right the first time.

        • "The reason for a QA department isn't that they've better at finding bugs in new features, they aren't. Rather the QA department gets to focus on the testing that developers have no incentive to do."

          The real reason for a QA department is that developers are rarely allocated enough time to get something working, let alone working bug-free. By having a separate group, whose sole function is testing, then that guarantees - *mostly* - that the testing gets done.

          Yeah, there are issues with developers not being

      • EVERYONE on your team should be a developer.

        Just because QA should automate the test, doesn't mean the same skill set should be used for developing and QAing. Specialize or be bad at everything.

        Developers testing their own shit subconsciously avoid the shoddy parts.

        Which is why QA shouldn't know which parts to test and which not to test. They should make their own assumptions about how things should work based on how the product is described to the outsiders. Which means they need to be external to the project.

      • Re:Rotate (Score:4, Informative)

        by Darinbob ( 1142669 ) on Sunday November 05, 2017 @12:45AM (#55491843)

        Sort of. QA does development, but not on the project, they develop automated testing scripts and the like. The skill sets are different. A developer should do basic testing, but the developer is way too close to the code and is very likely to miss things. I see developers who assume no testing is needed, it was an obvious fix, and then it turns out to have bugs. Developers are often not good at integration testing, because they may have troubles seeing the big picture of how all the parts fit together.

        QA can read the product's code certainly, they can be involved in the code reviews, but they shouldn't develop that code either. For a good QA person, testing, creating the test plans from the functional specs, automating the tests, and so forth, is a full time job in itself. In a lot of ways it is harder than development.

        Every time I have seem companies try to merge two groups or departments together it has not turned out well. And this cost saving measure won't turn out well either.

      • by Quarters ( 18322 )

        100% absolutely not. The skill sets of developers and quality assurance personnel very rarely overlap.

      • by PPH ( 736903 )

        This.

        Having developers do some QA will teach them what pitfalls to watch for. And make them better devs as time goes by. The argument that development and QA have different mindsets is both wrong and harmful. Devs need to code with eliminating bugs as a priority. Not just hope that QA will catch them before product release.

        This is how it is typically done in engineering. A designer will hand their work over to another engineer to be checked. And the roles are swapped, so everyone gets to do both tasks. Th

        • by Altrag ( 195300 )

          Having developers do some QA will teach them what pitfalls to watch for.

          Having developers do some QA as a training exercise is one thing, having them do actual QA for a releasable product is another.

          The argument that development and QA have different mindsets is both wrong and harmful.

          The argument that they have the same mindset is even more wrong because its simply not true, at least in general. Developers tend to need time to ponder a problem and then rapid-fire out lines of code until they have a working solution. A very binge-and-purge style of work. QA on the other hand needs to be methodical and repetitive -- exactly the kind of tasks that programmers ty

    • Fuck you, a competent developer already does qa, or we wouldn't release it to qa.

      Wait, is it possible that QA didn't write the code, doesn't know its limits, and can put user testing on top of functional testing?

      YES you fucking idiots.

      • by serbanp ( 139486 )

        You have a lot of anger, why?

        If you would understand the power of good design specifications, you would see that useful QA (or DV for hardware design etc) does not start from the implementation but from these specs, the same as the designer.

        It is obvious that you have no idea about how QA should work. There is a lot of very good information in this discussion thread about the subject (and some not so good), spend some time educating yourself.

      • What do you mean? QA write their tests against the same SRS, they need not know what the code does, they only need to know what requirements it needs to fulfil.

  • by datavirtue ( 1104259 ) on Saturday November 04, 2017 @10:47PM (#55491419)

    "It just makes people lazy..."

    Totally agree. I have lobbied for developers to build integration tests against their code. I started by lobbying for QA to do this but they weren't technical enough and couldn't handle it. The run-of-the-mill QA process is a fluffy user acceptance test. I have proof. We continue to find the real problems in production.

    • How often do developers actually use their own code in a production environment?

      The user can and will find some obscure or previously unexpected way to break the application.

      Bugs will continue to be found in production.

      That said, I do extensive QA on any of my code that users will come into contact with. Lot of pokemon error handling. Though there are still bugs which crop up from time to time in product.
    • by WarJolt ( 990309 )

      Good QA teams are what crappy Engineers lean on to explain why their shitty code is taking so long to release.

    • by Darinbob ( 1142669 ) on Sunday November 05, 2017 @01:02AM (#55491907)

      You need a series of gates and software/firmware/hardware has to pass through all those gates. The later a bug is found the more expensive it is to fix it. When the customer finds a bug it can be catastrophic; as in layoffs are going to happen because of loss of revenue. So find the bugs early, and make sure you've got all the gates set up.

      - Developer must unit test the code - developer must to the basic test of flipping it on, making sure that something happens and that it does what the developer thinks it does. Finding and fixing a bug here is very cheap.
      - Other developers must code review the changes; spread out the knowledge to more than one person, have other sets of eyes looking at the code, let someone point out where the clever change is breaking the design or violating preconditions from elsewhere in the project.
      - QA must then test the feature; positive and negative tests. Run it for long periods, try to break it, compare it one by one to each requirement and specification. This can happen before all features are complete.
      - QA then tests the integration; not just one feature but all of the features, so it's done once there's a code freeze for the release. Maybe it repeats some of the earlier tests, but it should run through a full regression, use it like a customer would use it.
      - If it's applicable, do a full system test. End to end, from hardware board to back end server. Test different combinations. Finding a bug here is expensive; but it's still less expansive than having the customers find the bug.

      If you still find bugs after that, then review how the bug slipped past, fix the gates, and do better next time.

  • by peterofoz ( 1038508 ) on Saturday November 04, 2017 @10:48PM (#55491421) Homepage Journal
    QA done right can add value to a project by rooting our ambiguous requirements and holding developers accountable. Developers are worried about a lot of issues like function, aesthetics, security, performance, scalability. Having QA work from the same specs to derive the functional tests from a different perspective can provide a huge value to developers. But you have to involve QA early in the game.
  • no (Score:5, Insightful)

    by cascadingstylesheet ( 140919 ) on Saturday November 04, 2017 @10:55PM (#55491453)
    No, for the same reason professional writers need an editor. Someone else will see things that you didn't. Yes, they will.
    • Re: (Score:3, Interesting)

      by Maxo-Texas ( 864189 )

      Exactly. Some people are good at writing code. Some people are good at finding errors.

      Modern writing has gone to hell compared to even cheap books written when I was young. As long as the word is correctly spelled it gets thru to being published. You read the atrocious error, get jarred out of the book, then resume reading.

      In development, the atrocious error is a loss of all data on the screen, or the program hangs, or you can't update an amount if you come at the screen from one particular set of comma

      • "Some people are good at writing code. Some people are good at finding errors."

        And what drives people is different. Some people get a thrill out of trying to break someone else's code or driving it to its performance limits. Other people like implementing new features. Or maintaining a codebase.

        While I agree QA shouldn't be done exclusively by Devs, QA and Dev really need to work together and Dev needs to avoid a "throw it over the wall" culture. Help guide QA to write tests that provide optimal cov

      • Exactly. Some people are good at writing code. Some people are good at finding errors.

        That's really a small part of it. Mostly, it's that a fresh set of eyes will see things you didn't, even if they aren't as good at finding errors as you are.

    • by Calydor ( 739835 )

      The entire problem is that you remember what it is SUPPOSED to say, and so won't notice if 'tricks' is spelled 'ticks'.

  • Microsoft (Score:4, Insightful)

    by JBMcB ( 73720 ) on Saturday November 04, 2017 @10:57PM (#55491461)

    Didn't Microsoft move to this model just before the Windows 10 upgrade, uh, issues?

    https://www.windowscentral.com... [windowscentral.com]

    I've done development and QA work, and QA approaches testing from from a whole different perspective. The problem is context switching. It's difficult for a developer to approach a piece of functionality from the user's perspective (especially if they don't necessarily know all the parameters for how the functionality is going to be used) It takes time to switch out of that mode, and, in an effort to make the code perfect, you can come short of ever making the code good to begin with.

    A QA engineer is going into testing with the mindset of an end user. What are they going to expect to have happen? What are they likely to do? What corner cases aren't worth exploring? It's useful for developers to have some experience in this, but I don't think it's reasonable to expect them to be experts.

    In general, you want developers to spend time and cognitive effort making sure the code is functional and maintainable. Heaping on the cognitive load of completely switching gears and thinking like an end user isn't always the best use of their resources.

    • I've done development and QA work, and QA approaches testing from from a whole different perspective.

      So does proofreading. Have you ever done any?
  • Short version: No. (Score:4, Insightful)

    by RyanFenton ( 230700 ) on Saturday November 04, 2017 @10:58PM (#55491469)

    Longer version: NOOOOOOOOOOOOOOOOOOO.

    I've been at places where managers get a variation on this lovely idea: Hey, maybe you would be more careful coders if you had to do your own testing. You're all so sloppy!

    Oddly enough, this is mostly from non-coders, or folks without technical acumen.

    Here's the deal: There ARE coders that claim to not make mistakes, and create bug-free code. Those are known as inexperienced or self-deluded coders, or folks only making one-off projects negotiating exact terms that make it not REALLY programming so much as configuring.

    Even just from a simple level as just writing down words, if you've ever written a work longer than a couple of paragraphs, you'll virtually always reach a point where there's some subtle or stupid mistake you make without noticing it. Even VERY experienced journalists and writers need proofreaders on any serious project.

    It's not a matter of taking your work seriously, or being unable to face responsibility - you're always going to be blind to the flaws in your own code, and although testing crews ARE expensive - they exist for many, MANY important reasons. They're cheaper than the same churn without a proper quality process. Sure, some of that effort may be being duplicated, or used badly - but few possibilities would be worse than expecting everyone to 'man up' and expect bug count to go down.

    In other words - certainly, I can test my own code if you'd like, but I'm a VERY expensive testing staff member, and my perspective is going to cause me to overlook a lot of things - since it already 'makes sense' to me and I won't spot the inconsistencies a percent of the time.

    Ryan Fenton

    • by Hairy1 ( 180056 )

      Coders should be responsible for the quality of their work, but the environment should support them. That means they should be given the time to write unit tests and perform code reviews. Code reviews are not just about reviewing implementation but reviewing the requirements to ensure the developer understood the requirements and implemented what was required. Code review is about ensuring that the unit tests for the code properly test the code. It is never a case of throwing new code over the wall to let

    • I've been at places where managers get a variation on this lovely idea: Hey, maybe you would be more careful coders if you had to do your own testing. You're all so sloppy!

      But most likely, that's not even how it went down. What most likely happened was that they created toggles for every new piece of functionality. Then they wrote a script that would switch to the old version of the functionality as soon as an error with of the new functionality was detected. In other words, they wrote a safe-mode to relaunch their application under, every time an unrecoverable error or a runtime exception were detected.

      And by itself, this is actually not a bad idea. It's one more way to prot

  • by Maxo-Texas ( 864189 ) on Saturday November 04, 2017 @11:10PM (#55491525)

    This is dumb.

    QA people think like QA people. Developers think like developers. And developers can't bring a fresh eye to their own code.
    Heck, most developers can't even develop remotely comprehensive test data.

    There is some evidence in the nuclear and aerospace industries that having things checked twice results in people being lazier (because they think the other person will catch it).

    But for mainstream business developers, I've never seen them able to test as well as QA does.

    • But for mainstream business developers, I've never seen them able to test as well as QA does.

      I've been developing complex software systems for about 30 years now, and I couldn't agree with you more. We developers are the absolutely worst people in the world to QA our own projects, as we develop tunnel vision towards how the software is used; and we develop an unconscious aversion towards doing things that will cause problems for our systems.

      I test my code extensively during development, but am still absolutely amazed at how quickly my customers find errors that didn't occur during my testing. My

  • I can say, as a lead developer and architect at several companies over the last 12 years, having a good QA team to catch all the edge cases makes me a much more productive developer. I have worked in many situations where I didn't have Quality Assurance, and the quality for when I did have QA was much much better.
    • The two teams do need to communicate. When quality goes down you can often trace it back to having bad communications. Ie, features aren't documented well, documentation wasn't written up front, and so forth. When you run across a developer who thinks QA is an adversary, then that's a problem that needs fixing.

  • by king neckbeard ( 1801738 ) on Saturday November 04, 2017 @11:10PM (#55491535)

    External testing is necessary to avoid a number of cognitive biases. Developers should do some degree of testing new builds, especially if the tests can be automated, but at the very least, you at least need roughly the software equivalent of a proofreader. Proofreaders are needed, not because a writer lacks the ability to understand basic grammar and spelling rules, but because the brain suffers from the opposite of NIH syndrome.

    Does that necessarily have to take the form of a QA department? No, but this mostly sounds like trying to cut manpower or get around addressing flaws in the testing process. Another PHB trying to preserve his existence by pulling out pieces of a machine they don't understand.

  • I've done QA and I moved to development. The jobs are very different. Devs should check their own work and write unit tests, obviously, but QA really is it's own step and should not be minimized. Sure, you can code regression testing and unit test just about anything in the right environment, but a good QA tester will do things you will never think of and catch a bug before it is found on production.
    • QA also does things that developers don't have time for. QA is a full time job, and should be done in parallel with development. Ie, one day of coding, one day of unit testing, then QA may be spending a full week just on feature testing after that, then a week of integration testing, and several weeks of regression testing (hopefully automated).
      That's nto counting documentation; ie, dev is writing specs and QA is writing test plans off of those specs and that's in parallel.

      If theupper management thinks the

  • I code in 3 different languages and write my own QA in each, but only because THERE IS NO QA department. Writing useful QA always takes way longer than initial coding of the script to do its intended purpose.

    Coder's lament: "Why would a user *ever* do that?"

    Because they are a different set of eyes and don't do the same things the coder does.

    2 weeks ago I torpedoed my own QA handling an important script that had been working fine, QA 100% successful (so my report said). A "simple", backwardly compatible enha

  • by ghoul ( 157158 ) on Saturday November 04, 2017 @11:17PM (#55491571)

    QA is needed. However far too often QA teams are used as dumping grounds for Developers who cannot cut it

    In my perspective the most complicated part in actually figuring out the tests. if you can figure out the tests it means you have nailed down the scope. if you can do that before you start coding the coding part is actually easy. However if you are only doing QA but not doing coding your coding skills will rust.

    The best place I have worked had 2 scrums working on a project - Team Blue and Team Orange. For one release Team Orange was dev and Team Blue was QA. For next release Team Blue was Dev and Team Orange was QA. They kept switching around keeping both their Dev and QA skills updated but at the same time the context switch is easier to do at 6 weeks intervals than continuoisly during the day.

  • by Cerlyn ( 202990 ) on Saturday November 04, 2017 @11:23PM (#55491593)

    "Test is dead" [youtube.com] was the keynote presentation of a Google Test Automation Conference six years ago.

    My personal view is that if you are doing web development where your company rapidly & repeatedly deploys releases on behalf of customers, you might be able to get away with not having much of a QA department so long as the impact is low. If a problem arises it can be quickly fixed without much of a financial loss.

    But if you are in a regulated industry, failure of your software will result in significant lost revenue while its being fixed, or your software is deployed only every few weeks or months by customers who do their own acceptance and integration testing, then you probably need to do more QA work & dedicated QA work upfront. In such scenarios the software producer and their customers may encounter significant losses and/or inconvenience because something faulty snuck through.

    • There is no ultimate process that is best for everyone. They might be right that when selling fashion, being able to make quick changes is more important to their customers than reliability. On the other hand, they do handle money and personal data, so discovering problems in production can have lasting consequences.

      I think reactive QA is the real story here, not whether QA is a role handled by developers or by dedicated testers.

  • I suppose if you donâ(TM)t care if your site and backend stop working one day then having devs do QA is fine. It all depends on your requirements and expectations. Oh, and who checks to see that toggle a doesnâ(TM)t affect code path c? End-users, of course.

  • Why? All my code compiles and runs perfectly the first time.

    But seriously, the first level of QA should be the developer. Before it leaves the developer, the code should work reasonably well or at leave pass some sanity checks. Not, "it compiles and links, so I'm done."

  • This is the problem I see: Developers are inherently held to a higher standard than everyone in a dev shop. Their code must pass the parser, the compiler, and execute properly. This means not a single typo. No errant semicolons or brackets. Every character must me right. Nobody else has that harsh or a standard. I've seen high level PR and corporate releases that have typos in the first sentence. Cringe-worthy stuff. Even the New York Times and Washington Post issue articles with typos in the headlines, d
    • Of course, I failed as a dev. My comment doesn't compile. Alas, I can't edit it properly. The fixes for the compiler are:
      "that harsh of a standard" and
      "Every character must be right."

      Maybe I've been working too much this weekend, and drinking the whole time. Who knows?
  • Developers should be required to QA all their own code, but there still needs to be a second QA pass done by someone who did not write it. Basic human psychology causes natural blind spots in one's own logical thought processes. It's impossible to be completely unbiased about stuff you wrote yourself. So, while a developer is the best person to QA their own work, they're also usually inherently incapable of completing the task with reasonable accuracy in a reasonable time frame. Therefore, 2 people at m

  • If you run a shop without continuous integration/deployment, full testing automation, A/B deployment, constant feedback loops, etc... then you had better have a second pair of eyes (without the bias of a software developer) to purely represent the customer stakeholder. With those functions mentioned being in place however, it is entirely possible that the role of QA shifts strictly to that of test automation and monitoring of A/B deployment results. The majority of development houses only dream of such orga
    • If you run a shop without continuous integration/deployment, full testing automation, A/B deployment, constant feedback loops,

      Then you're not doing web development. Seriously, 'Software development" is not a different way of saying "Doing web-dev".

  • by Anonymous Coward

    This has to be the absolute worst idea I've heard in a long time. Decent developers generally don't make good testers. Developers don't test their own code well -- I suspect because they know the workflow and don't try enough out of the box things. I'm not suggesting that the developers should be absolved from testing, they should write unit tests, end to end tests and everything in the middle. I am suggesting that a second tier of testers is a huge improvement.

    A corollary, if you ever have a good or gr

  • Also known as "why do we need testers if we already have customers?"
  • "If our site is down now, people will generally come back later," Brennan adds

    In other words: we've conditioned our users to accept being shat upon.

  • by SvnLyrBrto ( 62138 ) on Sunday November 05, 2017 @01:14AM (#55491947)

    I've been a software engineer, a QA engineer, traditional IT SysAdmin, and have now found my niche in SysOps/DevOps/SRE/whatever-we're-calling-it-this-week. And good lords of Kobol, QA is absolutely necessary. The usual reasons definitely apply. A separate pair of eyes will find things you've been blind to because you're too close to your own work. Building things and breaking things are different skillsets and different mindsets. There will be edge cases that even the spec didn't anticipate. Blah blah blah.

    But try being on the operations front line some time, and you'll REALLY appreciate a good QA team. Right now, I'm responsible for multiple products from teams that, due to acquisitions and re-orgs and different companies doing things differently, some have dedicated QA and some let the developers test their own code. I see the errors and failures before anyone else. And the QA'd stuff invariably causes me less grief than the "we trust the developers" stuff. Hell, if I had my way, we'd have QA vet ops changes and updates too; because I'm under no illusions of infallibility myself.

  • by James McGuigan ( 852772 ) on Sunday November 05, 2017 @01:27AM (#55491979) Homepage

    As a developer solo-managing a legacy codebase, it is still important to have various rounds of external QA.

    The first issue is that the developer can be so far into the code that they can completely miss what is obvious or un-intuitive to a non-technical user. This may also involve legacy bugs or interactions with code they havn't written themselves.

    The second issue is that the the spec itself may not have been fully defined, or what you have is the developer's interpretation of the spec, and this may require clarification or feedback once someone has seen the end result.

    The third issue is that a developer will focus their attention on the things they consider most important. Sometimes the best way to achieve this is to just give a developer a list of minor bugs/features, which is a great way of focusing attention by clarifying the spec and letting a developer speed though a bunch of quick fixes which have a predefined spec (defining the spec is half the mental effort).

    So in general a developer should be capable of doing a first round of internal QA on their own code, but a second pair of eyes is still occasionally needed, as the developer has a very peculiar set of perception filters.

  • by EmperorOfCanada ( 1332175 ) on Sunday November 05, 2017 @02:12AM (#55492063)
    Developers should absolutely have unit and integration tests and only deliver a product to QA that they think will be immediately passed onto the customer. But within many organizations this just doesn't make people happy, and unhappy people make bad decisions. QA can give a seal of approval that gets better buy in within the company. This should not be the way it is but that's life.

    Lastly, a good QA will look to see that what was produced is what was actually asked for and promised. This is no small thing, because many programming teams can lose sight of what the contract actually says. "But it's better" is something that should be negotiated, not just delivered.

    But that said, QA can be a major road block. It is not uncommon that you throw things over the fence only to find that QA has hyper focused on something that isn't a requirement. For instance you might have some system that does echo location to find the distance to a wall. Your requirements call for one successful echo location every 10 seconds. You fire out 10 per second and around 50% fail. The reality is that you are getting roughly 5 ranges per second and never will you miss your target of one every 10 seconds. Yet QA gets all wound up about the 50% failure rate and reports that the product has a 50% failure rate. This is where you begin convincing the accountants that the QA department needs a head count reduction.
  • by hyades1 ( 1149581 ) <hyades1@hotmail.com> on Sunday November 05, 2017 @02:50AM (#55492117)

    It's cheaper. Did we mention that? It's 'way cheaper.

  • independent review (Score:4, Informative)

    by Speare ( 84249 ) on Sunday November 05, 2017 @02:59AM (#55492141) Homepage Journal
    In aerospace, independent verification and validation is a critical part of the regulatory process. Every change has a reason, every reason is documented, and every change is reviewed by someone else.
  • The problem with this is that the programmer knows too much about the code. If there is a form involved, the developer knows what goes where, and if the field name is nonsense, he will still enter the right thing. If the field says 'Name', and the average joe enters his name here, and the developer wants the guys Userid... The developer won't catch this... the Q/A guy will enter hist name, not his userid, and the test will fail. There are tons of examples where the look and feel is wrong, or the error

  • by Ihlosi ( 895663 ) on Sunday November 05, 2017 @05:31AM (#55492443)
    ... and not from a company that makes stuff that kills people when it malfunctions. Airplanes, autonomous vehicles, medical devices, etc.
  • by luis_a_espinal ( 1810296 ) on Sunday November 05, 2017 @06:22AM (#55492563) Homepage

    Should Developers Do All Their Own QA?

    If you can do that without deleterious consequences, then your work is trivial and/or not overtly complex.

    Otherwise, such a thing is insane, specially if you work in developing actual products (shrink-wrapped software or software bundled in hardware), or complex services. ) And let us not even mention software related to avionics, weapons or medical devices (where QA is done by a separate organization to remove bias.)

    The primary concern is always bias, bias that come from first-hand knowledge of a thing (or assumptions about a thing.)

    It is a lot easier and effective to have a 3rd party dedicated to methodical black box and end-to-end integration/UA testing than to get a developer to test his own creation to the breaking point.

    There is a reason why organization hire external teams to perform penetration testing of their infrastructure. He who knows nothing of the internals a-priori is the perfect person to black-box test the shit out of something with nothing but specs.

  • QA is a cost center, management is always looking to cut or make more efficent cost centers (and at the same time maximize profit centers)

    It's no wonder every manager under the sun thinks they're the genius who can "solve the QA problem" and return 1-6 people's worth of payroll to the bottom line for that big annual bonus. The problem is that while YES you can get away with no QA in a very high functioning development team, on certain products,

    A) most teams are not high enough functioning, e

  • by ggendel ( 1061214 ) on Sunday November 05, 2017 @07:22AM (#55492727)

    This is really a lame-brain position. As a long time developer I have two mottos:
    * If it isn't tested, it doesn't work.
    * A developer make assumptions all the time.

    I can guarantee that, if I was responsible for QA on my own code, things would be broken in subtle ways.

    Many years ago I did QA consulting for AT&T. We set up a system where the human factors engineering group would document an application in a specification. One copy went to the developers and one copy went to the QA personnel. The QA staff designed a test plan from this spec and then implemented it against the product sent from the developers. I can't tell you how many things the developers missed because of assumptions they made that were caught by the test plans.

  • QA/QE/whatever you care to call it is critical when doing anything important. They provide an independent analysis that the job was done right. Not only in code, but in pretty much any industry. Creators fall in love with whatever they are creating, and think it's perfect. QA should provide a more objective view. Just this week, I was editing some code. I'd tested it and my tests were fine. Send it for review, one reviewer says "why did you change this". Turns out, I'd changed something in an area I didn't
  • On who's going to do QA.
    If she's a QA professional, then it's ok.
    Otherwise do it yourself to save resources.

  • If you are standing up some trivial web application and call it your 'software project', then maybe you are overthinking it to have a lot of people in different roles. Of course if your web application is more involved or more severe, then you need testing from a separate perspective.

    If you are writing some particularly custom software for a particular market, then you should probably have a pool of testers that may not know how to program, but they are probably more in tune with your target userbase. I o

  • Developers can add assertions and write unit tests that can test conditions that should never reach a particular module. That should be responsible for producing unit tests and running all tests as a regression suite. The best team I worked for had a policy that when a bug was reported you wrote a unit test to reproduce it at the lowest level, then fix it. This test was added to the test site and solid to every future version. However this is not sufficient, a QA team needs to test the resultant project. I
  • "If our site is down now, people will generally come back later,"

    They don't care about quality and how it affects their customers. They think their customers have all sorts of extra time to do double work because of software errors. I have to wonder if they company takes the same lackadaisical approach to the quality of their products? ~It's OK if our products are bad, because the customers will just buy another...~

  • This is the quality of QA you get when you have your dev team do all the testing.
    https://blog.codinghorror.com/... [codinghorror.com]

  • You pay QA people less than you pay devs and it's a different skillset.

    So this guy is wasting money and getting a worse product. So win/win.

  • Developers should be responsible for making sure their code is run through static analysis, that it has adequate unit tests (that pass), that automated integration testing is in place and passing, that peers have reviewed the code, and similar 'best practice' sort of things. They should also be engaged in using the product and so on, and if management wishes to spend their time this way, testing OTHER developers code.

    A developer testing their own code outside developing and maintaining automated tests is
  • (OK, I found one occurrence.) But seriously, this is what's being sold in management consulting "corporate DevOps transformation" packages. The first rule is almost always, "Go fire your QA department and make your developers write automated tests." This is exactly what the executives paying the management consulting firm for their "digital transformation strategy" want to hear, and that's magnified by a bunch of Google "visionaries" touting the same. Not every business is Google, and IMO DevOps works only

  • " the company has now moved all of its QA workers into engineering roles."

    So inexperience engineers are going to be releasing code with no QA.

    You can't make this stuff up.

Money is its own reward.

Working...