How Do You Get Better Bug Reports From Users? 205
itwbennett writes "You can try to train them, you can try to streamline or automate the process, you can demand that all bug reports go through a middleman (i.e., a QA tester) or you can throw up your hands and accept that users will forever submit bug reports that in no way help you solve the problem. Like the stages of grief, you've probably tried or experienced all of these at some point. But have you found any approach that really works for getting useful bug reports from your users?"
Worse error messages (Score:5, Interesting)
When writing error messages, don't have it spit out something sensible. Have it spit out something completely crazy but memorable, which you can then grep the code for. Something along the lines of "The Cake has hit the Fan" or "The Chickens are eating Pie". This improves the odds the user will remember it and report it correctly, giving you some hope of finding where the bug is in the code.
Speak to the Submitter (Score:4, Interesting)
The sooner, the better. You need an agile bug response process.
The BUG! Button (Score:5, Interesting)
On every page of the site that I did for this company, in the upper right corner, there is a button labelled "BUG!". Click on it and a dialog box comes up that says simply "What's wrong with this page?" and has a big blank area to fill in. The dialog box has a "Submit" button.
What is not visible is that the JavaScript code for the BUG! button is grabbing all the information it can from the browser itself - what the current URL is, all the global JavaScript variables, name of the current logged-in user, time and date, browser type, web page contents, etc. All grabbed automatically.
And all stored in a special file on the server. The only thing the user has to tell me is what she doesn't like about that page.
The BUG! button was invented - by me - as a reaction to Bugzilla, which seems to be designed to keep users from reporting problems. I can ignore pointless bug reports, but I want to hear everything.
Re:Follow up (Score:4, Interesting)
I had just sent the below list of things required in a bug report to a group of users/testers this morning......
In general, the following items should be included when possible:
â Reproduction steps
â Expected behavior
â Observed behavior
â If available, any related steps or values that worked (for example when a bug âoesometimesâ happens, itâ(TM)s useful to know when it works and when it doesnâ(TM)t)
Then, the trick is to reject or de-prioritize any bug that doesn't conform to the above. Train the users that if they provide accurate information, they get quicker response. It works for Pavlov.
Re:Follow up (Score:4, Interesting)
Maybe, the user didn't want to confuse you by sending in multiple problem reports at once. Or maybe the user's manager thought that would overload IT, or make it appear they were just complaining too much. :)
I had users who just ignored, or worked around errors, errors that had never been reported. Some were user errors (resolved by both program changes to prevent those errors and user training), some were UI errors that didn't impact the results, and some were program errors with actual consequences or impact on the data or utility of the software. I found out about these errors by watching the users, or as a side issue when they were having some other problem that they did report. I explained that reporting all errors ASAP gives the developers a broader view of the potential problem, and allows all errors to be fixed faster and more completely. I trained them to report every problem, no matter how small, and made a point of addressing those errors as quickly as practical to reinforce the behavior of reporting them. And, of course, I explained that unreported errors were very unlikely to be fixed, ever.