How effective is Code Coverage Analysis ?
How effective is Code Coverage Analysis ? In your company is this activity done by developers or by QA ? In your opinion who should do this activity ? What tools do you use to instrument the code ? Have there been instances where instrumenting the code changed the logic of the application (not performance) ?
Good Answers (2)
I use code coverage (EclEmma in development and Cobertura) along with JUnit in my team to asses only how much code is hit by our tests. I stress that this doesn't mean the software is good (that relies on well written tests) and the coverage % has to be taken with some thought. i.e. 100% is impractical for a real app, but we always aim for 100% coverage through something like a math formula, or critical path of the application.
In answer to your questions:
Its very effective in the right situation but should be done by developers, who should also write the unit tests etc. Test Driven Development. QA for me is there to assess higher level requirements at a user story level, not a code level. Tools we use are above (java tools though) and i think that any deeper understanding of your code (tests, profiling, coverage, etc) can always impact on your design.
The effectiveness of Code Coverage Analysis is a function of:
- Your coverage goals
- Your development processes
- Your coverage tool
For reference, my teams were developing mission critical software in C++ using CppUnit for testing and LDRA as a coverage tool.
Your Coverage Goals
The number one benefit of coverage analysis is finding functions and paths you forgot to test. Beyond that, you have to balance goals such as 100% coverage with your project schedule. Do you really need to cover every accessor and mutator? Not really. You do not want a situation where you are writing tests that do not add value just to check a box in your development plan.
Your goal should be to find the maximum amount of bugs with the least amount of effort.
Your Development Process
Developers should setup the coverage scenarios while QA should verify results during gate reviews. (This depends on your QA department. Ours was not very technical.) Since developers perform unit testing, they will get the greatest benefit from running the coverage analysis themselves to see what they might have missed.
We found it makes sense to run coverage towards the end of testing rather than early on since we did not want the overhead of running a coverage analysis every build.
Your Coverage Tool
Automation is key. New coverage scenarios must be easy to set up. The more time your developers spend instrumenting code, the less efficient they will be. Make sure you have a user friendly tool so you get maximum impact for your money. All scenarios should be executable from a script and easy to rerun. Like unit testing, the more you automate the process the more likely it will get done effectively.
I’ll say now that our unit-testing tool was terrible. We started using the product when it was immature and it ended up costing us a ton of time for training and waiting for bug fixes. We eventually switched to the Rational products instead. Also, see if there is an open source solution, if there are problems, you can fix them yourself and move on.
See the web resources for coding coverage definitions and a list of tools.
Links:
More Answers (1)
Mohammed Hussain K
Head - Content & Client Servicing at K WEBMAKER™
Best Answers in: Using LinkedIn (22), Web Development (3), Personnel Policies (2), Lead Generation (2), Project Management (2), Professional Networking (2), Education and Schools (1), Job Search (1), Mentoring (1), Occupational Training (1), Venture Capital and Private Equity (1), Staffing and Recruiting (1), Offshoring and Outsourcing (1), Business Development (1), Customer Relationship Management (1), Search Marketing (1), Writing and Editing (1), Non-profit Fundraising (1), Non-profit Management (1), Personal Investing (1), Positioning (1), Professional Books and Resources (1), Career Management (1), Computers and Software (1), Wireless (1)
N.A>
