Do you use requirements management in the QA process?
Following some internal discussions about plans for our next releases, we shed some blood arguing about requirements management. Therefore, I'd like to ask all professional out there with hands on experience - do you use requirements management in the QA process? Or is it just some theory you don't find useful in practice? If you do, what tool do you use for it?
Thanks!
Clarification added July 2, 2008:
Just to make it clear, I'm not talking about using requirements management in our own development process, but adding a feature for managing requirements in Testuff (the on-demand test management tool we are developing), possibly some sort of Requirement Traceability Matrix or something similar.
Good Answers (2)
Jay P
Information Technology and Services Consultant
Best Answers in: Software Development (10), Using LinkedIn (8), Computers and Software (3), Writing and Editing (1), Blogging (1), Databases (1), Web Development (1)
Requirement management in the QA process is very important to have. As a QA professional you would want to map your test cases & defects to the requirements so you can determine where the gaps are. Using a requirements matrix to tie the mapping is good so you can determine (below are just some key reasons):
- What test cases are tied to what requirements
- What defects are tied to what requirements
- Determine if there are additional test cases needed
- Determine if there are requirements missing
- Determine if there are more defects surrounding a particular requirement/set of requirements
As you can see requirements management is good to have in the process for many reasons.
If you have any other questions/comments please feel free to contact me.
Do you mean if you should collect requirements and write spec? Everytime. Even for the smallest thing (turning to be a big one as soon as you start with it). And the req/spec is not the goal, but gathering them is.
What I have learned is that you can throw away the specification as soon as you finish it, because the result will look different anyway (because of technology obstacles or because somebody changes his mind at some development stage). But the things you learn while writing the specification is what you have to go through. When you are writing something down, that's the time you actually start thinking about all the consequences, asking yourself and other people the right questions etc.
We develop an MDA based ERP and we implemented our own requirements management. For us it makes even more sense because we charge our clients for the changes, so we have to sign the requirements with our customers. But as I say, even if it is all internal requirements, you should do them.
What I also learned is that when we write the requirement down very early, we do not have to discuss things again and again and it also helps when you start forgeting the small details. It helps to keep all the details written down, nobody is perfect. When the software grows, you start getting lost in the details.
And one for all, see the link. They also do a decent RQ management/bugtracking software we also use internally.
Links:
Clarification added July 2, 2008:
And something more to it. When I was the only developer and had everything in my mind and was the only one talking to the customer, I did not have to write anything down. But as soon as the software grew a bit and other people got involved, we started to write everything down.
Clarification added July 2, 2008:
To include RQ/Management with your solution? For us it makes sense. We specify use/test cases from the requirements and those we test. That way we can be sure that shipping functionality is tested based on requirements coverage.
But I think RQ/Management is completely another topic and it will double your product. It has to do a lot with collaboration, documents, workflows, etc. A lot of new features to your current product.
You should listen to your customers, first.
More Answers (6)
Phil S
Founding Director at Twygrove Limited
Best Answers in: Software Development (3), Offshoring and Outsourcing (2), Organizational Development (2), Career Management (2), Budgeting (1), Personnel Policies (1), Contracts (1), Public Relations (1), Change Management (1), Project Management (1), Quality Management and Standards (1)
Well, for starters, I would hope you already do peer review?
Following on from that, how about putting in place some simple metrics to judge the quality of the requirements documents? I find DRE (defect removal efficiency) very useful in that regard - you should be able to dig up some references on the internet and probably a spreadsheet or two to help with the calculations and related metrics.
Assuming that you aren't doing function point counting you can use some rough 'rules of thumb' which are better than nothing, so for example roughly 1000 words describes a function point, then you can do some rough estimates of how many errors/defects you would expect for that function point. Of course, doing proper function point counts is more scientific.....
I would (as always) recommend two books - Software Assessments, Benchmarks and Best Practices by Capers Jones and Quality Software Management by Jerry (Gerald) Weinberg. Jerry's book is a series of 4 - so start with the first and see how you get on.
Good luck!
Links:
Keith T
now working at GMI in Bellevue
Best Answers in: Software Development (2), Web Development (1)
The best QA practice IMO is one that is holistic instead of focused on the narrow view of post-development assessment.
Many defects can be detected at the spec level which saves time later having to change the spec, redevelop, and retest. I am a big fan of going through this pain as much as possible at the spec development phase rather than at the development phase.
That being said, it depends on your organization whether that is a QA role or a separate project role such as a BA or TA. I would argue in favor of getting those roles as close together as possible.
Someone mentioned peer review. Peer review does not always include QA (or a specific role such as BA), unfortunately, though it should. Product managers and developers are not always enough. This may be because that organization's QA arm is not broadly developed but instead based focused on rigid TPs and a team of low-overhead button pushers. (This stems from the endemic problem of QA often being an afterthought or considered an unnecessary expense.) In such an organization you should either advance the QA team expertise or hire an analyst -- either one -- in order to fulfill that peer review role.
Raj S
Thinker
Best Answers in: Project Management (5), Quality Management and Standards (4), Hotels (1), Business Analytics (1), Change Management (1), Organizational Development (1)
For any System requirement, there should be testability requirement analysis, which will
- Help to know the requirements that must be fullfilled by system to make it testable
- Help to know the specfic requirements for test environments
- what type of Qualification activities (Note there is a difference between Qualification of requirements with testing of requirements) required and out of that what amount of testing is needed.
Richard Z
President at ZULTNER & COMPANY
Best Answers in: Project Management (8), Software Development (3), Planning (2), Quality Management and Standards (2), Organizational Development (1), Manufacturing (1), Market Research and Definition (1), Engineering (1), Product Design (1), Computers and Software (1)
A basic aim of QA is the assure that the results of development can satisfy customers -- competitively.
So the emphasis should NOT be on doing requirements management in QA, but for QA to focus the entire development process, and especially the front-end, on key requirements. Not to "manage" them, but to execute them superbly well. Once you understand what the key business requirements and high-value customer needs are, everyone should be focusing on doing their best on those items throughout the entire development process. Gold Box Testing is this focus applied to all QA activities, especially testing.
Most requirements are not the difference between winning and losing. A few are. Why not focus on those, and win?
This is the opposite of constructing a large requirements traceability matrix -- where much effort is spent tracking many requirements of trivial importance. In Gold Box Testing you track only the few high-value requirements throughout development -- not so you don't "lose" them, but so everyone can do their best on them. End-to-end excellence on what matters most to customers. Isn't that what QA is really about?
Links:
Ido
In defence acquisition, we use requirements as part of the QA process particularly in the acceptance phase of a project. This is essential because we are allocating Tax Payer's money and require a clear and robust audit trail to justify expenditure; although taxpayers money is probably just as precious as shareholders money.
We create a User Requirement Document (URD) from information that the User community (warfighters) provide about their needs. This is the baseline document against which we map our requirements against military tasks. This is the equivalent to mappig requirements to required functions.
In QA terms, the URD reflects the NATO, Defence and civilian standards against which we will be procuiring the equipment. These requirements then feed 2 QA related documents, the System Requirement Document which definies System Performance and the Integrated Test, Evaluation and Acceptance Plan. These are the documents against which we assess the QA specific issues of Quality, Performance and Fitness for purpose. These documents are robust checklists and are used by both military and technical specialists as part of the acceptance process.
In any audit trail we can then trace a QA issue back throught the process to the specific military requirement and assess the impact of that quality or performance shortfall.
The process is cumbersome and time consuming but offers the best approach to ensuring the quality of military equipment.
The Acquisition Operational Framework can expand on this apporach if you feel you would like to explore it more.
Regards
Martin
Links:
In CMMI PA - REQM , CMS ( change management system) needs to be in place during your planning phase. If you do not have a CMS in place then you should think about other factors through which you can keep the track of changes to your documents and work products. Like for e.g a tracebility matrix.
During testing REQM - CMS is just to see or satisfy the entry criteria for your testing process. Like for e.g your work product and document history for the latest files to be utilized for testing.
Therefore CMS implementation is must for any project to track the changes and hence to ensure that proper Work products gets tested.
