Successful use of Agile Programming in Software Product Development
Agile programming seems to be a popular iterative programming methodology in use for software development projects today. However, in many ways it's in conflict with accepted New Product Development (NPD) practices. For example, you can't practically complete a full launch of the product for each short iteration. Generally Product Roadmaps are forward looking for several quarters at least, while Agile wants to evolve requirements and priorities with each short release.
I'm very interested in hearing some success stories in using Agile programming in software NPD and understanding how you overcame the inherent conflicts between Agile and commonly used NPD process. Please share your experiences.
Good Answers (3)
Ben S
Entrepreneur
Best Answers in: Software Development (4), Web Development (3), Market Research and Definition (1), Engineering (1), Interface Design (1), Business Plans (1), Incorporation (1), Computers and Software (1)
I am certainly no expert in formal NPD, but I can speak about my experiences leading Product Management at a small software company that used an Agile development approach (it was a modified Extreme Programming, with 2-week iterations but no pair programming).
We were careful to distinguish between internal and external releases of our software. In each iteration, we would only accept features that passed acceptance testing and were "releasable." However we only made new versions publicly available in accordance with our Product Roadmap, which was intentionally specified at a higher level. Usually, we would have 6-8 iterations between external releases.
What we found was that in general, we would complete the current release as planned with all of the major feature areas covered. However, in the process of finishing the current release incrementally, we would have significantly more opportunity for collecting feedback from prospective customers, Sales, etc. The value of that added feedback was that we would shift our focus in subsequent releases on the Roadmap. If we hadn't been producing incremental releases along the way, the feedback would have come too late to adjust the next release.
Another benefit of the iterative approach arose when dealing with the inevitable emergencies that pop up. If a customer needed a fix or feature right away, we were never stuck with completely unstable code - we could always build off of the last iteration's release. When a critical demo needed some extra pizazz, we could use code that was at most 2 weeks old, rather than having to agonize whether to roll back to a 3-month-old version or not.
Ultimately, Agile programming is a means to an end. The end is building useful software. But the means is designed to cope with the realities of business that don't fit neatly into an 18-month Product Roadmap: specifically, that requirements will change, emergencies will happen, and some of your assumptions will always be wrong. Agile programming simply says, "let's assume that change will happen and build a process that helps identify the need for change early and that is flexible and manageable when change occurs."
On top of that process, you can easily layer Product Roadmaps and Product Launches. The place where you do need to alter your high-level process is to adopt an agile approach with it as well. After all, what good is a beautiful, clear, long-term Product Roadmap when you learn that release 2.5 has features in it the market wants now, while release 2.1 has enhancements planned for an area of the product that isn't being widely used?
I hope this helps.
MDS Sciex out of Toronto is the best example I've been associated with in terms of integrating Agile Software Development with Product Development. There is a paper you can obtain through FastTrack (a bit spendy though) that details their method but this is basically what it says:
Title: "Embracing Ambiguity: MDS Sciex Pilots Rolling Wave Project Management to Facilitate Agile Product Development"
Abstract: "How can a team create accurate plans and schedules in the face of an uncertain future – especially with projects that demand long lead times? Faced with the problem of prediction accuracy, MDS Sciex piloted the Rolling Wave approach to project management. The Rolling Wave methodology creates a window of highly detailed and accurate plans for near-term activity. Activities in the more distant future are given rough order-of-magnitude estimates but are not planned or scheduled in detail. MDS Sciex generated numerous expected and unexpected benefits from this fast and flexible approach to project planning."
Our practice also focuses much more on the Dynamic Systems Development Method out of Great Britain that does much more upfront planning and integration of business requirements, architecture, and business planning than do most of the other Agile methods. At the end of the day, what you need is an agreement between the folks who manage the product development lifecycle and those who manage the software development lifecycle so that all parties are coordinating their effort, adapting to new information when appropriate and creating a constant flow of business value. To that end, one of the best texts on Agile and product development is Jim Highsmith's "Agile Project Management: Creating Innovative Products". Many of the things Jim talks about in his book were successfully implemented in the MDS environment.
We are currently working with Control4 (a home automation company) to help answer the very question you've asked in this inquiry. Our approach is to help everyone on the executive team see the product and software development processes as one "throughput" exercise, plan for what we think is possible, exercise the agile software development and testing practices as appropriate, and iteratively adjust both processes as necessary to meet expected outcomes. In the worst case scenario we'll uncover any major problems sooner, resolve them quicker where possible, and/or save future sunk costs.
If you are interested in how our firm does what we do, please don't hesitate in visiting us at www.aamngt.com or by calling 801-633-0962.
Links:
- http://fasttrack.roundtable.com/app/content/knowledgesource/item/158
- http://www.dsdm.org
- http://www.amazon.com/exec/obidos/ASIN/0321219775
David S also suggests these experts on this topic:
Andy B
Entrepreneur, Manager and Consultant
Best Answers in: Air Travel (1), Staffing and Recruiting (1), Futures Markets (1), Computers and Software (1), Software Development (1), Web Development (1)
In short - agile is not throwing long term planning away, it just renounces clinging to those plans as something more than they are. Many major software products from well-know companies were frequently delayed, even by years (Vista is an all-too-obvious example). Delays in release etc. and changes of the roadmap are a reality to embrace, not shovel under the carpet. Agile is not stipulating that there should be no plan for more than one iteration ahead. It just says that this plan should be potentially changeable. It always is anyway - isn't it? Even if, for example, a date is set in stone by some external force (say, national elections for a votes counting system) then scope (functionality) can be changed along the way.
My suggestions - read Mike Cohn's book. Or talk to him.
Links:
Andy B also suggests this expert on this topic:
More Answers (8)
Michael S
President and CEO of MSCC
Best Answers in: Software Development (5), Offshoring and Outsourcing (2), Lead Generation (2), Business Analytics (2), Enterprise Software (2), Databases (2), Accounting (1), Venture Capital and Private Equity (1), Economics (1), Risk Management (1), Government Policy (1), Personnel Policies (1), Intellectual Property (1), Internet Marketing (1), Business Development (1), Sales Techniques (1), Wealth Management (1), Career Management (1), Green Business (1), Computers and Software (1), Information Security (1), Telecommunications (1), Web Development (1), Using LinkedIn (1)
I don't see an inherent conflict with what you call "accepted NPD practices".
Consider Agile as an iterative subprocess. Each iteration provides a subset of the overall functionality. It may take several iterations where you will have product 1.0, where product 1.0 represents a shippable product.
The big difference and difficulty with Agile development is that you no longer have the ability to 'fix bid' the work. How do you fix bid a project where at each iteration, there is "scope creep"? I'm sure that there is an agile expert who will beg to differ, but when I saw an RFP that said fixed bid and "agile development", I walked away. (Not that I like to do fixed bid work in the first place...)
I've used agile development in an in-house effort where the end product was a conceived idea, but didn't have all of the requirements or inputs fully fleshed out. It allowed me to work on components, get feedback and then flesh out the functionality over the iterations.
Agile development works well when you have power sponsors that realize that they have a problem and need a solution, but don't have a complete grasp as to what they want in a solution. Most times, they understand the Business Process at a high level, but not all of the components. Rather than getting overwhelmed by the complexity of the overall model, we can focus on one section and divide it in to smaller subsections which will fit in to the agile time structure.
Jim H
Strategic Business Consultant
Best Answers in: Positioning (2), Planning (1), Market Research and Definition (1), Business Plans (1), Starting Up (1)
Jim H suggests this expert on this topic:
He is now an executive Manager at Wachovia, but if he has time, he's brilliant on this subject...
Andreas S
Experienced operator and business executive. General Manager ProChain, Board Member TIE-DC
My company is almost exclusively implemeting advanced project management methodologies in NPD organizations of Fortune 500 companies. Also, we develop our own software. By doing that we are using some of the principles of Agile: Short iterations (e.g. 2 weeks), incremental increases in functionality, embedded test cycles. From my point of view it appears that it is an Agile type approach works well with smaller development teams, that are ideally collocated. Even then, however, we choose to overlay the development process with a project management process that plans out all critical phases and stage gates on a higher level. We use Critical Chain for that purpose. The methodology allows to model complex programs and/or a number of development initiatives that are loosely coupled (potentially only by having certain resources in common). We see that approach very successfully deployed in large scale environment. It also integrates very well with stage gate processes.
I could go in any level of detail on this subject. If you have further questions, then please don't hesitate to contact me directly.
Rajasekar N
SAP Business Objects Consultant / Microsoft BI Consultant
Best Answers in: Using LinkedIn (3)
Rajasekar N suggests this expert on this topic:
Siddharta is an Expert in Agile and he has developed Agile Project Management Tool
Check @ http://www.silverstripesoftware.com
Cindy,
I was about to write you a long detailed answer, but this article (amazing incident) actually does it even better...
http://www.pragmaticmarketing.com/publications/topics/07/does-agile-have-to-be-acrimonious/
Hope it helps,
Jonathan
Sam L
Product, Process, Change and Requirements Management Consultant
Best Answers in: Product Design (5), Corporate Governance (2), Project Management (2), Engineering (2), Sales Techniques (1), Business Analytics (1), Organizational Development (1), Planning (1), Quality Management and Standards (1), Market Research and Definition (1), Professional Books and Resources (1), Enterprise Software (1), Software Development (1)
Depends a lot what sort of software you are doing; are you delivering software based on client requirements, are you creating your own product, or are you delivering software in a custom client project. Agile methods fit these scenarios differently, least performing to the first, and best for the last.
Actually, its all about discovering, what exactly needs to be done. When dealing with custom client developments, the element of change is the driver. For your own product plans, the variation in the target are moderate, but some new ideas might spun out of development which might serve a need on the market. Finally, working with strict supplier requirements hardly poses any big new ideas to discover, and change is something you really try to avoid.
Scope creep (change control) and discovery of unknowns (contingency) need to be budgeted for in project planning no matter what the methodology.
When product owners ask for a change, the change is estimated and the product owner is asked to decide if they are willing to spend this amount of change control budget on this change. When the change control budget is exhausted the user's decision becomes one of increasing the time/resource budget or removing some of the yet undeveloped features of the product. When halfway through your kitchen remodel, you tell your contractor you have decided on Italian marble counters instead of Formica, they do not feel compelled to anything other than give you an estimate of the additional cost and a new completion date. Users get to ask for whatever they want. As managers our responsibility is to give them accurate estimates, not to meet arbitrary time lines no matter what is asked for.
Similarly a contingency budget allows for discovery time.
If, during planning, the feature requirements are broken down in a fairly granular manner, experienced developers and managers are capable of estimating with a decent degree of certainty the time each feature will take to model, construct and test. These estimates may also be used later during trade off decisions by the product owner.
Agile development is about anticipating change and producing high quality software, not about setting off on a journey with no idea where it will lead or how long it will take.
follow the link to Jeff DeLuca's site for more info.
Links:
Bob W
Senior Manager eCommerce Marketing
Best Answers in: Customer Service (1), Internet Marketing (1)
The essential conflict here is between the flexibility of an Agile process and the structured approaches of NPD or Waterfall software development. The processes conflict with each other which also causes internal organizational conflict when trying to use the Agile method.
My organization has talked some about creating Agile development groups, (even put it on job descriptions) but has never full supported the idea and structured the organization to support it.
So you are right in that you can't fully support all of the tenets and processes of the structured approaches while implementing the Agile methodology. It's a different mindset and people have to be willing to put the decision making capabilities on the ground level so that quick decisions can be made to support the agile work group. The trick would be how to build in some of the important elements of the NPD process to ensure quality in the final output.
