Norm B. • The ID environment/industry is such that we even pretend that we can achieve more in a foreign country than withing our own. Every donor country has its own problems with poverty, social issues and lack of opportunity for aboriginals, etc. but these issues offer enormous challenges even with all the assets at our disposition. Yet we feel that we can go to a foreign country with its own culture and pretend that we can fix everything. As was said in previous comments, we need to adapt and develop realistic objectives using what is available on site and accept the long term commitment it takes for progress to take effect.
I was recently referred to the Coady Asset-Based Community Development Institute (ABCD) the idea that originated this concept seems to reflect the concerns identified in this discussion and may be useful to some of you, if you are not already familiar with it. Its principles are used for local as well as international assistance programs.
http://coady.stfx.ca/work/abcd/

Rashid M. • First, a limited percentage of funds for international development sector reach the common man, woman, boy and girl at the grass root. A lot of money is spent on workshops, expensive cars, travel and on consultants. Second, there is high pressure for results in a very short-term period without taking into account the speed of change within the local operating context (quick fix approaches i.e 2 years interventions lead to democracy or good governance in an environment where democracy have never been practiced for the last 50 years). Third, when you exclude money for emergencies and relief, the funds for international development sector is still low to bring about substantial development or desired growth in poor countries compared to government resources in these countries. Thus, limited input and limited outputs. Even the limited resources are not put into proper use as in the first point.

Paulette L. • I agree with most of what you say, Rashid, except the line, "Limited input,
limited outputs." This is the Jeffrey Sachs trap, which correlates money
to sustainable achievement. Let's return momentarily to the Peace Corps
adage, "Don't buy a fish, teach the (wo)man to fish." Of course, the
(wo)man has to want to fish...

Jennifer L. • My favorite: "If you give me a fish, you have fed me for a day. If you teach me to fish, then you have fed me until the river is contaminated or the shoreline seized for development. But if you teach me to organize, then whatever the challenge I can join together with my peers and we will fashion or our solution." ~Richardo Levins Morales

Paulette L. • lol! Love that one, Jennifer!

David W. • From Russell's initial post:

"Why does the ID sector spend so much money for so little result?"

What's your benchmark it, and how do you run the comparison? Really: compared to, say, building a car, or providing Internet services in California, development aid operates in the messiest and most difficult environments, addressing some of the most intractable. Who's to say that aid is poor value for money? I'd like to see the transparent methodology for that one.

I work in large scale aid-funded schools projects. The projects deliver teacher training, text books, education management, and classrooms to poor communities that have no schools. The schools are themselves community-built through grant funding, and cost about USD4,000 per classroom. Thousands are built every year. And your complaint is... ?

"It really is a litany of issues that are being raised and commented on."

If you walk into any organisation, even "stars" like Apple Computer or the Grameen Bank, you will find a litany of issues being raised and commented on. That's a good sign, not a bad sign. When you walk into an industry or organisation where there are no issues being raised or commented on: then you know you are in deep shit.

"The sad part is that with so many people in agreement, we still can't get it right."

What does this mythical "get it right mean"? You mean "get it right" as in:

• the war on drugs
• the US education system
• the war on poverty
• the wars in Iraq in Afghanistan
• environmental protection
• pollution control
• democratic governance
• efficient and effective bureaucracies
• the criminal justice system
• the airline industry
• urban planning
• workers' rights
• child protection
• local government
• the health system
…

I challenge you to come up with one serious, complex, area of endeavour in which people are sitting on their laurels, glad that they "got it right." Development aid and humanitarian aid, as an area of endeavour, is no better or worse than any of these.

And in those areas where you think things are better than in development, there is usually a simple answer: the budget per "client" is a hundredfold or a thousandfold greater. That simple.

Russell L. • Perhaps I should have rephrased the question -

"Why does the ID sector spend so much money while achieving so few outcomes?"

By that I mean that although many outputs are produced, there are very few outcomes achieved.

David W. • Part, I think, is because the idea of an "outcome" is donor-driven.

Once upon a time, donors were satisfied with inputs. Then they noticed that the inputs weren't producing "results", so they shifted to "outputs." Then they shifted to "outcomes". And now we have "impacts". All common sense, at one level: but the fact is that in most cases the poor were quite happy with the inputs.

Donors have also shifted from a charity model, to development, to sustainable development.

And most donors see their giving nowadays as "investments", rather than… giving.

So what drives this ongoing donor dissatisfaction?

But they are aiming at, I think, is the investment ideal of reaching a point where a change is sustained internally, and no further investment is required.

That's a worthy ideal… but ideals can be worthy while being in practice not feasible.

There are many places in the world where sustainable development is clearly not feasible. One area is the PICTs: the Pacific Island Countries and Territories. They are structurally incapable of internally sustaining the standards of living which both they themselves and donors have set. I happen to know the PICTs quite well, but I'm also wondering how many other countries in the world also fall into that category.

If we look at the international system as a whole, the idea that every country will one day "middle income" is in the first place mathematically impossible. It may also be the case that constant global competition and accumulation will mean that some countries will always be poor systemically, regardless of the education of their people or the natural wealth of their country.

Here again, I'd draw comparison with the welfare system.

IF the US, still top of the GDP pile, a country which has over the last 50 years poured much more (I should check how much) per capital into the welfare of its own poor, within its own borders and governance systems, can still today be at a place where 1 in 7 of its children are subsisting on food stamps, THEN why do we imagine that better outcomes should be reached when same money is poured into foreign countries, with worse governments, weaker economies, less education, at a fraction of the per capita rate.

I mean: on the face of it, people should be amazed that there are any development outcomes AT ALL.

But I take an optimistic view. I think that if were to able to compare IDA outcomes between nations on a social return per dollar per capita, we would find that IDA produces a far better ROI than the internal welfare system. And that therefore what we should be doing is taking this comparative success, and applying it from the foreign field, to our own home turf.

Now: this assertion would require some numbers to support it. But here are three bits of evidence:

• The Australian National Audit Office (ANAO) recently reviewed AusAID, and determined it's functioning efficiently and effectively.

• The ANAO also recently examined the GOA's investment in Indigenous welfare, and excoriated it as being almost a complete waste of money, with nothing to show for billions spent.

• One Indigenous leader, Professor Mick Dodson, AM, 2009 Australian of the Year, has said of Australia's aid program "that the principles AusAID uses in delivering aid throughout the region should be applied to the delivery of assistance to Australia's indigenous communities."

You've put up a thesis. I'm offering a counter-thesis: that when we look at IDA compared to in-country programs, and factor in the lower per capita investment rate, even without allowing for the problems of cross-border and cross-cultural work, IDA fares well.

Russell L. • Actually I agree with most if not all of what you are saying - well not quite all - but certainly the fact that we have not achieved anywhere near the success in Australia that Australia has achieved internationally - particularly in terms of visible outputs. But I do disagree that we are achieving outcomes, particularly when I see that the objectives are couched in outcome terms and the measurements in outputs - mind you, that occurs throughout the public sector. It isn't limited to AusAID or other donors. I sometimes feel that there are actually very few people who actually understand the difference. I haven't mentioned impacts, because I actually think they are entirely different and I'm not sure that we are measuring them properly either - I still think donors are relying on outputs generated rather than imapcts assessed.

I also fully agree with you on the low probability of the PICTs achieving achieving any great improvements without on-going aid programs - as you say, with a few exceptions, they are structurally incapable of achieving true self-sufficiency to the standards they and the donors have set.

However, if the ID community is doing so well, then why is the perception so strong that it isn't performing well at all - or perhaps there is significant room for improvement particularly in terms of ensuring that ID efforts collectively are working towards the same goals and objectives in full cooperation and not as it seems in competition.

David W. • Hi Russell. You're in Australia, too, hey? A fellow Melburnian perchance?

I think that the M&E community has a few communications problems. Often, great learnings from great projects get digested into fuzzy generalities that don't constitute the learnings at all. But that's another conversation.

The distinction between output/outcome/impact is certainly hard for me to keep clear. I once got a hold of UNDP M&E definition sheet, and mapped out visually all the terms. My current understanding is:

OUTPUTS: the "SMART" objectives of the project, being specific and measurable, also termed "deliverables". This is what the project "delivers" the "target" population.

OUTCOMES: The effect on the population of what is delivered. Thus, if outputs are doses of vaccine delivered, the outcome is reduced incidence of the disease immunised against, or increased resistance.

IMPACT: My reading of the UNDP definition is that whereas OUTCOMES is restricted to the scope of the goals of the project, IMPACT takes a broader view, and includes not only the project-envisaged changes in the population, but also other changes not envisaged. For instance, we might get better relations between health workers and citizens in the target area; children fearing health workers bearing big needles; greater awareness of the impact of the disease among both health workers and clients; better engagement of parents in the schools where the doses were delivered, etc.

What do you reckon?

Re the current wave of criticism of ID: look, it could be something particularly wrong happening, but I see other possible factors, including:

• The Eye of Mordor. This is what a friend of mine used to call the attention of senior management in a mining company. This is the fact that politicians and journalists are always scanning the horizon for something to pounce on: boat people, speed cameras, overpaid consultants, government waste, alcoholic Aboriginals, train timetables, politician perks, sex between any public figure and another... Some Indigenous clients of mine told me that the Government would always give them seven years of stability, after which they would pull the plug on everything, turn the whole system upside down, and the Aboriginal leaders would have count their losses and start again. In other words: this attention might be more driven by political/news cycles.

• Academic Haymaking. At least in the US, I find the claims of people like the CGD and the J-PAL astounding. "Do we really know anything in development?" (CGD). "Development is in the stage of barbers applying leeches. We will do for development what we did for medicine." (J-PAL). This is just people do heavy duty marketing to get money. (If you look at the pages of Nature… real scientists don't talk like that :-).

Look at the periodic outrage and interest the oil industry attracts due to its weekly price changes. From this, can we really conclude that there's something wrong with the industry?

Suzanne H. • As a postgraduate in media and international development, where we think and analyse development critically.... where we question and think of new ways to make ID more effective, so some reason or another when graduates go into the world of development work, these thoughts go. Whether this be because ID organisations are unwilling to change their strategies and approaches is a big shame... and we up back to square one!

Russell L. • Hi David - Brisbane actually.

I like your "Impacts" definition - which thankfully for me, supports my thinking that the impacts are the results of the intervention whether intended or not - and MAY include the outcomes.

To me "outputs" are production - e.g. widgets

"Outcomes" are the result of production - e.g profit or improvement (community health for example).

Very simplistic I know - but in a world where so few people actually know the difference, I find it necessary to use the KISS principle.

David W. • Hi Russell.

I was thinking of doing a graphic which mapped a program as a process, and all the terms as places in that process.

There's a UN glossary of M&E terms here:
http://www.un.org/Depts/oios/mecd/mecd_glossary/index.htm

Impact is defined as: "The overall effect of accomplishing specific results. In some situations it comprises changes, whether planned or unplanned, positive or negative, direct or indirect, primary and secondary that a programme or project helped to bring about. In others, it could also connote the maintenance of a current condition, assuming that that condition is favourable. Impact is the longer-term or ultimate effect attributable to a programme or project, in contrast with an expected accomplishment and output, which are geared to the biennial timeframe."

I wince when I see the phrase "ultimate effect". Do effects ever cease?

Question: Do you have any insights into how the Indigenous programs could borrow from the ID programs?

David W. • Suzanne, here are some thoughts on your comment:

1 • One problem is that new graduates don't have any authority, so that it's likely that anyone who goes into an institutional setting will not be able to lead change for at least 10 years. That might create the impression of "those thoughts go."

2 • I find that the ability to innovate and lead change is highly variable across actors. I think that the NGOs, especially the large ones which are outcome-driven rather than single-issue or ideologically-driven, are very good at gathering results, reflecting on them, innovating. Oxfam I think is good. I think the World Bank's IDA is good. Private sector contractors do innovate in a limited way, but are always constrained by their clients. Bilaterals are constrained by their ministers. All actors are constrained by the fact that they operate in a complex mesh of issues and relationships, and they can't be like the Two Steves starting Apple in a garage. They have to mediate the needs of partner governments, the people that provide them the money, and the effort required to make any kind of change. (Change costs money: sometimes big money.)

3 • The biggest obstacle is that everybody is flat out all the time. Knowledge managers in every org know that it's very difficult to get people to take time out to stop and reflect. And too much pressure on "value for money" and "accountability" can spell death for innovation. In professional service organisations, the rule is that if more than 70% of time is spent on delivery, then you are eating into your future. Better if its 60%, and (allowing time for admin) you can spend a good 20% of time learning, thinking, and working on better ways of doing things.

So: not as bleak I think as you paint it; but there are obstacles, and I think it would be good to identify them, and then remove them.

Brian I. • Hi David and co,

Is it at all feasible that one of the issues we face is something like the 'remembering self' versus the 'experiencing self'? The great mind, Dan Kahneman, founder of behavioural economics,addresses this in the linked TED Talk below.

http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html

Can his cognitive analysis apply to an ID project? I believe even the thought experiment in this talk, about choice of vacation, can be applied to an ID project. Listen to his talk first, and then read on.

Kahneman's 'remembering self' in ID projects is the few peaks and troughs that can be identified from a distance, over time. It is the 'highlights' and 'lowlights', not the day-to-day activity and interaction that actually in my view defines so many good projects. That certainly sounds like most external evaluations I have ever been subject of. If you look at the M&E processes in most projects, our usual M&E experience could be best characterised as the 'remembering self', not the 'experiencing self'.

So what is the ID project equivalent of the 'experiencing self'? I believe the experiencing self in a project is, as anywhere, a completely different thing, and is embodied in the day-to-day project life, where experts and beneficiaries interact, where targets have a workshop, where field visits occur, and it is the cultural exchanges such as the karaoke with experts and beneficiaries side by side.

In my experience,all projects have great moments that have deep personal impact on the various actors but are simply overlooked, not recorded, and are forgotten. They are also VERY hard to observe for the external evaluator, especially post facto. The only hope of capturing the day to day magic of an ID projects is in the regular - almost daily - recording of far more than just facts and figures, by internal observation. How many M&E systems are structured to capture the 'feeling' of a project? How many M&E systems capture attitudes and relationships as the glue that binds the institutions and people involved in a project?

When evaluating, I use an analytical construct I call PAIR analysis: People, Attitudes, Institutions, and Relationships. This has proved especially effective in examining, for instance, national counter-human trafficking measures. The people in place are one thing, and the institutions in place are another, but it is what binds them together, and what characterises their interactions, that are the best predictors of whether together they are successful or failing.

I certainly can attest that the vast majority of M&E processes I have witnessed focus exclusively on people and institutions, not relationships and attitudes. The 'remembering self' of ID projects tends not to capture these moments: M&E systems as we use them are like the poorest example of the photographic memories of a vacation, and abjectly fail to account for the daily magic of the journey that is almost any ID project.

And, like Kahneman's vacation, when building an M&E system that is like the remembering self, at the end we remember only what we see again and again in the photos: fading fragments of only the lowest lows and highest highs (or rather, not even all the lows and highs, but just those moments where we had the presence of mind to take the photo!).

I think we almost only ever evaluate ID projects in a style of the remembering self, not the experiencing self. And that is going to come up a loser almost every time. I do think Kahneman has hit on something for us to think about in how we experience or remember ID projects.

Jacquie O. • Easterly's The White Man's Burden deals so well with this subject. ID is full of big plans but very poor implementation. There is so much that can actually be achieved at the local level because that's often where the most resourceful and innovative people are...because they often have to be in order to survive.

Marc D. • Russell, after years of evaluating development projects both private and public sector led, I wish I had the silver bullet answer to your question. My only conclusion is that change is complex. Here are three things I have learned:

Aligning change expectations (the developers vis those developing) with a development outcome that has long-term impacts (i.e., changing local socio economic structures to benefit the poor) is hard as hell.

Related, defining the a viable (efficient and effective) developmental process which actually allows you to implement actions successfully leading to desired outcome is harder as hell.

Gearing developer (e.g., UN, USAID etc.) time, resource, and policy dynamics to support a viable process leading to desired outcome is harder than hell.

The best and most rapid “success” I have seen in the field that leads me to hope overcoming turgid and expensive development projects leading not always to undesired or half desired outcomes is the case of mobile phone banking.
Mobile phone banking in developing countries is catching on fire because it uses simple technology everyone wants to use, it is an efficient and accessible tool for everyone, and financial services are those that people want, namely savings and fund transfer services (notice no credit services!). Phones can also provide important health, weather, and crop price information and is often provided to low income and rural folk with very little or no public sector support. A great example is Vodaphone in Fiji where the United Nations Capital Development Fund used modest financial subsidies and some timely expert policy and product advice to seed the rapid development of mobile phone financial services.

Elizabeth S. • Even with all our experience in ID, we still haven't learned the fundamental lesson that handouts simply don't work.
What does work is providing expertise to individuals to develop local market-driven initiatives that serve local needs. Supporting hyper-local initiatives that create goods and services targeted to meet local market demands provides sustainable results. As Drew said earlier, this means that we're on the ground long-term, living within the communities and modeling the behaviours we plan to change.

Norm B. • If we speak of measures, we need to include the commitment at all levels of government in support of any ID initiative, even if not directly related. On the local side, this means the support and commitment of the municipal and provincial governments, departments such as law enforcement and justice, and the local private sectors including foreign. And the same needs to happen donor side with all the partner countries, the UN, NGO's. Alignment and continuity are essential. Any local project, as good as it is, will be subject to the support of these entities in the long term. The problem is that these entities are very reliant on the political condition of the time and, accordingly, ephemeral in nature and in need of "announceables" to boost the political clout. We've all seen good outcomes focused in a defined local area when a number of things were in place but even these projects will run out of gas after everyone is gone if appropriate infrastructures are not in place. The good local people then come back to the outside agencies to keep going but everyone else has moved to other priorities. I have seen many politicians speak at local functions on commitment and priorities at the beginning of a project but when these have changed some hard working people on the ground are left alone and, sadly, still believing int he commitment professed at the time. We cannot measure the progress on the ground without keeping measures on the long term political commitment on both sides.

Duane W. • I am enjoying reading this discussion.

I would like to comment that Project Management practices of any kind were absent in the non-construction projects I worked on and observed in Iraq and Afghanistan.

There is growing support for PM4Dev/PM4NGO concepts, but I could not understand given that most SME's come from developed nations, why projects are being managed frequently on an adhoc basis when there is serious money involved.

http://ngolearning.org/pm4ngos/pages/PMD%20Pro1%20Prep.aspx

There are hundreds of program managers, but I know of only 2 Certified Project Managers in the 100 or so expats I met, and they werent being used in that PM Capacity, but in their technical areas of expertise.

In my opinion, program managers, should be managing a bunch of project managers, delivering a project, drawing on SME's to implement the deliverable's or similar etc.

Many of the projects I know of were / and still are still lacking in basic documentation, decision making governance processes, which are essential for describing objectives, deliverable's, stakeholders, risk, quality, lessons learnt, and project plans that are reviewed updated and maintained by a "Project Manager" dedicated to that task.

My point comes back to managing for results - I think there are alot of people who are great subject matter experts, being expected to manage projects without training in that profession, and the perceived lack of results is due to a lack of tailored robust project management practices which are readily applied to the unique dev environments.

All that said, today I met with a client for 1 hr, walked down the street and bought them coffee. ....... Compare that to two years ago where that would require extensive coordination, at least 4 rides on a blackhawk at 2am, a 4 armoured truck convoy and about 20 heavily armed marines .... now that is spending alot of money for a very little result as the people I arrived to talk with had the day off but we werent told! ahahaha.

Go Tonga.