Wednesday, April 2, 2014

Re: [MFP] Paper available: "How Do the Poverty Scorecard and the PAT Differ?"

 

Dear Sean, Peter and All,
  
In regards to measuring changes in poverty status over time and linking progress (reduction in poverty) with the resources deployed (and the activities undertaken) in order to get this progress, I'd like to share with you Truelift's Pro-Poor Principles and the related Indicators Tool. At the core of our work is a simple theory of change--Measure. Learn. Change.--with the assumption that good data and analysis leads to good decision-making in favor of poor clients.

Pro-Poor Principles
1.      Purposeful Outreach to People Living in Conditions of Poverty
2.      Services that Meet (and Adapt to) the Needs of Poor Clients
3.      Tracking Progress of Poor Clients

The Truelift Indicators Tool (click this link to register to download the tool) serves as an introduction to the Pro-Poor Principles and can also be used for self-assessment by microfinance practitioners. It helps an institution to identify its strengths and weaknesses in reaching poor clients, meeting their needs (what activities, or products and services are offered?), and tracking progress over time. You can read more about the tool here. Truelift has also collaborated with the PPI to define benchmarks for poverty outreach by country. 

The Truelift framework creates accountability, focuses on the importance of learning from measuring outcomes, promotes the adoption of new practices, and facilitates the replication of scalable innovations across a large number of service providers through its poverty-focused community of practice. 


We invite your feedback and questions as we pilot the tool and accompanying guidelines Wiki. 

Thanks,
Bridget


On Mon, Mar 3, 2014 at 2:05 PM, steve wright <steve@conches.org> wrote:
 

Peter,

I have two comments on your excellent insights. The first is in response to your comment: "Better still, if a system based on these concepts was to be linked to mobile apps for data input and analysis, all sorts of amazing things could become possible. "

At Grameen we have built TaroWorks (http://www.TaroWorks.org) explicitly in response to the dynamics that you describe.  This is a tool with an Android "front end" for data collection, and not just data collection.  The mobile app also provides tools for performance and task management of a field force of employees.  The cloud based "back end" provides management and data analysis tools.  We have integrated the PPI in to the tool and have made it possible to integrate other standard or custom indicators/surveys.  A next step will be to make data aggregation of these standards easy.  Again, we will start with PPI data but will ultimately be able to aggregate data from any predictable set of standard indicators.

My second comment is in response to: "The measure we should be looking for is one that links progress (reduction in poverty) with the resources deployed (and the activities undertaken) in order to get this progress."

A colleague of mine recently pointed me to an excellent paper that is about integrating Measurement & Evaluation (M&E) in to the "design space".  The paper starts with an excellent look at the history of M&E from which comes this quote.  "In practice however, ‘project evaluation’ has been used to mean three completely different things, for which we propose three distinct terms: 1) project valuation, 2) implementation evaluation and 3) impact evaluation"  The first is focused on the cost of inputs and the benefit (in terms of cost) of the outputs. It is a set of aspirational and speculative questions that one asks to determine if it makes sense to allocate funds. The second, answers very practical questions.  Did the intended activities occur?  Were the expected outputs produced? The third is concerned with causality - did the activities which generated the outputs cause outcomes? 

Embedded in your comment are all three questions. The coming innovations will enable us to efficiently and effectively manage our work explicitly at each stage from ideation to pilot to implementation to scale. From the paper mentioned above: "We introduce structured experiential learning (which we add to M&E to get MeE) which allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real-time performance information with direct feedback into the decision loops of project design and implementation."  This of course necessitates effective and transparent process and data management of the type that we work to enable with our TaroWorks product.

The social enterprise's progenitor - the financial enterprise - only had to manage a P&L to understand success.  Our analog for profit is multifaceted and, while it includes financial sustainability, it also includes vibrant human communities which we must learn to realize with the same rigor and intention with which we realize profit.

On Mon, Mar 3, 2014 at 9:01 AM, Peter Burgess <peterbNYC@gmail.com> wrote:
 

Dear Colleagues

As I read this, the question of 'Why we want to do measurements?' comes to mind. Throughout my career, I have made use of measurement and data, perhaps in part because I started out as an engineer, and without measurement you don't build anything. In the corporate world the purpose of measurement (accounting) was to report profit performance to the investors and to help (with cost accounting) to get everyone in the company pulling in the same direction. I would also add that in the corporate world measurement had to cost very little and the use of these data had to be very effective!

So how does this fit into the measurement of poverty? I argue that the measure we should be looking for is one that links progress (reduction in poverty) with the resources deployed (and the activities undertaken) in order to get this progress. My impression is that in the development environment people tend to measure only one part of this ... not both parts. I may be mistaken but I don't think so. 

One of the strengths of double entry accounting is the idea that change in state (the balance sheet) is a measure of profit. This same idea can be used to measure progress. How much poverty is there at the beginning of the period, and how much at the end of the period. What is the change? How much resources and what activities went into achieving this progress ... or lack of? 

I believe a system designed around these concepts would be much more useful than what we have at the present time. Better still, if a system based on these concepts was to be linked to mobile apps for data input and analysis, all sorts of amazing things could become possible. 

Peter Burgess - TrueValueMetrics 
Multi Dimension Impact Accounting 

____________
Peter Burgess
TrueValueMetrics ... Meaningful Metrics for a Smart Society
Multi Dimension Impact Accounting
twitter: @truevaluemetric @peterbnyc 
mobile: 212 744 6469 landline 570 431 4385
email: peterbnyc@gmail.com
skype: peterburgessnyc


On Mon, Mar 3, 2014 at 9:55 AM, <margaretelise@gmail.com> wrote:
 

Hi Sean, 

I think this paper offers a great example of how PPI was used in combination with PWR and another household survey to target services of BASIX (India). Fonkoze also has a number of documents describing how it has used PPI data (in conjunction with other data) for social performance management, notably to manage impacts of the earthquake.

But point-in-time and change-over-time analysis is essentially what these tools are designed to do (as are most M&E activities). As Julie said, the tools are being used in contexts outside of microfinance (most PAT users are not MF): FHI360's STRIVE project has used both tools (documentation forthcoming) in the context of VCD; PAT is being used as part of a larger tool to assess impacts of a women's empowerment program in multiple countries. Another project was working on integrating GPS data to track changes by location more accurately. The LIFT project is using PPI & PAT (modified) as part of a larger survey to develop a referral system to help health clinic patients link with economic strengthening services. 

I've also helped people analyze their PAT data more deeply to learn from specific indicators, client data and disaggregated results (many people use the rural/urban and regional PAT indicators to compare results). 

I will point out that a number of organizations have found that these tools are not generally sensitive enough to track changes over short periods of time. I'm working on a new project with Abt Associates that attempts to address the issue of change over time sensitivity; we are piloting tools in Uganda & Tanzania - if anyone has particular interest in these countries please contact me directly as we are looking for input on their design.

For this and other reasons (like getting a more holistic view of poverty), I generally recommend people use more than one tool: PAT (and PPI) users will combine a tool with food security measures, a housing index, asset indicators, self-reported income data, etc (see these videos I produced last year; essentially poverty tools 101). Though even doing so people will often see inconsistent results. We did an e-consultation through POWG (now STEP UP) a few years ago to discuss issues Freedom From Hunger and Trickle Up had in interpreting PPI & food security data (basically, the results were inconsistent: PPI score going down while food security increasing). FFH has since developed an Impact Stories methodology that really does a lot to highlight the complexity of poverty and how household decisions are made (I'm a big fan of this and anything that uses qualitative info to explain quantitative findings). And everything we know about impacts of MF (summarized in Goldberg 2005, Odell 2010, EPPI Centre 2010) points to inconsistent client outcomes related to poverty and its many dimensions (food security, housing, education, empowerment, etc.)

So back to your question about how the tools are being used, Sean, I brought up the points in I did above to emphasize that they are being used to tell a piece of the story of poverty outreach and changes over time but that the story is incomplete without deeper measures. 

Thanks,
Margaret Richards
PAT Help Desk Director (helpdesk@povertytools.org)
STEP UP Facilitator & consultant (margaretelise@gmail.com)





--
Steve Wright
Dir. Social Performance
Grameen Foundation




--


Bridget Dougherty | Program Manager, Truelift
e: bridget@truelift.org | skype: dougherty.bridget | Twitter | Facebook
m: +1.630.334.2199 (CST/GMT – 6:00) 

Truelift creates a trust mark - in microfinance and other forms of social business - to signify commitment to positive and enduring change for people living in conditions of poverty. 

Help us raise $2,500 to fund the completion of the Truelift Indicators Tool Guidelines website!

Support Truelift here and get involved from the ground floor with this exciting grassroots initiative! 

__._,_.___
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (10)
WARNING! If you hit REPLY, your message will go to the entire listserve, not just the original author!
.

__,_._,___

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)