Rigorous social scientific research runs through the core of SAFI's programmes and activities. The organization's founders met while conducting qualitative research on community institutions and politics in Laikipia, and SAFI's initial rural solid waste management programme was implemented as a randomized field experiment funded by Yale University. Since then, SAFI has utilized both quantitative and qualitative research methods to design and evaluate its programmes, both on its own and in collaboration with co-founder and Trustee Ryan Sheely. The ongoing Participant Driven Evaluation project takes this engagement with research one step further by assessing the extent to which research methods can be used as a tool to promote empowerment and accountability.
The major academic research projects that SAFI has worked on are as follows:
The SAFI Project Waste Management Experiment (2007-2010)
Impact of institutional arrangements on trash levels
This research project, which was SAFI's first activity, was designed to simultaneously jumpstart solid waste management efforts in rural town centres in Laikipia, while also assessing how community mobilization and enforcement of anti-littering rules shape the sustainability of waste management efforts and littering behavior over time. This research resulted in Ryan Sheely's PhD dissertation in Political Science at Yale and his in-progress book manuscript, as well as several research working papers, which are in the process of being submitted for publication in Political Science research journals. This research was funded by the MacMillan Center, Institute for Social and Policy Studies (ISPS), and Leitner Program at Yale University.
The findings of the research are as follows: First, through interviews, participant-observation, and archival data, we mapped the local institutional diversity in Laikipia, finding that although a wide variety of state and community institutions solve many public goods problems on an ongoing basis, none of these institutions were harnessed to maintain public sanitation in the region at the time of the study. Second, the research team utilized this institutional diversity as the basis for a randomized field experiment that randomly assigned a waste management and anti-littering program to three different institutional arrangements that incorporated different mixes and types of government and community institutions. The major findings from this experiment are consistent with the theory developed in the associated manuscript- localities in which there was no explicit punishment for littering experienced more sustained reductions in public waste and littering behavior vis- a-vis localities in which government chiefs could punish littering and localities in which traditional leaders could punish littering. Furthermore, survey evidence indicates that this difference is driven in part by lower rates of community clean-ups in localities assigned to one of the two treatments with rules allowing the punishment of littering.
The findings of the research are as follows: First, through interviews, participant-observation, and archival data, we mapped the local institutional diversity in Laikipia, finding that although a wide variety of state and community institutions solve many public goods problems on an ongoing basis, none of these institutions were harnessed to maintain public sanitation in the region at the time of the study. Second, the research team utilized this institutional diversity as the basis for a randomized field experiment that randomly assigned a waste management and anti-littering program to three different institutional arrangements that incorporated different mixes and types of government and community institutions. The major findings from this experiment are consistent with the theory developed in the associated manuscript- localities in which there was no explicit punishment for littering experienced more sustained reductions in public waste and littering behavior vis- a-vis localities in which government chiefs could punish littering and localities in which traditional leaders could punish littering. Furthermore, survey evidence indicates that this difference is driven in part by lower rates of community clean-ups in localities assigned to one of the two treatments with rules allowing the punishment of littering.
The LASDAP Mobilization Experiment (2009)
Impact of Mobilization on Meeting Attendance
This project was motivated by SAFI's interest in engaging the Laikipia County Council (the region's elected local government) to support the community waste management efforts started during the initial Waste Management Experiment. In particular, SAFI was interested in understanding whether mobilizing communities to participate in a local government budgeting and planning process (the LASDAP) could increase local government funding for solid waste management activities.
The core of this research project was a randomized field experiment in which half of the local government wards in the sample were randomly assigned to receive a treatment in which SAFI mobilized community members to attend a local government planning meeting and to publicly support local government funding for waste management at that meeting. Enumerators assessed the level of participation and meeting outcomes through structured observation of meetings, and supplemented these observations with administrative records of the actual project proposals that were submitted by the local government to the central government. The results of the experiment show that the NGO’s mobilization had a large and significant effect on citizen participation in planning meetings, particularly in ethnically homogenous wards. This increased participation had no effect on the likelihood that the NGO’s preferred project was funded or in the match between the projects selected at meetings and the final projects selected by the local government, but did cause a shift in the type of discrepancies observed in final allocations towards less visible types of interference by the local government.
This research resulted in a working paper by Ryan Sheely, and was also integral in the design of the Innovation for Poverty Action-Kenya (IPA-K) Dispenser Choice Evaluation (DCE) project, which in turn led to SAFI's own work with installing and maintaining Chlorine Dispensers. This research was conducted with the support of the Sustainability Sciences Program at Harvard Kennedy School.
The core of this research project was a randomized field experiment in which half of the local government wards in the sample were randomly assigned to receive a treatment in which SAFI mobilized community members to attend a local government planning meeting and to publicly support local government funding for waste management at that meeting. Enumerators assessed the level of participation and meeting outcomes through structured observation of meetings, and supplemented these observations with administrative records of the actual project proposals that were submitted by the local government to the central government. The results of the experiment show that the NGO’s mobilization had a large and significant effect on citizen participation in planning meetings, particularly in ethnically homogenous wards. This increased participation had no effect on the likelihood that the NGO’s preferred project was funded or in the match between the projects selected at meetings and the final projects selected by the local government, but did cause a shift in the type of discrepancies observed in final allocations towards less visible types of interference by the local government.
This research resulted in a working paper by Ryan Sheely, and was also integral in the design of the Innovation for Poverty Action-Kenya (IPA-K) Dispenser Choice Evaluation (DCE) project, which in turn led to SAFI's own work with installing and maintaining Chlorine Dispensers. This research was conducted with the support of the Sustainability Sciences Program at Harvard Kennedy School.
Participant Driven Evaluation (PDE) (2011-2013)
Community Research on French Beans, Mutaro Village
The Participant Driven Evaluation (PDE) project was motivated by the experience of the first two randomized field experiments that Ryan Sheely conducted with SAFI, as well as an analysis of the politics of social science research in Kenya that Ryan wrote in 2011.
The core idea of the Participant-Driven Evaluation project is that the incorporation of community members into research design, data collection, and analysis has the potential to both improve the quality of data and to fully enable the use of research as an empowerment tool, allowing communities to hold politicians and civil society organizations accountable for project performance. Yet, although the possible gains from combining experimental and participatory methods are intuitively appealing, a number of unanswered questions have prevented attempts to combine these methods into a toolkit that can be used by communities, researchers, and policymakers.
With the support of three Harvard Research Funds (the Weatherhead Center's Medium Faculty Grant, the Milton Fund, and the Ash Center for Democratic Governance's Faculty Grant), the PDE project has developed an adult education curriculum that teaches community members the principles of scientific research and enables them to design and implement their own research projects for the purposes of local problem-solving, project monitoring, and mobilization of resources. Moreover, SAFI is currently evaluating the effects of this project using a "randomized phase-in" experimental design. The sample for this evaluation is a randomly selected set of 32 rural villages in Laikipia East and Central Districts. Sixteen of these villages were randomly assigned to the first wave of PDE workshops and 16 were assigned to the second wave of workshops. The random assignment to roll-out phases makes it possible to conduct surveys before and after the first roll-out, which in turn makes it possible to treat the second roll-out group as a control group for the purpose of evaluating the impact of the PDE intervention. The results of this evaluation will be shared with all 32 Village Research Committees after the second wave of workshop implementation, during a Final Meeting.
Although final results will be available after the conclusion of the endline survey in August 2013, preliminary analysis of evidence from the baseline survey and qualitative observations yield a number of initial findings: 1) A broad cross-section of community members were able to grasp the core concepts and methodologies of social science research, and were also willing and able to develop their own research agenda and conduct research;
2) While non-participatory research methods can alienate community members and dampen their faith in research, engaging community members in the research process can actually reverse this association;
3) Mainstream research practices impart community members with a feeling of being less involved in research than professional researchers, academics, government and private sector researchers, to the point that community members believe they benefit less from research than this group;
4) Participating in the process of conducting research can empower community members by enabling them to see themselves as partners in the research process and by changing the ways in which they interact with politicians, NGOs, and other development actors.
When complete, this research will result in a set of training tools and materials that will be shared under a creative commons license, as well as a series of academic articles by Ryan Sheely, Vanessa Zhang (the PDE Project Coordinator who led the design of the PDE curriculum and the evaluation), and Tara Grillos (a Harvard PhD student who has worked on the design of the PDE curriculm and the evaluation of the effect of PDE on the empowerment of participants).
The core idea of the Participant-Driven Evaluation project is that the incorporation of community members into research design, data collection, and analysis has the potential to both improve the quality of data and to fully enable the use of research as an empowerment tool, allowing communities to hold politicians and civil society organizations accountable for project performance. Yet, although the possible gains from combining experimental and participatory methods are intuitively appealing, a number of unanswered questions have prevented attempts to combine these methods into a toolkit that can be used by communities, researchers, and policymakers.
With the support of three Harvard Research Funds (the Weatherhead Center's Medium Faculty Grant, the Milton Fund, and the Ash Center for Democratic Governance's Faculty Grant), the PDE project has developed an adult education curriculum that teaches community members the principles of scientific research and enables them to design and implement their own research projects for the purposes of local problem-solving, project monitoring, and mobilization of resources. Moreover, SAFI is currently evaluating the effects of this project using a "randomized phase-in" experimental design. The sample for this evaluation is a randomly selected set of 32 rural villages in Laikipia East and Central Districts. Sixteen of these villages were randomly assigned to the first wave of PDE workshops and 16 were assigned to the second wave of workshops. The random assignment to roll-out phases makes it possible to conduct surveys before and after the first roll-out, which in turn makes it possible to treat the second roll-out group as a control group for the purpose of evaluating the impact of the PDE intervention. The results of this evaluation will be shared with all 32 Village Research Committees after the second wave of workshop implementation, during a Final Meeting.
Although final results will be available after the conclusion of the endline survey in August 2013, preliminary analysis of evidence from the baseline survey and qualitative observations yield a number of initial findings: 1) A broad cross-section of community members were able to grasp the core concepts and methodologies of social science research, and were also willing and able to develop their own research agenda and conduct research;
2) While non-participatory research methods can alienate community members and dampen their faith in research, engaging community members in the research process can actually reverse this association;
3) Mainstream research practices impart community members with a feeling of being less involved in research than professional researchers, academics, government and private sector researchers, to the point that community members believe they benefit less from research than this group;
4) Participating in the process of conducting research can empower community members by enabling them to see themselves as partners in the research process and by changing the ways in which they interact with politicians, NGOs, and other development actors.
When complete, this research will result in a set of training tools and materials that will be shared under a creative commons license, as well as a series of academic articles by Ryan Sheely, Vanessa Zhang (the PDE Project Coordinator who led the design of the PDE curriculum and the evaluation), and Tara Grillos (a Harvard PhD student who has worked on the design of the PDE curriculm and the evaluation of the effect of PDE on the empowerment of participants).