Grameen Foundation came to our San Francisco Datadive with some great questions about their Community Knowledge Workers program. Life is uncertain if you're a subsistence farmer in Africa, where your livelihood is deeply affected by changing weather, market prices, and livestock health. To help battle that uncertainty, Grameen Foundation hires local Community Knowledge Workers (CKWs) to travel to farmers and offer them cellphone-based knowledge services through which they can inquire about market prices, weather conditions, or general health inquiries.
Emily Tucker from Grameen Foundation wanted to know more about how well this program was working, what makes for a good CKW, and whether certain interventions (e.g. introducing bicycles) helped CKWs reach more people and do their jobs better. Below we've highlighted some of our findings, but please see our Wiki if you'd like to read more details about the team's work.
Our teams began to explore the data by looking at the seasonality of what farmers were searching for. The figure above shows what terms are searched for when. These graphs showed the result of certain interventions (the introduction of "cultures vivrieres" as a search term) as well as raised attention to the seasonality of farmer search patterns. Our scientists and Grameen better understood the spikes in searches during planting season in the fall, which would help later analysis.
In order to understand how CKWs were operating and how to evaluate their performance, our teams looked at both the number of visits each CKW made by district as well as the number of searches they did. These plots above show very interesting trends by district, including some districts (Nwoyo) where the average number of visits decreased, but the number of queries stayed constant. We also found some cases where a single CKW would do over 700 searches a day. Does this mean that some CKWs are artificially inflating their searches or are farmers just asking for more searches over time? These are questions Grameen can take back and look into more deeply now that they have these plots.
Grameen also wanted to understand if introducing bicycles helped increase the geographic range of CKWs, the number of searches they did over time, and / or the number of unique farmers they visited per day. Two separate teams tackled this problem—and given the data we had—found the data to be inconclusive. There was simply too little information after the bicycles were introduced and there were too many seasonal trends to allow the teams to conclude which changes in activity were due to bikes and which just to the fact that it was harvest season.
It also raised the question of what a "win" was for this program: did CKWs need to show an increase in distance across the board or was it enough for 5% of CKWs to go 10% further? Our teams worked with Grameen Foundation to determine what information would be needed to answer these questions for next time and helped them nail down ideas of what success would be for such a program.
One of the big questions facing Grameen was whether CKWs started good and stayed good (where "good" means they visit a lot of farmers and do a lot of searches) or whether some CKWs went from being bad to good or vice versa. Our team worked with them to establish a tool that would not specifically say a CKW was "good" or "bad", but would allow Grameen to compare relative CKW performance over time. The figure above shows the relative rankings of CKWs per month per district in terms of unique farmers visited. Blue is a high rank and yellow is a low rank.
What we immediately see is that some CKWs do start "good" and stay good while others start "bad" and stay bad. Why they are that way, however, is something that only Grameen can answer. For example, is a CKW who sees one farmer a day lazy or do they live in an area with only one farmer? What the figure above allows for, however, is the ability to drill into the CKWs who are high and low ranks and understand what makes them that way. In addition, we can see many cases where CKWs started good and turned bad or started bad and turned good. Grameen could use this tool to identify CKWs who are slipping or improving and could then understand the factors contributing to both. This finding could vastly help them improve their CKW workforce or catch early signs of CKWs slipping in performance.
Our teams also explored other interesting facets of the data, such as search volume over time by gender, seasonality of activities based on harvest seasons, and mapping CKW locations over time. Some of these findings are documented on our Wiki and we will be continuing work with Grameen to better understand these unexpected external findings.
Our Data Without Borders + Grameen Foundation team was able to better understand their data by digging into it and asking questions as they unfolded. The team tackled the question of whether bicycle interventions worked or not and ended up getting to a bigger question of what constituted success and what data was available. There were also surprising findings about the way CKWs were operating and the way district activity was changing seasonally. Finally, they arrived at a tool that can help Grameen better understand their CKW's relative performance so they can see more in this data than ever before. Overall, the weekend was a fantastic opportunity to learn more about these datasets and to help Grameen understand the nature of their program better.