Blog
Optimizing Data Pipeline Platforms: DataKind San Francisco Partners with Grameen America
October 27, 2022

By DataKind San Francisco

Background

How can data infrastructure be optimized to help support the financial empowerment of women?

Currently, only four percent of all small business loans from mainstream financial institutions go to women, according to a report by the National Women's Business Council. As MDRC’s recent impact evaluation report states, “Safe and affordable financial products are not always available to individuals with limited material assets or credit histories. Microlending programs such as Grameen America aim to fill that gap, offering borrowers loans to invest in small businesses that have the potential to generate income and eventually improve financial well-being.”

Grameen America is the fastest-growing microfinance organization in the U.S. Since 2008, it’s helped entrepreneurial women who live in poverty build businesses to enable financial mobility through microloans, financial training, and direct support to members across 23 cities in the U.S. Grameen America has disbursed over $2.6 billion to more than 155,000 low-income women through its unique social capital model pioneered by Nobel Laureate Professor Muhammad Yunus.

In 2022, DataKind San Francisco (DKSF) was privileged to partner with Grameen America as part of its second cohort of DataAdvisory projects to provide the right recommendations to leadership on building out their data infrastructure to solve existing pain points and improve data insights. 

Data Advisory Project Execution

For this project, Grameen America wanted comprehensive recommendations to select a set of integrated vendors to create a new internal data analytics platform. The goal of this data advisory project was to robustly evaluate the universe of potential vendors and subsequently  provide final recommendations on which tools Grameen America should choose for their platform. Upon implementing these recommended tools, Grameen America hopes to modernize their internal infrastructure, create dynamic dashboards, and automate critical reports. 

The project began with preliminary scoping to understand the current state of Grameen America’s data analytics platform as well as the future requirements of this platform. This involved a deep-dive into analyzing the current infrastructure, identifying the specific pain points, and deriving insight into what user needs and use cases are in the end state. Through multiple conversations, DataKind San Francisco was able to deeply understand how Grameen America envisioned the flow of data through their data analytics platform through four main stages: Extract & Load, Store, Transform, and Visualize.

undefined

After preliminary conversations, DKSF worked with Grameen America to develop criteria to evaluate potential vendors across the four data stages. DKSF ranked those criteria based on how critical they were to organizational needs, while also incorporating criteria informed from the data team’s extensive experience managing data projects.

undefined

Once DKSF aligned on key criteria across the data processing stages, DKSF narrowed down an extensive list of 50+ potential vendors to a few that would address the above criteria. DataKind San Francisco’s experienced volunteers did outside research on vendors based on Grameen America’s criteria as well as relied on industry expertise on what vendors are most often used in practice. Initially, when optimizing for only data performance and efficiency, DKSF generated three integrated tech stack options, prioritizing cloud integration through Amazon or Google Cloud. Through multiple iterations and a deliberative evaluation process, DKSF was able to arrive at a final recommendation considering financial costs. DKSF hopes these vendor recommendations will help inform Grameen America’s data analytics strategy in the future. A visual representation of the vendor selection process can be seen below.

undefined

Key Learnings

  • Building the team with the right skill sets and experience is critical.
    • At the onset of this project, we realized that deeper data engineering competency was required to complement the expertise already available in data science. We recruited additional team members who had experience in data architecting and engineering, which was helpful as we scoped vendors through the stages of the data pipeline.
  • Most nonprofits are resource-limited so cost plays an important role in developing data architecture that’s sustainable. 
    • We were initially generating and evaluating the vendors purely on data performance attributes. Through conversations with Grameen America, we added financial cost as a key metric to consider, as data architecture costs can accumulate to burdensome levels. By clarifying budget requirements, we were able to identify a more appropriate solution for Grameen America and balance the tradeoff between the needs of the nonprofit and the unique technical requirements needed for a financial services institution, including additional security and storage capacity to host financial transaction data.
  • Open communication was critical to manage changing priorities and expectations, as well as avoiding scope creep.
    • Many nonprofits have competing demands for their time so it’s important to ensure the organization has the necessary time and resources to devote to data analytics work. In our case, Grameen America had seven main initiatives to focus on, of which data and analytics was one of them. This year, other priorities took precedence and Grameen America will be revisiting the recommendations next year. 

undefined

As Data Advisory at DataKind San Francisco grows, we hope to partner with more nonprofits in need of data infrastructure guidance! If you or your organization are looking to partner with us, please reach out to sfbayarea@datakind.org to learn more.

The Grameen America Data Advisory project team and those who co-wrote this blog included: 

  • Marcus Berkowitz, Dana Lunberry, and Patricia Choi from Grameen America
  • Jaya Pokuri, Azeem Ansari, Richa Sharma, and Anjana Sundaram from DataKind San Francisco

We’d like to thank DataKind Global and San Francisco team members Mallory Sheff, Jay Bhatt, Melinda Tellez, Abhishek Kapatkar, and Melissa DiLoreto for their support and guidance throughout this project. 

Join the DataKind movement.

Quick Links

Read more posts
September 10, 2021
Lessons from a year of discovery in the economic resilience and well-being of communities around the world
From facing an unexpected bill to losing your job, to being repeatedly refused loans to start your own business, economic shocks can come in many forms and have wide-ranging effects on the independence, opportunity, and...
Read full story
June 28, 2021
Creating Data Science Solutions to Serve the Good of Many: Highlights from DataKind’s Discovery Day
The COVID-19 pandemic has deepened inequalities in accessing economic opportunity around the world, impacting those who were previously economically disadvantaged the hardest.
Read full story
December 20, 2021
Shining a Light on Community: Looking Back at DataKind’s Virtual DataDive® Event
We hosted a DataDive® event in fall 2021, and with it being the season of giving, we thought what better time to share some highlights and express our deepest gratitude to our partners, volunteers, and sponsors for making this yet...
Read full story
March 15, 2021
Powering Public Data for Communities: Highlights from Virtual DataDive®️ Event
On day one of our first virtual DataDive event of the year, volunteers who signed up for the housing insecurity project with the Future of Land and Housing (FLH) Program at New America rolled up their sleeves to get to work.
Read full story
Blog Archive