The Story...or rather...the Randomized Control Trial... of His Glory

Jeff Bloem has been working with the ICM Marketplace Ministry team as the research assistant in Kitale, Kenya since September of last year.  He will be with us until July of this year, after which he will return to the US and enter grad school in September.  I have worked with a number of interns over the years, all of whom have been unique in their own way.  Jeff, too, is unique and it has been a joy to work with him and get to know him.  Jeff lives and breathes economics.  Seriously.  All the time.  It's quite remarkable how much this guy reads and processes, and thinks and processes, and debates and processes, and processes and processes.  We almost never have a quiet car ride as we spend hours in the car each week driving from city to city to do this work.  It's been great fun and I've learned a lot from him. 

He wrote the article below about his reflections on his work here and it does a great job of defining the problem and explaining how our work is trying to address it.  Enjoy!

================================================================

Mary's bakery is bustling. So much so, that Ushindi Snacks cannot keep up with the demand for it’s fresh baked goods. Her business has room to grow but seems trapped at its current level.

Mary (left) owner of Ushindi Snacks

Beatrice sells beans and grains by the cup. Her shop is small, consisting of seven or eight bags of beans and rice under a temporary structure of lopsided scraps of wood and plastic tarp. Her unnamed business doesn’t grow; she has sold her goods off of the same 4x6 table for the past eight years.

Edward manages a consulting and management business, aptly named Strategic Management Firm LTD. His business grows on a quarterly basis and his employees are encouraged and equipped to work creatively and productively.

These are just three of the sixty-three business owners who signed up for a basic business skills training organized by their church in Kakamega, an emerging city in Western Kenya. And here lies the inherent problem.

The problem is not that Mary has to turn down sales and profits on a daily basis. It is not that Beatrice’s tabletop business has been the same for the last eight years. It is not that Edward is perhaps one of only three business owners attending the training that may actually be realizing his or her gifts and expressing traits of entrepreneurship.

While those are all very real issues, they are tangential to the larger problem at hand. The problem is obvious, almost never understood, and until recently rarely acted upon. Mary, Beatrice, and Edward are all different from each other. Half of this seeming banal platitude goes without saying, while, unfortunately, the other half often goes unsaid.

This problem is only exacerbated by the all too clear fact that the other sixty-three business owners are all different from each other in the same way Mary, Beatrice, and Edward are different. 

Ushindi Snacks in full operations

They all have different desires, different needs, different skills, different knowledge, and different businesses. A singular story about one person from this training would be—depending on your purpose for reading the story, either: (a) interesting and entertaining, (b) informative but probably misleading, or (c) uninformative and completely useless.

Here’s the thing. Those tangential problems discussed above remain because the larger problem is rarely addressed. Over the past half century developed countries have spent about $6.5 Trillion to assist the developing world develop and grow. Most of this money has been guided by loose fitting macroeconomic trends, fun to read stories containing anecdotal evidence, and bleeding heart good intentions.

Invariably what happens is some of the loose fitting macroeconomic trends predict actual growth, some fun to read stories containing anecdotal evidence are true for whole populations, and some good intentions lead to beneficial outcomes.

The negative rub is recognized when experts have a hard time discerning what worked and what didn’t. Today we have experienced remarkable progress in terms of poverty reduction and development around the world, and very little knowledge as to why that progress actually occurred. Until recently, poverty eradication and development have been run like a business that doesn’t keep accounting records.

This is all changing. In recent years researchers have been making remarkable advancements to project evaluation. The major innovation: randomized controlled trials. The methodology is taken almost directly from the medical profession and drug testing.

Jeff interviewing Edward, gathering statistics before the project starts.

For example how do we know if giving textbooks away to schools in developing countries will help raise test scores and improve school attendance numbers? The logic checks out—free textbooks mean more students read, and more kids learn. There is undoubtedly a heart-warming story of a young student who, after receiving a free textbook, went on to become a doctor. But does reality align with the logic? Was the student who would later become a doctor simply an exceptional student? Beyond speculation, what was the actual impact of these free textbooks?

The way to test this is to identify double the number of schools the program has funds to support. Randomly select half of those schools to receive free textbooks, while leaving the other half with none. Then compare the test scores of these two groups of schools to see if the textbooks had any sort of recognizable impact.  [As it turns out, giving textbooks away for free doesn’t work—at least in Western Kenya, where this four-year study took place. (Read more about it here.)]

Randomized control trials provide great feedback for governments and organizations looking to solve any sort of social problem. The major roadblock is evaluations of this nature is that they are often expensive and always challenging to execute well. Studying human beings in real life is much more challenging than studying a medical drug in a controlled laboratory environment. Human behavior, achievement, and success are dependent on seemingly endless factors. The best randomized evaluations control for as many of these factors as possible, but to do this completely in a real world setting is extremely challenging. It is impossible to say, “Lets run 2008 over again and this time there is no global economic crisis.” 

Beatrice's cereal business

For the past six months and the next twelve months a randomized evaluation is occurring in Western Kenya on the impact of the International Christian Ministries (ICM) Marketplace Ministry on churches and business owners, utilizing the curriculum from Partners Worldwide. This business-training curriculum is used in many countries where Partners Worldwide operates. In all of these places an NGO or financial institution organizes the training. Earlier this year the question was raised: would the effects of this faith-based business-training be increased if it were presented in a church supported context? In short, what happens when we train business skills in a church with an affirming pastor and a supporting community of believers?

The results of the evaluation will not be complete until 2015, but initial effects seem to be intriguing. During a training session last week the question was asked, “how many priests or ministers are in this room?” One hand was raised; it was the actual pastor of the church. After a discussion on the Biblical foundation for work, a Godly perspective on wealth, and a missional understanding on God’s intentions for the tasks and responsibilities of His people from Monday through Saturday, the question was asked again. This time, with the understanding that their work in their businesses was just as spiritual as the church’s work on Sunday, everybody in the room raised their hand. Everybody identified themselves as a priest or minister in their own life and work.

The significance of this monumental attitudinal shift remains to be seen. Remember, this story is anecdotal; the facts, however true they may seem, may not necessarily be reliable. The most encouraging aspect of this whole story is that, in the face of their differences, Mary, Beatrice, Edward and all the other attendees of the class experienced something similar and behaved in a uniform manner. 

This is the kind of finding the final evaluation aims to discover, a uniform change across a diverse population. We will then know if training business people in a church works better than training them in a financial institution. We will begin to actually know what works and what doesn’t.  

The Bible tells the story of His glory. Part of the fulfillment of that story is being told through hundreds of stories in Western Kenya. It lacks charm and rhyme but this is a randomized control trial for His glory. 

Jeff Bloem is a Research Assistant working on this evaluation in partnership with International Christian Ministries and Partners Worldwide at the African Theological Seminary in Kitale, Kenya. Follow him on Twitter @JeffBloem or email him at jeffb@partnersworldwide.org.