Top Marks for Rising in Final Year Liberia Evaluation

The final report of a randomised controlled trial of Liberia’s flagship education reform has praised Rising Academies for producing “a consistently positive pattern of results across learning, access, and safety dimensions.”

The three year ‘gold standard’ evaluation of the Liberia Education Advancement Program (LEAP) found that students in Rising schools progressed more than twice as fast as children in comparable government control schools.

Rising also had a positive impact on student enrolment, retention and attendance. School management and child safety were also better in Rising schools than control schools.

IPA Preliminary Results Brief: https://www.poverty-action.org/sites/default/files/publications/Beyond%20Short-Term%20Learning%20Gains.pdf

An earlier report based on data from the first year of the program had highlighted Rising’s promising learning outcomes. Over the subsequent two years, Rising remained one of the top-performing operators, while expanding its program to reach six times as many students and cutting its unit cost by 75%.

IPA Preliminary Results Brief: https://www.poverty-action.org/sites/default/files/publications/Beyond%20Short-Term%20Learning%20Gains.pdf

The LEAP (formerly Partnership Schools for Liberia) program was launched in 2016 to test whether inviting third-party providers to run government schools could help tackle longstanding challenges with Liberia’s education system. In Liberia, which recently ranked fifth from bottom of the World Bank’s Human Capital Index, few education reform initiatives have been able to demonstrate significant effects at scale, and fewer still at a cost that is financially viable.

The evaluation, led by researchers Justin Sandefur of the Center for Global Development and Mauricio Romero of ITAM, concludes that the results of the experiment were mixed. The LEAP program overall helped raise learning outcomes, but more modestly and at a higher cost than originally hoped. There were also large variations between the seven different providers in academic effectiveness, cost, and their impact on the welfare of students and teachers.

Based on the track record Rising has built in the first three years of the LEAP program, this year the Government of Liberia has asked Rising to expand to 87 schools, serving more than 21,000 students across 8 counties.

Commenting on the report, Rising’s Managing Director in Liberia George Cowell said: “We’ve learnt a huge amount in our first 3 years in Liberia and are pleased that the evaluation confirms we are delivering strong results in partnership with the Ministry. We now have a foundation from which we can scale our offer to reach many more children.”

Notes for editors:

1. The Liberian Education Advancement Partnership (LEAP), originally known as Partnership Schools for Liberia (PSL), began in 2016 with 93 public schools, and has since expanded to an additional 101 schools. The model is similar to charter schools in the United States or academies in the United Kingdom. LEAP schools remain public schools, charge no fees, and are staffed by public school teachers, but each school is managed by one of eight private contractors. More info here: https://www.poverty-action.org/sites/default/files/publications/Beyond%20Short-Term%20Learning%20Gains.pdf

2. The LEAP program, formerly PSL, started with 8 operators in September 2016: https://www.cgdev.org/publication/can-public-private-partnership-improve-liberias-schools

3. Rising Academies started with 5 schools in Year 1 of the program: http://www.risingacademies.com/blog/liberiapsl. In Year 2, Rising expanded to 29 schools across 7 counties (Montserrado, Bomi, Grand Bassa, Rivercess, Sinoe, Maryland and River Gee) after receiving an “A-rating” by the MoE:  http://www.risingacademies.com/blog/pslyear2. In Year 3, there was no expansion of the program due to the changing gov administration.

4. Rising achieved the highest learning gains in the midline report (July 2017) http://www.risingacademies.com/blog/pslrct.

5. Rising is a certified “B Corporation” http://www.risingacademies.com/blog/bcorp and began operations in Freetown, Sierra Leone, where the organisation runs low-cost private schools.

Contact: George Cowell, Managing Director

george.cowell@risingacademies.com 

+231881983963

Becoming a B Corp

I’m delighted to announce that Rising has become a Certified B Corporation®, and I wanted to take a minute to explain why.

We’re a company with a social mission. We exist to provide access to a high quality education to some of the millions of children around the world to whom it is currently denied, while also providing a satisfactory financial return to our investors. Some people would call us a social enterprise, a social business or a social venture. But even these terms are ambiguous, and mean different things in different places and to different people. The point is we are for-profit and for-purpose.

Whether non-state and particularly for-profit actors, whatever their professed motives, have a role to play in tackling the global learning crisis is controversial. (For a critical perspective, see this report by the Global Campaign for Education. For a defence of for-profit operators in education, see this article by Professor James Tooley.)

Around the world, this vigorous debate is reflected in the democratic process and the different policy choices that different countries have made about the appropriate role for private providers of education. 

At Rising, we can hardly pretend to be neutral but we’ve always tried to be respectful of the strongly held views on both sides of this debate. And while we may not be able to change many minds, we believe we have a responsibility to be open about what we stand for, transparent about we do, and clear about why we believe it is possible to be for-profit and for-purpose.

But we’ve found over the last few years that when we talk to people about Rising they struggle to reconcile in their minds our status as a company with our stated commitment to our purpose. And it’s not just among the global education community or the Twittersphere. A few months ago I joined an all-hands meeting with our team in Liberia, and a member of the operations team explained that when he’d first taken a job with Rising he’d had a hard time explaining to his friends and family that, while he was joining a company, it was a company dedicated to a social mission.

"There is one and only one social responsibility of business - to use its resources and engage in activities designed to increase its profits”, wrote Milton Friedman. Nearly fifty years later and it seems his words still cast a long shadow. 

That’s why we’re delighted to become a B Corp™. In so doing, we join a global community of companies, spanning 60 countries and more than 150 industries, that are committed to proving Friedman wrong - to being a force for good not just for their investors, but for their customers, employees, communities and other stakeholders.

To become a B Corp, every aspect of our business had to be assessed and audited in order to ensure that we meet rigorous standards of accountability and transparency, and that we are giving due consideration to the impact - in the broadest sense of that word - that Rising has on the world. 

The B Corp designation was created by a global non-profit called B Lab®. The number of Certified B Corps in East Africa is growing rapidly, and we’re proud to be the third in West Africa to achieve the designation - and the first in Sierra Leone and Liberia.

“Show, don’t tell” is a great maxim for writers and a pretty good one for companies too. It’s what we do, not what we say, that we should be judged on. But becoming a B Corp is an important symbol - for the students and families we serve, for our staff, for our investors, for other stakeholders - of our commitment to putting purpose at the heart of what we stand for as a company.

Evaluation results show strong gains - especially for girls

Isatu’s story

Schools are supposed to be a safe space for learning, but for Isatu Kabba, 14, that’s not how it felt at her old primary school. “There was no proper monitoring of who comes in and out of the school,” she recalls. “There was a lot of noise, a lot of bad language, bad behaviour from some of the boys, and less concentration.”

That’s changed since she enrolled for secondary school at one of Rising’s schools in Sierra Leone. She says she feels safer in the school environment, and more motivated because of the encouragement and quality teaching she gets from school staff. In last year’s public exams, she received one of the highest marks of any Rising student.

As a first-generation learner - neither her mother, a petty trader, nor her father, currently out of work, ever had the opportunity to go to school - Isatu is passionate about her education. She has continued on to the senior secondary level, focusing on the science stream. When she finishes, she wants to study medicine at university and dreams of becoming a doctor.

The experience of girls like Isatu, and just how different it is from the experience of girls in other schools, is one of the highlights of the comprehensive final report of a three year impact evaluation of Rising’s work in Sierra Leone. The study, by Dr David Johnson and PT Jenny Hsieh of Oxford University, finds that girls in Rising schools make faster progress than the boys, and 2-4 times the progress of girls in comparison schools.

As ever, if you want to jump straight to the study, it’s here. We’ve also put together this factsheet with the headline findings. For more of the background to the study and our reactions to it, read on.

Background to the study

I’ve written about the background to the study previously (here), but to briefly recap:

• The study assessed the learning gains of Rising junior secondary students in reading and maths from January 2016-May/June 2018. 

• These learning gains were benchmarked against the progress of comparison groups of students attending comparable private schools and government schools in the same neighbourhoods. 

• The incoming ability levels of these students at baseline were approximately similar.

• This period of 2.5 calendar years actually represents 3 academic years: the September 2015-July 2016 academic year was truncated as a result of the Ebola epidemic. 

• The study used an innovative computer-adaptive testing software to estimate the progress made by individual students more precisely than is usually the case with a paper and pencil test.

• In addition to cognitive measures, the study also explored progress in non-cognitive domains.

Figure 1. How the evaluation defines impact

There are some limitations to the study:

• It is a quasi-experimental not a fully experimental research design. Although the specific students sampled in each school were randomly selected, the schools themselves were purposively sampled. This creates the risk of sample selection bias, where the outcomes achieved by different schools are caused in part by differences in the backgrounds of the students themselves. 

• This is dealt with in part by checking that students’ prior achievement was approximately similar, and then focusing on progress from that baseline i.e. comparing learning gains (differences-in-differences) rather learning levels. But there are differences in the socio-economic status of students (SES) in the study, particularly between students at private schools (both Rising and other private schools) and their peers in government schools. The Oxford team plans to do more work to explore the impact of these differences.

• The research team also experienced significant sample attrition (that is, students surveyed at baseline not presenting for follow-up assessments) across all three school types. Again, this creates a risk of outcomes being shaped by differences in the composition of students presenting for assessments rather than because of anything to do with the schools themselves.

• The team have dealt with this mainly by reporting average scores for three sub-groups of students:

  a. All students presenting on that particular test (irrespective of whether or not they presented on any other test)

  b. All students who presented at the baseline, each end of year test, and the endline 

  c. All students who presented at the baseline and endline

• Group (a) is not useful for making comparisons over time as the students taking a test in one period may not be the same students taking that test in the next period. Group (b) is in some ways the most interesting as it allows us to see the shape of their learning trajectories over time. But the sample that took all 4 of these assessments is very small. Group (c) is larger, allows for a fair comparison of progress over time and is therefore the best group to focus on. While the pattern is broadly the same whether you look at Group (b) or Group (c), there are important differences in the results which is why it is helpful for the team to present both.

The baseline reports, and Year 1 and Year 2 midline reports, are all on our website if you want to check them out.

Headline findings

The key findings largely echo what the two midline reports found: RAN students make significantly larger gains than their peers in other schools, and these gains are more equitably distributed.

Finding 1. RAN students make significantly more progress than students in either comparison group in reading. 

• RAN students make 48% more progress in reading than the private comparison students. That is, RAN students learn as much in 1 year of schooling as their peers learn in 1.5-1.75 years. For those who care about such things, this is an effect size of 0.41.

• RAN students make 160% more progress in reading than the government comparison students. That is, RAN students learn as much in 1 year of schooling as their peers learn in 2.5 years. This is an effect size of 0.77.

Finding 2. RAN students make significantly more progress than students in either comparison group in maths. 

• RAN students make 120% more progress in maths than the private comparison students. That is, RAN students learn as much in 1 year of schooling as their peers learn in 2.2 years. This is an effect size of 0.42.

• RAN students make 133% more progress than the government comparison students. That is, RAN students learn as much in 1 year of schooling as their peers learn in 2.3 years. This is an effect size of 0.46.

Finding 3. Girls do much better in RAN schools than in comparison schools.

• In all school types, girls start behind boys at baseline. In government and private schools, they fall further behind every year - in fact, girls’ progress is less than half of the progress of boys. By contrast, in RAN schools girls actually progress faster than boys, and much faster (2-4x) than girls in either comparison group.

• Indeed, a lot of the difference in overall learning gains between Rising and non-Rising schools seems to be driven by the differential performance of girls.

Figure 2. Reading gains by gender

Figure 3. Maths gains by gender

Finding 4. RAN schools also do a better job of improving performance of students with lower prior achievement. 

• The Oxford team set thresholds to divide the sample into four ability bands.

• In reading 74% of RAN students who started in the bottom two ability bands at baseline had progressed to a higher band by the endline, compared to 48% and 47% in private and government comparison schools respectively. 

• In maths, 65% of of RAN students who started in the bottom two ability bands at baseline progressed to a higher band by the endline, compared to 34% and 39% in private and government comparison schools.

Reactions and reflections

First and foremost we’re grateful to the Oxford team for their work on this project over the last few years, and to the staff and students of the comparison schools for taking part.

We’re particularly delighted by the progress of our girls. Globally, the conversation about girls’ education can, at times, fall into one of two traps. The first is to get stuck at the why. As Echidna Giving argues. “it’s time to move from asking ‘Why serve girls’ to asking ‘How?’” The second is to talk about educating girls as if they are a different species. Yet as Dave Evans and Fei Yuan have recently argued, “one of the best ways to help girls overcome the learning crisis may be to improve the quality of school for all children.” A lot of the most effective interventions for improving outcomes for girls, they find, don’t specifically target girls. This study perhaps lends some weight to that view. Get the basics right for all students, and girls will flourish.  

I’m very proud of the hard work of our teachers, school leaders and central team in achieving these results. I heard a talk from an impact evaluation expert the other day who made the argument that most organisations try to do evaluations too quickly, before they have worked out all the kinks and settled on a stable model. I confess we’ve probably been a little guilty of that, launching this evaluation when we were less than 2 years old and when our schools had only been up and running for 6 months. But it also says a lot about our team that they welcomed this kind of rigorous scrutiny of their work when there was so much we were still figuring out. 

That relates to a third point. To be honest, the very idea of a stable model isn’t one that particularly resonates with us. Yes, I think we do have a more defined model today than we did three years ago. But whether it’s our curriculum or our school oversight systems or our approaches to teacher coaching, we’re always looking for ways to innovate and improve. 

The results themselves show why this is so important. A mantra at Rising is that “however well we do, we always strive to do better.” Compared to other schools, the learning gains here are impressive. It’s particularly striking that they are achieved by enabling more disadvantaged groups of learners - girls, and those with lower prior achievement - to progress much faster than is the case in other schools. 

But in absolute terms, our students still lag behind where they should be. Their learning trajectories are much steeper than their peers in other schools, but there’s so much more we need to do to keep bending them upwards even further.

So, to repeat something else we like to say round here, “our first draft is never our final draft.” Here’s to the next draft.

Leading the Way Forward

This post by Natalie Lamprell, Rising’s Director of Curriculum and Teacher Development, originally appeared on the Global Schools Forum blog.

---

Principal Chitra Vishwanathan welcomes us into her office at Abhuyudaya Nagar MPS, a government school in Mumbai supported by the Akanksha Foundation. We sip on chai as Chitra leans forward in her chair, smiles and tells us about her school. She has 35 teachers; some are veterans, some are ‘Teach for India’ newbies and some, regardless of experience, need more of her instructional support. She has 565 students and invests time in building strong relationships with their parents. Her walls are adorned with personalised ‘Super Mum and Dad’ displays.

Super Dad Display in the office of the Abhuyudaya Nagar school, Mumbai

But there’s only so much chai and chat Chitra can spare. As Uncommon Schools Chief Schools Officer Paul Bambrick-Santoyo states in his book Leverage Leadership, school leaders should be “relentless about spending time on the most important things”. Before long, Chitra says her goodbyes and heads off into the school and back to her core mission. She begins the day with a learning walk before monitoring today’s attendance data.

Seeing Chitra and her school in action filled me with hope – an excellent school leader is the key to unlocking excellent teaching and learning. But it also made me think about how you address this critical need for focused, skilled school leaders in more remote locations and at scale.

Chitra was just one of the inspiring school leaders I met during a week-long study tour to Mumbai and Delhi arranged by the Global Schools Forum. There are 260 million schoolchildren in India attending over 1.5 million schools. The task of guaranteeing them a great education is gargantuan. The Study Tour was designed to expose participants to some innovative non-state operators in this context. Many of the attendees work in Sub -Saharan Africa where the scale of the task is also daunting and is set to get even greater; by 2050 three quarters of a billion children will attend school. Throughout the week we found ourselves grappling with the same key question: how can we recruit, equip and retain excellent school leaders? 

Phonics lesson in progress in a Peepul classroom, Delhi

At the Peepul-SDMC partnership school in Dayanand Colony in Delhi, Education Director Urmila Chowdhury takes a grow-your-own approach to her future school leaders. She recently promoted two teachers working in the school to Academic Coordinators. They may be younger and less formally qualified than potential external hires, but their time with the organisation means that they have the DNA of Peepul – a relentless focus on best classroom practice. They are deeply committed to the community they were already serving as teachers and they are here to stay.

Matthew Bidwell, a Professor of Management at the Wharton School, has studied the merits of hiring externally versus promoting from within. Much of it comes down to a question of relationships. In a 2011 paper he notes, “we have relationships in organizations that are key to getting work done and a set of structures and routines we need to know” . Bidwell’s analysis feels especially true of schools, whereschool leaders must build a multitude of relationships with students, parents, teachers, middle leaders, the wider community and education ministries. Mainak Roy CEO of Simple Education Foundation (S.E.F) agrees that relationships are paramount. The management team of S.E.F spend lots of their time in the government schools they support to build these relationships before rolling out new initiatives.

In his paper Bidwell goes on to say that an external hire often slows down the performance of the whole team while they try to get the new person up to speed. By contrast, an internal hire can hit the ground running and serve as an example to the rest of the team about what behaviours get rewarded. Seeing their peers progress into management roles can motivate other teachers to strive to achieve the same progress, ensuring there is a talent pipeline within the organisation who, if needed, can be deployed to locations further afield.

At the Pratham learning centre in Delhi

But the step from teacher to school leader is a big one. How do we equip our leaders with the tools to approach their role with what Bambrick-Santoyo calls “an engineer’s obsession”? A clue came during one of our first visits, to Lead School Mangaon Campus, three hours from Mumbai. Their bespoke school management system allows all the school leaders in their network to log on and crunch item level student assessment data and then coordinate effective remediation programmes (like the rigorously evaluated TaRL – Teaching at the Right Level - sessions Pratham have pioneered). Accurate, regular, timely data is driving the academic performance of their students and HQ can look at these results in real time remotely.

Across the week we discussed how different school networks were finding their own route to recruiting, developing and supporting school leaders. But running through all these approaches was a common thread: put your mission at the heart of what you do. 

I am keen to continue the discussions we began in India about school leadership. In the new year GSF’s Keya Lamba and I intend to set up a Chief Academic Officer community of practice and I’d like to work with this group to chart the roadmap for our instructional leaders from novice to excellent. 

Thank you to GSF for hosting the trip, to the schools we visited for being so welcoming and inspiring, and to my fellow delegates for all the learning (and the laughs along the way).