What 53,000 Maths Responses Can Teach Us About AI

How Rising Academies is sharing real-world learning data to improve AI-powered assessment and support research into education.

Artificial intelligence has significant potential to improve teaching and learning. But realising that potential will require more than developing new tools. It will also depend on making high-quality data, research and technology more accessible to the people and organisations working to improve education.

At Rising Academies, we believe that some of the most valuable contributions we can make to AI in education are those that others can learn from and build upon.

One example is AMMORE (African Middle-School Math Open Response), a publicly available research dataset developed through Rising's work on Rori, our AI-powered maths tutor.

53,000 student responses, one valuable research resource

AMMORE brings together 53,298 student responses to mathematics questions from 2,508 Rori users, covering topics including algebra and number operations.

The dataset includes the questions students were asked, their responses and human-reviewed assessments of whether those answers were correct.

Led by Rising researchers Owen Henkel and Hannah Horne-Robinson, alongside academic collaborators, the work was presented in the 2024 research paper Learning to Love Edge Cases in Formative Math Assessment.

The research explores an important challenge: how can AI accurately assess the many different ways students answer open-ended mathematics questions?

What did the research find?

The findings were encouraging.

Using large language models to assess particularly challenging responses, researchers found they could:

  • Increase overall grading accuracy from 98.7% to 99.9%.

  • Correctly grade 92% of challenging responses that the existing rules-based system struggled to assess.

  • Reduce errors in estimating student mastery from 6.9% to 2.6%.

These findings matter because even small improvements in grading accuracy can influence how educational technology understands a student's progress and identifies where additional support may be needed.

The research also demonstrates the value of real-world educational data from African learning contexts, which remain underrepresented in global AI research.

Sharing what we learn

AMMORE is publicly available for non-commercial research and development, with attribution, allowing other researchers and organisations to explore the data and build on the findings.

It also reflects a wider direction for Rising.

Alongside sharing research and datasets, we are exploring open-source AI infrastructure, including Kunye, an agent framework designed to help others build AI-powered educational tools.

We are also investing in rigorous research on AI tutoring and working with governments and technology partners to understand how AI solutions can be tested, adapted and adopted responsibly within public education systems.

Our aim is not simply to develop better tools ourselves, but to contribute evidence, resources and practical knowledge that help the wider education community improve learning.

Explore the resources:

Previous
Previous

New RCT Evaluation Underscores Rising’s Best-in-Class Results

Next
Next

Rising signs landmark MoU with City of Kigali