Testing Reading Skills

Since 2006, early grade reading assessments (EGRA) have been used to rigorously diagnose national and system level gaps in reading competencies among students in over 50 countries and 70 languages. Results have informed education policy reform and improvement of teacher training around the world, contributing to a reading revolution in international development. For example, USAID’s latest education strategy aims at improved reading skills for 100 million children in primary grades by 2015. The Global Fund for Education is also promoting investment in early math achievement, and a similar, complementary assessment methodology—the early grade math assessment, EGMA—exists.

Increasingly projects are required to collect extensive and rigorous data to show evidence of program impact. While the traditional, paper-based forms of EGRA and EGMA provide high quality, informative data for this purpose, the costs of measuring large samples (2,300 is the average size for national baseline samples, according to the EGRA Tracker on www.eddataglobal.org) two or three times over the course of a project is significant, and up to 20% of those costs can be directly related to costs of the paper-based administration (logistics, copies, supplies, etc.). The largest share of paper-related costs is the preparation of the data entry system and transcription from paper forms into the system for data analysis. This process is prone to human error, compromising the quality of the data and delaying the availability of the data for analysis and decision-making. 

Solution

Using an Amazon Kindle Fire to conduct an oral reading assessment with a child in Nairobi, Kenya. (Photo: S. Pouezevara)

RTI has designed and developed an electronic data collection system for EGRA and EGMA called Tangerine®. Tangerine is  open source data collection software designed for mobile computers, particularly touch-screen tablets. It is used to record student responses to early grade reading and mathematics assessments.

Tangerine uses cutting-edge open source technologies including CouchDB and JQuery Mobile providing compatibility across a wide range of mobile devices and languages. A critical feature is that it enables devices to synchronize data from various assessors working simultaneously on different assessments with a central database over a variety of connection types. Some of the advantages to this approach include:

• Rapid availability of assessment data
• Reduced amount of paper and supplies, as well as the logistics required to prepare and manage the paper assessments (photocopying, sorting, stapling, packaging, etc.)
• Reduced measurement errors due to missing fields, data transcription errors, invalid data types or formats, illegible or incomprehensible data, etc.

Field Testing

Tangerine was field tested in Nairobi, Kenya in January 2012 with a sample of 200 children; each child completed English reading, Swahili reading, and math tests for a total of 600 tests administered with the Tangerine interface.

The hardware chosen for this pilot was the Kindle Fire, by Amazon, a low-cost tablet e-Reader with the Android operating system. The necessary components—Mobile Futon and Tangerine—were loaded directly onto the devices through wireless replication from a central server without making any changes (“rooting”) the operating system. This means it can also continue to be used for its intended purpose—as a mobile tool for reading eBooks and consuming multimedia—a useful complementary feature for teachers and trainers in the program.

The results of the field trial were important for confirming the feasibility of such an approach and also discovering ways in which Tangerine can be improved to further ensure quality through usability and data security.