^{*}

Edited by:

Reviewed by:

*Correspondence:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Evaluation of educational interventions is necessary prior to wide-scale rollout. Yet very few rigorous studies have been conducted on the effectiveness of tablet-based interventions, especially in the early years and in developing countries. This study reports a randomized control trial to evaluate the effectiveness of a tablet intervention for supporting the development of early mathematical skills in primary school children in Malawi. A total sample of 318 children, spanning Standards 1–3, attending a medium-sized urban primary school, were randomized to one of three groups: maths tablet intervention, non-maths tablet control, and standard face-to-face practice. Children were pre-tested using tablets at the start of the school year on two tests of mathematical knowledge and a range of basic skills related to scholastic progression. Class teachers then delivered the intervention over an 8-weeks period, for the equivalent of 30-min per day. Technical support was provided from the local Voluntary Service Overseas (VSO). Children were then post-tested on the same assessments as given at pre-test. A final sample of 283 children, from Standards 1–3, present at both pre- and post-test, was analyzed to investigate the effectiveness of the maths tablet intervention. Significant effects of the maths tablet intervention over and above standard face-to-face practice or using tablets without the maths software were found in Standards 2 and 3. In Standard 3 the greater learning gains shown by the maths tablet intervention group compared to both of the control groups on the tablet-based assessments transferred to paper and pencil format, illustrating generalization of knowledge gained. Thus, tablet technology can effectively support early years mathematical skills in developing countries if the software is carefully designed to engage the child in the learning process and the content is grounded in a solid well-constructed curriculum appropriate for the child’s developmental stage.

Research with digital educational software has shown increased motivation (

According to the ^{1} standards in mathematics across Malawi are very poor. In tests conducted in 2004, 98% of pupils in Malawi did not possess mathematical skills beyond basic numeracy and Malawi was ranked the second lowest for mathematics of the 14 participating countries in the report. Similar findings were reported by USAID in tests conducted across 50 primary schools in Malawi in 2010^{2}. The USAID report stated that children could “only answer the most elementary and procedural of items with any sense of confidence” (p. 42). Thus radical shifts in the teaching of mathematics are needed in Malawi to raise academic standards.

Recent advances in digital technology could provide an alternative or additional means of support to current classroom practices if technology-based interventions are shown to be effective. Mobile technologies, such as tablets, might be particularly suited to developing countries like Malawi, where class sizes are typically large (average 76 primary school children per class in Malawi in 2011 according to the UNESCO Institute for Statistics^{3}) and the number of classrooms is usually small, leading to severe overcrowding. Classes are based on ability (Standard) rather than age and repetition rates are very high in Malawi, as many children fail to progress over a school year, so are required to take the year again. Typically, girls fare less well than boys in primary education in Malawi, with many dropping out of school by age 8 years. This results in a long-lasting gender disparity that affects economic growth in multiple ways. In addition, the quality of teaching in Malawi can vary greatly and many teachers may have received poor or little training. Coupled with few resources to assist with teaching, the learning environment is often extremely impoverished. Consequently, within the current primary school system in Malawi it is difficult to track developmental progression of individual children, identify those that are falling behind, and provide child-centered individual tuition at a consistent level of quality.

Digital technologies, such as tablets, afford many advantages to both the class teacher and their pupils, if the software is well-designed and the content is grounded in a solid well-constructed curriculum that is appropriate for the child’s developmental stage (

Yet despite the affordances of mobile technologies to support learning, even in the early years, there is a dearth of research providing an evidence base to justify their use in an educational context. A recent meta-analysis that examined 39 studies published between 1990 and 2012 reporting on the use of technological games in an educational context showed that technological game-based approaches were significantly more effective than typical instructional methods in improving learning (

This study reports the first randomized control trial (RCT) to evaluate the effectiveness of a tablet-based intervention to support mathematical ability in primary school children in Malawi. onebillion©, a not-for-profit publishing company, developed the tablet-based maths intervention that was evaluated. The intervention is currently being piloted in several primary schools in Malawi. Based on the National Primary Curriculum for mathematics delivered throughout primary schools in Malawi it is given in Chichewa – the official language in Malawi – via individual electronic tablets (Apple© iPad minis) and incorporates all of the features of well-designed software outlined above. The main aims of this study were to: (i) evaluate the effectiveness of the maths tablet intervention over normal classroom practice in supporting the development of mathematical ability in primary school children; (ii) establish the most appropriate Standard in which to implement the maths tablet intervention; and (iii) determine if girls respond differently to the maths tablet intervention than boys.

A RTC was conducted in which children from Standards 1–3 at a medium-sized primary school (Biwi Primary School) situated in an urban area of Lilongwe, the capital city of Malawi, were randomized to one of three groups: maths tablet intervention (treatment group); non-maths tablet control (placebo group); and normal practice (control group). Children were tested on mathematical ability and basic skills associated with scholastic progression immediately before (pre-test) and after (post-test) the 8-weeks intervention period.

The Ministry of Education in Malawi gave consent for the study to take place and selected Biwi Primary School as the test school in Lilongwe. Prior to study commencement, consent was also gained from the parent association at Biwi Primary School and the Community Chief of the region where Biwi Primary School is located.

Composition of the study sample at each stage of the RCT.

Study phase | Number of children | ||
---|---|---|---|

Eligible (all children in S1–S3 attending school on first and second day of school year) | 350 |
||

Randomized (to one of three groups) | 318 |
||

Excluded (due to group size study design constraints) | 32 |
||

Randomized to group | 115 |
90 |
113 |

Pretested | 112 |
83 |
109 |

Not pretested (i.e., absent at pre-test) | 1 |
2 |
3 |

Received intervention | 113 |
85 |
112 |

Did not receive intervention (i.e., transferred before pre-test) | 2 |
5 |
1 |

Post-tested | 105 |
82 |
103 |

Lost to follow up (i.e., absent or transferred by post-test) | 8 |
4 |
9 |

Excluded from analysis (i.e., absent at pre-test or post-test or transferred) | 11 (9.6%) |
11 (12.2%) |
13 (11.5) |

In total, 113 children from Standards 1–3 received the tablet-based maths intervention and 112 children from Standards 1–3 received normal classroom practice during the 8-weeks intervention period. Additionally, 85 children from Standards 2–3 received a tablet-based control intervention that did not involve the maths software. This group was critical as it served as a placebo group to enable the generic effects of using tablets over the specific effects of the onebillion© maths software to be differentiated. In addition, this placebo intervention controlled for additional factors that might influence performance compared to normal classroom practice. For example, the tablet interventions were delivered in a small classroom to groups of 25 children (see Location of Interventions) whereas within the normal classroom practice maths instruction was delivered in a large classroom to groups of 70–80 children. Children in the tablet interventions (treatment and placebo) were thus singled out from the rest of the class whilst receiving the intervention, so received additional attention compared to their normal practice peers. Children taking part in the tablet interventions (treatment and placebo) also had more practice at using the device compared to children in the normal practice group. As two of the three assessments of mathematical ability given at post-test were delivered via the tablets, this placebo group was thus crucial in distinguishing specific effects of the maths software from these other extraneous variables associated with the tablet interventions that could influence performance. As such, this placebo group served to control against novelty and/or Hawthorne effects. Due to the limited numbers of children enrolled in Standard 1 at the start of the school year, Standard 1 children were not allocated to this non-maths tablet control intervention.

For all of the analyses below, a final sample was determined which included all children from Standards 1–3 (

Descriptive statistics of the final sample.

Standard | Group |
|||
---|---|---|---|---|

Maths tablet | Non-maths tablet | Normal practice | ||

NA | ||||

Age (months) | 84 (10.8) 73–118 | 81 (4.5) 74–93 | ||

Gender ( |
10:12 | 5:15 | ||

Age (months) | 91 (8.7) 74–109 | 91 (9.5) 79–119 | 92 (10.7) 75–125 | |

Gender ( |
18:20 | 19:16 | 17:20 | |

Age (months) | 106 (15.9) 87–161 | 109 (13.4) 87–147 | 105 (14.2) 79–139 | |

Gender ( |
22:22 | 24:20 | 24:19 | |

Children from Standards 1–3 were randomly allocated to one of the following groups.

This intervention consisted of four different apps developed by onebillion©: Masamu (Chichewa for Maths) 1, Masamu 2, Count to 10, and Count to 20. The apps are based on the National Primary Curriculum that is delivered in Malawi and teach core mathematical concepts (MCs) in a structured manner through several colorful and engaging sets of activities delivered in the local language, Chichewa. Children worked through the apps at their own pace and could practice particular activities as often as they desired. To progress to the next set of activities, children needed to ‘pass’ a quiz built into the software that assessed knowledge of the set of activities the child had been working on. Teachers monitored progress of individual children through achievement charts, in which a star was awarded to each child as they passed a particular quiz. Passing a quiz required 100% accuracy, thus thoroughly assessing children’s progression through the apps.

This placebo intervention consisted of four different apps that are freely available to download from the Internet: Music Sparkles developed by Kids Game Club©, Drawing Pad developed by Darren Murtha Design©, and Toca Tailor and Toca Hair Salon developed by Toca Boca AB©. These apps were chosen because they are educational (supporting musical ability and design skills), receive good customer ratings, are non-verbal, and do not involve concepts taught in the maths tablet intervention. Additionally, these apps require children to interact with the tablet in terms of manual and attentional processes in a similar manner to the onebillion© maths apps, thus equating, as far as possible, practice of manual and attentional skills across the two tablet groups. Throughout the intervention period, children were free to choose whichever app they wanted, as often as they wanted.

This consisted of normal instructional practice delivered in primary schools across Malawi. Children followed the Malawi National Primary Curriculum, delivered in Chichewa by their class teacher. Basic numeracy (including mathematics) is taught from Standard 1. Delivery was face-to-face with the class teacher giving maths instruction through the aide of a chalkboard, to a group of children sat on the floor. Most of the children had small notebooks and pencils in which to practice particular maths questions, written on the board or dictated orally (for classes taken outside) by the class teacher. For example, the teacher might write the following sum on the chalkboard, 7+2, and wait whilst the children completed this, in their notebooks. The children would then bring their notebooks to the teacher for their answer to be marked.

As the maths tablet intervention is also based on the Malawi National Primary Curriculum, children in each of the groups were exposed to the same basic curriculum. However, the different modes of implementation (tablet versus face-to-face) resulted in instructional differences, as both the content and delivery of teaching mathematics varied across the groups. For example, the tablet technology afforded individual child-centered learning so each child in the maths tablet intervention could work through the program at their own pace. Accordingly, the amount of time spent on particular topics varied across individuals within the maths tablet group, as did the rate of progression across the program. Children also received feedback through their interactions with the maths software. Positive feedback was received immediately upon successful completion of an item, after which the software introduced the next item to the child. In contrast, with normal classroom practice, the class teacher determined pace of delivery and amount of time spent on particular topics, and teaching was delivered to a large group of children (∼70–80 children). Class teachers gave feedback to individual children when they presented their response to the teacher after completing a particular question posed. However, compared to the maths software intervention, feedback per item to individual children was delayed in the standard instructional practice, as class teachers simply could not reach individual children as they completed a particular task. Although both of the maths interventions followed the same curriculum, the content was specific to each intervention. In the maths tablet intervention the content was consistent for all children and had been written by a well-reputed maths author. In standard classroom practice, the class teacher determined the content, as there was not a standard maths text that was followed.

Both of the tablet-based interventions (treatment and placebo) were administered in a purpose-built ‘Learning Centre’; a small classroom within the grounds of Biwi Primary School but detached from the main school buildings. The Learning Centre housed up to 25 children at a time, sat on bamboo mats, individually using tablets connected to personal headsets. A class teacher was present in the Learning Centre during the tablet-based interventions primarily to assist the children with using the technology. The Malawi branch of VSO provided additional technical support to class teachers throughout the intervention period. Class teachers responsible for overseeing the tablet-based interventions were given a training session prior to the intervention starting, in which they were shown how to use the tablets including turning the tablets on and off, connecting the headsets, storing the headsets, charging and securely storing the tablets, and navigating through the software. They were also shown how to record attendance and progress with the maths software for individual children throughout the intervention period using attendance/achievement charts specifically designed for this study that were pinned to a board in the Learning Centre. Children allocated to the normal practice intervention received the National Primary Curriculum in their usual classroom setting.

Tablet technology was used in the assessment of individual children on tasks of mathematical ability and basic skills as it enabled performance to be measured objectively, with large groups of children, within a short period of time. Accordingly, a new assessment app was developed specifically for this study with instructions given in Chichewa. The app consisted of two measures of mathematical ability and six measures of basic skills known to be associated with scholastic progression, namely manual processing speed, manual coordination, visual attention, short term memory, working memory, and spatial intelligence (

Two measures of mathematical ability were developed for the tablet-based assessment app Maths curriculum knowledge (CK) consisted of 50 quiz items taken from the Masamu 1 and Masamu 2 apps and thus assessed CK that is specific to the maths intervention. MCs consisted of 48 questions assessing a conceptual understanding of mathematics, similar to that used in the

To assess generalization of CK learned through the tablet-based intervention to a more conventional paper and pencil context a third measure of mathematical ability was developed. Maths curriculum knowledge generalization (CKG) was an additional measure given at post-test only, that included 50 new items based on those used in the maths tablet intervention, that the children had not seen before. Paper tests were prepared for each child, with their photograph and study number on the front page to ease identification of individual children. These were administered in groups of 25 children in the Learning Centre, with an assistant pointing to each question as a class teacher read out the question in Chichewa.

The study commenced in September 2013 on the first day of the school year and continued for a total of 10 weeks. Enrolment and pretesting was carried out over the first week by the author and three assistants (a VSO volunteer and a programmer and a translator from onebillion©). The intervention took place over the following 8 weeks and was implemented by class teachers at Biwi Primary School, with technical support by VSO. Post-test assessments were then conducted in November 2013, in week 10, by the author and two assistants (the VSO volunteer and the programmer from onebillion©).

Over the first 2 days of the school year all children that were eligible for the study were enrolled, randomly allocated to one of the three groups, and assigned a study number. Enrolment included photographing each child so as to ease identification of individual children assigned to the different groups. Gender, Standard, and Classroom were also recorded for each child and date of birth was collected through school records. To ensure children were enrolled just once, a temporary mark was placed on the child’s hand to indicate they had been registered. A computer program, written by onebillion©, was used to automatically assign children randomly to one of the three groups, prior to children being given the pre-test assessments.

Groups of up to 50 children maximum were pre-tested or post-tested at the same time, in a regular classroom in the main school building. Prior to assessment, the tablets were set up with each child’s photograph, study number, and intervention group. This aided conducting the pre-test and post-test assessments as the tablets could be handed out reliably to individual children and their data was stored automatically to their allocated group. Children were tested per group (e.g., Standard 1 maths tablet intervention) where possible. Children were collected from their classroom by one of the assistants and escorted to the classroom where the assessments were carried out. When entering the assessment classroom the researchers (author) or assistant handed each child the tablet set up specifically for them and a headset. Children sat on the floor to carry out the assessments. The researcher stood at the front of the group and demonstrated each task in the assessment app via a large tablet with instructions being orally translated into Chichewa by a class teacher. Children first completed a brief demonstration of how to touch and move objects on the tablet screen, so as to familiarize them with using the technology. They then carried out each task in the assessment app, in a specified order. To start each task children were required to swipe a large white dot on an introduction screen after which the first practice trial of that task was administered. Children worked through each task, in sequence, until they had completed all of the assessments. A large star appearing on the screen and exploding into smaller stars marked the end of the assessment app. Children took around 40–60 min to complete the assessment app, depending on their level of ability. Upon completion, children handed their tablet and headset to the researcher or assistants and returned to their class.

The intervention period lasted 8 weeks (40 school days). Each of the tablet-based interventions (treatment and placebo) was administered for 20 days. Children from Standards 2–3 received the tablet-based intervention to which they had been allocated for 1 h on alternate school days (20 h maximum across the 8-weeks intervention period). Children from Standard 1 received the maths tablet intervention for 30 min on alternate school days (10 h maximum across the 8-weeks intervention period). Class teachers responsible for overseeing the tablet-based interventions drew up a timetable, in which one set of teachers was allocated to the maths tablet intervention and a second set of teachers was allocated to the non-maths tablet control intervention. As far as timetabling permitted, children from Standards 1–3 that were allocated to the maths tablet intervention group received the maths tablet intervention whilst children in the other two groups (non-maths tablet intervention and normal practice) received the standard maths instruction that is normal practice in Malawi. This was in an attempt to equate total time spent on maths education across the three groups.

The tablets used in this study were iPad Minis as these are a suitable size for young children to hold and interact with whilst seated on the floor. They also have the battery capacity required for implementing this intervention on a daily basis. onebillion© placed 50 iPads into Biwi Primary School specifically for this study. This enabled 25 iPads to be used for the intervention on alternate days whilst the other 25 iPads were being charged. To ensure children received the correct intervention, 25 iPads with red covers had only the maths intervention software installed on them, and the other 25 iPads with blue covers had only the non-maths software installed on them. Furthermore, for the maths intervention group, iPads were personalized to individual children using their photographs and study number allocated at enrolment. This ensured that each child used the same iPad throughout the intervention period and allowed electronic data to be recovered on individual usage. Class teachers overseeing the tablet-based interventions were responsible for charging the iPads when not in use. At the end of the study, the maths tablet intervention was made accessible to all children from Standards 1–3 at Biwi Primary School, so as to enable each child to potentially benefit from this new educational technology resource.

For all of the analyses conducted the final sample of 283 children, reported in

To evaluate the effectiveness of the maths tablet intervention over normal classroom practice in supporting the development of mathematical ability, mean performance for each of the groups was compared across time, at both pre-test and post-test, using mixed (where Time was the within-groups variable and Group was the between-groups variable). Analyses of variance (ANOVAs), for Standards 1–3, on each of the measures of mathematical ability. Separate 2 (Time: pre-test, post-test) × 3 (Group: maths tablet, non-maths tablet, normal practice) mixed ANOVAs were conducted for Standard 2 and Standard 3 and a 2 (Time: pre-test and post-test) × 2 (Group: maths tablet, normal practice) mixed ANOVA was conducted for Standard 1. In addition, for Standard 2 and Standard 3, separate one-way between-groups ANOVAs (Group: maths tablet, non-maths tablet, normal practice) were conducted to examine generalization of maths CK to a different paper and pencil format.

To establish the most appropriate Standard in which to implement the maths tablet intervention, between-group effect sizes (using Cohen’s

To determine if girls respond differently to the maths tablet intervention than boys, mean performance for the combined measure of mathematical ability of girls and boys receiving the maths tablet intervention was compared to that of girls and boys receiving normal practice at pre-test and post-test, using a 2 (within-groups variable Time: pre-test, post-test) × 2 (between-groups variable Group: maths tablet; normal practice) × 2 (between-groups variable Gender: girls, boys) mixed ANOVA.

Results from the analyses reported above are summarized in

Group performance [mean (SD), minimum–maximum] at pre-test and post-test and percentage performance gain for the tests of mathematical concepts (MCs; maximum score 48), Maths curriculum knowledge (CK; maximum score 50), and Maths curriculum knowledge generalization (CKG; maximum score 55).

Standard |
Group |
|||
---|---|---|---|---|

Maths tablet | Non-maths tablet | Normal practice | ||

NA | ||||

Pre | 2.0 (1.7) 0–5 | 3.4 (4.3) 0–13 | ||

Post | 5.1 (4.6) 0–17 | 4.5 (4.3) 0–13 | ||

Pre | 2.4 (2.7) 0–12 | 2.7 (1.7) 0–6 | ||

Post | 7.7 (6.3) 0–24 | 5.9 (5.3) 0–20 | ||

Pre | 8.6 (6.5) 0–20 | 10.1 (5.7) 0–21 | 8.4 (6.3) 0–19 | |

Post | 14.6 (6.6) 0–24 | 12.5 (7.3) 0–23 | 10.5 (6.5) 0–19 | |

Pre | 6.4 (6.3) 0–24 | 8.3 (8.1) 0–28 | 5.5 (5.3) 0–21 | |

Post | 20.7 (10.3) 0–37 | 15.1 (8.5) 2–31 | 10.8 (7.4) 0–26 | |

Post | 23.8 (9.1) 7–41.5 | 17.4 (10.3) 0–36 | 20.7 (8.4) 4–39 | |

Pre | 14.9 (5.7) 0–23 | 15.4 (5.6) 0–22 | 14.9 (5.2) 0–22 | |

Post | 19.5 (5.2) 5–27 | 19.3 (5.1) 4–26 | 18.6 (6.0) 0–26 | |

Pre | 13.5 (9.3) 0–34 | 14.8 (10.1) 1–37 | 11.0 (10.3) 0–33 | |

Post | 35.2 (7.0) 19–46 | 24.9 (8.0) 3–36 | 23.4 (6.9) 7–36 | |

Post | 33.5 (7.3) 19–47.5 | 20.0 (6.7) 7–36 | 27.2 (8.1) 10–43 |

Between-group effect sizes (Cohen’s

Standard | Measure |
||
---|---|---|---|

MC | CK | CKG | |

1 | 0.135 | 0.310 | NA |

2 | 0.626 | 1.119 | 0.354 |

3 | 0.161 | 1.698 | 0.818 |

Performance [mean (SD) minimum–maximum] of girls and boys across Standards 1–3, at pre-test and post-test, on the combined measure of mathematical ability (maximum score 49).

Standard |
Group | |||
---|---|---|---|---|

Maths tablet | Normal practice | |||

Pre-test | Girls | 2.3 (1.4) 0.0–4.0 | 3.6 (3.0) 0.5–8.5 | |

Boys | 2.1 (1.9) 0.5–6.0 | 2.9 (2.7) 0.0–9.5 | ||

Post-test | Girls | 7.4 (5.4) 0.5–15.5 | 5.5 (4.7) 0.0–10.0 | |

Boys | 5.6 (3.8) 2.5–14.0 | 5.0 (3.9) 0.0–13.0 | ||

Pre-test | Girls | 6.3 (5.2) 0.0–20.5 | 6.0 (4.7) 0.0–14.5 | |

Boys | 8.5 (6.3) 0.5–21.5 | 7.7 (5.4) 1.0–19.0 | ||

Post-test | Girls | 15.6 (8.5) 0.0–30.0 | 9.0 (6.1) 0.0–20.0 | |

Boys | 19.5 (7.1) 5.0–30.0 | 12.1 (6.4) 0.5–22.5 | ||

Pre-test | Girls | 14.5 (6.9) 3.0–26.5 | 11.7 (4.4) 1.0–20.0 | |

Boys | 13.9 (6.7) 0.0–27.5 | 14.6 (6.5) 1.5–25.5 | ||

Post-test | Girls | 28.3 (5.0) 16.5–35.0 | 20.4 (5.9) 4.0–26.5 | |

Boys | 26.4 (5.9) 13.0–33.5 | 21.8 (4.4) 13.0–29.5 | ||

For the test of MCs the 2 (Time: pre-test, post-test) × 2 (Group: maths tablet, normal practice) mixed ANOVA revealed a significant main effect of Time,

Similar results were found with the test of CK. The 2 (Time: pre-test, post-test) × 2 (Group: maths tablet, normal practice) mixed ANOVA revealed a significant main effect of Time,

For the test of MCs results from the 2 (Time: pre-test, post-test) × 3 (Group: maths tablet, non-maths tablet, normal practice) mixed ANOVA showed a significant main effect of Time,

With the test of maths CK, ANOVA results revealed a significant main effect of Time,

In addition, one-way between-groups ANOVA (Group: maths tablet, non-maths tablet, normal practice), conducted on the CKG test administered at post-test, revealed a significant main effect,

For the test of MCs the 2 (Time: pre-test, post-test) × 3 (Group: maths tablet, non-maths tablet, normal practice) mixed ANOVA revealed a significant main effect of Time,

With the test of CK, ANOVA results showed significant main effects of Time,

These effects of the maths tablet intervention on CK generalized to paper and pencil format at post-test. One-way between-groups ANOVA on the CKG test revealed a significant main effect of Group,

Results from the ANOVAs conducted for each Standard revealed the maths tablet intervention group showed: (1) Significantly greater gains in performance over time compared to the normal practice control group on the MCs test for Standard 2 children. (2) Significantly greater gains in performance over time compared to both the non-maths tablet placebo group and the normal practice control group on the Maths CK test for Standard 2 and Standard 3 children. (3) Significantly higher performance on the paper and pencil generalization test (CKG) at post-test than the non-maths tablet placebo group for Standard 2 children, and both the non-maths tablet placebo group and normal practice control group for Standard 3 children.

Whilst clear benefits were found after using the maths tablet intervention for Standard 3 children, results for Standard 2 children are somewhat mixed. For Standard 2 children on the test of MCs, although mean performance at post-test was higher for the maths tablet group (14.6) than the non-maths tablet placebo group (12.5), this difference did not reach significance. Despite this, over the 8-weeks intervention period, the maths tablet group made more improvement (12.5%) than the non-maths tablet placebo control (5%) whose performance gains was similar to the normal practice control group (4.4%). This arises from the higher level of performance at pre-test on the MCs test for the non-maths tablet placebo group (10.1) compared to both the maths tablet group (8.6) and the normal practice control group (8.4). Accordingly, within-groups effect sizes computed across the intervention period were large for the maths tablet group (Cohen’s

Overall, these results indicate that intervention with the tablet-based maths software can add significant value to normal classroom practice in Malawi at supporting development of mathematical skills in primary school children. Dedicated time spent interacting with tablet technology in small groups away from the rest of the class is insufficient to support mathematical skills, as the gains made by the non-maths tablet placebo group did not differ significantly from the normal practice control group. In general, compared to normal practice, intervention with the tablet-based maths software seems to support a conceptual understanding of mathematics in Standard 2 and more specific curriculum-based knowledge in Standards 2 and 3, which generalizes to different formats by Standard 3, as shown by the paper and pencil test.

Effect size analyses conducted at post-test across the maths tablet intervention group and the normal practice control group are reported in

These results suggest that implementing the maths tablet intervention in Standard 2 is optimal for supporting development of both conceptual and specific curriculum-based mathematical abilities. It is important to note, however, that Standard 1 children received 20 h of intervention with the maths software whereas children in Standards 2 and 3 received 40 h. Although the effect sizes in Standard 1 were small (MC) to small-to-medium (MQ), it is possible that larger effects of the maths tablet intervention may be obtained by increasing the time on task in Standard 1.

Using the combined measure of mathematical ability (maximum mean score 49), mean performance for girls and boys in each Standard, at pre-test and post-test, was determined. Results are reported in

A 2 (Time: pre-test, post-test) × 2 (Group: maths tablet, normal practice) × 2 (Gender: girls, boys) mixed ANOVA was conducted to explore if girls responded differently to boys to the maths tablet intervention. Results showed no significant main effect of Gender,

These results suggest that both girls and boys responded similarly to the maths tablet intervention and normal practice over time. As performance gains by children receiving the maths tablet intervention were significantly higher than those exposed to normal classroom practice, this analysis shows that the maths tablet intervention is just as effective for girls as it is for boys. Independent three-way mixed ANOVAs conducted on the separate measures of MCs and CK confirmed there were no significant gender effects in this sample.

This study reports the first RCT to be conducted to evaluate the effectiveness of a new maths tablet intervention currently being trialed in primary schools in Malawi. To differentiate the effectiveness of the maths software from the tablet hardware and small group setting of the Learning Centre built to administer the tablet intervention, this RCT comprised of three treatment arms: the maths tablet intervention delivered in the Learning Centre, a non-maths tablet placebo intervention also delivered in the Learning Centre, and a normal classroom practice control delivered in the typical classroom setting. Ecological validity was ensured through the tablet interventions being administered by class teachers, with technical support being provided by a local VSO volunteer. This represents a sustainable model for scale-up. Baseline measures of mathematical ability were taken prior to the intervention period. These consisted of assessments of conceptual understanding of maths and specific curriculum-based mathematical knowledge. The intervention period lasted 8 weeks, with children receiving the tablet-based interventions for the equivalent of 30 min a day. Children were reassessed on the measures of mathematical ability at the end of the 8-weeks intervention period so that learning gains could be determined.

Results clearly showed that the maths tablet intervention was significantly more effective at improving mathematical attainment than current instructional practice for primary school children in Malawi. First, Standard 2 children receiving the maths tablet intervention made significantly greater gains in performance over time (12.5%) compared to children receiving normal classroom practice (4.4%) on a measure of MCs. (Note: although the non-maths tablet placebo group only made a 5% increase over time on this test, their performance at post-test did not differ significantly from the maths tablet group) As this measure was comprised of items that were not used in any of the interventions, it illustrates genuine advances in mathematical understanding by Standard 2 children after interacting with the maths tablet intervention. Second, whilst all groups made significant improvements in CK over time, children receiving the tablet-based maths intervention showed greater performance gains than the control children, and this was significant for children in Standards 2 and 3. To some extent this might be expected, as children in the maths tablet intervention received some practice with the items included in the maths curriculum test, as these were drawn from the quizzes used in the maths tablet intervention. Thus, they would have been exposed to these items at least once whilst interacting with the software, if they had completed the maths intervention program in full. In addition, this test was administered via tablets, so familiarity with using the devices might have advantaged children receiving the maths tablet intervention over children receiving normal classroom practice. However, the non-maths tablet placebo group was incorporated into this RCT design specifically to guard against effects of enhanced familiarity with tablet use, as well as other key factors that might influence performance (such as small versus large group teaching). Results showed no advantage for the non-maths tablet intervention placebo group over the normal practice control group in maths CK, indicating that the tablet mode of delivery of this test cannot adequately account for the pattern of results found. Furthermore, the maths CK acquired by children in Standard 3 through interaction with the maths tablet intervention generalized to novel items, that were not practiced during the maths tablet intervention, and were delivered in a different, paper and pencil, format. This test of generalization was included to guard against practice effects with specific items in the maths tablet intervention, and thus shows that the higher levels of CK observed by Standard 3 children receiving the maths tablet intervention was unlikely to arise from practice effects with specific items used in the intervention. Despite this, the primary use of tablet technology to assess maths performance at post-test can be considered a shortcoming of the present study. Future studies should include additional tests of generalization to novel items and formats. Measures of application of new mathematical knowledge gained through the tablet intervention to real life contexts could be particularly useful to establish.

Importantly, the enhanced learning gains found by children interacting with the maths tablet intervention can be attributed to the maths software rather than the tablet hardware and small group setting of the Learning Centre, as children who received the critical non-maths tablet placebo intervention did not make significant gains in performance relative to the control children exposed to normal classroom teaching. The non-maths tablet intervention placebo group was incorporated into this RCT design to guard against other factors, such as size and location of class, that differed across the maths tablet intervention group and the normal practice control group, from influencing results. This placebo group also controlled for enhanced familiarity with using tablets, which might have affected performance on the tablet-based assessments at post-test. Importantly, a paper and pencil test assessing generalization of CK administered at post-test, showed significantly enhanced CK on novel, unseen items, by Standard 3 children who had received the maths tablet intervention compared to both normal practice and non-maths tablet controls. This shows that familiarity with tablets cannot account for the enhanced learning gains obtained through the tablet technology by Standard 3 children. Rather, the results illustrate that learning had been embedded within the children’s cognitive systems, allowing them to generalize this knowledge to a new, paper and pencil, context. Thus, the mechanism for supporting the substantial learning gains reported in this study focuses on the engaging, child-centered, curriculum-based, maths software, rather than the tablet technology and small group setting

It is important to note that time on task was equated, as far as possible, across group within each Standard. Thus, the enhanced learning gains found by Standards 2 and 3 children after interacting with the maths software compared to children receiving normal classroom practice are most likely to be attributable to the software itself. This might be due to the immediate feedback and active retrieval features that are built into the onebillion© maths software. As described previously, for each item in the software feedback is given to the child as they interact with the program. Within each item, feedback is given by way of a ‘negative’ sound when an error is made and a ‘positive’ sound when a correct action is performed. Some items require several actions to be performed for completion. Successful completion of an item is marked by the appearance of a big yellow tick and a ‘positive’ sound. This interactive feature of the software is clearly not possible in normal classroom practice where face-to-face delivery is to large groups of children. In addition, progression through the maths software is dependent upon successful completion of each item in set sequence of delivery that is graded in terms of difficulty. As a child must complete an item successfully before they can progress to the next item, this ensures graded learning that is firmly embedded within their cognitive system. Furthermore, a block of items that train a particular concept must be completed prior to progressing to another block of items training a different, more difficult, concept. At the end of a set of nine blocks of items, each block training a different concept, children complete a quiz that assesses their knowledge of the concepts trained over the previous nine blocks. This engenders ‘retrieval-based’ learning which is known to enhance not only encoding of new information but also the application of knowledge (e.g.,

In many developing countries, such as Malawi, delivery of standard instructional practice is dependent on various factors, such as weather (which determines if classes take place indoors or outside) and teacher availability (which depends on whether or not they have been paid, external family matters which can be frequent and significant, etc.). Consequently, daily timetables are highly flexible. This means that when conducting school-based studies in developing countries facing these conditions, it is not possible to quantify the precise time that each child has spent learning different subjects delivered through standard practice. Although attempts were made in this study to equate time-on-task across the different groups, it is not possible to establish how much time each child spent learning maths or any other subject through normal classroom practice. Only in the maths tablet intervention was time-on-task measured objectively as the tablet technology affords precise monitoring of time-on-task to be recorded. This can be seen as an additional benefit of using tablet technology in developing countries, such as Malawi, where is it presently very difficult to assess the amount of instruction a child has received in any given subject. Being able to monitor objectively time-on-task will enable studies to assess the relationship between duration of intervention and individual children’s progression, which will lead to greater efficiency in implementing educational interventions. Furthermore, this study has illustrated that tablet technology can be implemented readily within the standard primary school timetable to support development of basic skills. This gives teachers an opportunity to support individual children’s development, without the need to increase teachers’ time in class. Hence, tablet technology can be used effectively to increase overall school efficiency, thus justifying the costs associated with equipping schools with this technology.

Effect size analysis conducted across ability levels suggested that the maths tablet intervention was most effective at supporting conceptual understanding of mathematics when implemented in Standard 2 (Cohen’s

Finally, a gender analysis showed that the maths tablet intervention was just as effective at supporting the development of early mathematical skills in girls as it is for boys. This is an important finding for countries, such as Malawi, where girls typically fare less well than boys in primary education, and many drop out of school by age 8 years. As outlined in the Introduction, this gender disparity currently persists throughout society in Malawi and hinders economic growth. The results from this study provide robust evidence to support the continuation of girls’ education across primary and secondary school, and further into the higher education sector.

The use of tablet technology, coupled with well-designed, curriculum-based, engaging software, that allows the child to work at their own pace, could help to radically transform scholastic progression and attainment in developing countries, such as Malawi, that are currently facing significant educational challenges. With school days as short as 3 h for Standard 1 children in Malawi, fast learning gains are needed to optimize time spent at school. Tablet-based interventions that can be delivered in 30 min a day, that are shown to give rise to significantly enhanced meaningful learning, over a short period of time (e.g., 8 weeks, as in this study), could revolutionize early years education in developing countries, such as Malawi. However, when scaling up to a national level, careful deployment is required for these technologies to be optimized and deeply embedded within the educational system. For example, teachers need to be confident in using tablet technology within their everyday practice and they need to be aware of the potential that tablet technology can have in supporting learning. This will require investment in teacher training of both trainee teachers within teacher training institutions and existing teachers through continuing professional development courses. Technological support will be necessary to ensure maximal usage and uptake of tablet interventions. In developing countries, this could be implemented through capacity building exercises, such as training community-based volunteers, supported by organizations like VSO. Funds need to be available, not only for the initial investment in hardware but also for its maintenance and upgrading over time to ensure sustainability. Critically, software that is implemented to support the development of scholastic skills needs to be formally evaluated, through scientific studies such as this, in order to determine its effectiveness and potential to impact on educational attainment. Additional research could explore how to optimize tablet interventions, by manipulating the context in which they are used. For example,

The Guest Associate Editor Robert Samuel Savage declares that, despite having collaborated on the Research Topic (Using technology to revolutionize learning: assessment, intervention, evaluation, and historical perspective) with author Nicola J. Pitchford, the review process was handled objectively. The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Thanks go to the following people and organizations that helped to make this study possible: Andrew Ashe (CEO of onebillion©), Richard Howeson (Chairman of EuroTalk©), Jamie Stuart (Chief Programmer), Michal Safier (Programmer), Alexandra Turner (Translator), Dario Gentili (Country Director of VSO Malawi), Frederick Gaghauna (VSO Technical Support), the Malawi Ministry of Education, especially Anjimile Oponyo (then Principal Secretary for Basic and Secondary Education), Raphael Agabu (Director of Inspectorate for Schools), Henry Mphiningo (Primary Education Advisor Area 36), Guy Pickering (from Kumbali Lodge, Lilongwe) for building the Learning Centre at Biwi Primary School, and the Head teacher, class teachers, and children at Biwi Primary School. This study was funded in part by a grant “Digital Tools for Rural Schools” (#M10/032) awarded to EuroTalk© from the Scottish Government.