AR helping to improve literacy skills
The Arete H2020 Project is exploring how the introduction of Augmented Reality (AR) technology could help primary school students (9 to 12 years old) with reading and spelling difficulties and improve their literacy skills. Many of these students struggle through the education system underachieving, and only a very small minority have access to effective, evidence-based educational programmes that align explicitly with their particular learning profile.
Pilot 1 app
WordsWorthLearning© is an evidence-based online remedial programme for primary and secondary school students that are having difficulty learning to read and spell. The Arete Project conversion of the programme to a native App (iOS and Android) that contains Augmented Reality technology coupled with real-time Voice Recognition and Markerless technology, will hopefully provide even more potential to increase the prospect for schools and parents to help those students to reach their academic potential.
The following information provides a brief outline of the approach taken for the design, development and build of the "READ SPELL" Android and iOS Apps for improving English language reading and spelling skills.
Content - UI/UX & AR
WordsWorthLearning© is an evidence-based online remedial programme for primary and secondary school students that are having difficulty learning to read and spell. The Arete Project conversion of the programme to a native App (iOS and Android) that contains Augmented Reality technology coupled with real-time Voice Recognition and Markerless technology, will hopefully provide even more potential to increase the prospect for schools and parents to help those students to reach their academic potential.
The following information provides a brief outline of the approach taken for the design, development and build of the "READ SPELL" Android and iOS Apps for improving English language reading and spelling skills.
AR Infrastructure
The most preferred SDK development tools were Unity and Vuforia (Fan et al 2020). As an SME, we had to consider the EU funded project budget and the availability of the technical skillsets required to design and develop the native Apps and decided to use Unity AR Foundation for the App development which offered a cross platform feature for Unity Editor and ARCore and ARKit to introduce “State of the Art” native AR Apps for mobile devices (Specifically Tablets).
Markerless technology
Apple (ARKit) and Google (ARCore) continue to update features and support for their A developer platforms and devices with advances such as Simultaneous Localization and Mapping (SLAM) that allowed mobile apps to map an environment and attach AR objects in a real-world environment and track movements within it. Markerless AR is a concept growing fast in the EdTech sector and this provided an opportunity to move away from the 'marker' based technology that we had introduced in our Web-AR version. We decided to introduce "Markerless" AR for the Apps which include colour coded audio-visual AR 3D animated objects that provide ‘memory hooks’ to embed essential reading & spelling sounds and rules in the students' long-term memory.
Child-voice recognition (API)
Most voice recognition products are designed for the "adult" voice, and considering the age of the stakeholders and their literacy difficulties, a decision was taken to introduce a "voice recognition" API (Soapbox Labs), that is specifically designed for the "Child" voice. It provides reading exercises in "real-time" and the voice recognition software gives them immediate onscreen feedback - producing performance data that is logged to the Arete Project - Learning Locker. It measures reading accuracy at a phonemic level e.g. pronunciation, fluency (rate) and comprehension to indicate student literacy development level. Providing data from a "phonemic" baseline offers future research potential, as the data could be used across 'multi disciplines' to improve teaching methods for children with reading and spelling difficulties.
xAPI - Research data
The xAPI research data will track various timestamps and provide "Actor - Verb - Object - Result" data to measure students' actions and results for lessons, games, reading & spelling exercises, quizzes, reflecting their progress through the App. The data is transferred to the Arete Project - Learning Locker to provide (i) real-time data for the teacher to monitor and assess a child's progress throughout the App and (ii) provide metadata for further project research going forward, that may be pertinent in "multidiscipline" and "neuro-diversity" research projects involving literacy improvement.
