News & Events

The Development of Behavioral Expectation Lessons for Positive Behavior Interventions and Support (PBIS) Supported by Augmented Reality for ARETE

Introduction  

In the ARETE consortium we strive to improve education for youth around the world by incorporating novel technological developments into existing educational curricula. We specifically seek to develop Augmented Reality (AR) content and applications to enhance students’ (and teachers’) educational experience. If you’re unfamiliar with the work of the ARETE consortium, please check out this blog and/or visit our website.

In the blog you’re reading right now we’re going to zoom in to the third pilot study within the ARETE project. In this pilot, the social behavioral education of early adolescents is addressed. More specifically, we seek to develop AR content for an application that can be used within the Positive Behavior Interventions and Support (PBIS) framework. To shortly explain, PBIS is a goal-directed and schoolwide approach aimed at increasing desired behavior in students and at decreasing problem behavior, thereby creating a positive social learning environment that facilitates learning and prevents behavioral problems. As such, PBIS does not only support students’ social behavioral learning, but also the development of their executive functioning such as their behavioral and emotional regulation and their inhibitory control. If you want to know and/or learn more about PBIS, please check out this earlier ARETE blog and/or visit the official PBIS website. As you can probably imagine, a lot of developmental steps were taken - and still are to be taken - to realize our so-called PBIS-AR application and curriculum. In this blog we would like to address the steps the ARETE consortium has undertaken until now in the development of the PBIS-AR application and curriculum as well as to hint at some of the exciting future steps still to come.

Research & development

Schools that use PBIS normally start by defining a small set of school values to promote a safe and healthy school environment that allows their students to flourish. Schools are free to choose the school values that they deem the most important. Examples of such school values are respect, responsibility, and safety, but other values (e.g., love [each other]) are also sometimes used. After the school values have been established, the next step for a PBIS school is to develop a set of behavioral expectations—or behavioral rules—for students (and school staff!) to adhere to in all school settings based on these school values. That is, the school values via the behavioral expectations guide students’ behavior and actions not only in the classroom, but also in for example the hallway, when they go to the restroom, and/or when they are playing on the playground. These behavioral expectations are summarized in a so-called Behavioral Expectation Matrix. Special lesson plans are subsequently developed to help students learn the expected behavior at school in a positive way following guidelines that are backed-up by both theory and practice.

As all PBIS schools are free to determine their own school values and their own behavioral expectations, there was no pre-existing and universal behavioral expectation matrix that we could use in the development of our PBIS-AR application and curriculum. Therefore, an important first step in our research & development process was to create a universal—or as we like to call it the ARETE—behavioral expectation matrix. Our original intention was to execute a set of focus groups with stakeholders (i.e., PBIS teachers) to ask them about what they thought would be the most important content in terms of school values and behavioral expectations, so that we could include these in the PBIS-AR application and curriculum. Unfortunately, the COVID-19 pandemic was not bothered by our intentions and made it rather impossible to run a sufficient set of focus groups to gather the information we so desired. Therefore, the Dutch team members of the ARETE Consortium came up with the contingency plan to review the literature on PBIS behavioral expectation matrices and to collect behavioral expectation matrices of a large set of schools in the Netherlands, the United Kingdom, the United States, and in Australia (due to language restrictions other countries were not included in the literature review). This process resulted in a (rather) longlist matrix of behavioral expectations for all school settings that were based on the most common school values of respect, responsibility, and safety.

This longlist was however—as the name implies—long, in fact it was too long to be useful for the purpose of creating AR content for the PBIS-AR application and curriculum. Following initial deduplication and reduction efforts by the Dutch team members of the ARETE Consortium, a shortlist matrix of behavioral expectations remained. In this shortlist—conforming to PBIS guidelines—a maximum of five positively stated behavioral expectations per school value and per school setting remained. However, this shortlist—unlike the name implies—was still too long to allow us to realize our intended final set of behavioral expectations for which AR content and 12 behavioural lesson plans could be developed. Again, we intended to execute a set of focus groups with stakeholders (i.e., PBIS-teachers) to ask them about the most important 12 behavioral expectations of the ARETE behavioral expectation matrix. However, with the COVID-19 pandemic still unbothered by our intentions, we again needed to activate a contingency plan to come to the final set of behavioral expectations.

In the first half of 2021, the Dutch team members of ARETE therefore planned to execute an online survey in 10 European countries to see whether we could narrow down the shortlist matrix of behavioral expectations to a manageable set of behavioral expectations. However, the survey was also used to double-check whether the school values of respect, responsibility, and safety were indeed the most common school values in European schools, and whether we might have missed important behavioral expectations in our research & development process.Ultimately, the survey could only be executed in four European countries (i.e., the Netherlands, Italy, Portugal, and Lithuania) as the teachers in most countries were still suffering the consequences of mandatory school closure caused by the COVID-19 pandemic (e.g., teaching was prioritized over participating in research, overflowing workload for teachers). Nevertheless, the data of several hundreds of students and teachers provided us valuable information for the development of the PBIS-AR lesson plan curriculum. First, the school values chosen for our universal ARETE Behavioral Expectation Matrix, that is, respect, responsibility, and safety, were highly rated by all respondents and other school values mentioned by participants could be considered to fall under the same umbrella terms. More importantly, the ratings of students and teachers converged on a small set of 15 behavioral expectations for the universal ARETE behavioral Expectation Matrix. The final set of behavioral expectations can be found in Table 1 presented below.

Table 1: the final set of behavioural expectations of the ARETE Behavioural Expectation Matrix ranked in order of importance (i. e. averaged over student and teacher ratings 

Current and future steps

Although slightly delayed in our research & development process, we were now ready for the next phase in the development of the PBIS-AR application and curriculum. That is, we were ready to start creating the AR content for the behavioral expectations and to develop lesson plans in which the PBIS-AR applications could be applied. This is also the current phase of pilot 3 in the ARETE project. At present two things are being done simultaneously. First of all, the Italian team members of ARETE are working on creating the actual AR content for the PBIS-AR application. You can read more about this here. At the same time—and as the COVID-19 pandemic started to get impressed with our intentions—the Dutch team members of ARETEstarted to work together with stakeholders (i.e., PBIS-teachers) in June of 2021 to develop PBIS-proof lessons that will constitute the core of the PBIS-AR curriculum. In this developmental phase, small teams of PBIS-teachers together with PBIS-experts and the Dutch team members of the ARETE consortium work on systematically developing and evaluating a total of 12 behavioral lessons based on the final set of 15 behavioral expectations in the universal ARETE Behavioral Expectation Matrix. The lesson plans that are being developed will be designed in such a way that the lessons can be tailored to the specific needs of students in a particular classroom yet following a universal plan. The final set of behavioral expectations (chosen by the teacher design teams) for which the 12 behavioral lesson plans are being developed as part of the ARETE PBIS-AR curriculum—including the planned order of presentation within the curriculum—can be found in Table 2 presented below. In a future blog we will present more detailed information about this lesson design study.   

 Table 2: The ARETE PBIS-AR curriculum of lesson plans including the order of presentation within the curriculum

The ARETE-PBIS lesson plans that are currently being developed with stakeholders will be finalised in Fall of 2021 ans subsequently pilot tested starting after the Fall school break of 2021 in a set of 10 Dutch 5^th and 6^th grade classrooms (i. e., students aged 9-12) of five Dutch primary schools. During this time, the lessons will be mainly tested on pedagogical content and - as the AR-content is still being developed- will be tested without the enhancement with AR-animations. What we intend to figure out in the upcoming months is: Are the developed behavioural lesson plans reaching their intended goals? With the knowledge gained during this phase of ARETE pilot 3, and in combination with a beta version of the PBIS-AR application, we will finetune the ARETE behavioral expectation lessons for PBIS and incorporate the AR component to the lessons in early 2022. The final PBIS-AR application and curriculum will subsequently be pilot tested starting in September through December of 2022 in another set of 10 Dutch 5^th and 6^th grade classrooms of five Dutch primary schools. Moreover, the effectiveness of the ARETE PBIS-AR application and curriculum on the behavioral, emotional, cognitive development of participating students will be compared to those of a matched control group of 10 Dutch 5^th and 6^th grade classrooms of five Dutch primary schools.

Exciting times are thus ahead of us in the coming years! So, keep your eyes out for future posts about the ARETE PBIS-AR application and curriculum. Also, be sure to check out the official website of the ARETE consortium frequently and to subscribe to our social media channels (Twitter, LinkedIn, Facebook, YouTube), so you can rest assured that you will always stay up to date about all exciting technological and educational innovations the ARETE consortium is working on!

Jeroen Pronk, Wilma Jongejan, & Sui Lin Goei

Vrije Universiteit Amsterdam, the Netherlands