Class leaderboards cause 114% increase in minutes student spend speaking Spanish each week

“Don't undertake a project unless it is manifestly important and nearly impossible.” - Edwin Land

At Beepboop, we are motivated to re-engineer the classroom so that each additional student enhances the learning of their classmates. We know that as class size grows, additional learners usually detract from student learning. But technology and advancement in the implementation of science-based, empirically-proven methods can, will, and should help solve this issue, leading to improvements in educational outcomes, globally.

How do we ensure that educators can effortlessly use proven teaching methods at scale? How do we make students comfortable being vulnerable and making mistakes in front of peers? How can we make sure students hold each other accountable? Answering these questions require relentless experimentation — exactly what we’re doing.

Class Leaderboards

On January 16, 2023, we beta launched class leaderboards, which is a daily competition where classes compete to see who can create the most supportive learning environment. Each student is emailed the following day after taking a Drill a report detailing what their contribution to their class was and how their class stacked up relative to other classes. The class that is the most supportive each day wins a prize (like free VIP access).

From previous experiments with leaderboards, we’ve learned that the majority of our students STRONGLY disliked being compared to students within their classes and actually found it demotivating. This knowledge helped inform how we structured the class leaderboard experiments.

Also, if you’re wondering how we objectively measured how “supportive” a learning environment was, look at the example Class Leaderboard Email at the bottom of this post.

The preliminary results have been shocking.

In just three weeks of tweaking, the average amount of time active users spent learning in Drills increased 114% from 24 to 51 minutes each week. Users also increased the Celebrations they give to peers 17,160% from 0.1 to 14.9 Celebrations per user per week. A Celebration is a button classmates can press while a student is speaking that animates a little celebration on the speaking student’s screen. Students have also increased the amount of post-class feedback they submit to their instructors by 91%, which gives our instructors more information on how they can improve.

Sometimes experiments fail…

During the last three weeks some experiments have gone horribly wrong and as a result some students have been negatively impacted. We’re sorry for that.

One notable failure was our effort to make our more self-conscious students feel more comfortable getting Beepboop’d (aka making mistakes). To test this, we had our instructors at the beginning of every class say “Beepboops earn you more Quokka Point than correct answers.”

Before we went live with the experiment, we understood that a negative externality of our implementation is that it would incentivize students to fake mistakes. What we didn’t know is if students would actually follow through with faking mistakes given that doing so could be viewed by peers as anti-social behavior and may negatively impact the learning of others in the class.

One day into the experiment, we learned that about 5% of students would aggressively fake mistakes to see how many points it would earn them. We would learn of these instances from upset students and instructors who detected insincerity. So after only one day, we killed the experiment and went back to awarding Beepboops and correct answers with the same amount of points.

Not to worry! We’ll find other ways to encourage students to speak their first words of a new language and embrace making mistakes.

V1 of the Class Leaderboard Email

That’s it for now! If you’d like to support our mission and help us help you learn, become a VIP!

Regards,

Devon Saliga
CEO | Beepboop