Is online course beta testing, a thing of the future?

Whilst sipping a cup of coffee and taking 5 minutes from the chaos around, I found myself scrolling through the stories on Edsurge and one happened to catch my eye; online course beta testing.

As I read through the article, at first I was thinking of how I think it’s a fantastic idea. We all strive for a world where online learning material is not confusing and is thoroughly engaging for everyone that takes it. Yet, as the article continues, I couldn’t help but think of all the possible inaccuracies that could come from doing this. Firstly, not everyone can afford to pay people for their review of a course, would this sway the accuracy of the data? Secondly, even if you are being paid, would that also change the accuracy of the data?

The article states,

“We’re still trying to find the magic bullet that motivates people to review other people’s courses when they’re not being paid,” he says. “Even when people have the best intentions, they’re busy.”

However, I don’t think that the issue just lies with a motivation side of things from the respondent. When you collect data using a survey method, you can often slant a bias unless your questions are very well structured. In fact, studies show something called “Wisdom of the Crowd”, whereby, if survey questions are structured in a way that ask you if you think other people would like/use/want/etc. it, you’re much more likely to get a more accurate response.

The Devil’s in the Detail

We could go into great detail around collection of data, and how it’s not just ensuring that you have correctly motivated respondents, but also an unbiased structuring of your line of questioning. We really could. Hours. The fact remains though, when you attempt to get a response from someone, you can’t beat observational studies. By that I’m talking about taking the information from someone directly from their actions. It’s kind of what we do here at emotuit. It’s also what those clever, sneaky companies do, in order to know what to advertise to you online. They could ask you what you like, but you might say supercars and exoctic tropical island holidays. When the reality is, you could be more likely to buy kitty litter and Domino’s pizza.

The article also states:

“It would be lovely if universities would consider ways of adopting the practice of beta testing,” says Phillip Long, chief innovation officer and associate vice provost for learning sciences at the University of Texas at Austin. One factor, though, is cost. “How do you scale that at a university that has thousands of courses being taught,” 

Which is something we could never discourage. If more people took an interest in the efficacy of their online content, they’d see the direct collocation with their student performance. One of the key things in online learning is ensuring that you can keep the attention of your students, and as such keep them learning. With the likes of course beta testing you will get some data and with 2500 samples, you get a decent chunk of “data”. Although, again, the data side of me would state that that isn’t enough to help you understand how YOUR students will respond to that online content.

We know students are all different and respond differently, so then why not take that data from the students that are taking your course in the first place?

Obviously with emotuit, you can. Which is why we built what we have, in the way that we do. To ensure all of the aspects that I’ve talked about here are taken into consideration when looking at the efficacy of content and data sampling of students.