Looking for Proof: Evaluating LG2G

"The big question is – Is there real proof to show that these games can have an impact?" says Shani Reid of the firm Macro International. As one of the evaluators for LG2G, it's part of Shani's job to find an answer to that very big question.

Evaluation looms especially large for LG2G, according to Shani, for two reasons: the grant that funds the project requires that it be evaluated using rigorous science-based research protocols; and the whole initiative really is something of an "experiment." "There's not a lot of research on how educational games work to improve student subject knowledge," Shani says. "So it's a little like we're trying to evaluate a moving target."

So how do you evaluate something for which a set of standards hasn't even been developed yet? Well, you develop some standards. Macro International's first step in the process has been to work with the good folks at CTE to develop a set of criteria by which they'll evaluate the LG2G game. Initially, the team will evaluate the game purely in terms of things like game play and mechanics (since the game can only be successful if kids actually like playing it). Later in the process, Macro will do more testing, in the form of randomized control trials, to measure the impact of the game on students' academic performance. The chance to evaluate the LG2G game from its early design stages, says Shani, demonstrates that "evaluation doesn't just happen at the end of a project," and that "it's equally important at the beginning."

In addition to evaluating the actual LG2G game, Macro is also working with the Advance Team of teachers who are now beginning to incorporate existing "off-the-shelf" games into their classroom lessons. The evaluation team will use data from the teachers' experiences to help determine the kinds of teacher supports that will accompany the LG2G game. And this, says Shani (herself a former teacher), is an essential part of the project. "The thought of learning something new can be overwhelming for teachers," she says. So it's imperative that the teachers who ultimately use LG2G in their classrooms feel both supported and well prepared for that task. And that, in turn, says Shani, will help ensure that the game is used effectively to promote student learning.

Shani says that although she herself isn't a gamer, she sees great potential in the use of games in the classroom. "I don't think it's going to be a fix-all," she says. "But I think games can become another important tool in teachers' tool belts." And while being in on the ground level of something so new certainly presents its challenges, Shani and the other evaluators are feeling pretty good about working on LG2G. "I feel privileged to be a part of it," says Shani. "It feels like we’re setting the foundation for work in this field. To be on the frontline of something like this is very exciting."

Back to LG2G Newsletter