Playtesting: Round One

I'd like to reflect briefly on the playtesting I've done so far and what I plan to do moving forward.

So far, I have been able to playtest with 6 people in person while I have written Levels 1-4. In general, players seem to be getting the learning objectives. At the end of each level, I quizzed players to see what they had learned and everyone was able to give basic answers. Players overall seemed interested and engaged during the levels, but the bias here is that I only conducted this experiment with my friends and family. Most didn't have many suggestions to offer, just polite encouragement. Hopefully future play testers won't feel the need to do this. 

The most useful play tester so far was my partner's roommate. He knew nothing about the Renaissance before playing, so I could see clearly where he got confused and when the game was able to answer his questions without my interjection. He also is a hobbyist filmmaker, so he felt more comfortable giving me constructive criticism because of his experience with similar creative projects. In particular, we were able to critically discuss how to handle the symposium speeches in Level 3 so that they were more accessible to students.

I also have been able to meet with my first reader, Dr. Clulow. He had many good notes. The biggest takeaway was that I need to keep in mind player engagement. Players need the illusion of choice to be ever-present, and they need feedback for their decisions. This feedback can be simple (for example, a slight change in an NPC's expression) because all it has to do is indicate to the player that their choices matter, even though they rarely do. 

Moving forward, I hope to have many more people playtest these levels. Watching people play the game in-person is ideal in many ways. You learn the most about the gameplay experience from watching players.  For example, it allows me to see how engaged students are, how much thought they give to the choices offered, and if they appear to be having fun or seem bored/frustrated, where they get stuck, etc. That being said, during the Covid-19 pandemic, I can only have that experience with the handful of people in my "Covid Bubble." This number changed a bit during the recent snow and power crisis, but for me it is usually only 4-5 people.

It is necessary to test my game out with more people, so the second best option is to have an online alpha test. Many game studios already do this, and at the start of the pandemic last spring, my game professors required it. Here's how it works. At each stage of game development, I will post my playable work to Itch.io. I must write clear instructions on how to play the game so that everyone can play it without in-person support. I must also design playtest surveys for each level that can evaluate how successful the level is based on what I am trying to accomplish.

I designed my playtest surveys to be short enough that students would actually fill them out, but still tell me a handful of important things. To keep them brief, I kept them to only one section. All of the questions are on the same page, and they are all optional. That way, if students want to quit halfway through the form, they might still submit something useful.

I began by evaluating whether or not students had learned the things I intended to teach.

Then, I made sure to include a section to evaluate how enjoyable the experience was. 

Lastly, I gave students a chance to give any feedback I hadn't thought to ask about. Often those responses are unhelpful,  but sometimes players have really good ideas or point out a problem I wouldn't have noticed.

Doing alpha testing online is a great way to standardize my testing and to reach more people during Covid. However, it isn't as rich of an experience as working with playtesters in person. It also can be difficult to get people to play the game on their own time. When you playtest in person, you can often host an event with pizza or some other incentive to get people in the door. During Covid, all I can do is send out an ad for playtesting in my class group chats and ask my friends to forward it. There isn't any consequence if people don't help me, and there isn't really any reward either. 

I'm still thinking about ways to get around this issue. Perhaps I can ask my professors to send out an email to their students? I had some light success with that method last semester when Dr. Mayhew advertised my Russia game. But many students played it and neglected the feedback forms, which ultimately wasn't helpful. 

I'll keep thinking on it.