Opinions are like assholes: everybody has one. But which ones (opinions) are valuable? We have a very vibrant indie community in the Netherlands and a lot of wonderful creative people tested my game before its release. Some opinions were crucial to the improvement of the quality of Should Shoot. Some obviously were not… It can be very hard for a designer to decide which ideas and suggestions to take with you and which ones you should disregard. This story should help you with dilemmas of taking or disregarding feedback.
One of the first things I learned as a designer is to always listen to feedback. Always listening is not the same taking everything into account. Developing your own game can often feel like raising a child. Everybody want your child to grow up to become a wonderful human being, as a parent you should listen to everybody’s feedback, right? Not really. As a designer, you must have a clear idea of the value of your game and the kind of experience you want to create in order to evaluate the value of the feedback on your game.
|Whether we notice it or not: this seems to be our default attitude.|
There’s a natural flow to my testing processes when designing a game. I’m a teacher of Usability/Playability classes at Utrecht University of Arts (HKU) and my lessons mostly focus on play testing processes for teams of 5 or more. My play-testing processes in an early phase of a project is very different from the more formal approach (also taught in my lessons) at a later stage. This developer story I want to share some best practices for the earliest forms of play-testing. These practices could also help you while conducting a play-test in a later stage of development.
|Ok this is probably the last lame image in this developer story.|
My first experience of showcasing Should Shoot was early in development in its Tap Tap Shoot phase. It was during a network lunch hosted by the Dutch Game Garden. The Dutch Game Garden is an organisation that is houses multiple Dutch Indies at various locations. Their also host to the network lunch: a gathering of IT professionals where they get to present their projects or just have brief conversations with one another. More then 30 people played Tap Tap Shoot that day. Most players enjoyed the game a lot. The best comment was: “This game is great, it’s simple yet brilliant. I’m surprised there are no existing similar games.”. But quotes rarely give a proper impression of the average experience(s) of playing a game. I took notes and made a list of problems I identified during play-sessions. Because I tested early in the development process and my design goal was a bit unclear, I got a bit confused by all the feedback. I took a step back and used my experience as a play-test monitor in the AAA scene to formulate some take-aways to help me get valuable data and determine what to do with it.
As the designer: You’re the worst observer in the world.
As a designer, you’re probably the worst observer in the world. There is a big chance you’ll only monitor how testers respond to your intended design. For instance, if you’ve placed an ammo box near a boss encounter and the player walks right past it whilst you planned for players to pick it up you might just note ‘player walks past ammo box’. While someone else might note ‘Player walks straight to enemy boss'. There’s no necessarily something wrong about either observation, it’s just that feedback by someone other then the designer might shine a light on aspects of a design that the designer him/her self would not have noticed. If you’re a lonely indie like me it might help to co-operate on play-tests or to reflect with someone (could be another designer) on video footage from the test later on.
Designers are the worst (play) testers in the world
Designers contributed greatly to the development of Should Shoot. So I wouldn't say never to test with designers but be aware of the fact that their experts at evaluating an experience. They won't think and respond to your design. They'll try and think like 'an average user' and analyze the events, look and feel in your game from the persona they created in their head. Sometimes this can help you spot errors in play-tests with players sooner. Sometimes it can be total crap and another designer-test-participant will create a problem where there really is none. And because most designers are super analytical certain features won't have any impact at all. Instead of having the experience of using a heavy impact weapon they might feek like they're making your camera shake (and critique the way it shakes too!).
|User perception from 'Interaction Desig: Beyond Human Computer Interaction' by Priece (2002)|
Note observations, not conclusions
This is very important! Try and avoid drawing conclusions when observing your test-participant’s behaviour. For example: when a test participant is playing and lets out a ‘sigh' it could mean he/she is bored. But it could also be a false conclusion. If the timing is right: try and ask what’s going on. If the test participant describes events in the game that would fit a bored state then you can always ask for the test-participant to describe their own condition. Only when they say they’re bored you can write down something like ‘test-participant says he/she’s bored'.
Be aware of your tester's likes, dislikes and background
Everyone has certain tastes and preferences, likes or dislikes. And even when we think we’re aware of that while giving feedback, it often still shines through. Someone who might not be a fan of action games might start giving suggestions that help to slow the game down or they might hint at mechanics that influence the weight of strategical thinking in order to nudge your game towards something they like. This is ok. And it can be quite insightful and even useful! But it can also be really confusing. I set up a google form and asked people to fill in a questionnaire when they’re done to get a profile of my test-participants. This led to me ignoring some data or it helped me to interpret some off comments or remarks. Try and stick to your design goals and evaluate whether feedback helps you achieve those goals or not.
Be aware of your social environment
This might not be important to everyone but it’s still valuable to be aware of your social environment. Like I said, I am a teacher and sometimes my test-participants were either alumni or students. This kind of relationship influences the way you and your test-participants communicate with each other. For example, in my case, some students commented on every single asset in my game hoping to impress with their skills as a designer. Some test-participants can also be really impressed with you as a game-designer. If possible, distance yourself from your product. Sometimes it’s even best to lie and say ‘I didn’t make this game.’ in order to enable your test-participant to feel free to talk.
There’s some great software out there that’s very useful for play-testing alone. As a Mac user, I’m really fond of Silverback. If you want to get more hardcore this might be an interesting read. The good thing about Silverback is it’s performance. For my iPhone/iPad games however, these tools were no good. I had to resort to a small go Pro aimed at the faces and a direct capture. Capturing the screen was a horrible tragedy as I was not able to use the Lightning to HDMI dongle by Apple because of HDCP on the HDMI.
One particular issue that came back a lot in the beginning is the presentation of score in the game. At first, the game only showed the player’s score. Players tended to tap to continue playing the next round real fast and they missed their' or their opponents score. I considered putting a back button at the end screen but it seemed weird to add another button for a player to go back to something he/she wanted to see earlier.
Next thing I tried to implement was both player’s scores. Another big design decision: where to put the player’s score?
- Scenario 1: Always put the highest number first, second highest number second. Both numbers can belong to either players. I would need a clear visual representation of each player. A number could be confusing. Picking an avatar or color would mean a whole new menu.
- Scenario 2: Always put player 1’s score first, player 2’s score second. This is what I went for the first time. I thought consistency would really help player’s identify which score belonged to who.
- Scenario 3: Always put the player first, the show the score of the other player. This is the solution I went for. It worked really well. Every play-tester I asked for their score responded with the correct answer. Consider this a tip: when building a game where players can check their own score, always show the ‘owner’s' score first, and the other player’s score second.
Asking for feedback is always a good thing. Listening to every voice isn't. It can make you confused and there's a chance you'll be making someone else's game instead of your own. Formulate your design goals and make sure you know about the preferences, likes and dislikes of the person that is providing you with feedback. Take all points mentioned above into consideration when asking for feedback or organizing an actual play-test. If you'll do this and quitely reflect on all the data I'm sure you'll be able to improve the quality of your project.
For more tips on play-testing, check out this awesome video by extra credits.