I have been working on a research project at EA this semester. The goal of this project has been to see if we can predict what types of games people will like based on their in-game behavior. Yes, the goal was that general. As the designer on the project it was pretty much up to me to come up with a plan of action as to how we might accomplish this. I thought a lot about the way people play together, and looked into everything from the work of Chris Bell to this huge Gamasutra article on player types. I’m not going to go into player types too much as you can read the article, which reviews them extensively, if you are super interested. However, what I am going to touch on is our process, the models we choose, and how we have been using them thus far.
Many people may be familiar with Bartle types. These were originally adapted for analyzing players of MUDs, but have since been adapted for multiple uses. GamerDNA.com uses a pretty good Bartle Test to analyze what type of player you are based on your behavior in MMORPGS. But, basically Bartle breaks down players into four categories (sometimes eight, but for our most commonly and for our purposes four): Killers, Achievers, Socializers, and Explorers. But, Bartle doesn’t apply to everything and we needed something a little more complex.
If you know Richard Bartle, you may also be familiar with Nicole Lazzaro. She breaks players down by why they play. Again, she puts players into four categories by the type of fun they are pursuing: Serious, Hard (Fiero), People, and Easy.
These two models of types correlate with each other very well. For example, an achiever who prefers to pursue leveling, achievements, equipment, and customization; these players are are drawn to fiero. I mean by this statement that they gain pleasure from overcoming difficult situations, like puzzles, acting on difficult problem in the world (whether these are combat challenges or perhaps building challenges) and will gain a sense of satisfaction from the achievements of a mentee or partner in game. Below is a map of how these groups correlate.
So we have developed a structured way of observing players. First, we created a Dead Space 3 level which serves as a microcosm for the whole game. What do I mean by that? Well, we only have about an hour with each player to see if we can predict their player type, and see if our prediction about their player type will accurately allow us to predict whether they will like another game. In this case we are trying to predict if they will like Army of Two: The 40th Day. So we get 40 minutes with Dead Space and 15 minutes with Army of Two. In order to observe as much as possible in 40 minutes we needed to have a lot of the normal game’s possible activities shoved into a shorter experience. For example, we needed to have more opportunities for customization, exploration, frustrating combat, and puzzles. This way, we could see what a player would do quickly. We are trying to automate this process, so in the future a system will have the whole game over which to collect data, but again, we only have 40 minutes.
At the end of the test, after we have made our prediction about what their type is, and whether they will like Army of Two, we give them a Bartle test to get a boolean on whether we are hitting close to the mark with our predictions. We also ask them some questions, amongst these is whether they like Army of Two or not.
First off, we have determined there are a few things we thought would be good predictors that aren’t. For example: time spent in EVA (floating around in outer space) is not a good determiner of whether someone is an Explorer since some people can’t orient themselves well in a zero-g environment. Also, the way people play is highly influenced by player types they are with. Explorers tend to have a negative experience when playing with Killers, as Killers will try to drive the experience quickly through a level. Socializers on the other hand will go with the flow and can interact well with either of those types (usually, we did have a gross exception).
On the whole we have found that our process gives us good results. We can predict a player’s type over 65% of the time so that it correlates with the first Bartel type returned on a Bartle test administered at the end of the testing session. Also, we had hypothesized that Achiever and Socializer player types would like Army of Two the most and this has proven to be true for the most part. We can predict whether a player will like Army of two with about 68% accuracy going by this hard and fast rule. Socializers almost always like Army of Two once they play it, because they are attracted to the co-op aspects, and especially like the camaraderie system. Achievers on the other hand like Army of Two only about half of the time. This may be because they sometimes do not have enough time to get into the customization systems available to them, and don’t play enough to experience any fiero. Trust me, having played the game, there are definitely opportunities to experience fiero. However, our predictions about who will not like the game are spot on. We have only had one explorer who liked the game at all, and they said, “It was okay, I might play it if my friend had it.” Our killers like it less than 20% of the time, this is probably because they can’t complete the game all by themselves or force the experience to move forward without their partner.
We have a lot more testing to do, and it remains to be seen whether we can get our automated system working to make these predictions. But, if nothing else results from this project, I am happy that we came up with a way to predict this much. I feel that the way we predict player types in Dead Space 3 can easily be transferred to a wide variety of games from platformers to open world RPGs.
More to come as we wrap up this project.
Update Post Project May 2013:
I just wanted to say that we were able to reassess our criteria (looking for Socializers, Achiever Socializers, and Killer Socializers specifically) for predicting who would like certain games, specifically Army of Two based on the results of our testing. We found that we can now predict with 75% accuracy whether or not a player will like AO2 based on how they played a 40 minute session of our DS3 testing level. Most of the numbers I have given in this blog are rounded off, that one is not. Amazingly when we crunched the numbers it actually came out to a flat 75% (where as before our accuracy rate was actually 68.4% ). I will link our postmortem documentation when I get back to San Francisco.