Writing Better Practical Tests For The Games Industry

making a great test for the games industry

This is a repost of my article from the Tiny Hydra blog posted May, 31st 2021.

I have been doing design and development tests for over a decade now (I even did some art tests early in my career) and I have come to the conclusion that most tests are bad. The idea of testing people isn’t bad, in fact, it’s imperative in the games industry. Truth is, most tests fail to accomplish what they set out to do. Which is: to find out whether a potential recruit has the skills required for the role they applied for. Out of the dozens of tests I have completed in my career, I can count on one hand the tests that were well crafted. I’ve written this article in the hope that it will help you (the test giver) to write better tests for almost any discipline. 

When I refer to tests I do not mean multiple-choice exams that you might expect from school, or even the short evaluations with minutes long exercises often given as a preliminary evaluation to see if people have a baseline understanding of a knowledge base. The types of tests that might be used to see if people actually know how to code in a language they claim on their resume, or understand universal concepts of UX design. When I talk about tests in this article, I am referring to practical exercises given to candidates to evaluate their broader skill set. These tests often require candidates to plan or build something based on an existing project, or even create a wholly original experience.

What do I mean when I talk about a ‘well crafted’ test? Oh, I’m so glad you asked. By well crafted I mean that the tests were well scoped, the turn around requirements were respectful, the deliverables were well defined, and the test had been well vetted.

So what does it take to make a good design test? Typically it’s a fairly straightforward exercise, related to the work the potential hire will be doing, which is primarily geared toward testing the way a person approaches a problem, communicates solutions, and which gives you an opportunity to assess how a person handles feedback. You will typically also want to validate a person’s ability to use a specific tool or language. In this article I’ll expose some common issues with test exercises and the kinds of instructions you tend to see out there, and discuss what we can do better as an industry. Also, I’ll walk you through what I think makes a good test for a professional in the games industry.

Why Should You Listen To Me?

The common question when reading something like this is, “well what does he know?” In the last ten years I have been in 3 separate management roles on game development projects where I was responsible for writing tests (and one on an IoT project). I was also a professor of game design and development, and during my tenure in that gig I wrote a ton of tests and created other forms of evaluations. I have made ALL the mistakes I talk about in this blog, and have outlined some things we do differently here at Tiny Hydra. I’ll admit, providing a good candidate testing experience does take more time and effort. But, I think you’ll find that it’s really worth it. It leaves the candidate feeling valued (whether they passed the test or not), gives you meaningful insights into your own process, and helps build better connections even with the people you don’t hire.

When Should The Test Be Due?

This is actually a really hard question to answer. You want to be fair and make sure you give people equal time to do the test, right? You want them to have ample time to turn the test around, and you want to be sure you are testing enough.

The problem is that each case is different, and you don’t know what a person’s schedule is like. If you tell your applicant, “Here is the test, please return it by this date and within so many hours after downloading the files,” you’re failing to take their schedule into account. Why? 

Let me walk you through a scenario. You’re working full-time but for whatever reason you’ve been applying around and have a couple jobs you’ve been interviewing for. Because this is a highly skilled industry, every position has a test. With: fielding tests, interviewing, and doing your full time job – you already have a lot on your plate. Another company you are very interested in gets back to you several weeks after you applied. They request you sign an NDA so they can give you a technical evaluation. ‘Great,’ you think to yourself. You sign the NDA on a Thursday and they send you the test at the end of that day. They don’t interview you first (a problem we will talk about later) and when you open up the test it has the following requirements:

  1. Complete a task that will take a full day of work +/- 2 to 4 hours
  2. Complete the task within 24 hours of starting it.
  3. You need to return it by Wednesday the following week.

No big deal right. I mean you were just already spending your nights and weekends working on tests, you can slot in one more… Right? Given your busy schedule, what would you do in this case?

This example demonstrates how important it is to discuss the deadline with your applicant. Telling someone when a test is due without a discussion disregards and disrespects their other responsibilities and timelines. The thing is: you don’t know other people’s schedules.

How Long Should The Test’s Exercise Take?

Most of the time tests are written by people working within that discipline on the team. But, because they are written by people who already know the everyday way of things, I would say that almost all of them are poorly scoped. They don’t plan time for discovery and they rarely allow for learning the intricacies of someone else’s project or technology. Also, we tend to forget how much value comes from working with a team in contrast to working alone. Consider a situation where you are unsure about something you are doing, you can turn to a peer and ask, “Hey, how would you do this?” But, someone working alone has to do extensive research, and when you are working in an innovative field like games maybe there is more than one way to do something and none of the common practices are super well documented.

And then there’s tests with ridiculous time requirements. It is never OK to expect an applicant to spend 30 to 40 hours of (free labour, ahem) working on a test. Firstly, let’s go back to the previous example where you are interviewing with multiple companies, and all the positions are pretty exciting to you at this stage. How would a 30-40 hour test fit? What if you’re still at another full time job? If you believe that a job at your company is well worth the 30-40 hour test investment (next to the time dedicated to applying, interviewing, and repeating those steps at X number of other companies), well, have I got news for you: your company is not the center of another person’s universe. 
A good rule of thumb is to choose a small task that will allow a person to showcase their competency. You want to give them a task that should take a MAXIMUM of 8 hours. If you can’t tell from 8 hours worth of work whether someone is competent then there is probably something wrong with your task. My suggestion is to figure out what you (or a proficient member of your team) could do in a day’s worth of work. Then, cut the task in half to account for complexity. If this sounds unreasonable to you, ask yourself, “Why don’t programmers actually program 40 hours a week?” or, “Why can’t concept artists spend all their time drawing?”

How Do You Define Test Deliverables?

One of the biggest problems I see with tests today is a lack of clarity in the deliverables. For example, an applicant might receive the following task: 

Create a “research plan” and a presentation of recommendations to the team based on your preliminary findings from playing through the first time user experience. 

But what counts as a research plan and a presentation? How much information is enough? One applicant might put in over 20 hours of effort to create an extensive research plan that accounts for conducting the research over the course of production for the game, and a 15 minute presentation aimed at peers breaking down all the findings from the preliminary research and going through their recommended design changes based on those findings. Another applicant might put in 6 hours of work and come back with a simple outline for a research plan and a 5 minute presentation with a plan of attack suitable for an executive level audience. Both would fulfill the requirement but demand a different tax on time, and level of involvement. In the first case, an interviewer might even say the applicant has done too much. But how is an applicant supposed to know from the instructions that were given? 

Be very clear about what you want from a deliverable. If you are asking someone to build a level for a game, give them a max and minimum size, and how long it should take a player to complete the experience. If you are asking for a presentation, clearly define how long it should be and who the audience will be. Also, be very specific about what you are looking for. Where should a person focus their efforts?

Here’s an example of a poor deliverable description:

Using simple mechanics, produce a pitch document for a minigame.

  • Maximum one page of text.
  • Theme the game to a sci fi setting
  • Include a short visual pitch on a separate page.
  • Include a list of required assets to build the game on a separate page.
  • Explain your thought-process and reasoning on a separate ‘designer notes’ page.

Let’s set aside the fact that this could easily take several days worth of work to do well.  What else is wrong with the deliverables description:

  1. What are they looking for?
    • Minigames could be interpreted in many ways. Checkout this video on the 7 Best Minigames That Deserve Their Own Game. Are these good examples of the type of thing the test is asking for? Arguably most of these have very “simple mechanics”. But, maybe the writer of the test is looking for something much more narrow in scope like the Mass Effect 2 hacking minigame. Also, is this minigame part of a 3D Action RPG? Or maybe a 2D Visual Novel? How often will the player see the minigame? Do you need to be able to produce a bunch of different versions of it, or will this be a one off experience? This task lacks specificity and thus is way too open to interpretation.
  2. What is the scope of the visual pitch?
    • Who is the audience for this pitch? Is it the design team? Maybe it’s a product manager? Should this be a slide deck or a one pager? A tester might read this and decide they know exactly what they need to do, then find that the test’s authors expected something completely different.
  3. The list of requirements itself is a huge task.
    • The scope of this task is directly related to how the tester interprets the definition of minigame. If the author asks the tester to create a minigame for an existing game, then at least they have a reference they can use to figure out what resources are already available to them. But, designing in a vacuum with no frame of reference the tester might do a great job on their due diligence for a fairly ambitious design that wasn’t even close to what the authors were looking for.

There are some good things about the above deliverables description. It tells the designer that they have to be succinct enough to fit their design documentation into a single page of text. It also solicits designer notes and asks again for brevity. 

Here is one way I would retool this description, keeping the same scope:

Using simple mechanics, produce a pitch document for a 2D minigame, using puzzle mechanics, for our game <NAME OF RPG GAME>. We are looking for an experience that takes less than 5 minutes on average to complete, requires minimal instructions to learn, and uses a non-diegetic UI. We want to be able to present this minigame multiple times within the larger game’s experience and still have it feel fairly fresh each time. Good references include the hacking minigames from Mass Effect 2, or Prey. 

  • Please keep your written description of the minigame to one page or less.
  • Include a visual pitch in the form of a slide deck. This pitch will be meant to get buy-in from department heads and should allow them to quickly understand the concept from a high level. Your pitch should take about 5 minutes, please be prepared to present it to the team during your test review.
  • Include a list of required assets (above and beyond those that already exist in the game) to build the game on a separate page. If you are able to repurpose existing game assets please give a brief explanation of how you intend to do this.
  • Explain your thought-process and reasoning on a separate ‘designer notes’ page.

Still this isn’t perfect and some people will interpret these instructions in different ways. This brings me to my next point…

Should You Encourage Questions About The Test?

Even the most experienced people write tests poorly. As a professor I revised every test I gave every semester based on feedback from my students. Still, every year I had students come to me asking for specifics about one exercise or another. Honestly, I don’t believe that you can give a perfectly clear test. One of the problems is personal interpretation.

It’s a mistake to think that you will get the best applicant by giving them a test and having them perform in isolation. In reality, the better applicant might be the one who asks questions and looks for opportunities to collaborate, and to investigate. Often I find companies putting up a wall between those evaluating the test and those taking the test. Many companies don’t encourage applicants to ask questions, and when they do there is often an involved process where the test taker asks the HR representative, who asks the team, and sometimes the answer applicants get back isn’t to the question they originally asked.

My suggestion for a great candidate experience is to encourage candidates to ask questions. If you can afford it, setting up a quick meeting to  onboard someone to the test exercise you are giving them is a great way to do this. That way the test taker has an opportunity to ask and follow up on questions, and you find out whether your applicant has a collaborative mindset. You also get something out of this – you get feedback on your test.

If you don’t make it clear that questions are welcome, many applicants will assume that their ability to interpret the questions is part of the test. This might well be the case, but is usually a bad measure of someone’s critical thinking ability. The flaw inherent in this way of thinking, again, is the assumption that you have written the instructions well. 

A test is just like a product. It needs iteration if you want to achieve a really good result. The first exercise you put together is very unlikely to be perfect. The level design test you made last week should probably be very different from the one you’re giving three years from now. You can get some meaningful feedback from looking at the numbers and statistics on how many people did well versus who failed, and whether the people who tested well later performed as expected on the job. But, you can also supplement this. Do those on-boarding meetings, and ask test takers to fill out a short post-test survey about their experience.

Is A Project You Work On Regularly A Good Testing Resource?

We talk about this all the time when we are running tests on the games we make. The worst testers are those on the development team. They have too many expectations, and know too much about the project. Consider the story from James Portnow’s Extra Credits where he talks about a consulting job where he was asked to try a game his client was developing. He was completely unable to figure out how to open the player character’s inventory. Eventually, he was told he had to triple-click the player character. Everyone there was so used to it that they had completely lost touch with the fact that it was not intuitive to anyone else. This is an example of excess familiarity being a problem. 

Bottom line, you are too close to the work and have too many expectations. In recruiting there is this idea of culture-add rather than culture-fit. The principle is that you should be trying to find people who will add to your team rather than be a reflection of who you already have. The same principle should apply to the work. But this takes a lot of effort. It is hard to evaluate someone’s ideas on the basis of whether they are good even when they aren’t what you would have come up with given the same assignment. To help achieve that objective it is a good idea to remove some of your partiality.

The primary reason we test with projects we already have, is that it’s easy. We look for people who can come up with the same, or at least similar solutions. Someone who provides them at a high quality. But there’s an inherent flaw there. We forget that we spent so much time in pre-production refining our ideas, and then tested and iterated on them until they were good. We can’t expect someone starting fresh with a project to have the same frame of mind and the knowledge we have. However, when you give someone a test derived from a project you’ve been working on for several years, no matter how impartial you try to be about the results the evaluation will be colored by your personal experiences and biases.

I suggest designing functional tests that get at core principles and knowledge without being colored by best practices in the context of a specific situation. You want to evaluate how well someone uses a tool or knows a language. Give them a test that you don’t have a perfect answer for. If you work for a studio with multiple games, one way to accomplish this concept while still keeping the overhead cost low is frame the test for your team around the work of another’s (just keep anyone who transferred from that team out of the test review).

How Often Should You Give Applicants Feedback On Their Test?

Very early on in my career I took a test for Insomniac (I only mention this because the story doesn’t make much sense without this information). I didn’t get a follow up interview after submitting my test, a huge disappointment for a then-student who was super excited to get a test from a major AAA developer. When I asked for feedback on my submission I was only told, “It wasn’t Ratchety enough.” In retrospect I have to laugh at that response, given the urban dictionary definition of ratchet. But, I ask you, what could I have done with that feedback in order to grow and improve?

Maybe the test results aren’t what you expected. Maybe you were looking for something very different. Maybe you don’t think the way they presented the material was structured correctly. Or maybe, their test results just didn’t show the seniority level you were expecting. Fine. If they completed your test, you should do a follow up interview with them anyway. They just spent hours if not days on a test for you! The least you could do is help them improve by giving them 30 minutes of your time and a little feedback. Who knows, their explanations might surprise you.

Who Should Be Attending The Test Review Meeting?

There is a tendency in the industry to leave the test review process up to “someone else.” Especially in development it tends to get seen as a chore, or “not real work.” I totally understand this instinct. I remember early on as a lead I got really tired of doing the post test reviews even with candidates I thought were qualified. I would foist that meeting off on more junior members of the team. People who I thought had the chops to do the review, but who I thought had less demands on their time than me. Or, if I’m being real, I thought I had more important things to do.

This was a terrible mistake. Honestly, in my opinion, it’s a mistake to even bring someone junior to one of these meetings. They have a tendency to be less secure about their position and see the testing interview as an opportunity to showcase how well they understand things. Even if it is subconscious, they have a tendency to want to show off how they understand the team’s way of working, pipeline, design thinking, or whatever they think should be the focus. The problem is they also tend to demonstrate this by verbally attacking the candidate. They often don’t mean to, but it is the rare person in this position who doesn’t aggressively latch onto and dissect everything they think a candidate did wrong.

There are two problems here. One is obvious, the candidate who receives this treatment has a terrible experience and may decline to move forward even if you liked them. The other one is that these evaluations are often unfair or narrow minded. The junior team member is so focused on how this person didn’t do things the “right way,” that they might fail to realize the value in the candidate’s approach.

Who Tests The Test?

Can your people pass your test? Is the assignment clear? Do they produce the deliverables you expect? Can they do it in significantly less time than you would require a prospective applicant to finish in?

Ideally you can answer these questions by testing internally before disseminating your test to applicants. But, maybe you don’t have the inhouse expertise. Maybe you had to hire someone to write a test for you to build a department you don’t have yet. Or maybe you have a one person team you want to expand. You still have a network you can leverage. If you want your test to be reasonable in scope, a good indicator of whether someone will do the job, and elicit consistent results from well qualified candidates; it’s probably a good idea to reach out to people you trust and get some initial feedback.

TL;DR

Given that most functional tests out there are poorly done, here are some guidelines for making sure your test can be effective and your testing process will be both beneficial to you and the candidate:

  1. Discuss and be flexible with your test deadlines, because people are usually interviewing with multiple companies, have full time jobs, and real lives.
  2. Scope your test to take hours rather than days. If you think your test might be over-scoped, assume it is.
  3. Describe deliverables clearly being specific about what skills and knowledge you are trying to test.
  4. Encourage questions so candidates really understand what is being asked of them. If possible, have a test onboarding session with the candidate and someone from the functional department they are applying for.
  5. Don’t test with projects you work on every day, you have too many biases about the “right way” to do things.
  6. Always give feedback to candidates who finish your test. This helps them learn and grow, and might help them come back in six months and be right for the job. Also, they just did a bunch of work for you, don’t treat them like that’s nothing.
  7. Don’t have junior and new employees be part of the review process. They tend to have something to prove and their inclusion can result in a hostile candidate experience.
  8. Test your test. Make sure that people you already have (or people you know that are like the people you want to have) understand the test and can complete it in the time you expect. Also, tests are like a product, make sure you are constantly gathering feedback (going back to point 4), and spending the time to improve the test.

Hope you found the article helpful. Let us know in the comments below if you did!

Great (But Not Really Easy) Sourdough Recipe

sourdough bread

I consider myself an experienced chef. I started cooking in restaurants when I was 16, well I started cleaning dishes in restaurants but I moved onto salads fairly quickly. By the time I was in my sophomore year of college I knew I did not want to be a professional chef, and quit that job to restore furniture for a while before moving on to HVAC work when I moved and changed majors. However, I continue to cook for pleasure. Often, I find myself looking up recipes. Something I’ve noticed is that all the top (search) ranking recipes for just about anything claim to be “simple” or “easy” or “no-fail” plans. I’ve been experimenting with sourdough for some time now and can tell you that this is not one of those recipes (and thus it will probably never rank well, not to mention the fact I care little for SEO on my personal blog).

Every step of making sourdough is pretty easy, but put all together making good sourdough is a long and involved process that can easily go wrong. What I have for you here is a well tested recipe that will result in great tasting sourdough that looks something like this (depending on flour/s and/or beer you choose to use):

IMG_20210502_103734_845.jpg

But, before we get there you will need to have a nice healthy starter going. If you haven’t already, please checkout my post about making a great starter using wild yeast (and maybe some sour beer bacteria if you are into that sort of thing). You’ll be able to make your first loaf of bread in a few days, though don’t expect any miracles. It will be a couple weeks before that starter is really mature enough to make excellent bread.

Without further ado, here is my recipe in 15 EASY steps. Just so you know, the way I make sourdough takes 6 to 8 hours (or more) from fridge to cooling rack so plan accordingly. We’ll talk a bit about each step in turn.

Sourdough Recipe


  • 500g of a 50/50 mix whole wheat and spelt (sifted to remove much of the bran)
  • 200g of warm water (or beer)
  • 100g of milk
  • 150g of starter
  • 25g melted butter
  • 1tsp honey
  • 10g fine sea salt
  • ½ tsp baking soda 
  • fine ground cornmeal (for dusting)
  1. Feed your starter the night beforehand and leave it out overnight

You want a happy and active starter for tomorrow morning.

  1. Soak flour the night beforehand for a lighter softer loaf.

For best results I suggest an overnight soak. To do this, sift and measure out your flour then mix in the 200g of water (or beer) evenly distributing the liquid. Put the whole thing in the fridge overnight. It should look a little like shredded cloth.

I like to use a mix of whole wheat and spelt. I got into making bread for health reasons. I have bad reactions to the preservatives in store bought bread, and I’m trying to cultivate a good gut biome, for more on gut health checkout my diatecian’s site Positive Gut. But, if you don’t care about all that, using multipurpose flour will work just as well and probably give you a better rise. Though it tends to come out a bit more wet. The beer will add a little flavor, a strong (but pleasant) scent, and some color (if you go for a darker beer). You will not get good hop aromas from using a nice IPA so don’t waste one on trying. The sugars in the beer will also get the wild yeasts in the flour working a little and can help with getting a better rise (in case you haven’t guessed it, getting a good rise is the hardest part and we’ll talk about that in due time).

  1. Prep your mixture.

I suggest waking up early on baking day and setting out your soaked flour to warm up. The first few times I did this I took the four right out of the fridge and started the mix, but getting a good consistency takes a lot of hand work and the warm butter isn’t enough to heat up all that flour so your hands tend to cramp up a bit.

Whisk the starter, milk, melted butter, and honey together in a large bowl. Add the soaked flour and salt. Mix all that goodness together with your hands, squeezing all the big lumps out as much as you can

Feed the starter with 70grams of freshly ground wheat (or all purpose flour if your feeling lazy) and store it for a return to its regular feeding schedule.

  1. Let the dough mixture rest for 1 hour.

Cover the bowl with a damp cloth and leave it out to rest for an hour. I’ve found the ideal temperature for resting and rising the dough is about 30℃ (or roughly 86℉). In cool weather I like to place my bowl near the radiator. Spring is hard because you don’t want to be heating your house when it’s 18℃ outside so I suggest taking all the times in this recipe and multiplying by 1.5 when the temperature outside is warm but not hot. In the Summer, if your house is temperature controlled you can place the bowl out on a porch or something. Just be careful of critters, and make sure that cloth is really damp, not so much that it’s dripping water onto your dough, but enough that it keeps flies from getting in there.

  1. Work the dough into a rough ball.

Get all the little bits off the side of the bowl, should take less than a minute. Here’s a pic for reference:

  1. Let the dough rise for 30 minutes, then stretch and fold.

This will get some nice air into your dough and help the yeast get busy.

Do this no less than 8 times at a resting temp of 30℃. At colder temps you will probably want to do this more like 10 to 12 times. The dough should have a consistency somewhere between Gak and warm Playdough, and hold together pretty well when stretched to a 3 ply toilet paper thinnes. If you use beer sometimes you will end up on the Gak side of things, that’s okay but you might get some spreading during the final bake. 

In the event of an overly wet dough, to help with surface tension building you might throw as much as 25 grams of all purpose flour in there around fold 4 or 5 (by then you should know where you’re heading. I suggest only doing this once you’ve made a few loaves and have a feel for the dough. 

Cover the bowl with a damp cloth between foldings. Here’s a nice video on the stretch and fold technique for reference. That’s also a good reference for the consistency of the dough you are looking for, though a little less gooey is fine especially if you are using whole grains.

  1. Knock the dough back.

Prior to shaping, sprinkle some flour over your work area and start spreading and folding the dough to give it good surface tension. Here is a link for reference, this is also a good starter video for a few references like dough consistency, and this loaf is a bit easier to make. I started with this process and you could too (if you’re getting scuured). 

  1. Shape the dough. 

After you’ve knocked the dough back, spread it out like a small pizza and add half the baking soda as a thin dusting on the surface. Fold it up and do that again. Then shape the dough. Here’s a nice shaping video. It’s probably important to keep in mind this person has already dusted the surface of their work area with flour. Don’t forget that bit.

  1. Proofing rise.

Place the dough surface down into a floured proofing basket. If you don’t have a proofing basket a bowl with a thin weave towel in it will work. Put some flour on top of the dough and let it rise for about 1.5 to 3 hours depending on the room temperature. When it goes in it should look something like this:

Keep an eye on it, you want to bake at peak rise. When it is at peak rise it should look like this (this is a good rise don’t worry if you don’t quite get this height on your first try):

  1. Prep the oven.

When you know you are about 20 to 30 minutes from peak rise start preheating the  oven to 235℃ (455℉) with your dutch oven inside. Once you are at the desired heat, take the dutch oven out and place it on a hot pad. Quickly but carefully coat the bottom of the dutch oven with a dusting of cornmeal.

  1. Drop and score the dough. 

While the dutch oven is still piping hot, turn the proofing basket over as close to the inner surface of the dutch oven as you can manage without burning yourself. Immediately slash the surface along the length with a sharp knife or bread blade.

  1. Spraying the surface of the shaped dough generously with water.

I like to use a spray bottle that has garlic water in it (water that had bits of garlic soaking in it for a day). The garlic water leaves a nice aroma when cooking, doesn’t hurt the bread, and can be used to keep aphids off your plants. The water itself adds to the steam which is really great during the covered bake.

  1. Covered bake

Reduce the oven setting to 220℃  (428℉). Place your covered dutch oven in to bake for 20 minutes.

  1. Uncovered bake

Reduce the oven setting to 200℃ (392℉). Uncover the dutch oven and bake the bread for an additional 30 minutes (or until the outside is  the right shade of brown). If you want to take a scientific approach as I did the first 10 or so times, then take the internal temp of the bread (start after 30 minutes and take every 5 minutes). The bread is done when the internal temp is btw 96℃ – 99℃ (205℉-210℉). The bread should look something  like this:

  1. Cool for at least an hour on a cooling rack. 

After one hour you can slice it. If the bread seems gummy (leaves gum on your knife) stop cutting and let it cool for longer. Here’s a first cut of a loaf for you:

That’s it, that’s my recipe. It takes a while but gives good results. You can do fun things like rolling the dough in sesame seeds before placing it in the proofing basket, or maybe throw some herbs and spices in the dough during the shaping. Have fun and be creative.

GIST for Games (Development): Is GIST the Right Tool for Your Project Planning?

Hydra thinking about gist

One of the weaknesses of the game industry is that we are fairly slow to adopt new processes. A great framework that has come out in the last few years is GIST. It allows teams to eliminate the use of road-maps and stay Agile in their planning as well as their implementation. This article will give you an overview of the GIST framework, and drill down into how it can help solve common issues encountered while making games. It will also address some of GIST’s limitations and try to give you an idea of whether it might be right for your team.

What is GIST

GIST is a lightweight planning system designed with Lean and Agile goals in mind. It is built to be change-friendly, have lower management overhead, improve team velocity, help foster autonomy, and facilitate better cross-disciplinary alignment. The focus is on building things the customer wants with decision making driven by evidence over feelings and gut reactions.

The framework is named after its main building blocks: Goals, Ideas, Step-projects, and Tasks. There is a fun little coincidence of naming here. In Dutch, “gist” means yeast, the driving force behind beer fermentation, bread making, and other fun stuff. Since I’m an avid beer brewer and the co-founder of a Dutch-based organization, I liked this little correlation.


Going back to the GIST system, here is a breakdown of the building blocks.

Goals describe the company strategy in terms of desired outcomes without specifying solutions. Goals have a planning horizon of one or more years, encouraging your team to think long term.

Ideas describe possible ways to achieve specific goals. Ideas are constantly collected and prioritized (more on this later).

Step-projects are small objectives that can be achieved in 10 weeks or less and are additive to achieving an overall goal. Step projects are defined on a quarterly basis, the team selects goals they want to focus on and ideas they want to pursue that quarter and then defines step-projects accordingly.

Tasks are the smallest part of the system and well understood by agile planning, they are bite-size activities that break down the step project. If you are using scrum you could think of these as traditional “back-log” items. Tasks have a typical planning horizon of one to two weeks and fit in nicely with existing Scrum and Kanban approaches to development.

Idea Prioritization with GIST

One of the key components behind GIST is how ideas are prioritized. The process uses a scoring method that takes into account three main attributes: Impact, Confidence, and Ease (of implementation). Because we all love acronyms this type of scoring is referred to as ICE.

Impact is an estimate of how much the idea will positively affect the key metric/s you are trying to improve. Examples might be: Daily Active Users, Conversion Rates, Lifetime Value, Perceived Usability, and Session Times. Whatever is meaningful for your team to measure based on the type of game and play experience you are looking for.

Confidence is an estimate of how sure we are about the impact the idea will have. This measure helps us stay honest about our impact and ease estimates (things we are bad at estimating by themselves), and it gives us a way to quantify sources of data that inform our confidence level. Lowest confidence is given to things like self-conviction and planning, while highest confidence is given to live ops data and the results of targeted testing.

Ease (of implementation) is an estimate of how much effort will be required to realize the idea. It is estimated on the basis of weeks with less than one week providing the highest Ease score, and 26+ weeks providing the lowest.

An ICE score is found by doing a simple multiplication of these three variables. Here is an example table of ICE scoring for a number of features for a AAA PC 4x game:

IdeaImpact ConfidenceEaseICE Score
Twitch integration80.5832
Quick add Discord invites828128
Custom unit builder674168
Crossplay with new Online Subsystems992162
New scenarios70.279.8

In the above scenario, the team should focus their efforts on a custom unit building feature, and crossplay (maybe a Discord integration). The other ideas can be revisited when they have higher confidence or if it becomes easier to execute on them. The idea bank should be constantly updated and if you conduct a study that indicates strongly that your players would really like a Twitch integration then you can upgrade that confidence to 3 and maybe work on that in the next quarter. 

This has just been a quick look at GIST. For a deeper dive, I suggest checking out the website of Itamar Gilad, the creator of the GIST framework. He has a number of articles on GIST there and gives workshops on how to use this system effectively. For now, I’d like to move on to a discussion of how GIST can be useful for games.

Applying GIST To Games

Problems In Game Development That GIST Can Help Address

You have disagreements about what the player wants

Have you ever had an argument about what “the player” wants? Even if you are working from personas and gathering tons of data it can be hard to agree about this. Things get even worse when you consider early phases of development like pre-production when you don’t have data from your own game at all. Beyond that people just get really attached to their ideas, and tend to defend them as if they couldn’t just conjure up more if they put their mind to it. GIST can be a really effective tool at helping mitigate some of these issues. By requiring evidence based justifications for decision making you can create a more open exchange where there are rules of engagement for idea advocacy. 

There is a danger here. Some people might try to use GIST situationally while subverting it in other contexts. The framework as laid out by the original author doesn’t get super specific. You have to spend some time defining for your team how to measure confidence and what score to give a playtest involving twenty-five participants versus a structured survey with a few hundred respondents. You need buy-in from everyone on the team to get consistent scoring with informed estimates of ease, a structured approach to impact, and reliable confidence measures.

The most outspoken team members get their way all the time

I know as a designer that it is often the loudest person in the room that gets their way. Ambition tends to be one of the unifying attributes of game designers. We want to see our ideas accepted and implemented. We game designers are house Slytherin, not Gryffindor. GIST helps mitigate the effect of all those designers weighing in. It helps give room to evidence-based decision making so that Ravenclaws and even Hufflepuffs can find their voice at the table. 

You have ever-shifting requirements

GIST is built for the ever-evolving landscape of the lean startup with a focus on tech. As such it is highly adaptable to changing requirements. The hardest part is to still maintain the traditional sprint structure where you see through your tasks set for a specific time period. GIST is a process of constant reevaluation that helps surface the most promising ideas and has multiple planning horizons. It’s like it was tailor-made for a highly fluid and cross-functional endeavor like making a game.

The current process involves too many meetings and a lot of overhead

Making games often involves a lot of meetings which seem to get in the way of “real work” but don’t seem like something you can scale back. Part of the problem is that we often have warring priorities and there is a lack of clarity on who is handling what and when. GIST is lightweight, it is a single source that is flexible in a way that traditional road-mapping and adjusted project planning isn’t. Everyone can get a helicopter view of what each team is working on very quickly by checking out their GIST idea banks, and with more access to information and greater cross disciplinary transparency comes a decreased need for all those meetings to “coordinate” efforts.

When To Use GIST

The GIST framework grows in value the more mature the project

One thing that is important to note is that the GIST framework tends to grow in value as the project matures. Early on confidence in a team’s decision making is always going to be driven by personal experiences, historical data, and market trends. Since you don’t have anything to test really you can’t have medium to high confidence evidence. That said, your game might be a sequel to another game, a fast follower, or a time-tested genre that you are applying a simple innovation to. In these cases, you can at least have medium to medium-low confidence in many of your decisions even at the preproduction stage. The key will be to adopt a lean build measure and learn feedback loop. Get started on those grey box prototypes and interactive UI mockups so you can get some data and start having higher confidence in your decisions.

You should try to adopt GIST early

My point is, you should probably adopt GIST early. Ideally, start pre-production with the idea of using GIST in mind. I don’t think it would be healthy to shut down some of the creativity of the initial ideation phases of pre-production by demanding everyone ICE score their ideas, but it might help in the late stages to manage scope (even if confidence is going to be low overall). I mean we all know what a beast scope creep can be.

You can still migrate over to GIST late in production

You can also migrate over to using GIST later in the development process. As I’ve stated, GIST’s value is really being maximized late in the development process when you are actively testing a lot of more polished experiences, and during live ops on a game when you are getting a lot of data in real-time that can help you drive decision making. GIST easily plugs in with Scrum and Kanban. The first project I used GIST on, we used traditional Agile with Scrum until beta release when we became aware of GIST. We did have some problems with decision making being driven a lot by anecdotal evidence and gut reactions of some senior managers. When GIST came along and gave us a way to quantify and prioritize everything it was a chance we couldn’t pass up. We had fully integrated GIST into our process in only a few weeks, and it really helped us to focus our efforts. It also continued to be effective when we decided to move from Scrum to Kanban during live ops. 

GIST is probably best for mid to large teams.

GIST is probably better for medium to large teams. I’ll get into this a little later in this article, but many smaller teams don’t have a dedicated Producer or PM, and the work associated with GIST is probably best handled by someone whose job description includes owning all the production planning process. If you have to, you can spread the work around and mitigate the impact on any one team member. Though this will probably put a little more strain on your Leads and Product Owners. With a large team, you can do both, spread around some of the costing/scoring work, and have a dedicated individual owning and advocating for the process. This isn’t a law of project planning by any stretch. If you’re on a team of less than ten people, and still have a dedicated Producer (or are just super process-oriented) then GIST could still be a good fit for you.

Issues with GIST

GIST is “lightweight”, but lightweight is really in the eye of the beholder. It still requires a dedicated Product Manager or Producer to own the GIST framework and help organize the teams. Either that, or everyone has to make small but meaningful contributions and take ownership of their part. This can be problematic. A common structure for teams is to have Game Designers serve as Producers and/or Product Managers. If you add ICE scoring and idea bank management, along with coordinating goal setting, and quarterly step-project selection to an already full plate it might become a bit much for someone. However, it does replace a lot of often updated and hard-to-maintain processes like road-mapping. My point is, you need to be careful and clear about who will be responsible for the process ownership and being an advocate for the GIST framework (that might be the same person, it might be multiple people). 

Like any framework, you need to get buy-in from the whole team. You can’t have one department decide they don’t like using GIST and won’t adopt it, while everyone else is using it. Not only will this cause confusion: if you have to justify your decisions through confidence measures with clearly defined thresholds while another department can just “go with their gut” you might tend to resent them.

Finally, there aren’t as polished and sophisticated tools for using GIST. The Atlassians, Mondays, and Asanas of the world (to name a few) don’t have GIST workflows yet. Though, I imagine the lack of tools for GIST will change over the next few years as more companies realize its value. But, there are some pretty slick solutions for using roadmaps these days, like next gen projects in JIRA and roadmaps in ProductPlan. That said, there are still many people out there building gantt charts in excel, which just illustrates that you don’t need really sophisticated tools to use a framework.

TL;DR Summary

To sum it all up, GIST is a fairly lightweight framework with multiple planning horizons that puts a focus on evidence-driven decision making, over instinct and seniority. It plugs in nicely to existing Agile methodologies and encourages test-driven design and development. Will it automatically make you produce rainbow unicorns or the next Pokemon Go? I don’t know. Does it have the potential to make things a little easier for you? Very much so. Go check it out and see for yourself if it’s right for your project.

Niche Game Development Disciplines: How to Specialize in Technical Design

This is a repost of my post to the Tiny Hydra blog. To read the original go here.

I’m often asked by my former students, “What skills should I really be learning to get into the industry?” The answer to this question is really complicated, and depends largely on what the person wants to do. However, there are a few jobs that are very difficult to fill for many companies and if you have an aptitude for these disciplines it can be a great career path for you. This is the second in a series of blogs, each discussing niche positions, what they do, how that varies from studio to studio, and some good ways to get started. Checkout the previous post on Technical Art.

Both Technical Design and Technical Art are services Tiny Hydra offers. If you’re interested in being one of our ‘Hydra Heads’ so you can help studios make better games, take a look at our recruitment page. We’re happy to have new entrants into the industry with demonstrable skills. We make a special effort to train applicants in the soft skills side of working in games, not only the technical aspects of specific disciplines. We also specialize in rapid onboarding and cross-functional collaboration.

For this second article in the series, we will be discussing the Technical Designer role. A little attribution before we continue. Because I have been doing Technical Design for most of my career, I felt like I might have some very strongly held opinions that were not always accurate, so I enlisted the help of my friend Brian Jennings Lead Technical Designer at NZXR, while coming up with ideas for this article, and I want to thank him for his contribution here, especially in coming up with “beginner” Technical Design projects.   

What is a Technical Designer?

A Technical Designer is a facilitator, helping to move concepts from ideation to implementation. They serve as a practical bridge between disciplines (design and engineering), and as a professional “unblocker” for the design team.

Where do Technical Designers come from?

I think most Technical Designers come out of design. They tend to be people who got fed up with how tools were working. Most disciplines are driven by questions. The artist asks, “How can I make this look cooler?” The UX Designer asks, “How can I remove a click?” The Game Designer asks, “How can I make this more fun?” (or some variation on those). The Technical Designer asks “How can I make designing this go faster?” If you often find yourself asking that same question and motivated to do something about it, then you just might be a good Technical Designer. 

Like Technical Art, Technical Design is a discipline in its infancy. There are no programs that I know of (please let me know if you know different), where you can major in Technical Design. And even today, if you’re a designer looking to make tools (or worse, an engineer looking to make the design team’s life easier), it might be an uphill battle to convince your team your time would be better spent on that than something else. You might even be told to, “Stay in your lane.” 

There are actually a fair number of barriers to entry to Technical Design. While Technical Art is getting to be pretty well understood by most people in the industry these days, as of the time of writing this article Technical Design is still not as common at studios, or the value is not as well understood. Part of the problem is, when some people see you moving outside of what they consider to be “your role” they tend to get a bit uncomfortable. Other people might feel threatened, like you are encroaching on their territory. And so, artificial barriers to entry have been set up around roles like Technical Design. Usually, you can get past these just by showing the value of what you’re trying to do. But, that may require you to do work on your own time to sell your ideas, and it doesn’t always work out.

So, where do Technical Designers come from? Most of the time they come out of Design. People with an aptitude and a desire to solve problems. They can also come out of Technical Art. Often Technical Artists find themselves asked to build tools to facilitate solving design problems, and eventually have so much knowledge of an engine and the design processes they are working with that they become the go-to person for technical design tasks. Sometimes, it’s an engineer with a propensity for design and a desire to make tools or rapid prototype ideas.

What do Technical Designers do?

Here’s a one-sentence definition: A Technical Designer is a Designer who can create functions for their game’s high-level scripting language, or creates interfaces/tools for design.

The answer here is a bit more straightforward than with Technical Art. Okay maybe it’s a byte more straightforward, but still not much more (look Mom, I haz programmer jokes).  In fact, Technical Designers do many of the same things as Technical Artists but just from a different perspective.

One of the most common functions of a Technical Designer is building tools for designers. If you are totally new to the concept of Technical Design, you might wonder what that means. Well, it can mean a lot of things: It could mean creating two-way communication between a game engine and the spreadsheets that System Designers work out of. It could mean building a way to procedurally generate game objects based on a set of criteria. Or, it might mean creating widgets/gizmos/brushes/ or other interfaces to make level design quicker and easier. The breadth and depth of tools a Technical Designer can create are really only limited by the imagination, what’s possible with the latest hardware, and the development skills of the Technical Designer .

Technical Designers might also be the person tasked with rapid prototyping ideas, and might not make tools at all. They explore game mechanics and systems by making a lot of quick and dirty minimum playable experience (more on this in a future blog post, I promise). These prototypes can be tested and used to discover what’s fun and engaging so the team can focus their efforts on what will make the game compelling to users. This side of Technical Design means throwing away a lot of your work when it doesn’t test well. So if this is the sort of thing you’re interested in, you really need to marry your brain to Jesse Schell’s concept of how to handle ideas:

Ideas are not like fine china, ideas are like paper cups — they are cheap to manufacture, and when one has holes in it, go get another one.”

It’s the “be willing to kill your darlings” philosophy of game design, and it will probably serve you well no matter what discipline you’re in. But, as a Rapid Prototyping specialist, you will probably be doing this more often than not. So if you are a person who loves exploration and scientific experimentation, and is not super in love with iterating on the same thing for years, until it’s “perfect”, a rapid prototyping specialization might be the right fit for you. 

The typical differences between what a Technical Artist does and what a Technical Designer does are the focus (art vs design), and the technologies they tend to work with. This is not always the case mind you. I see more Technical Artists working to facilitate the use of Digital Content Creation (DCC) tools using languages like MEL, MaxScript, Python, and VEX. While I see Technical Designers focused more on extending the in engine functionality and typically using languages like C#, C++ and LUA (yeah creating Blueprints too).

If you can do both (tools and prototyping) then you’re batting a thousand (did I use that right? I know next to nothing about Baseball, but I think that’s the right idiom), and should probably start focusing your resume and applications on Technical Design. A general distinction to be made here is that some Technical Designers are programming to support their own creations (high-level gameplay programming and rapid prototyping) and some to facilitate the work of others (tools and pipeline development). 

What Should I Be Doing to Become a Technical Designer?

Again, the first answer is: “get good” at math. After that, it really depends on what you want to do. But, there are a few things that will make Technical Design tasks more accessible to you from a high level:

  1. Learn an object-oriented programming language.
    1. Don’t Panic! 
    2. If you want to use Unity then (no contest) the choice is C#. But, generally, you should probably learn C++ first (or you know you could learn Objective-C; after all they are both just preprocessors in front of C). Straight up, if you learn C++, you will be far more efficient in Unreal than if you just learn Blueprints. 
    3. If you find C++ (or even C#) overwhelming, you could always start with something like Scratch, which makes the common lessons of programming super approachable.
    4. If you do want to start with a web development first approach consider JavaScript, it’s a pretty solid intro language. There are even some fun JavaScript based engines out there, like PlayCanvas.
    5. Whatever you do, don’t waste your time with JAVA (he said, showing his bias). And, don’t be like me, I started object-oriented programming with PHP (back then though I thought I’d be pursuing a career in web development).
  2. Learn about data structures, serialization, and reading/writing files from your programming language of choice. 
    1. These are subsets of learning to program. But you want to become pretty solid on these as teams will use a ton of different solutions for managing designs, and engineers will have a lot of different opinions on how to best handle data. And you are going to want to marry those two using tools in many cases. Because, you aren’t going to get designers to stop using spreadsheets, and engineers aren’t going to handle a bunch of CSVs when they can demand JSON… You see what I mean?
  3. Learn about Design with a capital D.
    1. You should learn Design concepts really well so that you can easily communicate design intent to engineers. Once you can do that, you should have no problem understanding the needs of your design team and solving them yourself (provided you have the programming chops).
  4. Learn an engine.
    1. Some people will tell you that once you have a good grasp of a language then you can learn an engine. But it is really about the approach. For me, it was always about getting things done. As a consequence, when I was getting started I tended to learn the basics of the engine first. After that, I learned the basics of the language. Finally, I learned how to use the language in the engine thus increasing my understanding of both by an order of magnitude.

What Makes a Good Technical Designer?

As with so many things, being an effective communicator is really important. Technical Designers are a bridge role that needs to be able to bring Design and Engineering closer. We’ve already talked about how it might be necessary to advocate for the value of Technical Design. Beyond that, as a Technical Designer, you will need to “sell” your prototypes, or tools. Getting good at presenting, showing an ability to empathize with people from other disciplines, and speaking the language of your peers regardless of their focus: all these soft skills will serve you well as a Technical Designer.

You’ve got to be problem focused. You need to demonstrate an ability to identify, and define a problem quickly and work towards implementing a solution. 

As a Technical Designer, it is important to be scientific. You need to be test driven, and understand the value of research. Whether you are serving the needs of others on your team by facilitating their work through well designed tools, or creating lots of testable prototypes; being willing and able to gather and act on useful feedback is key to doing strong work.

It also helps to be a bit of a jack/jill of all trades. The technical design position is something of a generalist position. One day you might be putting on a pure design hat, the next doing UX research for something you’re creating, another creating some rough art assets to help convey the idea of a prototype you built. Having at least a working knowledge of things like 3D modeling, digital illustration, VFX, sound design, UI design, etcetera; will help you make stronger work while also allowing you to better understand the needs of your teammates. 

Another thing that makes a good Technical Designer is their ability to design. There is no one size fits all way to be a good designer. Some people love to design by taking inspiration from other games and building on that. That’s fine. In fact, there are designers out there who I look up to and that seems like the only thing they do. They play a lot of games, they experiment, and they make. But, imho, the best designs come from personal experiences. In the words of Jess Hider (Technical Game Designer at Rare):

“Hobbies and life experiences are the most important things for a game’s designer, because it means we can pull on first hand experiences, not regurgitate somebody else’s.”

See what I did there? ‘I’m such a hypocrite.’ No, but seriously, this is a perception I have long held and is the reason I have spent my professional life moving around, why I’ve traveled everywhere I can as much as I can, and why I’ve picked up all sorts of hobbies from brewing beer, to snowboarding, to rock climbing, to maker work. So, get out there, so you bring your own life experience to the table in the future. 

Where Should I Start?

Like with anything, start small. Try reproducing solutions to problems other people have already solved (without straight up copying). Maybe expand on a game project by making all the settings that drive the experience easy to modify outside the code (accessible at the engine level to non-technical designers). Then move on to bigger game (yeah, you know you like my punz: game like hunting game in the context of a game development blog, hold your nose against my maximum pun-age). You could also try a training site like Code Wars.

Below, I’ve compiled a few projects you can try to get yourself started and explore whether Technical Design might be for you. I want to say that these projects are nontrivial undertakings. In the case of the more advanced projects, many of the things here would take a skilled Technical Designer several days of serious work to accomplish, and are likely to take a novice a much longer period of time (like probably a lot longer). As with anything, practice makes perfect you better.

Getting Started

  1. Map generation tool. 
    1. Create a project that procedurally generates a map for games that makes use of tiles or hexes (like a 4X or RTS). Can you log your seed and any custom properties for later use? Can you build a tool to take the map you like, load it up, and then change aspects of it? What about being able to change individual tiles (or hexes) via a brush or maybe execute batch changes via a menu?
  2. Enemy generator.
    1. Let’s say you have libraries of armor and weapons your enemies can use, and base stat ranges for your enemies with standard modifiers based on their level. Can you build a tool that will generate variations (5, 10, 20, 100) of an enemy with a single click? Can you reference a file to pick any number of these generated enemies and present it in game?
  3. Quick add tool.
    1. What if any time you dropped an asset is into a folder it became part of a library of characters that spawn in an environment? Can you use tags in the name of the asset to define properties or assigned behaviors?

More Advanced & Bigger Projects

  1. Procedural localization tool. 
    1. Make a tool that will search all prefabs in a scene, grab all the text mesh components, throw the text into a spreadsheet, localize it using something like Polyglot, then be able to reference the results and change what’s presented in the game based on a language setting.
  2. Asset usage lookup tool. 
    1. Click on a project asset with this tool open and see all instances where it is used in the project. The same tool could also search all your assets and find anything that isn’t being used (find assets without references). That would be super useful for cleaning up bigger projects.

Resources

Here are some things you should look at. 

  • http://blog.cleancoder.com/
    • Uncle Bob is a friendly inspiration to all of us. He’s one of the authors of the Agile manifesto and a development historian. Engaging to read and watch, he’ll make better recommendations for things you should check out than I ever could.
  • Game Maker’s Toolkit
    • A great youtube channel on design and development. Maybe start with versatile verbs.
  • http://stonetronix.com/
    • Stone Librande is a very smart man who says smart things, mostly about design. Check out his website, I’m especially a big fan of his one page design doc philosophy. 
  • https://www.redblobgames.com/
    • Math and math visualizations that make it more approachable (or at least cool to look at).
  • https://unrealslackers.org/
    • Unlike Technical Artists who have tech-art.org, Technical Designers do not have a go-to community maintaining an extensive forum and active slack (to my knowledge, please let me know if I am wrong as I would like to join). In lieu of that maybe join this discord, or the next one, or both. 
    • Side note here, if you happen upon this post and are interested in running a Technical Design group I would be happy to support you, I would start one myself but (as with everyone) I’ve got too much going on already.
  • https://discord.me/udc
    • Same as above, but for Unity devs.
  • http://www.squidi.net/three/
    • Three Hundred Mechanics is a project by Sean “Squidi” Howard, and is an interesting exploration of game design and (some) implementation. Worth checking out.
  • http://gameprogrammingpatterns.com/
    • Not going to skip the about page that offers to sell you this free book, as the author deserves the chance to sell it to you. But it is free, and super useful if you want to go from understanding that programing is just “assignments, if statements, and loops” to understanding how things can be more efficient.
  • https://www.habrador.com/
    • This collection can be a bit difficult to parse but there is some good stuff there, especially in the Unity tutorials. That said, those aren’t so much tutorials as an explanation followed by a code dump, so don’t skip straight to this stuff.
  • https://catlikecoding.com/unity/tutorials/
    • There are a lot of good tutorials there, and honestly, there’s something for everyone here from a development perspective. 
  • Game Design Resources
    • A giant workbook with everything from General Design resources, to Tools, to Career advice. It’s a lot to parse but totally worth it.

A Sourdough Starter For Homebrewers

As a brewer I have a lot of ingredients that, surprisingly, we don’t think about much when it comes to baking. In my home I have malted grains, I have wheat, I have yeast. Oh we think about this stuff in the sense that we think about where we need to go buy this in it’s preprocessed form to follow our baking recipe. But, when was the last time you ground your own grain flour? Okay, yeah, I realize that question sounds pretentious, but bear with me.

A few years ago I got pretty fat, and over the last year I’ve been working with a dietician to get healthy. In 2020, I really focused on cultivating good gut health and getting my blood pressure down. And, while I shed a few kilos, this year is the year of weight loss. All this is to say that I’ve drastically changed my eating habits, and for the most part, when I eat bread now, it’s sourdough.

However, good sourdough is kind of a chore to go buy, especially in winter in the Netherlands. It might not seem like a 15 minute bike ride down to the baker isn’t a big deal, but over icy terrain in the bitter cold… Would you be psyched to do that? Ah, well, maybe you would, I’m not. Also, I’ve been starting my own company, so my budget is tight and artisan bread is expensive. So, I decided to make my own sourdough, after all I have some good ingredients on hand, at the very least for a starter.

A starter is a wild yeast cultivation, and I have a ton of wild yeast floating around in my home, presumably, I mean I brew beer at least once a month. So, I figured, ‘Why not try to get a good one going?’ Then I can make all the sour dough I want and I won’t have to ride across town for it. I did some research on starters and all you really need is water and flour.

The key to a good starter is the natural occurring microbes that eat all the simple sugars provided by your water and flour slurry. The two big players (or really groups of players) here are the wild yeasts and lactobacillus strains (lactic acid creating bacteria), the later being responsible for the sourness of your starter. But, don’t take my word for it, you can read more here. While store bought flour probably has enough lactobacillus on hand, I thought some brewers wheat would have quite a bit. Which brings me to step 1.

Note about the coming steps: While you want to cultivate wild yeast you probably don’t want any old thing in there. So wash your hands, and sanitize your measuring devices and utensils (with boiling water). 

Step 1: Finely grind 50g of wheat.

For this I used a clean hand coffee grinder, if you’re a brewer you might be thinking your grain mill will be okay but it probably won’t get it fine enough.

This is what mine looks like.

Step 2: Combine with 50g of water in a 1 liter covered jar, and place in a warm spot for 48 hours.

You’re looking for 24℃ to 27℃ (Roughly 75-80℉). I put mine near the radiator, but not too close. Don’t pour off the hooch (the brown watery substance that smells like feet that have been in boots too long) if you get any. In the future this will be a strong indicator your starter needs feeding, and you’ll pour this off prior to doing just that. But, for this first period, just let all your little microbes grow, it will be fine.

Step 3: First feeding.

Add an additional 50g of water along with 50g of all-purpose flour. If, like me, you wish to make your own all purpose flour it is pretty simple (if time consuming), check out this video. After you’ve added your water and flour, cover the jar and let it warm again for 24 hours.

Step 4: Feeding 2 & optional inoculation.

Pour off any hooch and then discard a bit more than half the starter. Feed it 100g of all-purpose flour and 100g of water. You can leave it like that and skip ahead to Step 5 or take a chance on the optional step below.

Optional Step: Now, here’s where things get interesting for my brewer friends (not that anyone reads this blog). If you have a favorite lactobacillus strain and you want to get some really interesting bread, go ahead and throw some in there. If you are new to brewing or don’t have your own cultures, that’s okay: probably the easiest thing you can do in this case is get one of the WildBrew bacterial cultures from Lallemand (you can buy these at most online homebrew supply retailers). Add a small amount to your starter (like the smallest amount you can measure on your scale, for me that’s 1g).

The great thing here, again if you are a brewer, is you are creating an interesting microbiome that will be good for making sourdough, but also for making funky sours in the future. Though I would suggest using a kettling methodology, otherwise some not so nice yeasts could out-compete any beer yeast you pitch. If you just straight up use the sourdough starter to create a yeast starter or direct pitch (and then don’t kettle after 3-5 days), you will probably end up with something containing butyric acid (which might smell vaguely of vomit or rancid cheese), so I recommend against this course of action. Anyway, my point is if you keep your sourdough starter alive it can “wear many hats” in your home funkatorium. Ooooh, new idea, I wonder if this could make interesting kimchi…

Step 5 – 7: Continue Feeding

Pour off any hooch and then discard about half the starter. Feed it 100g of all-purpose flour and 100g of water. Do this about once every 24 hours. All the little microbes should be pretty active now and the starter should start to bubble up quite a bit between feedings. When things start to settle down (the starter begins to fall back down to its size when it was last fed) then you know it’s time to feed the little beasties again.

Step 8 – Infinity: Transfer, Store & Feed Periodically

Get a nice clean jar and transfer your starter to that. If you can, keep it at room temperature and feed it everyday (the lactobacillus strains will be competitive at room temp where as in cool temperatures they will be less active than their wild yeast counterparts). Or, put it in the fridge and feed it (same way as before) about once a week. Keep an eye out for hooch formation, as I said before that’s a good sign the starter needs to feed.

7th Day Results:


Bread Recipe Coming Soon…

Honestly, I’m not happy with the density (texture) of my finished sourdough yet. But, the flavor is good. I think it is just a mater of water ratios and rise times (I’ve been a little impatient and tried to do a sourdough in a day). When I have a method that yields a result I’m happy with, I’ll try to take the time to post it here. If anyone actually reads this and wants to point me to their favorite bread making process (that can be done with a simple oven) shoot me an email or tweet at me.

Game Systems Analysis: Creating a Competitive Pacifistic Build in Stellaris

With Covid-19 we are all spending a lot of time at home, and have taken up many hobbies. But it can’t all be fun and… never mind, sometimes your professional appetites leak in. To that end I’ve been trying to do a deep dive on a highly complex game Stellaris. If you haven’t played Stellaris, it is a 4X game about space based empire building. It’s a really wonderful game and one of the best offerings from Paradox (imho), however it certainly isn’t for everyone and that is largely because it has a ton of systems all intertwined and takes a long time to learn, and even longer to “get good” at.

One of the things that bugs me about playing Multiplayer in Stellaris is that the dominant builds are all tech rush war builds (and often xenophobic slavers). Honestly I don’t find war that interesting, especially in a multiplayer game where I want everyone to have fun. I also want to play closer to my personal preferences in terms of civics and ethics. To that end I have been trying to create an effective Pacifistic, Xenophilic, and Technophilic civilization that can compete against things like crazy Slaver Materialists and their ilk. It’s been largely an uphill battle with some builds performing ok, but most ending up at the bottom of the pack and getting exterminated by warmongering empires that (sometimes literally) want to eat my people. 

However I have had great success recently with a build I created that exemplifies all the traditions of openness, inclusion, and discovery I want, but also is highly competitive. I’d like to walk you through this build and why it is competitive in both multiplayer and against Grand Admiral AI. Please keep in mind that this empire model is effective in 2.7.1 (and 2.7.2 test) and if you are reading this later on it might be nerf’d. A note before reading further, this is not a beginners guide to Stellaris by any stretch. This blog assumes you have some experience with the game or at least are willing to read up on its systems. If you’ve never played Stellaris, I recommend you go play a bit, or watch some beginner tutorials before reading further.

The Build: Rogue Servitor Remnant Pacifistic Technologists

This is a machine empire build with an organic civilization that produces tons of unity for you. Only the civilization you will use is not strictly organic because early game you will only have one Bio-trophy civ to take care of and they should be Lithoids (rock eaters). 


The thinking: This build makes for a simplified and powerful economy that is easy to ramp up early game. You can take a long term lead in certain tech (which you can boost by clearing remnants on your homeworld), and can afford to ignore food focused technologies early game since you won’t need them. Your super awesome economy can just out perform everyone else, allowing you to play tall (even though everyone else has to play fairly wide in the current meta).

Let’s walk through the traits, government, and ethics:

First off you are a machine intelligence so straight out the gate you have no choice (Gestalt Consciousness and Machine Intelligence are the only ways you can go. But, you do have some choices when it comes to your Civics:

Rogue Servitor: This Civic is the backbone of this build. It gives you biotrophy pops you can spread across your civilization and as long as they are happy they create a bunch of unity for you. Your empire will have massive stability and if you embrace Lithoids as your starting bio trophies, you can just focus on a mineral and energy based economy with no need for food. Later in the game you can quickly ramp up food production as you open your borders to the scattered masses of organic pops that will be desperate for a new home after all your neighbors start going to war with each other. At that point you should be technologically dominant (or at least one of a few tech super powers) and it will be a simple matter for you to ramp up food production for those organics.

You have two choices now for a second Civic. I like Maintenance Protocol for its boost to early game unity, but another viable choice here is Rapid Replicator. Either choice will give you awesome pop growth and great unity, the only question is which one of those you want to maximize. Population growth is very dominant but getting through the Discovery and Expansion traditions super quickly will arguably give you just as much of an edge (and sometimes you just won’t be able to colonize quickly enough to make use of maximized pop growth early game). Again this is dealer’s choice, but you really need to go with one of these two civics. 

Next, let’s talk Traits: I’ve gone with Emotion Emulators, Mass Produced, and Superconductive. This is a pretty standard set of traits for this type of empire and I don’t want to deep dive all of these too much, but you do have a choice here. You could swap out Superconductive for Logic Engines. This will make the power of your economy less of a given (because you are sacrificing a big boost of energy credits) but it will definitely increase your ability to tech rush, and (again) early game you can ignore food and agricultural techs so a maximized tech rush can also be very targeted. Either way you should be able to get ahead of most, if not all, empires in terms of military tech and be able to defend yourself effectively through the use of suped up star bases and an advanced but fairly minimum defensive fleet.

As for the negative traits: High Bandwidth is pretty much a given especially with this build, since you will ideally play tall and empire sprawl is just not that big of an issue. The ROI is high as it gives you 2 points to use for positive traits. I’m happy with Repurposed Hardware, to free up the other point you need for this build, as a slow gain of leader levels isn’t that big of a deal. However robot upkeep is also going to have low impact on your civ so High Maintenance is also a good way to go. Personally, I feel like you have to think about it more, and this game gives you a lot to think about all the time already. So, if you aren’t the most experienced player in the world the leader leveling is very much a passive system and I recommend going that route.

Now for your Bio-trophy pops:

As I said you are going to want to go Lithoid to simplify and streamline your economy early game. Lithoids also can live pretty much anywhere so they are less of a problem to spread around your empire and get all your colony worlds producing tons of unity for you. After that Conservationist and Traditional are the best choices here. These are best because the first will allow you to really not worry about consumer goods (you can in fact sell your consumer goods early game), and the second will just boost your unity more. I have been running Natural Engineers as my final choice because Engineering technology is the most dominant in the game (given the current meta) and a slight boost to this early game can pay huge dividends late. However, it is a very slight boost, and it is perhap just as good to go with a two point trait here. Good choices include Intelligent (for the tech bonuses though I honestly wouldn’t spend the extra point as Natural Engineers will get you what you want without the extraneous fluff) but maybe a better choice is a Lithoid specific trait: Volatile Excretions. This trait will give you access to motes in the early game and allow you to clear remnant blockers on your homeworld earlier, those blockers will give you tech boosts in turn and the overall effect of that can be huge.

One note here, if you use a 2 point trait you will need to pick up another negative trait for your bio trophies. A good choice here is Deviants, though you could also scrap Decadent and go for Repugnant which gives you two points. I personally would just not add more negative traits because while their impact will be minimal they will counter some of the good choices you made earlier. That is why I opt for Natural Engineers, even though it means slower exploitation of my remnant world (but slow and steady ramp up wins the race). 

Some last notes. You really need to focus on alloys early game so you can expand (so build up alloy production before a research center on your home world). Unlike with most empires, go with Discovery over Expansion first in the traditions. The bonuses are better for a tall technophilic empire. Finally, all your star bases need strike craft so you don’t get pulled into the early wars. Your fleet power will be low otherwise and all those militaristic empires will start thinking of ways to carve you up. But, if you rush strike craft and star base technologies, you will be able to fight defensive wars effectively with a small peace keeping fleet (provided you secure your choke points).

That’s it for now. As the Vulcans say, “Peace, and long life.”

Musings on Improved Efficiency During COVID-19 Shelter in Place

Does a time come for every business when the CEO thinks, ‘We really should just get rid of our offices…’? Right now, I’m hoping our CEO is thinking something along those lines. Maybe not getting rid of our offices entirely, but just paring down to a nice place to show visitors and a storage space for all the stuff we can’t really store in the homes of our various employees. I imagine our boss showing a group of investors around a plush meeting room and saying, “This is the magic of MiniBrew.” Inevitably someone will try to look behind the innocuous little door in the back – beyond which lies a storage space that looks like the walk-in closet of a mad inventor or a child who likes to take apart the consumer electronics his parents left him alone with – and my boss with throw himself in their way. “Nothing to see here.”

The company I work for, MiniBrew, like so many companies, has given a “work from home” order in response to the emergence and rapid dissemination of the coronavirus (soon to be re-branded as the Bud-Light virus). It isn’t the first time I’ve worked from home, but it’s the first time I have really noted the stark contrast in terms of what I get done in the office, and what I get done in the same amount of time at home.

In his book, The Goal: A Process of Ongoing Improvement, Eliyahu Goldratt posits the idea that in a truly efficient work environment there is downtime. Basically: if you are constantly working then there is something wrong with your operations management. I read the goal recently and I have thought a lot about this idea ever since. Because, at the office, I’m constantly working and I have done many things to make my processes more efficient and optimize my work life. It never really goes away. The problem is mitigated, there is less stress sure, but I always run out of daylight.

However, in the last few days, I have noticed a very different paradigm emerging. I have time left over. One thing I started a while back, is at the end of a week I look at my tasks for the following week and try to time-box them in my calendar (leaving some wiggle room to shuffle things around, deal with unplanned for challenges, and pick up the odd last-minute meeting). Since I started working from home, I’ve noticed I get ahead of my time boxing (which I’ve gotten really good at estimating). In fact, I am typically done with my planned for tasks by 15:00 (3 PM). I then take a look at what I can do with the last few hours of my day and knock that out.

So, what’s changed? Well not that much. I’m still working the same way, I’m still engaging in the same meetings and scrum ceremonies. I’m still facing the same hurdles, in fact in some cases getting over certain hurdles (like getting a back-end developer to run down a latency in our alpha environment) actually takes longer and that’s mainly because I can’t just walk over to a developer’s desk and say, “Could you help me resolve this issue real quick?” But, guess what? No one else can do that to me. That’s where the difference lies. 

On any given day I must get interrupted somewhere between eight to ten times on average. Most of the time it’s just for a “quick chat.” A hardware developer wants to know what I think about testing a new prototype with users, a software dev needs to understand requirements better, or a customer service rep wants insight on when to tell someone to expect a new app feature we are about to roll out. But it breaks my stride, I have to stop what I’m doing, pay attention to that person, then re-engage in what I’m doing. If each of these quick chats take between five and ten minutes (building in some time spent getting to a good stopping point, switching my focus, and then getting back into it): I’m probably spending around… a ton of time each day breaking my focus in an unstructured way.

When people tell me they hate Slack, I think, ‘Well I don’t, but they’re entitled to their opinion.’ However, honestly, it’s the best. It isn’t like people have stopped hitting me up to have a quick conversation. They just aren’t sitting next to me. I’m not rudely ignoring them if I finish what I’m in the middle of before I respond to their inquiry. And that makes all the difference. 

Sure some people call me and interrupt, but usually, what they’re calling about is pretty important. Most of the time, they are kind enough to send me a quick note first saying, “Hey, can we chat for a minute?”

Animated Shorts MDC 2017

Last semester I taught the Production course for Animation Studio at MDC. It was a really challenging experience. I was told, after the projects were pitched, that I would have about 40 students in the capstone production course. We didn’t, I started production with only 17 students and two of them dropped. As their faculty mentor I basically served as a line producer on the project, and it was a real challenge managing schedules with students juggling work on multiple shorts at the same time.

We had students wearing many hats and working on at least two shorts at a time. They were stressed and overworked and it is amazing what we got done. No, we did not accomplish all our goals, but these kids made something they should be proud of, and I’m certainly proud to showcase it here.

Pitching a Game At MDC

Today I am a professor at Miami Dade College in a new program called MAGIC (Miami Animation & Gaming International Complex). This spring will mark the first time our students will pitch capstone projects and I recently gave a lecture on creating game design documentation as part of their pitch. If a student had me for Intro to Game Development then this was all redundant and woefully truncated, but there were many students that needed this primer. For those that were unable to attend I’ve made my deck open to view online: tinyurl.com/mittnerworkshop

While this is certainly not the end all be all I think it is a good place to start if you are thinking about pitching a game. Check it out, I welcome feedback.

What Makes a Good Game Designer?

I’m between jobs right now, looking for the right fit. Consequently I’ve been taking a few design tests. On the last test I took there was a section devoted to game design philosophy. Questions like,  “What is game design?” or “Where does design fit in a game development pipeline?” so on and so forth. One of my favorites was, “What makes a good game designer?” Now there are some obvious answers here: some knowledge of other disciplines, the ability to problem solve, the ability to communicate effectively, etc. However, in my opinion one thing is often overlooked: experience.

“Oh that’s obvious Martin!” you say dismissively. But I’m not talking about 5+ years of industry experience and two shipped titles. Trust me I’ve met people with more industry experience than that who didn’t have enough life experience to identify with the outsourcers from Germany that they had to work with on a deliverable. Life experience is crucial, and has nothing to do with brainstorming, understanding best practices, or sitting behind a desk writing documentation. What I’m talking about is having done “things and stuff.” Having connected with people from other cultures, having lived abroad, having tried to navigate a waterfront cave system; that sort of thing.

A while back I watched a video called Humans Need Not Apply. If you haven’t watched this film you probably should. This short talks extensively about how humans are being replaced in the workforce. Much like the emergence of the automobile put horses out of work, humans are being replaced at a rapid pace by new technology. It gets pretty scary talking about how low-skill, white -collar workers, and professionals are all in danger of being replaced by machines (of one sort or another). Yes, even doctors. At one point in the film the narrator says “Perhaps you are un-phased because you’re a special creative snowflake.” He then goes on to say that human creativity is no big deal and bots can be taught to replicate it. The narrator gives the example of Emily Howell, a bot that composes music. But, Emily is no Mozart. I didn’t hear anything that actually “moved” me while listening to its music.

My point is that, to spite what the video claims, there are human emotions, experiences, an essential human condition which you must relate to in order to produce truly compelling creative work. If programmers used machine learning to try and create a Shakespeare bot, feeding it everything from how to construct prose in iambic pentameter, to what makes for a good plot line, to knowledge of every cultural nuance of the world of Elizabethan England; I still don’t think the bot would come up with anything as compelling as Othello or even All’s Well That Ends Well (my least favorite work by old Will Shakes). If we created a game design bot and told it to design an unique puzzle game using optical illusions, what do you think the chances are that it would come up with something as beautiful and with as much mystique as Monument Valley?

Did you ever watch Star Trek: The Next Generation (TNG)? If you haven’t, then for this reference you just need to know that there is a major protagonist named Data (seriously do you live under a rock), an Android, basically a sophisticated robot that appears human. One of the reoccurring themes of the show concerns Data’s efforts to become “more human.”  Data was created without emotions, and while he is superior to humans in many ways (like strength and his ability to do complex math in his head) he can never really be one with the species he was designed to emulate because of this lack. Often Data’s search to understand humanity takes the form of creative endeavors like painting or acting. While he might give an adequate performance of Ebenezer Scrooge ultimately it is derivative, stemming from his studies of  “every known acting master.” Data admits that he does not “effectively convey the fear called for in the story.” It is pointed out to him that he has never known fear, but he should be able to approximate it as an acute observer of human behavior. Data criticizes this idea because it isn’t an appropriate basis for an effective performance. Data says he is a proponent of method acting, a technique where actors, in themselves, cultivate the thoughts and feelings of their characters. The idea of Data as a method actor is oxymoronic.

A computer, even one that emulates emotions, does not understand emotions or have experiences and associated personal interpretations that makes a person’s creative endeavors unique while still compelling to others. For this reason I believe that, as creatives, game designers should have as much life experience as possible. Understanding the exhilaration one feels on the half pipe at a snow park will make you better able to design an enjoyable snowboarding game or even a racing game. But it goes beyond this. Life experience will make you better at working with others. If you’ve never spent time with Chinese or Indians then you probably know little of their cultures. When you have to work with people from these countries you may find it hard to communicate. Try referencing Alice in Wonderland and watch the blank looks propagate. Within any culture there are things we take for granted that “everybody knows.” We reinforce this belief through media. An American produced television show will typically depict the US as the center of the world, the same for British TV. Have you ever watched anime? What country is the world’s hub of culture, trade, and diplomatic clout in those shows? Japan. By immersing yourself in other cultures you become better able to understand people, not just your people.

If you never try to have life experiences. If you aren’t adventurous and tend to eat the same few meals you’re comfortable with, never tried yoga, never visited a foreign country, watch only reality television, and have never spent a night camping in the middle of a forest (sans RV), then you are missing out. You don’t have to do those things, exactly, but without life experience, ultimately, you aren’t much better than a creative computer program. All your knowledge and understanding of humanity is mostly theoretical or stems from a very narrow perception of the world. In the end we are designing for other people. The person who designs for themselves is lost, tending not to make great games. The more life experience you have the better you will be at understanding the needs and motivations of those you are designing for. While books on Game Design are great, I’d prefer to work with the designer who has never cracked one but who has spent three months working on a shipping vessel in the Atlantic. But, that’s just how I feel.