1. Lesson 47: Kleenex Testing

      written by Ray Benefield
      Happy Sunday/Monday folks. If you've been living under a rock the first thing you may notice is that RP undergoing some visual changes. There are still some kinks in the layout, but atleast it is functional. Speaking of the layout I have promoted DavidJCobb as an admin for RP who will be focused on working with the layout and maybe even coding other awesome things for the site. Congratz on the new position Cobb. And maybe you don't know about our latest map Think Twice. I've been working with A 3 Legged Goat, Jonzorz 124, and Sven Nietzsche. All four of us have worked hard on Think Twice and the Public Beta was front paged by Urk over on BNet. A special thanks to you Urk if you are reading this, as per usual. ;) The Public Beta has given us LOADS of feedback and thanks to that the map has gone through tons of changes. I think one or two more weeks should give us time to fully finish up the map and give people a little bit more time to return with feedback. Before moving on to the lesson below you can find a link to the public beta download (6900+ downloads, almost 1K per day), the feedback thread, and the new layout issues thread in our forums. Thanks for all of your guys' support. Laterz peepz. ;)





      Next >>


      So I don’t need to tell you how important playtesting your map is. I cover that pretty well as a way to apply your design decisions and it is pretty self explanatory. A good majority of people already know that they need to test their map to ensure that it is at least playable and to find any flaws that they may not have seen otherwise. What most people might not think of is how important it is to not “soil” a player’s playtesting experience.


      Fresh fully clean


      First let me start by expressing how much more beneficial it is to you if your playtesters know as little about your map as physically possible before playing it. Why? Well because when players find your content to download somewhere there is a good chance that they know very little or even nothing about the map. If knowledge is power, then a player knowing nothing about your map is the worst possible scenario that a player can be in. Improving this worst possible scenario can improve the first and overll impressions that your map leaves on players. Attempting to share as little information as possible with your testers is known as Kleenex Testing. The term is created from the concept of never using a dirty Kleenex to blow your nose… it just isn’t right and you always want the most clean Kleenex that you can get your hands on. Kleenex testing allows you to replicate the actions and reactions of players that download your final creation while not know anything about it allowing you to study and adjust your map to improve this scenario.


      Sterilizing your environment


      Pure Kleenex testing is extremely difficult to pull off if you don’t have a reputation built already. Why? Because in order to get playtesters, advertising is needed. And to garner a good first impression you’ve got to use your skills and show off some beautiful static perspectives or films to make people want to invest their time in your map. Sometimes even doing that won’t get you playtesters. The best that you can do is to withhold as much information as physically possible. When you have new players playing your map, don’t tell them where to find the power weapons. Let them find it themselves. Also don’t pick up any important weapons so you can observe whether or not players are finding them. If you grab a power weapon then how do you know if new players will find it? You can even try not calling out anything and observing how players move around your map when un-influenced by more knowledgeable players. The worst thing you can do is explain everything about your map before playing it with new players. Because then at that point you have tainted their fresh experience. Keep your mouth shut and just watch.


      Studying the Kleenex


      Use Kleenex testing as a resource of valuable information. Watching players play your map can tell you a lot. Ask your players what they thought about the map as soon as you finish or during the session while the experience is fresh in their mind. Find out which of your incentives are working out and drawing attention. Ask if players had an easy time finding their way around the first time through. This is your chance to observe how your creation will do out in the wild when you release it to the public when you can’t be there. Use this technique wisely.


      Next >>

      9 comments:

      Schnitzel said...

      I'm not down with this one. Sorry.
      Here's how we run tests over at MLG: Load up the map, all players go to the same team, party lead lets players familiarize themselves with map, all power weapons and pointed out... Take about 5 minutes. End game, pick teams, playtest 2 to 3 times before moving on to next map. Rinse/Repeat.
      Following that same dynamic is also a very solid way to run your own test lobbies. Remember though, its for the players to familiarize themselves with the map, not complain or make suggestions (although this may be inevitable)... Collect feed back after the game ends.
      Also, make sure you go to theater and watch the games played. Watch spawns, player movement, high kill zones and low trafficked zones. Try to do this with as many games as possible.

      As far as getting tests being a hard accomplishment to pull off, I cry FALSE! Reputation and advertising have nothing to do with it. The best way is through networking. Reach out to as many forgers as possible, especially forgers who have already made a name for themselves. Many of us are more than willing to help test maps, granted of course the design looks like it has potential... All you have to do is ask for help and most of us are willing to come lend a hand. ;)

      GodlyPerfection said...

      Schnitzel, what you are talking about is the exact opposite of this, veteran testing. Veteran testing is where you test out balance through ensuring that all players know of as many details about the map as possible. MLG as a community always bases map selection off of the competitive value of a map... its lasting impression. MLG maps do not need to conform to first impressions as much as standard maps because of exactly what you pointed out. Your guys' tests allow players to familiarize themselves with the map first. For standard maps, people do not do this... they just load up the map and play. Like I said... everything you refer to is covered in the next lesson. Kleenex Testing is all about improving First Impressions... Veteran Testing is all about improving Lasting Impressions.

      MLG maps don't have to consider eye catching or anything of the like for their power weapons because it is assumed that players should already know where they are as an MLG player. If you don't then it is your fault for not studying the map. Standard maps and regular casual players do not follow the same guidelines as MLG does, even though the MLG way is much better for gameplay purposes. People don't have that kind of time so you have to build to make their first time on the map as best as possible. You have to build to ensure that players can find every weapon easily and effectively. MLG players are competitive enough that they have the time to familiarize themselves with maps. Casual players do not have the time... they have real life to attend to. ;) lol... you see what I did there?

      Networking requires advertising and reputation to be successful. And don't lie to me when you say that all you have to do is ask for help and most of us are willing to come lend a hand. Because you aren't even willing to play what you consider "bad" gametypes that are casual and fun. If you can't even do that, then there is no way you are just going to play a bunch of maps by a bunch of randoms. If you were asked by thousands of people to test their maps, you would only take the time to look at a few and you would pick based on their reputation that you are aware of, and how they advertise themselves to you or their map. As much as I want to help every person that asks me to test their maps, I cannot. I have to pick and choose because I do not have all of the time in the world so Reputation and Advertising play a huge role in who I decide to help.

      Schnitzel said...

      I really wouldn't say its "veteran testing", as this is the way I've been testing since I started forging. I built a vast network of friends who forge, and then slowly worked my way up to establishing connections with more well known forgers. It was a lot of work, but it was done through networking.
      As for the MLG tests, its not this way because its MLG but because the only way to get truly reliable results is to eliminate confounding factors that unbalance the teams (such as map knowledge). If we didn't give players the chance prior to the game, we'd have to play an additional game to see how it plays after every one has overcome the learning curve. Its really a matter of time management.

      Oh Godly, you know I hate when people act like MLG maps follow different design rules than normal maps. Of course "eye catching" isn't necessary for power weapon placement, or at least in regards to having some fancy weapon holder or aesthetic of the likes, but it should be "eye catching" in the manner that the power weapons are in a blatantly obvious and easy to find location. This ties in to orientation as with any map, MLG or not, players should be able to find their way around the map easily.

      I'll say networking doesn't require advertising, at least not in the manner that you advertise, in order to be successful. I understand some level of involvement with communities (such as FH or XF for example) can be considered advertisement, but I consider it part of inter-personal relations that comes into play with networking.

      I'm not lying when I say that reaching out a hand for help will be accepted. I do not accept random invites to play in other lobbies where I don't know what is to be expected, nor do I accept any invites if I'm busy with something else. Same goes for most experienced forgers. When I say that we're willing to help if asked, I mean that by contacting us through another medium (such as a forum, AIM or xbl msg), we'll most likely be willing to come into forge (granted there is time available) to look over a map and give some insight on it. Throwing on a map without knowing what it is runs the risk of destroying custom lobbies (unless you have an EXTREMELY dedicated group that has nothing better to do) as if the map is bad, you risk rage-quiters. Its not worth the hassle of re-populating a solid customs lobby because of one bad map. Furthermore, it damages the reputation of the party host as other players will expect bad maps again in the future.

      There's a method to my madness, and it all ties into performance and quality control.

      To counter your final paragraph, I'll hit on orientation again. It should not be necessary to familiarize the players with the map prior to playing as it should be easy for them to figure it out themselves. Unfortunately, for more reliable testing, it is a necessity.

      Captainpineapple44 said...

      In my opinion you guys are both right. Kleenex testing is the best way to test first impressions. Veteran testing is a good way to test everything else.

      Jonzorz_124 said...

      Schnitzel I still think that you are missing the point of this method of testing. What you describe in your MLG tests, as Godly has called it "Veteran Testing," involves creating the most balanced gameplay environment possible. This is done so that you can get a true feeling of the way your map plays once people have learned your map in its entirety a.k.a. when the players have become quote "veterans" of the map.

      The purpose of Kleenex Testing is to create the most sterile, or uninfluenced environment possible. This is done so that the creator can test how "easy" or "hard" it can be for players to find the power weapons or tactical positions within the play space.

      These two testing methods, Kleenex and Veteran, along with Mass Out Testing are testing methods that go hand in hand. Every map should undergo this type of testing regiment, but unfortunately it is not a perfect world and we do not have unlimited time or testers to perform these tests. As such, designers have to cut corners and while it is unfortunate, it is the way it is.

      Schnitzel said...

      Kleenex testing is good for judging first impressions, but only that. Its rather silly when you think about it as it doesn't accurately test the map itself... It simple gauges players opinion of the map based off of a blind run. Kleenex testing might have some value for testing how easily players orientate themselves with a map, or how the aesthetics or frame rate fare, but it is completely useless in judging the map itself as the players have not yet learned how to play the map.
      Think of it this way, did you KNOW how to play any of the maps that shipped with Reach proficiently upon the fist time? Or did it take you time to learn and master these maps?
      The point is, Kleenex testing has its use... But its extremely insignificant in the broader scope of map testing.

      GodlyPerfection said...

      But you are testing the map itself. Part of the map is how quickly it teaches its players about the important parts of the map. The quicker a player gets up to speed, the faster a player can get a good grasp of how they should play on a map. This is important for custom maps since there is no guarantee that people will play the map more than once. If you are testing for a map that is guaranteed to go into matchmaking then Kleenex Testing is useless. But unless you are Bungie, you cannot force players to play your map more than once... or even for one full play through. That is why it is important to have your map to teach your players the map as fast as possible so that they will be able to enjoy the map to its fullest the first time around.

      I don't know why I'm telling you this because all of this is covered in lessons 1 and 2. We didn't need to know how to play the maps that shipped with reach because we are guaranteed to play them multiple times in matchmaking so eventually we will learn them. Having a good first impression allows you to ensure that players enjoy the map on their first time around and want to play it a second time around. If players don't enjoy your map the first time, why the fuck would they play it again? And if they don't play it again then what is the point of all of the testing you put into balancing? Good first impression can be key in helping you advertise your map... the more people that like it, the higher chance that they will tell their friends. This concept is known as Virality, a currently new concept in the world of marketing that has just started to come to light thanks to things like Youtube and Facebook. That's actually one of the future lessons.

      Whether or not you think it is important in the grand scheme of things is up to you. This site is called Reaching Perfection, and you can only reach perfection if you take every little detail into account and learn how you can make that detail better than it is. So while it may seem insignificant, as you said it does have its use so if you want to reach perfection then learning how to make that detail better can overall make your map better. If you don't want to improve this aspect then you are missing out in a key area where you can improve your map. Orientation is a very important subject in level design. Kleenex testing can help tell you if the things you use for Orientation are working or not. You can't test that with veterans because they already know the map... you HAVE to test that with Kleenex Testing.

      Schnitzel said...

      Godly, I'm done entertaining this argument with you. You really need to work on humility man. Your not always going to be right ya know.

      Rashad1595 said...

      "All you have to do is ask for help and most of us are willing to come lend a hand. "
      Despite the fact that, relatively speaking, I am a novice at forging, I'd be happy if a few veterans could check out a map of mine and provide feedback. My gamertag is Knives 69 you (sorry about that)