If comments are welcome why do you delete them?

Your survey is skewed, or do you consider not giving a person an option to say "none of those answers is my answer" something other than forcing them into a decision that they don't choose?

Can you be a little more specific? Your past comment was deleted because it was vague, unconstructive, hostile and came across like a troll post more than anything else.

Forced choices do not result in a skew because they do not inherently cause respondents to answer one way more likely than another. They are also a reflection of many real-world situations where you are forced to make a choice (such as whether you will vote or not - there is no 65% "yes" response). And finally, they reinforce the intent of the question. For example in the "flight-vs-invis" question, you may say you want neither, but the point of the question is - if you HAD to choose, which would you choose. It's also the case that most questions in the survey are not forced choice, but provide a spectrum around the intended construct of the question.

Constructive feedback is always welcome. Trolls and spam are routinely deleted.

If I wanted to troll I would have said something far different. Just because a person exhibits frustration does not automatically mean they're being unreasonable.

In any case, I understand your reasoning behind your belief that the lack of an "N/A" results in "something related" to real life decision-making... but, I really want you to consider what the data would reflect if you had 20-30% of your respondents choosing an N/A option.

What I mean by that is - and I've had this discussion countless times on various gaming forums - is that your studies do not accurately reflect the "average" gaming mindset when your data does not take into account that the options you give may not be the motivating reasons for a person to perform "x" action in a game.

If you have 7,000 people respond to a question with the choices you present - there's nothing wrong with the raw data, of course, but the extrapolation of that data becomes innaccurate. Take for example, your first question in your current survey... Your answers require me to select whether or not I find that sort of "attack" to be appealing. It assumes that I can make an educated response as to whether or not I find such to be appealing - but if I only played TSO, for instance, your question would be completely irrelevant since there is no such thing in that game.

As it so happens, I do know what those attacks are, and I do have a response to give based on that knowledge - but if you have 20% of your population who just "pick" an answer... I wonder just how telling the data is going to be for that question - or ones that are similar.

If you look at the second set of questions (the question on page 2) - your second question on that page leaves out something that may very well change the way a person could answer that question... I know that *for me*, that which you ask is only important to me depending upon whether or not such an item is going to relate to my game by making my character better; if it's going to be just a "collectible" with no special properties; if it will allow my character access to specific content; etc.

Or take your "Good vs. Evil" character preference - my answer to that question is not static. My answer to that question rests solely in the "it depends on the game concept" answer that doesn't exist.

I very much respect the work you do, and the effort you and your colleagues put into this work is tremendous. However, when the majority of people with whom I associate in the games I play tell me either "Nick Yee's data PROVES that "x" is true" or "I won't take his surveys because his answers aren't indicative of how I want to answer the question", I get very frustrated.

With regards to the TSO example, that's precisely why only MMORPG players are targeted for respondents. I do not target TSO / There.com players at all. The survey questions are framed with a specific audience in mind - the audience that is targeted as respondents.

Flexibility and precision are trade-offs in this methodology. To capture diversity and depth, you'd have to use open-ended questions (which I also do), but it becomes hard to quantify open-ended answers and categorize them.

That's actually exactly why I'm collecting both close and open ended answers.

But let me give you a better answer to your sense that 20% of people just give a bogus answer because of lack of NA option and that this severely skews data. I'm most interested with the data on the level of age, gender and personality differences. Say older players prefer "invisibility" while younger players prefer "flight", and this is statistically significant. If 20% of answers were bogus, that would make this finding even more impressive because it's saying that the effect is so strong that even bogus answers didn't wash it out (because in those answers, the choice wouldn't relate to age). And the finding can't be an artifact of the bogus answers because they are the minority of answers, and bogus answers are presumed to be random. My point is that if you believed that 20% of answers are bogus, you are in fact claiming that my findings are much more impressive and significant than they are.

In regards to this: "I do not target TSO / There.com players at all."

I know TSO players who have taken your survey. I know MUD players who have taken your surveys. Regardless of their own inability to realize you aren't targeting them, and knowing that they didn't all specify the name of the game which they actually do play - I assume you are not considering answers in your publication from people who noted TSO as a game they play? It's unfortunate that the only way to 'guarantee' you are getting your target audience's responses is to depend upon their uncontrollable decision to specifically tell you that they belong to the unwanted demographic using whatever method you've given them to do that.

In regards to this: "but it becomes hard to quantify open-ended answers and categorize them." I absolutely agree with that, but I'm not convinced that "Other - please explain" is the only way to account for the flexibility trade-off you mention. I myself am a "gamer". I own more game titles than a sane person should, but even I - a member of your target audience - find it problematic to take your surveys. I'll give you an example. My favorite game to play is one where there aren't any "classes", but I do often play games that have classes. So, I have different characters who have different abilities and skill-sets. Which one I play depends heavily upon what it is I am trying to accomplish at the time I play as well as which game I am playing. Your questions relating to "How appealing are the following abilities when you create or "spec" characters?" is a difficult series to answer because the importance of each thing you list is relative to which character I am playing and in which game I happen to be playing.

To ask me how important it is to spec a character with "ranged attacks" for instance, is completely irrelevant to me or completely important to me depending upon which title I'm playing. I'm not sure what it was you were looking for in that series - but the only logical way for me to answer your entire series is "that depends on the character, the game, and how many characters I can have on my account.

When I played SWG and CoH, I considered having a ranged attack to be essential because of the game mechanics. When I play UO and DAoC, however, that preference is totally dependent upon what I am wanting to do when I log into the game. So, my answers to your survey would have been less arbitrary (because, trust me, my answers were wholly arbitrary) if the questions I'd been asked were qualified with "In the game which you enjoy playing the most, if you could only have one character, how important would "x" be to you?"... etc. Further still... it would probably be interesting to know that the game I play most often is not the game I enjoy playing most often. :D

In regards to this: "My point is that if you believed that 20% of answers are bogus, you are in fact claiming that my findings are much more impressive and significant than they are."

Oh! So you agree with me! I hope you realize that your statement here is only true if you are testing for and discounting the random answers - and as you have no way of tracking the "randomness" in your targeted demographic, such could then never be the case. Your findings, therefore, can only be as significant as the quality of the data - and as long as you have no way to account for the randomness, your data suffers from the quality it otherwise could have.

With regard to: "I hope you realize that your statement here is only true if you are testing for and discounting the random answers - and as you have no way of tracking the "randomness" in your targeted demographic, such could then never be the case."

No, my point is that any statistically significant finding is probably driven by a real difference, and that if a significant result emerges despite any presumed X% of bogus answers, then that implies those differences would only have been stronger if I had a cleaner sample. The finding itself is still valid.

In other words, bogus anwers only affect the statistical magnitude but not direction of the finding. Arguing that there is an X% of bogus answers actually STRENGTHENS my findings because you're saying that even an X% of bogus DIDN'T wash out the effect. If I find a signficant effect at p

But ultimately, I don't think there are methodologies, questions, or response options that will make everyone happy. Again, that's why I also ask open-ended questions.

If you'd like to continue this discussion, let's switch to email. It is becoming like a typical message board debate where we're repeating ourselves, talking past each other, and stops being interesting to anyone else. So if you'd like to continue, let's continue over email.

There's no need to go to email. I have no desire to make this some enormous debate really. I simply wanted to point something out to you - and it seems you have glossed over the last point I was making. You certainly have the right to do think about your study in the terms you have set out. Personally, I'm amused that you can say if it is true there are "x%" providing bogus answers that your findings are even MORE significant - that you don't seem to get what you are also saying - that without a way to know if there are bogus answers that your results are possibly less significant than they would be if you were measuring for those bogus answers.

Carry on. I'll leave your site now.

Hmmm. NY, your comment about randomly chosen answers occasionally enhancing your results is somewhat misleading. I understand of course that if you see a correlation *despite* some percentage of random answers, that other correlations (age/gender/etc) are strengthened by this, but this presupposed that a N/A desire by the testee results in a random choice, and not, say, a disposition towards the first choice... You will need to either randomize the order of the possible answers, or give different tests (and a control) to each survey taker in order to eliminate this problem. Nice work, overall, by-the-way.

Tamooj - With regards to the "disposition towards the first choice" - that's why the gender / age / personality differences are so compelling, because they did not all pick the first choice, but according to another variable such as gender. These differences suggest there is no overall disposition towards the first choice, and even if there were, it suggests that the actual difference is stronger than this predisposition.

remove the "wish" detail in the initial question to or not to enter the survey.

satryc - What's the reasoning behind this? It's actually set and approved by the IRB, so it's harder to change than one might think. And does it really matter whether it's phrased as "wish" or "agree"?

Wow. First time I've ever seen a flame on a comment panel. @_@

I think you could have cut the whole discussion short if you had pointed out to the angry poster that "true" is not a statistically valid concept, therefore, anyone using your survey results (or any other survey results) to "prove" something is true, has misunderstood the purpose of conducting a survey in the first place.

"True" is an irrelevancy because truth assumes it applies to anyone, anytime, anywhere, thus it always returns a statistically invalid reponse. ^_^V

Peace, out!

I just wanted to say that your site has greatly inspired me. As an undergraduate student who is interested in conducting research projects of my own in the future, reading your research projects and papers have given me a great deal of insight and information both concerning the methodology and presentation of ideas, data, and interpretation in research.
Thanks very much for sharing your work online. Good luck!

Hey. Just making a comment, I took both versions of the survey and I am greatly interested in participating in your research. I have been jumping around free MMORPG’s for nearly five years and what I have been trying to explain to people, you have been gathered information on for via internet relationships in character and out of character, how lives are connected to the game, why we play the games, and what we like/dislike about them. Honestly, I am impressed, and I want to thank you. I hope to be able to participate as research goes on.

i read your first chapter, and i've read some comments. you will no doubt garner the support of mmorpg fans trying desperately to justify their addiction, but it's abundantly clear that anyone in their right mind who reads your report will see the poison of these games and most blatantly their effect on you. EQ, star wars galaxies, final fantasy XI... all are no more than vaccums which would have consumed my soul had i not walked away and returned to the real world. Unless you have a hideous disfigurement or are socially inept, i promise you that nights are better spent at bars or parties or just home with you and your girl than in some virtual cult. To anyone who thinks me wrong, please, for your own sake, go to your local library and take out Plato's The Republic, book VII. It contains the allegory of the cave, which should serve to explain your curent state of affairs.

It seems that what you are trying to do is show the positive aspects of mmorpg's, and the "serious social phenomina" which occur within them. I suggest that you abandon your approach and take a u-turn to the real story- Why the fuck do people spend hours upon hours in some virtual world of extremely limited possibilities when there is a perfectly good one OUTSIDE? Is it because they are shy, insecure, or do they just want to feel a part of something? Or is the world just a sad and lonely place? Does it scare them? Is EQ their only means of escape? Are they too pussy to face reality? nothing positive has come from mmorpg games. To hold them in the light like you are doing is morally wrong. these games are a trap, and they will ensnare others just like they did you.

Jonny - It's ironic that you ask me to explore the questions you bring up without realizing that those questions have already been answered in the book chapter. The phrase "problematic usage" is in the chapter subtitle and there is a section on addiction - is that what you meant by holding them "in the light"? My goal is not to justify MMORPGs but to show the incredible depth and complexity that many are willing to ignore because these are "just games".

The allegory of Plato's cave is appropriate, although I would question whether the real or virtual world is the cave. Can a world where we have freedom over our identities be more cave-like than the world where we are bound to a single identity?

In response to your first question, i feel that the mmorpg phenominon is potentially very dangerous. When you express your fascination with the social lives of avatars, you unintentionally glorify it, which is what i mean by "holding it in the light." By doing so, you stray from the topic at hand and come across as a writer/mmorpg player trying to justify his hobby. I don't deny that you have fulfilled your goal, i only suggest that you look deeper into what these games have become and question whether this virtual reality is a proper way to live one's life, which leads me to your next question.

Indeed, which world is the cave? Although it seems pretty obvious that the game is, one could just as well argue that someone who's never played a video game in his life is in a cave just as deep. None of us can provide evidence which distinguishes reality from irreality, real life from everquest- since who is to say that what one percieves as real is in fact fake. What if you were fed intravenusly and lived your whole life in front of a computer logged onto everquest- and saw nothing else. Instead of learning to speak, you would learn to type, and that would seem perfectly normal to you. If one day you were released into the "real" world, you would percieve it just as we percieve these mmorpg's- virtual relaity, role playing, fake. You might try and kill yourself to get back to your homepoint, but obviously that wouldn't work.

My point is that I can't answer your last question, however I can leave you with a thought. Human beings are equipped with a plate on our heads, which, when sunlight reaches it, makes us happy. It seems that we are meant to be outdoors, since it is how we are evidently designed to live.

Its hard to sit back and watch these comments fly, so I must speak.
How about this idea for starters… your sitting here blah blah blah about how terrible these games are because they are addictive. Well just a few days ago I was hearing about a girl addicted to your average over the counter medicine, Tylenol. So… how dangerous is that? Everyone knows limitations with medicines and that they are addictive, and yet some choose to deny that because well, they like the addiction. For MMORPG’s, or for any game for that matter, yes they are addictive, every single one of them, if you like them enough yes you can suck yourself into that little unsocial cave and not shower for a week because your almost to the next level. Well, personally, I –am- addicted to a MMORPG, but I can pull my head out of it to go on dates, hang out with friends, have my family hour and so on. I spend about 35 hours on the game, but I got myself into collage, I got myself a drivers license, I go out every weekend. Honestly the games aren’t the fault, they were developed by many people for the value of mild entertainment and to get them clothes food and four walls. They aren’t Satan as you seem to be describing them. Everyone is exactly where they want to be, and if that’s sucked into a virtual world, well, then that’s where they are. Another metaphor? A game is a like a gun. It itself is not dangerous, the person behind it is.

Watch a child grow up with the Computer as his friend and the child will have a very limited view on the TERM "REAL WORLD"... the kids are starting younger to play the games online and the fact that "older players" find some logic in the entire process is "justified".( they have a life ) the younger kids will learn the Values of the RPG's and the content within.

The survey will show if the player is in the RPG scene long enough the answers will be motivated by the AGE and not the Experience...

I don't see the correlation between MMORPGs and addiction. There's a difficult line drawn from compulsory behavior and behavior. I think you should study many things people do to pass time, and correlate those results with these (like skiing, or baseball, or study-making). I bet in doing so you would find that compulsory people exist, scattered throughout every caviat of life. Congratulations on completing your study; however, like so many studies before it, nothing gained and nothing lost. Perhaps creating studies to solidify one's idea as belief is, in itself, an addiction. Maybe we should be so blessed that things like MMORPGs exist at all.

And speaking to Jonny,

Condemming someone as morally wrong is difficult to do on an individual basis. To answer your question:

"Why the fuck do people spend hours upon hours in some virtual world of extremely limited possibilities when there is a perfectly good one OUTSIDE?"

I would answer with a question:

What is outside, to you? If that involves bars and nights with your girl, so be it. But deep down you know that these things are not universal. The same emotional tingle you get fraternizing or drinking a beer or discussing the future with your girl might be the same emotional tingle others get from MMORPGs.

Don't play an MMORPG if you don't like it. Don't be gay if that's not your cup of tea. Go outside, drink a beer. Murder someone if you feel you must. All these things are as real as the next.

Condemning people because they choose for various reasons to sink time into a game is contrary to allowing people to excersize free will.

Let me elaborate and also touch on the cave metaphor. How many men in the country drink beer and watch football every week? How many men drink beer and watch tv in general whatever it may be? Please tell me the difference between sinking time into this hobby vs. sinking time into gaming. It's all a matter of perspective really.

I suppose you could say that people could spend time reading and visiting the library but that once again is a matter of preference as much as drinking a beer at a bar and hanging out with sleezy or not so sleezy women. Once again a matter of preference. Personally I don't drink so bars aren't my scene, dance clubs maybe but once again personal preference. And there is nothing stopping me from playing my games, playing board games A&A with friends, and rolling to a club on the weekend.

For those who condemn people who play video games because they think in their minds eye that they are living in a cave your scrutiny of others lives through your shallow trough needs to be widened and allow for understanding to the desires and wants of other human beings who are simply enjoying themselves how they see fit.

end of discussion on that matter imo.

As far as mmorpgs having no relevant life skill carry over to r/l that is wholely subjective and bias if it is negative. I know that I learned plenty of skills while playing in several mmorpgs. Just to rattle a few organization of large events, conflict management, problem solving and writing skills. Specifically raids/guild events, flaming and forum moderation, guild diplomacy, and newspapers/emails/forum posts. These all translate into r/l skills which I use every day. In a mmorpg you are also free to experiment and practice people skills I could go on and on, I won't though.