Sunday 17 March 2013

Sunday Sermon: Where was I?

    
    


I've obviously been a little reticent of late (I'm sure you've all noticed). My productivity levels have certainly been 'low' when it comes to output on this here Blog. I've not been sitting in a darkened room rocking gently backwards and forwards though... OK... so maybe I have done a little bit of that, but mainly after attempting to do things like wallpapering and other DIY tasks. Have any one you ever attempted to lay massively uneven natural stone tiles on a floor that resembles the Himalayas? Well I have, and let me tell you, I have a new found respect for tilers. Quite often we look at the stuff other people do from the outside looking in and we think 'I could do that'. Sometimes we're right, we almost certainly could do that, but I think we all too often like to underestimate the difficulties associated with other peoples work.

As a researcher / consultant I often here all the jokes about giving consultants watches so they can tell you the time. They bring a wry smile from me, and I understand some of the hostility towards 'outsiders' coming in to tell people either A) what they already know or B) how to do things. It's a difficult line to tread and get right. Quite often if you are asked to do something for somebody else the reasons are very similar. The most common reason I get asked to do something is because there is a lack of resources at an organisation to get a short term piece of work over the line. This can be the most challenging environment, mainly because the organisation will almost certainly have people who are capable of doing the task themselves... it's just their day job is getting in the way. In these situations I find it is best to acknowledge that fact, and work with those people, keep them informed and pick their brains.

The other reason people call on you for help is difficult for totally different reasons. Quite often you'll be brought in to do a piece of work because an organisation hasn't got a clue how to set about doing the task themselves. Here the first task is trying to work out what the ultimate end goal is. Often somebody will think they know what they want, but actually they don't. They just think that a piece of research might just answer all the questions they have. Convincing people to do something different to what they think they want and need can be tricky, but ultimately if you are unable to guide them they end up disappointed and unsatisfied with your work, even if it exactly what they asked for. So although people might think research and analysis is easy, I can assure you it is not.

Often I'll read statistics and cock an eyebrow at how little they actually tell us about a phenomenon. My first thought always used to be 'shoddy workmanship'. Now, I know a little bit more about the truth. The reason some of the statistics you see in articles are as simplistic as they are is because that's what most people will understand. I have seen many people's eyes glaze over at the mention of inferential statistics, or multivariate analysis. I've gotten better at talking to people in a language and a way they can comprehend easily, and the narrative that I try to tell with my work is always now tailored to the audience. Now, I can hear some of you asking 'what the hell has this got to do with wargames and boardgames?'... well if you stick with me I might be able to shed some light on that for you.

This wonderful graphic tells us absolutely nothing, yet games news outlets have run with it and claimed it's massively insightful. Really? What's the sample size, methodology and more importantly where are the comparative statistics?

I have often been asked to play test products for people down my long and distinguished career as a games nerd. Perhaps it is my keen analytical mind that has made me a popular 'go to' play tester. Or perhaps it's my breadth of experience with games of all types. Or perhaps it's because I'm able to articulate my thoughts in a way that is enlightening, or helpful (we are rarely both at the same time). It's actually more likely an unholy combination of all of those things. Lately though I have had more input than I normally would as a straight up and down play tester with a number of products. I've been involved with solving design issues and sorting out statistical sequences for gameplay mechanics. I've even helped with discerning whether or not there's a market for certain types of product. Simply put it hasn't been easy. Yeah sure it has been fun at times, and really educational, but it has proved far tougher than I thought it would be.

I've relished the challenge though, and it has given me a totally different perspective on the production of games, and how bloody difficult it can be to produce something that does what it is supposed to, while also being entertaining. There's a lot of work that goes into the whole design process, and quite often I think we as gamers don't really appreciate the complexity of the task at hand. I thought I did, and honestly I didn't. Now I'm not suggesting we cut games designers more slack when they screw things up. Nope. I'm just saying that unless you've walked in their shoes perhaps we shouldn't be so quick to judge people. Sure judge their product, but that isn't necessarily the same thing as judging them, which sadly there has been a lot of in our hobby. I've been keeping up with news within the industry, and I've been reading Blogs... and I've noticed an upswing in whinging.

Now this is for two main reasons. Firstly it is experienced gamers, who have significant levels of knowledge regarding products and know what they're talking about, and they can now see that certain games have intrinsic flaws with their design. Well, from their perspective. They could have been 'done' better is often the claim. Well yeah maybe, but they might be missing the bigger picture, the product might not be designed for them anymore. The second complaint is from people who don't know what they actually want from a game. They think they do, but when they get it, well it turns out it wasn't what they wanted after all. I guess the similarity in the root cause of a number of gripes, to the gripes that researchers often have to put up with struck a chord with me. Maybe if we as consumers spent a little bit more time understanding what it is we actually want from products, we'd spend far less time being frustrated at what we all buy. Peace out!

26 comments:

  1. Amen, brother! Especially to the comment about veterans complaining about products that might not be designed with them in mind. There's a surprising amount of inertia among old gamers concerning their game of choice, and I'm often struck by the fact that butterfly gamers like me are often happier with the games we play than long-term gamers, who whinge endlessly but never look outside their box to see what else is out there.

    Incidentally, I was interested to hear of your consulting as a statistician for a game company, after hearing Craig Gallant of the D6G podcast and writer for Wild West Exodus mention their employing a statistician. Is this coincidence?

    ReplyDelete
    Replies
    1. The veterans complaining thing has started to really get on my nerves a bit. It normally takes a significantly long time before I lose my patients with people. I have a pretty long fuse and I'm normally pretty placid, calm and understanding, not quite as calm a Hindu cow mind you, but not far off. So the fact that it's finally starting to get to me tells me that it must have either A) being going on for a long time, or B) there's just a megacrapton of complaining going on.

      We're not talking about constructive criticism or feedback here, we're talking whinging at a ridiculous level about anything and everything and not realising that if they find something that disagreeable they should just move on. I play lots of different types of games, be they war, board, card or even computer games. If one starts to annoy or grind my gears... I move on to something else. We are supposed to be having fun when we do these things. But often it seems like people are involved in the hobby just to get angry and whinge.

      Maybe I'm in the wrong and I just don't get it, but I would have thought for ones own mental health it might be advisable not to continually expose yourself voluntarily to something that makes you unhappy and angry. Maybe as Minitrol says I'm mad after all.

      PS. I know conspiracy theorists will have a field day, but I can assure you that I have had no input with Wild West Exodus, or spoken with Craig Gallant. A lot of designers are starting to come round to the ways of statistics though. We can help work through a lot of theoretical situations and outcomes of mechanics changes in a very short space of time and show their probable outcome.

      Delete
    2. I agree. Most gamers who stays with their game through multible editions of their game seems to smile less when gaming, than people who switch between systems "a lot".

      Delete
    3. I think that's with most things in life... well it is for me. Variety, spice and all that jazz. ;)

      Delete
  2. It's insane how many hours of work get thrown in the bin because the conclusion from a professional hired to answer a question, isn't the one that top management had hoped to hear.

    It's too bad that being able to seperate an idea from a person can be as difficult as it is necessary. Both with criticism that you give as with criticism that you recieve.

    ReplyDelete
    Replies
    1. Yes, I've become accustomed to the "but that's not what we wanted it to say" people. I don't work with them anymore. There's no point. They'll just come back to you angry when the lie you told to make them happy turns out to be a lie. So short term you get the work and the money, long term you damage your reputation. I recently had someone who shouted at me because my report "didn't f***ing back them up" come and tell me my report saved their company a lot of money and they were sorry they went off of the deep end at me. You take the rough with the smooth, I've been right way more times than I'm wrong and I'm happy with my success ratio. Doesn't mean I'm immune to the odd blunder though, so I'm always willing to listen to counter arguments and learn. I've received a fair bit of criticism over the years for my work, it's part of the territory as a researcher. People expect you to say the things they want to hear because nobody thinks they're wrong, or perhaps that things aren't going as well as they could be. Then some seemingly smug git turns up with their graphs and stats and tells them they're way off target. It's not an easy message to sell at times, and the 'criticism' you receive can be quite harsh. You just have to learn not to take any of it personally... even if they get personal.

      Delete
  3. Hmm I was always terrible at statistics in school. I always like the joke "47.5% (just insert any number really) of all statistics are completely made up".

    Wall papering and tiling? I'm surprised you're still sane.

    ReplyDelete
    Replies
    1. @Sean all statistics are made up. Doesn't mean they're not right. The thing with statistics is that they are a social construct, and without context narrative or a compelling explanation they are meaningless. Without showing that the figures have been collected from a representative sample, without showing how they were defined and collected, without having comparative statistics and fully explaining the phenomenon stats are just meaningless numbers on a page. Tiling is like the worst form of jigsaw puzzle, that you have to live with. Your shoddy workmanship mocks you every time you look at it. Some of the wallpaper I put up was stupidly expensive as well, and had a 100cm drop. I was literally throwing money away... stupid hand printed wallpaper.

      @Minitrol, I most definitely am not sane!!! I have never once made claims to the contrary. I've been utterly bonkers for years. I think it goes with the territory of being a gamer.

      Delete
  4. I have been involved with a number of different game designs over the years, especially railroad games. I have found that trying to point out the potential flaws to a game designer is the most difficult thing I have ever done and it seems that no amount of empirical evidence will satisfy them that there really is a problem. And then when the game is published and the flaw is still evident and it rapidly becomes apparent that it is really a flaw they go off the deep end claiming that you and the other play testers deliberately concealed the information from them. Thank goodness for email strings!

    ReplyDelete
    Replies
    1. Hi Kris, you're right. There are some games designers who just don't want to listen, no matter how you broach the subject or present the information. That's absolutely fine though. I've never had a problem with a designer saying "thanks for the input but this is my baby, my magnum opus and I think I want to do it this way". That's cool and I quite like designers who have a strong sense of what their product is, and where they want to go with it. In many respects they're easier to work with because you know you are there to support them and their vision. However, the guys who then come back and blame you when you explicitly expressed concerns with regards to some mechanics or points... well that drives me bonkers.

      As you say that can be really difficult to take. When stats, play testing and everything else says that something is a mistake, and more importantly won't give the design team what they want and they go with it... biting your tongue can be hard to do. then when they throw the resulting mess back in your face it can be a bit much. I was in that situation with a game in the early noughties and I decided to step away from it and refused to work with them again until they apologised. They never did and now they're bankrupt. not that they're bankrupt because they didn't listen to me, they didn't listen to anybody!!!

      Equally though I had a really good experience with a designer recently who was willing to listen to every piece of criticism and suggestions. He took on board my knowledge of stats and tweaked a number of little things with mission design and positioning of objectives on the games tiles. It was really gratifying to work with somebody who actually communicated effectively what they were after, and what they wanted to achieve with every mission and aspect of the game. I might not have agreed with every goal they wanted to achieve, or the direction they wanted to take every aspect of the game, but the fact they knew what they wanted and could tell me precisely meant that I was able to support them as best I could in achieving their goals. That was really enjoyable and very gratifying.

      Delete
  5. Agreed. I think that Google and Wikipedia have made many people think that they know a lot more than they do about everything. Those technologies make research seem easy, when actually in a lot of ways it is now harder to do quality research because there is so much more crap to sort through. Misapprehensions spread like wildfire through social media, like old wives' tales. There is also a common misconception that anything you would want to know is out there somewhere, it's just a question of finding it. This is, obviously, not true. But if you believe it is then you can easily come to believe that anything you can't easily find out mustn't be worth knowing.

    Where I'm going with this is that people, especially online where it's safe, have developed a disturbing tendency not to listen to experts or give them the benefit of the doubt. And games designers are experts. It doesn't matter how much of your spare time you think about and play games, it's not as much as someone who does it for a living.

    People thinking that they can do other people's jobs really gets my goat. My partner used to work in a scientific research facility and her colleagues were THE WORST I have ever seen for this. They seemed to think being a scientist meant they knew everything about everything. I once sat through a BBQ with gritted teeth while a bunch of them whinged about how they could do a much better job at catching speeding drivers and generating revenue than the poor simple-minded police. Really? you think you can catch criminals better than the police? I'd love to see that, maybe you could hit them with your PhD in plant phenomics.

    ReplyDelete
    Replies
    1. James there is nothing more irritating than somebody claiming they could do somebody else's job better. Especially when that job is a 'profession' and they themselves have no experience of it. I too have heard the "I could do a better job than the rozzers" line. It winds me up no end actually. Especially having worked with them on producing crime maps and all sorts and hearing why they can't just go and do there work in certain 'ways' because it would impinge on all our civil liberties.

      I also get massively annoyed at the criticism nurse receive, my sister is a nurse and has been spat at while at work, assaulted and abused. Yet she still tries her hardest to smile and care while pulling 12 hour shifts some days... but the amount of people who could do a better more 'caring' job than nurses astounds me... and all for the shitty pay they receive in the UK too!!! I'm amazed we aren't awash with awesomely happy nurse curing every disease known to man.

      I'm used to people telling me as a researcher/analyst that I don't know what I'm talking about. Funnily enough months latter when my predictions come true they always seem to come back for more of my 'piss poor' research. I also think part of the problem is the level of further education in our society now. Fundamentally we all think we're well educated and therefore qualified to speak with clout on many topics. Put bluntly we aren't. I think you're also spot on with Wiki.

      Someone once told me I was using the wrong statistics and that I should used a T-Test instead of an ANOVA. Rather than get annoyed I asked them why they thought that was the case, they shrugged their shoulders and said they'd read it was the best test on Wiki. Hmm, I replied, best test for what? I dunno they responded, but I'm your boss and I want you to use that. Honestly I thought my head was going to explode... I did the work and it told us nothing. I was then asked why I'd done it by the same manager who'd told me to do it. You learn to smile and get on with these things, safe in the knowledge that at least you know what you're doing.

      Sometimes.

      :P

      Delete
  6. Welcome back...we missed you.

    ReplyDelete
    Replies
    1. You're the sole survivor of my Hobby Curse: Every time I find cool blog and bookmark it, it goes under. From The Warp, Warp Signal, Game Over, and a scattering of "minor" ones.

      The force is strong with you.

      Delete
    2. Well I should hope so... I'm a Sith Lord... erm... I mean Jedi Knight!!!

      Move along, nothing to see here.

      *waves hand*

      Delete
  7. I'm a software engineer by trade, with a focus on interface design, so I understand the challenges of trying to craft something which will receive a necessarily subjective response. I find a lot of similarities between games design and software in that both seem to be populated by people who are dead sure that they know better than the designer what it should be. That's obnoxious, and it's exhausting to deal with all the time.

    But on the flip side, designers need to be open to the idea that they're not perfect, and that they're not designing just for them. You see this in interface design a lot, especially with open source software. An otherwise-good product is made unusable and horrible because of an interface which is designed for engineers (who aren't the target user) or, worse, for the developer themselves. One of the biggest compliments I ever got came from a colleague when I left my last job, who said "I really liked working with you, when someone pointed out a problem you'd actually fix it."

    It's a hard balance to strike, and in the gaming world the abundance of overwrought positions (both supporting and attacking) certainly don't make it any easier.

    ReplyDelete
    Replies
    1. Buhallin, much of what you say is true for researchers too. I'm always trying to find new ways of representing data in a visual form that's easy to understand and digest. I might get what an output of 0.00346 means from a piece of statistics, but you can bet your bottom dollar most people won't. Graphs and histograms also confuse people far more than we like to admit.

      I'm always willing to listen to feedback on my work. Primarily because I want people to find what I produce useful. For it to be useful it must first be easy to understand. Having people look at things and say "what the hell does that mean" is actually incredibly useful feedback for someone like me. It tells me that what I assume to be a clear message is actually indecipherable to many.

      It can be hard to strike a balance as you say between actually conveying a message as accurately and as precisely as possible, and conveying a message that people can comprehend and find useful. As always buddy thanks for your thoughts.

      Delete
  8. "I also think part of the problem is the level of further education in our society now. Fundamentally we all think we're well educated and therefore qualified to speak with clout on many topics. Put bluntly we aren't."

    Yes. A thousand times yes.

    Just about everyone is highly educated as you say, but in some sort of specialist field. Then they go into an even more specialized job. For some reason this has had the bizarre effect of causing many people to think they know about everything else outside their specialization, or a least that they can find out with a few mouse clicks. It's madness.

    You know those futurists who predict that in 50 years everyone's IQ will be double what it is today? Well I think we have some growing pains to get through first in terms of handling the immense about of info at our fingertips in a discerning and humble way.

    The next time someone pulls that wiki thing on you maybe you should excuse yourself for ten minutes, edit the wiki page to support your advice and then tell hem to read it again. Maybe they'll get the message!

    ReplyDelete
    Replies
    1. The other qualified statements I hate James are "as a parent", "as a mother" and "as a father". So what? Your ability to procreate makes you qualified to speak on every topic now? Good lord it winds me up!!!

      The chap who actually pulled the wiki thing on me is a gentlemen I actually had a great deal of time for, and a HUGE amount of respect for. He was a manager at an organisation I worked for. He was an old school professional finance expert, and he had no idea what I did for a living. Or indeed why the big boss thought I was needed and head hunted me. That man had forgotten more than I'll ever know, and I'm immensely grateful to him for sharing a hell of a lot of his wisdom with me. After I proved my worth he was one of my biggest supporters. My only regret is I never got the opportunity to thank him properly for all his help and advice before he sadly passed away...

      I did put him straight on Wikipedia though!!! lol.

      Delete
    2. Point of order ;)

      I DO use "as a parent", and "as a father" but, this is important, it's not a qualifier. It's a positioning statement to give perspective on how I have arrived at a particular view.

      I am sure some people do use it in the manner you have expressed but perhaps not as many as you may think.

      Delete
  9. Since this blog has become pretty irregular are you going to do as you said and get other people in to do articles?

    ReplyDelete
    Replies
    1. Sasquatch I have never said I was going to get other people in to write on this Blog. This Blog is about my journey as a gamer. I've said I'd be willing to work with other people on other Blogs, but this little space, as quiet as it has been recently, is my corner of the Interwebs, for my thoughts on things. So no, I'm not going to get other people in to write articles.

      Delete
    2. My mistake. The post I was thinking of was about how you'd handle things if you stopped blogging and not about hiatuses as I thought.

      Judging by the tone of the response my question seems to have come off a little pithier than was intended. :)

      Delete
    3. There wasn't any tone to my response I can assure you. If you think there was then honestly that wasn't my intention. Just explaining that it's not going to happen. Part of me would rather see the site die, than hand it over to someone else as well. Because if it is no longer about my journey then what would the point be?

      Delete