Game testing- do people want it?

Discussion in 'Game Development (Technical)' started by Reactor, Mar 14, 2008.

  1. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Yesterday I posted an announcement about a new game (and media) testing option named Plasma Beam that's now available, and although it's only been one day, the post is sinking towards the bottom of the announcement list faster than the Titanic. Since the feedback forum is so active and I seem to spend a lot of my time offering feedback for software companies these days, I've put everything else on hold to offer the service. I even quit my job to do it full time (it wasn't a great job, and I needed a change anyway!)

    If people recall, I asked late last year if this is something that people might want, and I got the impression a number of people would. But, now it's up and running I really have to get things rolling. So the question is- do people want solid feedback about their games? Will they pay for it, or are they happy with forum members offering up a few minutes of their spare time? If I get the impression there just isn't a market I'll shut the things down and move on. But, as I mentioned... with the popularity of the feedback forum it's obvious to me people both want and need some kind of feedback. Larger companies certainly do, and they often pay good money for it. But, I know people on here aren't rich so I've done my best to make it as affordable as possible.

    Let me know guys. I hope this post also doesn't do a titanic, and people either take advantage of what I'm doing here, or give me some solid feedback so I can transform this into something that'd be even more helpful. I know it isn't the culture around here to pay for feedback, but there really are huge advantages to it, just as there are huge advantages to paying for artwork or music when you're a programmer, and you know you need help.

    On a side note, if this isn't the best place to post this, could the mods please move it? I know where advertising for art or music/sound creation should go, but this is such a new thing to this community it's hard to know how to get the word out.
     
  2. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    As a one-man operation, without some sort of credentials and testimonials, it's going to be awfully hard to convince people that paying you $200 is superior to asking some guys on a forum what they think.

    To me, your website doesn't look trustworthy. It has the slick, linkless black I associate with a certain kind of con artist (like, say, people peddling turnkey sites). It promises things it can't deliver. You refer to yourself as 'we' and then later admit there's only you. There's no example of any work you've done, in ANYTHING. No information about the credibility of the person providing the service. (Obviously here on the forum we have a link to something else you've done, but your website doesn't.)

    Also:

    ... See, I just gave you feedback free! :)

    Good feedback is valuable, but not everybody's feedback is good feedback. How are your customers supposed to know that that your feedback is worth the money?

    One thing you might consider would be to do something along the lines of a "free initial consultation" - in which you provide some very brief comments. Then if they want more detail out of you, or want your feedback on their changes and updates, they have to pay for that. Obviously this means you might have to do a lot of work for free, but it might also help open the door to a better business relationship.
     
    #2 papillon, Mar 14, 2008
    Last edited: Mar 14, 2008
  3. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Thanks papillon, I love good feedback ;)

    I agree, and had thought about offering a free trial run for a few people, in exchange for a testimonial. I thought I'd see if people wanted the service first, though.

    I went with the black because to a lot of people that's professional (many service websites for video media tend to be black), but I can see where you're coming from. It's a fine balance, and depending on how things go I'll certainly be updating the site to look less 'scammy' ;)

    Many service websites have people contact the person involved to ask about credibility. I chose to do it that way, because it's better to discuss credibility than try and sell it with a blurb... simply because an awesome resume of work doesn't necessarily make you good at testing. As you say, "not everybody's feedback is good feedback". Testimonials are a different matter, however.

    At the moment they aren't so much, but then that's why I'm trialing it here, where I'm somewhat known. If I were to for example post on Gamedev, I'd want some testimonials and the like.

    Again, thanks for the feedback. I'll offer some free testing days, but I'll see what other people suggest first (if any do)...

    EDIT:

    Actually, sorry... where do I say, "We"? I can't find it. I do refer to myself as Plasma Beam at one stage, but that's all. Is that what made you think there was more than one person to the thing?
     
    #3 Reactor, Mar 14, 2008
    Last edited: Mar 14, 2008
  4. Cartman

    Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    618
    Likes Received:
    0
    I would agree that your web site needs some work, but that's not what you are really asking. In my opinion the first impression of your site should give me an idea of what you do, even without me reading it. I didn't get an impression that you doing testing from a quick glance at your web site.

    Here are some suggestion.

    1) Many articles that show examples of your work. Writing blogs about testing software, and providing some general guidelines may help people feel that you are qualified and bring traffic to your website.

    2) Some portals provide feedback on a game. Therefore if you going to submit to a portal, you will get some feedback from them about what sells on their site. As well as the feedback from here. That's not to say good testing is not valuable. However, my point is that you have some competition with the resources that the portals have themselves. So you really need to emphasize why someone would want to pay for your services.

    I wish you luck on your business.
     
  5. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Thanks Cartman, those are also great suggestions. That's an excellent point about the portal situation. I'll factor that into a website redesign.

    With point number 1), I'm going to release a few videos that demonstrate why dedicated 'Plasma Beam style' (if I can coin that phrase) testing is clearly an advantage over casual developer or portal feedback. A blog is a great idea also, though.
     
  6. luggage

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    2,132
    Likes Received:
    0
    I'm pretty sure you could find some kind of market for testing services but as the others have pointed out there's nothing to say how good a service you'll be getting.

    How about a couple of test cases on your site for already released games? This way we can see what we'd be getting. For example, if you want your game tested for bugs but you end up with a lot of "wouldn't it be nice if..." suggestions then it's virtually useless. Not that I think your testing would be like that but with such an empty site we have no idea.

    The other thing that would be really useful for indie developers would be good quality compatibility testing. Testing across a good range of hardware and software to see if anything screws up.
     
  7. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Good food for thought, thanks.

    I'd like to be able to do compatibility testing, but it's both a costly and difficult thing to do well. And, if I can't do it well, I don't want to pretend I can. I mean, even if I had ten different systems, I still wouldn't get anywhere near the kind of compatability testing you'd find by releasing the game to the general public.
     
  8. Nexic

    Indie Author

    Joined:
    Nov 5, 2004
    Messages:
    2,436
    Likes Received:
    0
    "We give you with the targets so you can knock them down"

    It's says 'we' and it doesn't make sense.

    This kind of testing service is something I might be interested in, but I feel $200 is pricey. When Realspace 3 came out we were offering $100 each for feedback and we had tons of people interested, most of which had good previous experience in game media (reviews, blogs, working in the industry etc). Bare in mind ArcadeTown was paying for this, if it was me I would be looking at paying more like $40 per person. But even at this price, I'm sure I'd get a lot of takers.

    My suggestion would be that you:

    1. Get some good credentials (start a blog and try to get a good rep)
    2. Show clear examples of what you get
    3. Get testimonials from well known developers
    4. Find 10 other people with a variety of specs and strike a deal where they will test a game and look for bugs (just load it up and play for 5 minutes) for $5 each.
     
    #8 Nexic, Mar 14, 2008
    Last edited: Mar 14, 2008
  9. Desktop Gaming

    Moderator Original Member Indie Author

    Joined:
    Feb 24, 2005
    Messages:
    2,295
    Likes Received:
    12
    He's right. Doesn't bode well for your games testing career if you didn't spot a whopping great mistake like that. :p

    [edit]

    I should probably add something a bit more constructive.

    1. Your website is all "text text text text text". I just couldn't be bothered reading it all. Consider breaking it up into smaller pages, narrower columns, etc. Keep the text brief and to the point. Don't go jabbering on and on for about 20 screens full of text. Nobody will read more than a few lines.

    2. A big problem you're going to have; if I were to hire you, how do I know you're doing anything? What's to stop you taking my $200 and sending an email "yeah that's fine!" 24 hours later?

    3. I often think its better for games to be tested by non-technical people - the kind of people who are going to be buying and playing my games. I've had people test my games in the past, with feedback collated and emailed via a project manager with some of the most inane comments, i.e. "The grass is the wrong shade of green". I used to get suggestions and requests to change things around, then in the next version I got told that "it would be better if <whatever>".... which is exactly what it did in the first place! I guess what I'm saying is that technically minded people just talk for the sake of having something to say and they come out with complete and utter bollocks half the time.

    I honestly can't see there being many takers on this. Quality Assurance is a very difficult job (to do properly) - and I know because I've done it and it aint a bed of roses. A casual game is going to need at least a couple of weeks of QA, and that's more than most people can afford or be willing to invest.

    Me, I prefer to test games out on people around me, so I can watch them and see what they're trying to do. I find that to be a much better way of finding potential problems.
     
    #9 Desktop Gaming, Mar 14, 2008
    Last edited: Mar 14, 2008
  10. svero

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    3,392
    Likes Received:
    6
    Yeah I saw the post and sometimes I need testing but my first question was... Why this guy? What does he know? Is the feedback going to be worth the money? Can I trust him with a private beta? What's his experience in the industry? What exactly is this service providing I can't do on my own? And so on...

    Here's a better sales pitch...

    Jake has been working in the industry for 25 yrs and has worked on games x,y,z as a developer and produced the following game with publisher P. Through us you can target the casual audience. (or whatever audience) We have a list of so many testers who are all avid players of games like bejweled and mcf etc..

    ... Something like that and I might get interested. But to just hand a beta off to some random person I've never heard of. Not likely.
     
  11. Spore Man

    Indie Author

    Joined:
    Jun 19, 2007
    Messages:
    842
    Likes Received:
    0
    Are you kidding me? Your website is supposed to be MARKETING your service and conveying credibility. If I found 20 sites by Googling, are you seriously saying that your potential customers are going to contact all 20 and then decide based on one-to-one discussion? No. They are going to cross off the list any that did not seem credible, or professional and then contact the rest, or say, the top 5 that seem the most credible.

    Your job is to market yourself. As I see it, such a service is a major risk because it involves putting unreleased games into the hands of a stranger. You could be part of a pirate group for all anyone knows. We pay you $200 for feedback, maybe some pirate group will pay you $200 for "0-day warez".

    Look at it from the perspective of people who have never heard of you and don't visit indiegamer.

    Don't fret though. It takes Microsoft 3 versions before they get something right, so give it a round of improvement.

    Good luck.
     
  12. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Thanks guys. Sorry for the long reply post...

    Nexic:

    I think it makes sense, but great pickup! I searched through all of the text but forgot to check the images. See what a great example I am of how much everyone needs external input? :)

    I'm not sure about less than $200. If I charge less I may as well buy a box and go live on the street, because even at that price I'm hardly going to be making a great living. (that's okay by me, it'd be a start for other things) Charging $40 just isn't realistic- there are already a lot of willing participants around the web who are good at giving brief feedback. Add that to the issue that at $40 I need to be helping at least four people a day, five days a week and the chances to find enough work seems almost impossible. To go with that, I just don't think a quarter of a day is enough for a decent test.

    That said, I'm not ignoring your comments on pricing... that's just some info behind why I've asked for what I have (which I don't consider ideal, just better than undercharging).

    Interesting idea, but fairly implausible considering the timeframe for each test (especially with the $40 model). It's also a privacy issue nightmare.


    Desktop Gaming:

    Thankfully no one is perfect ;)

    Funny you should mention that... it's because of time I didn't get to do what I originally wanted- to have less text and go with a video that kept everything brief. This is good confirmation, thanks. You'll notice the Cellblock website keeps things quite brief. It is interesting though that you say nobody will read more than a few lines, yet people suggest I have a blog, so I don't know how true that is.

    You can only get to the store after talking to me, and I'll only send you a link after we make a binding agreement through email. So, if you purchase and nothing happens, you can track me down and take legal action if you like.

    I agree, and that's because I'm a not too technical person. I know about technical issues, but I'm an avid gamer and psssionate developer. So, my testing process is quite different from every other place you'll find on the 'net.

    I try to avoid complete and utter bollocks as much as possible ;) I know what you mean though, I've seen many random comments on bug trackers.

    Excellent point, and that's why what I'm offering is different. I'll take some steps to show why, but from my perspective a lot of the main issues in media projects can be identified in a short period of time. Not a 'five minute' period of time, but not a huge amount more. Certainly not weeks, anyway. There are other testing solutions out there, but they're different in that they test for everything and anything, and it takes weeks (or months). As you say, for most people this is simply unaffordable. And, that's why I'm coming at this from a different direction.


    svero:

    I have legal and privacy protection in place, but fair enough. I'll do my best to address that.


    Spore Man:

    That's true, but people who have loads of cash will mark my website off the list in an instant anyway, and those who don't will mark the others of their list. As far as I can see, I am the only one on the 'net putting forward a lower-end testing solution (in business form- I'm not talking about individuals who dabble in the odd bit of testing here and there).

    That's not really true, larger companies do this all the time. To protect themselves they get testers to sign non-disclosure agreements and other such things. That said, I agree I need to do more to demonstrate who I am and what I've done in the past.

    Thanks! :) I may have to get a job in the meantime, but I might just push forward with testing and see if I can develop it at a later date. I'm glad I threw up a quick website and am receiving good feedback about the idea- at least I know where I stand.
     
  13. TimS

    Original Member

    Joined:
    Feb 9, 2005
    Messages:
    686
    Likes Received:
    0
    To add to the heap, the quote:

    "We give you with the targets, so you can knock them down"...

    is to me... so... off... that it makes me think I'm dealing with a non-native english speaker. Reading your posts here makes me think otherwise, but... "we give you with the targets"? Wha?

    "We give you" cannot be followed by "with" under ANY circumstances that I'm aware of. And it isn't the 'we' that's the problem, it's the 'with'. Take it out and it'll be ok. "We give you the targets" etc.

    Also...

    The very first line welcomes me to a unique testing facility -- which is possibly ok, but it calls into the imagination much more than is really for sale. If your 'testing facility' is in fact you sitting at your personal computer looking at my game, you should probably stop trying to sell it like it involves a bank of DoD neural nets with a bunch of blinking LEDs and the kid from the Manhattan Project movie overseeing the whole operation.

    It isn't a unique testing 'facility'... it's you, providing feedback. Which is fine, and could well be worth $200 a day (if you're worth $25 an hour).

    Another point... charging by the day is odd. It doesn't include information about how many people will be involved, so it ends up looking cheap when it really isn't. People who pay for testing are familiar with higher prices, but they're getting the mini-army of a testing company with the resulting information. If you make your pricing structure more transparent I think the indie crowd will be more likely to use your services.

    It's really just a matter of deciding if you want to cater to the indies or to the 'big dogs' and writing your promotional website to fit the bill. Right now it is a bit muddled between the two, and as a result it turns off everyone.

    That's my two cents. Hope it helps!

    -Tim
     
  14. Reactor

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    1,635
    Likes Received:
    0
    Now I feel like a total dummy... I completely missed the 'with'. Now I see why it makes no sense! Thanks :p It looks like I'll have to work double time to prove I'm not totally incompetent (you know, only partly)

    That's also good confirmation... I wrote a lot of the main page in a rush, and wasn't that happy with the line. It'll be changed soon anyway, but thanks for the feedback.

    It's unique because I'm the only one on the 'net looking to do it the way I am. Yes, it's me providing feedback, but someone doing that in a full-time capacity and providing more than an email or forum posted list of bugs isn't something I've seen before.

    Even for one guy $200 a day is quite cheap, but I'm going to adjust my pricing anyway (it'll soon be much cheaper, and free for a while too), but I'll make an effort to make clear what it is people will receive.

    It's pretty obviously I'm not exactly targeting the big guys (because hey, who'd hire one guy when they could afford that mini-army you mentioned?)
    but I certainly am paying the price for not getting my website right. In many ways I expected this though, because it's half impossible to research the thing. It was pretty much expected I wouldn't hit the mark first time out... I mean, I've missed it by a mile, but... :)

    It does Tim, thanks.
     
  15. luggage

    Moderator Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    2,132
    Likes Received:
    0
    This bit confuses me a touch. From my experience in games testing it does take a fair old amount of time to clear a game of actual bugs. There's really three different types of testing.

    1) Compatibility testing - you've addressed that you don't have the resources to provide this service.

    2) Bug testing - Looking for areas of the game that don't work as intended. eg. Does saving the game work? Does it crash if I do this...? I'd be willing to pay for some good quality bug testing.

    3) Design issues - These are things like "It would be better if it did this...". and so on. These are the kind of things you would pay a designer or a focus group for.

    If I was to pay for testing I'd only really want type twos reported (presuming there's no compatibility).

    Regarding your website, don't follow other 'service' websites for your template. You keep saying you're offering a unique service to indie developers so don't model yourself with a bunch of sites who aren't offering the same things.

    It would also be useful if you could list the specs of what you do your testing on. And maybe use Virtual PC along with several versions of windows so you can test different OS and DirectX\OGL combinations.

    Testing for actual bugs is something that small indies will struggle with - it's notoriously difficult to test your own game - and there could well be a market for some cheap testing.
     
  16. jcottier

    jcottier New Member

    Joined:
    Jul 12, 2006
    Messages:
    1,385
    Likes Received:
    0
    Does anyone really think he can make a living with it? Personally, I don't think so. Why? Because the people who has a good budget will have their own ressource do to this (designer/tester). The one who are really tight on budget will just do it themselves and will not want to give few 100s for this service.

    You might get a few clients, but as far as doing that for a living, I really think it is close from impossible.

    If really you want to invest some time on this thing, do it as a side project and see how it goes. It always take a lot of time to build a business, you new a catalog of clients/credibility/experience/money. You don't seem to have any of those.

    I am not trying to sound pessismistic, but realistic.

    JC
     
  17. Desktop Gaming

    Moderator Original Member Indie Author

    Joined:
    Feb 24, 2005
    Messages:
    2,295
    Likes Received:
    12
    I agree.

    As I said, I don't think anyone on a budget (which is most of us) will want to invest that much in testing, when the money would arguably be better spent on art/audio.
     
  18. Sectarian

    Sectarian New Member

    Joined:
    Dec 1, 2006
    Messages:
    67
    Likes Received:
    0
    why not have the indie peeps test your game and then send their input to you via email for free as a testimonial that you need to promote your game(s). it's easy if you paid me to test your game, i'd do it
    it's easy if you paid me to tell the world about your game(s) i'd do it but, I wont pay you two cents and give you input about your game(s) when i'd like to test your game and tell you about how much fun it was for me to play. I would be lying through my teeth if I said Quote I bought a ready to fly r/c airplane from la tee da and it flies great yet I don't have the product and never tried the hobby. Almost same senerio
    So perhaps if you offered a way for peeps to try your game(s) and then if they like it would buy it maybe offer them a questionaire along with the game(s).

    When I first tried Ferion, I played the newbie arena this is free, and played several arenas without buying a key to unlock the particular arena and played for free then I decided that I liked the thought of being brutal and it went on from there. And I did'nt pay the game owner one red cent to say Yeah your game is cool.
    :)
     
  19. Desktop Gaming

    Moderator Original Member Indie Author

    Joined:
    Feb 24, 2005
    Messages:
    2,295
    Likes Received:
    12
    I have no idea what you just said. :confused:
     
  20. Nexic

    Indie Author

    Joined:
    Nov 5, 2004
    Messages:
    2,436
    Likes Received:
    0
    My point was that such testing might make it worth $200 (as opposed to only your feedback, which without credentials is worth maybe $40 to me). You'd still get $150 at $5 each. If you could find some people who were willing to do this regularly and you knew them well enough to trust them I don't think it would be a problem.
     

Share This Page

  • About Indie Gamer

    When the original Dexterity Forums closed in 2004, Indie Gamer was born and a diverse community has grown out of a passion for creating great games. Here you will find over 10 years of in-depth discussion on game design, the business of game development, and marketing/sales. Indie Gamer also provides a friendly place to meet up with other Developers, Artists, Composers and Writers.
  • Buy us a beer!

    Indie Gamer is delicately held together by a single poor bastard who thankfully gets help from various community volunteers. If you frequent this site or have found value in something you've learned here, help keep the site running by donating a few dollars (for beer of course)!

    Sure, I'll Buy You a Beer