Hi guys, I'm at a stage in life where I'm fairly unhappy in my current job, and now isn't the right time to start developing a game again (though I'm always watching for the right time!). So, I had a thought to start a freelance graphics design company, or to do something a little different. I'll no doubt get hammered for putting this on here, but one of the ideas I had was to provide a service as a development consultant, who would test products and search for bugs, interface and usability issues, etc etc. (there are many things I could do) Most developers seem happy to create a bit of software, and then submit it to a community as if by doing that, they're getting good feedback on it. On this forum, I'd agree with people if they said the feedback they'd gotten on here was of a high standard. But, this forum is rare. I've purchased a fair few programs recently and have spent hour upon hour on forums creating giant bug-lists and feature suggestions (all well received) that it's become obvious to me things like beta periods with certain community members (and other such attempts at bug-hunting) just aren't all that effective. Sometimes, having a specialist cast a discerning eye over a project is much, much more helpful. So, my question is, and ignoring how much this service might cost, would something like this be helpful for indie game developers? Obviously indie's don't have cash to throw around everywhere, and the issue of hiring other people (whether artists, sound people etc) is already quite difficult for most, so... that's why I ask. For myself personally, I already spend quite a few hours helping companies to develop their products into stronger, better working ones... and sometimes I think they should pay me for the reports I submit!
A small update- I noticed FreshlySqueezed has something already happening, so to clarify, I'd be offering this service in regards to any media... not limited to game design.
It's hard to give an answer if you ignore the price. For example, if the price was £1 per game then it would be a (very) worthwhile service. If it was £100,000 a game it would not. I'd be more interested than a detailed amount of bug and compatibility testing rather than feature suggestions. Trouble is testing is not as easy a task as some people think. I've worked with a lot of poor testers and very very few good ones. If I was paying money for someone to bug test my game I'd expect to get a decent job. As an example, working on a PS2 game I got a bug along the lines of "The default mouse speed could do with being a bit faster." This was flagged as a class A (crash bug, showstopper, game can't go out with it in there). Got my ear chewed off by the boss for having a class A crop up. I asked the tester why he made it a class A and got told "You wouldn't sort it if it wasn't a class A".
If it helps, I've been involved with bug testing before and although I'm not an A-class programmer, I know enough not to flag something like that as a serious problem I have seen people do that though, and it musn't be all that helpful in a bug tracker. My brother is a darned good programmer, and I've been able to sit and watch how he deals with issues... when a bug report is helpful, and when it isn't. Most of the time though I think the best thing to do for a programmer is provide a list of issues, how to produce them (if possible) and leave the rest up to them... unless they've has asked otherwise, of course Anyway, thanks for the feedback! In the situation of the mouse speed, I'd put that more in the interface/game usability box, and not in a bug report one. EDIT: Sorry, on the topic of price... that's a good question at this point. I have the option of asking for an hourly rate similar to what you'd get hiring a web designer (nothing too crazy, I'm not a money hungry guy) or a per-project fee based on what would be required. For yourself, what's more appealing?
How the pricing is worked out is a tricky one. If it's a flat rate per project regardless of genre then it could mean a lot of work for you. Would take a lot more effort to bugtest an RPG than a Bejewelled game. On an hourly basis is a bit better but it's always a little uncomfortable contracting someone to work per hour when it's someone you don't know (no offence intended!). Probably wiser to ask for a recent build of the game, give it a quick play and then give a reasonable quote based on how much work you think is involved. That way both parties have room to negotiate a fee you're both happy with.
You're making one big glaring mistake of an assumption here. That the feedback is no good. It IS good. The problem is with devs NOT BOTHERING TO MAKE THE CHANGES. I suppose if a dev PAID for the feedback, they'd be more inclined to implement the feedback, but from what I see here, there's plenty of good feedback, along with plenty of "well I'm finished with this game, so I don't want to spend more time on it". So even a "specialist" may be of no use to devs with that kind of attitude.
On this forum the feedback about games is much more detailed than what you'd find on many software forums. When the community of a software project (game, app, whatever it might be) provide some kind of feedback, it is actually good. So in that, I agree. But, the feedback is often very limited to the workflow, biases and time constraints of the community member providing it. It's true that someone like myself would also be affected by certain workflow limitations and biases, but the difference comes in that I aim to be broard in how I look at things, whereas community members often don't. A few cases come to mind- the beta period for Zbrush 3, and for Daz3D's Hexagon 2 application. Both programs had beta periods (6 months-1yr) with 3D veterans who tested the application for a long period of time. Blogs and forums posts indicated all was working well with both programs, but as soon as they were released I had to post giant laundry list bug reports for the silliest little things. The testers of Hexagon 2 (mostly community members) stood behind their testing of the program, even though the rest of the community complained about the horrible state it'd been released in. After chatting to a number of people it became clear both programs did work well, providing no one stepped outside of the specific workflows of the beta testers, who'd made sure the functions they were after worked for them. Because they were focused on making sure the new fangled features worked well, the basics of interface functionality went completely out of the window. I hope I'm making sense The long and short of it is- yes, there's some good feedback from people around here, and on a few other forums. But, from my point of view there's a large difference between getting some feedback from other developers, friends or community members, and hiring someone to go over the program (or media) with a fine tooth comb. Perhaps in time I'll be able to demonstrate that. Thanks for the feedback tho'... I'm taking it all on board!
There's only so much time for regression testing. You'd need the department to grow exponentially. On a project I was working on, we were getting near beta when someone clicked on a pull-down list widget. The cursor disappeared and when the user moved their mouse, the selection in the list followed. There was no way to get the cursor back. It was something that had been working for months or years and suddenly it stopped working. We checked out old copies of the project and it had been broken for a while. Apparently, the testers hadn't tested this one widget in months so when it broke, no one noticed. It took hours to track down the bug. It turned out to be a race condition between event handlers. In this case, and who knows how many more, one event fired ahead of another and put the widget and mouse into invalid states.
Heh... interesting story... True, but in this case I'm not sure that was the reason behind the issues.
You could start with low prices and build up a client list of successful games to display on your site as evidence of your utility. You could use that list to charge more for your service over time. One thing that I would find particularly useful is a service that gives advice on interface design as bad interfaces can ruin good games.