To mobile game developers: how do you test your games?

Discussion in 'Indie Business' started by baegsi, Feb 12, 2006.

  1. baegsi

    Original Member

    Joined:
    Oct 8, 2004
    Messages:
    424
    Likes Received:
    0
    Hi,

    to those who are sucessfully developing mobile games: how do you port and test your games?
    • Rely on emulators hoping the real devices behave similar (which they don't)?
    • Buy as many devices as you can afford and test on your own?
    • Outsource the testing and porting task?
    Personally, I'd like to follow the last: outsource the whole task. Question is: would this pay off? Has someone done this already? If yes, can you recommend someone? And how are the cost calculated: per hour, per handset or an other way?

    Any tips are very appreciated.
     
  2. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    Hi Baegsi,

    I have built up a library of code so I now know what works on what mobiles etc.

    But in reality after first testing on emu's and also some phones here, the final end code gets tested by the publishers in their own in house test dept.
    Feedback of any errors comes back as an excel sheet, tho there are rarely any errors or probs these days because of the first point above really that came with a bit of hard work and time in the first place of course.

    It's much like reaching a critical mass cut off point with the code, in that once you know the skeleton code runs on all mobiles there is not much else to it unless the game engine in use needs to be totally re-written for some reason perhaps.

    Thats how I do it here anyway.

    Cheers,
     
    #2 Adrian Cummings, Feb 12, 2006
    Last edited: Feb 12, 2006
  3. baegsi

    Original Member

    Joined:
    Oct 8, 2004
    Messages:
    424
    Likes Received:
    0
    Thanks for your answer, Adrian.

    That's a nice thing to have. Is that a standard service of publishers to have them test a game they publish?

    Every month there's this plethora of new phones available. Do you have to adjust your library often, or are mobile phones actually getting less buggy these days?

    What do you think about other libraries who address this problem, like j2mepolish?
     
  4. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    Regards the testing via publishers... hard to say as I have only ever been offered this type of deal twice in practice, so I dont know if it is the norm.

    The new phones are not really all that new tho thats the thing :) they are say like Nokia N80/N90 to me just a few changes to screen size equates and many are now midp2 so I know my midp1 code will run on them anyway before even getting to test - it's getting better now this sort of thing not worse in reality like before.

    J2MEPolish is fine but I have a large list of equates for all the main makes/models sets/subsets of phones and just use that to churn out the builds. This includes things like canvas size key maps and various details about each phone/set of devices form Nokia, Sharp thru to Siemens etc. etc.

    Somebody asked me recently how I did this and yes it took me months to create the working list of magic numbers with test on test after test and eventually I got most of the market place phones covered tho I am still missing some like panasonic, sagem but I've got about 75-80% of old and new devices now with a few missing here and there.

    Cheers,
     
    #4 Adrian Cummings, Feb 12, 2006
    Last edited: Feb 13, 2006
  5. StGabriel

    Original Member

    Joined:
    Jan 27, 2006
    Messages:
    26
    Likes Received:
    0
    I work professionally, not independently, making J2ME mobile games. We do test on each actual phone and contract a lot of this work out, but then we deal directly with the carriers with no publisher go-between. With a lot of work you can figure out what most of the differences are going to be. Resolution is easy, correct values for the softkeys is easy. Handling pauses, handling heap overruns, handling sounds (especially during interrupts) can get quite tricky.

    What phones you are selling to will depend a lot on your target market (US, Europe, etc. -- Europeans go through phone models pretty darn quickly). What issues you run into will depend on what phones you target and what you are trying to do with your game. For example, you may never have to know that there are restrictions on image width/height on Motorolas if you never use really wide filmstrip images -- but as soon as you do you'll start getting errors on those phones. If you want your games to talk to a server you just took on a whole lot more issues, issues which can be not just phone-specific but also carrier-specific.

    In general I'd say that emulators aren't going to cut it, you'll need access to actual phones in some manner or another.
     
  6. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    Depends what you are trying to write I guess to some extent and for what phones perhaps?.

    It's not an easy choice going down the mobile route there are many pitfalls of course, but we all have to start somewhere. Don't be put off just look at the problem wisely and buy the key phones your end (2nd hand off eBay would do just fine heh!).

    Mobiles are getting better now and the market is moving on all the time, so it's not as hard as it was say cramming everything into a 3510i or 7210 (even tho you still have to most of the time at the moment). Things are getting easier with the better phones in my opinion, well for me they are anyway and I reckon I should know being me and writing and porting all my games on my own for the publishers/portals and carriers :)

    It's hard but not that hard once you get going - really - it's just that 'getting going' is the hard part from my experience as an indie mobile games developer.

    Cheers,
     
  7. Flint

    Original Member

    Joined:
    Feb 14, 2006
    Messages:
    3
    Likes Received:
    0
    I don't know if I would agree that things are getting better in that area. We may not be squashing everything into 65k limits anymore, but as quality expectations rise, the higher limits are starting to become just as problematic. I also find that while MIDP2 standardized on and added a couple of useful items, implementation quality varies greatly (especially in media handling!) and there are lots of things that still need to be standardized-for example the fact that there are still not standard softkey codes. The MIDP market has a long way to go before platforms are in any way consistent.

    Anyway, baegsi you asked about testing. Depending on where in your development phase you are and what your budget is, you may want to look at something like getjar.com that allows you to submit software for testing by unpaid beta testers. Obviously being able to pay for testing across multiple handsets is great, but for those of us that can't afford it, a free service like getjar's is incredibly valuable. We found the reports we recieved really useful, especially inconjunction with handset information we were able to package along with high score submissions.
     
  8. Jason Chong

    Original Member

    Joined:
    Jul 30, 2004
    Messages:
    264
    Likes Received:
    0
    I use to code mobile apps and games.

    I got so sick of it because there's simply no way as an independent will I ever be able to penetrate this industry easily, without being screwed by so many people in between.

    Even with a good game I think it will be difficult to sell because there's no way anyone will publish your game unless you've got multiple ports (pain in the ass) and no way you can do it yourself and sell direct to consumers.

    It's a monopoly by telco and aggregrators so even after I stop coding mobile games and apps as a day time job, i still refuse to code them as an indie.

    The barriers for indies like me is just too high and more and more people are pumping out rubbish daily like a rubbish mill. Too much crap out there and things are not helping with companies claiming to have game engines that lets you finish making a game in 2 weeks or less.

    It's one area of industry I am very disappointed in because of the layers between you and the customer in addition to the ports you have to do and money you have to fork out to invest in the devices for testing.

    The costs of mobile game development for indies could actually exceed that of what we're all doing now making casual games.
     
  9. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    Hmmm oh dear well maybe not?... you are entitled your own opinion of course but you have to ask how I am doing then??? - er I'm an indie as well, and am doing fine and while I take what your saying as a point it does not apply to everyone really.

    It can be done - I'm doing it - and on my own!.
     
  10. Flint

    Original Member

    Joined:
    Feb 14, 2006
    Messages:
    3
    Likes Received:
    0
    Well then I have to ask, what is your secret? Do you have any specifically mobile related advice? Because to be honest Jason's comments ring true to what i have experienced as well. It really does seem as if there is less and less likelyhood of 'little guys' making it these days in the mobile market. Competing for distributors' attention against bigger publishers that insist on deluding themselves into beliveing that mobile gamers want inferior ports of console titles, and lame movie tie ins sometimes seems impossible-especially since the distros would apparently really like that to be true because it makes their job of actually selling titles easier.
     
  11. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    The secret is two things...

    1) Find a publisher that will handle nearlly all if not all the big carriers and portals for you so that your titles get into mainstream and you just concentrate on development.

    2) Have a burning desire (and 100% ability) to deliver the titles on as many mobile devices as can be managed and quickly.

    I started on just Nokia and SE and now support most devices across the board form Sharp to Siemens etc.

    The hard part, and it took me about 8 months was step 1 above and hence that is what I meant by it is harder to get going, than keeping going in the mobile industry at least.
     
    #11 Adrian Cummings, Feb 17, 2006
    Last edited: Feb 17, 2006
  12. Flint

    Original Member

    Joined:
    Feb 14, 2006
    Messages:
    3
    Likes Received:
    0
    Then if I may ask (excuse me if I'm pulling this thread off topic), what in your experience is the focal point for publishers? Are they more concerned about graphical quality, great gameplay, or broad device support?
     
  13. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    Some I have had meetings with in the past where only interested in brands of course and I did'nt really like their business models anyway when it came to payment.

    Others (and I mean I could count them on one hand) were after broad device coverage and quality of titles that were apt for the current market at the time.

    They are not really interested I have found if you want to write what YOU want to write, they are more interested in quick delivery to market given a mix of mid to high quality and what THEY want you to write.

    Afterall it is the publisher that has a much better understanding of what will sell on the portals and by carriers and not the developer really.

    Just to add, if your in it for the fun using this route forget it, but if you are in it for the money (if not, then you have to ask what is the point?) then thats about it! :)
     
    #13 Adrian Cummings, Feb 17, 2006
    Last edited: Feb 17, 2006
  14. Oaf

    Oaf
    Original Member

    Joined:
    Jul 30, 2004
    Messages:
    136
    Likes Received:
    0
    You do need to test on hardware - emulators are fine for testing screen layouts and so on, but you can't rely on them to verify your game will work on an actual handset.

    GetJar is useful - but I've also a bunch of people who test games for me; fellow developers, friends on forums and so on. Not only can they tell you if the game works or not (and how quickly!), but they also can give good feedback on the game itself - something a testing house might not do....

    You can succeed at this on your own, but you're going to have to be a good salesman as well as a developer. In fact, I think you'll need to be a better salesman than you are developer! That means forging and maintaining good relationships with operators and aggregators, sending them as much marketing material as you can, even sending them presents.

    The market has taken a downturn over the last two years, as everyone jumps on the license bandwagon. As someone said, some aggregators will only take branded content. But I think some of them are beginning to realise (and Glu stated publicly) that taking a crap game and slapping a license on it "will not do". Perhaps they've figured that, by the time they've paid the licensor, paid the developer, there's not much money left in the pot for them....

    However, I've recently taken the decision to hand over all the sales aspect of the business to another person, and split the proceeds. This leaves me free to make good games, without having to be continually nice to people! :D

    Finally, you can sell mobile games direct from your own website. Take a look at sharewire.net - they cover most operators and countries, BUT they take 25 euros a month from your revenue (providing you've actually made 25 euros a month!). It's the best method out there at the moment.
     
  15. Sysiphus

    Original Member

    Joined:
    Jul 26, 2005
    Messages:
    433
    Likes Received:
    0
    Greatly interesting thread. I work for mobile as a all-kindda-graphics grunt (less workers than a hand can count, though, in this company) ,but like to dream of getting into this market with a friend. I have done many games(the graphics) at the company in months, so I guess this can be profitable if done for my own. Most of what I read here is what more or less I was aware of...But I admire people like Adrian, as indeed, I see it as a very hard market to get in.

    Does someone here made it nicely or averagely well via sharewire.net , anyway ? Is better than Handango ?
     
  16. Oaf

    Oaf
    Original Member

    Joined:
    Jul 30, 2004
    Messages:
    136
    Likes Received:
    0
    Handango's hopeless for J2ME. Don't bother.

    The success of sharewire is dependant on the popularity of your own web site - it allows you to sell direct to consumer via your site. Although they've murmered about aggregating their customer's content and offering it to operators, I've yet to see anything from it.
     
  17. Sysiphus

    Original Member

    Joined:
    Jul 26, 2005
    Messages:
    433
    Likes Received:
    0
    I see. We had not very good experience when did put the game at Handango. Sharewire then could be a last option.
     
  18. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    If you cant get a mainstream publisher to take your content then Sharewire or ClickGamer are the only real decent options, as I dont think (I recall) that Handango even take j2me stuff on anymore - cant remember for sure tho ok?.

    Anyway I think we all agree that you do need to test on hardware but emu's are fine to get going with while you build up a stock of phones to test on if you are going to do it on your own, or use getjar or similar of course.

    Overall tho it can be done you just have to really want to do it and keep trying like I did through thick and thin - just like anything in life really no?.

    That's me done on this thread tho as I think we have all answered Baegsi's original question :)

    Happy birthday to me I'm 41 today and really should'nt be on here at this time of the morning heh.

    Cheers,
     
  19. mj32

    Original Member

    Joined:
    Jun 24, 2005
    Messages:
    39
    Likes Received:
    0
    Mobile games for a living can be done. In fact it´s all ive done since end of 2002 as an indie developer - apart from some contract work on mobile development and its the only income ive had since March 2004.

    It´s hard work to find decent distribution, people will try and f**k you over when you start up but it can be done! And never underestimate the need to have phones to test on, emu´s lie one way or another - if you don´t check it out before releasing it you will get burned one day - trust me :rolleyes:

    I still think maybe a "índie mobile" board maybe a good thing to add to the forums due to the constant stream of questions here and there.... then we can collate all these posts into one place.

    Anyway - just thought I would add a positive slant to this thread agreeing with Adrians - amongst all the negatives :D
     
  20. Adrian Cummings

    Indie Author

    Joined:
    Feb 15, 2005
    Messages:
    922
    Likes Received:
    0
    I agree with mj32, yes it is hard but it's no use being negative of course you have to stay positive to get anywhere in this game!.

    I've only been on this board for about a year I think, but I have noticed quite a fair bit of negative attitude when people like myself share the inside info a little bit - it's not like I have to do that of course, but having been through some really hard times myself in the past, I like to offer help and advice (when required) to others who have been less fortunate than me of late.

    I'm not into all that bad karma style'eee stuph either but you ain't gonna get anywhere in life if all you do is moan and gripe - negative attitude usually amounts to negative rewards in my opinion (i'm 41 now so can talk from experience having done all that in the past in the games biz) however maybe it is just something you have to learn on your own and then get over it - I don't really know? :p

    Anyway, I'm still up for the seperate 'indie mobile' board also it has to be said :) as mobile games are not going to go away - moreover they are the future in many respects, albeit it's all still in it's infancy at this time.

    Stay happy and cheers,
     

Share This Page

  • About Indie Gamer

    When the original Dexterity Forums closed in 2004, Indie Gamer was born and a diverse community has grown out of a passion for creating great games. Here you will find over 10 years of in-depth discussion on game design, the business of game development, and marketing/sales. Indie Gamer also provides a friendly place to meet up with other Developers, Artists, Composers and Writers.
  • Buy us a beer!

    Indie Gamer is delicately held together by a single poor bastard who thankfully gets help from various community volunteers. If you frequent this site or have found value in something you've learned here, help keep the site running by donating a few dollars (for beer of course)!

    Sure, I'll Buy You a Beer