On fixing crashes that don't happen on your own computer.

[RinkuHero]

I'm developing a game and it's currently in playtesting. The last version, v0.5, worked perfectly for all playtesters (about 10 to 20 people).

The next and current version, v0.6, works perfectly for me and for several playtesters, but three of them experienced problems: one has it soft crash (it freezes, but doesn't crash the computer) randomly with seemingly no pattern, and two others have it hard crash (it crashes their computer entirely), and in a consistently repeatable place -- however, that place is *different* for those two (one is when he starts a stage, the other is when a countdown preparation part of a stage reaches 0). I've asked all three of the people it crashes on for their system and driver etc. specs, and can find nothing in common.

Now, I'm not asking this as a technical question, obviously you'd have to know far more about the game than I mention here in order to help with that. What I'm asking is a more general question: when you are confronted with situations like this, where a game has a problem on several computers but you don't have access to any computer it has problems on, what exactly can you do? Do you try to solve it remotely? Do you try to gain access to a computer that it crashes?

What I probably need to do is send them v0.51, v0.52, v0.53 and so on and see at which point the crashes were introduced, narrowing it down to particular changes I made, but that will take awhile, and it would be annoying to them -- they're doing me a favor by playtesting it and I'd be asking them to repeatedly try something which they know will crash their computer.

[JoshuaSmyth]

I hate it when that happens... and I'd like to know what others have to share on the matter.

Whenever it's happened to me in the past, its usually been something to do with either having a different operating system version (eg xp vs 98) or having a different video card and/or older drivers.

The only way I've managed to think of is simply to never throw your old computer away, keep it for testing.

[mDEV]

It sounds a lot like memory access violation (pointer problem) gaining access to the crashing computer wouldnt help much without having some sort of tracking mechanism for debugging.

What I would do is checking all the new pointers since your last non-crashing version, maybe overriding new () and delete () operator with some sort of simple tracking mechanism would possibly help if you're using C++ that is..

hth..

[RinkuHero]

I'm actually using Game Maker *^^* -- and that uses an interpreted language with no real memory pointers accessible to me, which makes this even stranger, because I don't think I changed anything between the two versions which would cause these crashes.

[cliffski]

general tips:

add lots of asserts, and make it easy to make a debug build with the asserts in (actually my release builds keep them in, but my games aren't performance critical).

Make it easy in the options for your game to totally disable certain things, especially sound. Being able to 100% rule out the problem being related to your sound system is massively helpful.

Build in some debug log code so that once the game crashes, you have some idea what was going on shortly beforehand, and log all warnings or errors to it.

If possible, build code that offers to email you the error message when an assert is triggered, and make sure any logging or email also contains as much system info as you might need, especially the version of the O/S and the video card data.

Good luck!

[princec]

Quick tip: if it's crashing hard and requiring a reset that means it's crashed in kernel mode (after ruling out overheating and duff memory problems), which means it's crashed in a driver (probably a graphics driver but sound can occasionally go wrong too).

Cas

[papillon]

Geeky playtesters who have machines that they don't use for any serious purposes and are quite happy to repeatedly crash them for your whims (as well as swap in/out a variety of crappy graphics cards) are GOLDEN.

(I have a captive linux geek, who built a winbox almost entirely to be used for my evil purposes, and I bought him a range of graphics cards. Since it's not his 'real' computer, frequent resets, if necessary, don't matter that much.)

But really, try to find devoted testers instead of just people 'willing to take a look at it'. Willing to take a look at is okay, but not nearly as helpful as someone willing to keep downloading new versions and breaking things until they stop breaking.

[LilGames]

First you need to rule out the stupid---err, I mean Human factor. For example, I get emails to support: "I used to play Game XYZ on your site every day, and now all I get is a black box".

First attempt to gain more info, they claim "I haven't changed anything on my computer!"

Dig deeper and I find out "Oh I uninstalled Flash because it was causing pop-ups".

DERRRRR

Maybe your crash testers upgraded to Vista and didn't bother to consider that? Who knows...

[RinkuHero]

Thanks everyone!

I did have code to disable the sound system actually, so I'm sure it's not that. I never really used assert() though (or built a Game Maker equivalent -- it doesn't have one) so I'll consider doing that in the future. I've never really understood how to use assert() though, aside from making sure you don't divide by zero.

I agree on the importance of dedicated playtesters, but only one of the three people it crashed for is in that category (and he said he's willing to try to crash it).

The human stupidity factor is true, but I took care of by making them go back and play the older v0.5 version to be sure that it doesn't crash.

[lennard]

Cliffski's idea of wrapping your assert with an email back is good although you will have to go back in and sprinkle them everywhere.

My games tend to run under a single FSM so this may not help you but, if you can open a dos box (or any other info window or file) then you can also spew out state information which could help you quickly narrow in on where the thing goes south.

Intermittent bugs are the worst.

[ZuluBoy]

Asserts are useful, but they won’t help you detect memory access violations, memory leaks, double deletions etc…
On Win32, there is a good (and free) tool for tracking them, it is gflags (http://www.microsoft.com/whdc/devtoo.../default.mspx).

[RinkuHero]

Fixed! The problem was that sometimes I was drawing a graphics primitive -- a triangle fan -- using only a single vertex instead of three. Which apparently some video setups crash on.

[TeeGee]

You solved your problem before I could reply, but your explanation proves the point I've had in mind, so I've decided to bring it on .

Most errors that cause those dreaded crashes are really silly things. The ones that make you slap your head saying: "How I could miss that!?". Especially on simple engines/languages like the GameMaker.

In most cases, if the new version of your game crashes, it's some really stupid error in the most recent piece of the code. Different machines react in different ways to the same bugs/memory leaks/whatever, but crash is a crash. Best idea is to not think too much and just check every piece of the new code once again. If few users are experiencing crashes, there has to be something in there and hopefully it's the reason of all the problems.
That's why I prefer to update my test version as soon as I implement something new. Of course it needs some actual beta-testers to be possible, but it's really not that hard to find dedicated people over the internet.

Of course there are people that will always have issues with your game. My most favorite type is a guy that sends you an e-mail saying that your game sucks because he can't even run it and when you ask him politely what are his system specifications, he answers:

"Nothing special...

Standard 386DX with B/W screen taken from my grandpa. I use directX 3.0 because in 4.0+ CIA implemented their tracking device and Windows sucks anyway!

Oh and I'm using steam engine instead of my CPU, I don't have a keyboard because it sucks and I use my 2MBs of RAM as a frying pan. And I also kick my PC all the time while playing your game, but it shouldn't cause any problems! It's all YOUR fault!"

In that case - just ignore the problem. If 10k of your gamers are happy and 1 always has some issues - you'll have to live with that. Really - most problems that are not caused by silly coding mistakes, come from users ignoring the "system requirements" part. At least from my experience...

[Nikster]

If you're using directx, all the more reason to turn on dx debugging, I've had something similar with scissor regions where any point that sits outside a canvas would crash on some cards and would be handled on others. wouldn't have caught it without dx debugging though.

[zoombapup]

One of the main things you can do to protect yourself from these kinds of bugs, is automate a checking system for yourself, so essentially add things like units tests and asserts a-plenty that compile out with a switch.

In your case, I was going to suggest getting a demo trial of Purify, but it probably wouldnt have helped you. Using the debug DX runtime and getting the output into a window might well have.