OpenGL on Windows rears ugly head, Indies taken by surprise

[princec]

In logtastic #1 I earlier used an incorrect query to determine my game installation failure rate. When I put the correct query in I was basically astounded to discover a failure rate on XP of just 2.5%.

Yes, that means what you think it means. It means that just 2.5% of the Windows XP Ultratron installation logs I have show installs where the user is unable to actually play the game.

Only two variables affect the accuracy of this figure:

1. Firewalls blocking my logging service. But they should be fairly evenly spread and shouldn't affect the % either way signficantly.

2. VM crashes. I can't trap VM crashes with my logger. Broken drivers usually cause a VM crash. The best I can do is if they run the game again it writes a crash recovery log. So basically if someone runs the game, it barfs, and they give up immediately, I don't know about those.

What are you going to make of that then, software-only/DirectX zealots?

Cas :)

[PeterM]

I'm a big OpenGL fan, who unfortunately has to support D3D as well.

The stats show to me a link between people who like Ultratron and people who know how to install drivers. Not necessarily your average casual or indie gamers.

[luggage]

Do you have stats on how many of these people alread had one of your other games and had managed to get that to work? Like what percentage are Puppy Veteranss?

[princec]

Fraid not. Most of these people are just random new downloaders from all over the place.

If BrianF gives me permission I'll post up the Arcadetown stats, which are even more interesting because they're from the casual games audience. Let's just say that they're even more impressive.

Cas :)

[princec]

See post wrt. Arcadetown logs.

I really am just bringing in casual gamers. And they really do all have drivers. The vast majority of XP users for example are running SP2 as well.

Cas :)

[svero]

Ok. Now try it with a big general audience of regular folk!

[Nexic]

Cas, I think you will find ArcadeTown players are slightly less casual than people in other places. Eitherway, thats still great news!

[soniCron]

What are you going to make of that then, software-only/DirectX zealots? While I'm not a zealot, at least I don't seem to have to think about developing a Direct3D render driver as well! Curious, could you post the results of this query? And when you say you can't trap the VM crashing, but it usually crashes because of driver issues, do you mean video driver issues?

[princec]

Yes, only actual broken video drivers cause a VM crash. For example ATI Rage Pro 128 Mobility drivers crash if you call glDrawElements. Drivers that simply aren't up to scratch or not installed don't cause a crash and instead I get a log with runtime=0 and played=0. It is conceivable that someone will launch the game and not play it and just quit but it's unlikely so I don't worry about that situation much; and even if it did happen, it only makes the figures worse.

Which query did you want the results for?

Cas :)

[soniCron]

Which query did you want the results for? I'm curious about lesser Windows versions (98SE-). But since we're rapping about queries, any luck with the high-score thing (http://forums.indiegamer.com/showpost.php?p=51313)?

[princec]

Can't do the hiscores thing easily and I'm not so sure it really would glean us any useful information. 98 manages a failure rate of 16.3%.

Cas :)

[soniCron]

Can't do the hiscores thing easily and I'm not so sure it really would glean us any useful information. 98 manages a failure rate of 16.3%. Thanks! Looks like OpenGL it is! :) If you think the highscore theory is bunk, then what else could account for the totally inconclusive results?

[princec]

No idea. I'm just going to sit back and watch how it develops.

Cas :)

[Jack Norton]

What are you going to make of that then, software-only/DirectX zealots?

Cas :)
I was a OpenGL zealot :) while I don't have such complex log system, since I put both renderer available (DX/GL) I noticed a 40% sales increase.
Why use one when you can use both? :cool:

[Jay_Kyburz]

I'm confused, are you saying that 2.5% of xp machines not being able to play the game is good?

Is there some sarcasm I'm missing. I would have expected a much lower failure rate.

[soniCron]

I'm confused, are you saying that 2.5% of xp machines not being able to play the game is good?

Is there some sarcasm I'm missing. I would have expected a much lower failure rate. For broken OpenGL drivers? No way. Microsoft doesn't give half a rat's ass about OpenGL and thus doesn't do much to support it. If they did, you could bet the failure rate would be much lower. Reflexive had to support software rendering in Wik because the OpenGL failure rate was so high. I think a lot of us have been expecting 10%+ failures (perhaps even 25%+), so hearing it's only 2.5% is a good thing.

[Reactor]

Cellsquads is Opengl, and apart from the odd glitch here and there (due to driver changes) we've had no reports of failures.

[soniCron]

Cellsquads is Opengl, and apart from the odd glitch here and there (due to driver changes) we've had no reports of failures. That may be so, but also consider your audience.

[PeterM]

Well if lots of players can play Cas' games, then I think it's good news. It's about time he got some rewards for his work.

I'll continue to support both D3D + OpenGL until MS start supporting OGL again (if that ever happens).

[princec]

It would appear MS still do support OpenGL, but if you buy your XP upgrade from a shop, you'll find that the drivers on the CD don't come with GL support. However the vast majority of people who have XP had it bundled with their machine, and that machine came with all the drivers for the hardware already installed. At least, this is what I surmise.

I'm not sure why Jay_Kyburz thinks a 2.5% failure rate is bad. Or even remotely bad. About 30% of the games I've played from Indiegamer developers don't work properly on one or more machines I've played them on. (Always, for some reason, the DirectX ones). DirectX may be a more stable and even more prevalent API, but you C++/DX coders have really got to get your act together and get the code watertight :)

Also, Jay - the reason why this post is so interesting to many is that about 2 years ago or whenever it was when I originally started logging this stuff, about 35% of Windows machines didn't have OpenGL drivers. Now it's about 2.5% which is a seriously good turn of events for those of us who haven't got the time nor expertise to write and support DirectX software!

Cas :)

[luggage]

How do your figures measure up against Wik's? If it was only a 2.5% failure rate I can't imagine they'd have gone to the hassle of putting it into software.

Mind you I suppose they could have tested it on 10 machines - found it not working on 5 of them and then sort it.

[Jay_Kyburz]

I'm not sure why Jay_Kyburz thinks a 2.5% failure rate is bad. Or even remotely bad.

I work in a retail PC games company and I know that an off the self game has a much better failure rate.

Is it legal to redistribute OGL drives in the same way it is to distribute new versions of direct x?

[princec]

This is precisely why I take everyone else's "facts" round here with such a pinch of salt - even the venerable James C. Smith - because without accurate logs, no-one knows the actual truth of the matter. Not that James was wrong to switch to a software renderer: ISTR he claimed that the volume of support issues they were getting was high with GL-only. However, I think this is more likely because he had an immature OpenGL renderer: it's taken me 3 years and 91,000 logs to get mine to the level of compatibility it currently enjoys. He figured it'd be easier to ditch it than fix it which is a perfectly reasonable call and the kind of decision project managers make every day.

I should hope that an off-the-shelf game has a better failure rate - <1% with any luck - because being distributed on CD they should come with the required drivers as well. Given that I'm not distributing the drivers I think that I'm doing particularly well here. It means that I'm losing 2.5% of my sales potential - big deal. I lose more than that just by writing nonstandard games in the first place.

My conclusion to this thread is: whether I exclusively use hardware drivers, and whether I exclusively use OpenGL or not, is no longer the biggest factor I have to worry about to drive sales (after exposure). I can just forget about the whole issue and worry about designing games that people enjoy and want to buy happy in the knowledge that a fairly small fraction of the people downloading won't get to find out what all the fuss is about.

Cas :)

[princec]

I work in a retail PC games company and I know that an off the self game has a much better failure rate.?

Actually I just re-read this statement and once again I feel the need to ask this: where are the facts to be found about this and how were they measured? I spent a long time in the dark trusting people about all sorts of issues like this until I actually went and found out for myself that "facts" like this are usually "not facts".

Eg. the revered 5mb download limit which turned out to be complete bollocks
Eg. Java is slow and crap and has a massive JVM which also turned out to be complete bollocks
Eg. serverside key verification means support calls and lost sales which turned out to be completely untrue (never had a single issue)
Eg. games requiring 8 keys to play are too complicated and won't sell which is still being proved wrong every day :)
Eg. openGL drivers are broken and buggy and not distributed on Windows
Eg. that logging graphics card drivers would upset potential customers (not one complaint, and interestingly, no change in sales when I turned it off)

and so on. So not that I'm personally attacking you here but... have you got some actual figures and sources because it'd be interesting to compare.


Cas :)

[tolik]

Just a quick note - default AGP bios settings on some motherboards prevent OpenGL from working in hardware accelerated mode.

princec, have you tried to break driver installation/opengl yourself and see if your game is able to post this information before the driver crash or if non-accelerated mode is selected?

[James C. Smith]

... discover a failure rate on XP of just 2.5%...What are you going to make of that then, software-only/DirectX zealots?

You guys must realized that Windows XP is not the only Windows operating system in common use by players of downloadable game.

I am not surprised OpenGL runs great on Windows XP. If you are willing to target XP, I don’t see a problem with using OpenGL. But you want to target the mass market casual gamer that is a very different story. I suspect even the indie gamer is a different storry.

So who has stats on what percent of paying customer of casual game or downloadable indie games are using WindowsXP?

Could you make a lot of money selling games to Windows XP players only? You sure could. Could you make MORE money if you included Max OS X support? You bet! Could you make even more if you also support Windows 2000, Windows ME, and Windows 98? Absolutely!

I bet there are more users of Windows 98 than Mac OS, yet Mac OS Xgets more attention than 98. Maybe it is just because it is easier to target Mac OS X then it is to target Win98. It depentds what tools, languages, and APIs you use. For me, Wind98 is easier to support because it is very similar to the features of WindowsXP used in my games. But if you game uses Open GL and is written in Java than I guess MacOS
X would be more similar to Windows XP than Windows 98 is.

[princec]

So who has stats on what percent of paying customer of casual game or downloadable indie games are using WindowsXP?

Me. The answer is here (http://www.puppygames.net/ultratron_10_failure_rate.html) and shows that just 53 people actually had a DOS-based windows product out of 1247 Windows downloaders. That's just 4.2% of the Windows population. And remember I don't target anyone specifically - everyone who downloads my games is a random Joe. There's no ads up, no portal presence to speak of. The notable lack of Windows 95 entries backs this up: my game doesn't run on 95, so no logs.

<edit>Just realised you said "paying" but I suspect the figures are closely related given the general way of things.

<edit2>Just so happens the code is identical for OSX and 98 so there's no difference whatsoever for me to support XP, OSX, 2k, 98, ME, etc.

Cas :)

[luggage]

I dunno Cas - I'd have thought you'd still get more of a hardcore audience to your site than the soccer moms?

[princec]

I'd have thought that for every soccer mom, she's got at least one member of the brood who "fixes the computer" for her every now and again... I just don't know how this assumption can be made based on the sales of casual games to this so-called "soccer mom" group. In fact I don't even know if they really exist. No-one's got any real stats on it as usual.

There's so many assumptions being made here it's almost not funny! Like - soccer moms exist. Or soccer moms are the main purchasers of casual cames. Or soccer moms don't have OpenGL drivers. Or soccer moms, whoever they are, outnumber all other groups of players so significantly. Or soccer moms don't download shooty robot games. Or soccer moms don't buy shooty robot games. No-one ever has any facts, just hearsay, all based on extrapolating a game style and sales on some portal or other :(

Cas :)

[svero]

Well one has to assume the big portals do demographic studies of their customer base. They believe customers buying are skewed towards women a bit and of a certain age range. It wouldnt do them any good to get this wrong. I expect it's pretty accurate and based on some solid measure. Still we're talking 55-45 and 60-40.. not 90-10

[princec]

Have you ever been asked your age and gender and IQ when buying a portal game?

Cas :)

[svero]

So I looked into this a bit and I was told that big companies buy demographic info from the credit card companies and that's how they know who's buying.

- S

[princec]

Again based on the spurious supposition that the person with the credit card is the one that the game is being bought for... imagine for example if Zuma was actually being played by 13 yr old boys. Well none of those boys have credit cards so who do they ask to buy the game for them? Mum of course. Etc.
<edit>Also - it doesn't say anywhere on a credit card application whether you're a "soccer mom" or not nor your IQ or whether you've got a job when you bought the game (in fact last I heard it was rather difficult to get a credit card if you weren't employed. Well it is over here anyway)

Cas :)

[luggage]

The situation is odd. The portals can't be completely wrong - they're making more and more money so they're doing something right.

Then again it's also self serving - they're aiming at the soccer moms and therefore they're getting more soccer moms.

[soniCron]

It's quite interesting, and seemingly uncommon, to have such a high Mac userbase. Your MacOSX users outnumber all Windows users by 4 to 1. That just doesn't seem right. I don't know what I'm talking about, of course, but it's been my impression that:

There are far more Windows users than Mac users.
The sale of Windows games almost always outsell Mac sales, but certainly don't lose 4 to 1.

[princec]

I've always enjoyed > 50% of my sales to Macophiles. I like to think it's something to do with artistic taste, intelligence and appreciation for excellent gameplay that the the chattering classes who own most of the Macs in the world have :)

Cas :)

[tolik]

I love princec ignoring specific questions.

[princec]

Smartass. Sorry, missed your post.

Yes, I've disabled nvogl.dll (renamed it) to test what happens when GL isn't present.

Cas :)

[tolik]

Yeah, but as I mentioned, the most common thing (at least on Win2k) isn't the absence of OpenGL, but a SOFTWARE renderer (that reference OPENGL32.DLL sitting in system32). Remember how I was pissed that I wasn't able to play Elvis? Are you checking for software OpenGL rendering? The game would perfectly run except the speed... 2 fps on P4 2.4. People who would encounter it will just close the game after 20 seconds (seeing just couple dozens of frames).

[princec]

We specifically check for software rendering, and throw an exception.

Cas :)

[James C. Smith]

I just don't know how this assumption can be made based on the sales of casual games to this so-called "soccer mom" group


I don’t have lots of statistics, just some anecdotal evidence. It may have been atypical at the time, and now it is a year old. But I was turned off by Open GL when I couldn’t run the OpenGL game we were developing on many of the computers I tried it on. My wife players lots of popular causal games on her computer but could not run Wik. I could not run Wik on my own laptop. My brother could not play it on his computer. These were all computers that were frequently used to play many downloadable games. I asked the customers on the Ricochet forums to play Wik and many of them could not get it to run. These were all people who purchased a game from us.

Some of this may have been bugs in out game. But I didn’t find any evidence of that. Every case I dug into revealed a computer that did not have Open GL drivers installed.

I have no idea if there really are soccer moms who buy games, I don’t know what drivers they have or what OS they use, I don’t know if they would be willing to update their drivers. But I personally found a lot of game players who could not play my game. Most of them would have no clue how to update drivers. Someone like my wife could have asked me (or some bloke) to update her computer for her, but she probably wouldn’t bother if it was just to get some free game demo to work. There are plenty of other free game demos that work without tinkering with the drivers.

Like I said, I don’t know if this is typical or if the situation has changed in the last year. For example, I now have a new laptop and my wife is about to get a new computer. But these were some of the events that make me hate OpenGL a year ago. It was really frustrating to have family and friends who played Ricochet but could not play Wik without mucking around with their OS configuration.

[arcadetown]

Cas, feel free to post info from data logs you collected with Ultratron here. As for #s please keep in mind at launch UT hit middle of our Top 20 downloads chart (http://www.arcadetown.com/game_download.asp) then slowed down from there.

We're more willing to accept risky titles here due to our user type, and a 2.5% rate is lower than expected, but even so it's still too high if your plan it to be taken seriously by real big guys. We do see a higher than usual support and return issue for GL games so overall I'd say avoid GL if possible.

[princec]

I still don't even trust that 2.5% figure. It should be at least as bad as OSX, so I'm tweaking the logging to see if I can get it more accurate. Will have more information in a month or so.

And you'll be needing the new version as well Brian - I'll get that to you asap.

I wish someone else would get some damned logs.

Cas :)