The 2006 2D Sprite Render Test

[Mike Wiering]

I just made a version for OpenGL (in Delphi), but I can't disable vsync and use double buffering at the same time, so I made two different versions. The version without double buffering blinks all the time, but the other one runs at your desktop refresh rate.

www.wieringsoftware.nl/tmp/sprites.zip

Results:

                          100     500    1000    2000    4000
-------------------------------------------------------------
VB              window    460     226     140      80     n/a
(dxgame)         full     428     216     138      80     n/a
-------------------------------------------------------------
C++/DX 8.1 *    window    512     250     155      86      46
(undersan)       full     808     312     173      94      50
-------------------------------------------------------------
BlitzMax        window    636     342     216     126      68
(indiepath)      full    1150     462     266     144      76
-------------------------------------------------------------
OpenGL          window    912     408     240     133      70
(wiering)        full     950     425     252     139      73
-------------------------------------------------------------
Java (princec)   full     760     280     158      85      45
-------------------------------------------------------------

* 128/512/1024/4096 instead of 100/500/1000/4000

IndiePath, maybe you could compile your version for OpenGL and see if that makes a difference.

[dxgame]

This is all very interesting. This has shown me that at least my rendering methods are not taking advantage of certain hardware abilities in some video (newer??) cards. But while on older cards, onboard cards, the overall performance in all of the demos seems to be relatively level.

[dxgame]

Just did a fresh reboot from my main dev box for updated results:

Intel OnBoard Extreme Graphics
Celeron 2.93ghz

2000 Sprites

Undersan (C++/DX8): 36fps
IndiePath (BlitzMax/DX7): 35fps
DXGame (VB6/DX8): 34fps
Wiering(Delphi/OpenGL): 28fps

[princec]

Hey, give this Webstarted one a test:

Java sprite test with LWJGL

(Should run on Windows, MacOS PPC & Intel, Linux x86)

Cas

[Mike Wiering]

Hey, give this Webstarted one a test:

Java sprite test with LWJGL

(Should run on Windows, MacOS PPC & Intel, Linux x86)

Cas

I've updated my post above with the results.

[princec]

oh yeah - whack enter to change to windowed mode.

Cas

[Pyabo]

Still, I think displaying the text influences the results too much in dxgame's program: with 100 sprites, I get about 320 fps if the "100" is displayed and 455 if not, but that is with fps display on. The moment I press F, it shows 550 or more for a second.

Interesting. That's a whole 'nother metric to test now... the speed of the built-in text rendering methods in various engines.

[Pyabo]

THIS is the reason these kind of tests are great. For some reason my engine is performing poorly on your Radeon 9600 Pro compared to your engine. Perhaps there is a hardware render state I should be setting when running on a 9600 pro?

I thought the point was comparing higher-level engines? I really don't want to be forced to worrying about things like this... that's why I'm using PTK (or BlitzMax, PopCap) to begin with!

[undersan]

Thanks to everyone who has contributed! To anyone posting results, I'd appreciate it if you'd specify your video card (most of you have).

princec, can you double-check that your app is using 32-bit color throughout? I notice some banding on the little blue-and-pink tile.

Here are my results:

windowed/fullscreen framerates for 2000 32x32 sprites
nt = not tested by me
for wiering, I used sprites.exe.

Radeon 9600 Pro
dxgame      91/91
undersan    204/234
indiepath   210/232
wiering     171/188
princec     165/165

Geforce4 Ti 4400
dxgame      100/98
undersan    183/216
indiepath   201/220
wiering     nt/nt
princec     nt/nt

Geforce2 GTS
dxgame      70/72
undersan    54/60
indiepath   75/91
wiering     72/82
princec     nt/91

I thought the point was comparing higher-level engines?

I agree. It would be nice to see some commercial engines here. As far as indiepath's BlitzMax-powered app, I'm unclear whether he's using his customized version of BlitzMax, or the "stock" version.

[princec]

I think step 2 now is to use sprites from 4 different textures (randomly) and have them all rotating and moving with floats. And animate each sprite each frame to use a frame from each texture one at a time in sequence

Cas

[dxgame]

"I think step 2 now is...."

We STILL need TGB, PTK, demos here first before moving on.

Cas - your web app fails on my machine. But your games run fine on this box.

"I thought the point was comparing higher-level engines? I really don't want to be forced to worrying about things like this... that's why I'm using PTK (or BlitzMax, PopCap) to begin with!"

Agreed, any TGB, PopCap, PTK coders here??????

[Jack Norton]

PTK coders are busy finishing their 10th released game...

[princec]

Cas - your web app fails on my machine. But your games run fine on this box.

Any stacktraces to hand? (Java Preferences, Advanced, Show Console etc.) Works fine on 2 different Macs and a PC here... assuming the PC has OpenGL drivers, right?

Cas

[Indiepath]

OpenGL BMAX version : http://www.indiepath.com/tim/SpriteTestOGL.zip

BMAX DirectX - 196/230
BMAX OpenGL - 258/308

[Mike Wiering]

oh yeah - whack enter to change to windowed mode.

Cas

Unfortunately, I can't read the frame rate in windowed mode...

[oNyx]

Unfortunately, I can't read the frame rate in windowed mode...

Broken driver. Recreating the context and reloading the textures would workaround it tho.

[princec]

Broken driver. Recreating the context and reloading the textures would workaround it tho.

I'm far too lazy to workaround driver bugs Besides, my sprite engine's being blown into the weeds by BlitzMax. Not that BlitzMax does half the nifty things mine does

Cas

[Emmanuel]

Okay, it's sunday morning, I'm in a hotel and I wasn't awake enough to work on our new title before I go out for lunch, so, here's a ptk version:

SpriteTestPTK

Open up options.ini to switch from D3D (default) to OpenGL and from windowed (default) to fullscreen.

I'll let you measure and update the charts, as measurements on my own machine are meaningless to you. The app starts at 2000 sprites but you can go up to 50,000.

It'd be nice to add some useful metrics to the chart:

Development time -- 35 minutes.
Portable to mac -- yes.
Field tested -- yes.
Total cost of tools -- $215.

OpenGL is nice but D3D is a lot more compatible if you're serious about releasing actual games on Windows.

Best regards,
Emmanuel

[Mike Wiering]

Here are my updated test results. I've also made a version for DelphiX (new version with hardware accelleration by Jaro Benes), which you can find here: DelphiXTest.exe.

Computer: P4, 2.0 GHz
Video card: NVIDIA GeForce4 MX 460
OS: WinXP

                          100     500    1000    2000    4000
-------------------------------------------------------------
VB          DX  window    460     226     140      80     n/a
(dxgame)         full     428     216     138      80     n/a
-------------------------------------------------------------
C++/DirectX     window    512     250     155      86      46
(undersan)       full     808     312     173      94      50
-------------------------------------------------------------
BlitzMax    DX  window    636     342     216     126      68
                 full    1150     462     266     144      76
            GL  window    596     292     180     102      54
(indiepath)      full     676     380     214     116      60
-------------------------------------------------------------
OpenGL      GL  window    912     408     240     133      70
(wiering)        full     950     425     252     139      73
-------------------------------------------------------------
Java (princec)   full     760     280     158      85      45
-------------------------------------------------------------
PTK         DX  window    256     252     202     115      61
                 full      85      85      85      85      63
            GL  window     85      85      85      83      42
(Emmanuel)       full      75      75      75      75      74
-------------------------------------------------------------
DelphiX     DX  window    544     235     137      75      39
(wiering)        full     578     248     145      79      40
-------------------------------------------------------------
NC (Phil    -   window     63      32      20      11       6
Steinmeyer)      full      63      33      20      11       5
-------------------------------------------------------------
Blitz3D         window    253     210     160      95     n/a
(electronicStar)
-------------------------------------------------------------
Asphyre         window    650     366     237     139      76
(DraculaLin)     full    1712     684     395     214     111
-------------------------------------------------------------

* 128/512/1024/4096 instead of 100/500/1000/4000

[aaron]

Broken driver. Recreating the context and reloading the textures would workaround it tho.

I had the same issue with it on my machine. I've got a geforce4 mx440 running the latest nvidia drivers (fresh XP install yesterday).

Worked ok full screen but wouldn't work in windowed mode.

[Chozabu]

OpenGL is nice but D3D is a lot more compatible if you're serious about releasing actual games on Windows.

Ok, D3D is nice(well, not imo tbh...), but OpenGL is alot more compatable if i want to try and run the games im developing!
i guess if youre developing an app for linux, its not such a prob to use DirectX
cross compile only, and run with wine (or link with winelib...)
the installer/package can easily do a startup script that runs it through wine

though, to be sure it makes sense to have dual rendering backends

[Phil Steinmeyer]

OK, here's mine:

http://www.newcrayon.com/files/NCSpriteTest.zip

Mine's a custom (proprietary) engine, that's entirely SOFTWARE based (no use of HW Accel).

On my machine (AMD 4600+, Radeon X300 SE vid card), with 2000 32 x 32 sprites, with alpha (on the sprite as a whole), I get 32 fps, in both full screen and windowed. For reference, the BlitzBasic example gets 58, though I think it's using HW accel

(Oh - I used my own texture, but it's the same size as the others - feel free to save the other texture on top of mine, in \graphics\texture.jpg)

Anybody else got a SOFTWARE engine to test?

[Slayerizer]

Intel 3.2ghz HT
Nvidia 7800GS (agp)

I did the test with 2000 sprites for each demo

-------------------------------------------------------------
VB          DX  window    145
(dxgame)         full     145
-------------------------------------------------------------
C++/DirectX     window   473
(undersan)       full     480
-------------------------------------------------------------
BlitzMax    DX  window    520
            DX  full    530
            GL  window    892
            GL  full    1020
-------------------------------------------------------------
OpenGL      GL  window    894+
(wiering)        full     950+     
-------------------------------------------------------------
PTK         DX  window    253
                 full      253     
            GL  window     256
                 full      256
-------------------------------------------------------------
DelphiX     DX  window    820
(wiering)        full     871
-------------------------------------------------------------
NC     soft  window    30
         soft  full    31

[Pyabo]

PTK         DX  window    256     252     202     115      61
                         full      85      85      85      85      63

Hmmmm... sure looks like you're vsync limited on the full screen runs.

And is this app using your batching update, Emmanuel?

[Indiepath]

OK, here's mine:

http://www.newcrayon.com/files/NCSpriteTest.zip

Mine's a custom (proprietary) engine, that's entirely SOFTWARE based (no use of HW Accel).

On my machine (AMD 4600+, Radeon X300 SE vid card), with 2000 32 x 32 sprites, with alpha (on the sprite as a whole), I get 32 fps, in both full screen and windowed. For reference, the BlitzBasic example gets 58, though I think it's using HW accel

(Oh - I used my own texture, but it's the same size as the others - feel free to save the other texture on top of mine, in \graphics\texture.jpg)

Anybody else got a SOFTWARE engine to test?

Runs at 5FPS here where as the BMAX version was almost 200FPS. However there is one thing I forgot to mention, the BMAX (DirectX) version will fallback to software if it can't detect a 3DCard - it also sounds like the BMAX version maybe syncing to refresh on your machine for some odd reason.

[dxgame]

OK, here's mine:
http://www.newcrayon.com/files/NCSpriteTest.zip
Mine's a custom (proprietary) engine, that's entirely SOFTWARE based (no use of HW Accel).
On my machine (AMD 4600+, Radeon X300 SE vid card), with 2000 32 x 32 sprites, with alpha (on the sprite as a whole), I get 32 fps, in both full screen and windowed.

Hi Phil,

Very cool demo but I wonder if it's relevant to consider a software based render....in 2006/2007? On my main dev box which is a 2006 HP Pavillion, 2.93ghz Celeron with onboard Intel Extreme Graphics, your app runs at 13fps. What's the advantage of using a software based engine is it takes an AMD 4600+ to get a decent frame rate? I apologize if I'm a little thick here, but wouldn't targeting a 400mhz CPU, Geforce 1/2/3 give you better performance at nearly the same overall compatbility level?

[dxgame]

I brought out one of my older test machines:

Emachines ETower 400ix
Windows XP
400mhz celeron
Geforce 2 MX/MX 4400

2000 Sprites 32bit color Windowed Mode

BlitzMax - 40 fps
SpriteTest PTK - 30fps
VB6 DXGame - 30fps
SpriteMark - 26fps
NC Sprite Test (software) 2fps

[Phil Steinmeyer]

Hi Phil,

Very cool demo but I wonder if it's relevant to consider a software based render....in 2006/2007? On my main dev box which is a 2006 HP Pavillion, 2.93ghz Celeron with onboard Intel Extreme Graphics, your app runs at 13fps. What's the advantage of using a software based engine is it takes an AMD 4600+ to get a decent frame rate? I apologize if I'm a little thick here, but wouldn't targeting a 400mhz CPU, Geforce 1/2/3 give you better performance at nearly the same overall compatbility level?

Well, I don't push 2000 alpha-blended sprites for my main game.

Most casual games (including mine) use a number of sprites in the low 100s. IIRC, my previous game (Bonnie's Bookstore), ran at 30 fps on pretty much anything that was ~800 Mhz CPU or better, and I think my current game will be about the same (though I really haven't done much low end testing yet). For that matter, running at 15 fps on really low end stuff (i.e. your 400 Mhz Celeron) is acceptable with non-action titles such as my own.

So it's fast enough, and by using software:

1) Compatibility is essentially 100% - show me a hardware accel game without SOME compatibility issues (especially given that many of our users have old systems, with old drivers, etc.)
2) Lower test times (see #1)
3) Fast to develop for
4) Easy to develop unusual pixel FX for. If I can think it up, I can implement it, and it works on all machines. So I can easily get some interesting looks that will hopefully be novel for the user.

Will it ever hit 400 fps? No, but nobody really cares about that except fellow programmers. 100% compatibility, ease of dev. and testing, and flexibility are the key issues.

[Slayerizer]

Hi Phil,

Very cool demo but I wonder if it's relevant to consider a software based render....in 2006/2007? On my main dev box which is a 2006 HP Pavillion, 2.93ghz Celeron with onboard Intel Extreme Graphics, your app runs at 13fps. What's the advantage of using a software based engine is it takes an AMD 4600+ to get a decent frame rate? I apologize if I'm a little thick here, but wouldn't targeting a 400mhz CPU, Geforce 1/2/3 give you better performance at nearly the same overall compatbility level?

On a DELL laptop with a crappy intel adapter I got 30fps with Phil's software engine. I certainly dont think that someone need to draw 2000 alpha blitted sprites each frame. Without a doubt, a good software engine is a plus for me.

I got really good scores when doing the hardware accelerated tests but doing games that rely on 3d accelerated engines do have additional problems.

- you need to re-arrange your tiles to match specific texture size
- watch out for old low memory video adapter

[princec]

If you wanted to write one of my sorts of game though you'd be a bit buggered by the abysmal fill rate though... and we still maintain 60fps is the minimum acceptable

Also quite probable a software renderer would be even slower on OSX due to the fact that the surfaces are effectively then fed back through OpenGL when you could be going direct.

Cas