Page 1 of 4 123 ... LastLast
Results 1 to 30 of 111

Thread: The 2006 2D Sprite Render Test

  1. #1
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default The 2006 2D Sprite Render Test

    Yes, I'm hoping for something REALLY valuable for reference in this thread.

    TGB, PTK, BlitzMax, Flash, Custom Engines, etc, etc. Alot of 2D tools have been released the past year. I would like to propose that a new sprite render benchmark be outlined so we can see some real world tests of various rendering engines and how they perform on our own equipment. After the program structure is agreed on, hopefully a few versions of the test would be coded and made available for others to download.

    Here's my basic program proposal we can start the discussion with:

    Resolution: 800x600x32. Should be able to run in window or full screen.
    Texture: A single texture used, 256x256 in size. (png or jpg)
    Sprites: 32x32. Inside the texture 16 different sprites would be defined.

    Program Design:

    Windows Platform, DirectX (any version) Preferably a "2D via 3D" solution.

    Vsync turned off so program can render as fast as possible.

    Up to 2000 sprites will be rendered.

    *Program starts off with 500 sprites.

    Left arrow key removes 100 sprites.

    Right arrow key adds 100 sprites, up to 2000 total.

    * When a sprite is reset or initialized, it should have a random start up screen position. A random image from the texture. And a random alpha value.

    Sprites move in a pong fashion around the screen, bouncing off the edges.

    Sprites move 1 pixel per frame update. (No time based code, etc.)

    Frame rate display should be toggled on/off by pressing the "F" key. When turned off, we can validate the frame rate by using something like Fraps, etc. Even though the Fraps I have only supports DX8 and up?

    Number of sprites on displayed should be toggled on/off by pressing the "S" key.

    ESC key exits program.

    I would love to see how TGB stands up to PTK or BlitzMax, etc, etc.

    Hopefully someone can design the basic texture to use.

    And finally, I hope this thread does not die with no interest.

  2. #2
    Moderator
    Join Date
    Jul 2004
    Location
    Zürich, Switzerland
    Posts
    1,973

    Default

    Sounds cool, although it may not be representative of a real game. There are other variables and other limitations to consider. The texture and the sprites are conveniently power-of-2, for example. I have code to deal with arbitrary shaped textures and arbitrary shaped images, which has a little performance penalty.

    The only big variable you left out is windowed vs fullscreen.
    Gabriel Gambetta
    Google Zürich - Formerly Mystery Studio

  3. #3
    Senior Member
    Join Date
    Feb 2005
    Location
    Fenton, MO
    Posts
    757

    Default

    Are the sprites opaque (alpha 255), transparent (alpha 255 or 0), or translucent (alpha 0, 1, ... 254, 255)?
    Bonnie's Bookstore - Casual Game Blogs (Multiple blogs by different developers) - My Game Dev Blog

  4. #4
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    I think that any system should be out of the box without any 3rd party modules or improvements. If not then the results could be way off depending on the code used.

    For example, here are some results of my Single Surface module for BMAX, the speed increase is compared against the standard product.

    DirectX - 40% Less Video Ram 28% speed increase
    OpenGL - 42% Less Video Ram 28% speed increase AND 57% faster than DirectX
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  5. #5
    Member
    Join Date
    Oct 2004
    Location
    Orlando, FL
    Posts
    52

    Default

    The biggest problem, IMO:
    With 32x32 sprites, these apps will likely be bound by the sheer number of pixels involved (fill-rate). The per-sprite cost--things like setting render states and setting up vertices--will be completely hidden. Why is this bad? Because, for any hardware-accelerated renderer, it's the per-sprite processing that really impacts the CPU, and most games are CPU-bound, not GPU-bound.

    One possible solution:
    How about buttons to double/halve the sprite size, from tiny (1x1 pixels) up to something absurdly large, say, 256x256? With larger sprites, you'll see the renderer's per-pixel efficiency (fill-rate). With smaller sprites, you'll see its per-sprite efficiency ("sprite-rate" ).

    One more thing:
    Don't cap the number of sprites at 2000. I'd again suggest buttons to double/halve the sprite count, from 1 up to something absurdly large, say, 1M.

    I might be willing to put up a DX8 version (w/ source) at some point. Indiepath is right, though--the biggest value of this benchmark will be comparing straight-out-of-the-box commercial engines.

  6. #6
    Senior Member
    Join Date
    Jul 2004
    Location
    Netherlands
    Posts
    245

    Default

    I think it would be more interesting to not have any requirements like (only Windows, must use DirectX, only use 1 single 256x256 texture, out-of-the-box functions, etc.), but just say what the program must do (display 500 moving sprites, etc.) and let people implement this in the best possible way (as long as you provide the source code). It might be interesting to see how OpenGL compares with DirectX, and Windows vs. Mac / Linux.

    What exactly are you planning to measure? The number of FPS for 500 sprites? I think it would be more interesting to measure max. number of sprites possible in 100 FPS for example. Then there should't be a limit of 2000 sprites.

    I'ld split this into:
    - test 1: transparent sprites, no alpha blending
    - test 2: translucent sprites with random alpha (if supported)
    so you can compare this with DirectDraw / GDI / older 2D libs as well. When choosing a system/library for a game, you'll probably have decided wether or not you're planning to use alpha blending.

    Some libraries might have very slow text functions (since you'ld normally use bitmap fonts instead of text), so I'ld suggest to only display the last FPS value for a moment when you press F (counted without any text on the screen), or use bitmap fonts and display it real-time.
    Mike Wiering
    Wiering Software - http://www.wieringsoftware.nl/

  7. #7
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    hmm.. the more complicated it gets, the more likely no one will ever code something up. I still think a simple sprite render test using 32x32 sprites with random alpha sprites moving around via a pong style would be good enough. I also think any method is relevant, so if you have your own mod to an engine or whatever, let's see that as well. Source to me is not manditory. I'm sure if someone writes a demo that is performing ALOT faster than everyone else's inquiries will have to be made.

    I guess allowing more then 2000 sprites is fine, but probably would only matter on higher end computers.

  8. #8
    Senior Member
    Join Date
    Jul 2004
    Location
    Seattle, WA
    Posts
    1,316

    Default

    I think this is a great idea...

    Coincidentally enough, I've already written most of this test to run under PTK, when I was evaluating it. I think the sprite I used is only 16x16, but they bounce around the screen pong-style... and I can adjust the number of sprites and change between fullscreen and windowed on the fly.

    Dxgame, since this is your idea... maybe you should write a template C++ program to run the test. Just leave some nice comment blocks where any library-specific code needs to go.

  9. #9
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    You might also want to test with a couple'a hundred thousand sprites since GPU tech nowadays is going to cope with pretty much anything you chuck at it.
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  10. #10
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    couple'a hundred thousand sprites? jeesh.. I pretty much run only on older gpu/cpu systems with onboard graphics capabilities.

    "Dxgame, since this is your idea... maybe you should write a template C++ program..."

    I'm a VB6 person, sorry. I am going to throw together a demo to get things rolling. I'm sure my engine will probably be the slowest, because it's so high level, but oh well. Should have something ready later today.

  11. #11
    Senior Member
    Join Date
    Feb 2005
    Posts
    2,069

    Default

    I have written a prototype of 2D engine that fills exactly these conditions (complete with the pong-like collisions) in JAVA (using JOGL for OpenGL).
    I'll see if I have enought time to clean it up and upload it.
    The 2000 sprites barrier should be no problem I seem to remember testing with up to 3000 alphaed sprites on an AMD 2600+ with GF ti4200.

  12. #12
    Member
    Join Date
    Apr 2006
    Location
    Washington DC
    Posts
    53

    Default

    This is a great idea, but here's the rub; the harder you try to compare oranges to oranges, the further the stats drift from real-world performance.

    I think the most doable way to approach it would be to start with a reference implementation that everyone can copy and a set of assets that everyone can share. I'm currently moving my GDI+ vector graphics code to 2.5D using Direct3D, and I'd be happy to write a managed (C#/.NET) version.

    Don't use random positions, images or alpha values. The position, texture, alpha, etc. of each sprite should be identical each time the program is run, like 3DMark, with no user user input during the benchmark. You could put all the relevant sprite info into an XML or comma-delimted file to be sure everyone is using the same values.

    It would also be helpful to have the benchmark run a series of tests in multiple resolutions and windowing modes.

  13. #13
    Senior Member
    Join Date
    Feb 2005
    Posts
    2,069

    Default

    Quote Originally Posted by PaulModz
    This is a great idea, but here's the rub; the harder you try to compare oranges to oranges, the further the stats drift from real-world performance.

    I think the most doable way to approach it would be to start with a reference implementation that everyone can copy and a set of assets that everyone can share. I'm currently moving my GDI+ vector graphics code to 2.5D using Direct3D, and I'd be happy to write a managed (C#/.NET) version.

    Don't use random positions, images or alpha values. The position, texture, alpha, etc. of each sprite should be identical each time the program is run, like 3DMark, with no user user input during the benchmark. You could put all the relevant sprite info into an XML or comma-delimted file to be sure everyone is using the same values.

    It would also be helpful to have the benchmark run a series of tests in multiple resolutions and windowing modes.
    I think the original idea was just to see different implementations in action, what would be the point to just copy and compile the same thing for everybody?

  14. #14
    Senior Member
    Join Date
    Jul 2004
    Location
    Netherlands
    Posts
    245

    Default

    Quote Originally Posted by PaulModz
    Don't use random positions, images or alpha values. The position, texture, alpha, etc. of each sprite should be identical each time the program is run, like 3DMark, with no user user input during the benchmark. You could put all the relevant sprite info into an XML or comma-delimted file to be sure everyone is using the same values.
    Or use a very very simple random generator. That's alot easier than reading values from a file. Something like:
    Code:
    Global Seed = 0
    Function Random (X)
      Seed = (101 * Seed + (Seed Shr 5) + 5659) And $FFFFFF
      Return Seed Mod X
    End Function
    Mike Wiering
    Wiering Software - http://www.wieringsoftware.nl/

  15. #15
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    Looks like it's hard for developers to agree on the simple things sometimes. In any event, here's a little demo I wipped up that shows what I was talking about:

    http://www.dxgame.com/download/SpriteTest.zip (400k)
    Requires Windows XP, DirectX8, VB6 runtimes (already on XP systems)

    F: Toggle Framerate Display
    S: Toggle Sprite count display
    Right Arrow: Add 100 sprites (up to 2000)
    Left Arrow: Remove 100 sprites
    R: Init sprites with rotation
    ESC: Exit

    Each time the sprites are initialized, a random sprite image is defined from the texture, a random alpha value, a random direction to Pong around the screen with.

    I threw the rotation option in just to emphasize a real usage scenario because I'm always rotating sprites in my games.

    I already know my engine is slower than others, but I'm mainly interested to see how TGB, PTK, Blitz, etc, perform in a similar test.

  16. #16
    Member
    Join Date
    Oct 2004
    Location
    Orlando, FL
    Posts
    52

    Default

    http://www.ericology.com/temp/SpriteMark_1_0.zip (537k)

    This is DX8.1 and C++. I wanted to see if I could reach my Radeon 9600 Pro's advertised peak fill-rate of 1.6 gigapixels/sec, so I went kinda overboard with the render-state options. I got up to 1.3, meh, pretty close.

    Note to benchmarkers: Even with the full display turned off, my app's framerate is still displayed top-right (only every 6th frame, to save time!)

    I haven't compatibility-tested my D3D framework, so I apologize in advance if this crashes on certain cards.

    As dxgame predicted, his VB6 version is slower than my C++ version. Here is the comparison on my system (2000 32x32 sprites, windowed):
    dxgame: (deleted, see my later post for correct results)
    undersan: (deleted, see my later post for correct results)
    Update: My initial testing was slightly flawed. I had the DirextX debug runtime libraries enabled--d'oh!
    Last edited by undersan; 06-24-2006 at 02:31 PM.

  17. #17
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    Hi Undersan! Thanks for coding your example.

    "As dxgame predicted, his VB6 version is slower than my C++ version. Here is the comparison on my system (2000 32x32 sprites, windowed):
    dxgame: 56 fps
    undersan: 190 fps"

    hmm... this is a pretty big difference so I ran some tests on my system(s) to compare yours, especially ingteresting since they are both using DX8!

    My results were radically different than yours:

    On my Intel Extreme Graphics Card my VB6 engine actually performed better than your C++ version, in both windowed and full screen with 2000 sprites.

    dxgame: 40fps
    undersan: 39fps

    I then tested it on my Raiden Express 200 Series computer, results were slightly faster but both engines were VERY close again:
    undersan: 69fps
    dxgame: 65fps

    THIS is the reason these kind of tests are great. For some reason my engine is performing poorly on your Radeon 9600 Pro compared to your engine. Perhaps there is a hardware render state I should be setting when running on a 9600 pro?

    Thanks again, lets hope some others chime in. This could be very interesting!

  18. #18
    Senior Member
    Join Date
    Jul 2004
    Location
    Netherlands
    Posts
    245

    Default

    Still, I think displaying the text influences the results too much in dxgame's program: with 100 sprites, I get about 320 fps if the "100" is displayed and 455 if not, but that is with fps display on. The moment I press F, it shows 550 or more for a second.
    Mike Wiering
    Wiering Software - http://www.wieringsoftware.nl/

  19. #19
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    "Still, I think displaying the text influences the results too much.."

    If I was using bitmapped fonts the performance hit would have been alot less, just a few more sprites then. In this example I'm using the standard DX8 Text Rendering which is extremely slow and should never be used for anything serious in my opinion. I suggest using a 3rd party fps displayer like Fraps to help verify frame rates.

  20. #20
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    Here's the BlitzMax (DX7) Version : http://www.indiepath.com/tim/SpriteTest.zip

    T: Toggle FullScreen/Window.
    F: Toggle Framerate Display
    S: Toggle Sprite count display
    Right Arrow: Add 100 sprites (the sky is the limit)
    Left Arrow: Remove 100 sprites
    R: Toggle Rotation
    ESC: Exit

    #################

    Here are my results for 2000 Sprites Windowed

    - BlitzMax : 196FPS
    - DX Game : 94FPS
    - Undersan :108FPS

    Here are my results for 2000 Sprites FullScreen

    - BlitzMax : 230FPS
    - DX Game : 102FPS
    - Undersan : 119FPS

    FYI, I can run 10,000 Sprites in BMAX at 60FPS.
    Last edited by Indiepath; 06-24-2006 at 03:13 AM.
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  21. #21
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    Thanks IndiePath for the BlitzMax version!

    hmm... I'm seeing a disturbing trend. We seem to be getting better performance on our own machines compared to running on other machines. IndiePath stated speeds better than DOUBLE my engine on his machine, yet on my machine the BlitzMax version runs slower than the 2 previous demos. The attached screen grab shows it running at 36fps.

    "FYI, I can run 10,000 Sprites in BMAX at 60FPS." I got 10,000 at 10fps on my box.
    Attached Images Attached Images  

  22. #22
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    Results from another test machine

    BMAX: 40FPS
    DXGame: 35FPS
    Undersan: 23FPS
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  23. #23
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    More Results: (This is my highest end machine that I dare develop on.)
    AMD Sempron 3100+ 1.8ghz
    Geforce 4 MX 4000

    2000 Sprites Full Screen

    BMax: 76fps
    DXGame: 68fps
    Undersan: 46fps

  24. #24
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    I think I know why my initial results were very high. BMAX will use TnL if it's available, if not then it falls back to the standard HAL renderer (thanks Tim). The card on this machine supports TnL, my second test machine does not.
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  25. #25
    Senior Member
    Join Date
    Feb 2005
    Location
    Slovenia
    Posts
    1,004

    Default

    hmm... I'm seeing a disturbing trend.
    It seems your machines like your engines the most

    undersan I think you have a bug , when It says 4 sprites I see 3, when it says 8 I think I see 6, when 2 - 1 and when 2000 I see 1978 sprites

    ps. joking about the last one

  26. #26
    Senior Member
    Join Date
    Jul 2004
    Location
    Netherlands
    Posts
    245

    Default

    Quote Originally Posted by jankoM
    undersan I think you have a bug , when It says 4 sprites I see 3...
    Maybe there is one running around with alpha 0?
    Mike Wiering
    Wiering Software - http://www.wieringsoftware.nl/

  27. #27
    Senior Member
    Join Date
    Mar 2005
    Posts
    267

    Default

    "TnL" Ok, I'm thick on this one, what's TnL?

  28. #28
    Senior Member
    Join Date
    Aug 2004
    Location
    Dorset, England
    Posts
    999

    Default

    Quote Originally Posted by dxgame
    "TnL" Ok, I'm thick on this one, what's TnL?
    Hardware transformations and lighting.
    Indiepath Ltd
    "do good things - make money"
    And that is not the general opinion of Indiepath Ltd - etc... legal .... blah..

  29. #29
    Senior Member
    Join Date
    Feb 2005
    Posts
    2,069

    Default

    Quote Originally Posted by Mike Wiering
    Still, I think displaying the text influences the results too much in dxgame's program: with 100 sprites, I get about 320 fps if the "100" is displayed and 455 if not, but that is with fps display on. The moment I press F, it shows 550 or more for a second.
    I might very well be the consequence of a bug (eg fps not measured when writing is OFF for example)

  30. #30
    Member
    Join Date
    Nov 2005
    Location
    Cleveland, OH
    Posts
    66

    Default

    I tried various numbers:

    At 2,000 sprites:
    Blitz - 506
    DX -103
    Undersan - 423

    At 1,000 sprites:
    Blitz - 885
    DX -227
    Undersan (1,024 sprites) - 620

    At 500 sprites:
    Blitz - 1430
    DX -346
    Undersan (512 sprites) - 680

    At 100 sprites:
    Blitz - 2750
    DX -603
    Undersan (128 sprites) - 720

    At 10,000 sprites on the Blitz demo, I got 104fps.

    My system specs are 3.2 Ghz Pentium (dual CPUs), 4 Gig ram, 256M nVidia geForce 6800.

Page 1 of 4 123 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •