Advice on Anti-Aliasing Sprites?

[Sybixsus]

I have some sprites which look pretty aliased when they're drawn in-game. I'd like to clean them up a bit ( I'm using Photoshop ) but I'm not having a lot of luck. I tried selecting a 2-4 pixel border and applying a soft gaussian blur, but that doesn't seem to do it. If anyone has any better suggestions, I'd appreciate it.

[Ska Software]

It sounds like what's happening is that whatever graphics library (DirectX or OpenGL) you're using is picking up the blurred pixels that are close to the color key but not *the* colorkey.

If you're using DirectX, you could consider alpha maps with slightly blurred edges, but this is expensive (space- and processor-wise).

Otherwise, get rid of the gaussian blurred edges. Unless you're specifically using an alpha map, DirectX or OpenGL don't apply alpha based on how close a color is to the key, its either on or off.

Hope I answered the question, and made sense at that.

[Ste Pickford]

I have some sprites which look pretty aliased when they're drawn in-game. I'd like to clean them up a bit ( I'm using Photoshop ) but I'm not having a lot of luck. I tried selecting a 2-4 pixel border and applying a soft gaussian blur, but that doesn't seem to do it. If anyone has any better suggestions, I'd appreciate it.

Assuming everything is fine from a technical point of view in your code, what you are doing to your bitmaps still sounds pretty horrendous.

What I do with bitmap alpha channeled stuff is to draw everything at at least double the size it will appear in game. Say you have a 100x100 pixel sprite, I would draw this in photoshop at, say, 256x256. I would keep my work file at this size and not worry about anti aliasing or blurring. Its often a good idea to only use flat colours (depending on the graphical style), which makes re-editing the graphic much easier as you have clean boundries between each area of colour.

Then, when you're happy with your work, shrink the graphic to 100x100 and export it in the format you need for the game, and the resampling process generates free, perfect, anti-aliasing.

Another advantage to this method is that if you keep your work files at a pretty large size, you are free to change the size of your exported game graphics any time you like without having to redraw anything. Want that 100x100 sprite to be 120x120? Just re-shrink your 256x256 pixel original and re-export it!

Its also surprising how much more detail you can get into your exported graphics - effectively sub-pixel detail at times; fairly fine detail on your large graphic of survives intact on the shrunken version, and looks much better than you would ever be able to draw at the smaller size.

Or, you can be fairly sloppy and slapdash when drawing your large bitmap, sometimes the tiny little rough bits, which take time to clean up, are unnoticable in the shrunken version, saving time and effort.

[Sysiphus]

dunno,gab... the usual problem is...You'd better know where the gfx is gonna be put over...If is a black/dark backgound, do smash it to a similar or identical -better- layer, as when u would cut the thing, some antialiasing of that colors is what u blended in the borders: it'll never look like having dirty bright greys in the borders, if that's ur problem...If u have loads of diferent background colors, u can :
-do a version for each and load one or another depending on the background..
-smash the bg layer to the icon, and load as whole over the bg...
-if you can and looks well, do all to a dark or mid-dark grey, that one tends to look at least ok in most cases...

All this imagining that's the prob u mean...

rarely a blur will be a good idea in this case.In case needed, at least apply an strong unsharp mask later, often work averagely decently.


check that when u resize the image is NOT in indexed color mode. That will allways wreck it all when reducing or trying to make soft borders.

[vjvj]

Are you talking about aliased borders for alpha tested sprites/textures? Or aliased interiors? I'm guessing the former. If that's the case, preprocessing is going to make very little difference in image quality.

Only two solutions come to mind right now:

- Use more than one bit of alpha and apply a gradient alpha falloff as you near the edges. Render with alpha blending, and you should get reduced aliasing. Keep in mind that with this technique you will need to sort sprites by depth (if you have overlapping sprites), otherwise you might get artifacts.

- Implement MSAA in software. Could be a lot of work :)

In D3D/OGL land, this is a problem that has been screwing us up for years. Hardware support for Transparency AA just finally showed up a couple months ago (GeForce 7). So we can certainly empathize ;)

[Olivier]

Sorry to hijack a bit the thread but I wanted to know:
According to you game developers, are aliased sprites a serious issue? Do players really care?

[ggambett]

For what it's worth, I do what Ste Pickford says in our games, and the results look quite good.
According to you game developers, are aliased sprites a serious issue? Do players really care?
I'm not sure if they care, but I do think they believe they care. If every game looks fantastic but yours has jagged edges, players won't feel it's "as good as the others". Especially with the 30805283048 Zuma clones, it's hard enough to differentiate, so you should have *at least* what the others game have - and that includes anti-aliased sprites.

[Robert Cummings]

- how do you generate your sprites?
- premultiplied alpha or no alpha?

I render all my sprites at stupidy high resolutions then scale back down. Most of my stuff is digimatte, that is I will render some and paint some to get the desired effect. It is essential to have very high quality alpha channels. You don't feather or blur edges, keep them strong and use the alpha channel for soft edges.

[BantamCityGames]

Wait, I'm a little confused... are you guys talking about real in-game anti-aliasing? Or just anti-aliasing your prerendered graphics?

[Ste Pickford]

Do players really care?

I doubt most players understand enough about alpha channels to know whether they care or not - heh, even game developers have trouble discussing it amongst each other!

However, I'm sure players care about whether they like the look of a game or not, even if they don't understand the technical issues involved.

Its up to the developer to make the decision as to whether spending time working on anti-aliasing makes the game look better, and whether its worth the time and effort involved.

[Bad Sector]

Do a scan in the whole sprite and for each transparent pixel check if one of the following is true:

1. there are pixels above and at the right
2. there are pixels above and at the left
3. there are pixels below and at the right
4. there are pixels below and at the left

and if so set this pixel to be 50% translucent (or a percentage that looks good to you anyway). The color will be the average of the surrounding non-transparent pixels.

This will increase the quality of the outline of your jaggied sprites a bit (however it produces better results when used with fonts... :-P so you may want to modify the checks a bit for better results).

I used the above in a program i wrote three years ago and it had very good results :-)

[soniCron]

Do a scan in the whole sprite and for each transparent pixel check if one of the following is true:

1. there are pixels above and at the right
2. there are pixels above and at the left
3. there are pixels below and at the right
4. there are pixels below and at the left

and if so set this pixel to be 50% translucent (or a percentage that looks good to you anyway). The color will be the average of the surrounding non-transparent pixels. This is nothing more than what he's already doing (blurring). There's no good way to anti-alias a raster sprite at its native resolution. Your best bet is to blur the alpha mask slightly (0.25 - 1 pixel) and increase its contrast while decreasing the brightness. This will create an anti-aliased mask but with some loss of edge definition. (Sharp and thin extrusions will get rounded off, etc.)

Your only other automated, high quality option would be to create a vector mask from the alpha mask, adjust and tweak as necessary, and finally eliminate the raster alpha mask and extrude the color data while rendering the vector mask to rasterize a new RGBA sprite. Unfortunately, this takes a lot of proprietary programming which is far from easy to write. I'd go with the first method, but again, there's no easy way to do this once you've created the sprite. ;)

Finally, you can just "smear" pixels by hand, but this is far from automatic and quite time consuming! :)

[oNyx]

The best option is of course to create the art with tools which give you AA for free. That means no extra work and the AA will be pretty much perfect.

So either vector stuff (Moho (http://www.lostmarble.com/moho/index.shtml) looks nice... you can export animations as 32bit pngs) or pre-rendered 3d models.

And half automatic touch-up... well, thats quite complicated and doesnt look half as good. The best thing I can think of right now is...
-duplicating the layer
-blurring the lower one
-save in a 32bit format
-use a special tool to discard the alpha (I mean discard... not blending against a color [like this (http://kaioa.com/k/ASplit.java)])

[If you're using outlines you can just fill the transparent area with the outline-color]

Then back to the original...
-get the alpha channel
-scale up 400% (bi-linear)
-trace it (vectorize)
-export in 25% size (size of the original)

And finally combine the two images.

Source:
http://kaioa.com/k/mario1.png

Result:
http://kaioa.com/k/mario2.png

Pretty subtle difference, but it does look better than expected :)

Well, its rather long winded, but if you put all sprites into one image you only need to do it once.

[Bad Sector]

This is nothing more than what he's already doing (blurring). There's no good way to anti-alias a raster sprite at its native resolution.

No this not blurring! Blurring (as he described what he does in photoshop) the outline will blur each pixel in any case. I think i can express myself better with some images :-)

sprite_original.png is the original image i made and used :-)
sprite_gaussian.png is the sprite with a gaussian filter at the outline
sprite_mymethod.png is the same sprite filtered with the method i said above

See the difference? Gaussian (as he seem to used it) puts an "average" pixels in every case, around the sprite, while my method "emulates" what an artist could do to smooth the jaggies in the sprite (ok an artist could do it better :D but my method is closer to it than gaussian filtering).

sprite_compare.png is a comparison between the two methods: the first sprite uses my method, the second is the original and the third is the gaussian filtered.

[svero]

I generally just render out a mask with the sprite by lighting it with a white constant surface texture over a black background and re-rendering it. If I did have a sprite in 2d that wasn't antialiased and I wanted to antialias it I would create an alpha mask for it by selecting the area around the sprite and painting it pure white. Then I'd use a line drawing tool and edge the sprite with a white line with the antialiasing feature of the paint program. For a faster method I'd feather the selection and paint it black (or white depending on how the selection is inverted), but that doesnt produce as clean results.

[Robert Cummings]

Its up to the developer to make the decision as to whether spending time working on anti-aliasing makes the game look better, and whether its worth the time and effort involved.

It shouldn't take any extra time if done right from the beginning.

[lexaloffle]

sybixsus - perhaps you could post one of the sprites so that we know what you're dealing with?

Maybe slightly OT, but here's my method for automatic AAing (at code level) which I use a lot in my painting program:

For each pixel, find the difference between the four pairs of pixels neighbouring the pixel in question (i.e. abs(pixel(x-1, y-1) - pixel(x+1, y+1)) etc.

Find the lowest of these four values: i.

A high value of i indicates that the pixel lies on a corner. If the pixel was part of a straight line, or an area of even intensity, at least one of the gradients (the values we just calculated) would be very low.

For this result to be useful, we need to normalise it by the range of intensities in the local area. Find the difference between the lightest pixel in the 3x3 area and the darkest one: j

Now we can have a normalised measure of cornerness, k, which ranges from (0..1). k = i / j

The final step is to blur the pixel according to k - the higher k is the more colours from neighbouring pixels should bleed in.

This picture shows the original, the normalised corner detection, and the result:

http://www.lexaloffle.com/joseph/pixel/autoaa.gif

-J

[Sybixsus]

They actually were drawn at a much higher resolution and scaled down. My art skills are pretty low, so that's how all my art is done. That's why I was surprised that they looked so rough. I'm gonna have to have a play with the code, because I think I might be doing something stupid in code for the images to show up rough. They don't look aliased in Photoshop, so I'll need to check if I'm doing something stupid like using the wrong blend mode or scaling them ever so slightly with a floating point error. I appreciate all the tips. I'll work through them if I don't find a code error.

Just to clarify, I wasn't actually blurring the sprite itself, I was blurring a duplicate on the layer below just to get the soft pixels on the outside. But since it didn't make any difference, that backs up my "I've done something stupid. Again." theory.

[Indiepath.T]

I use a combination of Layer Mask and Alpha Channel to give me the aliasing I require (PhotoShop).

If you are scaling these down in-game, and I assume you are using BMAX, then make sure you have FILTEREDIMAGE and MIPMAPPEDIMAGE filters set when you load the image.

[Indiepath.T]

I've actually found a much nicer method, it's utilising a free dos app called hqx4. I used hqx4 to increase the size and smooth the image and then photoshop to remove the background (with the magic wand tool) and scaled down by 50%.

Original Image :- http://kaioa.com/k/mario1.png

New Image 100% larger and Antialiased :- http://www.indiepath.com/tim/mario2.png

Pretty darn nice I say.

[Sybixsus]

HQX4 is indeed very impressive, I've played with it before. ( I think it was you that put me onto it before, come to think of it. ) It only works with certain types of graphics though. With cartoony stuff it works great, but with other types of art, it can be less useful.

I'm still not sure what I got wrong, but I must have messed something up because I tore out a chunk of code and rewrote it, and the problem went away. I'm not scaling or rotating the images yet, so that wasn't the problem. Perhaps a stray blend mode.

[Indiepath.T]

You using FILTEREDIMAGE or MIPPMAPPEDIMAGE flags?

[soniCron]

I've actually found a much nicer method, it's utilising a free dos app called hqx4. I used hqx4 to increase the size and smooth the image and then photoshop to remove the background (with the magic wand tool) and scaled down by 50%. Save yourself some steps and use this Photoshop Action (http://www.solaristudios.com/anti-alias.atn) instead. No need for HQX4, and the results (http://www.solaristudios.com/anti-alias.png) are identical. Pretty darn easy, I'd say. ;)

[Indiepath.T]

That's nice, one more thing I just learnt about photoshop.