DirectX: Stencil buffer vs Depth buffer

[VengefulPastry]

For my latest DirectX non-trick:

I am having trouble with the stencil buffer not playing well with the z-buffer.

The stencil buffer works by not writing any pixel value of zero (black) onto the scene. The z-buffer works by not writing any pixel that is farther away from the camera than the pixel in front of it.

Here is the problem: when writing a texture to the scene, it seems all of the black pixels (total value zero) are nonetheless being given a depth value. So, if it just so happens this texture is closer to the camera than the next texture, all of those black pixels are being written to the final scene, because although they are black, they are closer to the camera....

Has anybody else experienced this, and did you come up with a solution? Bumbling through the MicroSoft archives, I have found you can alter behavior if the stencil passes but the depth fails, but I can't find out if there is something I can do if the stencil fails but the depth passes (which seems to be the case, here).

Thanks!

[C_Coder]

To my knowledge to write to the stencil buffer (SB) you have to enable it. Then anything you write will be written to the SB. Once you finish you use the SB to occlude some rendering...or is this the volume shadow technique I'm referring here...hmmm... :confused:

I never used the SB myself so I cannot help much. I only checked about it but never when in depth. Sorry! :)

[Sante]

IIRC, you can't enable and disable Z writting based in the result of the SB test. So you either write it all to the Z Buffer, or don't. Maybe I'm wrong, but I didn't see any method or option to allow that.

I'm not sure what effect you want to achieve, but you should take a different approach. Maybe using a 8-bit SB would give you more ground to play. It really depends on what you want to do.

Good luck with it!