Hi, I'm trying to figure out how to perform a custom z-prepass in directx, but I'm not sure how to do it. I set my depth buffer as usual:

deviceContext->OMSetRenderTargets(5, m_pRenderViews, m_depthStencilView);

deviceContext->RSSetViewports(5, m_viewports);

but now how do I write data to ‘m_depthStencilView’? do I have to use the SV_DEPTH semantic in the pixel shader output? Similarly, how do I write data to the stencil buffer? I am not aware of a HLSL semantic that refers to the stencil buffer.

Since I am not doing a custom z-prepass as of now, does directx do this automatically? If so, at which point is it performed? Thanks.

Its been a while, but the basic idea for a z-prepass is this:

You render your entire scene once, but only to the depth-buffer. What that means is, in DX10 upwards, you can bind a null-pixelshader. So you render each object with its normal vertex-shader, but without any pixel-shader. You bind the depth-buffer that you wan to fill, with the normal settings like if you would render your scene (write+read, fail-if-behind-or-equal). And then once your scene has been processed once, this is your z-prepass. Oh, you should optimally also try to cut everything else for that pass. If you can, you keep position in a separate vertex-buffer and only bind this for this pass (unless you do texture-based vertex displacement of course).

The next point - your normal pass. You use the same depth-buffer, and now you render everything as usual. With a small modification - you set the depth-state to “read only", and set it to fail-if-behind (basically only fragments at the same z should pass).

Unless I'm really mixin something up, thats it. I don't think this is something that can be done automatically, but it can give you a good gain if done correct, especially if you have complex fragment-shader and lots of overdraw. The z-prepass as previously descripted is really cheap - no fragments are emitted at all, and if you can bind only the position than the restrictions on bandwidth are also pretty minimal. Then during your regular pass, you have to execute way lass fragments than you would have to do - of course this does depend on your scene (and is also pretty useless for deferred rendering if I recall correctly since the fragment-shader for the scene-pass there is already pretty cheap).

There is nothing really special about a z-prepass in that is just like any other render pass you would make with the deciding factor being the desired output of the said pass. In this case, the desired output is scene depth values. Stencil writes are usually enabled by render state(s) and if you are not really doing any stencil writes then all you have to do is just write depth in your fragment/pixel shader or just use a null pixel shader as the mentioned above.

@juliean @cgrant Thanks for your replies. What if I want to omit certain pixels based on transparency, for example to get accurate SSAO on foliage? As i understand this would be not possible with only the vertex shader.

I think you could technically use an alpha-clipped shader during your z-prepass, but it would need to be conservative (anything that is not fully opague is discarded). And also it would add overhead and potentially bandwidth (not exactly sure what the GPU does when you have no render-targets bound be a pixel-shader set). Just keep in mind in general that

a) you don't have to include everything in the z-prepass (like promit said, you can exclude everything that is alpha-blended)

b) with a z-prepass, you are spending an amount of time (X) to save some time later (Y). The more complex operations you perform in the Z-prepass (=the higher X is), the less time you actually save during your normal pass.

Using alpha-tested geometry with a Z prepass can work, but as already stated you'll need to measure whether the cost of rendering this geometry in the prepass actually helps (or hurts) performance.

The idea is still basically the same as before but with a few caveats:

• You need a pixel shader now. It should be as simple as possible - basically sample your alpha texture and discard the pixel based on the value you get back, otherwise do nothing.
• Since you have a pixel shader bound you can't rely on just enabling depths writes to be enough. You should also clear any bound non-depth render targets and disable writes to them (I don't remember the exact terminology in DX land).
• You'll want to consider rendering all alpha-tested geom after all normal opaque geom to limit pixel shader invocations.

thanks for your replies everyone very helpful!!!

Key_C0de said:
Do you just do a rendering pass on the DepthStencilView of all opaques, with `DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL`. Then for all other passes you `DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ZERO` and Z is magically

Basically, yes.

Key_C0de said:
Don't you have to sample this rendered depth texture later from a Pixel Shader later and use it for calculations or something similar?

in DX11, you can bind the depth-buffer as a shader-resource view. Thus you can just read that directly, eliminating the need to render a separate depth-texture at all. You can even reconstruct the (view-space) positions from this depth-buffer, saving a huge amount of bandwidth.