Thomas, 2008-06-18, "Character version 2.0"

[Thomas]

In "normal" rendering called Forward Rendering you check what models a light intersects, render these, then check what models intersects with the next light render these and so on.

In deferred shading one renders all models to a special G-Buffer (which is bunch or textures storing normals, color, depth, etc). Now using this G-buffer one iterates the lights and draws them according to the information the G-buffer.

My main reasons for using deferred shading is because it makes it easier to maintain shader combinations, is faster when rendering many small (as in screen coverage) lights and makes it easier for the artist to optimize.

[eliasfrost]

Ok, thanks!

[starstutter]

HPL2 uses deferred shading

My main reasons for using deferred shading is because it makes it easier to maintain shader combinations, is faster when rendering many small (as in screen coverage) lights and makes it easier for the artist to optimize.

ahhhh, you're a step ahead of me

Personally I think forward rendering is simply becoming a thing of the past. It's complicates scene management and it's slower in practically all next-gen scenerios.

Just curious, I "came up" with a technique called deferred shadowing and with it I've been able to almost double my shadow map resolution and sample count with almost no real fps impact compared to forward shadow rendering. I've looked everywhere for any other game or engine that uses the techniqe but so far I simply havent found anything. I say "came up" because usually I dream up something and then later find out it's already made =/

[Thomas]

starstutter:

I do not think that deferred shading is a technique to rule them all. Forward shading is still very worthwhile and HPL1 will outdo HPL2 in certain kinds of scenes in terms of frame rate.

As for deferred shadowing, if you mean that you render the shadow map result(s) to texture and then using them in the light rendering then that is used by Crysis. That technique is mostly for forward shading though, so perhaps you mean something else.

[starstutter]

As for deferred shadowing, if you mean that you render the shadow map result(s) to texture and then using them in the light rendering then that is used by Crysis. That technique is mostly for forward shading though, so perhaps you mean something else.

I read the paper on the cryengine and while the techniqe is somewhat similar, it's not what I had done. There is no forward rendering of the shadowing results. By results I mean (and I guess you mean) is the final output from sampling the depth map. This techniqe is different, it works on the same principle as deferred lighting where not a single invisible pixel gets depth tested.

It's done in 3 fairly quick passes:

1. Render the depth scene from the lights point of view (whatever resolution that may be)

2. In a forward rendering pass, render all the geometry in your view fulstrum (from your point of view) to a seperate render target. As an optimization, you can preform culling for both the light's fulstrum and the camera fulstrum at the same time. To these surfaces, preform no depth testing, but use a floating point render target to record the SM coordinates to each surface (the 0 to 1 2d measurement of the projection matrix). R and G hold the sm coordinates, B holds the distance away from the shadowmap (in w coordinates of course). Unfortunatley this render target needs to be 4 channels with 32 bits for each to avoid precision problems with storing the depth.

3. In the final pass, render the shadow caster as geometry (same as you would do with light) and project the previously rendered scene onto it. In the shader, extract the SM coordinates and depth, and then compare. That may not be exactly clear instructions so:

float2 coords = tex2d(coordinateTexture, screenUV).rg;
float depth = tex2d(coordinateTexture, screenUV).b;
*then for sampling:*
float result;
for (i = 0; i < numsamples)
{
result = (depth < tex2d(depthMap, coords + jitteredDisk[i]))? 0:1;
}
result /= numsamples;

Personally, the results for me when moving to this method were:
Forward shadowing: 1024 sm with 8 samples : 65 fps
Deferred shadowing: 2048 sm with 11 samples + screen space (smart depth) blur : 62 fps

Thanks for taking the time to listen btw

[Thomas]

Kinda tired right now, but I fail to see the gain of your technique.

In #1 you do a simple Z only rendering using the light frustum. This is the first step of any shadow mapping. After that I simply do projection in the light shader and I do not understand how adding more steps will speed things up. Also your technique seems to not take advantage of hardware PCF.

I might be missing something though, so please correct me

[starstutter]

Also your technique seems to not take advantage of hardware PCF.

I might be missing something though, so please correct me

True, it doesn't take advantage of hardware PCF, but I have not attempted to modify it to use the method. Yes it does add an extra step and it is initially slower, but the idea scales with scene complexity and the number of shadow casters.

Using this method is the same theory behind deferred lighting where shadows are applied as a post-process. Yes, there is an initial extra step, but it is a very cheap one and saves loads of calculation for the complex shadow processing.

Like deferred lighting, the starting framerates are initially lower. Like in my example of the fps gains, I should have provided better comparison. I attempted to do a 2048 shadow map with 11 samples and a post blur with a forward techniqe, but the result leveled out to about 39 fps, wheras with the deferred shadowing it was 62 fps. This is basicly because so many texture samples are saved.

Because of the intermidiate step the basic form of the method is slightly slower. In the first test of deferred shadowing, I had a 1024 SM with 8 samples and no blur (as was the test with forward shadowing), the results were leveling out to 64 fps, one fps lower than the forward technique. So there is little benefit gained if lower resolution and sample counts are used, but as mentioned before the method scales extremley well with scene complexity and shadow quality.

As for the last part, maybe we are using slightly different methods for lighting. The way I do mine is by, drawing to g-buffers, then on the lighting pass drawing a geometric object on the screen that is the screen-space size of the light. Like a point light is a sphere, a spot light is a cone ect. In the shader I project the g-buffer onto the geometry using the cameras projection matrix (projective texturing) and preform lighting from there.

With the deferred shadowing, I do the same thing except I project the frame rendered in step #2 onto the geometry and the scene is not drawn again. So, no shadowing is actually applied directly to the scene geometry.

btw, please tell me if I begin to be annoying lol. I just want to suggest but not push anything.

[Thomas]

You are not annoying, I find it interesting hearing about new ideas

I still do not get your algorithm though. Here is how I do it (which is basically the "normal" way I guess):

1. Render geometry to shadow map in a Z-only pass.
2. Render the shadow casting light (using light geometry: sphere, cone, etc) to scene and use shadow map from #1 to calculate occlusion.
3. Discard the shadow map from #1.

Now as I understand your algorithm, after my step#1, you render the shadow map data and all world geometry again to a new 128 bit render target and then use that render target when rendering the light. Since your step #1 seems to be exactly like mine, I cannot see how you get a speed gain, unless the your technique greatly improves the performance of the light shader (which I fail to see).

Again, tell me if I am wrong about anything

[starstutter]

2. Render the shadow casting light (using light geometry: sphere, cone, etc) to scene and use shadow map from #1 to calculate occlusion.

I guess what I don't quite understand is your #2 step. You're saying you render the scene as the light geomerty, but obviously you have to re-draw the entire scene geomerty to get the projection coordinates and distance away.

From what you just said, it sounds as if you're testing the depth map against the scene without this data, so I'm obviously missing something 0_o. Do you mean that you are redrawing the scene geometry with the shadow shader and using cone geometry to cull it (such as the stencil or depth buffer)?

Anyway though, from the sounds of it, you're doing shadowing as a completley forward pass. This allows possiblity for overdraw, and on my graphics card, there is a noticable speed drop even without overdraw (probably due to vertex coordinate recalculation between pixel sets or something like that).

EDIT: I misunderstood what you were saying, fixed in the post below.

you render the shadow map data and all world geometry again to a new 128 bit render target and then use that render target when rendering the light.

Yes, this is the step we are kind of miscommunicating on. The 128 bit render target makes absolutley no reference to the depth map previously drawn. It could actually be an in entirley different shader all together. If you're like me you probably understand code better than description so let me psuedo-code the algorithem.


setup 2 render targets:
light depth map R32F (or could be shadowmap format)
coordinate map G32B32R32F (ehh, didn't type that exactly right)
#1.
VS() //vertex shader
{
out.position = in.position * (lightMatWVP);
}
PS()
{
out.depth = in.position.x / in.position.w;
}
render target = light.depth map

#2.
VS()
{
out.position = in.position * (cameraMatWVP);
worldPositon = in.position * MatW;
out.lightPos = worldPositon * (lightProjectionMatrix)
}
PS
{
----------- //transform shadow coordinates (forget the exact code)
float2 coord = ShadowCoordinates; //(ranging from 0-1 in 2 directions)
out.color.r = coord.x;
out.color.g = coord.y;
out.color.b = in.lightPos.z / in.lightPos.w ; //(the depth away from the light according to the lightProjection matrix)
}
render target = coordinate map
//just if you're curious, this texture if displayed normally should look like a rainbow colored pallate

#3.
To be clear, in this step there is NO scene geometry rendered, there is only 1 single cone in the shape of the shadow caster. The out.cameraPos variable is unnecassary, but I put it there for the sake of clarity that we are projecting a texture onto the geometry from the camera. Technically you could create this as a fullscene quad and not even need a vertex shader, but that would not take advantage of what the algorithem offers. The "jittered disk" is simply an array of float2's that are used to create a random sampling pattern (replaces banding with noise)

shader.SetTexture("coordMap", coordinate map);
shader.SetTexture("depthMap", light.depthMap);
VS() //just transforming the cone
{
out.position = in.position * (cameraMatWVP);
worldPositon = in.position * MatW;
out.cameraPos = worldPositon * (cameraProjectionMatrix)
}
PS()
{
----------- //transform camera projection coordinates
float2 fSMcoords = tex2D(coordMap, out.cameraPos).RG;
float depth = tex2D(coordMap, out.cameraPos).B;

float result;
for (i = 0; i < numsamples)
{
result = (depth < tex2D(depthMap, fSMcoords + jitteredDisk[i]))? 0:1;
}
result /= numsamples;
}
render target = mainSceneTarget

[starstutter]

OH OH OH! I get what you're saying.

In your step 2, you're rendering the light geometry, not shadow-specific geometry

"2. Render the shadow casting light (using light geometry: sphere, cone, etc) to scene and use shadow map from #1 to calculate occlusion."

Yes, now I see, you project the gBuffer onto the cone, calculate light and then mix that with a "shadowing" texture that had been forward rendered. What I'm trying to convey is that the shadow map has a g-buffer of it's own in the form of the "shadowCoordinate" texture, which does not end up looking black and white, but more rainbow colored like a normal or position buffer. This way the lighting and the shadowing coud be drawn on the exact same piece of geometry (and in the same pass I guess if you really wanted to, but that would make for some shader bloat).