3D HBAO+ Normal Map Artefact Fix
HBAO+ is clearly the best AO option in 3D, and it looks great on vegetation, but it's not quite perfect on larger flat surfaces with detailed normal maps. I spent some time tonight working on it in Far Cry 4 and have sucesfully been able to fix it: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67096/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67097/[/img] That is the worst case scenario for HBAO+ - a large wall with a lot of high contrast detail in it's normal map, and the camera not looking directly at it. It can be a rather difficult effect to find all it's shaders with hunting alone since it operates in so many separate passes - I recommend using frame analysis to find it and looking for a sequence like this (alternatively you might have some luck if you can identify the vertex shaders and use ShaderUsage.txt to find other matching pixel shaders). HBAO+ always starts out by scaling the depth buffer into a W buffer. Once I have identified this shader I will set it as a trigger to start dumping at this point in further analysis, saving having to wait for everything before it: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67098/[/img] That might be a bit clearer if I rescale it to maximise the dynamic range: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67099/[/img] A few shaders later I see it output a normal map of the scene: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67103/[/img] But wait! You might have missed it, but there's something very important hidden in the alpha channel of that image (which you will only get with dump_rt_dds - the jps versions don't have an alpha channel). Let me extract it and boost the contrast a little: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67100/[/img] Ah ha! This is the first time we see the broken effect in the frame analysis dump - that tells us that this is a likely candidate for a shader that will need fixing. Moving on there's another shader that is called a whole bunch of times taking that broken texture (and the W buffer) as an input... but unfortunately we don't see an o0 for it (it's actually writing out Texture3Ds, which frame analysis doesn't support yet). The next time we see an o0 it looks like this: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67105/[/img] But looking closer at this shader we notice this is in it's ps-t0 input: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67101/[/img] That particular texture is broken as well. Compared to the output of this shader it's equally as broken, suggesting that this shader might be ok. But, although it's hard to tell, it is more broken then the previous broken one we found, suggesting that there's another broken shader between the last one and this. In fact it is - the shader that is called a bunch of times that we didn't see an o0 for is broken. More dumps from the frame analysis to give you an idea of what it's doing (the next two had an alpha channel that I replaced with black for uploading - they are working out which parts of the image to shade and which to leave alone): [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67102/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67104/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67106/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67107/[/img] And fixed: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67108/[/img] The pattern to fix the shaders is here: https://github.com/bo3b/3Dmigoto/commit/6fcb2354435b866d3697b588c982d3f44c0ec552 It's fairly straight forward, pretty much just requiring the view-space correction formula a whole bunch of times, [s]but it does need the projection or inverse projection matrix copied in with 3DMigoto to fix accurately regardless of the FOV. It is still reasonable to fix without that since it won't matter as much as shadows if it's just approximate.[/s] Update: Actually, with my new fancy constant buffer debug shader I was able to identify that the required info is already available in the shader - See this version of the fix from WATCH_DOGS2 where I found it in -cb0[2].z: https://github.com/DarkStarSword/3d-fixes/commit/667bd4945503d3195eaaecc8caab5704f04ac316
HBAO+ is clearly the best AO option in 3D, and it looks great on vegetation, but it's not quite perfect on larger flat surfaces with detailed normal maps. I spent some time tonight working on it in Far Cry 4 and have sucesfully been able to fix it:

Image

Image

That is the worst case scenario for HBAO+ - a large wall with a lot of high contrast detail in it's normal map, and the camera not looking directly at it.


It can be a rather difficult effect to find all it's shaders with hunting alone since it operates in so many separate passes - I recommend using frame analysis to find it and looking for a sequence like this (alternatively you might have some luck if you can identify the vertex shaders and use ShaderUsage.txt to find other matching pixel shaders).

HBAO+ always starts out by scaling the depth buffer into a W buffer. Once I have identified this shader I will set it as a trigger to start dumping at this point in further analysis, saving having to wait for everything before it:

Image

That might be a bit clearer if I rescale it to maximise the dynamic range:
Image

A few shaders later I see it output a normal map of the scene:
Image

But wait! You might have missed it, but there's something very important hidden in the alpha channel of that image (which you will only get with dump_rt_dds - the jps versions don't have an alpha channel). Let me extract it and boost the contrast a little:
Image
Ah ha! This is the first time we see the broken effect in the frame analysis dump - that tells us that this is a likely candidate for a shader that will need fixing.

Moving on there's another shader that is called a whole bunch of times taking that broken texture (and the W buffer) as an input... but unfortunately we don't see an o0 for it (it's actually writing out Texture3Ds, which frame analysis doesn't support yet). The next time we see an o0 it looks like this:
Image

But looking closer at this shader we notice this is in it's ps-t0 input:
Image

That particular texture is broken as well. Compared to the output of this shader it's equally as broken, suggesting that this shader might be ok. But, although it's hard to tell, it is more broken then the previous broken one we found, suggesting that there's another broken shader between the last one and this. In fact it is - the shader that is called a bunch of times that we didn't see an o0 for is broken.

More dumps from the frame analysis to give you an idea of what it's doing (the next two had an alpha channel that I replaced with black for uploading - they are working out which parts of the image to shade and which to leave alone):
Image

Image

Image

Image

And fixed:
Image

The pattern to fix the shaders is here:
https://github.com/bo3b/3Dmigoto/commit/6fcb2354435b866d3697b588c982d3f44c0ec552

It's fairly straight forward, pretty much just requiring the view-space correction formula a whole bunch of times, but it does need the projection or inverse projection matrix copied in with 3DMigoto to fix accurately regardless of the FOV. It is still reasonable to fix without that since it won't matter as much as shadows if it's just approximate. Update: Actually, with my new fancy constant buffer debug shader I was able to identify that the required info is already available in the shader - See this version of the fix from WATCH_DOGS2 where I found it in -cb0[2].z:
https://github.com/DarkStarSword/3d-fixes/commit/667bd4945503d3195eaaecc8caab5704f04ac316

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#1
Posted 11/18/2015 06:08 PM   
That is some very interesting thing there! I looked a bit throug the shader code to understand what you did there;) Pretty cool stuff! Also those frame analysis screens look fabulous:)) Like them more than the final composition lol:)) AWESOME AND FABULOUS JOB AS ALWAYS DarkStarSword!!! I'll need to play with this myself at some point;))
That is some very interesting thing there! I looked a bit throug the shader code to understand what you did there;)
Pretty cool stuff! Also those frame analysis screens look fabulous:)) Like them more than the final composition lol:))

AWESOME AND FABULOUS JOB AS ALWAYS DarkStarSword!!!

I'll need to play with this myself at some point;))


Life is as empty without Terror, as is without Love...
But thou Chaos can be beautiful...
It can not last...


-
2x Asus GTX 980Ti Strix (OC edition), Custom Water-cooling to kick the Temperature & Noise arse ^_^
-3x Asus VG278HE 3D Vision 2 monitors (3D Surround)
i7 4790K @ 5.0Ghz ontop of Asus Maximus VII Ranger
DDR3 Corsair Vengeance 16GB 2133
RAID0 256GB SSDs
RAID0 2TB Mechanical HDD (WD Black)
-
Alienware M17x R5
GT880M (8GB VRAM)
120hz 3D Samsung LCD Panel

16gb 1600Mhz DDR3
256GB SSD + 1TB mechanical HDD
-
Alienware M14x R1
GT555M (1.5GB VRAM) - Regular LG LCD
8GB 1600Mhz DDR3
256 SSD + 512GB mechanical HDD
-
Paypal for donations: tavyhome@gmail.com

#2
Posted 11/18/2015 07:19 PM   
Interesting screenshots/explanation, and great result!
Interesting screenshots/explanation, and great result!

#3
Posted 11/19/2015 12:20 AM   
I see what you did there... Well visually I do, mentally...no comprendo Very interesting nonetheless
I see what you did there...

Well visually I do, mentally...no comprendo

Very interesting nonetheless

#4
Posted 11/19/2015 05:09 AM   
I revisited Nvidia's technology article on HBAO+, wrapped my head around it a little more. Interesting how it uses fast Interleaved Rendering technique to decrease overhead vs a slow Fullscreen Pass For those interested, links below http://www.geforce.com/hardware/technology/hbao-plus/technology https://developer.nvidia.com/sites/default/files/akamai/gamedev/docs/BAVOIL_ParticleShadowsAndCacheEfficientPost.pdf
I revisited Nvidia's technology article on HBAO+, wrapped my head around it a little more.

Interesting how it uses fast Interleaved Rendering technique to decrease overhead vs a slow Fullscreen Pass

For those interested, links below

http://www.geforce.com/hardware/technology/hbao-plus/technology

https://developer.nvidia.com/sites/default/files/akamai/gamedev/docs/BAVOIL_ParticleShadowsAndCacheEfficientPost.pdf

#5
Posted 11/19/2015 10:27 AM   
That. Is. Awesome. Broke open the HBAO+ glitches. Very cool. Really appreciate the writeup as the approach is even more interesting to me than the fix.
That. Is. Awesome.

Broke open the HBAO+ glitches. Very cool. Really appreciate the writeup as the approach is even more interesting to me than the fix.

Acer H5360 (1280x720@120Hz) - ASUS VG248QE with GSync mod - 3D Vision 1&2 - Driver 372.54
GTX 970 - GTX 760 Dedicated PhysX - i5-4670K@4.2GHz - 12GB RAM - Win7x64+evilKB2670838 - 4 Disk X25 RAID
Latest 3Dmigoto Release
Bo3b's School for ShaderHackers

#6
Posted 11/21/2015 08:24 AM   
Pattern to fix the same kind of SSAO artefact in Unity games: https://github.com/DarkStarSword/3d-fixes/commit/1c991948555fa451ef3ca125b7848a70d7bad8ac I've added those two shaders to my Unity 5 DX9 template - they are both used in The Forest and Stranded Deep (both updated on the blog), so it stands to reason they might be used in other games as well (although I do not believe they ship with Unity).
Pattern to fix the same kind of SSAO artefact in Unity games:

https://github.com/DarkStarSword/3d-fixes/commit/1c991948555fa451ef3ca125b7848a70d7bad8ac

I've added those two shaders to my Unity 5 DX9 template - they are both used in The Forest and Stranded Deep (both updated on the blog), so it stands to reason they might be used in other games as well (although I do not believe they ship with Unity).

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#7
Posted 12/14/2015 05:03 PM   
Updated pattern for HBAO+ in WATCH_DOGS2 - in this case I was able to use my new fancy constant buffer debug shader to locate the horizontal FOV (as it would typically be specified in the inverse projection matrix) already available in the shader in -cb0[2].z - I was able to verify that value matched (other than being negated) the value in a known inverse projection matrix in another shader, and updated as the FOV changed: https://github.com/DarkStarSword/3d-fixes/commit/667bd4945503d3195eaaecc8caab5704f04ac316 Note that in WATCH_DOGS2 there was only one HBAO+ shader that needed to be corrected, but the process to find it was much the same as other games (and this time it wasn't hidden in an alpha channel, so in a way it was easier ;-)
Updated pattern for HBAO+ in WATCH_DOGS2 - in this case I was able to use my new fancy constant buffer debug shader to locate the horizontal FOV (as it would typically be specified in the inverse projection matrix) already available in the shader in -cb0[2].z - I was able to verify that value matched (other than being negated) the value in a known inverse projection matrix in another shader, and updated as the FOV changed:

https://github.com/DarkStarSword/3d-fixes/commit/667bd4945503d3195eaaecc8caab5704f04ac316

Note that in WATCH_DOGS2 there was only one HBAO+ shader that needed to be corrected, but the process to find it was much the same as other games (and this time it wasn't hidden in an alpha channel, so in a way it was easier ;-)

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#8
Posted 01/13/2017 05:43 AM   
Does Witcher 3 suffer from this issue with HBAO+ that this awesome find could fix? I never really noticed but maybe I am just used to how it looks or it doesn't apply to the game.
Does Witcher 3 suffer from this issue with HBAO+ that this awesome find could fix? I never really noticed but maybe I am just used to how it looks or it doesn't apply to the game.

1080 GTX 8GB SLI | I7-4770K@4.5GHz | 16GB RAM | Win10x64
Asus ROG Swift PG278Q | 3D Vision 2

#9
Posted 01/15/2017 06:51 PM   
mx-2 tracked down the HBAO+ shaders in Witcher 3 and applied this fix (Thanks mx-2 :)
mx-2 tracked down the HBAO+ shaders in Witcher 3 and applied this fix (Thanks mx-2 :)

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#10
Posted 01/16/2017 01:46 AM   
Scroll To Top