I fixed Unity reflections, and got more than I bargained for
You may have seen in the update to Dreamfall Chapters I found a way to fix the reflections. Now, aside from the obvious stuff: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67453/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67452/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67457/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67454/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67455/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67450/[/img] It also fixed some things that were not strictly speaking reflections - this is more like a fake interior texture: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67456/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67458/[/img] There were also some more subtle changes, like the glint in Zoe's eyes and lips: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67445/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67443/[/img] And greatly improved her leather jacket: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67447/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67451/[/img] Although I am now jealous of this NPC's ultra reflective outfit: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67449/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67448/[/img] It even improved the look of her hair: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67446/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67444/[/img] Then I went and updated the fix to The Forest - and finding that there were no sources of the inverse model-view matrix I could use for the usual lighting fix in the intro sequence any more, I switched it over to use this new technique which does not depend on that (It uses the inverse MVP and the _Object2World matrices instead), and of course that fixed the obvious things here as well: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67466/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67465/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67463/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67464/[/img] But it had some more interesting effects in this game... Take a look at this sand - just a flat texture on the ground right? [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67469/[/img] Then apply the reflection fix to the deferred rendering shaders: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67472/[/img] ok, that's an improvement, the specular highlights look better, but the sand is still just a flat texture... but then look what happened when I applied the fix globally: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67473/[/img] Well, that was a surprise! The sand is now fully 3D! This game uses a technique called "parallax occlusion" to take a flat 2D texture and distort it depending on the camera angle to hide parts of the texture that should not be visible from that angle. Well, it turns out that the technique works even better in real 3D when it computes each view using separate camera angles! It's pretty convincing, but does have a few minor issues. Note here that the shadow is lined up with the original surface, and not where the ground now appears to be: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67475/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67474/[/img] Here's some more screenshots of this fix: [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67477/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67476/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67468/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67467/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67471/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67470/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67462/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67460/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67459/[/img] [img]https://forums.geforce.com/cmd/default/download-comment-attachment/67461/[/img] I have also applied the same technique to Stranded Deep (not updated on the blog yet), which uses the same effect and this also makes the sand and ground textures 3D, but the effect is much less pronounced (and the misalignment perhaps more pronounced).
You may have seen in the update to Dreamfall Chapters I found a way to fix the reflections.

Now, aside from the obvious stuff:
Image
Image
Image
Image
Image
Image

It also fixed some things that were not strictly speaking reflections - this is more like a fake interior texture:
Image
Image

There were also some more subtle changes, like the glint in Zoe's eyes and lips:
Image
Image

And greatly improved her leather jacket:
Image
Image

Although I am now jealous of this NPC's ultra reflective outfit:
Image
Image

It even improved the look of her hair:
Image
Image


Then I went and updated the fix to The Forest - and finding that there were no sources of the inverse model-view matrix I could use for the usual lighting fix in the intro sequence any more, I switched it over to use this new technique which does not depend on that (It uses the inverse MVP and the _Object2World matrices instead), and of course that fixed the obvious things here as well:
Image
Image
Image
Image

But it had some more interesting effects in this game... Take a look at this sand - just a flat texture on the ground right?
Image
Then apply the reflection fix to the deferred rendering shaders:
Image
ok, that's an improvement, the specular highlights look better, but the sand is still just a flat texture... but then look what happened when I applied the fix globally:
Image

Well, that was a surprise! The sand is now fully 3D! This game uses a technique called "parallax occlusion" to take a flat 2D texture and distort it depending on the camera angle to hide parts of the texture that should not be visible from that angle. Well, it turns out that the technique works even better in real 3D when it computes each view using separate camera angles!

It's pretty convincing, but does have a few minor issues. Note here that the shadow is lined up with the original surface, and not where the ground now appears to be:
Image
Image


Here's some more screenshots of this fix:
Image
Image

Image
Image

Image
Image

Image
Image

Image
Image

I have also applied the same technique to Stranded Deep (not updated on the blog yet), which uses the same effect and this also makes the sand and ground textures 3D, but the effect is much less pronounced (and the misalignment perhaps more pronounced).

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#1
Posted 12/12/2015 11:47 AM   
Now that is very interesting;) I read/know about parallax occlusion yet I've never seen it "in action" until know in the screenshots with the sand;) That is really interesting! Can you please add an explanation on the technique used? (some source code will be helpful as well). I read that this new technique uses "(It uses the inverse MVP and the _Object2World matrices instead)". What was the old technique using? A quick explanation + source code added to this thread I believe will help others as well;) (I know I can always look on GitHub ^_^ but I think others don't know where to look for). I am also interested to see if this technique is familiar with something that I used in "Layers of Fears" fix;) Awesome work again!! Cheers!
Now that is very interesting;)
I read/know about parallax occlusion yet I've never seen it "in action" until know in the screenshots with the sand;) That is really interesting!

Can you please add an explanation on the technique used? (some source code will be helpful as well).
I read that this new technique uses "(It uses the inverse MVP and the _Object2World matrices instead)". What was the old technique using?

A quick explanation + source code added to this thread I believe will help others as well;)
(I know I can always look on GitHub ^_^ but I think others don't know where to look for). I am also interested to see if this technique is familiar with something that I used in "Layers of Fears" fix;)

Awesome work again!!
Cheers!


Life is as empty without Terror, as is without Love...
But thou Chaos can be beautiful...
It can not last...


-
2x Asus GTX 980Ti Strix (OC edition), Custom Water-cooling to kick the Temperature & Noise arse ^_^
-3x Asus VG278HE 3D Vision 2 monitors (3D Surround)
i7 4790K @ 5.0Ghz ontop of Asus Maximus VII Ranger
DDR3 Corsair Vengeance 16GB 2133
RAID0 256GB SSDs
RAID0 2TB Mechanical HDD (WD Black)
-
Alienware M17x R5
GT880M (8GB VRAM)
120hz 3D Samsung LCD Panel

16gb 1600Mhz DDR3
256GB SSD + 1TB mechanical HDD
-
Alienware M14x R1
GT555M (1.5GB VRAM) - Regular LG LCD
8GB 1600Mhz DDR3
256 SSD + 512GB mechanical HDD
-
Paypal for donations: tavyhome@gmail.com

#2
Posted 12/12/2015 12:40 PM   
Incredible, results are awesome and "paralax occlusion" in real 3D are totally 3D game changer! Really tired with flat ground in 3D, we need paralax occlusion in all games engine ;) Do you know other games which are using this paralax occlusion technique? Congrats dude :)
Incredible, results are awesome and "paralax occlusion" in real 3D are totally 3D game changer! Really tired with flat ground in 3D, we need paralax occlusion in all games engine ;)
Do you know other games which are using this paralax occlusion technique?

Congrats dude :)

http://photos.3dvisionlive.com/chtiblue/album/530b52d4cb85770d6e000049/3Dvision with 49" Philips 49PUS7100 interlieved 3D (3840x2160) overide mode, GTX 1080 GFA2 EXOC, core i5 @4.3GHz, 16Gb@2130, windows 7&10 64bit, Dolby Atmos 5.1.4 Marantz 6010 AVR

#3
Posted 12/12/2015 12:54 PM   
Impressive! I will immediately update my (your) fixes on the mentionned games.
Impressive! I will immediately update my (your) fixes on the mentionned games.

Intel Core i7-3820, 4 X 3,60 GHz overclocked to 4,50 GHz ; EVGA Titan X 12VRAM ; 16 GB Corsair Vengeance DDR-1600 (4x 4 GB) ; Asus VG278H 27-inch incl. 3D vision 2 glasses, integrated transmitter ; Xbox One Elite wireless controller ; Windows 10
HTC VIVE 2,5 m2 roomscale
3D VISION GAMERS - VISIT ME ON STEAM and feel free to add me:
http://steamcommunity.com/profiles/76561198064106555/edit
Image

#4
Posted 12/12/2015 02:51 PM   
[quote="helifax"]Can you please add an explanation on the technique used? (some source code will be helpful as well). I read that this new technique uses "(It uses the inverse MVP and the _Object2World matrices instead)". What was the old technique using? A quick explanation + source code added to this thread I believe will help others as well;) (I know I can always look on GitHub ^_^ but I think others don't know where to look for). I am also interested to see if this technique is familiar with something that I used in "Layers of Fears" fix;)[/quote]Sure :) In high level terms, the technique is actually quite simple - all it's doing is adjusting the _WorldSpaceCameraPos, which is used by many shaders in Unity. It uses the inverse MVP matrix to convert a correction amount at depth=0 for the camera to local object coordinates, then _Object2World to get back to world-space coordinates. In the rest of this post I'll go over the background of how I arrived at the current technique - in the next post I'll post how to use my scripts to automatically apply this technique to a DX9 Unity game. I was originally trying to fix the specular highlights and cube map reflections in Dreamfall, which I already knew from previous Unity 5 fixes the same deferred lighting shaders as the shadows. After a while I worked out that the only way I could adjust them in the right direction was by adjusting _WorldSpaceCameraPos. At that point I was still using the old method I came up with for Unity shadows (read other threads if interested in the details - as you will soon see I have now replaced this), so I already had the necessary part of the inverse projection matrix to perform a view-space correction, and the lighting shaders had the _CameraToWorld matrix that I could then use to apply a world-space correction. I figured that since I was adjusting the camera position, I could take the depth as being 0, so I came up with this (I've cut the original shader code since it is not that interesting, but I've left in the headers that my tools add so you can see how to identify matrices and constant registers in Unity games): [b]NOTE: This code works, but it's obsolete compared to what I'm doing now - keep reading![/b] [code] // CRC32: B1717770 | Unity headers extracted from Internal-DeferredReflections.shader // Shader "Hidden/Internal-DeferredReflections" { // Properties { // _SrcBlend ("", Float) = 1 // _DstBlend ("", Float) = 1 // } // Fallback Off // SubShader 1/1 { // Pass 1/2 { // ZWrite Off // Blend [_SrcBlend] [_DstBlend] // GpuProgramID 34194 // Program "fp" { // SubProgram "d3d9 " { // Matrix 0 [_CameraToWorld] 3 // Vector 4 [_ProjectionParams] // Vector 3 [_WorldSpaceCameraPos] // Vector 5 [_ZBufferParams] // Vector 6 [unity_SpecCube0_BoxMax] // Vector 7 [unity_SpecCube0_BoxMin] // Vector 9 [unity_SpecCube0_HDR] // Vector 8 [unity_SpecCube0_ProbePosition] // Vector 10 [unity_SpecCube1_ProbePosition] // SetTexture 0 [unity_SpecCube0] CUBE 0 // SetTexture 1 [_CameraDepthTexture] 2D 1 // SetTexture 2 [_CameraGBufferTexture0] 2D 2 // SetTexture 3 [_CameraGBufferTexture1] 2D 3 // SetTexture 4 [_CameraGBufferTexture2] 2D 4 // } // } // } // } // } // // Headers extracted with DarkStarSword's extract_unity_shaders.py // https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py <snip> def c220, 0, 1, 0.0625, 0.5 dcl_2d s13 <snip> // To move the reflections & specular highlights to the correct depth, we need // to adjust _WorldSpaceCameraPos - trying to adjust anything else won't look // quite right // Use a coord with W = 0 to suppress position transformations in the // _CameraToWorld matrix since we only want the adjustment amount: mov r29, c220.xxxx // Amount to adjust camera position by. Camera can be thought of as being at // depth=0, so the formula would put it at separation * -convergence * fov // ("fov" is from the inverse projection matrix and is passed from the vertex // shader in v5.x): texldl r31, c220.z, s13 mul r29.x, r31.x, -r31.y mul r29.x, r29.x, v5.x // Convert to world coordinates with the _CameraToWorld matrix: dp4_pp r28.x, c0, r29 dp4_pp r28.y, c1, r29 dp4_pp r28.z, c2, r29 // And finally adjust _WorldSpaceCameraPos: mov r30, c3 add r30.xyz, r30, -r28 // Replace uses of _WorldSpaceCameraPos with our modified version: // add r0.xyz, r2, -c3 add r0.xyz, r2, -r30 <snip> // Specular cube map reflections sampled here: texldl_pp r5, r5, s0 [/code] The funny thing was that only fixed some of the reflections of specular highlights, but not others - and a single surface could have some fixed and some not fixed simultaneously. As it turns out, the ones that were not fixed were not being rendered in any of the deferred shaders - they were being rendered directly by the surface's pixel shader, which also used _WorldSpaceCameraPos. Now, at the time I didn't have any matrices I was copying with Helix Mod that I could use for a world space correction, and while I could have copied the _CameraToWorld matrix out of the lighting shaders, it would have been one frame out of date due to the draw order. For the first couple of these I looked at, the corresponding vertex shader did have both glstate_matrix_mvp and _Object2World matrices - so I was able to use Helix Mod to inverse the MVP matrix to convert a stereo correction performed in clip-space into local object coordinates, then use the _Object2World matrix to convert it to world-space and adjust the camera position, which went something like this (note that since this was the second matrix copy slot in Helix Mod and Helix Mod only has two slots, I had to pass the _Object2World matrix to the pixel shader via three spare outputs from the vertex shader): [b]NOTE: This code works, but it's obsolete compared to what I'm doing now - keep reading![/b] [code] ; Fix for specular highlights that are not using deferred lighting shaders. ; These need the world space camera position adjusted, which we do by ; multiplying the adjustment amount by the inversed MVP matrix then the ; _Object2World matrix. We use Helix Mod to inverse the MVP matrix and pass ; _Object2World from the VS to the PS in spare texcoords: [VS68D32F2E] ; Matched 12 variants of 3 shaders: Chickenlord/Detail/Bumped Specular, Dreamfall/Unity5/Standard_BarutiJacket, Dreamfall/Unity5/Standard_BumpedDetail_Ilum GetMatrixFromReg1 = 0 InverseMatrix1 = true [PS898E1E30] ; Standard_BumpedDetail_Ilum.shader UseMatrix1 = true MatrixReg1 = 180 [/code] [code] // New inputs from the vertex shader with the _Object2World matrix: dcl_texcoord7 v6 dcl_texcoord8 v7 dcl_texcoord9 v8 def c220, 0, 1, 0.0625, 0.5 dcl_2d s13 texldl r31, c220.z, s13 // Take the camera as being at depth=0 and calculate the adjustment amount in // projection space. We use W=0 in this coordinate to suppress position // transformations in the matrices: mov r29, c220.x mul r29.x, r31.x, -r31.y // Run the adjustment amount through the inverse MVP matrix, which Helix Mod // has copied from the vertex shader, inverted and put in c180-c183: dp4 r28.x, c180, r29 dp4 r28.y, c181, r29 dp4 r28.z, c182, r29 dp4 r28.w, c183, r29 // We are now in local coordinates, but we need to make sure we are rotated // back to world coordinates, so multiply by the _ObjectToWorld matrix, which // has been passed from the vertex shader in some spare texcoords: dp4 r29.x, v6, r28 dp4 r29.y, v7, r28 dp4 r29.z, v8, r28 // And finally adjust _WorldSpaceCameraPos: mov r30, c0 add r30.xyz, r30, -r29 //add r1.xyz, -r1, c0 add r1.xyz, -r1, r30 [/code] And that worked beautiflly, pushing specular highlights back to the correct depth and the same pattern was able to fix the flat reflections on the river in Propast :) However, as I wandered through the game applying this pattern to more and more pixel shaders I came across some that didn't have enough free output registers to pass the _Object2World through to the pixel shader. At first this was ok - the vertex shaders seemed to also use _WorldSpaceCameraPos, so I calculated the correction there and just passed that through to the pixel shader in just a single output. I even found that correcting _WorldSpaceCameraPos in the vertex shader fixed some extra effects as well - namely some of the shop windows in Propast that rendered a fake interior texture behind them. But then I started hitting some with no free outputs from the vertex shader at all, and others that didn't have the _Object2World and/or _WorldSpaceCameraPos in the vertex shader - I had been concerned about these possibilities since I had already maxed out Helix Mod's ability to copy things around and had been relying on the vertex shaders having this stuff. Plus, I really wanted to be able to script this technique since I was finding that I was adjusting a *lot* of shaders, but there were too many variations (especially in terms of what outputs I was using from the vertex shaders) to cover everything (I did script two variations before I came up with the final approach I'm using now). So, I started wondering if I could change the technique I used to fix the deferred shadows in Unity to work with the same matrices I needed for this, thereby freeing up those matrix copy slots so that I could use them both for the inverse MVP and _Object2World matrices and allowing me to copy them into any shader I needed them in, regardless of whether it was a vertex or pixel shader and removing the need for the shader to already have either matrix. Well, the answer is yes - to fix lighting shaders in Unity we had been applying a view-space correction right before the _CameraToWorld matrix was used in the pixel shader to convert it to world-space, so it should be fairly easy to apply a world-space correction just after that point instead. What's more is we can use the same depth value that we have already found in the shader so we can skip converting the coordinate to view or projection-space first: [code] // Reflection/specular part of the fix. // Take the camera as being at depth=0 and calculate the adjustment amount in // projection space. We use W=0 in this coordinate to suppress position // transformations in the matrices: mov r29, c220.x mul r29.x, r31.x, -r31.y // Run the adjustment amount through the inverse MVP matrix, which Helix Mod // has copied from elsewhere, inverted and put in c180-c183: dp4 r28.x, c180, r29 dp4 r28.y, c181, r29 dp4 r28.z, c182, r29 dp4 r28.w, c183, r29 // We are now in local coordinates, but we need to make sure we are rotated // back to world coordinates, so multiply by the _ObjectToWorld matrix copied // in to c190-c192 with Helix Mod: dp4 r29.x, c190, r28 dp4 r29.y, c191, r28 dp4 r29.z, c192, r28 // And finally adjust _WorldSpaceCameraPos: mov r30, c3 add r30.xyz, r30, -r29 <snip> // This is where the lighting/shadow fix used to go - we would apply a // view-space correction to r0 here based on the depth in r0.z and either the // inverse projection matrix passed from the directional lighting shader, or // derived from the inverse MV and forwards MVP matrices // c0-c2 is the _CameraToWorld matrix dp4_pp r2.x, c0, r0 dp4_pp r2.y, c1, r0 dp4_pp r2.z, c2, r0 // depth in r0.z // This is where the lighting/shadow part of the fix now goes // This is the adjustment amount - the depth comes from the register we noted // was used as in input in the above _CameraToWorld matrix multiply. We use W=0 // in this coordinate to suppress position transformations in the matrices: mov r29, c220.x add r29.x, r0.z, -r31.y mul r29.x, r29.x, r31.x // Run the adjustment amount through the inverse MVP matrix, which Helix Mod // has copied from elsewhere, inverted and put in c180-c183: dp4 r28.x, c180, r29 dp4 r28.y, c181, r29 dp4 r28.z, c182, r29 dp4 r28.w, c183, r29 // We are now in local coordinates, but we need to make sure we are rotated // back to world coordinates, so multiply by the _ObjectToWorld matrix copied // in to c190-c192 with Helix Mod: dp4 r29.x, c190, r28 dp4 r29.y, c191, r28 dp4 r29.z, c192, r28 // And finally adjust the world-space coordinate: add r2.xyz, r2, -r29 [/code] And that's all well and good, but there is still one other place we need to apply a view-space correction for a Unity lighting fix - we need to adjust the ray in the vertex shader, otherwise the lights will break at certain distances as it switches between different shadow map resolutions (I think that's what it's doing anyway). Fortunately, we only need to apply that particular adjustment for point and spot lights - and in those cases the vertex shader has a valid set of MVP and MV matrices (they are not valid when that same shader is processing directional or physical lighting), so we can use them just like we did in previous Unity techniques to derive the inverse projection matrix for the view-space correction. And what's more, I already wrote an optimised version of the code to do that without using up any of Helix Mod's copy slots, so all we have to do is rip out the code from the vertex shader that was for the directional/physical cases and simplify it a little: [b]This is the COMPLETE shader and can be dropped into any Unity 5 game as is[/b] [code] // CRC32: 05F7E52C | Matched 65 variants of 2 shaders: Hidden/Internal-DeferredShading, Hidden/Internal-PrePassLighting // // Unity headers extracted from Internal-DeferredShading.shader // Shader "Hidden/Internal-DeferredShading" { // Properties { // _LightTexture0 ("", any) = "" { } // _LightTextureB0 ("", 2D) = "" { } // _ShadowMapTexture ("", any) = "" { } // _SrcBlend ("", Float) = 1 // _DstBlend ("", Float) = 1 // } // Fallback Off // SubShader 1/1 { // Pass 1/2 { // Tags { "SHADOWSUPPORT"="true" } // ZWrite Off // Blend [_SrcBlend] [_DstBlend] // GpuProgramID 36028 // Program "vp" { // SubProgram "d3d9 " { // 0004000: Keywords { "DIRECTIONAL" "SHADOWS_OFF" "UNITY_HDR_ON" } // 0000002: Keywords { "DIRECTIONAL" "SHADOWS_OFF" } // 0080000: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" "UNITY_HDR_ON" } // 0000040: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" } // 0020000: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" "UNITY_HDR_ON" } // 0000010: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" } // 0100000: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" "UNITY_HDR_ON" } // 0000080: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" } // 1000000: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" "UNITY_HDR_ON" } // 0000800: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" } // 0200000: Keywords { "POINT" "SHADOWS_CUBE" "UNITY_HDR_ON" } // 0000100: Keywords { "POINT" "SHADOWS_CUBE" } // 0002000: Keywords { "POINT" "SHADOWS_OFF" "UNITY_HDR_ON" } // 0000001: Keywords { "POINT" "SHADOWS_OFF" } // 2000000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" "UNITY_HDR_ON" } // 0001000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" } // 0400000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "UNITY_HDR_ON" } // 0000200: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" } // 0010000: Keywords { "POINT_COOKIE" "SHADOWS_OFF" "UNITY_HDR_ON" } // 0000008: Keywords { "POINT_COOKIE" "SHADOWS_OFF" } // 0800000: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" "UNITY_HDR_ON" } // 0000400: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" } // 0040000: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" "UNITY_HDR_ON" } // 0000020: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" } // 0008000: Keywords { "SHADOWS_OFF" "SPOT" "UNITY_HDR_ON" } // 0000004: Keywords { "SHADOWS_OFF" "SPOT" } // Bind "vertex" Vertex // Bind "normal" Normal // Matrix 4 [glstate_matrix_modelview0] 3 // Matrix 0 [glstate_matrix_mvp] // Float 9 [_LightAsQuad] // Vector 7 [_ProjectionParams] // Vector 8 [_ScreenParams] // } // } // } // } // } // // Unity headers extracted from Internal-PrePassLighting.shader // Shader "Hidden/Internal-PrePassLighting" { // Properties { // _LightTexture0 ("", any) = "" { } // _LightTextureB0 ("", 2D) = "" { } // _ShadowMapTexture ("", any) = "" { } // } // Fallback Off // SubShader 1/1 { // 0000001fff: Pass 1/3 { // 0003ffe000: Pass 2/3 { // 7ffc000000: Pass 3/3 { // Tags { "SHADOWSUPPORT"="true" } // ZWrite Off // 0000001fff: Blend DstColor Zero // 7fffffe000: Blend One One // 0003ffe000: GpuProgramID 113408 // 7ffc000000: GpuProgramID 155461 // 0000001fff: GpuProgramID 57115 // Program "vp" { // SubProgram "d3d9 " { // 0008004002: Keywords { "DIRECTIONAL" "SHADOWS_OFF" } // 0100080040: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" } // 0040020010: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" } // 0200100080: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" } // 2001000800: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" } // 0400200100: Keywords { "POINT" "SHADOWS_CUBE" } // 0004002001: Keywords { "POINT" "SHADOWS_OFF" } // 4002001000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" } // 0800400200: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" } // 0020010008: Keywords { "POINT_COOKIE" "SHADOWS_OFF" } // 1000800400: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" } // 0080040020: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" } // 0010008004: Keywords { "SHADOWS_OFF" "SPOT" } // Bind "vertex" Vertex // Bind "normal" Normal // Matrix 4 [glstate_matrix_modelview0] 3 // Matrix 0 [glstate_matrix_mvp] // Float 9 [_LightAsQuad] // Vector 7 [_ProjectionParams] // Vector 8 [_ScreenParams] // } // } // } // } // } // // Headers extracted with DarkStarSword's extract_unity_shaders.py // https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py vs_3_0 def c10, 0.5, -1, 1, 0 dcl_position v0 dcl_normal v1 dcl_position o0 dcl_texcoord o1 dcl_texcoord1 o2.xyz def c220, 0, 1, 0.0625, 1 dcl_2d s0 // Rearranged to simplify shader: dp4 r0.x, c0, v0 dp4 r0.y, c1, v0 dp4 r0.z, c2, v0 dp4 r0.w, c3, v0 mov o0, r0 // moved up before the stereo correction // Apply texcoord correction only when not drawing a full screen quad: if_ne r0.w, c220.y // Apply stereo correction to screen coords stored in texcoord: texldl r31, c220.z, s0 add r31.w, r0.w, -r31.y mul r31.w, r31.w, r31.x add r0.x, r0.x, r31.w endif mul r1.x, r0.y, c7.x mul r1.w, r1.x, c10.x mul r1.xz, r0.xyww, c10.x mad o1.xy, r1.z, c8.zwzw, r1.xwzw dp4 r1.x, c4, v0 dp4 r1.y, c5, v0 dp4 r1.z, c6, v0 // Apply view-space ray correction only when not drawing a full screen quad: if_ne r0.w, c220.y // This technique has changed a little starting with the Unity 5 update // of Dreamfall Chapters. The old technique still works just fine, but // I wanted to adjust _WorldSpaceCameraPos in a number of shaders and // my quest to find a way to solve both problems reliably (by script) // led me to find a way to fix the lighting shaders using world-space // coordinates instead of view-space coordinates - that way, I could // keep the number of matrices I needed to copy with Helix Mod down to // 2 (the limit). // // This is the one part of the lighting fix that still needs a // view-space correction, but fortunately we only need to do this when // the shader has valid MV and MVP matrices so we can still derive the // inverse projection matrix to get the FOV like usual. // We can't use Helix mod to invert the MV matrix since we are already // using both matrix copy slots for the inverse MVP and _Object2World // matrices, and the MV matrix in Unity 5 is only a 3x4 matrix, while // Helix Mod expects a 4x4 matrix. // So let's do it directly in assembly instead! It sounds complicated, // but we can make some large simplifications based on assumptions // about the MV and MVP matrices and the fact we only need one cell of // the (inverse) projection matrix, which means it is actually possible // to do in a farily small number of instructions :) // Use a temporary register for MV[1] to get around the fact that some // of the below instructions need to read from more constant registers // than there are available read ports: mov r26, c5 // 1. Calculate 1/determinant of the MV matrix, simplifying by assuming the // 4th column of the MV matrix is 0,0,0,1 // // mathomatic simplified it to: // 1 / ((m12*((m20*m01) - (m21*m00))) + (m02*((m21*m10) - (m20*m11))) + (m22*((m00*m11) - (m01*m10)))); // // Replace row numbers with register components (assumes column-major order): // (mv2.x*((mv0.y*mv1.z) - (mv0.z*mv1.y))) // + (mv2.y*((mv0.z*mv1.x) - (mv0.x*mv1.z))) // + (mv2.z*((mv0.x*mv1.y) - (mv0.y*mv1.x))) // Do some multiplications in parallel with SIMD instructions: mul r20.xyz, c4.yzx, r26.zxy // mv0.y*mv1.z, mv0.z*mv1.x, mv0.x*mv1.y mul r21.xyz, c4.zxy, r26.yzx // mv0.z*mv1.y, mv0.x*mv1.z, mv0.y*mv1.x // Do the subtractions: add r20.xyz, r20.xyz, -r21.xyz // mv0.y*mv1.z - mv0.z*mv1.y, mv0.z*mv1.x - mv0.x*mv1.z, mv0.x*mv1.y - mv0.y*mv1.x // Now the final three multiplications and the two additions: dp3 r22.w, r20.xyz, c6.xyz // And finally get 1/determinant: rcp r22.w, r22.w // 2. Calculate the 1st row of the inverted MV matrix, simplifying by assuimg // the 4th column of the MV matrix is 0,0,0,1 // // m00 = (mv1.y*mv2.z - mv1.z*mv2.y) / determinant // m01 = (mv1.z*mv2.x - mv1.x*mv2.z) / determinant // m02 = (mv1.x*mv2.y - mv1.y*mv2.x) / determinant // Do some multiplications in parallel with SIMD instructions: mul r20.xyz, r26.yzx, c6.zxy // mv1.y*mv2.z, mv1.z*mv2.x, mv1.x*mv2.y mul r21.xyz, r26.zxy, c6.yzx // mv1.z*mv2.y, mv1.x*mv2.z, mv1.y*mv2.x // Do the subtractions: add r20.xyz, r20.xyz, -r21.xyz // mv1.y*mv2.z - mv1.z*mv2.y, mv1.z*mv2.x - mv1.x*mv2.z, mv1.x*mv2.y - mv1.y*mv2.x // Multiply against 1/determinant: mul r20.xyz, r20.xyz, r22.www // 3. Multiply the first row of the inverted MV matrix with the 1st column of // the MVP matrix (MV.I[0,3] is 0, so only worry about the 1st three): dp3 r30.x, r20.xyz, c0.xyz rcp r30.x, r30.x // Calculate P.I[0,0] // Apply view-space stereo correction to ray: mad r1.x, r31.w, r30.x, r1.x endif mul r2.xyz, r1, c10.yyzw mad r1.xyz, r1, -c10.yyzw, v1 mad o2.xyz, c9.x, r1, r2 mov o1.zw, r0 [/code] Then, all that's left is adding all the sections to the DX9Settings.ini to copy the matrices from any suitable shaders into every shader where we want to apply the correction: [code] [VSEED89599] ; Copy inversed MVP matrix and _Object2World matrix in for Unity reflection fix UseMatrix = true MatrixReg = 180 UseMatrix1 = true MatrixReg1 = 190 ; Candidate to obtain MVP and _Object2World matrices: GetMatrixFromReg = 0 InverseMatrix = true GetMatrixFromReg1 = 4 [PSB1717770] ; Copy inversed MVP matrix and _Object2World matrix in for world-space variant of Unity lighting fix UseMatrix = true MatrixReg = 180 UseMatrix1 = true MatrixReg1 = 190 ; and so on... [/code] [color="orange"]But [b]there's a gotcha here[/b][/color] - some shaders appear to have the matrices we want, but attempting to use them will break everything! I have determined that any shaders with "LIGHTMODE"="SHADOWCASTER" in the headers are a no-no - these shaders are used while rendering shadow maps, when the matrices will be for the scene rendered from the perspective of a light source, but we need them for the perspective of the camera. There may be other shaders to avoid as well - I'm currently limiting the set I use to those that themselves use _WorldSpaceCameraPos which I figure are more likely to be safe. In the next post I'll show how I combine all my scripts together to automatically apply this technique to a DX9 Unity game and fix it from start to finish. Edit: Simplified 05F7E52C.txt
helifax said:Can you please add an explanation on the technique used? (some source code will be helpful as well).
I read that this new technique uses "(It uses the inverse MVP and the _Object2World matrices instead)". What was the old technique using?

A quick explanation + source code added to this thread I believe will help others as well;)
(I know I can always look on GitHub ^_^ but I think others don't know where to look for). I am also interested to see if this technique is familiar with something that I used in "Layers of Fears" fix;)
Sure :)

In high level terms, the technique is actually quite simple - all it's doing is adjusting the _WorldSpaceCameraPos, which is used by many shaders in Unity. It uses the inverse MVP matrix to convert a correction amount at depth=0 for the camera to local object coordinates, then _Object2World to get back to world-space coordinates.


In the rest of this post I'll go over the background of how I arrived at the current technique - in the next post I'll post how to use my scripts to automatically apply this technique to a DX9 Unity game.



I was originally trying to fix the specular highlights and cube map reflections in Dreamfall, which I already knew from previous Unity 5 fixes the same deferred lighting shaders as the shadows.

After a while I worked out that the only way I could adjust them in the right direction was by adjusting _WorldSpaceCameraPos. At that point I was still using the old method I came up with for Unity shadows (read other threads if interested in the details - as you will soon see I have now replaced this), so I already had the necessary part of the inverse projection matrix to perform a view-space correction, and the lighting shaders had the _CameraToWorld matrix that I could then use to apply a world-space correction.

I figured that since I was adjusting the camera position, I could take the depth as being 0, so I came up with this (I've cut the original shader code since it is not that interesting, but I've left in the headers that my tools add so you can see how to identify matrices and constant registers in Unity games):

NOTE: This code works, but it's obsolete compared to what I'm doing now - keep reading!
// CRC32: B1717770 | Unity headers extracted from Internal-DeferredReflections.shader
// Shader "Hidden/Internal-DeferredReflections" {
// Properties {
// _SrcBlend ("", Float) = 1
// _DstBlend ("", Float) = 1
// }
// Fallback Off
// SubShader 1/1 {
// Pass 1/2 {
// ZWrite Off
// Blend [_SrcBlend] [_DstBlend]
// GpuProgramID 34194
// Program "fp" {
// SubProgram "d3d9 " {
// Matrix 0 [_CameraToWorld] 3
// Vector 4 [_ProjectionParams]
// Vector 3 [_WorldSpaceCameraPos]
// Vector 5 [_ZBufferParams]
// Vector 6 [unity_SpecCube0_BoxMax]
// Vector 7 [unity_SpecCube0_BoxMin]
// Vector 9 [unity_SpecCube0_HDR]
// Vector 8 [unity_SpecCube0_ProbePosition]
// Vector 10 [unity_SpecCube1_ProbePosition]
// SetTexture 0 [unity_SpecCube0] CUBE 0
// SetTexture 1 [_CameraDepthTexture] 2D 1
// SetTexture 2 [_CameraGBufferTexture0] 2D 2
// SetTexture 3 [_CameraGBufferTexture1] 2D 3
// SetTexture 4 [_CameraGBufferTexture2] 2D 4
// }
// }
// }
// }
// }
//
// Headers extracted with DarkStarSword's extract_unity_shaders.py
// https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py

<snip>

def c220, 0, 1, 0.0625, 0.5
dcl_2d s13

<snip>

// To move the reflections & specular highlights to the correct depth, we need
// to adjust _WorldSpaceCameraPos - trying to adjust anything else won't look
// quite right

// Use a coord with W = 0 to suppress position transformations in the
// _CameraToWorld matrix since we only want the adjustment amount:
mov r29, c220.xxxx

// Amount to adjust camera position by. Camera can be thought of as being at
// depth=0, so the formula would put it at separation * -convergence * fov
// ("fov" is from the inverse projection matrix and is passed from the vertex
// shader in v5.x):
texldl r31, c220.z, s13
mul r29.x, r31.x, -r31.y
mul r29.x, r29.x, v5.x

// Convert to world coordinates with the _CameraToWorld matrix:
dp4_pp r28.x, c0, r29
dp4_pp r28.y, c1, r29
dp4_pp r28.z, c2, r29

// And finally adjust _WorldSpaceCameraPos:
mov r30, c3
add r30.xyz, r30, -r28

// Replace uses of _WorldSpaceCameraPos with our modified version:
// add r0.xyz, r2, -c3
add r0.xyz, r2, -r30

<snip>

// Specular cube map reflections sampled here:
texldl_pp r5, r5, s0


The funny thing was that only fixed some of the reflections of specular highlights, but not others - and a single surface could have some fixed and some not fixed simultaneously. As it turns out, the ones that were not fixed were not being rendered in any of the deferred shaders - they were being rendered directly by the surface's pixel shader, which also used _WorldSpaceCameraPos. Now, at the time I didn't have any matrices I was copying with Helix Mod that I could use for a world space correction, and while I could have copied the _CameraToWorld matrix out of the lighting shaders, it would have been one frame out of date due to the draw order.

For the first couple of these I looked at, the corresponding vertex shader did have both glstate_matrix_mvp and _Object2World matrices - so I was able to use Helix Mod to inverse the MVP matrix to convert a stereo correction performed in clip-space into local object coordinates, then use the _Object2World matrix to convert it to world-space and adjust the camera position, which went something like this (note that since this was the second matrix copy slot in Helix Mod and Helix Mod only has two slots, I had to pass the _Object2World matrix to the pixel shader via three spare outputs from the vertex shader):

NOTE: This code works, but it's obsolete compared to what I'm doing now - keep reading!
; Fix for specular highlights that are not using deferred lighting shaders.
; These need the world space camera position adjusted, which we do by
; multiplying the adjustment amount by the inversed MVP matrix then the
; _Object2World matrix. We use Helix Mod to inverse the MVP matrix and pass
; _Object2World from the VS to the PS in spare texcoords:
[VS68D32F2E]
; Matched 12 variants of 3 shaders: Chickenlord/Detail/Bumped Specular, Dreamfall/Unity5/Standard_BarutiJacket, Dreamfall/Unity5/Standard_BumpedDetail_Ilum
GetMatrixFromReg1 = 0
InverseMatrix1 = true
[PS898E1E30]
; Standard_BumpedDetail_Ilum.shader
UseMatrix1 = true
MatrixReg1 = 180

// New inputs from the vertex shader with the _Object2World matrix:
dcl_texcoord7 v6
dcl_texcoord8 v7
dcl_texcoord9 v8

def c220, 0, 1, 0.0625, 0.5
dcl_2d s13

texldl r31, c220.z, s13

// Take the camera as being at depth=0 and calculate the adjustment amount in
// projection space. We use W=0 in this coordinate to suppress position
// transformations in the matrices:
mov r29, c220.x
mul r29.x, r31.x, -r31.y

// Run the adjustment amount through the inverse MVP matrix, which Helix Mod
// has copied from the vertex shader, inverted and put in c180-c183:
dp4 r28.x, c180, r29
dp4 r28.y, c181, r29
dp4 r28.z, c182, r29
dp4 r28.w, c183, r29

// We are now in local coordinates, but we need to make sure we are rotated
// back to world coordinates, so multiply by the _ObjectToWorld matrix, which
// has been passed from the vertex shader in some spare texcoords:
dp4 r29.x, v6, r28
dp4 r29.y, v7, r28
dp4 r29.z, v8, r28

// And finally adjust _WorldSpaceCameraPos:
mov r30, c0
add r30.xyz, r30, -r29

//add r1.xyz, -r1, c0
add r1.xyz, -r1, r30


And that worked beautiflly, pushing specular highlights back to the correct depth and the same pattern was able to fix the flat reflections on the river in Propast :)

However, as I wandered through the game applying this pattern to more and more pixel shaders I came across some that didn't have enough free output registers to pass the _Object2World through to the pixel shader. At first this was ok - the vertex shaders seemed to also use _WorldSpaceCameraPos, so I calculated the correction there and just passed that through to the pixel shader in just a single output. I even found that correcting _WorldSpaceCameraPos in the vertex shader fixed some extra effects as well - namely some of the shop windows in Propast that rendered a fake interior texture behind them.

But then I started hitting some with no free outputs from the vertex shader at all, and others that didn't have the _Object2World and/or _WorldSpaceCameraPos in the vertex shader - I had been concerned about these possibilities since I had already maxed out Helix Mod's ability to copy things around and had been relying on the vertex shaders having this stuff. Plus, I really wanted to be able to script this technique since I was finding that I was adjusting a *lot* of shaders, but there were too many variations (especially in terms of what outputs I was using from the vertex shaders) to cover everything (I did script two variations before I came up with the final approach I'm using now).

So, I started wondering if I could change the technique I used to fix the deferred shadows in Unity to work with the same matrices I needed for this, thereby freeing up those matrix copy slots so that I could use them both for the inverse MVP and _Object2World matrices and allowing me to copy them into any shader I needed them in, regardless of whether it was a vertex or pixel shader and removing the need for the shader to already have either matrix.

Well, the answer is yes - to fix lighting shaders in Unity we had been applying a view-space correction right before the _CameraToWorld matrix was used in the pixel shader to convert it to world-space, so it should be fairly easy to apply a world-space correction just after that point instead. What's more is we can use the same depth value that we have already found in the shader so we can skip converting the coordinate to view or projection-space first:


// Reflection/specular part of the fix.

// Take the camera as being at depth=0 and calculate the adjustment amount in
// projection space. We use W=0 in this coordinate to suppress position
// transformations in the matrices:
mov r29, c220.x
mul r29.x, r31.x, -r31.y

// Run the adjustment amount through the inverse MVP matrix, which Helix Mod
// has copied from elsewhere, inverted and put in c180-c183:
dp4 r28.x, c180, r29
dp4 r28.y, c181, r29
dp4 r28.z, c182, r29
dp4 r28.w, c183, r29

// We are now in local coordinates, but we need to make sure we are rotated
// back to world coordinates, so multiply by the _ObjectToWorld matrix copied
// in to c190-c192 with Helix Mod:
dp4 r29.x, c190, r28
dp4 r29.y, c191, r28
dp4 r29.z, c192, r28

// And finally adjust _WorldSpaceCameraPos:
mov r30, c3
add r30.xyz, r30, -r29

<snip>

// This is where the lighting/shadow fix used to go - we would apply a
// view-space correction to r0 here based on the depth in r0.z and either the
// inverse projection matrix passed from the directional lighting shader, or
// derived from the inverse MV and forwards MVP matrices

// c0-c2 is the _CameraToWorld matrix
dp4_pp r2.x, c0, r0
dp4_pp r2.y, c1, r0
dp4_pp r2.z, c2, r0 // depth in r0.z

// This is where the lighting/shadow part of the fix now goes

// This is the adjustment amount - the depth comes from the register we noted
// was used as in input in the above _CameraToWorld matrix multiply. We use W=0
// in this coordinate to suppress position transformations in the matrices:
mov r29, c220.x
add r29.x, r0.z, -r31.y
mul r29.x, r29.x, r31.x

// Run the adjustment amount through the inverse MVP matrix, which Helix Mod
// has copied from elsewhere, inverted and put in c180-c183:
dp4 r28.x, c180, r29
dp4 r28.y, c181, r29
dp4 r28.z, c182, r29
dp4 r28.w, c183, r29

// We are now in local coordinates, but we need to make sure we are rotated
// back to world coordinates, so multiply by the _ObjectToWorld matrix copied
// in to c190-c192 with Helix Mod:
dp4 r29.x, c190, r28
dp4 r29.y, c191, r28
dp4 r29.z, c192, r28

// And finally adjust the world-space coordinate:
add r2.xyz, r2, -r29


And that's all well and good, but there is still one other place we need to apply a view-space correction for a Unity lighting fix - we need to adjust the ray in the vertex shader, otherwise the lights will break at certain distances as it switches between different shadow map resolutions (I think that's what it's doing anyway).

Fortunately, we only need to apply that particular adjustment for point and spot lights - and in those cases the vertex shader has a valid set of MVP and MV matrices (they are not valid when that same shader is processing directional or physical lighting), so we can use them just like we did in previous Unity techniques to derive the inverse projection matrix for the view-space correction.

And what's more, I already wrote an optimised version of the code to do that without using up any of Helix Mod's copy slots, so all we have to do is rip out the code from the vertex shader that was for the directional/physical cases and simplify it a little:

This is the COMPLETE shader and can be dropped into any Unity 5 game as is
// CRC32: 05F7E52C | Matched 65 variants of 2 shaders: Hidden/Internal-DeferredShading, Hidden/Internal-PrePassLighting
//
// Unity headers extracted from Internal-DeferredShading.shader
// Shader "Hidden/Internal-DeferredShading" {
// Properties {
// _LightTexture0 ("", any) = "" { }
// _LightTextureB0 ("", 2D) = "" { }
// _ShadowMapTexture ("", any) = "" { }
// _SrcBlend ("", Float) = 1
// _DstBlend ("", Float) = 1
// }
// Fallback Off
// SubShader 1/1 {
// Pass 1/2 {
// Tags { "SHADOWSUPPORT"="true" }
// ZWrite Off
// Blend [_SrcBlend] [_DstBlend]
// GpuProgramID 36028
// Program "vp" {
// SubProgram "d3d9 " {
// 0004000: Keywords { "DIRECTIONAL" "SHADOWS_OFF" "UNITY_HDR_ON" }
// 0000002: Keywords { "DIRECTIONAL" "SHADOWS_OFF" }
// 0080000: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" "UNITY_HDR_ON" }
// 0000040: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" }
// 0020000: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" "UNITY_HDR_ON" }
// 0000010: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" }
// 0100000: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" "UNITY_HDR_ON" }
// 0000080: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" }
// 1000000: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" "UNITY_HDR_ON" }
// 0000800: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" }
// 0200000: Keywords { "POINT" "SHADOWS_CUBE" "UNITY_HDR_ON" }
// 0000100: Keywords { "POINT" "SHADOWS_CUBE" }
// 0002000: Keywords { "POINT" "SHADOWS_OFF" "UNITY_HDR_ON" }
// 0000001: Keywords { "POINT" "SHADOWS_OFF" }
// 2000000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" "UNITY_HDR_ON" }
// 0001000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" }
// 0400000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "UNITY_HDR_ON" }
// 0000200: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" }
// 0010000: Keywords { "POINT_COOKIE" "SHADOWS_OFF" "UNITY_HDR_ON" }
// 0000008: Keywords { "POINT_COOKIE" "SHADOWS_OFF" }
// 0800000: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" "UNITY_HDR_ON" }
// 0000400: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" }
// 0040000: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" "UNITY_HDR_ON" }
// 0000020: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" }
// 0008000: Keywords { "SHADOWS_OFF" "SPOT" "UNITY_HDR_ON" }
// 0000004: Keywords { "SHADOWS_OFF" "SPOT" }
// Bind "vertex" Vertex
// Bind "normal" Normal
// Matrix 4 [glstate_matrix_modelview0] 3
// Matrix 0 [glstate_matrix_mvp]
// Float 9 [_LightAsQuad]
// Vector 7 [_ProjectionParams]
// Vector 8 [_ScreenParams]
// }
// }
// }
// }
// }
//
// Unity headers extracted from Internal-PrePassLighting.shader
// Shader "Hidden/Internal-PrePassLighting" {
// Properties {
// _LightTexture0 ("", any) = "" { }
// _LightTextureB0 ("", 2D) = "" { }
// _ShadowMapTexture ("", any) = "" { }
// }
// Fallback Off
// SubShader 1/1 {
// 0000001fff: Pass 1/3 {
// 0003ffe000: Pass 2/3 {
// 7ffc000000: Pass 3/3 {
// Tags { "SHADOWSUPPORT"="true" }
// ZWrite Off
// 0000001fff: Blend DstColor Zero
// 7fffffe000: Blend One One
// 0003ffe000: GpuProgramID 113408
// 7ffc000000: GpuProgramID 155461
// 0000001fff: GpuProgramID 57115
// Program "vp" {
// SubProgram "d3d9 " {
// 0008004002: Keywords { "DIRECTIONAL" "SHADOWS_OFF" }
// 0100080040: Keywords { "DIRECTIONAL" "SHADOWS_SCREEN" }
// 0040020010: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_OFF" }
// 0200100080: Keywords { "DIRECTIONAL_COOKIE" "SHADOWS_SCREEN" }
// 2001000800: Keywords { "POINT" "SHADOWS_CUBE" "SHADOWS_SOFT" }
// 0400200100: Keywords { "POINT" "SHADOWS_CUBE" }
// 0004002001: Keywords { "POINT" "SHADOWS_OFF" }
// 4002001000: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" "SHADOWS_SOFT" }
// 0800400200: Keywords { "POINT_COOKIE" "SHADOWS_CUBE" }
// 0020010008: Keywords { "POINT_COOKIE" "SHADOWS_OFF" }
// 1000800400: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SHADOWS_SOFT" "SPOT" }
// 0080040020: Keywords { "SHADOWS_DEPTH" "SHADOWS_NATIVE" "SPOT" }
// 0010008004: Keywords { "SHADOWS_OFF" "SPOT" }
// Bind "vertex" Vertex
// Bind "normal" Normal
// Matrix 4 [glstate_matrix_modelview0] 3
// Matrix 0 [glstate_matrix_mvp]
// Float 9 [_LightAsQuad]
// Vector 7 [_ProjectionParams]
// Vector 8 [_ScreenParams]
// }
// }
// }
// }
// }
//
// Headers extracted with DarkStarSword's extract_unity_shaders.py
// https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py

vs_3_0
def c10, 0.5, -1, 1, 0
dcl_position v0
dcl_normal v1
dcl_position o0
dcl_texcoord o1
dcl_texcoord1 o2.xyz

def c220, 0, 1, 0.0625, 1
dcl_2d s0

// Rearranged to simplify shader:
dp4 r0.x, c0, v0
dp4 r0.y, c1, v0
dp4 r0.z, c2, v0
dp4 r0.w, c3, v0
mov o0, r0 // moved up before the stereo correction

// Apply texcoord correction only when not drawing a full screen quad:
if_ne r0.w, c220.y
// Apply stereo correction to screen coords stored in texcoord:
texldl r31, c220.z, s0
add r31.w, r0.w, -r31.y
mul r31.w, r31.w, r31.x
add r0.x, r0.x, r31.w
endif

mul r1.x, r0.y, c7.x
mul r1.w, r1.x, c10.x
mul r1.xz, r0.xyww, c10.x
mad o1.xy, r1.z, c8.zwzw, r1.xwzw
dp4 r1.x, c4, v0
dp4 r1.y, c5, v0
dp4 r1.z, c6, v0

// Apply view-space ray correction only when not drawing a full screen quad:
if_ne r0.w, c220.y

// This technique has changed a little starting with the Unity 5 update
// of Dreamfall Chapters. The old technique still works just fine, but
// I wanted to adjust _WorldSpaceCameraPos in a number of shaders and
// my quest to find a way to solve both problems reliably (by script)
// led me to find a way to fix the lighting shaders using world-space
// coordinates instead of view-space coordinates - that way, I could
// keep the number of matrices I needed to copy with Helix Mod down to
// 2 (the limit).
//
// This is the one part of the lighting fix that still needs a
// view-space correction, but fortunately we only need to do this when
// the shader has valid MV and MVP matrices so we can still derive the
// inverse projection matrix to get the FOV like usual.

// We can't use Helix mod to invert the MV matrix since we are already
// using both matrix copy slots for the inverse MVP and _Object2World
// matrices, and the MV matrix in Unity 5 is only a 3x4 matrix, while
// Helix Mod expects a 4x4 matrix.
// So let's do it directly in assembly instead! It sounds complicated,
// but we can make some large simplifications based on assumptions
// about the MV and MVP matrices and the fact we only need one cell of
// the (inverse) projection matrix, which means it is actually possible
// to do in a farily small number of instructions :)

// Use a temporary register for MV[1] to get around the fact that some
// of the below instructions need to read from more constant registers
// than there are available read ports:
mov r26, c5

// 1. Calculate 1/determinant of the MV matrix, simplifying by assuming the
// 4th column of the MV matrix is 0,0,0,1
//
// mathomatic simplified it to:
// 1 / ((m12*((m20*m01) - (m21*m00))) + (m02*((m21*m10) - (m20*m11))) + (m22*((m00*m11) - (m01*m10))));
//
// Replace row numbers with register components (assumes column-major order):
// (mv2.x*((mv0.y*mv1.z) - (mv0.z*mv1.y)))
// + (mv2.y*((mv0.z*mv1.x) - (mv0.x*mv1.z)))
// + (mv2.z*((mv0.x*mv1.y) - (mv0.y*mv1.x)))

// Do some multiplications in parallel with SIMD instructions:
mul r20.xyz, c4.yzx, r26.zxy // mv0.y*mv1.z, mv0.z*mv1.x, mv0.x*mv1.y
mul r21.xyz, c4.zxy, r26.yzx // mv0.z*mv1.y, mv0.x*mv1.z, mv0.y*mv1.x
// Do the subtractions:
add r20.xyz, r20.xyz, -r21.xyz // mv0.y*mv1.z - mv0.z*mv1.y, mv0.z*mv1.x - mv0.x*mv1.z, mv0.x*mv1.y - mv0.y*mv1.x
// Now the final three multiplications and the two additions:
dp3 r22.w, r20.xyz, c6.xyz
// And finally get 1/determinant:
rcp r22.w, r22.w

// 2. Calculate the 1st row of the inverted MV matrix, simplifying by assuimg
// the 4th column of the MV matrix is 0,0,0,1
//
// m00 = (mv1.y*mv2.z - mv1.z*mv2.y) / determinant
// m01 = (mv1.z*mv2.x - mv1.x*mv2.z) / determinant
// m02 = (mv1.x*mv2.y - mv1.y*mv2.x) / determinant

// Do some multiplications in parallel with SIMD instructions:
mul r20.xyz, r26.yzx, c6.zxy // mv1.y*mv2.z, mv1.z*mv2.x, mv1.x*mv2.y
mul r21.xyz, r26.zxy, c6.yzx // mv1.z*mv2.y, mv1.x*mv2.z, mv1.y*mv2.x
// Do the subtractions:
add r20.xyz, r20.xyz, -r21.xyz // mv1.y*mv2.z - mv1.z*mv2.y, mv1.z*mv2.x - mv1.x*mv2.z, mv1.x*mv2.y - mv1.y*mv2.x
// Multiply against 1/determinant:
mul r20.xyz, r20.xyz, r22.www

// 3. Multiply the first row of the inverted MV matrix with the 1st column of
// the MVP matrix (MV.I[0,3] is 0, so only worry about the 1st three):
dp3 r30.x, r20.xyz, c0.xyz

rcp r30.x, r30.x // Calculate P.I[0,0]

// Apply view-space stereo correction to ray:
mad r1.x, r31.w, r30.x, r1.x
endif

mul r2.xyz, r1, c10.yyzw
mad r1.xyz, r1, -c10.yyzw, v1
mad o2.xyz, c9.x, r1, r2
mov o1.zw, r0


Then, all that's left is adding all the sections to the DX9Settings.ini to copy the matrices from any suitable shaders into every shader where we want to apply the correction:

[VSEED89599]
; Copy inversed MVP matrix and _Object2World matrix in for Unity reflection fix
UseMatrix = true
MatrixReg = 180
UseMatrix1 = true
MatrixReg1 = 190
; Candidate to obtain MVP and _Object2World matrices:
GetMatrixFromReg = 0
InverseMatrix = true
GetMatrixFromReg1 = 4

[PSB1717770]
; Copy inversed MVP matrix and _Object2World matrix in for world-space variant of Unity lighting fix
UseMatrix = true
MatrixReg = 180
UseMatrix1 = true
MatrixReg1 = 190

; and so on...


But there's a gotcha here - some shaders appear to have the matrices we want, but attempting to use them will break everything! I have determined that any shaders with "LIGHTMODE"="SHADOWCASTER" in the headers are a no-no - these shaders are used while rendering shadow maps, when the matrices will be for the scene rendered from the perspective of a light source, but we need them for the perspective of the camera. There may be other shaders to avoid as well - I'm currently limiting the set I use to those that themselves use _WorldSpaceCameraPos which I figure are more likely to be safe.

In the next post I'll show how I combine all my scripts together to automatically apply this technique to a DX9 Unity game and fix it from start to finish.

Edit: Simplified 05F7E52C.txt

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#5
Posted 12/12/2015 03:31 PM   
[quote="eqzitara"]Unity games are dx9 but unplayable as far as i know[/quote] (quote from 5-19-2013) https://forums.geforce.com/default/topic/544546/ Unity games were and are such a mess in 3D and remain unplayable out of the box, but thanks to you DarkStarSword and all of the other fixers, coders, testers, etc.., the above statement no longer applies. Many thanks to all... ohh and it's also possible to fix Unity games in OpenGL now :) https://forums.geforce.com/default/topic/891735/
eqzitara said:Unity games are dx9 but unplayable as far as i know

(quote from 5-19-2013) https://forums.geforce.com/default/topic/544546/

Unity games were and are such a mess in 3D and remain unplayable out of the box, but thanks to you DarkStarSword and all of the other fixers, coders, testers, etc.., the above statement no longer applies.

Many thanks to all...

ohh and it's also possible to fix Unity games in OpenGL now :)

https://forums.geforce.com/default/topic/891735/

#6
Posted 12/12/2015 03:44 PM   
Dreamfall Chapters currently has [color="orange"][b]six thousand, five hundred and sixty seven[/b][/color] shaders that use _WorldSpaceCameraPos. Now while most of these probably won't have any noticeable effect when fixed in game, many of them will - certainly a lot more than I want to fix by hand anyway. So, this is my secret: [code] #!/bin/sh -e DIR=~/3d-fixes PATH="$DIR:$PATH" unity_asset_extractor.py Dreamfall\ Chapters_Data/Resources/* Dreamfall\ Chapters_Data/*.assets cd extracted extract_unity_shaders.py */*.shader --type=d3d9 cd ShaderCRCs cleanup_unity_shaders.py ../.. # Order from most specific targets to least specific targets shadertool.py -I ../.. --disable=0 --condition=c200.x Hidden_Coord/*/* Hidden_Depth/*/* Hidden_DepthBreaks/*/* Hidden_Final\ Interpolation/*/* Hidden_InterpolateAlongRays/*/* Hidden_Raymarch/*/* shadertool.py -I ../.. --fix-unity-lighting-ps-world --only-autofixed Hidden_Internal-Deferred*/fp/* Hidden_Internal-PrePass*/fp/* Hidden_ShadowSoftener-*/fp/* | unix2dos | tee -a ../../DX9Settings.ini shadertool.py -I ../.. --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed --ignore-register-errors */vp/*.txt | unix2dos | tee -a ../../DX9Settings.ini shadertool.py -I ../.. --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini [/code] This is a shell script - it's similar to a Windows batch file, only much more powerful (you'll see some of this in the examples for Stranded Deep and The Forest). These shell scripts come from the UNIX world, but you can install cygwin on Windows to use them here as well. Let's go through each of my scripts that calls: [code] unity_asset_extractor.py Dreamfall\ Chapters_Data/Resources/* Dreamfall\ Chapters_Data/*.assets [/code] This is my answer to the pain in the butt that was Unity Asset Manager - it extracts every .shader from the asset files in bulk to a directory called "extracted". It can handle all asset formats used by Unity for all the games I've thrown at it, including the new formats used by Unity 5. It currently only extracts shader files since that's all our community cares about, but it would be relatively straight forward to have it extract other files as well. Just give it a list of all the asset files - all Unity games have their asset files in the same locations - there will be a directory starting with the name of the executable and ending in _data. In that directory is a whole bunch of .assets files and another subdirectory called Resources with the standard Unity shaders. [code] cd extracted extract_unity_shaders.py */*.shader --type=d3d9 [/code] This script parses the .shader files from Unity and extracts shader assembly files with headers attached (which is necessary to find the matrices, especially as the shaders dumped from Helix Mod will not have any headers). If you have shaderasm.exe in the same directory as the script (just pull down my whole 3d-fixes repo) it will produce filenames that match the CRC that Helix Mod uses, so they can be copied straight in to ShaderOverride. If you are using this on a DX9 Unity 4 game, in some cases you might want to add --vs-fog (and perhaps --ps-fog). These options will duplicate all shaders that it finds and add some extra instructions to match the way Unity 4 handles fog. If you cannot loc.ate (huh, the word "l o c a t e" triggered the forum's IDS?) a shader dumped via Helix mod in the extracted shaders, it's likely you will need these options. These options will slow down the run (--vs-fog will double the number of vertex shaders it needs to calculate the CRC for and extract, and --ps-fog will tripple the number of pixel shaders). So far this does not seem necessary for Unity 5 games. [code] cleanup_unity_shaders.py ../.. [/code] You probably don't want to use this, and possibly shouldn't use this. It deletes any shaders in ShaderOverride and the git repository that are not found in the extracted game files. It is intended for use only in games that have been updated with no possibility that someone would still be using an old version. I'm using it for early access and episodic games where I can be fairly certain no one is using an old version. If you do use it, be sure you have some way to get the deleted shaders back in case it removes something you spent weeks working on (for my workflow, they are all tracked in git so this is not a problem). [code] cd ShaderCRCs shadertool.py -I ../.. --disable=0 --condition=c200.x Hidden_Coord/*/* Hidden_Depth/*/* Hidden_DepthBreaks/*/* Hidden_Final\ Interpolation/*/* Hidden_InterpolateAlongRays/*/* Hidden_Raymarch/*/* [/code] shadertool is my bread and butter tool for working with DX9 games. Run shadertool.py --help to see what sorts of things it is capable of. Here I'm using it to disable all shaders that are used in the volumetric ray-marched light shaft effect which I'm pretty convinced is unfixable. When it disables an effect it always adds a 3D enabled test, so if someone switches the game to 2D the light shafts will reappear, and I've also given it a --condition=c200.x, which is Const1 from DX9Settings.ini so they can also be re-enabled with a keybinding. I'm doing this first since I'm running it on one specific effect - the following commands are run on all shaders and could potentially match some of the shaders used by this effect as well, so I want this one to take priority (also, I'm not telling shadertool to overwrite shaders in these commands). [code] shadertool.py -I ../.. --fix-unity-lighting-ps-world --only-autofixed Hidden_Internal-Deferred*/fp/* Hidden_Internal-PrePass*/fp/* Hidden_ShadowSoftener-*/fp/* | unix2dos | tee -a ../../DX9Settings.ini [/code] Now we're starting to apply the new technique! Here I'm applying the new world-space variant of the Unity lighting fix, which also includes the reflection fix for these shaders (I may change that in future and split this into two separate options). Here I'm applying it to all "fragment shaders" (AKA pixel shaders, in blah/fp/*) for shaders I know to be lighting shaders used by this game. See the below examples in The Forest and Stranded Deep for a more generic method that does not need you to know the names of the shaders. You might also notice at the end of that line I'm "piping" (with the | character) the "standard output" of that command through unix2dos (which changes the newlines to Windows style) then to the tee command to append it to the DX9Settings.ini and simultaneously display it on the console. shadertool prints out any new sections that needed to be added to it's "standard output" stream, so by redirecting that into the DX9Settings.ini I can add them straight from the script! I could have alternatively used ">>" to have the shell append the file directly instead of using "| tee -a", but then I would not see the sections on the console - and I wanted a reminder that it had updated the DX9Settings.ini file. You might notice that shadertool prints out a lot of informational messages as well while it does it's thing, and these should not go in the DX9Settings. But don't worry - these messages are printed on it's "standard error" stream and won't be redirected with these commands. [code] shadertool.py -I ../.. --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed --ignore-register-errors */vp/*.txt | unix2dos | tee -a ../../DX9Settings.ini [/code] In this one command I am doing several things to all vertex shaders in the game: - I'm fixing (almost) all halo issues with --auto-fix-vertex-halo (my secret weapon, useful in almost any game)! - I'm adjusting _WorldSpaceCameraPos with --fix-unity-reflection - I'm accounting for differences between the way fog works in Unity between vs_2_0 and vs_3_0 for any shaders that the tool upgrades (unconfirmed if this is necessary on Unity 5, but it doesn't seem to hurt) - I'm telling the tool to skip any shaders that don't have any free sampler registers with --ignore-register-errors - I'm telling the tool to install the shader to ../../ShaderOverride/VertexShaders with -I, but only if either of the two autofixes I enabled were applied with --only-autofixed - I'm adding any extra sections I need to the DX9Settings.ini (mostly for the reflection fix, but it will also add sections for any shaders that were already using s0 so they use a different sampler instead) [code] shadertool.py -I ../.. --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini [/code] This time I'm running it on all pixel shaders and auto-fix _WorldSpaceCameraPos. --ignore-register-errors is there so that it won't stop if it finds a shader that does not have any free sampler registers, and will skip them instead. Now, the next time Dreamfall Chapters gets updated all I have to do is: [code] $ cd /cygdrive/c/Steam/SteamApps/common/Dreamfall\ Chapters/ $ rm -frv extracted $ ~/3d-fixes/Dreamfall\ Chapters/autofix.sh $ rsync -avP ShaderOverride/ ~/3d-fixes/Dreamfall\ Chapters/ShaderOverride/ $ cp DX9Settings.ini ~/3d-fixes/Dreamfall\ Chapters/ $ cd ~/3d-fixes/ $ ./mkrelease.sh Dreamfall\ Chapters/ dreamfall [/code] Presumably I would still run the game at some point to make sure the auto-fix is ok and check for any other issues (plus make sure my auto-HUD code is working as intended), but there's not much more to it - a Unity game transformed from unplayable to beautiful in a matter of minutes. Here's my autofix.sh for The Forest (and thanks to yesterday's 0.29 update I have tested that this does work to update a fix for real): [code] #!/bin/sh -e DIR=~/3d-fixes PATH="$DIR:$PATH" unity_asset_extractor.py *_data/Resources/* *_data/*.assets cd extracted extract_unity_shaders.py */*.shader --type=d3d9 cd ShaderCRCs cleanup_unity_shaders.py ../.. # Lighting fix - match any shaders that use known lighting vertex shaders: find . \( -name 05F7E52C.txt -o -name 678DC18B.txt \) -a -print0 | xargs -0 dirname -z | sed -z 's/vp$/fp\/*/' | xargs -0 \ shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-lighting-ps-world --only-autofixed | unix2dos | tee -a ../../DX9Settings.ini # Blacklist Ocean shaders as some of the vertex shaders do not have any free # sampler registers, so halos must be fixed in their pixel shaders instead find -maxdepth 1 -print0 | grep -zv '\(CetoTF_OceanTopSide\|CetoTF_OceanUnderSide\|Ceto_OceanMask\)' | sed -z 's/$/\/vp\/*/' | xargs -0 \ shadertool.py -I ../.. --stereo-sampler-vs=s3 --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed | unix2dos | tee -a ../../DX9Settings.ini # Water halo fix in the pixel shader: shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --adjust-input=texcoord4 --adjust-input=texcoord5 --adjust-multiply=0.5 --ignore-other-errors CetoTF_OceanTopSide*/fp/* | unix2dos | tee -a ../../DX9Settings.ini shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --adjust-input=texcoord4 --adjust-input=texcoord5 --adjust-multiply=0.5 --ignore-other-errors CetoTF_OceanUnderSide*/fp/* | unix2dos | tee -a ../../DX9Settings.ini # Reflection fix: shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini [/code] As you can see it's doing much the same thing as the Dreamfall Chapters fix, but there's a few differences: - It's using a custom sampler register for both vertex and pixel shaders. The vertex shader change is necessary in this game and Stranded Deep to work around Helix Mod bugs that break Ceto water with the default samplers, even if none of the Ceto shaders are in ShaderOverride. IIRC the pixel shader sampler change was also necessary to avoid breaking shadows in an older version of Stranded Deep (and again this did not matter that the shader was not in ShaderOverride). - It is locating Unity deferred lighting pixel shaders by searching for any that have the corresponding vertex shader 05F7E52C.txt or 678DC18B.txt. For this game I could just list them all again, but I wanted to find a way that would work for all Unity 5 games regardless of which custom lighting shaders they were using. - I'm blacklisting all the Ceto Ocean topside & underside vertex shaders. Ceto uses all four sampler registers in some of their vertex shaders and as a result they cannot be fixed with Helix Mod, and MUST NOT be copied to ShaderFixes under any circumstances otherwise Helix Mod WILL clobber one of their samplers! - I'm applying a 1/2 stereo correction to two of the texcoord inputs to the Ceto Ocean topside & underside pixel shaders to compensate for the fact that I did not auto fix their vertex shaders (note that the version of Ceto in Stranded Deep used a slightly different fix, but same concept).
Dreamfall Chapters currently has six thousand, five hundred and sixty seven shaders that use _WorldSpaceCameraPos. Now while most of these probably won't have any noticeable effect when fixed in game, many of them will - certainly a lot more than I want to fix by hand anyway.

So, this is my secret:
#!/bin/sh -e

DIR=~/3d-fixes
PATH="$DIR:$PATH"

unity_asset_extractor.py Dreamfall\ Chapters_Data/Resources/* Dreamfall\ Chapters_Data/*.assets
cd extracted
extract_unity_shaders.py */*.shader --type=d3d9
cd ShaderCRCs

cleanup_unity_shaders.py ../..

# Order from most specific targets to least specific targets
shadertool.py -I ../.. --disable=0 --condition=c200.x Hidden_Coord/*/* Hidden_Depth/*/* Hidden_DepthBreaks/*/* Hidden_Final\ Interpolation/*/* Hidden_InterpolateAlongRays/*/* Hidden_Raymarch/*/*
shadertool.py -I ../.. --fix-unity-lighting-ps-world --only-autofixed Hidden_Internal-Deferred*/fp/* Hidden_Internal-PrePass*/fp/* Hidden_ShadowSoftener-*/fp/* | unix2dos | tee -a ../../DX9Settings.ini
shadertool.py -I ../.. --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed --ignore-register-errors */vp/*.txt | unix2dos | tee -a ../../DX9Settings.ini
shadertool.py -I ../.. --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini

This is a shell script - it's similar to a Windows batch file, only much more powerful (you'll see some of this in the examples for Stranded Deep and The Forest). These shell scripts come from the UNIX world, but you can install cygwin on Windows to use them here as well. Let's go through each of my scripts that calls:

unity_asset_extractor.py Dreamfall\ Chapters_Data/Resources/* Dreamfall\ Chapters_Data/*.assets

This is my answer to the pain in the butt that was Unity Asset Manager - it extracts every .shader from the asset files in bulk to a directory called "extracted". It can handle all asset formats used by Unity for all the games I've thrown at it, including the new formats used by Unity 5. It currently only extracts shader files since that's all our community cares about, but it would be relatively straight forward to have it extract other files as well.

Just give it a list of all the asset files - all Unity games have their asset files in the same locations - there will be a directory starting with the name of the executable and ending in _data. In that directory is a whole bunch of .assets files and another subdirectory called Resources with the standard Unity shaders.

cd extracted
extract_unity_shaders.py */*.shader --type=d3d9

This script parses the .shader files from Unity and extracts shader assembly files with headers attached (which is necessary to find the matrices, especially as the shaders dumped from Helix Mod will not have any headers). If you have shaderasm.exe in the same directory as the script (just pull down my whole 3d-fixes repo) it will produce filenames that match the CRC that Helix Mod uses, so they can be copied straight in to ShaderOverride.

If you are using this on a DX9 Unity 4 game, in some cases you might want to add --vs-fog (and perhaps --ps-fog). These options will duplicate all shaders that it finds and add some extra instructions to match the way Unity 4 handles fog. If you cannot loc.ate (huh, the word "l o c a t e" triggered the forum's IDS?) a shader dumped via Helix mod in the extracted shaders, it's likely you will need these options. These options will slow down the run (--vs-fog will double the number of vertex shaders it needs to calculate the CRC for and extract, and --ps-fog will tripple the number of pixel shaders). So far this does not seem necessary for Unity 5 games.

cleanup_unity_shaders.py ../..

You probably don't want to use this, and possibly shouldn't use this. It deletes any shaders in ShaderOverride and the git repository that are not found in the extracted game files. It is intended for use only in games that have been updated with no possibility that someone would still be using an old version. I'm using it for early access and episodic games where I can be fairly certain no one is using an old version. If you do use it, be sure you have some way to get the deleted shaders back in case it removes something you spent weeks working on (for my workflow, they are all tracked in git so this is not a problem).

cd ShaderCRCs
shadertool.py -I ../.. --disable=0 --condition=c200.x Hidden_Coord/*/* Hidden_Depth/*/* Hidden_DepthBreaks/*/* Hidden_Final\ Interpolation/*/* Hidden_InterpolateAlongRays/*/* Hidden_Raymarch/*/*


shadertool is my bread and butter tool for working with DX9 games. Run shadertool.py --help to see what sorts of things it is capable of.

Here I'm using it to disable all shaders that are used in the volumetric ray-marched light shaft effect which I'm pretty convinced is unfixable. When it disables an effect it always adds a 3D enabled test, so if someone switches the game to 2D the light shafts will reappear, and I've also given it a --condition=c200.x, which is Const1 from DX9Settings.ini so they can also be re-enabled with a keybinding.

I'm doing this first since I'm running it on one specific effect - the following commands are run on all shaders and could potentially match some of the shaders used by this effect as well, so I want this one to take priority (also, I'm not telling shadertool to overwrite shaders in these commands).

shadertool.py -I ../.. --fix-unity-lighting-ps-world --only-autofixed Hidden_Internal-Deferred*/fp/* Hidden_Internal-PrePass*/fp/* Hidden_ShadowSoftener-*/fp/* | unix2dos | tee -a ../../DX9Settings.ini

Now we're starting to apply the new technique! Here I'm applying the new world-space variant of the Unity lighting fix, which also includes the reflection fix for these shaders (I may change that in future and split this into two separate options). Here I'm applying it to all "fragment shaders" (AKA pixel shaders, in blah/fp/*) for shaders I know to be lighting shaders used by this game. See the below examples in The Forest and Stranded Deep for a more generic method that does not need you to know the names of the shaders.

You might also notice at the end of that line I'm "piping" (with the | character) the "standard output" of that command through unix2dos (which changes the newlines to Windows style) then to the tee command to append it to the DX9Settings.ini and simultaneously display it on the console. shadertool prints out any new sections that needed to be added to it's "standard output" stream, so by redirecting that into the DX9Settings.ini I can add them straight from the script!

I could have alternatively used ">>" to have the shell append the file directly instead of using "| tee -a", but then I would not see the sections on the console - and I wanted a reminder that it had updated the DX9Settings.ini file.

You might notice that shadertool prints out a lot of informational messages as well while it does it's thing, and these should not go in the DX9Settings. But don't worry - these messages are printed on it's "standard error" stream and won't be redirected with these commands.

shadertool.py -I ../.. --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed --ignore-register-errors */vp/*.txt | unix2dos | tee -a ../../DX9Settings.ini

In this one command I am doing several things to all vertex shaders in the game:
- I'm fixing (almost) all halo issues with --auto-fix-vertex-halo (my secret weapon, useful in almost any game)!
- I'm adjusting _WorldSpaceCameraPos with --fix-unity-reflection
- I'm accounting for differences between the way fog works in Unity between vs_2_0 and vs_3_0 for any shaders that the tool upgrades (unconfirmed if this is necessary on Unity 5, but it doesn't seem to hurt)
- I'm telling the tool to skip any shaders that don't have any free sampler registers with --ignore-register-errors
- I'm telling the tool to install the shader to ../../ShaderOverride/VertexShaders with -I, but only if either of the two autofixes I enabled were applied with --only-autofixed
- I'm adding any extra sections I need to the DX9Settings.ini (mostly for the reflection fix, but it will also add sections for any shaders that were already using s0 so they use a different sampler instead)

shadertool.py -I ../.. --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini

This time I'm running it on all pixel shaders and auto-fix _WorldSpaceCameraPos. --ignore-register-errors is there so that it won't stop if it finds a shader that does not have any free sampler registers, and will skip them instead.

Now, the next time Dreamfall Chapters gets updated all I have to do is:
$ cd /cygdrive/c/Steam/SteamApps/common/Dreamfall\ Chapters/
$ rm -frv extracted
$ ~/3d-fixes/Dreamfall\ Chapters/autofix.sh
$ rsync -avP ShaderOverride/ ~/3d-fixes/Dreamfall\ Chapters/ShaderOverride/
$ cp DX9Settings.ini ~/3d-fixes/Dreamfall\ Chapters/
$ cd ~/3d-fixes/
$ ./mkrelease.sh Dreamfall\ Chapters/ dreamfall

Presumably I would still run the game at some point to make sure the auto-fix is ok and check for any other issues (plus make sure my auto-HUD code is working as intended), but there's not much more to it - a Unity game transformed from unplayable to beautiful in a matter of minutes.



Here's my autofix.sh for The Forest (and thanks to yesterday's 0.29 update I have tested that this does work to update a fix for real):

#!/bin/sh -e

DIR=~/3d-fixes
PATH="$DIR:$PATH"

unity_asset_extractor.py *_data/Resources/* *_data/*.assets
cd extracted
extract_unity_shaders.py */*.shader --type=d3d9
cd ShaderCRCs

cleanup_unity_shaders.py ../..

# Lighting fix - match any shaders that use known lighting vertex shaders:
find . \( -name 05F7E52C.txt -o -name 678DC18B.txt \) -a -print0 | xargs -0 dirname -z | sed -z 's/vp$/fp\/*/' | xargs -0 \
shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-lighting-ps-world --only-autofixed | unix2dos | tee -a ../../DX9Settings.ini

# Blacklist Ocean shaders as some of the vertex shaders do not have any free
# sampler registers, so halos must be fixed in their pixel shaders instead
find -maxdepth 1 -print0 | grep -zv '\(CetoTF_OceanTopSide\|CetoTF_OceanUnderSide\|Ceto_OceanMask\)' | sed -z 's/$/\/vp\/*/' | xargs -0 \
shadertool.py -I ../.. --stereo-sampler-vs=s3 --auto-fix-vertex-halo --fix-unity-reflection --add-fog-on-sm3-update --only-autofixed | unix2dos | tee -a ../../DX9Settings.ini

# Water halo fix in the pixel shader:
shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --adjust-input=texcoord4 --adjust-input=texcoord5 --adjust-multiply=0.5 --ignore-other-errors CetoTF_OceanTopSide*/fp/* | unix2dos | tee -a ../../DX9Settings.ini
shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --adjust-input=texcoord4 --adjust-input=texcoord5 --adjust-multiply=0.5 --ignore-other-errors CetoTF_OceanUnderSide*/fp/* | unix2dos | tee -a ../../DX9Settings.ini

# Reflection fix:
shadertool.py -I ../.. --stereo-sampler-ps=s15 --fix-unity-reflection --only-autofixed --ignore-register-errors */fp/*.txt | unix2dos | tee -a ../../DX9Settings.ini


As you can see it's doing much the same thing as the Dreamfall Chapters fix, but there's a few differences:
- It's using a custom sampler register for both vertex and pixel shaders. The vertex shader change is necessary in this game and Stranded Deep to work around Helix Mod bugs that break Ceto water with the default samplers, even if none of the Ceto shaders are in ShaderOverride. IIRC the pixel shader sampler change was also necessary to avoid breaking shadows in an older version of Stranded Deep (and again this did not matter that the shader was not in ShaderOverride).
- It is locating Unity deferred lighting pixel shaders by searching for any that have the corresponding vertex shader 05F7E52C.txt or 678DC18B.txt. For this game I could just list them all again, but I wanted to find a way that would work for all Unity 5 games regardless of which custom lighting shaders they were using.
- I'm blacklisting all the Ceto Ocean topside & underside vertex shaders. Ceto uses all four sampler registers in some of their vertex shaders and as a result they cannot be fixed with Helix Mod, and MUST NOT be copied to ShaderFixes under any circumstances otherwise Helix Mod WILL clobber one of their samplers!
- I'm applying a 1/2 stereo correction to two of the texcoord inputs to the Ceto Ocean topside & underside pixel shaders to compensate for the fact that I did not auto fix their vertex shaders (note that the version of Ceto in Stranded Deep used a slightly different fix, but same concept).

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#7
Posted 12/12/2015 05:05 PM   
As a bonus, you might notice that all the non-directional lighting shaders that shadertool fixes with --fix-unity-lighting-ps-world produce a section in the DX9Settings.ini file like this: [code] [PS10606CB1] ; Copy inversed MVP matrix and _Object2World matrix in for world-space variant of Unity lighting fix UseMatrix = true MatrixReg = 180 UseMatrix1 = true MatrixReg1 = 190 ; Copy _CameraDepthTexture and _ZBufferParams for auto HUD adjustment GetSampler1FromReg = 0 GetConst1FromReg = 9 [/code] Ok, the two matrices are obviously to allow us to correct world space coordinates in that shader, but what's this down the bottom about an auto HUD adjustment? Well, the tool doesn't do anything with those, but I added that for convinience since I've been experimenting a lot with auto crosshairs and auto HUDs recently, where I need access to the depth buffer and those shaders seemed like a convinient place to find it. Then I might do something like this to inject the depth buffer into a HUD shader: [code] [VSFE267E35] ; Copy _CameraDepthTexture and _ZBufferParams in for auto-crosshair depth: SetSampler1ToReg = 259 SetConst1ToReg = 150 [VBFE267E35.0] [/code] That would be suitable for adjusting a HUD based on a point at the center of the screen (suitable for a crosshair, or a game with a minimal HUD), or possibly to adjust each vertex independently (which would result in stretched icons). For Dreamfall Chapters, I got even more tricky: [code] ; HUD shaders [VS2FC2E74B] ; Copy _CameraDepthTexture and _ZBufferParams in for auto-HUD depth: SetSampler1ToReg = 260 SetConst1ToReg = 150 CheckTexCRC = true UseDefinedOnly = false ; Assign the first vertex FirstVertexPosReg = 170 ; We do NOT get the first vertex position from the shader, unless one of the ; below textures is matched: GetVertex = false ValNotDefined = 0 ValForDefined = 1 TexCounterReg = 251 DefinedTexturesVS = CAA23431;CC3C336C;CD161567;1DCD05B2;F7EDA18C;F842EB47; VBOffsetList = 0; [VB2FC2E74B.0] ; Since we are using FirstVertexPosReg in conjunction with DefinedTexturesVS we ; NEED to use the following additional TEX and VB sections. Additionally, since ; we are not using GetVertex in the main shader section, we set it in each ; individual texture section. This allows us to set the HUD depth based on the ; targetting icon, but stop adjusting the HUD when bringing up the selection ; icons, which keeps them all at a consistent depth with each other. [TEXCAA23431] ; eye GetVertex = true VBOffsetList = 0; [VBCAA23431.0] [TEXCC3C336C] ; targetting circle GetVertex = true VBOffsetList = 0; [VBCC3C336C.0] [TEXCD161567] ; cog GetVertex = true VBOffsetList = 0; [VBCD161567.0] [TEX1DCD05B2] ; talk GetVertex = true VBOffsetList = 0; [VB1DCD05B2.0] [TEXF7EDA18C] ; door GetVertex = true VBOffsetList = 0; [VBF7EDA18C.0] [TEXF842EB47] ; hand GetVertex = true VBOffsetList = 0; [VBF842EB47.0] [VS6AEB3DDF] ; Copy _CameraDepthTexture and _ZBufferParams in for auto-HUD depth: SetSampler1ToReg = 260 SetConst1ToReg = 150 CheckTexCRC = true UseDefinedOnly = false ; Get the position of the first vertex in the draw call: VBOffsetList = 0; ; Do NOT get the vertex information from this shader, but still assign it to ; the constant register. This allows us to use the vertex information from the ; icon shader instead GetVertex = false FirstVertexPosReg = 170 [VB6AEB3DDF.0] [/code] There I'm getting the position of the first vertex of the floating HUD when specific icons are displayed and using it to adjust those icons and the corresponding text to a consistent depth. This worked pretty well for Dreamfall, but was a no-go in The Forest since multiple icons are drawn simultaneously in that game (fortunately, most of the HUD in The Forest is already drawn at the correct depth in 3D). Nowadays 3DMigoto can do something similar to this, but it's quite a bit more powerful since you can use the current vertex ID to look up other vertices for the matching icon regardless of how many icons are drawn simultaneously. Here's the code from the Dreamfall Chapters HUD shader to auto-adjust it: [code] // HUD - floating icons // CRC32: 2FC2E74B | Unity headers extracted from Unlit - Transparent Colored.shader // Shader "Unlit/Transparent Colored" { // Properties { // _MainTex ("Base (RGB), Alpha (A)", 2D) = "black" { } // } // SubShader 1/2 { // LOD 100 // Tags { "QUEUE"="Transparent" "IGNOREPROJECTOR"="true" "RenderType"="Transparent" } // Pass 1/1 { // Tags { "QUEUE"="Transparent" "IGNOREPROJECTOR"="true" "RenderType"="Transparent" } // ZWrite Off // Cull Off // Blend SrcAlpha OneMinusSrcAlpha // Offset -1, -1 // GpuProgramID 19577 // Program "vp" { // SubProgram "d3d9 " { // Bind "vertex" Vertex // Bind "color" Color // Bind "texcoord" TexCoord0 // Matrix 0 [glstate_matrix_mvp] // } // } // } // } // } // // Headers extracted with DarkStarSword's extract_unity_shaders.py // https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py // UI - full screen fade to black and others vs_3_0 // Converted from vs_2_0 with DarkStarSword's shadertool.py dcl_position v0 dcl_texcoord v1 dcl_color v2 dcl_texcoord o0 dcl_position o1 dcl_color o2 def c220, 0, 1, 0.0625, 0.5 def c221, 1920, 1080, 0.0005208333333333333, 0.000925925925 dcl_2d s0 // For the auto HUD depth: dcl_2d s3 // Copy of _CameraDepthTexture defi i0, 255, 0, 0, 0 def c227, 0.00390625, 64, 0, 0.5 // 1/i0.x, 0.5 dp4 r0.x, c0, v0 dp4 r0.y, c1, v0 dp4 r0.z, c2, v0 dp4 r0.w, c3, v0 mov o0.xy, v1 mov o2, v2 // Determine an appropriate threshold coordinates near the corners for the // current resolution (1 - 2/resolution): rcp r30.z, c210.x // Instruction can only operate on one component at a time rcp r30.w, c210.y // And c210.zw doesn't seem to already be the reciprocal add r30.xy, r30.zwzw, r30.zwzw // 2/resolution add r30.xy, c220.y, -r30.xy // 1 - 2/resolution // Blacklist full screen coordinates at/beyond the screen corners: abs r31.xy, r0.xy mov r30.w, c220.y if_ge r31.x, r30.x if_ge r31.y, r30.y mov r30.w, c220.x endif endif // Check W == 1 to avoid double adjusting some icons (choice/balance icons) if_ne r0.w, c220.y mov r30.w, c220.x endif if_eq r30.w, c220.y texldl r31, c220.z, s0 // Auto depth: sample a number of points on the depth buffer to // determine an appropriate depth for this object, starting at the near // clipping plane and working towards original x + separation. // // You can think of this as a line in three dimensional space that // starts at each eye and stretches out towards infinity. We sample 255 // points along this line (evenly spaced in the X axis) and compare // with the depth buffer to find where the line is first intersected. // // Based on DarkStarSword's stereo crosshair code originally developed // for Miasmata, adapted to HLSL and made to use coordinates other than // (0,0), then adapted back to assembly for UE3, and now back to Unity. // Screen depth coordinates to sample around - Using the first // vertex position obtained with Helix Mod. Would be better to // use the center, but I don't think it is possible to look // that up in Helix Mod (would need to get the possition of a // diagonally opposite vertex). mov r23, c170 dp4 r22.x, c0, r23 dp4 r22.y, c1, r23 // near clipping plane - for Unity we get this from _ZBufferParams rcp r20.w, c150.w // offset = separation * (near - convergence) / near add r31.w, r20.w, -r31.y mul r20.x, r31.w, r31.x rcp r31.w, r20.w mul r20.x, r20.x, r31.w // Total distance to cover // distance = separation - starting X offset: add r20.y, r31.x, -r20.x // old_offset = offset mov r21.x, r20.x // Precompute some constants: // separation * convergence: mul r31.w, r31.x, r31.y // distance / 255: mul r20.z, r20.y, c227.x rep i0 // offset += distance / 255: add r20.x, r20.x, r20.z // Calculate W for this adjustment amount: // w = (separation * convergence) / (separation - offset): add r20.w, r31.x, -r20.x rcp r20.w, r20.w mul r20.w, r31.w, r20.w // sample_x = x + offset, sample_y = y (not flipped compared to UE3) mov r23.y, r22.y add r23.x, r22.x, r20.x // Convert to texture coordinates (coords / 2 + 0.5): mad r23.xy, r23.xy, c220.w, c220.w // Sample depth buffer: texldl r5, r23.xy, s3 // scale depth buffer to world Z (game dependent - look for scaling applied to // the depth buffer wherever else it is used. In the case of Unity we use // _ZBufferParams): mad r5.x, r5.x, c150.x, c150.y add r5.y, c150.z, c150.w // Derive 1/far from _ZBufferParams mul r5.x, r5.x, r5.y rcp r5.x, r5.x // If the sampled depth is closer than the calculated depth, exit the loop: break_ge r20.w, r5.x // old_offset = offset mov r21.x, r20.x endrep // Finally adjust the X coordinate: add r0.x, r0.x, r21.x endif mov o1, r0 [/code] This is the Unity version. This code has evolved quite a bit since the original incarnation I wrote for Miasmata, and there's now versions of it that work with Miasmata, a number of DX9 Unity games, UE3 DX9 (although I actually use that to fix bloom in Life Is Strange and Viscera Cleanup Detail), Witcher 3 (never worked very well - it's there but disabled) and Far Cry 4 (coming soon).
As a bonus, you might notice that all the non-directional lighting shaders that shadertool fixes with --fix-unity-lighting-ps-world produce a section in the DX9Settings.ini file like this:

[PS10606CB1]
; Copy inversed MVP matrix and _Object2World matrix in for world-space variant of Unity lighting fix
UseMatrix = true
MatrixReg = 180
UseMatrix1 = true
MatrixReg1 = 190
; Copy _CameraDepthTexture and _ZBufferParams for auto HUD adjustment
GetSampler1FromReg = 0
GetConst1FromReg = 9

Ok, the two matrices are obviously to allow us to correct world space coordinates in that shader, but what's this down the bottom about an auto HUD adjustment?

Well, the tool doesn't do anything with those, but I added that for convinience since I've been experimenting a lot with auto crosshairs and auto HUDs recently, where I need access to the depth buffer and those shaders seemed like a convinient place to find it. Then I might do something like this to inject the depth buffer into a HUD shader:
[VSFE267E35]
; Copy _CameraDepthTexture and _ZBufferParams in for auto-crosshair depth:
SetSampler1ToReg = 259
SetConst1ToReg = 150
[VBFE267E35.0]

That would be suitable for adjusting a HUD based on a point at the center of the screen (suitable for a crosshair, or a game with a minimal HUD), or possibly to adjust each vertex independently (which would result in stretched icons).

For Dreamfall Chapters, I got even more tricky:
; HUD shaders
[VS2FC2E74B]
; Copy _CameraDepthTexture and _ZBufferParams in for auto-HUD depth:
SetSampler1ToReg = 260
SetConst1ToReg = 150
CheckTexCRC = true
UseDefinedOnly = false
; Assign the first vertex
FirstVertexPosReg = 170
; We do NOT get the first vertex position from the shader, unless one of the
; below textures is matched:
GetVertex = false
ValNotDefined = 0
ValForDefined = 1
TexCounterReg = 251
DefinedTexturesVS = CAA23431;CC3C336C;CD161567;1DCD05B2;F7EDA18C;F842EB47;
VBOffsetList = 0;
[VB2FC2E74B.0]
; Since we are using FirstVertexPosReg in conjunction with DefinedTexturesVS we
; NEED to use the following additional TEX and VB sections. Additionally, since
; we are not using GetVertex in the main shader section, we set it in each
; individual texture section. This allows us to set the HUD depth based on the
; targetting icon, but stop adjusting the HUD when bringing up the selection
; icons, which keeps them all at a consistent depth with each other.
[TEXCAA23431]
; eye
GetVertex = true
VBOffsetList = 0;
[VBCAA23431.0]
[TEXCC3C336C]
; targetting circle
GetVertex = true
VBOffsetList = 0;
[VBCC3C336C.0]
[TEXCD161567]
; cog
GetVertex = true
VBOffsetList = 0;
[VBCD161567.0]
[TEX1DCD05B2]
; talk
GetVertex = true
VBOffsetList = 0;
[VB1DCD05B2.0]
[TEXF7EDA18C]
; door
GetVertex = true
VBOffsetList = 0;
[VBF7EDA18C.0]
[TEXF842EB47]
; hand
GetVertex = true
VBOffsetList = 0;
[VBF842EB47.0]

[VS6AEB3DDF]
; Copy _CameraDepthTexture and _ZBufferParams in for auto-HUD depth:
SetSampler1ToReg = 260
SetConst1ToReg = 150
CheckTexCRC = true
UseDefinedOnly = false
; Get the position of the first vertex in the draw call:
VBOffsetList = 0;
; Do NOT get the vertex information from this shader, but still assign it to
; the constant register. This allows us to use the vertex information from the
; icon shader instead
GetVertex = false
FirstVertexPosReg = 170
[VB6AEB3DDF.0]


There I'm getting the position of the first vertex of the floating HUD when specific icons are displayed and using it to adjust those icons and the corresponding text to a consistent depth. This worked pretty well for Dreamfall, but was a no-go in The Forest since multiple icons are drawn simultaneously in that game (fortunately, most of the HUD in The Forest is already drawn at the correct depth in 3D). Nowadays 3DMigoto can do something similar to this, but it's quite a bit more powerful since you can use the current vertex ID to look up other vertices for the matching icon regardless of how many icons are drawn simultaneously.

Here's the code from the Dreamfall Chapters HUD shader to auto-adjust it:
// HUD - floating icons

// CRC32: 2FC2E74B | Unity headers extracted from Unlit - Transparent Colored.shader
// Shader "Unlit/Transparent Colored" {
// Properties {
// _MainTex ("Base (RGB), Alpha (A)", 2D) = "black" { }
// }
// SubShader 1/2 {
// LOD 100
// Tags { "QUEUE"="Transparent" "IGNOREPROJECTOR"="true" "RenderType"="Transparent" }
// Pass 1/1 {
// Tags { "QUEUE"="Transparent" "IGNOREPROJECTOR"="true" "RenderType"="Transparent" }
// ZWrite Off
// Cull Off
// Blend SrcAlpha OneMinusSrcAlpha
// Offset -1, -1
// GpuProgramID 19577
// Program "vp" {
// SubProgram "d3d9 " {
// Bind "vertex" Vertex
// Bind "color" Color
// Bind "texcoord" TexCoord0
// Matrix 0 [glstate_matrix_mvp]
// }
// }
// }
// }
// }
//
// Headers extracted with DarkStarSword's extract_unity_shaders.py
// https://raw.githubusercontent.com/DarkStarSword/3d-fixes/master/extract_unity_shaders.py

// UI - full screen fade to black and others

vs_3_0 // Converted from vs_2_0 with DarkStarSword's shadertool.py
dcl_position v0
dcl_texcoord v1
dcl_color v2

dcl_texcoord o0
dcl_position o1
dcl_color o2


def c220, 0, 1, 0.0625, 0.5
def c221, 1920, 1080, 0.0005208333333333333, 0.000925925925
dcl_2d s0

// For the auto HUD depth:
dcl_2d s3 // Copy of _CameraDepthTexture
defi i0, 255, 0, 0, 0
def c227, 0.00390625, 64, 0, 0.5 // 1/i0.x, 0.5

dp4 r0.x, c0, v0
dp4 r0.y, c1, v0
dp4 r0.z, c2, v0
dp4 r0.w, c3, v0
mov o0.xy, v1
mov o2, v2

// Determine an appropriate threshold coordinates near the corners for the
// current resolution (1 - 2/resolution):
rcp r30.z, c210.x // Instruction can only operate on one component at a time
rcp r30.w, c210.y // And c210.zw doesn't seem to already be the reciprocal
add r30.xy, r30.zwzw, r30.zwzw // 2/resolution
add r30.xy, c220.y, -r30.xy // 1 - 2/resolution

// Blacklist full screen coordinates at/beyond the screen corners:
abs r31.xy, r0.xy
mov r30.w, c220.y
if_ge r31.x, r30.x
if_ge r31.y, r30.y
mov r30.w, c220.x
endif
endif

// Check W == 1 to avoid double adjusting some icons (choice/balance icons)
if_ne r0.w, c220.y
mov r30.w, c220.x
endif

if_eq r30.w, c220.y
texldl r31, c220.z, s0

// Auto depth: sample a number of points on the depth buffer to
// determine an appropriate depth for this object, starting at the near
// clipping plane and working towards original x + separation.
//
// You can think of this as a line in three dimensional space that
// starts at each eye and stretches out towards infinity. We sample 255
// points along this line (evenly spaced in the X axis) and compare
// with the depth buffer to find where the line is first intersected.
//
// Based on DarkStarSword's stereo crosshair code originally developed
// for Miasmata, adapted to HLSL and made to use coordinates other than
// (0,0), then adapted back to assembly for UE3, and now back to Unity.

// Screen depth coordinates to sample around - Using the first
// vertex position obtained with Helix Mod. Would be better to
// use the center, but I don't think it is possible to look
// that up in Helix Mod (would need to get the possition of a
// diagonally opposite vertex).
mov r23, c170
dp4 r22.x, c0, r23
dp4 r22.y, c1, r23

// near clipping plane - for Unity we get this from _ZBufferParams
rcp r20.w, c150.w

// offset = separation * (near - convergence) / near
add r31.w, r20.w, -r31.y
mul r20.x, r31.w, r31.x
rcp r31.w, r20.w
mul r20.x, r20.x, r31.w

// Total distance to cover
// distance = separation - starting X offset:
add r20.y, r31.x, -r20.x

// old_offset = offset
mov r21.x, r20.x

// Precompute some constants:
// separation * convergence:
mul r31.w, r31.x, r31.y
// distance / 255:
mul r20.z, r20.y, c227.x

rep i0
// offset += distance / 255:
add r20.x, r20.x, r20.z

// Calculate W for this adjustment amount:
// w = (separation * convergence) / (separation - offset):
add r20.w, r31.x, -r20.x
rcp r20.w, r20.w
mul r20.w, r31.w, r20.w

// sample_x = x + offset, sample_y = y (not flipped compared to UE3)
mov r23.y, r22.y
add r23.x, r22.x, r20.x
// Convert to texture coordinates (coords / 2 + 0.5):
mad r23.xy, r23.xy, c220.w, c220.w
// Sample depth buffer:
texldl r5, r23.xy, s3

// scale depth buffer to world Z (game dependent - look for scaling applied to
// the depth buffer wherever else it is used. In the case of Unity we use
// _ZBufferParams):
mad r5.x, r5.x, c150.x, c150.y
add r5.y, c150.z, c150.w // Derive 1/far from _ZBufferParams
mul r5.x, r5.x, r5.y
rcp r5.x, r5.x

// If the sampled depth is closer than the calculated depth, exit the loop:
break_ge r20.w, r5.x

// old_offset = offset
mov r21.x, r20.x
endrep

// Finally adjust the X coordinate:
add r0.x, r0.x, r21.x
endif

mov o1, r0

This is the Unity version. This code has evolved quite a bit since the original incarnation I wrote for Miasmata, and there's now versions of it that work with Miasmata, a number of DX9 Unity games, UE3 DX9 (although I actually use that to fix bloom in Life Is Strange and Viscera Cleanup Detail), Witcher 3 (never worked very well - it's there but disabled) and Far Cry 4 (coming soon).

Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword
Contact & PayPal: darkstarsword@gmail.com

#8
Posted 12/12/2015 05:43 PM   
Fantastic. The flat specularity in otherwise perfect 3D games was always a disappointing thing that sometimes can make surfaces look more fake in 3D than they do in 2D. That goes for parallax shaders as well but those are much more rare.
Fantastic. The flat specularity in otherwise perfect 3D games was always a disappointing thing that sometimes can make surfaces look more fake in 3D than they do in 2D. That goes for parallax shaders as well but those are much more rare.

#9
Posted 12/12/2015 07:36 PM   
Big Thank you for all the advanced and detailed information!!! I read through all of it and found it very interesting!! I know the next time will fix "the next unity game" all this information will be very appreciated and useful. Big big big Thank you again!!!
Big Thank you for all the advanced and detailed information!!!
I read through all of it and found it very interesting!!
I know the next time will fix "the next unity game" all this information will be very appreciated and useful.

Big big big Thank you again!!!


Life is as empty without Terror, as is without Love...
But thou Chaos can be beautiful...
It can not last...


-
2x Asus GTX 980Ti Strix (OC edition), Custom Water-cooling to kick the Temperature & Noise arse ^_^
-3x Asus VG278HE 3D Vision 2 monitors (3D Surround)
i7 4790K @ 5.0Ghz ontop of Asus Maximus VII Ranger
DDR3 Corsair Vengeance 16GB 2133
RAID0 256GB SSDs
RAID0 2TB Mechanical HDD (WD Black)
-
Alienware M17x R5
GT880M (8GB VRAM)
120hz 3D Samsung LCD Panel

16gb 1600Mhz DDR3
256GB SSD + 1TB mechanical HDD
-
Alienware M14x R1
GT555M (1.5GB VRAM) - Regular LG LCD
8GB 1600Mhz DDR3
256 SSD + 512GB mechanical HDD
-
Paypal for donations: tavyhome@gmail.com

#10
Posted 12/13/2015 12:05 PM   
As a long time fan of Dreamfall, I'm speechless with the amount of talent from you guys, I can't thank you enough.
As a long time fan of Dreamfall, I'm speechless with the amount of talent from you guys, I can't thank you enough.

All hail 3d modders Helifax, bo3b, mike_ar69, Flugan, DarkStarSword, 4everAwake, 3d4dd, eqzitara and so many more helping to keep the 3d dream alive, find their 3d fixes at http://helixmod.blogspot.com/ Also check my site for spanish VR and mobile gaming news: www.gamermovil.com

#11
Posted 12/13/2015 01:36 PM   
This is a revolution !!! I can't wait to see this on my own screen :) Thank you so much for your dedicated hard Work - you are a true hero DarkStarSword
This is a revolution !!!

I can't wait to see this on my own screen :)

Thank you so much for your dedicated hard Work - you are a true hero DarkStarSword

#12
Posted 12/13/2015 02:28 PM   
Scroll To Top