[quote="DarkStarSword"]
I've got an idea that might be able to get the projection matrix into the pixel shader if that would help with the FOV - GetConstNFromReg can copy a matrix passed into a VS and then SetConstNToReg can pass it in to the pixel shader. I haven't tried this yet, and I'm not familiar enough with the maths to know how to extract the FOV from the matrix even if this does work, but maybe someone can do something with this?[/quote]
Yes you can do this - I used this exact approach in Elder Scrolls Online - have a look at that fix to see what I did. There is no need to work out all the tan/atan stuff, the P_00 element already *is* the tan(FOV/2) value that you need to multiply the view space correction by. (I've often referred to it as P11 in the past because I think in terms of 1-4, but P00 is the nomenclature used in HLSL, basically its the first element). If you had the FOV you could work out that value, but if you have the Proj matrix you don't need to. Note that sometimes you need to divide by P_00 instead of multiply, but I can't remember why (I think this was the case in TESO).
DarkStarSword said:
I've got an idea that might be able to get the projection matrix into the pixel shader if that would help with the FOV - GetConstNFromReg can copy a matrix passed into a VS and then SetConstNToReg can pass it in to the pixel shader. I haven't tried this yet, and I'm not familiar enough with the maths to know how to extract the FOV from the matrix even if this does work, but maybe someone can do something with this?
Yes you can do this - I used this exact approach in Elder Scrolls Online - have a look at that fix to see what I did. There is no need to work out all the tan/atan stuff, the P_00 element already *is* the tan(FOV/2) value that you need to multiply the view space correction by. (I've often referred to it as P11 in the past because I think in terms of 1-4, but P00 is the nomenclature used in HLSL, basically its the first element). If you had the FOV you could work out that value, but if you have the Proj matrix you don't need to. Note that sometimes you need to divide by P_00 instead of multiply, but I can't remember why (I think this was the case in TESO).
Rig: Intel i5-3570K @4.5GHz, 16Gb Ram, SSD, 2xGTX 980 SLI, Win10x64, 3D Vision 2 Asus VG278, Optoma HD66, CyberPowerPC-Fangbook (i7-4810MQ, GTX780M), Driving Force Pro Wheel
I'm a bit concerned that none of the shaders seem to use the projection matrix by itself, only the MVP matrix.
Here are all the matrices used by shaders in Dreamfall Chapters - we have MV and MVP, is there some matrix operation I can do to extract the P from them? (I wish that NSW had included Matrix maths in the high school curriculum - I picked up a bit at university, but not as much as would be useful right now)
[code]$ grep -rih '^Matrix'|sort -k 3 -u
Matrix 0 [_CameraToWorld]
Matrix 5 [_FrustumCornersWS]
Matrix 0 [_FrustumRays]
Matrix 4 [glstate_matrix_invtrans_modelview0]
Matrix 0 [glstate_matrix_modelview0]
Matrix 0 [glstate_matrix_mvp]
Matrix 4 [glstate_matrix_texture0]
Matrix 8 [glstate_matrix_texture1]
Matrix 13 [_LightMatrix0]
Matrix 5 [_Object2World]
Matrix 0 [SkyMatrix]
Matrix 0 [_ToPrevViewProjCombined]
Matrix 13 [unity_World2Shadow]
Matrix 5 [unity_World2Shadow0]
Matrix 192 [unity_World2Shadow1]
Matrix 256 [unity_World2Shadow2]
Matrix 320 [unity_World2Shadow3]
Matrix 9 [_World2Object]
[/code]
http://docs.unity3d.com/Manual/SL-BuiltinValues.html has definitions of some of these, though the pre-processor has renamed a lot of them.
I'm a bit concerned that none of the shaders seem to use the projection matrix by itself, only the MVP matrix.
Here are all the matrices used by shaders in Dreamfall Chapters - we have MV and MVP, is there some matrix operation I can do to extract the P from them? (I wish that NSW had included Matrix maths in the high school curriculum - I picked up a bit at university, but not as much as would be useful right now)
ok, after playing around in numpy and re-familiarising myself with the maths, it looks like I just need to multiply the inverse of the model-view matrix by the model-view-projection matrix to get the projection matrix:
[code]
m=matrix.scale(10,10,10) * matrix.rotate_y(60) * matrix.translate(50,60,70)
v=matrix.rotate_y(25) * matrix.translate(4,5,6)
p=matrix.projection(near=0.1, far=99.0, fov_horiz=85, fov_vert=60)
matrix.extract_fov_w((m*v).I * (m*v*p))
84.99999999999999
[/code]
ok, after playing around in numpy and re-familiarising myself with the maths, it looks like I just need to multiply the inverse of the model-view matrix by the model-view-projection matrix to get the projection matrix:
Here's my Work in progress with almost all shadows and halo issues in the starting area fixed:
https://s3.amazonaws.com/DarkStarSword/3Dfix-Dreamfall+Chapters-WIP-2014-10-30.zip
As usual you can look at my commit history to see what I changed:
https://github.com/DarkStarSword/3d-fixes/commits/master/Dreamfall%20Chapters
The point light sources near Zoe's bed are still not quite right - they look fine up close but are a bit messed up from a distance and look like they are clipping incorrectly. I'm still not sure what's going on with them & will need to experiment a bit. The light under the nearest cliff is also off, and some distant mountains seem to be incorrectly in shadow.
I've added depth adjustment to some UI elements - use the keys on the number row to set the depth. Notably the mouse cursor is not adjusted - I may have just missed it, but it wouldn't surprise me if it's a hardware cursor - other Unity game's I've looked at recently have all had hardware cursors. (Note to self - see if the EnableStereoCursor or StereoPointer profile settings do anything)
To solve the FOV issue I used Helix mod to copy the MVP and the inverse of the MV matrix from the vertex shader into the pixel shader (an alternative approach I found to work was to add extra outputs to the VS - but then I'd need to do the matrix inverse manually):
[code]
// Get MV and MVP matrices from the VS inputs
[VS92E51A17]
// get glstate_matrix_modelview0 from c0-c3 and invert it
UseMatrix = true
GetMatrixFromReg = 0
InverseMatrix = true
// get glstate_matrix_mvp from c4-c7
UseMatrix1 = true
GetMatrixFromReg1 = 4
InverseMatrix1 = false
// Copy MVP and inverted MV matrices into the PS:
[PSFD2FC596]
// c180-c183 has the inverted MV matrix:
UseMatrix = true
MatrixReg = 180
// c190-c193 has the MVP matrix:
UseMatrix1 = true
MatrixReg1 = 190
[/code]
Then did a partial matrix multiplication in the pixel shader to get the FOV:
[code]
// We need to get the horizontal FOV, which is in the projection matrix at
// [0,0] (well, actually it's cot(fov_h / 2), but that's what we need).
//
// Unfortunately none of the Unity shaders use the projection matrix, so we
// need to calculate it. We have the MVP matrix in c190 and the inverted MV
// matrix in c180, which was copied from the VS with Helix mod (refer to the
// DX9Settings.ini).
//
// We can calculate the projection matrix by multiplying the inverted MV matrix
// by the MVP matrix: P = MV.I * MVP
//
// Since we only need one field in the projection matrix, we can avoid doing a
// full matrix multiplication here. We only need to multiply the first row of
// the inverted MV matrix with the first column of the MVP matrix. The matrix
// regs are columns (I think - feels backwards to me), which means we need to
// transpose the first component of each of the MV regs into a temporary
// register to get it's first row:
mov r21.x, c180.x
mov r21.y, c181.x
mov r21.z, c182.x
mov r21.w, c183.x
// Now calculate the dot product of the first row of MV.I with the first column
// of MVP and store the result in r20.x - this is Projection[0,0]:
dp4 r20.x, r21, c190
// We need to divide by the FOV, so invert it:
rcp r20.x, r20.x
// And now we can adjust the shadow:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.w, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mul r31.w, r31.w, r20.x // Adjust by FOV we just calculated
add r0.x, r0.x, -r31.w // Finally adjust the X coord
[/code]
Thanks to Mike and 4everAwake for putting me on the right track to find that fix :)
Rather than using the vPos to calculate the screen position as Mike's Might & Magic fix did, I uncorrected the vertex shader, which seems more accurate as it removed a 1px halo around Zoe (another alternative seems to be to 1/2 correct the texcoord output from the VS, but shadows need more work in that case).
The point light sources near Zoe's bed are still not quite right - they look fine up close but are a bit messed up from a distance and look like they are clipping incorrectly. I'm still not sure what's going on with them & will need to experiment a bit. The light under the nearest cliff is also off, and some distant mountains seem to be incorrectly in shadow.
I've added depth adjustment to some UI elements - use the keys on the number row to set the depth. Notably the mouse cursor is not adjusted - I may have just missed it, but it wouldn't surprise me if it's a hardware cursor - other Unity game's I've looked at recently have all had hardware cursors. (Note to self - see if the EnableStereoCursor or StereoPointer profile settings do anything)
To solve the FOV issue I used Helix mod to copy the MVP and the inverse of the MV matrix from the vertex shader into the pixel shader (an alternative approach I found to work was to add extra outputs to the VS - but then I'd need to do the matrix inverse manually):
// Get MV and MVP matrices from the VS inputs
[VS92E51A17]
// get glstate_matrix_modelview0 from c0-c3 and invert it
UseMatrix = true
GetMatrixFromReg = 0
InverseMatrix = true
// get glstate_matrix_mvp from c4-c7
UseMatrix1 = true
GetMatrixFromReg1 = 4
InverseMatrix1 = false
// Copy MVP and inverted MV matrices into the PS:
[PSFD2FC596]
// c180-c183 has the inverted MV matrix:
UseMatrix = true
MatrixReg = 180
// c190-c193 has the MVP matrix:
UseMatrix1 = true
MatrixReg1 = 190
Then did a partial matrix multiplication in the pixel shader to get the FOV:
// We need to get the horizontal FOV, which is in the projection matrix at
// [0,0] (well, actually it's cot(fov_h / 2), but that's what we need).
//
// Unfortunately none of the Unity shaders use the projection matrix, so we
// need to calculate it. We have the MVP matrix in c190 and the inverted MV
// matrix in c180, which was copied from the VS with Helix mod (refer to the
// DX9Settings.ini).
//
// We can calculate the projection matrix by multiplying the inverted MV matrix
// by the MVP matrix: P = MV.I * MVP
//
// Since we only need one field in the projection matrix, we can avoid doing a
// full matrix multiplication here. We only need to multiply the first row of
// the inverted MV matrix with the first column of the MVP matrix. The matrix
// regs are columns (I think - feels backwards to me), which means we need to
// transpose the first component of each of the MV regs into a temporary
// register to get it's first row:
mov r21.x, c180.x
mov r21.y, c181.x
mov r21.z, c182.x
mov r21.w, c183.x
// Now calculate the dot product of the first row of MV.I with the first column
// of MVP and store the result in r20.x - this is Projection[0,0]:
dp4 r20.x, r21, c190
// We need to divide by the FOV, so invert it:
rcp r20.x, r20.x
// And now we can adjust the shadow:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.w, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mul r31.w, r31.w, r20.x // Adjust by FOV we just calculated
add r0.x, r0.x, -r31.w // Finally adjust the X coord
Thanks to Mike and 4everAwake for putting me on the right track to find that fix :)
Rather than using the vPos to calculate the screen position as Mike's Might & Magic fix did, I uncorrected the vertex shader, which seems more accurate as it removed a 1px halo around Zoe (another alternative seems to be to 1/2 correct the texcoord output from the VS, but shadows need more work in that case).
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
I just gave it a try, and that looks absolutely fantastic. I can see the issues you've mentioned, but it's a huge step up from how it looked to start with. I've progressed to the second scene (in the same area) and I can see a couple other issues, but nothing too major. To get to the next scene, all you need to do is examine a few objects around the bed, then you'll see a light appear in the distance that can be interacted with.
I'm amazed at how good it's looking, I really appreciate all the time you've put into this so far!
I just gave it a try, and that looks absolutely fantastic. I can see the issues you've mentioned, but it's a huge step up from how it looked to start with. I've progressed to the second scene (in the same area) and I can see a couple other issues, but nothing too major. To get to the next scene, all you need to do is examine a few objects around the bed, then you'll see a light appear in the distance that can be interacted with.
I'm amazed at how good it's looking, I really appreciate all the time you've put into this so far!
[quote="DarkStarSword"]Here's my Work in progress with almost all shadows and halo issues in the starting area fixed:
https://s3.amazonaws.com/DarkStarSword/3Dfix-Dreamfall+Chapters-WIP-2014-10-30.zip
As usual you can look at my commit history to see what I changed:
https://github.com/DarkStarSword/3d-fixes/commits/master/Dreamfall%20Chapters
The point light sources near Zoe's bed are still not quite right - they look fine up close but are a bit messed up from a distance and look like they are clipping incorrectly. I'm still not sure what's going on with them & will need to experiment a bit. The light under the nearest cliff is also off, and some distant mountains seem to be incorrectly in shadow.
I've added depth adjustment to some UI elements - use the keys on the number row to set the depth. Notably the mouse cursor is not adjusted - I may have just missed it, but it wouldn't surprise me if it's a hardware cursor - other Unity game's I've looked at recently have all had hardware cursors. (Note to self - see if the EnableStereoCursor or StereoPointer profile settings do anything)
To solve the FOV issue I used Helix mod to copy the MVP and the inverse of the MV matrix from the vertex shader into the pixel shader (an alternative approach I found to work was to add extra outputs to the VS - but then I'd need to do the matrix inverse manually):
[code]
// Get MV and MVP matrices from the VS inputs
[VS92E51A17]
// get glstate_matrix_modelview0 from c0-c3 and invert it
UseMatrix = true
GetMatrixFromReg = 0
InverseMatrix = true
// get glstate_matrix_mvp from c4-c7
UseMatrix1 = true
GetMatrixFromReg1 = 4
InverseMatrix1 = false
// Copy MVP and inverted MV matrices into the PS:
[PSFD2FC596]
// c180-c183 has the inverted MV matrix:
UseMatrix = true
MatrixReg = 180
// c190-c193 has the MVP matrix:
UseMatrix1 = true
MatrixReg1 = 190
[/code]
Then did a partial matrix multiplication in the pixel shader to get the FOV:
[code]
// We need to get the horizontal FOV, which is in the projection matrix at
// [0,0] (well, actually it's cot(fov_h / 2), but that's what we need).
//
// Unfortunately none of the Unity shaders use the projection matrix, so we
// need to calculate it. We have the MVP matrix in c190 and the inverted MV
// matrix in c180, which was copied from the VS with Helix mod (refer to the
// DX9Settings.ini).
//
// We can calculate the projection matrix by multiplying the inverted MV matrix
// by the MVP matrix: P = MV.I * MVP
//
// Since we only need one field in the projection matrix, we can avoid doing a
// full matrix multiplication here. We only need to multiply the first row of
// the inverted MV matrix with the first column of the MVP matrix. The matrix
// regs are columns (I think - feels backwards to me), which means we need to
// transpose the first component of each of the MV regs into a temporary
// register to get it's first row:
mov r21.x, c180.x
mov r21.y, c181.x
mov r21.z, c182.x
mov r21.w, c183.x
// Now calculate the dot product of the first row of MV.I with the first column
// of MVP and store the result in r20.x - this is Projection[0,0]:
dp4 r20.x, r21, c190
// We need to divide by the FOV, so invert it:
rcp r20.x, r20.x
// And now we can adjust the shadow:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.w, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mul r31.w, r31.w, r20.x // Adjust by FOV we just calculated
add r0.x, r0.x, -r31.w // Finally adjust the X coord
[/code]
Thanks to Mike and 4everAwake for putting me on the right track to find that fix :)
Rather than using the vPos to calculate the screen position as Mike's Might & Magic fix did, I uncorrected the vertex shader, which seems more accurate as it removed a 1px halo around Zoe (another alternative seems to be to 1/2 correct the texcoord output from the VS, but shadows need more work in that case).[/quote]
Stellar stuff mate, well done! I tried doing this approach ages ago on GTA4 when I was just learning last year, but I did not know about the "Matrix1" stuff at that time and could not get it working with my own matrix inverse asm code. If you have GTA4, give it a shot :-)
The point light sources near Zoe's bed are still not quite right - they look fine up close but are a bit messed up from a distance and look like they are clipping incorrectly. I'm still not sure what's going on with them & will need to experiment a bit. The light under the nearest cliff is also off, and some distant mountains seem to be incorrectly in shadow.
I've added depth adjustment to some UI elements - use the keys on the number row to set the depth. Notably the mouse cursor is not adjusted - I may have just missed it, but it wouldn't surprise me if it's a hardware cursor - other Unity game's I've looked at recently have all had hardware cursors. (Note to self - see if the EnableStereoCursor or StereoPointer profile settings do anything)
To solve the FOV issue I used Helix mod to copy the MVP and the inverse of the MV matrix from the vertex shader into the pixel shader (an alternative approach I found to work was to add extra outputs to the VS - but then I'd need to do the matrix inverse manually):
// Get MV and MVP matrices from the VS inputs
[VS92E51A17]
// get glstate_matrix_modelview0 from c0-c3 and invert it
UseMatrix = true
GetMatrixFromReg = 0
InverseMatrix = true
// get glstate_matrix_mvp from c4-c7
UseMatrix1 = true
GetMatrixFromReg1 = 4
InverseMatrix1 = false
// Copy MVP and inverted MV matrices into the PS:
[PSFD2FC596]
// c180-c183 has the inverted MV matrix:
UseMatrix = true
MatrixReg = 180
// c190-c193 has the MVP matrix:
UseMatrix1 = true
MatrixReg1 = 190
Then did a partial matrix multiplication in the pixel shader to get the FOV:
// We need to get the horizontal FOV, which is in the projection matrix at
// [0,0] (well, actually it's cot(fov_h / 2), but that's what we need).
//
// Unfortunately none of the Unity shaders use the projection matrix, so we
// need to calculate it. We have the MVP matrix in c190 and the inverted MV
// matrix in c180, which was copied from the VS with Helix mod (refer to the
// DX9Settings.ini).
//
// We can calculate the projection matrix by multiplying the inverted MV matrix
// by the MVP matrix: P = MV.I * MVP
//
// Since we only need one field in the projection matrix, we can avoid doing a
// full matrix multiplication here. We only need to multiply the first row of
// the inverted MV matrix with the first column of the MVP matrix. The matrix
// regs are columns (I think - feels backwards to me), which means we need to
// transpose the first component of each of the MV regs into a temporary
// register to get it's first row:
mov r21.x, c180.x
mov r21.y, c181.x
mov r21.z, c182.x
mov r21.w, c183.x
// Now calculate the dot product of the first row of MV.I with the first column
// of MVP and store the result in r20.x - this is Projection[0,0]:
dp4 r20.x, r21, c190
// We need to divide by the FOV, so invert it:
rcp r20.x, r20.x
// And now we can adjust the shadow:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.w, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mul r31.w, r31.w, r20.x // Adjust by FOV we just calculated
add r0.x, r0.x, -r31.w // Finally adjust the X coord
Thanks to Mike and 4everAwake for putting me on the right track to find that fix :)
Rather than using the vPos to calculate the screen position as Mike's Might & Magic fix did, I uncorrected the vertex shader, which seems more accurate as it removed a 1px halo around Zoe (another alternative seems to be to 1/2 correct the texcoord output from the VS, but shadows need more work in that case).
Stellar stuff mate, well done! I tried doing this approach ages ago on GTA4 when I was just learning last year, but I did not know about the "Matrix1" stuff at that time and could not get it working with my own matrix inverse asm code. If you have GTA4, give it a shot :-)
Rig: Intel i5-3570K @4.5GHz, 16Gb Ram, SSD, 2xGTX 980 SLI, Win10x64, 3D Vision 2 Asus VG278, Optoma HD66, CyberPowerPC-Fangbook (i7-4810MQ, GTX780M), Driving Force Pro Wheel
[quote="chtiblue"]which profile did you use with?[/quote]I'm using the "3D-Hub Player" profile. If you use the DX9Settings.ini from the WIP it should select it automatically so long as the game hasn't been assigned to any other profile.
I'm using the "3D-Hub Player" profile. If you use the DX9Settings.ini from the WIP it should select it automatically so long as the game hasn't been assigned to any other profile.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Amazing work, DarkStarSword! Thank you for the work you put in to figuring this out. Also big thanks to mike_ar69 for the assistance. Can't wait to try out the WIP once I get home from work so I can study it further.
Amazing work, DarkStarSword! Thank you for the work you put in to figuring this out. Also big thanks to mike_ar69 for the assistance. Can't wait to try out the WIP once I get home from work so I can study it further.
I've fixed the remaining lighting issues in the starting area, here's the updated WIP:
https://s3.amazonaws.com/DarkStarSword/3Dfix-Dreamfall+Chapters-WIP-2014-11-03.zip
I've played through a couple of scenes and everything looks fine so far. I'm sure that things will be broken further into the game (surface shaders and any other variants of Unity's Hidden/Internal-PrePassLighting.shader for sure) and I haven't tacked the intro cinematic yet due to the short time broken effects are displayed making hunting difficult. My plan is to write a scipt to identify which dumped shaders correspond with which Unity shaders (and what variant) - I've already done that process manually, but automating it should give me a list of shaders that need to be fixed and a known method to fix them.
Additionally, I've had some success fixing things inside Unity itself - I've been able to get it to compile a modification of Internal-PrePassLighting.shader with the fix pre-applied. If I can match up the compiled variants with the CRC of the originals I should be able to bulk apply the fix to every variant of a given shader - I just need to write some more scripts to make that work.
In terms of what I've fixed since my last WIP, I worked out that the light was clipping incorrectly due to uncorrecting the vertex shader, however simply correcting the uv (texcoord0) output from the vertex shader broke the shadows again. I eventually worked out that I also needed to correct the ray output (texcoord1) from the vertex shader as well, following the exact same maths as I did to fix the pixel shader. This means I'm now applying the same fix in both vertex and pixel shaders - I feel it's probably possible to do this just once since the correction is acting on the same ray - I'll need to experiment further, but for now it works perfectly with the correction in both places.
I'm also getting a much better handle on the maths involved - like, I understand why we need the FOV for this correction, why we need to divide instead of multiply. Essentially, it's because we are adjusting a view-space coordinate and not a projection-space coordinate. For anyone studying this, here's my current understanding:
If the formula for the amount we need to adjust a projection-space X coordinate by is:
[code]separation * (W - convergence)[/code]
Then we need to unproject that to get the adjustment for a view-space coordinate.
I'm still using MV.I * MVP to find P, though naturally if you happen to have P handy you can of course just use it.
We can make some assumptions and simplifications based on a typical projection matrix used in games (some games may do something else, but it's uncommon):
[code]
P = fh 0 0 0 fh = cot(fov_h / 2)
0 fv 0 0 fv = cot(fov_v / 2)
0 0 q 1 q = far / (far - near)
0 0 v 0 v = -q * near
[/code]
Thanks to the gratuitous usage of 0s in the matrix we can calculate a typical unprojection matrix as:
[code]
P.I = 1/fh 0 0 0 1/fh = tan(fov_h / 2)
0 1/fv 0 0 1/fv = tan(fov_v / 2)
0 0 0 x x = too hard, don't care
0 0 x x
[/code]
Since we are only adjusting an X coordinate we only need the first column, and since it has three zeroes we only need the very first field, i.e. P.I[0,0], which is the same as 1 / P[0,0]. This is why we need the horizontal FOV and why we need to divide by P[0,0] instead of multiply.
This boils down to the following formula to correct a view-space coordinate (Note we use Z as the depth here, because Z becomes W after multiplying by the projection matrix. That said, the compiler may decide to re-organize the components, which is why it's W in some of the light pixel shaders, and Z in others - examine how texcoord1 is manipulated to determine which has the depth - I've added comments in my WIP to highlight this):
[code]X = X + separation * (Z - convergence) * P.I[0,0][/code]
Refer to VertexShaders/92E51A17.txt for the working code with a detailed comment. The maths is identical to the corrections in the pixel shaders, but I just understand things clearer now.
Here's a more concise version for copy+paste (of course you need to take care of getting the matrices into the shader with Helix Mod):
[code]
// This is a simplified version of the maths to find P.I[0,0] from MV.I and MVP
// Assumes the game is storing matrix columns in registers (highly likely)
// c180 is the inverse of the MV matrix (From DX9Settings.ini)
// c190 is the MVP matrix (From DX9Settings.ini)
// r0 is the view-space coordinate being corrected
// r0.z holds the depth (check the compiler hasn't re-organised this)
// c220.z has the magic coordinate 0.0625 to load the stereo settings
// s13 has the nVidia stereo texture
// r30 and r31 are temporary registers
// Transpose first component of the MV.I matrix regs into r30:
mov r30.x, c180.x
mov r30.y, c181.x
mov r30.z, c182.x
mov r30.w, c183.x
dp4 r30.x, r30, c190 // Calculate P[0,0]
// If you already have the projection matrix handy, you can skip to here
rcp r30.x, r30.x // Calculate P.I[0,0]
// If you already have the unprojection matrix handy, you can skip to here
// This is a variation of the stereo-correction formula for view-space coords:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.z, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mad r0.x, r31.w, r30.x, r0.x // X += separation * (depth - convergence) * P.I[0,0]
[/code]
I've played through a couple of scenes and everything looks fine so far. I'm sure that things will be broken further into the game (surface shaders and any other variants of Unity's Hidden/Internal-PrePassLighting.shader for sure) and I haven't tacked the intro cinematic yet due to the short time broken effects are displayed making hunting difficult. My plan is to write a scipt to identify which dumped shaders correspond with which Unity shaders (and what variant) - I've already done that process manually, but automating it should give me a list of shaders that need to be fixed and a known method to fix them.
Additionally, I've had some success fixing things inside Unity itself - I've been able to get it to compile a modification of Internal-PrePassLighting.shader with the fix pre-applied. If I can match up the compiled variants with the CRC of the originals I should be able to bulk apply the fix to every variant of a given shader - I just need to write some more scripts to make that work.
In terms of what I've fixed since my last WIP, I worked out that the light was clipping incorrectly due to uncorrecting the vertex shader, however simply correcting the uv (texcoord0) output from the vertex shader broke the shadows again. I eventually worked out that I also needed to correct the ray output (texcoord1) from the vertex shader as well, following the exact same maths as I did to fix the pixel shader. This means I'm now applying the same fix in both vertex and pixel shaders - I feel it's probably possible to do this just once since the correction is acting on the same ray - I'll need to experiment further, but for now it works perfectly with the correction in both places.
I'm also getting a much better handle on the maths involved - like, I understand why we need the FOV for this correction, why we need to divide instead of multiply. Essentially, it's because we are adjusting a view-space coordinate and not a projection-space coordinate. For anyone studying this, here's my current understanding:
If the formula for the amount we need to adjust a projection-space X coordinate by is:
separation * (W - convergence)
Then we need to unproject that to get the adjustment for a view-space coordinate.
I'm still using MV.I * MVP to find P, though naturally if you happen to have P handy you can of course just use it.
We can make some assumptions and simplifications based on a typical projection matrix used in games (some games may do something else, but it's uncommon):
P = fh 0 0 0 fh = cot(fov_h / 2)
0 fv 0 0 fv = cot(fov_v / 2)
0 0 q 1 q = far / (far - near)
0 0 v 0 v = -q * near
Thanks to the gratuitous usage of 0s in the matrix we can calculate a typical unprojection matrix as:
P.I = 1/fh 0 0 0 1/fh = tan(fov_h / 2)
0 1/fv 0 0 1/fv = tan(fov_v / 2)
0 0 0 x x = too hard, don't care
0 0 x x
Since we are only adjusting an X coordinate we only need the first column, and since it has three zeroes we only need the very first field, i.e. P.I[0,0], which is the same as 1 / P[0,0]. This is why we need the horizontal FOV and why we need to divide by P[0,0] instead of multiply.
This boils down to the following formula to correct a view-space coordinate (Note we use Z as the depth here, because Z becomes W after multiplying by the projection matrix. That said, the compiler may decide to re-organize the components, which is why it's W in some of the light pixel shaders, and Z in others - examine how texcoord1 is manipulated to determine which has the depth - I've added comments in my WIP to highlight this):
X = X + separation * (Z - convergence) * P.I[0,0]
Refer to VertexShaders/92E51A17.txt for the working code with a detailed comment. The maths is identical to the corrections in the pixel shaders, but I just understand things clearer now.
Here's a more concise version for copy+paste (of course you need to take care of getting the matrices into the shader with Helix Mod):
// This is a simplified version of the maths to find P.I[0,0] from MV.I and MVP
// Assumes the game is storing matrix columns in registers (highly likely)
// c180 is the inverse of the MV matrix (From DX9Settings.ini)
// c190 is the MVP matrix (From DX9Settings.ini)
// r0 is the view-space coordinate being corrected
// r0.z holds the depth (check the compiler hasn't re-organised this)
// c220.z has the magic coordinate 0.0625 to load the stereo settings
// s13 has the nVidia stereo texture
// r30 and r31 are temporary registers
// Transpose first component of the MV.I matrix regs into r30:
mov r30.x, c180.x
mov r30.y, c181.x
mov r30.z, c182.x
mov r30.w, c183.x
dp4 r30.x, r30, c190 // Calculate P[0,0]
// If you already have the projection matrix handy, you can skip to here
rcp r30.x, r30.x // Calculate P.I[0,0]
// If you already have the unprojection matrix handy, you can skip to here
// This is a variation of the stereo-correction formula for view-space coords:
texldl r31, c220.z, s13 // r31.x = separation, r31.y = convergence
add r31.w, r0.z, -r31.y // r31.w = depth - convergence
mul r31.w, r31.w, r31.x // r31.w = separation * (depth - convergence)
mad r0.x, r31.w, r30.x, r0.x // X += separation * (depth - convergence) * P.I[0,0]
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
The WIP looks awesome so far, DarkStarSword. It seems to be nearly ready. Just a few more halo issues, bloom, and some uneven HUD / UI elements remain. Not sure how far you've gotten, but I can lend a hand if you need.
Also thank you for the superb notes. I was doing some testing and I was able to achieve mostly positive results by duplicate your fix (correcting the ray output) in other games. In a couple of games, I'm getting intermittent "snapping" issues (w ≠ 1) only when I apply the FOV fix. I have to test these games further to see if I can somehow isolate the problem.
The WIP looks awesome so far, DarkStarSword. It seems to be nearly ready. Just a few more halo issues, bloom, and some uneven HUD / UI elements remain. Not sure how far you've gotten, but I can lend a hand if you need.
Also thank you for the superb notes. I was doing some testing and I was able to achieve mostly positive results by duplicate your fix (correcting the ray output) in other games. In a couple of games, I'm getting intermittent "snapping" issues (w ≠ 1) only when I apply the FOV fix. I have to test these games further to see if I can somehow isolate the problem.
Hmmm, I'd like to take a look at that snapping - is there any particular game where you can reliably reproduce it?
I'm wondering if it is related to this comment in the source code of the light vertex shader:
[code]// v.normal contains a ray pointing from the camera to one of near plane'
// corners in camera space when we are drawing a full screen quad.
// Otherwise, when rendering 3D shapes, use the ray calculated here.
[/code]
(v.normal is v1 in 92E51A17).
The Unity documentation has this to say on the subject:
[quote]Point and spot lights that do not cross the camera’s near plane are rendered as 3D shapes, with the Z buffer’s test against the scene enabled. This makes partially or fully occluded point and spot lights very cheap to render. Directional lights and point/spot lights that cross the near plane are rendered as fullscreen quads.[/quote]From http://docs.unity3d.com/Manual/RenderTech-DeferredLighting.html (worth reading the rest of this page as well).
That would suggest that spot & point lights can change form when they are close to the camera - does that by any chance correspond to the snapping issues you see?
Hmmm, I'd like to take a look at that snapping - is there any particular game where you can reliably reproduce it?
I'm wondering if it is related to this comment in the source code of the light vertex shader:
// v.normal contains a ray pointing from the camera to one of near plane'
// corners in camera space when we are drawing a full screen quad.
// Otherwise, when rendering 3D shapes, use the ray calculated here.
(v.normal is v1 in 92E51A17).
The Unity documentation has this to say on the subject:
Point and spot lights that do not cross the camera’s near plane are rendered as 3D shapes, with the Z buffer’s test against the scene enabled. This makes partially or fully occluded point and spot lights very cheap to render. Directional lights and point/spot lights that cross the near plane are rendered as fullscreen quads.
That would suggest that spot & point lights can change form when they are close to the camera - does that by any chance correspond to the snapping issues you see?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
[quote="DarkStarSword"]Hmmm, I'd like to take a look at that snapping - is there any particular game where you can reliably reproduce it?
I'm wondering if it is related to this comment in the source code of the light vertex shader:
[code]// v.normal contains a ray pointing from the camera to one of near plane'
// corners in camera space when we are drawing a full screen quad.
// Otherwise, when rendering 3D shapes, use the ray calculated here.
[/code]
(v.normal is v1 in 92E51A17).
The Unity documentation has this to say on the subject:
[quote]Point and spot lights that do not cross the camera’s near plane are rendered as 3D shapes, with the Z buffer’s test against the scene enabled. This makes partially or fully occluded point and spot lights very cheap to render. Directional lights and point/spot lights that cross the near plane are rendered as fullscreen quads.[/quote]From http://docs.unity3d.com/Manual/RenderTech-DeferredLighting.html (worth reading the rest of this page as well).
That would suggest that spot & point lights can change form when they are close to the camera - does that by any chance correspond to the snapping issues you see?[/quote]
This could very well be it. The snapping issues start to occur when you're close to it. But it also happens if you rotate the camera a certain way. Not sure if this is related, but in the light vertex shader, point & spot lights comprise of 10 or more textures.. all (seemingly) separated by angle. Directional lights only have one texture.
The games I'm seeing the issues in are Among The Sleep and Wasteland 2. In Among The Sleep the point light snapping issues were there by default (without a fix). I corrected the ray output in the light VS and the snapping went away. When I applied the FOV fix, the snapping issues reappeared. (You won't see this issue in the demo as the issue starts in Chapter 2).
In Wasteland 2, without a fix, directional light shadows show no snapping problems and is still ok when I correct the ray output. But when I apply the FOV fix, it starts to exhibit issues. This occurs at the very beginning of the game.
edit: I also tried using matrices from another shader, but I'm getting the same problem. Let me know if you want to check out the WIPs for these.
DarkStarSword said:Hmmm, I'd like to take a look at that snapping - is there any particular game where you can reliably reproduce it?
I'm wondering if it is related to this comment in the source code of the light vertex shader:
// v.normal contains a ray pointing from the camera to one of near plane'
// corners in camera space when we are drawing a full screen quad.
// Otherwise, when rendering 3D shapes, use the ray calculated here.
(v.normal is v1 in 92E51A17).
The Unity documentation has this to say on the subject:
Point and spot lights that do not cross the camera’s near plane are rendered as 3D shapes, with the Z buffer’s test against the scene enabled. This makes partially or fully occluded point and spot lights very cheap to render. Directional lights and point/spot lights that cross the near plane are rendered as fullscreen quads.
That would suggest that spot & point lights can change form when they are close to the camera - does that by any chance correspond to the snapping issues you see?
This could very well be it. The snapping issues start to occur when you're close to it. But it also happens if you rotate the camera a certain way. Not sure if this is related, but in the light vertex shader, point & spot lights comprise of 10 or more textures.. all (seemingly) separated by angle. Directional lights only have one texture.
The games I'm seeing the issues in are Among The Sleep and Wasteland 2. In Among The Sleep the point light snapping issues were there by default (without a fix). I corrected the ray output in the light VS and the snapping went away. When I applied the FOV fix, the snapping issues reappeared. (You won't see this issue in the demo as the issue starts in Chapter 2).
In Wasteland 2, without a fix, directional light shadows show no snapping problems and is still ok when I correct the ray output. But when I apply the FOV fix, it starts to exhibit issues. This occurs at the very beginning of the game.
edit: I also tried using matrices from another shader, but I'm getting the same problem. Let me know if you want to check out the WIPs for these.
Yes you can do this - I used this exact approach in Elder Scrolls Online - have a look at that fix to see what I did. There is no need to work out all the tan/atan stuff, the P_00 element already *is* the tan(FOV/2) value that you need to multiply the view space correction by. (I've often referred to it as P11 in the past because I think in terms of 1-4, but P00 is the nomenclature used in HLSL, basically its the first element). If you had the FOV you could work out that value, but if you have the Proj matrix you don't need to. Note that sometimes you need to divide by P_00 instead of multiply, but I can't remember why (I think this was the case in TESO).
Rig: Intel i5-3570K @4.5GHz, 16Gb Ram, SSD, 2xGTX 980 SLI, Win10x64, 3D Vision 2 Asus VG278, Optoma HD66, CyberPowerPC-Fangbook (i7-4810MQ, GTX780M), Driving Force Pro Wheel
3DSolutionGaming: http://www.3dSolutionGaming.com
Helixmod: http://helixmod.blogspot.com
3DMigoto: https://github.com/bo3b/3Dmigoto/wiki
Here are all the matrices used by shaders in Dreamfall Chapters - we have MV and MVP, is there some matrix operation I can do to extract the P from them? (I wish that NSW had included Matrix maths in the high school curriculum - I picked up a bit at university, but not as much as would be useful right now)
http://docs.unity3d.com/Manual/SL-BuiltinValues.html has definitions of some of these, though the pre-processor has renamed a lot of them.
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
I'll clean up what I've got and post the WIP :)
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
https://s3.amazonaws.com/DarkStarSword/3Dfix-Dreamfall+Chapters-WIP-2014-10-30.zip
As usual you can look at my commit history to see what I changed:
https://github.com/DarkStarSword/3d-fixes/commits/master/Dreamfall%20Chapters
The point light sources near Zoe's bed are still not quite right - they look fine up close but are a bit messed up from a distance and look like they are clipping incorrectly. I'm still not sure what's going on with them & will need to experiment a bit. The light under the nearest cliff is also off, and some distant mountains seem to be incorrectly in shadow.
I've added depth adjustment to some UI elements - use the keys on the number row to set the depth. Notably the mouse cursor is not adjusted - I may have just missed it, but it wouldn't surprise me if it's a hardware cursor - other Unity game's I've looked at recently have all had hardware cursors. (Note to self - see if the EnableStereoCursor or StereoPointer profile settings do anything)
To solve the FOV issue I used Helix mod to copy the MVP and the inverse of the MV matrix from the vertex shader into the pixel shader (an alternative approach I found to work was to add extra outputs to the VS - but then I'd need to do the matrix inverse manually):
Then did a partial matrix multiplication in the pixel shader to get the FOV:
Thanks to Mike and 4everAwake for putting me on the right track to find that fix :)
Rather than using the vPos to calculate the screen position as Mike's Might & Magic fix did, I uncorrected the vertex shader, which seems more accurate as it removed a 1px halo around Zoe (another alternative seems to be to 1/2 correct the texcoord output from the VS, but shadows need more work in that case).
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
I'm amazed at how good it's looking, I really appreciate all the time you've put into this so far!
http://photos.3dvisionlive.com/chtiblue/album/530b52d4cb85770d6e000049/3Dvision with 49" Philips 49PUS7100 interlieved 3D (3840x2160) overide mode, GTX 1080 GFA2 EXOC, core i5 @4.3GHz, 16Gb@2130, windows 7&10 64bit, Dolby Atmos 5.1.4 Marantz 6010 AVR
Stellar stuff mate, well done! I tried doing this approach ages ago on GTA4 when I was just learning last year, but I did not know about the "Matrix1" stuff at that time and could not get it working with my own matrix inverse asm code. If you have GTA4, give it a shot :-)
Rig: Intel i5-3570K @4.5GHz, 16Gb Ram, SSD, 2xGTX 980 SLI, Win10x64, 3D Vision 2 Asus VG278, Optoma HD66, CyberPowerPC-Fangbook (i7-4810MQ, GTX780M), Driving Force Pro Wheel
3DSolutionGaming: http://www.3dSolutionGaming.com
Helixmod: http://helixmod.blogspot.com
3DMigoto: https://github.com/bo3b/3Dmigoto/wiki
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
https://s3.amazonaws.com/DarkStarSword/3Dfix-Dreamfall+Chapters-WIP-2014-11-03.zip
I've played through a couple of scenes and everything looks fine so far. I'm sure that things will be broken further into the game (surface shaders and any other variants of Unity's Hidden/Internal-PrePassLighting.shader for sure) and I haven't tacked the intro cinematic yet due to the short time broken effects are displayed making hunting difficult. My plan is to write a scipt to identify which dumped shaders correspond with which Unity shaders (and what variant) - I've already done that process manually, but automating it should give me a list of shaders that need to be fixed and a known method to fix them.
Additionally, I've had some success fixing things inside Unity itself - I've been able to get it to compile a modification of Internal-PrePassLighting.shader with the fix pre-applied. If I can match up the compiled variants with the CRC of the originals I should be able to bulk apply the fix to every variant of a given shader - I just need to write some more scripts to make that work.
In terms of what I've fixed since my last WIP, I worked out that the light was clipping incorrectly due to uncorrecting the vertex shader, however simply correcting the uv (texcoord0) output from the vertex shader broke the shadows again. I eventually worked out that I also needed to correct the ray output (texcoord1) from the vertex shader as well, following the exact same maths as I did to fix the pixel shader. This means I'm now applying the same fix in both vertex and pixel shaders - I feel it's probably possible to do this just once since the correction is acting on the same ray - I'll need to experiment further, but for now it works perfectly with the correction in both places.
I'm also getting a much better handle on the maths involved - like, I understand why we need the FOV for this correction, why we need to divide instead of multiply. Essentially, it's because we are adjusting a view-space coordinate and not a projection-space coordinate. For anyone studying this, here's my current understanding:
If the formula for the amount we need to adjust a projection-space X coordinate by is:
Then we need to unproject that to get the adjustment for a view-space coordinate.
I'm still using MV.I * MVP to find P, though naturally if you happen to have P handy you can of course just use it.
We can make some assumptions and simplifications based on a typical projection matrix used in games (some games may do something else, but it's uncommon):
Thanks to the gratuitous usage of 0s in the matrix we can calculate a typical unprojection matrix as:
Since we are only adjusting an X coordinate we only need the first column, and since it has three zeroes we only need the very first field, i.e. P.I[0,0], which is the same as 1 / P[0,0]. This is why we need the horizontal FOV and why we need to divide by P[0,0] instead of multiply.
This boils down to the following formula to correct a view-space coordinate (Note we use Z as the depth here, because Z becomes W after multiplying by the projection matrix. That said, the compiler may decide to re-organize the components, which is why it's W in some of the light pixel shaders, and Z in others - examine how texcoord1 is manipulated to determine which has the depth - I've added comments in my WIP to highlight this):
Refer to VertexShaders/92E51A17.txt for the working code with a detailed comment. The maths is identical to the corrections in the pixel shaders, but I just understand things clearer now.
Here's a more concise version for copy+paste (of course you need to take care of getting the matrices into the shader with Helix Mod):
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
Also thank you for the superb notes. I was doing some testing and I was able to achieve mostly positive results by duplicate your fix (correcting the ray output) in other games. In a couple of games, I'm getting intermittent "snapping" issues (w ≠ 1) only when I apply the FOV fix. I have to test these games further to see if I can somehow isolate the problem.
I'm wondering if it is related to this comment in the source code of the light vertex shader:
(v.normal is v1 in 92E51A17).
The Unity documentation has this to say on the subject:
From http://docs.unity3d.com/Manual/RenderTech-DeferredLighting.html (worth reading the rest of this page as well).
That would suggest that spot & point lights can change form when they are close to the camera - does that by any chance correspond to the snapping issues you see?
2x Geforce GTX 980 in SLI provided by NVIDIA, i7 6700K 4GHz CPU, Asus 27" VG278HE 144Hz 3D Monitor, BenQ W1070 3D Projector, 120" Elite Screens YardMaster 2, 32GB Corsair DDR4 3200MHz RAM, Samsung 850 EVO 500G SSD, 4x750GB HDD in RAID5, Gigabyte Z170X-Gaming 7 Motherboard, Corsair Obsidian 750D Airflow Edition Case, Corsair RM850i PSU, HTC Vive, Win 10 64bit
Alienware M17x R4 w/ built in 3D, Intel i7 3740QM, GTX 680m 2GB, 16GB DDR3 1600MHz RAM, Win7 64bit, 1TB SSD, 1TB HDD, 750GB HDD
Pre-release 3D fixes, shadertool.py and other goodies: http://github.com/DarkStarSword/3d-fixes
Support me on Patreon: https://www.patreon.com/DarkStarSword or PayPal: https://www.paypal.me/DarkStarSword
This could very well be it. The snapping issues start to occur when you're close to it. But it also happens if you rotate the camera a certain way. Not sure if this is related, but in the light vertex shader, point & spot lights comprise of 10 or more textures.. all (seemingly) separated by angle. Directional lights only have one texture.
The games I'm seeing the issues in are Among The Sleep and Wasteland 2. In Among The Sleep the point light snapping issues were there by default (without a fix). I corrected the ray output in the light VS and the snapping went away. When I applied the FOV fix, the snapping issues reappeared. (You won't see this issue in the demo as the issue starts in Chapter 2).
In Wasteland 2, without a fix, directional light shadows show no snapping problems and is still ok when I correct the ray output. But when I apply the FOV fix, it starts to exhibit issues. This occurs at the very beginning of the game.
edit: I also tried using matrices from another shader, but I'm getting the same problem. Let me know if you want to check out the WIPs for these.