Parralax mapping does not work correctly using OpenGL and GLSL
I am trying to implement parralax mapping in my 3D engine using OpenGL and GLSL API, but the mapping is not correct. To learn and apply the complexity of such a method, I was inspired by the following PDF tutorial (pages 16, 17 and 18):
https://www.opengl.org/sdk/docs/tutorials/TyphoonLabs/Chapter_4.pdf
To create a very simple paraclax effect (without any lighting effect) I need to use 2 textures:
- 1 diffuse (color) texture (BPP: 24 -> RGB - format: JPEG)
- 1 displacement (height/grayscale) texture (BPP: 24 -> RGB - format: JPEG)
I used the famous and very useful CrazyBump software to create my displacement map. Also, this software can display a 3D view of what the paraclax mapping would look like in an external 3D application like mine.
First time, here's the display from "CrazyBump" (CrazyBump uses a lighting effect, but it doesn't matter here):
As you can see, the parralax effect is displayed correctly.
And now here's a render in my scene (using the same displacement structure created by CrazyBump and no luma. All I want to see is a fake surface deformation as above).
As you can see, the display does not match and is certainly not correct.
To try the same effect, I am applying the course in the PDF file that I talked about at the beginning of my post.
For information, I have already applied the "normal matching" method for my engine (so correct tangent and bitagent vectors!).
To execute my shader program, I need the position of the camera in world space and matrices (ModelViewProj, ModelMatrix and NormalMatrix).
Here's the C ++ client code used:
glm::mat4 modelViewMatrix = pRenderBatch->GetModelViewMatrix();
glm::mat3 normalMatrix = glm::mat3(glm::vec3(modelViewMatrix[0]),
glm::vec3(modelViewMatrix[1]), glm::vec3(modelViewMatrix[2]));
this->SetUniform("ModelViewProjMatrix", pRenderBatch->GetModelViewProjMatrix());
this->SetUniform("ModelViewMatrix", modelViewMatrix);
this->SetUniform("NormalMatrix", normalMatrix);
//Bound on channel 0
glActiveTexture(GL_TEXTURE0);
this->m_pTextureManager.PushAndBindTexture(
pMaterial->GetDiffuseTexture());
{
this->SetUniform("DiffuseSampler", 0);
}
//Bound on channel 1
glActiveTexture(GL_TEXTURE1);
this->m_pTextureManager.PushAndBindTexture(
pMaterial->GetDisplacementTexture());
{
this->SetUniform("HeightSampler", 1);
}
Vertex Shader:
#version 440
/*
** Vertex attributes.
*/
layout (location = 0) in vec4 VertexPosition;
layout (location = 1) in vec2 VertexTexture;
layout (location = 2) in vec3 VertexNormal;
layout (location = 3) in vec3 VertexTangent;
layout (location = 4) in vec3 VertexBitangent;
/*
** Uniform matrices.
*/
uniform mat4 ModelViewProjMatrix;
uniform mat4 ModelViewMatrix;
uniform mat3 NormalMatrix;
//Outputs
out vec2 TexCoords;
out vec3 viewDir_TS;
/*
** Vertex shader entry point.
*/
void main(void)
{
//Texture coordinates
TexCoords = VertexTexture;
//Vertex position in world space
vec3 Position_CS = vec3(ModelViewMatrix * VertexPosition);
//Vertex normal in world space
vec3 Normal_CS = NormalMatrix * VertexNormal;
//Vertex tangent in world space
vec3 Tangent_CS = NormalMatrix * VertexTangent;
//Vertex bitangent in world space
vec3 Bitangent_CS = NormalMatrix * VertexBitangent;
//View vector in world space
vec3 viewDir_CS = -Position_CS;
//TBN matrix
mat3 TBN = mat3(
Tangent_CS.x, Bitangent_CS.x, Normal_CS.x,
Tangent_CS.y, Bitangent_CS.y, Normal_CS.y,
Tangent_CS.z, Bitangent_CS.z, Normal_CS.z);
//2 others ways to compute view vector in tangent space
//mat3 TBN = transpose(mat3(Tangent_CS, Bitangent_CS, Normal_CS));
/*viewDir_TS = vec3(
dot(viewDir_CS, Tangent_CS),
dot(viewDir_CS, Bitangent_CS),
dot(viewDir_CS, Normal_CS)
);*/
//View vector converted in tangent space (not normalized)
viewDir_TS = TBN * viewDir_CS;
gl_Position = ModelViewProjMatrix * VertexPosition;
}
And finally, the Fragment Shader:
#version 440
layout (location = 0) out vec4 FragColor;
//Texture coordinates
in vec2 TexCoords;
//View (camera) vector in tangent space
in vec3 viewDir_TS;
//Diffuse texture sampler
uniform sampler2D DiffuseSampler;
//Displacement texture sampler
//(height map/grayscale map)
uniform sampler2D HeightSampler;
/*
** Fragment shader entry point
*/
void main(void)
{
//Parralax intensity {scale(s), bias(b)}
vec2 ScaleBias = vec2(0.04f, 0.02f);
//Height(h) range [0;1] (float) recovered from height map (HeightSampler)
float Height = texture2D(HeightSampler, TexCoords.st).r;
//Height scaled and biased according to the formula: hsb = h · s + b
float HSB = Height * ScaleBias.x + ScaleBias.y;
//View vector in tangent space normalized
vec3 viewDirNorm_TS = normalize(viewDir_TS);
//Computes texture offset according to the formula: Tn = To + (hsb · V{x, y})
vec2 textOffset = TexCoords + (viewDirNorm_TS.xy * HSB);
//Computes final diffuse texture color using parralax offset
FragColor = texture2D(DiffuseSampler, textOffset);
}
I have tried changing the scale and offset values without any success: the impressions are not correct.
I thought my offset structure was not loaded correctly, but it is not (I'm using the NVIDIA NSight dehydrator for information).
If I load the displacement map like this (GL_LUMINANCE):
glTexImage2D(this->m_Target, 0, GL_LUMINANCE,
this->m_PixelData.GetWidth(), this->m_PixelData.GetHeight(),
0, GL_BGR, GL_UNSIGNED_BYTE, OFFSET_BUFFER(0));
The pixel buffer starts with:
And if I load the displacement map like this (GL_RGB):
glTexImage2D (this-> m_Target, 0, GL_RGB, this-> m_PixelData.GetWidth (), this-> m_PixelData.GetHeight (), 0, GL_BGR, GL_UNSIGNED_BYTE, OFFSET_BUFFER (0));
The pixel buffer starts with:
In these two cases, we have shades of gray.
So my problem doesn't seem to be related to the texture loaded into memory. Maybe a matrix issue or a space issue. I am truly lost.
Can anyone help me please?
Thanks a lot for adavance for your help!
source to share
The only method textOffset
is the early parallax method, which makes a very large approximation: the offset depth will be the same. This is why the effect might look a little strange. I think in your case one of the tangent vectors collides with the wrong way. Try to deny textOffset.x
or as a test textOffset.y
. I'm more used to seeing shaders use eye space before jumping into tangent space, but can't see any issues with your code.
For best effect, trace (or "step") through the heightmap until you find an intersection. Use binary or secant search to improve it. Then use the coordinate of your intersection for the color. This is sometimes referred to as bump mapping, steep parallax mapping and parallax occlusion mapping.
This is untested, but hopefully gives an idea:
const int steps = 20;
const float scale = 0.1;
vec3 pos = vec3(TexCoords, 1.0); //1.0 as ray starts at surface/max height
vec3 dir = -viewDir_TS / viewDir_TS.z; //I assume viewDir_TS.z is negative
dir.xy *= scale;
dir /= steps;
//linear steps
float height;
for (int i = 0; i < steps; ++i)
{
pos += dir;
height = texture2D(HeightSampler, pos.xy).r
if (pos.z < height)
break;
}
//binary search
for (int i = 0; i < 4; ++i)
{
dir *= 0.5;
if (pos.z < height)
pos -= dir;
else
pos += dir;
height = texture2D(HeightSampler, pos.xy).r
}
//write output
FragColor = texture2D(DiffuseSampler, pos.xy);
A few smaller things: I see normal and tangent vectors multiplied by the normal matrix all the time. The tangents should indeed be multiplied by the model matrix, although it doesn't matter if the transformation is orthonormal, which is almost always the case. The computation and storage of binagent data is not required as it can be computed dynamically with a cross product while avoiding bit bandwidth.
source to share
The problem just came from the line:
float HSB = Height * ScaleBias.x + ScaleBias.y;
This is not an addition, but an expression:
float HSB = Height * ScaleBias.x - ScaleBias.y;
Screenshot 1:
Screenshot 2:
Of course, I added a normal display for the luminosity.
I hope this post is helpful!
source to share