Alpha mask of overlapping objects in OpenGL
What's the best way to get an alpha mask of overlapping objects in OpenGL? In the first image below, I have three meshes overlapping with an ellipsoid, depth checking is on. My goal is to get a result similar to the second image where white represents the alpha. Below are the depth check flags I am using.
glEnable(GL_DEPTH_TEST); glDepthMask(GL_TRUE); glDepthFunc(GL_LEQUAL);
source to share
Before drawing red, yellow and blue meshes, you have to enable stencil test and set glStencilOp stencil operations how to do it
glEnable( GL_STENCIL_TEST );
glStencilOp( GL_KEEP, GL_KEEP, GL_INCR );
glStencilFunc( GL_ALWAYS, 0, 1 ); // these are also the default parameters
This means that the stencil buffer is kept as it is if the depth check fails and is incremented if the depth test passes. After drawing, the stencil buffer contains a mask, where 0 means "black" and "0" means "white" in your case. Make sure your stencil buffer is cleared before drawing ( glClear( GL_STENCIL_BUFFER_BIT )
).
If you need to get this mask for a texture, you need to bind the texture to the stencil buffer:
GLint with = ...;
GLint height = ...;
GLuint depthAndStencilMap;
glGenTextures( 1, &depthAndStencilMap );
glBindTexture( GL_TEXTURE_2D, depthAndStencilMap );
glTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_STENCIL, with, height, 0, GL_DEPTH_STENCIL, GL_FLOAT, NULL );
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT );
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT );
GLuint frameBuffer;
glGenFramebuffers( 1, &frameBuffer );
glBindFramebuffer( GL_FRAMEBUFFER, frameBuffer );
glFramebufferTexture2D( GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_TEXTURE_2D, depthAndStencilMap, 0 );
glBindFramebuffer( GL_FRAMEBUFFER, 0);
Before drawing, you need to bind and clear the frame buffer:
glBindFramebuffer( GL_FRAMEBUFFER, frameBuffer );
glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT )
After painting, the texture depthAndStencilMap
contains a depth buffer in the red channel and a stencil buffer, which is your mask, in the Blue channel. Note that the scene is referring to the framebuffer, not the viewport.
source to share
"best" is a four-digit word;)
You can either
- use a stencil buffer
- clear it to 0 / false for "not your grids"
- stencil to 1 / true for "is your mesh"
- carry out the second pass
- include only the objects you want in the mask
- do not clear the depth buffer
- use depth == test
- use a different fragment shader for "these are your meshes"
source to share