Game Development Stack Exchange is a question and answer site for professional and independent game developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I can't seem to figure out my problem with stencil buffer in object outlining algo.

function init() {
   ...
   gl.enable(gl.STENCIL_TEST);
   gl.stencilOp(gl.KEEP, gl.KEEP, gl.REPLACE);
   ...
}

var mscaled = [];     // scaled model matrix of a cube  
function render() {
   gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT | gl.STENCIL_BUFFER_BIT);

    gl.stencilFunc(gl.ALWAYS, 1, 0xFF);
    gl.stencilMask(0xFF);
     // use program1
     // set attrib pointers 
     // bind cube's ebo buffer
     gl.drawElements(gl.TRIANGLES, 36, gl.UNSIGNED_SHORT, 0);

     gl.stencilFunc(gl.NOTEQUAL, 1, 0xFF);
     gl.stencilMask(0x00);

      // set scaled model matrix
      mat4.fromRotationTranslationScale(mscaled, cube.rotation, cube.translation, vec3.fromValues(1.1, 1.1, 1.1));

      // use program2
      // set attrib pointers 
      // bind cube's ebo buffer
      gl.drawElements(gl.TRIANGLES, 36, gl.UNSIGNED_SHORT, 0); 
}

And it just draws the whole second cube on top of the first, like there's no stencil test. I've tested with up-scale and down-scale.

the red cube should be the outliner. In the first image it is scaled up - but WebGL draws it just on top of the first one, in the second image it is scaled down - ideally it shouldn't been even drawn due to stencil test (of course, depth test is disabled in the case of a second image)

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.