FrameBuffer Objects for Dynamic Cubemaps in GLES 2.0 on Android

Update: Turns out this is a driver issue with the powerVR SGX in my Nexus S. Code works fluidly on all other devices I've tested it on.

I'll be making a smaller test case and submitting a bug report... to someone. Don't know who.

---

Hey guys,

First off, I'm working on a port of开发者_StackOverflow中文版 AndAR (ARToolkit for Android) to GLES 2.0 using the Java GLES2.0 bindings. The entirety of my code can be found here if you're curious, but I'll try to sum up the problem in this question. AndARShaders

I'm attempting to implement this paper to generate AR renderings which reflect and refract believably: Virtual Reflections in Augmented Reality Environments. In order to do this, the screen space bounding box of the object to be rendered is determined, then used to generate texture coordinates for planes representing each face of the cubemap. This means rendering a cubemap for each frame for each model. I'm only rendering one model at a time right now. I'm trying to use framebuffer objects to render the cubemap based on the method described in the paper.

Anyway, to the problem.

I have it mostly implemented. As far as I can tell, at least the front face of the cubemap has vertices and UV coordinates generated correctly. I can render my front face to the system provided frame buffer for the screen and it renders just fine without problems for as long as I'd like. The problem comes in with rendering it to a framebuffer object.

When I render my cubemap faces to a framebuffer object linked to a cubemap texture, GL eats ALL of my memory within a few seconds and crashes with a GLError 1285 (OUT OF MEMORY). If I don't bind the FBO, I can render the cubemap faces to the screen without any memory issues. Cube map texture size is 128 PX square, which should be reasonable for a mobile device. Somehow, GL is leaking memory

Here's the rough order I'm doing things. This is the entry to the render for this frame. (src/edu/dhbw/andar/ARGLES20Renderer.java ~Line 179)

// BEGIN TO DRAW FRAME.  DRAW BACKGROUND CAMERA IMAGE TO QUAD
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glUseProgram(mProgram);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0); // Camera image is stored in Texture0
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureName);

//load new preview frame as a texture, if needed
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, previewFrameWidth, previewFrameHeight, mode, GLES20.GL_UNSIGNED_BYTE, frameData);

//draw camera preview frame:
squareBuffer.position(0);
GLES20.glVertexAttribPointer(maPositionHandle, 3, GLES20.GL_FLOAT, false, GraphicsUtil.TRIANGLE_VERTICES_DATA_STRIDE_BYTES, squareBuffer);
GLES20.glEnableVertexAttribArray(maPositionHandle);
textureBuffer.position(0);
GLES20.glVertexAttribPointer(maTextureHandle, 2, GLES20.GL_FLOAT, false, GraphicsUtil.TRIANGLE_VERTICES_UV_STRIDE_BYTES, textureBuffer);
GLES20.glEnableVertexAttribArray(maTextureHandle);

Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mVMatrix, 0);
GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0);
GLES20.glUniform1i(mSamplerLoc, 0);

//draw camera square
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);

GLES20.glDisableVertexAttribArray(maPositionHandle);
GLES20.glDisableVertexAttribArray(maTextureHandle);

DRAW_OBJECTS();

And the code to draw the objects goes somewhat like this:

( src/edu/dhbw/andar/ARGLES20Object.java ~ Line 36 )

( src/edu/dhbw/andar/pub/CustomGLES20Object.java ~Line 55 )

// Use the new program for the object (Refract/reflect shader)
GLES20.glUseProgram( mProgram );

// Transform to where the marker is
Matrix.multiplyMM(mMVPMatrix, 0, glCameraMatrix, 0, glMatrix, 0);
GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0);

// Create a cubemap for this object from vertices
GENERATE_CUBEMAP( box.vertArray() );

// Feed in Verts
box.verts().position(0);
box.normals().position(0);
GLES20.glVertexAttribPointer(maPositionHandle, 3, GLES20.GL_FLOAT, false, VERTEX_NORMAL_DATA_STRIDE, box.verts());
GLES20.glEnableVertexAttribArray(maPositionHandle);

GLES20.glVertexAttribPointer(maNormalHandle, 3, GLES20.GL_FLOAT, false, VERTEX_NORMAL_DATA_STRIDE, box.normals());
GLES20.glEnableVertexAttribArray(maNormalHandle);

// Set Uniforms...
GLES20.glUniform4f(muColor, 0.0f, 1.0f, 0.0f, 1.0f);
...

// Draw the cube faces
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
...

GLES20.glDisableVertexAttribArray(maPositionHandle);
GLES20.glDisableVertexAttribArray(maNormalHandle);

Note the GENERATE_CUBEMAP( Vertices ) Toward the beginning of rendering the object. Here's what that does. Screen space bounding box [ssbb] has been calculated from vertices. (src/edu/dhbw/andar/ARGLES20Renderer.java ~Line 280)

// Grab the current viewport and program for restoration later
int[] OldViewport = new int[4], OldProgram = new int[1];
GLES20.glGetIntegerv(GLES20.GL_VIEWPORT, OldViewport, 0);
GLES20.glGetIntegerv(GLES20.GL_CURRENT_PROGRAM, OldProgram, 0);

// Update dynamic cubemap based on screen space bounding box for this frame     
mDC.UpdateUVs( DynamicCubemap.CorrectSSBB( ssbb ), widthcorrection, heightcorrection ); 

// Set up the program used to render to the texture
GLES20.glUseProgram(mProgram);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureName);
float[] projmatrix = new float[16]; // Projection Matrix
Matrix.orthoM(projmatrix, 0, -1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f);
Matrix.multiplyMM(mMVPMatrix, 0, projmatrix, 0, mVMatrix, 0);
GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0);
GLES20.glUniform1i(mSamplerLoc, 0); // Use the camera texture (bound in unit zero)

// Render to the front face of the cubemap
// Note:  If I don't bind the new Framebuffer, this 
// renders the face to the screen very nicely without memory issues
GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_CUBE_MAP, 0); // Ensure we aren't rendering to the same texture we're using
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBuffers[5]);
GLES20.glViewport( 0, 0, edu.dhbw.andar.Config.CUBEMAP_SIZE, edu.dhbw.andar.Config.CUBEMAP_SIZE);
GLES20.glClear( GLES20.GL_COLOR_BUFFER_BIT );
mDC.DrawFace( 5, maPositionHandle, maTextureHandle ); // Draw the front face with glDrawArrays

// Unbind the framebuffer, we no longer need to render to textures.
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

// Ensure the newly generated cubemap is bound to the correct texture unit
GLES20.glBindTexture(GLES20.GL_TEXTURE_CUBE_MAP, mCubeMapTexture);

// Bind the old program and viewport
GLES20.glUseProgram( OldProgram[0] );
GLES20.glViewport( OldViewport[0], OldViewport[1], OldViewport[2], OldViewport[3] );

And That's it... Here's how I initialize my FBOs and Cubemap Textures when the program starts. (src/edu/dhbw/andar/ARGLES20Renderer.java ~Line 128)

// Generate Cubemap Textures
int[] cubemaptextures = new int[1];
GLES20.glGenTextures(1, cubemaptextures, 0 );
mCubeMapTexture = cubemaptextures[0];
GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_CUBE_MAP, mCubeMapTexture);   

for( int i = 0; i < 6; i++ ) {
    GLES20.glTexImage2D(GLES20.GL_TEXTURE_CUBE_MAP_POSITIVE_X + i, 0, mode, CUBEMAP_SIZE, CUBEMAP_SIZE, 0, mode, GLES20.GL_UNSIGNED_BYTE, ByteBuffer.wrap(frame));
}
GLES20.glTexParameterf(GLES20.GL_TEXTURE_CUBE_MAP, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameterf(GLES20.GL_TEXTURE_CUBE_MAP, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_CUBE_MAP, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_CUBE_MAP, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);
GLES20.glBindTexture(GLES20.GL_TEXTURE_CUBE_MAP, 0);

// Create a set of FrameBuffers for the cubemap
mFrameBuffers = new int[6];
GLES20.glGenFramebuffers(6, mFrameBuffers, 0);
for( int i = 0; i < 6; i++ ) {
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFrameBuffers[i]);
    GLES20.glFramebufferTexture2D( GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_CUBE_MAP_POSITIVE_X + i, mCubeMapTexture, 0 );
    GLES20.glCheckFramebufferStatus( GLES20.GL_FRAMEBUFFER );
}
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

Perhaps my ordering is wrong, or my setup is incorrect?

Sorry for the LONG post. I really did everything in my power to make this as short as possible while still giving enough information to solve the problem. I cut out a lot of extra code which is application specific. If you're interested, or you think the problem might be caused elsewhere, I included links to the actual source files so you can take a quick peek.

Thanks for your time! I've wasted FAR too much time on this.

-Griff

Edit: clarified texture size