OpenGL Paint Cycle
Unlike most other OpenGL Applications, the cycle by which Compiz rasterizes the entire screen has multiple steps to it, each step interfacable and and serving a different function to the last. The main thing to keep in mind is that there is not anything special about the way Compiz uses OpenGL - at the end of the day, it is just drawing textures with vertex points on to the screen, each on top of the last, in order to produce the image that you see. You can draw anything you like on to the screen using OpenGL, whether it be a texture, or just core opengl drawing commands and primitives. If you hook the paint process at exactly the right section and draw there, it will be drawn in the z-order that you specify.
Output Paint Cycle
PrivateGLScreen::paintOutputs - All Output Stage, this is where the OpenGL Viewport is set up, and we set up output painting to redraw certain regions on the screen
GLScreen::glPaintOutput - Single Output Stage, this is where we redraw the contents of a region of a particular output. No windows have been drawn at this point
GLScreen::glPaintTransformedOutput - Single Transformed Output Stage, this is where we redraw the contents of a particular output which a plugin has indicated has been completely transformed. Considering that plugins may have drawn before the plugin that did the transforming, they may wish to clear their drawing and redraw again transformed
GLScreen::paintBackground - Paint background texture, this is where we draw the background texture on-screen. The background texture we do draw is the one set up by our display manager or whatever set a pixmap to draw as the background. There is always at least one of these or a single black pixmap. This is so that we do not end up having window trails drawn all over the screen
Window Paint Cycle
GLWindow::glPaint - Basic window paint phase, the OpenGL context is initialized here, but correct matrices have not been pushed. Nothing should be drawn with OpenGL during this phase. This phase is really just for updating window transformation matrices and updating paint attributes.
GLWindow::glDraw - The next phase on the drawing chain - here the matrices are set up, and you should be free to draw whatever you want within this window's matrix. Note here that we also have access to GLFragment::Attrib so we can start adding custom fragment programs to the window.
GLWindow::glAddGeometry - The next phase on the drawing chain - here we don't actually draw anything, but we add the rects which describe the various point's of the windows geometry. Here is where it gets tricky, since now we are describing windows as textures (so this step can be repeated a number of times with different geometries and textures on the next phases).
GLWindow::glDrawTexture - The next phase on the drawing chain - here is the last point before we start adding vertexes and enabling textures. You have access to the texture data GLTexture and all fragment programs which will be applied to it
GLWindow::glDrawGeometry - At this point we have enabled all fragment programs, enable the texture, do the modifications which allow it to be painted with brightness, saturation and opacity and start adding vertex point arrays.