3D support for X11 guests
Our 3D support for X11 guests is a bit of a dilemma for us. There are many, many issues which would be relatively easy to fix. Individually each one might be a day or two's work for a developer. Unfortunately it does not really make sense to address them individually, and just now none of our developers has much time to spend on this. Since a lot of the problems are things which could probably be fixed by users with reasonable 3D development skills, we have decided that for now it makes sense to declare 3D support for X11 guests as "user supported". We will be happy to integrate patches to fix individual problems if we can see without too much effort that they are not going to break other things, and we will be glad to provide other assistance as time permits.
Investigating problems
A good approach for many, though not all, issues is to start by creating the smallest possible test application which triggers the problem. As of that point, running the application with debug builds of the 3D guest libraries can help to see what is happening. Since many of the 3D applications running under X11 are free software, asking the developers of the applications for help might also be worth doing. We are open to suggestions about how we can help without investing too much developer time (including things to help build up a user 3D development community), and we are also happy to answer questions about how our 3D support works now. Please use the vbox-dev mailing list for this. We would particularly appreciate it if people contributing can document what they learn in a form that we can make publicly available.
Regarding the "3D applications always on top" bug, see ticket #12738 comment 9.
Quick overview of X11 3D pass-through
The 3D pass-through code lives in the VirtualBox source directory src/VBox/Additions/common/crOpenGL. The library VBoxOGL.so implements an OpenGL and a GLX library by respectively marshalling OpenGL calls into a data stream which is sent to the host (I haven't looked into how feedback from the host graphics card is handled) and talking to the guest X server to find out things like guest window position, so that 3D content can be placed on top of the right guest windows (try dragging guest windows with 3D content in them to understand the approach and its drawbacks). VBoxEGL.so is simply an X11-only EGL implementation wrapped around the GLX one. Anyone interested in making it work under Android or Wayland should probably expand this (eglGetDisplay() is designed to allow selecting different implementations) by studying the GLX code and using that as a basis. Wayland support would probably mean implementing a replacement libgbm.so too.
