Post-processing of large data sets produced on the HPCS may involve interactive visualisation software using 3D graphics. Broadly, there are three possible modes in which this might be attempted:
- The data is transferred over the network to the user's home site, and analysed there using a local graphical workstation. This requires time for the transfer, and that the user has sufficient storage and graphics capability available at home.
- The data remains at the HPCS, and the analysis software runs on a HPCS node. The dynamic graphical output (produced by OpenGL) is sent as a GLX protocol stream over an SSH tunnel (or tunnels) to a machine at the user's home site with a GLX-capable X display, and keystrokes and mouse events travel back through the same tunnels. (This is what happens automatically if you start a 3D X windows application on Darwin with SSH X forwarding enabled.) This may work slowly, or not at all depending on the application's support for indirect rendering.
- The data, the processing and the OpenGL 3D rendering all take place at the HPCS (using 3D graphics cards directly attached to the cluster), and only the final rendered pixels (possibly compressed) and keyboard/mouse events travel over the SSH tunnel.
Of these, the last makes the lightest demands on both the network connection, and the facilities at the home site, as neither large storage nor 3D graphics capability are actually required on the user side. Because only (compressed) rendered pixels are transmitted, interactive response should be best in this case, particularly if complex graphics are being generated. Finally, the method described below requires just a freely-downloadable remote desktop client and SSH software, which is available for Linux, Windows and MacOSX (no local X server being necessary).
Running a graphical program under VirtualGL
It is possible to use a limited number of NVIDIA Quadro FX5800 graphics cards installed at the HPCS to perform hardware-accelerated (direct) 3D rendering for remote display, using VirtualGLin conjunction with TurboVNC.
Now from the remote desktop session (which is assumed to be running on a node with a 3D graphics card, e.g. login-gfx1 or login-gfx2), start your graphical application through vglrun. E.g.
module load cuda/4.0 cd /usr/local/Cluster-Apps/cuda/4.0/NVIDIA_GPU_Computing_SDK/C/bin/linux/release/ vglrun smokeParticles
Note that omitting vglrun will start the program in the usual way, i.e. producing GLX commands which are expected to be rendered by the dummy X server providing your remote desktop - TurboVNC will simply error in this case with not GLX capable whereas NX will attempt to perform the rendering in software (which may fail outright, or just run very slowly). vglrun instead intercepts calls to the OpenGL library and arranges for the locally attached hardware to render the images, the resulting pixels of which are then sent uncompressed to the remote desktop.
When using TurboVNC, you will probably want to experiment with changing the compression level between the dummy X server on the login node and the viewer on your local machine - higher compression means greater interactivity at the cost of reduced image quality. This can be changed dynamically as needed.