Remote desktops & 3D visualization
In addition to the modes of interactive usage described in Interactive use, which can be characterized as either command-line or traditional X windows remote access via SSH, we also support remote Linux desktops via VNC which allows 2D and optionally 3D graphical applications (with full hardware acceleration) to be run directly on cluster nodes.
Starting a new VNC session
The recommended form of remote desktop uses the freely downloadable software TurboVNC. This is because TurboVNC, which is centrally installed on Darwin, contains an improved codec for better transmission of rapidly varying 3D images. Hardware accelerated 3D (see below) will still work with other forms of VNC (or e.g. NX), although in these cases the performance may be decreased depending on the codecs available for compressing and decompressing the desktop images.
Since the 3D graphics cards are attached to the login-gfx nodes, in order to use 3D the TurboVNC server should be started on one of these nodes. (Please also email support"at"hpc.cam.ac.uk and request access to the graphics hardware.)
If you don't plan to make use of 3D graphics, then it makes more sense to launch a VNC session on an ordinary login node (in the example commands below, replace login-gfx1 with the name of a particular login-sand or login-west node in this case).
To start a VNC session, first ssh/PuTTY in your usual way to a Darwin login node (use login-gfx1 or login-gfx2 if you need OpenGL 3D rendering). Obviously it is best to choose a relatively non-busy node (the commands top or w will display current system load). Then do:
(where in the default environment this will refer to the correct version of VNC). This will start a VNC session on the next available display number. The name of the display will be reported by the vncserver command in the form hostname:N where N is the display number. You will need to note the display number, and also remember the hostname.
Note that vncserver -list will list active sessions belonging to you which are running on the same node, and -kill can be used to terminate a specified display number. The first time you use VNC you will be prompted to create a VNC password - this will store the password under ~/.vnc. Please don't use your Darwin login password for this.
Having launched a VNC session on a login node, it is possible to log out of Darwin while leaving the session, and applications within the session, running and to reconnect to it later, perhaps from a different location. Please try not to create multiple sessions on different nodes by accident - there is no reason for anyone to have more than one operating at a time since a single virtual desktop can display applications running on any Darwin node (like any X windows display).
Connecting to an existing VNC session
This subsection deals with connecting to a pre-existing VNC session from your local client machine. The client software can be downloaded from the web site (.deb and .rpm files for Debian or RedHat-style Linux distributions, .exe for MS Windows, .dmg for MacOSX). For example, to connect to VNC display :N on login-gfx1 as user abc123, one might do from Linux:
/opt/TurboVNC/bin/vncviewer -via email@example.com localhost:N
where this assumes that the TurboVNC executables have been installed in their default location. This should first prompt for the normal login password to establish an SSH connection to Darwin, then prompt for the VNC password. The VNC connection is being tunnelled automatically through the initial SSH connection.
If, alternatively, your local machine is running Windows, then unfortunately the TurboVNC client cannot tunnel automatically and it becomes necessary to perform an additional step in order to forward local port 5900+N (where N is the VNC display number) to the same port number at the VNC server host (in the above example, assuming the display number is 1, this would be from local client port 5901 to localhost port 5901 on login-gfx1). On the Cygwin command line (similarly on MacOSX, and indeed on Linux if you wish to avoid the -via option) this tunnel can be created by doing:
ssh -L 5901:localhost:5901 firstname.lastname@example.org
(effectively this is what the Linux vncviewer -via option does internally). The equivalent step is also possible with PuTTY. Keeping the login session which this creates open, connect your local TurboVNC client to localhost:N - the SSH tunnel ensures that the connection is actually made (securely) to the correct VNC session.
A GNOME desktop session should then start up inside a window on your local machine. Note that F8 brings up a useful control panel, including a way to change the level of image compression on the fly (the default is tight + perceptually lossless JPEG which is probably fine for the CUDN; from home broadband I generally find tight + medium quality JPEG works better if I am running 3D applications). Note also the request lossless refresh option which forces an instant update of the desktop without compression (thereby removing any imperfections due to lossy compression). On the Windows client these features are accessed from the top menu bar.
Mouse cut and paste works in the obvious way between windows inside and outside the remote desktop window unless the window is a GNOME terminal which has strange clipboard handling; I usually start an xterm instead which works in the way you expect (select text by dragging then drop with middle button).
Note that there is also a (slower) java applet TurboVNC client, which provides a way to connect even if you don't have TurboVNC software installed on your local machine. To use this, login via ssh normally but set up tunnels from local client ports 5800+N and 5900+N, to the same ports at the VNC server host (where N is the VNC display number), e.g.
ssh -L 5901:localhost:5901 -L 5801:localhost:5801 email@example.com
(the first -L is similar to what is set up automatically by the -via option to vncviewer). Having done this, you should be able to point a browser at http://localhost:5801 to access display :1 on login-gfx1.
Post-processing of large data sets produced on the HPCS may involve interactive visualisation software using 3D graphics. Broadly, there are three possible modes in which this might be attempted:
- The data is transferred over the network to the user's home site, and analysed there using a local graphical workstation. This requires time for the transfer, and that the user has sufficient storage and graphics capability available at home.
- The data remains at the HPCS, and the analysis software runs on a HPCS node. The dynamic graphical output (produced by OpenGL) is sent as a GLX protocol stream over an SSH tunnel (or tunnels) to a machine at the user's home site with a GLX-capable X display, and keystrokes and mouse events travel back through the same tunnels. (This is what happens automatically if you start a 3D X windows application on Darwin with SSH X forwarding enabled.) This may work slowly, or not at all depending on the application's support for indirect rendering.
- The data, the processing and the OpenGL 3D rendering all take place at the HPCS (using 3D graphics cards directly attached to the cluster), and only the final rendered pixels (possibly compressed) and keyboard/mouse events travel over the SSH tunnel.
Of these, the last makes the lightest demands on both the network connection, and the facilities at the home site, as neither large storage nor 3D graphics capability are actually required on the user side. Because only (compressed) rendered pixels are transmitted, interactive response should be best in this case, particularly if complex graphics are being generated. Finally, the method described below requires just a freely-downloadable remote desktop client and SSH software, which is available for Linux, Windows and MacOSX (no local X server being necessary).
Running a graphical program under VirtualGL
It is possible to use a limited number of NVIDIA Quadro FX5800 graphics cards installed at the HPCS to perform hardware-accelerated (direct) 3D rendering for remote display, using VirtualGL in conjunction with TurboVNC.
Now from the remote desktop session (which is assumed to be running on a node with a 3D graphics card, e.g. login-gfx1 or login-gfx2), start your graphical application through vglrun. E.g.
module load cuda/4.0 cd /usr/local/Cluster-Apps/cuda/4.0/NVIDIA_GPU_Computing_SDK/C/bin/linux/release/ vglrun smokeParticles
Note that omitting vglrun will start the program in the usual way, i.e. producing GLX commands which are expected to be rendered by the dummy X server providing your remote desktop - TurboVNC will simply error in this case with not GLX capable whereas NX will attempt to perform the rendering in software (which may fail outright, or just run very slowly). vglrun instead intercepts calls to the OpenGL library and arranges for the locally attached hardware to render the images, the resulting pixels of which are then sent uncompressed to the remote desktop.
When using TurboVNC, you will probably want to experiment with changing the compression level between the dummy X server on the login node and the viewer on your local machine - higher compression means greater interactivity at the cost of reduced image quality. This can be changed dynamically as needed.
There are a number of 3D (OpenGL) applications currently available on Darwin that can be used with VirtualGL. Please send requests for additional packages to support'at'hpc.cam.ac.uk, bearing in mind that proprietary software needs to be properly licensed (to be arranged by your project).
The current list includes:
(i.e. module load modulename)
(see module help modulename)
|Availability on HPCS Systems||Website|
|PyMOL||1.3r2||pymol/1.3r2||Open Source (Python)||http://pymol.org/||Partiview||CVS 2009||Globally available||Freely available (Illinois Open Source License)||http://virdir.ncsa.illinois.edu/partiview/|
|VisIt||2.3.0||visit/2.3.0||Open Source (BSD)||https://wci.llnl.gov/codes/visit/|