---

Usage FAQ

System X Usage Frequently Asked Questions 

Below you will find a listing of the most commonly asked questions about System X and their answers:

1. How do I access System X?
2. What compilers are available on System X?
3. How do I compile programs on System X?
4. Are optimized versions of the BLAS available?
5. I'm having problems linking my code, and the problem seems to be with underscores in function names.
6. How do I submit jobs to the queue?
7. What about data storage?
8. Is there a debug cluster?
9. How do I get additional questions answered?

Answers

1. How do I access System X?

   Login to one of the compile nodes:

    'ssh sysx1.arc.vt.edu'
    'ssh sysx2.arc.vt.edu'
    'ssh sysx3.arc.vt.edu'
    'ssh sysx4.arc.vt.edu'      (Linux test)
   

2. What compilers are available on System X?

   Compilers available (default PATH already includes these directories):

   • gcc, located in /usr/bin
   • xlc and xlc++ family, located in /opt/ibmcmp/vacpp/6.0/bin
   • xlf family, located in /opt/ibmcmp/xlf/8.1/bin
   
   
3. How do I compile programs on System X?

   MPI compile scripts are located in /nfs/compilers/mpich-1.2.5/bin (also in the default PATH already). To compile and link MPI codes you should use one of these:

   mpicc  -- uses gcc compiler
   mpixlc -- uses xlc compiler
   mpif77 -- uses xlf f77 compiler
   mpif90 -- uses xlf f90 compiler
   

4. Are optimized versions of the BLAS available?

   Yes. Add "-framework vecLib" to your link line. See http:// developer.apple.com/documentation/Darwin/Reference/ManPages/man7/ vecLib.7.html for information on Apple's vecLib package. The package includes the Basic Linear Algebra Subprograms (BLAS), LAPACK, and other useful mathematical functions, some of which are vectorized. Note that only 32-bit operations are vectorized, however.

5. I'm having problems linking my code, and the problem seems to be with underscores in function names.

   For now, the vecLib framework and the MPI libraries are configured so that when they are linked from Fortran, they assume that all external names have an underscore appended to them. For example, if your Fortran code calls mpi_finalize, the linker will only find something called _mpi_finalize_ in the MPI library. If you use mpif77 or mpif90 to compile your code, this is automatically taken care of. But if you compile directly with xlf, you should use the -qextname command-line option to cause the underscore to be appended.

6. How do I submit jobs to the queue?

   We use Torque and Moab for resource management and scheduling. If you are familiar with Torque or OpenPBS qsub structure you should be able to submit jobs using the same commands.

   An example queue submission script can be found at:

   /nfs/docs/qsub-example.sh
   

   Additional documentation can be found here:

   users.html
   

7. What about data storage?

   We currently have a multiple TeraByte user storage resource that will be expanding soon. This storage is fault tolerant, but is not backed up. User data is the responsibility of the user, and it is up to the user to keep backups.

8. Is there a debug cluster?

   There is a debug cluster for System X users. This cluster is comprised of eight (8) nodes that are available through the following job startup info. Please note that this cluster is NOT for performance testing, but to test and correct execution errors. This cluster is for short jobs and is restarted every morning at 5:30 AM Eastern Time. To launch a job on the debug cluster use dbgrun:

   dbgrun [-printhostname] [-verbose] \
   -np N debug1 debug2 ... debug8 a.out [args]
   

   where N is number of nodes

   or:

   dbgrun [-printhostname] [-verbose] \
   -np N -hostfile hf a.out [args]
   

   where N is number of nodes and hf is a file with the nodes named (debug1-debug8)
   
   
9. How do I get additional questions answered?

   We have a very lean support staff at present, but we are working to expand it. Presently we offer the following response times for email and forum questions:

   Monday - Friday
   - maximum 4 hour turn around time between 8:00 AM and 5:00 PM
   
   Non-business Hours and Weekends
   - 12 - 24 hour turn around
   

   We have setup a listserve at tcf_support@listserv.vt.edu for support related questions. Support requests should include any .o and .e files generated by the job in question as attachments.