AMBER Archive (2009)

Subject: [AMBER] Compilation and usage of NAB in parallel ?

From: Marek Malý (
Date: Wed Jan 28 2009 - 16:25:24 CST

Dear Amber users,

I need urgently parallel version of NAB to use it for huge system (15000
atoms) analysis.

I found just a little about this topic in AmberTools manual pages 115-116.

Our platform is "SGI AltixXE 310" with "Intel Xeon Quad-core 5365"

I installed AmberTools (in serial using icc compiler) without any
problems and
also Amber using ifort compiler ( serial and also parallel with flag

Typical perfectly working script seems like this:

#number of nodes to load into virtual machine
#number of processors per node
#total number of processors allocated for the given job


#export NO_STOP_MESSAGE=" "
export EXE="/opt/amber/exe/sander.MPI"

#creation of virtual multiprocessor machine
mpdboot -f ~/.mpd.hosts -n $NODES

#run sander/pmemd
mpiexec -ppn $PPN -n $CPUS $EXE -O -i $IN -o $OUT -c $INPCRD -p $PRMTOP
-r $RST

#'destruction' of the virtual machine

I thaught that it will be easy to run like this also NAB in parallel but
it isn't :((
I tried something like this:



nab -O -o makej_MPI

mpdboot -f ~/.mpd.hosts -n $NODES

mpiexec -ppn $PPN -n $CPUS ./makej_MPI > results.txt



But it just loaded 8 serial jobs in one time nothing more.

The main problem is probably that I was not able to compile NAB in
parallel way.
I tried -mpi flag but using this option was absolutely useless it even
didn't add
the proper flags into config.h. When I tried "-openmp" flag situation was
a little
better and -openmp successfully propagated into config.h in this two lines:

-L$(MKL_HOME)/lib/em64t -lguide -lpthread -openmp
OCFLAGS=-wd117,177,266,880,1011 -O2 -DBINTRAJ -openmp -DOPENMP

But I am still not sure if even my compilation "nab -O -o makej_MPI" create now
parallel binary file "makej_MPI" since this should be probably clear from
the c source makej_MPI.c and
I didn't find any "MPI" strings here. Of course that also examination of
my binary file was not going in parallel,
although I tried play with OMP_NUM_THREADS constant and tried to run my
executable makej_MPI with or without mpiexec command.

In my opinion the root of the problem is that -intelmpi flag is not
allowed in case of NAB parallel configuration apart from
the AMBER where is available and works perfectly. Is there any chance how
to bypass this problem for example edit config.h
created with -openmp and put everywhere instead this flag just -intelmpi
or something like this ?

If is really no chance/trick how to use -intelmpi in case of parallel NAB
compilation what could be a solution for me ?
Since we have not ScaLAPACK installed yet, probably only one chance is to
use -openmp.

If "-intelmpi way" (IntelMPI) is really unpossible could anyone who has
some experience with parallel compilation/run of NAB with -openmp (under
provide a small "How to ?" = if and how modify "nab -O -o makej_MPI" (compilation of nab programs), how modify "mpiexec -ppn
$PPN -n $CPUS ./makej_MPI > results.txt" ...

Last my question is this: Is there any dependence of the number of
parallelisable NAB tools on the given flag (-openmp, -mpi, -scalapack ).
After reading of page 116 I think that yes and all the important
functions: conjgrad(), newton( ), nmode() ... are available in parallel
only using -scalapack flag. Am I right ? Could anyone please make simple
list of NAB functions which are parallelisable in each particular case (
-openmp, -mpi, -scalapack ).

Thanks a lot in advance ! for any relevant info or link/s to the relevant
materials which describes "NAB in PARALLEL" story a little more in detail
than AmberTools manual.


Tato zpráva byla vytvořena převratným poštovním klientem Opery:

_______________________________________________ AMBER mailing list