AMBER Archive (2008)

Subject: RE: AMBER: SANDER and PMEMD with openmpi. Failure 'make test.pmemd'

From: Francesco Pietra (chiendarret_at_yahoo.com)
Date: Sun Jun 01 2008 - 02:12:12 CDT

Hi Ross:
Great! Finally pmemd running with openmpi. All test PASSED, no diff file present.

I started chunk 6 from chunk 1-5 already carried out (for a large system in a POPC membrane) with sander.MPI using the command

mpirun -np 8 $AMBERHOME/exe/pmemd -O -i prod.in ....(as with sander)

where prod.in reads:

prod protein & ligand & membrane box80x80
 &cntrl
  imin=0, irest=1, ntx=5,
  nstlim=333334, dt=0.0015,
  cut=10, ntb=2, ntp=1, taup=2.0,
  ntc=2, ntf=2,
  ntpr=1000, ntwx=1000,
  ntt=3, gamma_ln=2.0,
  temp0=300.0,
 /

It is running with 0.3% MEM usage (out of the 24GB available), while with sander it was 0.4%. Of course all cpus 100%.

In view of pmemd and 8 cpus, do you see any improvement to the above prod.in?

I noticed your suggestion some time ago as to run with pmemd in chunks, though now I want to check the speed with respect to sander (and I am at modest 8 processors).

Thanks a lot
francesco

--- On Sat, 5/31/08, Ross Walker <ross_at_rosswalker.co.uk> wrote:

> From: Ross Walker <ross_at_rosswalker.co.uk>
> Subject: RE: AMBER: SANDER and PMEMD with openmpi. Failure 'make test.pmemd'
> To: amber_at_scripps.edu
> Date: Saturday, May 31, 2008, 1:57 PM
> Hi Francesco,
>
> If you are using an AMD chip then you need to use different
> vectorization
> options. The build I showed was for an Intel EM64T chip. I
> would suggest
> changing the following:
>
> Config.h
>
> Line 27: F90_OPT_LO = -tpp7 -O0
> To F90_OPT_LO = -O0
>
> Line 28: F90_OPT_MED = -tpp7 -O2
> To F90_OPT_MED = -O2
>
> Line 29: F90_OPT_HI = -tpp7 -xP -ip -O3
> To F90_OPT_HI = -axWPT -ip -O3
>
> This set of options should work on all modern AMD and Intel
> x86_64 chips
> using ifort > v10.0.1
>
> All the best
> Ross
>
> > -----Original Message-----
> > From: owner-amber_at_scripps.edu
> [mailto:owner-amber_at_scripps.edu] On Behalf
> > Of Francesco Pietra
> > Sent: Saturday, May 31, 2008 11:31 AM
> > To: amber_at_scripps.edu
> > Subject: Re: AMBER: SANDER and PMEMD with openmpi.
> Failure 'make
> > test.pmemd'
> >
> > Hi:
> >
> > With Amber10
> >
> > >cd $AMBERHOME/test/
> > >make clean (which removed previous
> TEST_FAILURES.diff)
> > >make test.PMEMD
> > make: *** No rule to make target
> 'test.PMEMD'.stop
> > I corrected PMEMD to pmemd, getting error about the
> cpu type
> >
> > What I did in detail:
> > Renamed previous amber10 installation. Then, I begun
> from scratch as on
> > previous compilation I applied bugfix.all to only
> amber10. Now also to
> > amber_tools.
> >
> > >i-fort -V
> > 10.1
> >
> > >icc -V
> > 10.1
> >
> > >gcc -v
> > 4.2.3 (Debian lenny 4.2.3-5)
> >
> > >uname -a
> > 2.6.24-1-amd64 #1 SMP May 10 09:28:10 UTC 2008
> >
> > >cat /proc/cpuinfo
> > ...
> > cpu family: 15
> > model: 33
> > model name: Dual Core Opteron (tm) Processor 875
> 2000.006MHz cache 1024MB
> >
> > >echo $MKL_HOME
> > /opt/intel/mkl/10.0.1.14/
> >
> > In my .bashrc:
> > declare -x MPI_HOME='/usr/local'
> > MPICH_HOME=/usr/local (this serves for DOCK6.2 nicely)
> >
> > >which mpirun
> > /usr/local/bin/mpirun
> >
> > >mpif90 -show
> > /opt/intel/fce/10.1.015/bin/ifort -I/usr/local/include
> -pthread
> > -l/usr/local/lib -L/usr/local/lib -lmpi_f90 -lmpi_f77
> -lmpi -lopen-rte -
> > lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil
> >
> >
> > --Installed amber_tools and recompiled after
> bugfix.all
> >
> > --untarred amber10 and applied its bugfix.all,
> compiled serial
> > (.configure_amber ifort) all tests - including QMMM -
> PASSED)
> >
> > >cd src
> > >make clean
> > >./configure_amber -openmpi -nobintraj ifort
> > >make parallel
> >
> > --all test.parallel.MM PASSED (-np 4)
> >
> > --test.parallel.QMM crashed at ./Run.1NLN.dspevd as
> expected for my system
> > (see previous thread)
> >
> > >cd src/PMEMD
> > >./configure linux_em64t ifort lam pubfft nobintraj
> >
> > ---Edited config.h as indicated below by Ross.
> >
> > >make
> > (no errors reported)
> >
> > > make install
> > installation of pmemd complete
> >
> > >export DO_PARALLEL='mpirun -np 2'
> > >set | grep DO_PARALLEL
> > (OK)
> >
> > >make test.parallel (OK, of course, it has already
> passed with -np 4)
> >
> > >make test.parallel.QMMM
> > same crash as with -np 4
> >
> > >make clean
> > (find . -name '*.dif' -o name
> 'profile_mpi' |\
> > while read dif ;\
> > do \
> > rm -f $ dif ; \
> > done ;\
> > rm -f TEST_FAILURES.diff
> >
> > > make test. PMEMD
> > make: *** No rule to make target 'test.PMEMD'.
> Stop
> >
> > Examining the Makefile, I corrected Ross' PMEMD
> with pmemd
> >
> > >make clean
> > >make install 2>&1 | tee .....
> > getting the error file:
> >
> > export TESTsander='../../exe/pmemd'; cd
> 4096wat && ./Run.pure_wat
> >
> > Fatal Error: This program was not built to run on the
> processor in your
> > system.
> > The allowed processors are: Intel(R) Pentium(R) 4 and
> compatible Intel
> > processors with Streaming SIMD Extensions 3 (SSE3)
> instruction support.
> >
> >
> > Fatal Error: This program was not built to run on the
> processor in your
> > system.
> > The allowed processors are: Intel(R) Pentium(R) 4 and
> compatible Intel
> > processors with Streaming SIMD Extensions 3 (SSE3)
> instruction support.
> >
> > ./Run.pure_wat: Program error
> > make: *** [test.pmemd] Error 1
> > ______
> > At this point I feel I need help. Even if I think
> about longer, little
> > chance. It does not like dual-opteron or I missed an
> appropriate flag.
> >
> > Thanks for help
> > francesco
> >
> >
> > --- On Fri, 5/30/08, Ross Walker
> <ross_at_rosswalker.co.uk> wrote:
> >
> > > From: Ross Walker <ross_at_rosswalker.co.uk>
> > > Subject: AMBER: SANDER and PMEMD with openmpi
> > > To: amber_at_scripps.edu
> > > Date: Friday, May 30, 2008, 3:18 PM
> > > Hi All,
> > >
> > > It seems a number of people are interested in
> building both
> > > sander and pmemd
> > > using openmpi and so I thought I would post a
> description
> > > of how I did this
> > > on my machine so others can benefit:
> > >
> > > ----------
> > > >ifort -V
> > > Intel(R) Fortran Compiler for applications
> running on
> > > Intel(R) 64, Version
> > > 10.1 Build 20070913 Package ID:
> l_fc_p_10.1.008
> > > >icc -V
> > > Copyright (C) 1985-2007 Intel Corporation. All
> rights
> > > reserved.
> > >
> > > >gcc -v
> > > gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)
> > >
> > > >cat /etc/redhat-release
> > > Red Hat Enterprise Linux AS release 4 (Nahant
> Update 6)
> > >
> > > >uname -a
> > > Linux ......... 2.6.9-67.0.4.ELsmp #1 SMP Fri Jan
> 18
> > > 05:00:00 EST 2008
> > > x86_64 x86_64 x86_64 GNU/Linux
> > >
> > > >cat /proc/cpuinfo
> > > ...
> > > cpu family : 15
> > > model : 6
> > > model name : Intel(R)
> Pentium(R) D CPU
> > > 3.20GHz
> > > ...
> > >
> > > >echo $MKL_HOME
> > > /opt/intel/mkl/10.0.1.014/
> > >
> > > >wget
> > >
> http://www.open-mpi.org/software/ompi/v1.2/downloads/openmpi-
> > 1.2.6.tar.gz
> > > >tar xvzf openmpi-1.2.6.tar.gz
> > > >cd openmpi-1.2.6
> > > >export CC=gcc
> > > >export CXX=g++
> > > >export F77=ifort
> > > >export FC=ifort
> > >
> > > >./configure
> > >
> --prefix=/usr/local/mpi/openmpi-1.2.6_ifort10.1.008
> > > >make
> > > >make install
> > >
> > > >export
> > >
> MPI_HOME=/usr/local/mpi/openmpi-1.2.6_ifort10.1.008
> > > >export PATH=$MPI_HOME/bin:$PATH
> > > >export
> LD_LIBRARY_PATH=$MPI_HOME/lib:$LD_LIBRARY_PATH
> > >
> > > >which mpirun
> > >
> /usr/local/mpi/openmpi-1.2.6_ifort10.1.008/bin/mpirun
> > >
> > > >mpif90 -show
> > > ifort -g
> -I/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/include
> > >
> -I/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/include
> > > -L/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/lib
> -lmpichf90
> > > -lmpichf90 -lmpich
> > > -lpthread -lrt
> > >
> > > >cd ~/
> > > >tar xvjf Amber10.tar.bz2
> > > >export AMBERHOME=~/amber10/
> > >
> > > >cd $AMBERHOME/src/
> > > >./configure_amber -openmpi -nobintraj ifort
> > > >make parallel
> > >
> > > >cd $AMBERHOME/src/pmemd/
> > > > ./configure linux_em64t ifort lam pubfft
> nobintraj
> > >
> > > edit config.h
> > > Line 17:
> > > Change MPI_LIBS = -L$(MPI_LIBDIR) -llamf77mpi
> -lmpi -llam
> > > -ldl -lpthread
> > > to MPI_LIBS = -L$(MPI_LIBDIR)
> > >
> > > Line 23:
> > > Change F90 = ifort
> > > to F90 = mpif90
> > >
> > > Line 35:
> > > Change LOAD = ifort
> > > to LOAD = mpif90
> > >
> > > >make
> > > >make install
> > >
> > > >export DO_PARALLEL='mpirun -np 2'
> > >
> > > >cd $AMBERHOME/test/
> > > >make test.parallel
> > > >make test.parallel.QMMM
> > >
> > > (Check TEST_FAILURES.diff - all tests passed file
> did not
> > > exist)
> > >
> > > >make clean
> > > >make test.PMEMD
> > >
> > > (Check TEST_FAILURES.diff - all tests passed file
> did not
> > > exist)
> > >
> > > --------
> > >
> > > All the best
> > > Ross
> > >
> > > /\
> > > \/
> > > |\oss Walker
> > >
> > > | Assistant Research Professor |
> > > | San Diego Supercomputer Center |
> > > | Tel: +1 858 822 0854 | EMail:-
> ross_at_rosswalker.co.uk |
> > > | http://www.rosswalker.co.uk | PGP Key available
> on
> > > request |
> > >
> > > Note: Electronic Mail is not secure, has no
> guarantee of
> > > delivery, may not
> > > be read every day, and should not be used for
> urgent or
> > > sensitive issues.
> > >
> > >
> > >
> > >
> -----------------------------------------------------------------------
> > > The AMBER Mail Reflector
> > > To post, send mail to amber_at_scripps.edu
> > > To unsubscribe, send "unsubscribe
> amber" (in the
> > > *body* of the email)
> > > to majordomo_at_scripps.edu
> >
> >
> >
> >
> -----------------------------------------------------------------------
> > The AMBER Mail Reflector
> > To post, send mail to amber_at_scripps.edu
> > To unsubscribe, send "unsubscribe amber" (in
> the *body* of the email)
> > to majordomo_at_scripps.edu
>
> -----------------------------------------------------------------------
> The AMBER Mail Reflector
> To post, send mail to amber_at_scripps.edu
> To unsubscribe, send "unsubscribe amber" (in the
> *body* of the email)
> to majordomo_at_scripps.edu

      
-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
      to majordomo_at_scripps.edu