AMBER Archive (2008)

Subject: Fw: RE: AMBER: SANDER and PMEMD with openmpi. NO MORE failure 'make test.pmemd'

From: Francesco Pietra (chiendarret_at_yahoo.com)
Date: Mon Jun 02 2008 - 05:22:38 CDT

Hi Ross:
Under the conditions described below, prod.in run with pmemd 38.6% faster than with sander.MPI.

francesco

--- On Sun, 6/1/08, Francesco Pietra <chiendarret_at_yahoo.com> wrote:

> From: Francesco Pietra <chiendarret_at_yahoo.com>
> Subject: RE: AMBER: SANDER and PMEMD with openmpi. Failure 'make test.pmemd'
> To: amber_at_scripps.edu
> Date: Sunday, June 1, 2008, 12:12 AM
> Hi Ross:
> Great! Finally pmemd running with openmpi. All test PASSED,
> no diff file present.
>
> I started chunk 6 from chunk 1-5 already carried out (for a
> large system in a POPC membrane) with sander.MPI using the
> command
>
> mpirun -np 8 $AMBERHOME/exe/pmemd -O -i prod.in ....(as
> with sander)
>
> where prod.in reads:
>
> prod protein & ligand & membrane box80x80
> &cntrl
> imin=0, irest=1, ntx=5,
> nstlim=333334, dt=0.0015,
> cut=10, ntb=2, ntp=1, taup=2.0,
> ntc=2, ntf=2,
> ntpr=1000, ntwx=1000,
> ntt=3, gamma_ln=2.0,
> temp0=300.0,
> /
>
> It is running with 0.3% MEM usage (out of the 24GB
> available), while with sander it was 0.4%. Of course all
> cpus 100%.
>
> In view of pmemd and 8 cpus, do you see any improvement to
> the above prod.in?
>
> I noticed your suggestion some time ago as to run with
> pmemd in chunks, though now I want to check the speed with
> respect to sander (and I am at modest 8 processors).
>
> Thanks a lot
> francesco
>
> --- On Sat, 5/31/08, Ross Walker
> <ross_at_rosswalker.co.uk> wrote:
>
> > From: Ross Walker <ross_at_rosswalker.co.uk>
> > Subject: RE: AMBER: SANDER and PMEMD with openmpi.
> Failure 'make test.pmemd'
> > To: amber_at_scripps.edu
> > Date: Saturday, May 31, 2008, 1:57 PM
> > Hi Francesco,
> >
> > If you are using an AMD chip then you need to use
> different
> > vectorization
> > options. The build I showed was for an Intel EM64T
> chip. I
> > would suggest
> > changing the following:
> >
> > Config.h
> >
> > Line 27: F90_OPT_LO = -tpp7 -O0
> > To F90_OPT_LO = -O0
> >
> > Line 28: F90_OPT_MED = -tpp7 -O2
> > To F90_OPT_MED = -O2
> >
> > Line 29: F90_OPT_HI = -tpp7 -xP -ip -O3
> > To F90_OPT_HI = -axWPT -ip -O3
> >
> > This set of options should work on all modern AMD and
> Intel
> > x86_64 chips
> > using ifort > v10.0.1
> >
> > All the best
> > Ross
> >
> > > -----Original Message-----
> > > From: owner-amber_at_scripps.edu
> > [mailto:owner-amber_at_scripps.edu] On Behalf
> > > Of Francesco Pietra
> > > Sent: Saturday, May 31, 2008 11:31 AM
> > > To: amber_at_scripps.edu
> > > Subject: Re: AMBER: SANDER and PMEMD with
> openmpi.
> > Failure 'make
> > > test.pmemd'
> > >
> > > Hi:
> > >
> > > With Amber10
> > >
> > > >cd $AMBERHOME/test/
> > > >make clean (which removed previous
> > TEST_FAILURES.diff)
> > > >make test.PMEMD
> > > make: *** No rule to make target
> > 'test.PMEMD'.stop
> > > I corrected PMEMD to pmemd, getting error about
> the
> > cpu type
> > >
> > > What I did in detail:
> > > Renamed previous amber10 installation. Then, I
> begun
> > from scratch as on
> > > previous compilation I applied bugfix.all to only
> > amber10. Now also to
> > > amber_tools.
> > >
> > > >i-fort -V
> > > 10.1
> > >
> > > >icc -V
> > > 10.1
> > >
> > > >gcc -v
> > > 4.2.3 (Debian lenny 4.2.3-5)
> > >
> > > >uname -a
> > > 2.6.24-1-amd64 #1 SMP May 10 09:28:10 UTC 2008
> > >
> > > >cat /proc/cpuinfo
> > > ...
> > > cpu family: 15
> > > model: 33
> > > model name: Dual Core Opteron (tm) Processor 875
> > 2000.006MHz cache 1024MB
> > >
> > > >echo $MKL_HOME
> > > /opt/intel/mkl/10.0.1.14/
> > >
> > > In my .bashrc:
> > > declare -x MPI_HOME='/usr/local'
> > > MPICH_HOME=/usr/local (this serves for DOCK6.2
> nicely)
> > >
> > > >which mpirun
> > > /usr/local/bin/mpirun
> > >
> > > >mpif90 -show
> > > /opt/intel/fce/10.1.015/bin/ifort
> -I/usr/local/include
> > -pthread
> > > -l/usr/local/lib -L/usr/local/lib -lmpi_f90
> -lmpi_f77
> > -lmpi -lopen-rte -
> > > lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil
> > >
> > >
> > > --Installed amber_tools and recompiled after
> > bugfix.all
> > >
> > > --untarred amber10 and applied its bugfix.all,
> > compiled serial
> > > (.configure_amber ifort) all tests - including
> QMMM -
> > PASSED)
> > >
> > > >cd src
> > > >make clean
> > > >./configure_amber -openmpi -nobintraj ifort
> > > >make parallel
> > >
> > > --all test.parallel.MM PASSED (-np 4)
> > >
> > > --test.parallel.QMM crashed at ./Run.1NLN.dspevd
> as
> > expected for my system
> > > (see previous thread)
> > >
> > > >cd src/PMEMD
> > > >./configure linux_em64t ifort lam pubfft
> nobintraj
> > >
> > > ---Edited config.h as indicated below by Ross.
> > >
> > > >make
> > > (no errors reported)
> > >
> > > > make install
> > > installation of pmemd complete
> > >
> > > >export DO_PARALLEL='mpirun -np 2'
> > > >set | grep DO_PARALLEL
> > > (OK)
> > >
> > > >make test.parallel (OK, of course, it has
> already
> > passed with -np 4)
> > >
> > > >make test.parallel.QMMM
> > > same crash as with -np 4
> > >
> > > >make clean
> > > (find . -name '*.dif' -o name
> > 'profile_mpi' |\
> > > while read dif ;\
> > > do \
> > > rm -f $ dif ; \
> > > done ;\
> > > rm -f TEST_FAILURES.diff
> > >
> > > > make test. PMEMD
> > > make: *** No rule to make target
> 'test.PMEMD'.
> > Stop
> > >
> > > Examining the Makefile, I corrected Ross'
> PMEMD
> > with pmemd
> > >
> > > >make clean
> > > >make install 2>&1 | tee .....
> > > getting the error file:
> > >
> > > export TESTsander='../../exe/pmemd'; cd
> > 4096wat && ./Run.pure_wat
> > >
> > > Fatal Error: This program was not built to run on
> the
> > processor in your
> > > system.
> > > The allowed processors are: Intel(R) Pentium(R) 4
> and
> > compatible Intel
> > > processors with Streaming SIMD Extensions 3
> (SSE3)
> > instruction support.
> > >
> > >
> > > Fatal Error: This program was not built to run on
> the
> > processor in your
> > > system.
> > > The allowed processors are: Intel(R) Pentium(R) 4
> and
> > compatible Intel
> > > processors with Streaming SIMD Extensions 3
> (SSE3)
> > instruction support.
> > >
> > > ./Run.pure_wat: Program error
> > > make: *** [test.pmemd] Error 1
> > > ______
> > > At this point I feel I need help. Even if I think
> > about longer, little
> > > chance. It does not like dual-opteron or I missed
> an
> > appropriate flag.
> > >
> > > Thanks for help
> > > francesco
> > >
> > >
> > > --- On Fri, 5/30/08, Ross Walker
> > <ross_at_rosswalker.co.uk> wrote:
> > >
> > > > From: Ross Walker
> <ross_at_rosswalker.co.uk>
> > > > Subject: AMBER: SANDER and PMEMD with
> openmpi
> > > > To: amber_at_scripps.edu
> > > > Date: Friday, May 30, 2008, 3:18 PM
> > > > Hi All,
> > > >
> > > > It seems a number of people are interested
> in
> > building both
> > > > sander and pmemd
> > > > using openmpi and so I thought I would post
> a
> > description
> > > > of how I did this
> > > > on my machine so others can benefit:
> > > >
> > > > ----------
> > > > >ifort -V
> > > > Intel(R) Fortran Compiler for applications
> > running on
> > > > Intel(R) 64, Version
> > > > 10.1 Build 20070913 Package ID:
> > l_fc_p_10.1.008
> > > > >icc -V
> > > > Copyright (C) 1985-2007 Intel Corporation.
> All
> > rights
> > > > reserved.
> > > >
> > > > >gcc -v
> > > > gcc version 3.4.6 20060404 (Red Hat 3.4.6-9)
> > > >
> > > > >cat /etc/redhat-release
> > > > Red Hat Enterprise Linux AS release 4
> (Nahant
> > Update 6)
> > > >
> > > > >uname -a
> > > > Linux ......... 2.6.9-67.0.4.ELsmp #1 SMP
> Fri Jan
> > 18
> > > > 05:00:00 EST 2008
> > > > x86_64 x86_64 x86_64 GNU/Linux
> > > >
> > > > >cat /proc/cpuinfo
> > > > ...
> > > > cpu family : 15
> > > > model : 6
> > > > model name : Intel(R)
> > Pentium(R) D CPU
> > > > 3.20GHz
> > > > ...
> > > >
> > > > >echo $MKL_HOME
> > > > /opt/intel/mkl/10.0.1.014/
> > > >
> > > > >wget
> > > >
> >
> http://www.open-mpi.org/software/ompi/v1.2/downloads/openmpi-
> > > 1.2.6.tar.gz
> > > > >tar xvzf openmpi-1.2.6.tar.gz
> > > > >cd openmpi-1.2.6
> > > > >export CC=gcc
> > > > >export CXX=g++
> > > > >export F77=ifort
> > > > >export FC=ifort
> > > >
> > > > >./configure
> > > >
> > --prefix=/usr/local/mpi/openmpi-1.2.6_ifort10.1.008
> > > > >make
> > > > >make install
> > > >
> > > > >export
> > > >
> > MPI_HOME=/usr/local/mpi/openmpi-1.2.6_ifort10.1.008
> > > > >export PATH=$MPI_HOME/bin:$PATH
> > > > >export
> > LD_LIBRARY_PATH=$MPI_HOME/lib:$LD_LIBRARY_PATH
> > > >
> > > > >which mpirun
> > > >
> > /usr/local/mpi/openmpi-1.2.6_ifort10.1.008/bin/mpirun
> > > >
> > > > >mpif90 -show
> > > > ifort -g
> > -I/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/include
> > > >
> > -I/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/include
> > > >
> -L/usr/local/mpi/mpich2-1.0.3_ifort9.1.039/lib
> > -lmpichf90
> > > > -lmpichf90 -lmpich
> > > > -lpthread -lrt
> > > >
> > > > >cd ~/
> > > > >tar xvjf Amber10.tar.bz2
> > > > >export AMBERHOME=~/amber10/
> > > >
> > > > >cd $AMBERHOME/src/
> > > > >./configure_amber -openmpi -nobintraj
> ifort
> > > > >make parallel
> > > >
> > > > >cd $AMBERHOME/src/pmemd/
> > > > > ./configure linux_em64t ifort lam
> pubfft
> > nobintraj
> > > >
> > > > edit config.h
> > > > Line 17:
> > > > Change MPI_LIBS = -L$(MPI_LIBDIR)
> -llamf77mpi
> > -lmpi -llam
> > > > -ldl -lpthread
> > > > to MPI_LIBS = -L$(MPI_LIBDIR)
> > > >
> > > > Line 23:
> > > > Change F90 = ifort
> > > > to F90 = mpif90
> > > >
> > > > Line 35:
> > > > Change LOAD = ifort
> > > > to LOAD = mpif90
> > > >
> > > > >make
> > > > >make install
> > > >
> > > > >export DO_PARALLEL='mpirun -np
> 2'
> > > >
> > > > >cd $AMBERHOME/test/
> > > > >make test.parallel
> > > > >make test.parallel.QMMM
> > > >
> > > > (Check TEST_FAILURES.diff - all tests passed
> file
> > did not
> > > > exist)
> > > >
> > > > >make clean
> > > > >make test.PMEMD
> > > >
> > > > (Check TEST_FAILURES.diff - all tests passed
> file
> > did not
> > > > exist)
> > > >
> > > > --------
> > > >
> > > > All the best
> > > > Ross
> > > >
> > > > /\
> > > > \/
> > > > |\oss Walker
> > > >
> > > > | Assistant Research Professor |
> > > > | San Diego Supercomputer Center |
> > > > | Tel: +1 858 822 0854 | EMail:-
> > ross_at_rosswalker.co.uk |
> > > > | http://www.rosswalker.co.uk | PGP Key
> available
> > on
> > > > request |
> > > >
> > > > Note: Electronic Mail is not secure, has no
> > guarantee of
> > > > delivery, may not
> > > > be read every day, and should not be used
> for
> > urgent or
> > > > sensitive issues.
> > > >
> > > >
> > > >
> > > >
> >
> -----------------------------------------------------------------------
> > > > The AMBER Mail Reflector
> > > > To post, send mail to amber_at_scripps.edu
> > > > To unsubscribe, send "unsubscribe
> > amber" (in the
> > > > *body* of the email)
> > > > to majordomo_at_scripps.edu
> > >
> > >
> > >
> > >
> >
> -----------------------------------------------------------------------
> > > The AMBER Mail Reflector
> > > To post, send mail to amber_at_scripps.edu
> > > To unsubscribe, send "unsubscribe
> amber" (in
> > the *body* of the email)
> > > to majordomo_at_scripps.edu
> >
> >
> -----------------------------------------------------------------------
> > The AMBER Mail Reflector
> > To post, send mail to amber_at_scripps.edu
> > To unsubscribe, send "unsubscribe amber" (in
> the
> > *body* of the email)
> > to majordomo_at_scripps.edu

      
-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
      to majordomo_at_scripps.edu