AMBER Archive (2002)

Subject: MPI (parallel) version of GIBBS

From: Etienne ENGLER (etienne_at_crypt.u-strasbg.fr)
Date: Mon Jan 07 2002 - 06:13:45 CST

(from amber maillist : Wed Jan 17 2001)

An experimental solution to gibbs MPI (Linux MPICH, IBM SP) :

1) To enable MPI

gib.f : lines 312-317

 Replace
c --- for now, Aug 1997, disable MPI_gibbs, since it fails the
c dynamically modified windows test .... dac
c
      write(6,*) 'MPI version of gibbs is currently disabled.'
      call mexit(6,1)
c

 with
c --- for now, Aug 1997, disable MPI_gibbs, since it fails the
c dynamically modified windows test .... dac
c
c write(6,*) 'MPI version of gibbs is currently disabled.'
c call mexit(6,1)
c

2) To avoid a locking (wait for other process reply)

gibb.f : lines 2372-2376

 Replace
C If IFRC>0 then IMES was > 0 on call. In this case only wanted
C derivates, so return now.
C
      IF (IFRC.GT.0) RETURN
C

 with
C If IFRC>0 then IMES was > 0 on call. In this case only wanted
C derivates, so return now.
C
      IF (IFRC.GT.0) then
#ifdef MPI
       if (ICORC.eq.1) call cor_mpi_wait(epert,itcumx)
#endif
        RETURN
      endif
C

3) Add this routine (in machinedep.f for example) :

#ifdef MPI
      subroutine cor_mpi_wait(epert,itcumx)
c
      IMPLICIT DOUBLE PRECISION (A-H,O-Z)
# include "mpif.h"
      DOUBLE PRECISION E58, E59, E60
      COMMON/JMTE/E58, E59, E60
      DIMENSION EPERT(*)
c
      DO 10 ITERA=1,ITCUMX
       ITER = ITERA
        DO 20 IFRE = 1,2
         IFR = IFRE
c
         e58 = 0.
         e59 = 0.
         call mpi_reduce(EPERT(58),e58,1,
     + MPI_DOUBLE_PRECISION,MPI_SUM,0,MPI_COMM_WORLD,ierr)
         call mpi_reduce(EPERT(59),e59,1,
     + MPI_DOUBLE_PRECISION,MPI_SUM,0,MPI_COMM_WORLD,ierr)
         EPERT(58) = e58
         EPERT(59) = e59
c
         IF (ITER.EQ.1 .AND. IFR.EQ.1) THEN
           e60 = 0.
           call mpi_reduce(EPERT(60),e60,1,
     + MPI_DOUBLE_PRECISION,MPI_SUM,0,MPI_COMM_WORLD,ierr)
           EPERT(60) = e60
         END IF
   20 continue
   10 continue
c
      end
#endif

Etienne Engler
MSM - UMR7551
ULP Strasbourg