AMBER Archive (2002)

Subject: increasing the number of processors (fwd)

From: Ioana Cozmuta (ioana_at_nas.nasa.gov)
Date: Tue Dec 03 2002 - 18:19:24 CST

---------- Forwarded message ----------
Date: Tue, 3 Dec 2002 16:17:58 -0800 (PST)
From: Ioana Cozmuta <ioana_at_nas.nasa.gov>
To: amber_at_heimdal.compchem.ucsf.edu
Subject: increasing the number of processors

Hi amber users,

I want to increase the maximum number of processors that amber is allowed
to use and I have a couple of questions.

1. The present version is supposed to run on a maximum of 128 CPU's.
However in $AMBERHOME/src/sander/parallel.h and ew_parallel.h the
MPI_MAX_PROCESSORS is defined as 512.
Is this value related to the definition in the Machine file (in this case
Machine.sgi) thus being set during the compilation of amber?
****
# Number of Processors
set NPROC = `mpadmin -n | tail -1`
@ NPROC += 1
****
If not why is it's value 512 is the limit is 128?

2. Also in the Machine.sgi_mpich there is a definition for the MAX_PRO
variable depending on the IP of the machine. What is the relation between
the two variables, meaning why if the $WHICHIP == 7 then MAX_PRO = 8
etc.??? So for example if you have an Origin3000 what should be the value
to set for Max_PRO?

3. If I change in parallel.h the value
integer logtwo(128)
into
integer logtwo(1024)

then do I need to modify also the
data logtwo definition as well

      data logtwo/ 0,1,0,2,0,0,0,3,0,0,0,0,0,0,0,4,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
     . 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,7 /

I see that there are 128 values in the definition above but I am not sure
what it means exactly and whether this is connected to the 128 maximum
CPU's limit of amber or not. SO if I am changing the 128 into 1024 do I
need to change this data logtwo as well and what are the rules? These
numbers look like the log base 2 values for the first 128 integers with
theresults converted to integers. Is this correct?

4. in the /src/gibbs/machinedep.f there is a definition for
SHARED_MEMORY_MAX_PROCESSORS 36 for SGI_MP usage. Does this value need to
be changed when increasing the maximum number of processors?

5. One obvious change has to be done in sander.f, i.e. replacing 128 by
1024 in:
if( numtasks.gt.1 .and. (i.le.0 .or. numtasks.gt.128) ) then
. ' and no greater than 128: ',numtasks

6. I was advised that by increasing the number of processors from 128 to
1024 I am also changing the memory layout of the code and this can have
negative effects ont he code.
For example, if not all the arrays that need to be increased in size are
found, then execution of the code will likely overwrite portions of memory
that were not overwritten with only 128 processors and this could lead to
wrong numerical results.

Are there some more issues that I have to be aware when changing the above
parameters? I would greatly appreciate your advice.

Thank you in advance,
Ioana