AMBER Archive (2008)

Subject: Re: AMBER: comparison of MD trajectories recorded with pmemd and sander

From: Carlos Simmerling (carlos.simmerling_at_gmail.com)
Date: Wed Dec 17 2008 - 09:12:32 CST

I often use sander for equilibration and pmemd for production, and consider
this homogeneous. I think that is fine. the differences between the programs
are comparable to differences one would expect between different
architectures or running on different levels of parallelism. the same would
be true for running the same force field with another program, under the
assumption that all other things match, such as thermostat/barostat, and
that the energies/forces have been validated to match sander, etc.

On Wed, Dec 17, 2008 at 9:04 AM, Thérèse Malliavin <terez_at_pasteur.fr> wrote:

> Dear Prof. Duke,
>
> I am sorry for having forced you to answer again to questions you already
> discussed in the past in the AMBER discussion list.
> The purpose of my mail is not to question the enormous work which was put
> into the pmemd development, and I am perfectly convinced that this program
> does bring certainly a lot to the AMBER package.
>
> But, I am concerned by the following problem. Molecular modeling studies
> are often based on the comparison of MD trajectories run with several
> conditions. In that way, two sander trajectories are recorded
> with different conditions and compared. If one trajectory is recorded with
> pmemd, and the other with sander, is the comparison still meaningful?
>
> Also, if one uses an additional trajectory recorded by CHARMM, GROMACS or
> NAMD with the AMBER force-field, will the pmemd trajectory be "closer" to
> the sander trajectory than the CHARMM, GROMACS or NAMD trajectory?
>
> If two trajectories are recorded with pmemd and sander starting from the
> same input, should we consider that they are no more different than
> two trajectories recorded with the same program (sander or pmemd) but using
> different initial velocities?
>
> Another question is: let one suppose that a trajectory was recorded using
> alternatively sander and pmemd for different time intervals, in the
> following way: some ns with pmemd, then restart with keeping velocities
> and then additional ns with sander. Should the complete trajectory obtained
> with these different interval be considered as an "homogeneous" trajectory
> which can be analyzed as a whole?
>
> I am sorry for insisting on these questions, but they are important for me
> in order to plan future calculations. I hope that I do not waste too much
> your time. Also, I realize that it is probably difficult to answer these
> questions, except by doing tests on each studied system, but I am just
> interested to read your opinion about these points.
>
> Best regards,
>
> Therese Malliavin
> Unite de Bioinformatique Structurale
> Institut Pasteur, Paris
> France
>
> On Tue, 16 Dec 2008, Robert Duke wrote:
>
> Okay, this has been discussed a lot. PMEMD should replicate sander
>> results for a couple of hundred steps at least, unless you have an
>> unbelievably bad starting configuration with a couple of atoms on top of
>> each other (in which case some of the force gradients are huge and the
>> simulation is bad anyway). However, the thing with MD is that there are on
>> the order of millions, if not billions, of calculations per step, including
>> additions, and the thing about addition of floating point numbers on
>> computers is that it is not truly associative - the order in which the
>> additions are performed DOES matter, due to truncation in the floating point
>> representation of the number. So what this means is that if you have an
>> algorithm that is different AT ALL, even in logically insignificant ways,
>> there will be a rounding error, and due to the nature of MD, this rounding
>> error will rather quickly grow. The main sources of difference between
>> pmemd and sander are probably the following: 1) a different splining
>> function for the erf() function in pmemd for some implementations (there is
>> an optimization, and pmemd is actually more accurate than sander), 2)
>> workload distribution differences running in parallel (which effect which
>> force additions will occur with net-limited precision of a 64 bit floating
>> point number), and 3) differences in the order of force additions arising
>> from differences in calculation and communication order. The thing to note
>> about rounding error - we are talking about a loss in precision down around
>> 1e-17 I believe - rather small. Now, the erf() splining errors are probably
>> closer to 1e-11 - probably the lowest precision transcendental we have, but
>> the other transcendental functions are probably between these two numbers in
>> precision (rough guess, have not looked recently, and it will be
>> machine-dependent). Now all this junk does not really matter, because your
>> calculation is probably off by at least 1e-5 (actually much worse) based on
>> precision of forcefield parameterization, the fact that coulomb's law does
>> not really get electrostatics just right, the fact that (substitute here the
>> next force term generator) just right, ... And the standard justification
>> for not being disturbed by all this - the different errors just mean that
>> you sample different parts of phase space, and if you run long enough, you
>> will get it all (this last point is why I have labored so long to make pmemd
>> fast). Run your system on some other software and you will see some more
>> dramatic differences in phase space sampling... Heck, just change the
>> cutoffs a bit, the fft grid densities, etc. etc. etc. I have gone on-and-on
>> about this stuff for the last several years on the amber reflector (see
>> ambermd.org for links), probably hitting different high and low points -
>> perhaps worth going back to look over, if you want the complete discussion.
>> I always jump on these questions, but am sort-of answering for Ross here
>> because I am 3 hrs closer to Europe and he is hopefully still asleep ;-)
>> Regards - Bob Duke
>>
>> ----- Original Message ----- From: "Thérèse Malliavin" <terez_at_pasteur.fr>
>> To: <amber_at_scripps.edu>
>> Sent: Tuesday, December 16, 2008 7:57 AM
>> Subject: RE: AMBER: launching a job works with sander.MPI and fail with
>> pmemd.MPI
>>
>>
>> Hi Ross,
>>
>> Thank you for your mail. Finally, I tried to use AMBER 10 in place of
>> AMBER 9, and pmemd runs without any problem. Now, I have another naive
>> question. I already realized that pmemd runs significantly faster than
>> sander even on 4 processors. But, if I compare the results obtained
>> by sander and pmemd starting from the same system, as for example the
>> total energy, the two runs seem not to be so much correlated. So, I would
>> like to know whether we have to expect that pmemd or sander should produce
>> the same numbers if the runs start from the same system. The
>> differences observed come probably from a different architecture of the
>> two programs, could you please tell me little bit more about that?
>>
>> Thank you for your help,
>>
>> Best regards,
>>
>> Therese
>>
>> On Mon, 15 Dec 2008, Ross Walker wrote:
>>
>> Hi Therese,
>>>
>>> First thing to check. PMEMD when built in parallel (which I assume you
>>> did)
>>> is called pmemd, not pmemd.MPI. Hence you should be getting a file not
>>> found
>>> error - which in parallel may be masking itself as a lamboot failure.
>>>
>>> Also I would make sure you do the following to run cleanly in your
>>> script:
>>>
>>> export AMBERHOME=/foo/bar/amber10
>>> lamboot
>>> mpirun -np 4 $AMBERHOME/exe/pmemd -O -i ...
>>> lamhalt
>>>
>>> Then you can nohup the entire script. You should probably make sure you
>>> kill
>>> any existing lambood or lamd instances on your machine first though since
>>> some will probably be left over from earlier runs. You should also make
>>> sure
>>> that pmemd was built with the same version of lam as your mpirun refers
>>> to.
>>> Makes sure you run the test cases:
>>>
>>> export DO_PARALLEL='mpirun -np 4'
>>> lamboot
>>> cd $AMBERHOME/test/
>>> make test.pmemd
>>> lamhalt
>>>
>>> Good luck,
>>> Ross
>>>
>>> -----Original Message-----
>>>> From: owner-amber_at_scripps.edu [mailto:owner-amber_at_scripps.edu] On
>>>> Behalf
>>>> Of Thérèse Malliavin
>>>> Sent: Monday, December 15, 2008 6:42 AM
>>>> To: amber_at_scripps.edu
>>>> Cc: terez_at_pasteur.fr
>>>> Subject: AMBER: launching a job works with sander.MPI and fail with
>>>> pmemd.MPI
>>>>
>>>> Dear AMBER Netters,
>>>>
>>>> I have a question about the use of PMEMD. It is probably a trivial
>>>> question, but, as I did not find an answer neither on the Web pages
>>>> neither in the manuals, I am asking it to you.
>>>>
>>>> I am doing the parallel calculations with sander.MPI using a lamd deamon
>>>> and the command nohup to launch the job, so I am doing:
>>>>
>>>> . /Bis/shared/centos-3_x86_64/etc/custom.d/amber9_intel8.1_lam-
>>>> 7.1.2_intel-8.1.sh
>>>> lamboot
>>>>
>>>> before starting the AMBER calculations. The typical command line for
>>>> sander.MPI is then:
>>>>
>>>> mpirun -np 4 ${AMBERHOME}/exe/sander.MPI -O -i mdr1.in -o mdr1.out -inf
>>>> mdr1.inf -x mdr1.crd -c eq7.rst -p prmtop -r mdr1.rst
>>>>
>>>> But, if I replace in the command line sander.MPI by pmemd.MPI:
>>>>
>>>> mpirun -np 4 ${AMBERHOME}/exe/pmemd.MPI -O -i mdr1.in -o mdr1.out -inf
>>>> mdr1.inf -x mdr1.crd -c eq7.rst -p prmtop -r mdr1.rst
>>>>
>>>> I get an error saying that lamboot was not started.
>>>>
>>>> I am trying to do these calculation on an 64 bits 8-proc Linux machine,
>>>> running under centos-3. The lam used is the version 7.1.2_intel-8.1.
>>>>
>>>> Also, I am only using features which should exist in PMEMD according to
>>>> the AMBER manual.
>>>>
>>>> Do you have any idea what I could check or what to find information to
>>>> fix
>>>> this problem?
>>>>
>>>> Thank you in abvance for your help,
>>>>
>>>> Therese Malliavin
>>>> Unite de Bioinformatique Structurale
>>>> Institut Pasteur, Paris
>>>> France
>>>>
>>>> -----------------------------------------------------------------------
>>>> The AMBER Mail Reflector
>>>> To post, send mail to amber_at_scripps.edu
>>>> To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
>>>> to majordomo_at_scripps.edu
>>>>
>>>
>>> -----------------------------------------------------------------------
>>> The AMBER Mail Reflector
>>> To post, send mail to amber_at_scripps.edu
>>> To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
>>> to majordomo_at_scripps.edu
>>>
>>>
>> -----------------------------------------------------------------------
>> The AMBER Mail Reflector
>> To post, send mail to amber_at_scripps.edu
>> To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
>> to majordomo_at_scripps.edu
>>
>

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
      to majordomo_at_scripps.edu