AMBER Archive (2006)

Subject: RE: AMBER: Problem with file size of sander outputs

From: Feng X Zhou (feng.zhou_at_abbott.com)
Date: Thu Feb 23 2006 - 11:42:22 CST

Dear Amber community:

Sorry for jumping off the topic a little bit.... But is it possible to use
better format for trajectory files in future versions of AMBER?

May not be such a bad idea to setup a mySQL database server and append each
trajectory frame as a table entry... this way the file size limit problem
maybe avoided and all past frames can be retrieved easily.... if a frame
happen to be bad and fail to commit, system can simply rollback and no
corruptions will occur. Also getting the analysis done maybe easier too - as
it now can use the features provided by relational database and SQL (such as
indexing, using the RAM efficiently, etc.)

In addition, given the stability of the database, results from multiple
simulations maybe deposited/retrieved into a central location from various
computers on the network. This may come handy if one has a lot of simualtions
run in parallel but want to combine them for analysis, or even just for
sharing to another research group? Seems compression of data is possible in
mySQL also...

Feng Zhou
UNIX admin/IT consultant
Abbott Labs

Wednesday, February 22, 2006 3:17 PM
To: <amber_at_scripps.edu>
cc:
From: "Ross Walker" <ross_at_rosswalker.co.uk>
Subject: RE: AMBER: Problem with file size of sander outputs

Dear JunJun,

Try editing the config.h file produced with your configure script and add
the following to the
AMBERBUILDFLAGS line:

-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE

Then do a make clean followed by make.

This 'may' depending on your system, compiler etc produce an executable that
can write trajectory files larger than 2GB. That said writing such large
trajectory files is NOT recommended. The reason for this is the larger the
file becomes the more likely it is to be corrupted. If a single frame in the
tajectory gets corrupted then you will have a great deal of trouble
extracting information beyond the corrupted frame. If you break your
simulation up into much smaller parts and one trajectory file gets corrupted
then you can simply re-run this small portion of the trajectory from the
restart file for that portion of the trajectory to recreate the mdcrd file.
Improvements to the trajectory file format in future releases of AMBER may
address this problem but for the moment there is no easy fix for corrupted
trajectories.

Splitting things up is much much much safer... When it comes to analysing
things it also makes things easier. For starters you can analyse several
trajectory files seamlessly in ptraj and VMD will allow you to load several
mdcrd files into a single prmtop (molecule). Havings things split up also
has the advantage that you can easily analyse just a small fraction of the
trajectory without having to load the whole thing.

Add this to the fact that if you copy files bigger than 2GB between 32 bit
and 64 bit machines over NFS shares or try to edit a file bigger than 2GB in
an editor (say vi) that does not have support for large files you can end up
truncating or corrupting the file fairly easily.

Thus unless you have good justification for wanting large coordinate files
you are better off just scripting your simulation into smaller pieces...

Just my 2c...

All the best
Ross

/\
\/
|\oss Walker

| Department of Molecular Biology TPC15 |
| The Scripps Research Institute |
| Tel: +1 858 784 8889 | EMail:- ross_at_rosswalker.co.uk |
| http://www.rosswalker.co.uk | PGP Key available on request |

Note: Electronic Mail is not secure, has no guarantee of delivery, may not
be read every day, and should not be used for urgent or sensitive issues.

> -----Original Message-----
> From: owner-amber_at_scripps.edu
> [mailto:owner-amber_at_scripps.edu] On Behalf Of JunJun Liu
> Sent: Wednesday, February 22, 2006 12:58
> To: amber_at_scripps.edu
> Subject: Re: AMBER: Problem with file size of sander outputs
>
> I have little experience about it, not related to sander but other
> program. At first I compiled that program using ifort9. The
> program died
> with "filesize limit exceeded " displayed when the produced
> file reached
> 2.1GB. Intel announced that intel compiler automatically
> enable "large
> file support", but I don't know why my program still has the 2GB
> limitation.
>
> Then I use PGI compiler with -Mlfs flag for the compiliation. The
> re-compiled program can now overcome the 2GB limitation.
> Maybe you also
> can try using PGI compiler with -Mlfs flag to re-compile
> sander. Hope this
> helps!
>
> On Wed, 22 Feb 2006 05:49:49 -0500, german
> <gsciaini_at_qi.fcen.uba.ar> wrote:
>
> > Try to divide your calculation into different output files
> restarting
> > them with NTX=5, and IREST=1...
> >
> > Good look
> > G.
> >
> >
> >
> > At 11:54 p.m. 21/02/2006 -0800, you wrote:
> >
> >> Dear amber community,
> >>
> >> I have been running a PME explicit solvent molecular
> >> dynamics for a 20 KDa protein and its ligand using the
> >> sander module of AMBER8.
> >> I set nstlim=40000 and ntpr=10. After a couple of
> >> hours , my MD run stopped at nstep =9970 giving me the
> >> following error message:
> >>
> >> Filesize limit exceeded
> >>
> >> My trajectory file size has reached to 2 GB, the MD
> >> output is 666 Kb and the coordinate output is 6.6 MB.
> >>
> >> I believe I will have to recompile sander changing
> >> some codes to increase the file size in sander. Could
> >> someone please let me know what codes in sander should
> >> be modified?
> >>
> >> cheers,
> >>
> >> jenk.
> >>
> >>
> >>
> >>
> >>
> >>
> >> --- Ross Walker <ross_at_rosswalker.co.uk> wrote:
> >>
> >> >
> >> > > ./configure -mpi ifort
> >> > > make parallel
> >> >
> >> > -mpi means native MPI libraries as provided by the
> >> > operating system. This is
> >> > the option to use for things like an SGI altix. For
> >> > a regular PC running
> >> > redhat you need to provide an mpi installation
> >> > either MPICH or LAM. This MPI
> >> > installation needs to be compiled with the same
> >> > compiler as you are using
> >> > for compiling Amber. Hence the lam that is often
> >> > installed by default with
> >> > redhat will not work as it is compiled with GCC.
> >> >
> >> > So, download either Lam or MPICH, compile it with
> >> > ifort and chech your PATH
> >> > is correct so that the correct installation is being
> >> > used.
> >> >
> >> > E.g. which lamboot
> >> >
> >> > should return /usr/local/lam/lamboot (or similar)
> >> > rather than /usr/bin/lamboot
> >> >
> >> > Once this is done create a config.h for the lam or
> >> > mpich:
> >> >
> >> > export LAM_HOME=/usr/local/lam..... (or where you
> >> > installed it)
> >> > ./configure -p4 -lam ifort
> >> > or
> >> > export MPICH_HOME=/usr/local/mpich.....
> >> > ./configure -p4 -mpich ifort
> >> >
> >> > Then:
> >> >
> >> > make clean
> >> > make parallel
> >> >
> >> > You should then not get the undefined references you
> >> > are seeing.
> >> >
> >> > All the best
> >> > Ross
> >> >
> >> > /\
> >> > \/
> >> > |\oss Walker
> >> >
> >> > | Department of Molecular Biology TPC15 |
> >> > | The Scripps Research Institute |
> >> > | Tel: +1 858 784 8889 | EMail:-
> >> > ross_at_rosswalker.co.uk |
> >> > | http://www.rosswalker.co.uk | PGP Key available on
> >> > request |
> >> >
> >> > Note: Electronic Mail is not secure, has no
> >> > guarantee of delivery, may not
> >> > be read every day, and should not be used for urgent
> >> > or sensitive issues.
> >> >
> >> >
> >> >
>To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu