AMBER Archive (2009)

Subject: RE: [AMBER] 60 giga output file .mdcrd file

From: Ross Walker (ross_at_rosswalker.co.uk)
Date: Mon Nov 02 2009 - 17:36:48 CST

Hi Silvia,

This sounds normal to me. Currently ptraj goes through and completely checks
the mdcrd file before it processes it. If you figure that your system can do
maybe 10MB per second sustained then just to run through a 60GB file without
doing any processing will be 1.7 hours. Try just creating a copy of the file
and this will give you an idea of the speed of your system, it will probably
take > 30 mins just to copy. Then figure at least twice this for ptraj to
process it for a basic strip command.

Options are to 1) Get yourself a faster system, probably with multiple
striped raid disks that will allow you to approach something like 120Meg a
second.

2) rerun your simulation and do not save to the mdcrd file as often.

3) break up your mdcrd file into chunks or use ptraj to reduce the number of
frames. This of course will still require you to run through it once.

AmberTools 1.3 will be released soon and this includes several improvements
for ptraj when working with large files, such as removing the initial check
etc. It can also do parallel i/o so you can try to take advantage of
parallel file systems such as GPFS or Lustre. In short though, if you are
running this on your desktop, rather than a well build supercomputer
connected by fiber to a true (and well designed) parallel file system then
you are unfortunately up against the hardware limitations of your system.

If you are using this on some NFS server that you are connecting to remotely
then I would just take a weeks vacation while you wait for it to process.

Sorry I can't help much more but unfortunately disk speeds have been largely
flat for the last 8 years while capacity has ballooned.

All the best
Ross

> -----Original Message-----
> From: amber-bounces_at_ambermd.org [mailto:amber-bounces_at_ambermd.org] On
> Behalf Of Silvia Carlotto
> Sent: Monday, November 02, 2009 9:42 AM
> To: amber_at_ambermd.org
> Subject: [AMBER] 60 giga output file .mdcrd file
>
> Dear user,
>
> I generate a.crd file of 60 Giga (a protein in a water box, 10 ns, n
> tot
> atom ca. 8000).
>
> I am using ptraj to strip water but
>
> the ptraj command is over 30 minutes on
>
> PTRAJ : trajin XXX.mdcrd
>
> checking coordinates : XXX.mdcrd
> Is it a normal?
>
> Is it possible to manupulate with ptraj a file of these dimensions?
>
> I have no other ideas to strip water to generate a movie with vmd.
>
> thanks to help
> _______________________________________________
> AMBER mailing list
> AMBER_at_ambermd.org
> http://lists.ambermd.org/mailman/listinfo/amber

_______________________________________________
AMBER mailing list
AMBER_at_ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber