AMBER Archive (2004)

Subject: Re: AMBER: Additional note on ld.so.conf vs. LD_LIBRARY_PATH

From: Robert Duke (rduke_at_email.unc.edu)
Date: Tue Apr 27 2004 - 17:36:51 CDT

Dave -
Actually, there are logical explanations for all the puzzles that fit within
the framework I posted earlier. The reason you were able to get rid of
stacksize problems by getting rid of -static is because it is a different
stacksize problem. If you run the jac benchmark (around 23K atoms), you
actually don't have to do a "limit stacksize unlimited" on a dual processor
run. This is the test you are using, I believe. If you run the factor ix
benchmark (around 91K atoms), you DO have to do a "limit stacksize
unlimited", or all the junk on the stack in the reciprocal space code will
get you. This benchmark is always my first stress test. There are good
performance reasons to put stuff on the stack, relating to page reuse. The
intel fortran compiler has been more conservative about actually putting big
stuff on the stack in the past; apparently it is getting more aggressive.
Now, as far as -static goes, that is a different stacksize problem. I may
not have the details exactly right, but basically redhat (at least) has
screwed up static threads libraries in that they use fixed size stacks of
fairly small size, and if you link statically to threads stuff, you are very
apt to overflow some thread or another's stack. I am incredulous about all
this junk, but there you are. So:
1) you shouldn't use -static from redhat 8 forward, at least.
2) if your problem is small, you may get away without doing a "limit
stacksize unlimited" in .cshrc. I don't recommend it.
Regards - Bob

----- Original Message -----
From: "David E. Konerding DSD Staff" <dekonerding_at_lbl.gov>
To: <amber_at_scripps.edu>
Sent: Tuesday, April 27, 2004 6:11 PM
Subject: Re: AMBER: Additional note on ld.so.conf vs. LD_LIBRARY_PATH

> Robert Duke wrote:
>
> >Folks -
> >I should have included this in my previous posting. All my comments
about
> >the caveats assume that you are set up to use rsh or ssh as your
mechanism
> >of starting processes (what I do using ch_p4 and mpirun on my machines).
If
> >you use ch_p4mpd and the mpd demon on your systems, then environments
should
> >not be inherited, and it is definitely preferable to use ld.so.conf as
> >opposed to LD_LIBRARY_PATH. Dave K. did not report problems with limits.
> >Regards - Bob Duke
> >
> >
>
> Actually, I did have the problem with stack size limits, and they
> magically disappeared when I stopped using -static. But, the program
> crashes for another reason (obscure one,
> not easy to debug because IFC8's -g flag causes PMEMD to fail to
> compile. Bob- have you seen this as well?) if I compile with -static.
>
> For those of you using mpd, if you do not want to modify
> /etc/ld.so.conf, use an mpirun command line like:
>
> mpirun -wdir `pwd` -m machines.all -np 8 -g 2 ./pmemd -O -i mdin -p
> prmtop -c inpcrd.equil -o mdout -MPDENV-
> LD_LIBRARY_PATH=/home/dsd/intel_compiler/compiler80/lib
>
> The -MPDENV- part at the end sets the LD_LIBRARY_PATH in the child
> pmemds running within the MPI environment.
>
> -----------------------------------------------------------------------
> The AMBER Mail Reflector
> To post, send mail to amber_at_scripps.edu
> To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu
>

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu