AMBER Archive (2007)

Subject: Re: AMBER: AMBER8 compiler with intel 9.1 compiler

From: Peter Z. Qin (pzq_at_usc.edu)
Date: Wed May 16 2007 - 15:37:53 CDT

----- Original Message -----
From: "Scott Brozell" <sbrozell_at_scripps.edu>
To: <amber_at_scripps.edu>
Sent: Wednesday, May 16, 2007 11:42 AM
Subject: Re: AMBER: AMBER8 compiler with intel 9.1 compiler

> Hi,
>
> On Tue, 15 May 2007, Peter Z. Qin wrote:
>
>> I am running AMBER8 on our school mainframe (Linux cluster). Recent
>> changes
>> at this facility forces me to re-compile AMBER8 with an Intel 9.1
>> compiler.
>> While my previous compilation of AMBER8 using an Intel 8.1 compiler
>> passed
>> all tests, the re-compilation (both the serial and the parallel version)
>> fails testing on several cases. A typical example, generated from
>> test/cytosine/Run.cytosine, is:
>>
>> 87,88c87,88
>> < NSTEP = 1 TIME(PS) = 5420.002 TEMP(K) = 301.21
>> PRESS -5.8
>> < Etot = -608.3990 EKtot = 1063.0604 EPtot
>> -1671.459
>> ---
>> > NSTEP = 1 TIME(PS) = 5420.002 TEMP(K) = 301.68
>> > PRESS -651.0
>> > Etot = -606.7460 EKtot = 1064.7135 EPtot
>> > -1671.459
>> 92c92
>> < EKCMT = 176.9122 VIRIAL = 180.9720 VOLUME =
>> 32637.922
>> ---
>> > EKCMT = 176.9122 VIRIAL = 635.6568 VOLUME =
>> > 32637.922
>> 94c94
>> < Ewald error estimate: 0.1512E-1
>> ---
>> > Ewald error estimate: 0.6367E+1
>>
>> My question is whether these differences are significant? and iff so,
>> whar
>> could be the fix?
>
> These are sig. diffs.

I guess that is the case, but it is still good to confirm. Thanks.

> What is the outpout of ifort -V ?

[pzq_at_hpc-login1 src]$ ifort -V
Intel(R) Fortran Compiler for 32-bit applications, Version 9.1 Build
20060323Z Package ID: l_fc_p_9.1.032
Copyright (C) 1985-2006 Intel Corporation. All rights reserved.
FOR NON-COMMERCIAL USE ONLY

> What config file was used ?

[pzq_at_hpc-login1 src]$ more config.h
#==============================================================================
# AMBER Makefile configuration for compiler/architecture: ifort
# Generated via command: ./configure ifort
#
# Configuration script written mainly by Joe Krahn, Scott Brozell, and
# Dave Case, with contributions from lots of people.
#==============================================================================

#------------------------------------------------------------------------------
# Main AMBER source root directory
#------------------------------------------------------------------------------
AMBER_SRC=/home/rcf-12/pzq/MD/amber8/src

#------------------------------------------------------------------------------
# AMBERBUILDFLAGS provides a hook into the build process for installers;
# for example, to build debug versions of the amber programs
# make -e AMBERBUILDFLAGS="-DDEBUG -g"
#------------------------------------------------------------------------------
AMBERBUILDFLAGS=

#------------------------------------------------------------------------------
# LOCALFLAGS is intended for program specific modifications to the
# Fortran build process and may be modified by the program's local makefile
#------------------------------------------------------------------------------
LOCALFLAGS=

#------------------------------------------------------------------------------
# Availability and method of delivery of math and optional libraries
#------------------------------------------------------------------------------
USE_BLASLIB=$(SOURCE_COMPILED)
USE_LAPACKLIB=$(SOURCE_COMPILED)
USE_LMODLIB=$(LMOD_UNAVAILABLE)

#------------------------------------------------------------------------------
# C compiler
#------------------------------------------------------------------------------
CC= gcc
CPLUSPLUS=g++
ALTCC=gcc
CFLAGS=-O2 $(AMBERBUILDFLAGS)
ALTCFLAGS= $(AMBERBUILDFLAGS)
CPPFLAGS= $(AMBERBUILDFLAGS)

#------------------------------------------------------------------------------
# Fortran preprocessing and compiler.
# FPPFLAGS holds the main Fortran options, such as whether MPI is used.
#------------------------------------------------------------------------------
FPPFLAGS= -P -I$(AMBER_SRC)/include $(AMBERBUILDFLAGS)
FPP= cpp -traditional $(FPPFLAGS)
FC= ifort
FFLAGS= -w95 -mp1 -O0 $(LOCALFLAGS) $(AMBERBUILDFLAGS)
FOPTFLAGS= -w95 -mp1 -ip -O3 $(LOCALFLAGS) $(AMBERBUILDFLAGS)
FPP_PREFIX= _
FREEFORMAT_FLAG= -FR

#------------------------------------------------------------------------------
# Loader:
#------------------------------------------------------------------------------
LOAD= ifort $(LOCALFLAGS) $(AMBERBUILDFLAGS)
LOADCC= gcc $(LOCALFLAGS) $(AMBERBUILDFLAGS)
LOADLIB=
LOADPTRAJ= ifort -nofor_main $(LOCALFLAGS) $(AMBERBUILDFLAGS)
XHOME= /usr/X11R6

#------------------------------------------------------------------------------
# Other stuff:
#------------------------------------------------------------------------------
.SUFFIXES: .f90
SYSDIR=lib
AR=ar rv
M4=m4
RANLIB=ranlib
SFX=
MAKEDEPEND=$(AMBER_SRC)/../bin/amber_makedepend

# default rules for Fortran and C compilation:

.f.o: $<
        $(FPP) $< > $(FPP_PREFIX)$<
        $(FC) -c $(FFLAGS) -o $@ $(FPP_PREFIX)$<

.f90.o: $<
        $(FPP) $< > $(FPP_PREFIX)$<
        $(FC) -c $(FFLAGS) -o $@ $(FPP_PREFIX)$<

.c.o:
        $(CC) -c $(CFLAGS) $(CPPFLAGS) -o $@ $<

> What are the machine details ?

      hpc0001 hpc0256 256 Dual Dualcore AMD Opteron 2.0 GHz
      4GB Memory 60GB main large quick X4100 opteron x86_64 dualcore

more details at http://www.usc.edu/hpcc/systems/use-l-0

>
> The simplest fix is to use an older intel compiler.

I was told by the adminstrator that intel 8.1 is no longer support on the
operating system

> I stay away from the latest from intel because I have found their
> quality assurance to be lacking.
> On the machines I use the latest versions are
> ia32 intel-compilers-9.0-20050916
> ia64 intel-compilers-9.1-20070111
> but I have not built amber with these versions (using 8.1 instead).
> I could not quickly find details about which intel versions
> have been used to build Amber at the web site.
> Other posters may be helpful here.
>
> The likely cause is changes in code meaning due to compiler
> optimizations. If you cannot use an older ifort then one
> could reduce the optimization level. There are clues in the diffs
> above, and more info about the other failing tests might enable
> someone knowledgeable about Amber's software architecture
> to identify a small number of files that have been affected.
>

1. For the cytosine case, below is the full difference between the
cytosine.out.save (from 02/2004) and my cytosine.out. It seems the input
parameters for sander are different, which is consistent with compiler
optimization level. But how do I reduce the optimization?

[pzq_at_hpc-login1 cytosine]$ more full.dif.cytosine
6c6
< | Run on 02/05/2004 at 17:33:30 

---
> | Run on 05/15/2007 at 17:29:37
44c44
< | Flags: MPI
---
> | Flags:
60c60
< |     Real              170832
---
> |     Real              166979
62,64c62,64
< |     Integer           100948
< |     Max Pairs          26479
< |     Max Rstack         43100
---
> |     Integer           155255
> |     Max Pairs         423666
> |     Max Rstack        117560
66c66
< |       Total             2224 kbytes
---
> |       Total             4539 kbytes
131,136d130
< |  Atom division among processors:
< |         0      81     161     241     321     401     481     561
< |       641     721     801     881     961    1041    1121    1196
< |      1271
< |  Running AMBER/MPI version on   16 nodes
<
152c146
< | Local SIZE OF NONBOND LIST =       6576
---
> | Local SIZE OF NONBOND LIST =     100304
155,156c149,150
<  NSTEP =        1   TIME(PS) =    5420.002  TEMP(K) =   301.21  PRESS 
  -5.8
<  Etot   =      -608.3990  EKtot   =      1063.0604  EPtot 
 =     -1671.4594
---
>  NSTEP =        1   TIME(PS) =    5420.002  TEMP(K) =   301.68  PRESS 
>   -651.0
>  Etot   =      -606.7460  EKtot   =      1064.7135  EPtot 
>  =     -1671.4594
160c154
<  EKCMT  =       176.9122  VIRIAL  =       180.9720  VOLUME     = 
32637.9223
---
>  EKCMT  =       176.9122  VIRIAL  =       635.6568  VOLUME     = 
> 32637.9223
162c156
<  Ewald error estimate:   0.1512E-01
---
>  Ewald error estimate:   0.6367E+01
....

2.  The following tests failed in the intel 9.1 compilation but passed for 
the intel 8.1 compilation.  Could this give us a clue
cytosine, dhfr, dhfr.noshake,mtrx, trx.cpln, alp.

-peter 

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" to majordomo_at_scripps.edu