AMBER Archive (2008)

Subject: Re: AMBER: Compiling Amber10 PMEMD with MVAPICH, InfiniBand support

From: Robert Duke (rduke_at_email.unc.edu)
Date: Wed Sep 24 2008 - 14:39:30 CDT

Looks to me like you should modify config.h as follows:

MPI_LIBDIR2 = /opt/ofed/1.2.5.5/lib64
MPI_HOME = /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9

modify the MPI_LIBS line to list the
libraries -libverbs -libumad -libcommon -lpthread after -L(MPI_LIBDIR2)
modify the MPI_LIBS line to list the libraries -lmpichfarg -lmpich
after -L(MPI_LIBDIR)

Alternatively, it may be most simple to use mpif77 in place of ifort in the
config.h file, and drop all the MPI_LIB* stuff; I have typically not done
this because I want to absolutely control what is happening, but if you find
the correct mpif77 for your installation, and the installation has not been
dinked with ("customized", perhaps not entirely correctly), that may prove
simpler. I think Ross has released some more detailed directions on this
option on the reflector; I may eventually give in and make this an easily
available option in configure (there is a bunch of other stuff going on in
the -link_info output here that you may or may not actually need; I
typically have no way of knowing without access to the machine).

Best regards - Bob Duke

----- Original Message -----
From: "Pablo Englebienne" <pablo.englebienne_at_mcgill.ca>
To: <amber_at_scripps.edu>
Sent: Wednesday, September 24, 2008 3:04 PM
Subject: AMBER: Compiling Amber10 PMEMD with MVAPICH, InfiniBand support

> I'm trying to compile PMEMD from Amber10 (fully patched) on an
> InfiniBand-connected cluster of Pentium4 Xeon nodes (Mammouth-Mp @ RQCHP,
> if anyone has experience on it), but with little luck.
>
> The short question is: how can I find where are the "Infiniband libraries"
> required for compilation of PMEMD?
>
>
> This is the walkthrough of what I did:
>
> Based on the info on the README file, this is the output of
> 'mpif77 -link_info':
> [englebie_at_ip05 pmemd]$ mpif77 -link_info
> ifort -fPIC -L/opt/ofed/1.2.5.5/lib64 -D__INTEL_COMPILER -Wl,-rpath-link -Wl,/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/shared
> -L/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/shared -L/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib
> -lmpichfarg -lmpich -L/opt/ofed/1.2.5.5/lib64 -libverbs -libumad -libcommon
> -lpthread -D__INTEL_COMPILER -lpthread -lrt
>
> This is the command I'm using to configure the compilation:
> [englebie_at_ip05 pmemd]$ ./configure linux_em64t ifort mvapich
> Intel ifort compiler found; version information:
> Version 10.1
> configure assumes Intel MKL is installed in /opt/intel/mkl/10.0.3.020
> File config_data/fft.pubfft being used...
> File config_data/linux_em64t.ifort being used...
> File config_data/interconnect.mvapich being used...
> Please enter directory where mvapich mpi files are installed:
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9
> Please enter name of directory where Infiniband libraries are installed:
> /opt/ofed/1.2.5.5
> PMEMD Configurate successfully completed.
>
> When compiling, there is a long list of errors like the following ones:
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitall.o): In
> function
> `MPI_Waitall':
> (.text+0x8f): undefined reference to `MPID_SendComplete'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitall.o): In
> function
> `MPI_Waitall':
> (.text+0x128): undefined reference to `MPID_SendComplete'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitany.o): In
> function
> `MPI_Waitany':
> (.text+0x131): undefined reference to `MPID_SendIcomplete'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitany.o): In
> function
> `MPI_Waitany':
> (.text+0x383): undefined reference to `MPID_SendIcomplete'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x362): undefined reference to `ibv_dereg_mr'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x384): undefined reference to `ibv_dereg_mr'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x475): undefined reference to `ibv_destroy_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x520): undefined reference to `ibv_destroy_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x607): undefined reference to `ibv_destroy_srq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x622): undefined reference to `ibv_destroy_cq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x650): undefined reference to `ibv_destroy_comp_channel'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x719): undefined reference to `ibv_dealloc_pd'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Finalize':
> (.text+0x734): undefined reference to `ibv_close_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1a53): undefined reference to `ibv_get_device_list'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1ac2): undefined reference to `ibv_get_device_name'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1aee): undefined reference to `ibv_open_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1b31): undefined reference to `ibv_query_port'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1b43): undefined reference to `ibv_query_port'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1b9e): undefined reference to `ibv_close_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1c29): undefined reference to `ibv_open_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1c53): undefined reference to `ibv_query_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1c6e): undefined reference to `ibv_alloc_pd'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1cb7): undefined reference to `ibv_query_port'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x1d8e): undefined reference to `ibv_free_device_list'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x2246): undefined reference to `ibv_query_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x2283): undefined reference to `ibv_query_pkey'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x2de1): undefined reference to `ibv_modify_srq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x3407): undefined reference to `ibv_modify_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x351c): undefined reference to `ibv_modify_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `MPID_VIA_Init':
> (.text+0x39be): undefined reference to `ibv_modify_srq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x4eb6): undefined reference to `ibv_create_comp_channel'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x4ef2): undefined reference to `ibv_create_cq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x4f44): undefined reference to `ibv_create_cq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x4fe0): undefined reference to `ibv_create_srq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x510e): undefined reference to `ibv_create_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x51d4): undefined reference to `ibv_query_device'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x521e): undefined reference to `ibv_query_pkey'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): In
> function
> `ib_init':
> (.text+0x5352): undefined reference to `ibv_modify_qp'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `async_thread':
> (.text+0x40): undefined reference to `ibv_get_async_event'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `async_thread':
> (.text+0x223): undefined reference to `ibv_modify_srq'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `async_thread':
> (.text+0x32d): undefined reference to `ibv_ack_async_event'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> function `MPID_DeviceCheck':
> (.text+0x482): undefined reference to `odu_test_new_connection'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `MPID_DeviceCheck':
> (.text+0x663): undefined reference to `odu_test_new_connection'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `MPID_DeviceCheck':
> (.text+0x703): undefined reference to `ibv_get_cq_event'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `MPID_DeviceCheck':
> (.text+0x740): undefined reference to `ibv_ack_cq_events'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `MPID_DeviceCheck':
> (.text+0xbe4): undefined reference to `cm_process_queue'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): In
> functio
> n `MPID_DeviceCheck':
> (.text+0xfe2): undefined reference to `cm_process_queue'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viapriv.o): In
> function
> `viadev_ext_sendq_send':
> (.text+0x155): undefined reference to `prepare_coalesced_pkt'
> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viapriv.o): In
> function
> `deregister_memory':
> (.text+0x15c6): undefined reference to `ibv_dereg_mr'
>
> Any suggestions?
>
> --
> Pablo Englebienne, PhD Student, Moitessier research group
> Department of Chemistry, McGill University
> 801 Sherbrooke Street West
> Otto Maass bldg, room 206
> H3A 2K6 Montréal, Québec, Canada
> Tel (514) 398-5501 Fax (514) 398-2382
>
> "Progress is made by lazy men looking for easier ways to do things." -
> Robert A. Heinlein
>
>
> -----------------------------------------------------------------------
> The AMBER Mail Reflector
> To post, send mail to amber_at_scripps.edu
> To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
> to majordomo_at_scripps.edu
>

-----------------------------------------------------------------------
The AMBER Mail Reflector
To post, send mail to amber_at_scripps.edu
To unsubscribe, send "unsubscribe amber" (in the *body* of the email)
      to majordomo_at_scripps.edu