AMBER Archive (2008)

Subject: Re: AMBER: Compiling Amber10 PMEMD with MVAPICH, InfiniBand support

From: Pablo Englebienne (pablo.englebienne_at_mcgill.ca)
Date: Thu Sep 25 2008 - 14:07:48 CDT


Thanks a lot Bob, that indeed made it work.

After a consultation with an analyst at the RQCHP supercomputer, here's
the modifications to the config.h that worked for me, I'll post them for
completeness:

MPI_HOME = /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9
MPI_LIBDIR2 = /opt/ofed/1.2.5.5/lib64

MPI_LIBS = -L$(MPI_LIBDIR) -lmpichf90nc -lmpichfarg -lmpich
-L$(MPI_LIBDIR2) -libverbs -libumad -libcommon -lpthread

Which is basically the same as suggested in the previous message, plus
the '-lmpichf90nc' flag which I'm unsure about its purpose...

Thanks again!

-- 
Pablo Englebienne, PhD Student, Moitessier research group
Department of Chemistry, McGill University
801 Sherbrooke Street West
Otto Maass bldg, room 206
H3A 2K6 Montréal, Québec, Canada
Tel (514) 398-5501    Fax (514) 398-2382

"Progress is made by lazy men looking for easier ways to do things." - Robert A. Heinlein

Robert Duke wrote: > Looks to me like you should modify config.h as follows: > > MPI_LIBDIR2 = /opt/ofed/1.2.5.5/lib64 > MPI_HOME = /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9 > > modify the MPI_LIBS line to list the libraries -libverbs -libumad > -libcommon -lpthread after -L(MPI_LIBDIR2) > modify the MPI_LIBS line to list the libraries -lmpichfarg -lmpich > after -L(MPI_LIBDIR) > > Alternatively, it may be most simple to use mpif77 in place of ifort > in the config.h file, and drop all the MPI_LIB* stuff; I have > typically not done this because I want to absolutely control what is > happening, but if you find the correct mpif77 for your installation, > and the installation has not been dinked with ("customized", perhaps > not entirely correctly), that may prove simpler. I think Ross has > released some more detailed directions on this option on the > reflector; I may eventually give in and make this an easily available > option in configure (there is a bunch of other stuff going on in the > -link_info output here that you may or may not actually need; I > typically have no way of knowing without access to the machine). > > Best regards - Bob Duke > > ----- Original Message ----- From: "Pablo Englebienne" > <pablo.englebienne_at_mcgill.ca> > To: <amber_at_scripps.edu> > Sent: Wednesday, September 24, 2008 3:04 PM > Subject: AMBER: Compiling Amber10 PMEMD with MVAPICH, InfiniBand support > > >> I'm trying to compile PMEMD from Amber10 (fully patched) on an >> InfiniBand-connected cluster of Pentium4 Xeon nodes (Mammouth-Mp @ >> RQCHP, if anyone has experience on it), but with little luck. >> >> The short question is: how can I find where are the "Infiniband >> libraries" required for compilation of PMEMD? >> >> >> This is the walkthrough of what I did: >> >> Based on the info on the README file, this is the output of 'mpif77 >> -link_info': >> [englebie_at_ip05 pmemd]$ mpif77 -link_info >> ifort -fPIC -L/opt/ofed/1.2.5.5/lib64 -D__INTEL_COMPILER >> -Wl,-rpath-link >> -Wl,/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/shared >> -L/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/shared >> -L/opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib -lmpichfarg -lmpich >> -L/opt/ofed/1.2.5.5/lib64 -libverbs -libumad -libcommon -lpthread >> -D__INTEL_COMPILER -lpthread -lrt >> >> This is the command I'm using to configure the compilation: >> [englebie_at_ip05 pmemd]$ ./configure linux_em64t ifort mvapich >> Intel ifort compiler found; version information: >> Version 10.1 >> configure assumes Intel MKL is installed in /opt/intel/mkl/10.0.3.020 >> File config_data/fft.pubfft being used... >> File config_data/linux_em64t.ifort being used... >> File config_data/interconnect.mvapich being used... >> Please enter directory where mvapich mpi files are installed: >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9 >> Please enter name of directory where Infiniband libraries are installed: >> /opt/ofed/1.2.5.5 >> PMEMD Configurate successfully completed. >> >> When compiling, there is a long list of errors like the following ones: >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitall.o): >> In function >> `MPI_Waitall': >> (.text+0x8f): undefined reference to `MPID_SendComplete' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitall.o): >> In function >> `MPI_Waitall': >> (.text+0x128): undefined reference to `MPID_SendComplete' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitany.o): >> In function >> `MPI_Waitany': >> (.text+0x131): undefined reference to `MPID_SendIcomplete' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(waitany.o): >> In function >> `MPI_Waitany': >> (.text+0x383): undefined reference to `MPID_SendIcomplete' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x362): undefined reference to `ibv_dereg_mr' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x384): undefined reference to `ibv_dereg_mr' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x475): undefined reference to `ibv_destroy_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x520): undefined reference to `ibv_destroy_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x607): undefined reference to `ibv_destroy_srq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x622): undefined reference to `ibv_destroy_cq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x650): undefined reference to `ibv_destroy_comp_channel' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x719): undefined reference to `ibv_dealloc_pd' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Finalize': >> (.text+0x734): undefined reference to `ibv_close_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1a53): undefined reference to `ibv_get_device_list' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1ac2): undefined reference to `ibv_get_device_name' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1aee): undefined reference to `ibv_open_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1b31): undefined reference to `ibv_query_port' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1b43): undefined reference to `ibv_query_port' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1b9e): undefined reference to `ibv_close_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1c29): undefined reference to `ibv_open_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1c53): undefined reference to `ibv_query_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1c6e): undefined reference to `ibv_alloc_pd' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1cb7): undefined reference to `ibv_query_port' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x1d8e): undefined reference to `ibv_free_device_list' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x2246): undefined reference to `ibv_query_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x2283): undefined reference to `ibv_query_pkey' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x2de1): undefined reference to `ibv_modify_srq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x3407): undefined reference to `ibv_modify_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x351c): undefined reference to `ibv_modify_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `MPID_VIA_Init': >> (.text+0x39be): undefined reference to `ibv_modify_srq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x4eb6): undefined reference to `ibv_create_comp_channel' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x4ef2): undefined reference to `ibv_create_cq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x4f44): undefined reference to `ibv_create_cq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x4fe0): undefined reference to `ibv_create_srq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x510e): undefined reference to `ibv_create_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x51d4): undefined reference to `ibv_query_device' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x521e): undefined reference to `ibv_query_pkey' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viainit.o): >> In function >> `ib_init': >> (.text+0x5352): undefined reference to `ibv_modify_qp' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `async_thread': >> (.text+0x40): undefined reference to `ibv_get_async_event' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `async_thread': >> (.text+0x223): undefined reference to `ibv_modify_srq' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `async_thread': >> (.text+0x32d): undefined reference to `ibv_ack_async_event' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In function `MPID_DeviceCheck': >> (.text+0x482): undefined reference to `odu_test_new_connection' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `MPID_DeviceCheck': >> (.text+0x663): undefined reference to `odu_test_new_connection' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `MPID_DeviceCheck': >> (.text+0x703): undefined reference to `ibv_get_cq_event' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `MPID_DeviceCheck': >> (.text+0x740): undefined reference to `ibv_ack_cq_events' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `MPID_DeviceCheck': >> (.text+0xbe4): undefined reference to `cm_process_queue' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viacheck.o): >> In functio >> n `MPID_DeviceCheck': >> (.text+0xfe2): undefined reference to `cm_process_queue' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viapriv.o): >> In function >> `viadev_ext_sendq_send': >> (.text+0x155): undefined reference to `prepare_coalesced_pkt' >> /opt/ofed/1.2.5.5/mpi/intel/mvapich-0.9.9/lib/libmpich.a(viapriv.o): >> In function >> `deregister_memory': >> (.text+0x15c6): undefined reference to `ibv_dereg_mr' >> >> Any suggestions? >> >> -- >> Pablo Englebienne, PhD Student, Moitessier research group >> Department of Chemistry, McGill University >> 801 Sherbrooke Street West >> Otto Maass bldg, room 206 >> H3A 2K6 Montréal, Québec, Canada >> Tel (514) 398-5501 Fax (514) 398-2382 >> >> "Progress is made by lazy men looking for easier ways to do things." >> - Robert A. Heinlein >> >> >> ----------------------------------------------------------------------- >> The AMBER Mail Reflector >> To post, send mail to amber_at_scripps.edu >> To unsubscribe, send "unsubscribe amber" (in the *body* of the email) >> to majordomo_at_scripps.edu >> > > ----------------------------------------------------------------------- > The AMBER Mail Reflector > To post, send mail to amber_at_scripps.edu > To unsubscribe, send "unsubscribe amber" (in the *body* of the email) > to majordomo_at_scripps.edu

----------------------------------------------------------------------- The AMBER Mail Reflector To post, send mail to amber_at_scripps.edu To unsubscribe, send "unsubscribe amber" (in the *body* of the email) to majordomo_at_scripps.edu