[oi-dev] Adding mpich/openmpi and allow switching between MPI implementations

[Aurélien Larcher]

Hi,
To precise the question about mpich/openmpi, I have been working on
packaging numerical libraries like PETSc, ATLAS, Hypre, ARPACK in the past
weeks. Unfortunately, research and teaching have slowed down the process a
lot...

I would like to open a discussion about where to install mpich [1] and
openmpi [2] and libraries built against them. There are two aspects to be
considered:
- the MPI implementation is compiled to be used with one compiler.
- the libraries built with mpich and openmpi cannot be mixed (they might
compile and link but will generally fail at runtime).

Since the MPI binaries and libraries should be compiler dependent they
cannot live in /usr/ and moreover the libraries compiled with mpicc/mpicxx
etc.. are MPI implementation-dependent.
In the past I have been installing the "bundle" (i.e for one pair (MPI
implementation, compiler)) in the prefix /usr/lib/$MPI/$COMPILER and
switching between them using the environment-modules tool [3]. Moreover
32bit and 64bit should be provided.

I wrote a file "mpi-macros.mk" which contains predefined variables and
paths like the following:

# Default to mpich
MPI_IMPLEMENTATION ?= $(MPI.mpich)
MPI_COMPILER = $(COMPILER)

MPI_BUNDLE =        $(MPI_IMPLEMENTATION)-$(MPI_COMPILER)

MPI_PREFIX =        $(USRLIBDIR)/$(MPI_IMPLEMENTATION)/$(MPI_COMPILER)

CONFIGURE_DEFAULT_DIRS=no

MPI_BINDIR.32 =     $(MPI_PREFIX)/bin
MPI_BINDIR.64 =     $(MPI_PREFIX)/bin/$(MACH64)
MPI_BINDIR    =     $(MPI_BINDIR.$(BITS))

(same with libraries, but data/doc/man system wide)

And added Makefiles to create packages with fmri library/mpich-gcc,
library/openmpi-gcc and other libraries with the same naming scheme. The
same could be done with studio if needed.
I chose the MPI_PREFIX like that as it is usually discouraged to add
something in /usr (/usr/mpich,/usr/openmpi) and since putting things in
/usr/gcc could not allow to get something consistent for studio (there is
no /usr/studio).

In each case, after including shared-macros.mk I declare MPI_IMPLEMENTATION
= mpich (or openmpi) and then

include $(WS_MAKE_RULES)/mpi-macros.mk


I would like to know if it sounds reasonable to you before I convert all my
makefiles to use oi-build.

Finally, I intend to provide modulefiles to be able to switch easily
between these "bundles" using environment-modules. This latter requires
bash-completion: has someone already packaged it ?
Some Linux distributions provide a tool to switch between MPI by changing
symlinks in /usr but this is not satisfactory and error-prone.

Thank you for your comments.
Best regards,

Aurélien


[1] http://www.mpich.org/
[2] http://www.open-mpi.org/
[3] http://modules.sourceforge.net/

-- 
-------------------------------------------------------------------------------
LARCHER Aurélien            | KTH, School of Computer Science and
Communication
Work: +46 (0) 8 790 71 42   | Lindstedtsvägen 5, Plan 5
Mob.: +46 (0) 7 09 46 40 17 | 100 44 Stockholm, SWEDEN
-------------------------------------------------------------------------------
Praise the Caffeine embeddings ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20131118/2b128b9f/attachment-0004.html>