[oi-dev] Numpy and Scipy
Adam Števko
adam.stevko at gmail.com
Mon Nov 18 08:38:57 UTC 2013
Hi Aurelien,
Speaking of prefixes, there are generally some rules (please, let the rest correct me if I am wrong):
a) if the software you are packaging has no reason to exist in multiple versions on the system, put it into /usr (for example vim/nano)
b) if there are reasons to use multiple version, then place it under /usr/<software>/<version> (for example, gcc 4.6 vs 4.7, postgresql 9.1 vs 9.2 vs 9.3) and create IPS mediated links into /usr/{bin,lib} and user will have the ability to choose.
However, creating sometimes mediated links for the library is tricky (alp could talk about that).
Python is rather an expection from b) as python 2.6 and 2.7 are both placed in /usr. Python binaries/scripts have a suffix -2.6/2.7 (depending on which version of python is used in shebang line of the script or which version of python is the binary for) and are mediated into non-suffixed link, e.g /usr/bin/python-2.6 becomes /usr/bin/python, if python mediator is set to 2.6 (at the moment, 2.6 is the only version available). Python modules are placed under /usr/lib/python-<PYVER>)/site-packages. Please check some python module IPS manifests (https://github.com/OpenIndiana/oi-userland/tree/oi/hipster/components/python). I am not sure that why Python is placed into /usr and not /usr/python, but I guess that it has to do something with the fact that IPS itself is written in Python. Could someone bring more light into this?
As for openmpi/mpich stuff, the locations sounds reasonable to me, but I am not sure how will this behave with other C++ libraries. It would be better if others comment on this.
Oh, and thanks again working on these libraries. If there are any other questions, just feel free to come back and ask.
Cheers,
Adam
On Nov 13, 2013, at 6:01 PM, Aurélien Larcher <aurelien.larcher at gmail.com> wrote:
> Hi Adam and Andrzej,
> sorry for the late follow-up, I have been caught in teaching and different research projects in the past weeks.
> Moreover I noticed that something broke my python modules again in hipster, I need to investigate.
> Anyway I installed a bunch of numerical libraries, python modules with oi-build that I could contribute to oi-userland but I have now a very concrete question for you that I mentioned in the past when I was packaging with SFE (but never got a conclusive answer).
> I think it requires some thinking before adding these packages.
>
>
> 1)
>
> For mpich and openmpi, one should provide packages compiled for a specific compiler and libraries (like PETSc and linear solvers) should be provided for a given pair (compiler, MPI implementation).
> For example, in the past I ended up providing mpich-gcc, openmpi-gcc, mpich-cc packages and for each library following the same scheme:
>
> library/mpich2-gcc (localhost) 1.5-0.0.151.1.7 i--
> library/openmpi-gcc (localhost) 1.6.3-0.0.151.1.7 i--
> library/math/mpich2/gcc/hypre (localhost) 2.9.0-0.0.151.1.7 i--
> library/math/mpich2/gcc/parmetis (localhost) 4.0.2-0.0.151.1.7 i--
>
> I then provided headers in /usr/include/mpich2 and /usr/include/openmpi (same thing for the documentation) and for each pair (compile, mpi) the libraries in /usr/lib/$(ARCH)/$(MPI)/$(COMPILER)
>
> For example: /usr/lib/amd64/mpich2/gcc
>
> My question is then if this scheme would be suitable for hipster and what should be the library prefix ?
>
> Obviously it cannot be just /usr/lib since you cannot mix C++ libraries from different compilers and you cannot mix MPI implementations.
>
> 2)
>
> One more thing is that I also use the environment-modules utility (http://modules.sourceforge.net/) to switch between mpi + compiler "bundles", it is a good way to set the environment without dealing with manual setting.
>
> Example:
>
> /afs/nada.kth.se/dept/na/ctl/pkg/@sys/modulefiles
>
> Contains the module definitions:
>
> #%Module1.0#####################################################################
> ##
> ## null modulefile
> ##
> proc ModulesHelp { } {
> global version
>
> puts stderr "\tThis module does absolutely nothing."
> puts stderr "\tIt's meant simply as a place holder in your"
> puts stderr "\tdot file initialization."
> puts stderr "\n\tVersion $version\n"
> }
>
> module-whatis "does absolutely nothing"
>
> # for Tcl script use only
> set version "3.1.6"
>
> set modroot /afs/nada.kth.se/dept/na/ctl/pkg/@sys/fiat/0.3.4
>
> prepend-path PATH $modroot/bin:
> prepend-path MANPATH $modroot/man
> prepend-path LD_LIBRARY_PATH $modroot/lib
>
> ## gcc paths
> prepend-path C_INCLUDE_PATH $modroot/include
> prepend-path CPP_INCLUDE_PATH $modroot/include
> prepend-path LIBRARY_PATH $modroot/lib
>
> ## python specific
> prepend-path PYTHONPATH $modroot/lib/python2.7/site-packages
> prepend-path PKG_CONFIG_PATH $modroot/lib/pkgconfig
>
> And the module root directory /afs/nada.kth.se/dept/na/ctl/pkg/@sys/fiat/0.3.4 contains the installed module
>
> /afs/nada.kth.se/dept/na/ctl/pkg/@sys/fiat/0.3.4/lib/python2.7/site-packages/FIAT/
>
> I could then provide modulefiles allowing to switch from one "bundle" (like "mpich2-gcc") to another.
>
> The main issue is: in which prefix shoud I put everything ? e.g is /usr/lib/$(ARCH)/$(MPI)/$(COMPILER) suitable of not ? or should we have /usr/mpich/$(COMPILER) and /usr/openmpi/$(COMPILER) prefixes ?
>
> Handling this is usually ugly even with Debian and FreeBSD.
>
> I also compiler Enlightenment 17 and have the same thought, what should be the prefix for a Desktop Environment.
> Same question applies to JNI modules.
>
> I think Hipster would benefit from guidelines to specify install prefix for different kinds of software.
>
> Thank you in advance.
> Best regards,
>
> Aurelien
>
>
>
>
>
>
>
>
> On Mon, Sep 16, 2013 at 12:21 PM, Adam Števko <adam.stevko at gmail.com> wrote:
> Hi Aurelien,
>
> how is it going? Do you need any help?
>
> Cheers,
> Adam
>
> Begin forwarded message:
>
>> From: Aurélien Larcher <aurelien.larcher at gmail.com>
>> Subject: [oi-dev] Numpy and Scipy
>> Date: September 8, 2013 6:36:57 PM GMT+02:00
>> To: OpenIndiana Developer mailing list <oi-dev at openindiana.org>
>> Reply-To: OpenIndiana Developer mailing list <oi-dev at openindiana.org>
>>
>> Hi everyone,
>> since the numpy python package available in hipster seems broken (segfaults), I installed the latest stable 1.7.0 compiled with gcc, without a linear algebra backend.
>> So I intend to add latest numpy, scipy (and atlas for the backend) in oi-userland if it is ok.
>>
>> How does it fit in the way python modules are packaged ? Should they be added to the python folder ?
>> Should 32 bit and 64 bits be both provided or only one of them ?
>>
>> What about gdb, is it possible to upgrade to 7.x to get python extensions or are there any known showstoppers ? (I installed the latest gdb with the 'historical' patches).
>>
>> I have also a few numerical libraries pending that I converted from my SFE specs.
>> Best regards,
>>
>> Aurelien
>>
>> --
>> -------------------------------------------------------------------------------
>> LARCHER Aurélien | KTH, School of Computer Science and Communication
>> Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 4
>> Mob.: +46 (0) 7 09 46 40 17 | 100 44 Stockholm, SWEDEN
>> -------------------------------------------------------------------------------
>> Praise the Caffeine embeddings ...
>> _______________________________________________
>> oi-dev mailing list
>> oi-dev at openindiana.org
>> http://openindiana.org/mailman/listinfo/oi-dev
>
>
>
>
> --
> -------------------------------------------------------------------------------
> LARCHER Aurélien | KTH, School of Computer Science and Communication
> Work: +46 (0) 8 790 71 42 | Lindstedtsvägen 5, Plan 5
> Mob.: +46 (0) 7 09 46 40 17 | 100 44 Stockholm, SWEDEN
> -------------------------------------------------------------------------------
> Praise the Caffeine embeddings ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20131118/8db84b84/attachment-0004.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20131118/8db84b84/attachment-0004.bin>
More information about the oi-dev
mailing list