2 years ago I had the task of running a python-program using scipy on our university cluster, using the Intel Compiler. I needed all those (as well as PyNIO and some other stuff) for running TM5 with the python shell [1] on the HC3 of KIT [2].
This proved to be quite a bit more challenging than I had expected - but it was very interesting, too (and there I learned the basics of GNU autotools which still help me a lot).
But no one should have to go to the same effort with as little guidance as I had, so I decided to publish the script and the patches I created for installing everything we needed.1 [3]
The script worked 2 years ago, so you might have to fix some bits. I won’t promise that this contains everything you need to run the script - or that it won’t be broken when you install it. Actually I won’t promise anything at all, except that if the stuff here had been available 2 years ago, that could have saved me about 2 months of time (each of the patches here required quite some tracking of problems, experimenting and fixing, until it provided basic functionality - but actually I enjoyed doing that - I learned a lot - I just don’t want to be forced to do it again). Still, this stuff contains quite some hacks - even a few ugly ones. But it worked.
This script requires and installs quite a few libraries. I retrieved most of the following tarballs from my Gentoo distfiles dir after installing the programs locally. I uploaded them to draketo.de/dateien/scipy-pynio-deps [4]. These files are included there:
satexputils.so also needs interpolatelevels.F90 which I think that I am not allowed to share, so you’re on your own there. Guess why I do not like using non-free (or not-guaranteed-to-be-free) software.
The hdf autotools patch only retrieves the last CFLAG instead of all:
export CC='gcc-4.8.1 -Wall -Werror' echo $CC | grep \ - | sed 's/.* -/-/' -Werror
If you have the regexp-foo [28] to fix that, please improve the patch! But without perl (otherwise we’d have to install perl, too).
Udo Grabowski [29], the maintainer of our institutes sun-cluster [30] somehow managed to get that working on OpenIndiana [31] with the Sun-Compiler, but since I did not need it, I did not dig deeper to see whether I could adapt his solutions to the intel-compiler.
Aside from some inline patches, the script uses the following patches:
This is the full install script I used to install all necessary dependencies.
#!/bin/bash # Untar for i in *.tar* *.tgz; do tar xvf $i || exit done # Install PREFIX=/home/ws/babenhau/ PYPREFIX=/home/ws/babenhau/python/ # Blas cd BLAS cp ../blas-make.inc make.inc || exit #make -j9 clean F77=ifort make -j9 || exit #make -j9 install --prefix=$PREFIX # OR for Intel compiler: ifort -fPIC -FI -w90 -w95 -cm -O3 -xHost -unroll -c *.f || exit #Continue below irrespective of compiler: ar r libfblas.a *.o || exit ranlib libfblas.a || exit cd .. ln -s BLAS blas ## Lapack cd lapack-3.3.1 ln -s ../blas # this has a hardcoded absolute path to blas in it: replace is with the appropriate one for you. cp ../lapack-make.inc make.inc || exit make -j9 clean || exit make -j9 make -j9 || exit cp lapack_LINUX.a libflapack.a || exit #make -j9 install --prefix=$PREFIX cd .. # C interface patch -p0 < lapacke-ifort.diff cd lapacke # patch for lapack 3.3.1 and blas for i in gnu inc intel ; do sed -i s/lapack-3\.2\.1\\/lapack\.a/lapack-3\.3\.1\\/lapack_LINUX.a/ make.$i; sed -i s/lapack-3\.2\.1\\/blas\.a/blas\\/blas_LINUX.a/ make.$i; done make -j9 clean || exit #make -j9 LINKER=ifort LDFLAGS=-nofor-main make -j9 # || exit #LINKER=ifort LDFLAGS=-nofor-main make -j9 install cd .. ## ATLAS cd ATLAS cp ../Make.Linux_HC3 . || exit echo "ATLAS needs manual intervention. Run make by hand first." #echo "just say yes. It makes some stuff we need later." #make #mv bin/Linux_UNKNOWNSSE2_8 bin/Linux_HC3 #for i in bin/Linux_HC3/*; do sed -i s/UNKNOWNSSE2_8/HC3/ $i ; done #rm bin/Linux_HC3/Make.inc #cd bin/Linux_HC3/ #ln -s ../../Make.Linux_HC3 Make.inc #cd - make -j9 install arch=Linux_HC3 || exit cd lib for i in Linux_HC3/* ; do ln -s $i ; done cd ../bin for i in Linux_HC3/* ; do ln -s $i ; done cd ../include for i in Linux_HC3/* ; do ln -s $i ; done cd .. cd .. # Numpy and SciPy with intel compilers # Read this: http://marklodato.github.com/2009/08/30/numpy-scipy-and-intel.html # patching patch -p0 < SuiteSparse.diff || exit patch -p0 < SuiteSparse-umfpack.diff || exit rm numpy ln -s numpy-*.*.*/ numpy patch -p0 < numpy-icc.diff || exit patch -p0 < numpy-icpc.diff || exit patch -p0 <<EOF --- numpy/numpy/distutils/fcompiler/intel.py 2009-03-29 07:24:21.000000000 -0400 +++ numpy/numpy/distutils/fcompiler/intel.py 2009-08-06 23:08:59.000000000 -0400 @@ -47,6 +47,7 @@ module_include_switch = '-I' def get_flags(self): + return ['-fPIC', '-cm'] v = self.get_version() if v >= '10.0': # Use -fPIC instead of -KPIC. @@ -63,6 +64,7 @@ return ['-O3','-unroll'] def get_flags_arch(self): + return ['-xHost'] v = self.get_version() opt = [] if cpu.has_fdiv_bug(): EOF # include -fPIC in the fcompiler. sed -i "s/w90/w90\", \"-fPIC/" numpy/numpy/distutils/fcompiler/intel.py # and more of that patch -p0 < numpy-ifort.diff rm scipy ln -s scipy-*.*.*/ scipy patch -p0 < scipy-qhull-icc.diff || exit patch -p0 < scipy-qhull-icc2.diff || exit # # unnecessary! # patch -p0 <<EOF # --- scipy/scipy/special/cephes/const.c 2009-08-07 01:56:43.000000000 -0400 # +++ scipy/scipy/special/cephes/const.c 2009-08-07 01:57:08.000000000 -0400 # @@ -91,12 +91,12 @@ # double THPIO4 = 2.35619449019234492885; /* 3*pi/4 */ # double TWOOPI = 6.36619772367581343075535E-1; /* 2/pi */ # #ifdef INFINITIES # -double INFINITY = 1.0/0.0; /* 99e999; */ # +double INFINITY = __builtin_inff(); # #else # double INFINITY = 1.79769313486231570815E308; /* 2**1024*(1-MACHEP) */ # #endif # #ifdef NANS # -double NAN = 1.0/0.0 - 1.0/0.0; # +double NAN = __builtin_nanf(""); # #else # double NAN = 0.0; # #endif # EOF # building # TODO: try again later cd SuiteSparse make -j9 -C AMD || exit make -j9 -C UMFPACK || exit cd .. # TODO: build numpy again and make sure it has blas and lapack (and ATLAS?) cd numpy python setup.py -v build_src config --compiler=intel build_clib \ --compiler=intel build_ext --compiler=intel || exit python setup.py install --prefix=$PYPREFIX || exit cd .. # scons and numscons cd scons-2.0.1 python setup.py -v install --prefix=/home/ws/babenhau/python/ || exit cd .. git clone git://github.com/cournape/numscons.git cd numscons python setup.py -v install --prefix=/home/ws/babenhau/python/ || exit cd .. # adapt /home/ws/babenhau/python/lib/python2.7/site-packages/numpy/distutils/fcompiler/intel.py by hand to include fPIC for intelem cd scipy PYTHONPATH=/home/ws/babenhau/python//lib/scons-2.0.1/ ATLAS=../ATLAS/ \ LAPACK=../lapack-3.3.1/libflapack.a LAPACK_SRC=../lapack-3.3.1 BLAS=../BLAS/libfblas.a \ F77=ifort f77_opt=ifort python setup.py -v config --compiler=intel --fcompiler=intelem build_clib \ --compiler=intel --fcompiler=intelem build_ext --compiler=intel --fcompiler=intelem \ -I../SuiteSparse/UFconfig # no exit, because we do the linking by hand later on. # one file is C++ :( icpc -fPIC -I/home/ws/babenhau/python/include/python2.7 -I/home/ws/babenhau/python/lib/python2.7/site-packages/numpy/core/include -I/home/ws/babenhau/python/lib/python2.7/site-packages/numpy/core/include -c scipy/spatial/qhull/src/user.c -o build/temp.linux-x86_64-2.7/scipy/spatial/qhull/src/user.o || exit # linking by hand # for x in csr csc coo bsr dia; do # icpc -xHost -O3 -fPIC -shared \ # build/temp.linux-x86_64-2.7/scipy/sparse/sparsetools/${x}_wrap.o \ # -o build/lib.linux-x86_64-2.7/scipy/sparse/sparsetools/_${x}.so || exit # done #icpc -xHost -O3 -fPIC -openmp -shared \ # build/temp.linux-x86_64-2.7/scipy/interpolate/src/_interpolate.o \ # -o build/lib.linux-x86_64-2.7/scipy/interpolate/_interpolate.so || exit # build again with the C++ file already compiled PYTHONPATH=/home/ws/babenhau/python//lib/scons-2.0.1/ ATLAS=../ATLAS/ \ LAPACK=../lapack-3.3.1/libflapack.a LAPACK_SRC=../lapack-3.3.1 BLAS=../BLAS/libfblas.a \ F77=ifort f77_opt=ifort python setup.py config --compiler=intel --fcompiler=intelem build_clib \ --compiler=intel --fcompiler=intelem build_ext --compiler=intel --fcompiler=intelem \ -I../SuiteSparse/UFconfig || exit # make sure we have cephes cd scipy/special PYTHONPATH=/home/ws/babenhau/python//lib/scons-2.0.1/ ATLAS=../../../ATLAS/ \ LAPACK=../../../lapack-3.3.1/libflapack.a LAPACK_SRC=../lapack-3.3.1 BLAS=../../../BLAS/libfblas.a \ F77=ifort f77_opt=ifort python setup.py -v config --compiler=intel --fcompiler=intelem build_clib \ --compiler=intel --fcompiler=intelem build_ext --compiler=intel --fcompiler=intelem \ -I../../../SuiteSparse/UFconfig cd ../.. # install PYTHONPATH=/home/ws/babenhau/python//lib/scons-2.0.1/ ATLAS=../ATLAS/ \ LAPACK=../lapack-3.3.1/libflapack.a LAPACK_SRC=../lapack-3.3.1 BLAS=../BLAS/libfblas.a \ F77=ifort f77_opt=ifort python setup.py config --compiler=intel --fcompiler=intelem build_clib \ --compiler=intel --fcompiler=intelem install --prefix=$PYPREFIX || exit cd .. # PyNIO # netcdf-4 patch -p0 < netcdf-patch1.diff || exit patch -p0 < netcdf-patch2.diff || exit cd netcdf-4.1.3 CPPFLAGS="-I/home/ws/babenhau/libbutz/hdf5-1.8.7/include -I/home/ws/babenhau/include" LDFLAGS="-L/home/ws/babenhau/libbutz/hdf5-1.8.7/lib/ -L/home/ws/babenhau/lib -lsz -L/home/ws/babenhau/libbutz/szip-2.1/lib -L/opt/intel/Compiler/11.1/080/lib/intel64/libifcore.a -lifcore" ./configure --prefix=/home/ws/babenhau/ --enable-netcdf-4 --enable-shared || exit make -j9; make check install -j9 || exit cd .. # NetCDF4 cd netCDF4-0.9.7 HAS_SZIP=1 SZIP_PREFIX=/home/ws/babenhau/libbutz/szip-2.1/ HAS_HDF5=1 HDF5_DIR=/home/ws/babenhau/libbutz/hdf5-1.8.7 HDF5_PREFIX=/home/ws/babenhau/libbutz/hdf5-1.8.7 HDF5_includedir=/home/ws/babenhau/libbutz/hdf5-1.8.7/include HDF5_libdir=/home/ws/babenhau/libbutz/hdf5-1.8.7/lib HAS_NETCDF4=1 NETCDF4_PREFIX=/home/ws/babenhau/ python setup.py build_ext --compiler="intel" --fcompiler="intel -fPIC" install --prefix $PYPREFIX cd .. # parallel netcdf and hdf5: ~/libbutz/ patch -p0 < pynio-fix-no-grib.diff || exit cd PyNIO-1.4.1 HAS_SZIP=1 SZIP_PREFIX=/home/ws/babenhau/libbutz/szip-2.1/ HAS_HDF5=1 HDF5_DIR=/home/ws/babenhau/libbutz/hdf5-1.8.7 HDF5_PREFIX=/home/ws/babenhau/libbutz/hdf5-1.8.7 HDF5_includedir=/home/ws/babenhau/libbutz/hdf5-1.8.7/include HDF5_libdir=/home/ws/babenhau/libbutz/hdf5-1.8.7/lib HAS_NETCDF4=1 NETCDF4_PREFIX=/home/ws/babenhau/ python setup.py install --prefix=$PYPREFIX || exit # TODO: Make sure that the install goes to /home/ws/.., not home/ws/... cd .. # satexp_utils.so f2py -c -m satexp_utils --f77exec=ifort --f90exec=ifort interpolate_levels.F90 || exit ## pyhdf # recompile hdf with fPIC - grr! cd hdf-4*/ # Fix configure for compilers with - in the name. patch -p0 < ../hdf-fix-configure.ac.diff autoconf FFLAGS="-ip -O3 -xHost -fPIC -r8" CFLAGS="-ip -O3 -xHost -fPIC" CXXFLAGS="$CFLAGS -I/usr/include/rpc -DBIG_LONGS -DSWAP" F77=ifort ./configure --prefix=/home/ws/babenhau/ --disable-netcdf --with-szlib=/home/ws/babenhau/libbutz/szip-2.1 # --with-zlib=/home/ws/babenhau/libbutz/zlib-1.2.5 --with-jpeg=/home/ws/babenhau/libbutz/jpeg-8c # finds zlib and jpeg due to LD_LIBRARY_PATH (hack but works…) make make install cd .. # build pyhdf cd pyhdf-0.8.3/ INCLUDE_DIRS="/home/ws/babenhau/include:/home/ws/babenhau/libbutz/szip-2.1/include" LIBRARY_DIRS="/home/ws/babenhau/lib:/home/ws/babenhau/libbutz/szip-2.1/lib" python setup.py build -c intel --fcompiler ifort install --prefix=/home/ws/babenhau/python cd .. ## matplotlib cd matplotlib-1.1.0 patch -p0 < ../matplotlib-add-icc-support.diff python setup.py build -c intel install --prefix=/home/ws/babenhau/python cd .. # GEOS → http://download.osgeo.org/geos/geos-3.3.2.tar.bz2 cd geos*/ ./configure --prefix=/home/ws/babenhau/ make make check make install cd .. # basemap easy_install --prefix /home/ws/babenhau/python basemap # fails but should now have all dependencies. cd basemap-*/ python setup.py build -c intel install --prefix=/home/ws/babenhau/python cd ..
To ease usage and upstreaming of my fixes, I include all the patches below, so you can find them directly in this text instead of having to browse external textfiles.
--- SuiteSparse/UMFPACK/Lib/GNUmakefile 2009-11-11 21:09:54.000000000 +0100 +++ SuiteSparse/UMFPACK/Lib/GNUmakefile 2011-09-09 14:18:57.000000000 +0200 @@ -9,7 +9,7 @@ C = $(CC) $(CFLAGS) $(UMFPACK_CONFIG) \ -I../Include -I../Source -I../../AMD/Include -I../../UFconfig \ -I../../CCOLAMD/Include -I../../CAMD/Include -I../../CHOLMOD/Include \ - -I../../metis-4.0/Lib -I../../COLAMD/Include + -I../../COLAMD/Include #------------------------------------------------------------------------------- # source files
--- SuiteSparse/UFconfig/UFconfig.mk 2011-09-09 13:14:03.000000000 +0200 +++ SuiteSparse/UFconfig/UFconfig.mk 2011-09-09 13:15:03.000000000 +0200 @@ -33,11 +33,11 @@ # C compiler and compiler flags: These will normally not give you optimal # performance. You should select the optimization parameters that are best # for your system. On Linux, use "CFLAGS = -O3 -fexceptions" for example. -CC = cc -CFLAGS = -O3 -fexceptions +CC = icc +CFLAGS = -O3 -xHost -fPIC -openmp -vec_report=0 # C++ compiler (also uses CFLAGS) -CPLUSPLUS = g++ +CPLUSPLUS = icpc # ranlib, and ar, for generating libraries RANLIB = ranlib @@ -49,8 +49,8 @@ MV = mv -f # Fortran compiler (not normally required) -F77 = f77 -F77FLAGS = -O +F77 = ifort +F77FLAGS = -O3 -xHost F77LIB = # C and Fortran libraries @@ -132,13 +132,13 @@ # The path is relative to where it is used, in CHOLMOD/Lib, CHOLMOD/MATLAB, etc. # You may wish to use an absolute path. METIS is optional. Compile # CHOLMOD with -DNPARTITION if you do not wish to use METIS. -METIS_PATH = ../../metis-4.0 -METIS = ../../metis-4.0/libmetis.a +# METIS_PATH = ../../metis-4.0 +# METIS = ../../metis-4.0/libmetis.a # If you use CHOLMOD_CONFIG = -DNPARTITION then you must use the following # options: -# METIS_PATH = -# METIS = +METIS_PATH = +METIS = #------------------------------------------------------------------------------ # UMFPACK configuration: @@ -194,7 +194,7 @@ # -DNSUNPERF for Solaris only. If defined, do not use the Sun # Performance Library -CHOLMOD_CONFIG = +CHOLMOD_CONFIG = -DNPARTITION #------------------------------------------------------------------------------ # SuiteSparseQR configuration:
--- configure.ac 2012-03-01 15:00:28.000000000 +0100 +++ configure.ac 2012-03-01 15:00:40.000000000 +0100 @@ -815,7 +815,7 @@ dnl Report anything stripped as a flag in CFLAGS and dnl only the compiler in CC_VERSION. CC_NOFLAGS=`echo $CC | sed 's/ -.*//'` -CFLAGS_TO_ADD=`echo $CC | grep - | sed 's/.* -/-/'` +CFLAGS_TO_ADD=`echo $CC | grep \ - | sed 's/.* -/-/'` if test -n $CFLAGS_TO_ADD; then CFLAGS="$CFLAGS_TO_ADD$CFLAGS" fi
--- lapacke/make.intel.old 2011-10-05 13:24:14.000000000 +0200 +++ lapacke/make.intel 2011-10-05 16:17:00.000000000 +0200 @@ -56,7 +56,7 @@ # Ensure that the libraries have the same data model (LP64/ILP64). # LAPACKE = lapacke.a -LIBS = ../../../lapack-3.3.1/lapack_LINUX.a ../../../blas/blas_LINUX.a -lm +LIBS = /opt/intel/Compiler/11.1/080/lib/intel64/libifcore.a ../../../lapack-3.2.1/lapack.a ../../../lapack-3.2.1/blas.a -lm -ifcore # # The archiver and the flag(s) to use when building archive (library) # If your system has no ranlib, set RANLIB = echo.
diff -r 38c2a32c56ae matplotlib-1.1.0/setup.py --- a/matplotlib-1.1.0/setup.py Fri Mar 02 12:29:47 2012 +0100 +++ b/matplotlib-1.1.0/setup.py Fri Mar 02 12:30:39 2012 +0100 @@ -31,6 +31,13 @@ if major==2 and minor1<4 or major<2: raise SystemExit("""matplotlib requires Python 2.4 or later.""") +if "intel" in sys.argv or "icc" in sys.argv: + try: # make it compile with the intel compiler + from numpy.distutils import intelccompiler + except ImportError: + print "Compiling with the intel compiler requires numpy." + raise + import glob from distutils.core import setup from setupext import build_agg, build_gtkagg, build_tkagg,\
--- netcdf-4.1.3/fortran/ncfortran.h 2011-07-01 01:22:22.000000000 +0200 +++ netcdf-4.1.3/fortran/ncfortran.h 2011-09-14 14:56:03.000000000 +0200 @@ -658,7 +658,7 @@ * The following is for f2c-support only. */ -#if defined(f2cFortran) && !defined(pgiFortran) && !defined(gFortran) +#if defined(f2cFortran) && !defined(pgiFortran) && !defined(gFortran) &&!defined(__INTEL_COMPILER) /* * The f2c(1) utility on BSD/OS and Linux systems adds an additional
--- netcdf-4.1.3/nf_test/fortlib.c 2011-09-14 14:58:47.000000000 +0200 +++ netcdf-4.1.3/nf_test/fortlib.c 2011-09-14 14:58:38.000000000 +0200 @@ -14,7 +14,7 @@ #include "../fortran/ncfortran.h" -#if defined(f2cFortran) && !defined(pgiFortran) && !defined(gFortran) +#if defined(f2cFortran) && !defined(pgiFortran) && !defined(gFortran) &&!defined(__INTEL_COMPILER) /* * The f2c(1) utility on BSD/OS and Linux systems adds an additional * underscore suffix (besides the usual one) to global names that have
--- numpy/numpy/distutils/intelccompiler.py 2011-09-08 14:14:03.000000000 +0200 +++ numpy/numpy/distutils/intelccompiler.py 2011-09-08 14:20:37.000000000 +0200 @@ -30,11 +30,11 @@ """ A modified Intel x86_64 compiler compatible with a 64bit gcc built Python. """ compiler_type = 'intelem' - cc_exe = 'icc -m64 -fPIC' + cc_exe = 'icc -m64 -fPIC -xHost -O3' cc_args = "-fPIC" def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose,dry_run, force) - self.cc_exe = 'icc -m64 -fPIC' + self.cc_exe = 'icc -m64 -fPIC -xHost -O3' compiler = self.cc_exe self.set_executables(compiler=compiler, compiler_so=compiler,
--- numpy-1.6.1/numpy/distutils/intelccompiler.py 2011-10-06 16:55:12.000000000 +0200 +++ numpy-1.6.1/numpy/distutils/intelccompiler.py 2011-10-10 10:26:14.000000000 +0200 @@ -10,11 +10,13 @@ def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose,dry_run, force) self.cc_exe = 'icc -fPIC' + self.cxx_exe = 'icpc -fPIC' compiler = self.cc_exe + compiler_cxx = self.cxx_exe self.set_executables(compiler=compiler, compiler_so=compiler, - compiler_cxx=compiler, - linker_exe=compiler, + compiler_cxx=compiler_cxx, + linker_exe=compiler_cxx, linker_so=compiler + ' -shared') class IntelItaniumCCompiler(IntelCCompiler):
--- numpy-1.6.1/numpy/distutils/fcompiler/intel.py.old 2011-10-10 17:52:34.000000000 +0200 +++ numpy-1.6.1/numpy/distutils/fcompiler/intel.py 2011-10-10 17:53:51.000000000 +0200 @@ -32,7 +32,7 @@ executables = { 'version_cmd' : None, # set by update_executables 'compiler_f77' : [None, "-72", "-w90", "-fPIC", "-w95"], - 'compiler_f90' : [None], + 'compiler_f90' : [None, "-fPIC"], 'compiler_fix' : [None, "-FI"], 'linker_so' : ["<F90>", "-shared"], 'archiver' : ["ar", "-cr"], @@ -129,7 +129,7 @@ 'version_cmd' : None, 'compiler_f77' : [None, "-FI", "-w90", "-fPIC", "-w95"], 'compiler_fix' : [None, "-FI"], - 'compiler_f90' : [None], + 'compiler_f90' : [None, "-fPIC"], 'linker_so' : ['<F90>', "-shared"], 'archiver' : ["ar", "-cr"], 'ranlib' : ["ranlib"] @@ -148,7 +148,7 @@ 'version_cmd' : None, 'compiler_f77' : [None, "-FI", "-w90", "-fPIC", "-w95"], 'compiler_fix' : [None, "-FI"], - 'compiler_f90' : [None], + 'compiler_f90' : [None, "-fPIC"], 'linker_so' : ['<F90>', "-shared"], 'archiver' : ["ar", "-cr"], 'ranlib' : ["ranlib"] @@ -180,7 +180,7 @@ 'version_cmd' : None, 'compiler_f77' : [None,"-FI","-w90", "-fPIC","-w95"], 'compiler_fix' : [None,"-FI","-4L72","-w"], - 'compiler_f90' : [None], + 'compiler_f90' : [None, "-fPIC"], 'linker_so' : ['<F90>', "-shared"], 'archiver' : [ar_exe, "/verbose", "/OUT:"], 'ranlib' : None @@ -232,7 +232,7 @@ 'version_cmd' : None, 'compiler_f77' : [None,"-FI","-w90", "-fPIC","-w95"], 'compiler_fix' : [None,"-FI","-4L72","-w"], - 'compiler_f90' : [None], + 'compiler_f90' : [None, "-fPIC"], 'linker_so' : ['<F90>',"-shared"], 'archiver' : [ar_exe, "/verbose", "/OUT:"], 'ranlib' : None
--- PyNIO-1.4.1/Nio.py 2011-09-14 16:00:13.000000000 +0200 +++ PyNIO-1.4.1/Nio.py 2011-09-14 16:00:18.000000000 +0200 @@ -98,7 +98,7 @@ if ncarg_dir == None or not os.path.exists(ncarg_dir) \ or not os.path.exists(os.path.join(ncarg_dir,"lib","ncarg")): if not __formats__['grib2']: - return None + return "" # "", because an env variable has to be a string. else: print "No path found to PyNIO/ncarg data directory and no usable NCARG installation found" sys.exit()
--- scipy/scipy/spatial/qhull/src/qhull_a.h 2011-02-27 11:57:03.000000000 +0100 +++ scipy/scipy/spatial/qhull/src/qhull_a.h 2011-09-09 15:42:12.000000000 +0200 @@ -102,13 +102,13 @@ #elif defined(__MWERKS__) && defined(__INTEL__) # define QHULL_OS_WIN #endif -#if defined(__INTEL_COMPILER) && !defined(QHULL_OS_WIN) -template <typename T> -inline void qhullUnused(T &x) { (void)x; } -# define QHULL_UNUSED(x) qhullUnused(x); -#else +/*#if defined(__INTEL_COMPILER) && !defined(QHULL_OS_WIN)*/ +/*template <typename T>*/ +/*inline void qhullUnused(T &x) { (void)x; }*/ +/*# define QHULL_UNUSED(x) qhullUnused(x);*/ +/*#else*/ # define QHULL_UNUSED(x) (void)x; -#endif +*/#endif*/ /***** -libqhull.c prototypes (alphabetical after qhull) ********************/
--- scipy/scipy/spatial/qhull/src/qhull_a.h 2011-09-09 15:43:54.000000000 +0200 +++ scipy/scipy/spatial/qhull/src/qhull_a.h 2011-09-09 15:45:17.000000000 +0200 @@ -102,13 +102,7 @@ #elif defined(__MWERKS__) && defined(__INTEL__) # define QHULL_OS_WIN #endif -/*#if defined(__INTEL_COMPILER) && !defined(QHULL_OS_WIN)*/ -/*template <typename T>*/ -/*inline void qhullUnused(T &x) { (void)x; }*/ -/*# define QHULL_UNUSED(x) qhullUnused(x);*/ -/*#else*/ # define QHULL_UNUSED(x) (void)x; -*/#endif*/ /***** -libqhull.c prototypes (alphabetical after qhull) ********************/
--- scipy-0.9.0/scipy/spatial/setup.py 2011-10-10 17:11:23.000000000 +0200 +++ scipy-0.9.0/scipy/spatial/setup.py 2011-10-10 17:11:09.000000000 +0200 @@ -22,6 +22,8 @@ get_numpy_include_dirs()], # XXX: GCC dependency! #extra_compiler_args=['-fno-strict-aliasing'], + # XXX intel compiler dependency + extra_compiler_args=['-lifcore'], ) lapack = dict(get_info('lapack_opt'))
I hope this helps someone out there saving some time - or even better: improving the upstream projects. At least it should be a nice reference for all who need to get scipy working on not-quite-supported architectures.
Happy Hacking!
: Actually I already wanted to publish that script more than a year ago, but time flies and there’s always stuff to do. But at least I now managed to get it done.
| Anhang | Größe |
|---|---|
| 2013-09-26-Do-installing-scipy-and-matplotlib-on-a-bare-cluster-with-the-intel-compiler.org [49] | 29.2 KB |
Links:
[1] http://sourceforge.net/projects/tm5/
[2] http://www.scc.kit.edu/dienste/hc3.php
[3] https://www.draketo.de/light/english/install-scipy-pynio-cluster-intel#fn.1
[4] http://draketo.de/dateien/scipy-pynio-deps
[5] http://draketo.de/dateien/scipy-pynio-deps/PyNIO-1.4.1.tar.gz
[6] http://draketo.de/dateien/scipy-pynio-deps/SuiteSparse.tar.gz
[7] http://www.cise.ufl.edu/research/sparse/SuiteSparse/current/
[8] http://draketo.de/dateien/scipy-pynio-deps/atlas3.6.0.tar
[9] http://draketo.de/dateien/scipy-pynio-deps/basemap-1.0.1.tar
[10] http://draketo.de/dateien/scipy-pynio-deps/cblas.tar
[11] http://draketo.de/dateien/scipy-pynio-deps/curl-7.21.7.tar.bz2
[12] http://draketo.de/dateien/scipy-pynio-deps/geos-3.3.2.tar.bz2
[13] http://draketo.de/dateien/scipy-pynio-deps/git-1.7.6.1.tar.bz2
[14] http://draketo.de/dateien/scipy-pynio-deps/hdf-4.2.7.tar.bz2
[15] http://draketo.de/dateien/scipy-pynio-deps/hdf5-1.8.7.tar.bz2
[16] http://draketo.de/dateien/scipy-pynio-deps/lapacke.tar
[17] http://draketo.de/dateien/scipy-pynio-deps/libgemm-20000228.tar.bz2
[18] http://draketo.de/dateien/scipy-pynio-deps/matplotlib-1.0.1.tar
[19] http://draketo.de/dateien/scipy-pynio-deps/matplotlib-1.1.0.tar.gz
[20] http://draketo.de/dateien/scipy-pynio-deps/netCDF4-0.9.7.tar
[21] http://draketo.de/dateien/scipy-pynio-deps/netcdf-4.1.3.tar.gz
[22] http://draketo.de/dateien/scipy-pynio-deps/numpy-1.6.1.tar
[23] http://draketo.de/dateien/scipy-pynio-deps/pyhdf-0.8.3.tar.gz
[24] http://draketo.de/dateien/scipy-pynio-deps/scipy-0.9.0.tar
[25] http://draketo.de/dateien/scipy-pynio-deps/scons-2.0.1.tar.gz
[26] http://www.gnu.org/distros/free-distros.html
[27] http://draketo.de/dateien/scipy-pynio-deps/zoneinfo-2010g.tar.gz
[28] https://xkcd.com/208/
[29] http://www.imk-asf.kit.edu/mitarbeiter_110.php
[30] http://www.imk-asf.kit.edu/english/763.php
[31] http://openindiana.org/
[32] http://draketo.de/dateien/scipy-pynio-deps/SuiteSparse-umfpack.diff
[33] http://draketo.de/dateien/scipy-pynio-deps/SuiteSparse.diff
[34] http://draketo.de/dateien/scipy-pynio-deps/hdf-fix-configure.ac.diff
[35] http://draketo.de/dateien/scipy-pynio-deps/lapacke-ifort.diff
[36] http://draketo.de/dateien/scipy-pynio-deps/matplotlib-add-icc-support.diff
[37] http://draketo.de/dateien/scipy-pynio-deps/netcdf-patch1.diff
[38] http://draketo.de/dateien/scipy-pynio-deps/netcdf-patch2.diff
[39] http://draketo.de/dateien/scipy-pynio-deps/numpy-icc.diff
[40] http://draketo.de/dateien/scipy-pynio-deps/numpy-icpc.diff
[41] http://draketo.de/dateien/scipy-pynio-deps/numpy-ifort.diff
[42] http://draketo.de/dateien/scipy-pynio-deps/pynio-fix-no-grib.diff
[43] http://draketo.de/dateien/scipy-pynio-deps/scipy-qhull-icc.diff
[44] http://draketo.de/dateien/scipy-pynio-deps/scipy-qhull-icc2.diff
[45] http://draketo.de/dateien/scipy-pynio-deps/scipy-spatial-lifcore.diff
[46] http://www.gnu.org/software/emacs/
[47] http://orgmode.org
[48] http://validator.w3.org/check?uri=referer
[49] https://www.draketo.de/files/2013-09-26-Do-installing-scipy-and-matplotlib-on-a-bare-cluster-with-the-intel-compiler.org