Code Saturne Compilation errors with "recompile with -fPIC"

[bsotelo]

Hi

I'm installing code saturne latest stable version 7.0.0 and building it from source, this is my configure line:

CC=mpiicc ../configure --prefix=/program/code-saturne-7.0.0/ --with-hdf5=/program/code-saturne-7.0.0/hdf5-1.12.1/ --with-cgns=/program/code-saturne-7.0.0/cgns-4.3.0/

As you see, I have included HDF5 and CGNS (installed prior) but I'm getting the following error when running make:

Code: Select all

libtool: link: mpiicc -O2 -std=c11 -restrict -funsigned-char -Wall -Wcheck -Wshadow -Wpointer-arith -Wmissing-prototypes -Wuninitialized -Wunused -wd981 -qopenmp -O -Wl,-export-dynamic -qopenmp -o cs_io_dump cs_io_dump-cs_io_dump.o  -lz -ldl
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(cgns_error.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(cgns_io.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(cgnslib.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(ADFH.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(ADF_interface.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(ADF_internals.o): relocation R_X86_64_32 against `sub_node_start_tag' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(cg_hashmap.o): relocation R_X86_64_32S against `BitLengthTable' can not be used when making a shared object; recompile with -fPIC
ld: /program/code-saturne-7.0.0/cgns-4.3.0//lib/libcgns.a(cgns_internals.o): relocation R_X86_64_32 against `.rodata.str1.4' can not be used when making a shared object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output
make[3]: *** [libsaturne.la] Error 1
make[3]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build/src/apps'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build'
make: *** [all] Error 2

Any help with how to diagnose this would be much appreciated

[bsotelo]

I forgot to attach my log files, they are now in this response.

On further debugging,

I added this flag "--enable-shared" to code saturne's configure line as well as added this system env var "export CXXFLAGS="$CXXFLAGS -fPIC"" before running configure but it yields the same error.

and on further reading, based on the log, it seems there are several static libraries from my CGNS compilation and I just realized I previously compiled CGNS without "--enabled-shared", is this an issue with incompatible static CGNS libraries and shared library with code saturne? how to fix this?

[Yvan Fournier]

Hello,

Did you force shared libraries when compiling CGNS ? Since you have a libcgns.a instead of .so, I guess not. So you need either to force shared libraries for CGNS, or force static (configure --disable-shared) for code_saturne. I usually prefer the first solution.

Since you are using very recent tool versions, I also recommend trying v7.0.4 rather than v7.0.0.

There have been updates to detection of the oneAPI compilers in the master branch, but not all of those might be in the v7.0 branch, so testing with the master branch first might be interesting also.

Best regards,

Yvan

[bsotelo]

Thank you for your response Yvan,

I realized yes that I had not compiled with CGNS option so i had to do that but I also had to add a -fPIC to FFLAG to CGNS configure step. Then I was able to compile CGNS as a shared library and I think this is now a thing of the past. The new error that I get when I run "make" is the following below. Why is it saying No module named 'code_saturne'?

This is my configure command:

PYTHONPATH=/program/code-saturne-7.0.0/venv/lib/python3.7/site-packages PYTHON=/program/code-saturne-7.0.0/venv/bin/python3 CC=mpiicc FC=mpiifort CXX=mpicxx ../configure --enable-shared --prefix=/program/code-saturne-7.0.0/ --with-hdf5=/program/code-saturne-7.0.0/hdf5-1.12.1/ --with-cgns=/program/code-saturne-7.0.0/cgns-4.3.0-shared/

Code: Select all

libtool: link: (cd ".libs" && rm -f "libcs_solver.so" && ln -s "libcs_solver-7.0.so" "libcs_solver.so")
libtool: link: ( cd ".libs" && rm -f "libcs_solver.la" && ln -s "../libcs_solver.la" "libcs_solver.la" )
PYTHONPATH=../../bin:../../../bin \
/program/code-saturne-7.0.0/venv/bin/python3 -B ../../../build-aux/cs_compile_build.py \
../../../src/apps/cs_solver.c
Traceback (most recent call last):
  File "/media/ephemeral0/code_saturne-7.0.0/build/bin/cs_package.py", line 30, in <module>
    from code_saturne import cs_config
ModuleNotFoundError: No module named 'code_saturne'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../../build-aux/cs_compile_build.py", line 354, in <module>
    from cs_package import package
  File "/media/ephemeral0/code_saturne-7.0.0/build/bin/cs_package.py", line 32, in <module>
    import cs_config
  File "/media/ephemeral0/code_saturne-7.0.0/build/bin/cs_config.py", line 309
    libdir = ""/opt/rescale/oneapi/2021.4.0/mpi/2021.4.0/lib/release"",
                                          ^
SyntaxError: invalid syntax
make[3]: *** [cs_solver] Error 1
make[3]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build/src/apps'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/media/ephemeral0/code_saturne-7.0.0/build'
make: *** [all] Error 2

[Yvan Fournier]

Hello,

You forcing of PYTHONPATH might interfere with the path management done by the code_saturne build.
Running inside a Python virtual environment, you should not need to define PYTHONPATH.

We do have working installs with a defined PYTHONPATH, but in this case, PYTHONPATH is usually exported.
Could you try exporting PYTHONPATH instead of just prefixing it ?

Best regards,

Yvan

[bsotelo]

Thank you for your new reply Yvan. I tried your suggestion and it did not work. I still get the same error:

Code: Select all

PYTHONPATH=../../bin:../../../bin \
/program/code-saturne-7.0.0/venv/bin/python -B ../../../build-aux/cs_compile_build.py \
../../../src/apps/cs_solver.c
Traceback (most recent call last):
  File "/media/ephemeral0/install/build/bin/cs_package.py", line 30, in <module>
    from code_saturne import cs_config
ModuleNotFoundError: No module named 'code_saturne'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../../build-aux/cs_compile_build.py", line 354, in <module>
    from cs_package import package
  File "/media/ephemeral0/install/build/bin/cs_package.py", line 32, in <module>
    import cs_config
  File "/media/ephemeral0/install/build/bin/cs_config.py", line 309
    libdir = ""/opt/rescale/oneapi/2021.4.0/mpi/2021.4.0/lib/release"",
                                          ^
SyntaxError: invalid syntax
make[3]: *** [cs_solver] Error 1
make[3]: Leaving directory `/media/ephemeral0/install/build/src/apps'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/media/ephemeral0/install/build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directo

So indeed I am running a python environment using pyenv and python 3.7.6. At a very high level, my install script launches inside a cloud instance with Centos7. I first install hdf5 and cgns from source. Then I install pyenv and install and activate python 3.7.6 and then pip install PyQt5. Finally I exported PYTHONPATH and launched a configure and run make:

Code: Select all

export PYTHONPATH=${PROGRAM_PATH}/venv/lib/python3.7/site-packages:$PYTHONPATH[/quote]

CC=mpiicc ../configure --disable-shared --prefix=/program/code-saturne-7.0.0/ --with-hdf5=/program/code-saturne-7.0.0/hdf5-1.12.1/ --with-cgns=/program/code-saturne-7.0.0/cgns-4.3.0/

The fact I'm getting an error with "No module named 'code_saturne'", it tells me I'm missing something fundamental. I don't think there is a code_saturne.py module that I'm supposed to install is there? I feel like it's probably a silly mistake but I can't seem to identify it. Can you comment more on what my specific error means?

On further review, I noticed inside the build directory, some files are created in <build-dir>/bin folder, and it shows there is a code_saturne but not a .py extension. Is this the module? I appreciate if you can comment on my issue. I think we are super close

[Yvan Fournier]

Hello,

Depending on configure/build options, "bin/code_saturne" is either a Python script, or a shell script calling a code_saturne.py script.

In any case, A Python module/directory named "code_saturne" should be present, and contain various files in directories, including "cs_config.py" file.

What is strange is that around line 29/30, cs_package.py should contain a "try/execpt" clause around
"from code_saturne import cs_config"

Which does not seem to be the case in your install. Was this file modified compared to the code_saturne 7.0 branch ?

Regards,

Yvan

[bsotelo]

Hi Yvan,

I'm able to reproduce the issue with "ModuleNotFoundError: No module named 'code_saturne'" with versions 7.0.0 and 7.0.4. It is worth mentioning that I was using the tarballs from this site: https://www.code-saturne.org/releases/

T

Code: Select all

raceback (most recent call last):
  File "/media/ephemeral0/install/build/bin/cs_package.py", line 30, in <module>
    from code_saturne import cs_config
ModuleNotFoundError: No module named 'code_saturne'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../../build-aux/cs_compile_build.py", line 354, in <module>
    from cs_package import package
  File "/media/ephemeral0/install/build/bin/cs_package.py", line 32, in <module>
    import cs_config
  File "/media/ephemeral0/install/build/bin/cs_config.py", line 309
    libdir = ""/opt/rescale/oneapi/2021.4.0/mpi/2021.4.0/lib/release"",
                                          ^
SyntaxError: invalid syntax
make[3]: *** [cs_solver] Error 1
make[3]: Leaving directory `/media/ephemeral0/install/build/src/apps'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/media/ephemeral0/install/build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/media/ephemeral0/install/build'
make: *** [all] Error 2

If I clone the repository directly from master branch, which you suggested in your earlier reply, I do not run into this issue. It seems the repository has a python/code_saturne folder while the site with tarballs do not have a python/code_saturne folder. I finally cloned the repository but from the v7.0 branch and that also got the same error above. So in conclusion only the master branch has worked so far.

You did mention that since I'm using Intel OneAPI 2021 compilers that I should stick with master branch. Are these the reasons? I also didn't know what version I'm getting from master branch since this is always going to change but at I ran "./configure --version" and it appears that TODAY it's 7.2-alpha.

Does code saturne install not come with its own bundled MPI? I didn't find any. The installation has worked from master branch so I can run the code serially. Next, will be to run the program with Intel MPI. We have our own wrappers so hopefully it's not too difficult.

[Yvan Fournier]

Hello,

There have zlso been some changes to the cs_package and cs_config module hansling between v7.0 and the master branch, so the issue in your case might be solved by these improvements rather than by better support of the oneAPI environment.

I'm not sure which of the 2 helps here.

In any case, we do not bundle our own MPI, for several reasons. On a simple Linux distribution, it is easy to install a packaged MPI, which will be compatible with other available packages. On a cluster, it is better to use the MPI libraries installed by the admins, so as to have a correct fast network driver configuration and a correct batch system integration. So we stopped bundling MPI in our installer some years ago.

Regards,

Yvan