Error building with Catalyst support

All questions about installation
Forum rules
Please read the forum usage recommendations before posting.
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

Thanks for your suggestions. Will try ASAP (I'm at work place now, the machine is a home workstation + I'm quite busy with everyday things now, I spent most of my spare time on weekend upgrading and updating Kubuntu and it was so kind to make it from 16 to 20.04 version :), but I didn't have time yet to check if it helped and what CMake version I have for now). May it be connected also with old OpenMPI version 1.8.4? Maybe I need to use more recent OpenMPI? What OpenMPI version would you suggest as the most stable/compatible?
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Error building with Catalyst support

Post by Yvan Fournier »

Hello,

I do not know the details of how CMake detects MPI, but switching to a more up-to-date version would be a good idea. In this case, using the latest OpenMPI, or whichever version is packaged on a Ububtu 20.04 would seem fine.

Best regards,

Yvan
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

I switched to out-of-source build (re-extracted fresh source tree), Kubuntu version is 20.04. The MPI issue disappeared but there is something other instead of this:

Code: Select all

...
checking ParaView/Catalyst... CMake Error at CMakeLists.txt:27 (add_executable):
  Target "CoProcessingTest" links to target "VTK::ViewsContext2D" but the
  target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at CMakeLists.txt:27 (add_executable):
  Target "CoProcessingTest" links to target "VTK::RenderingContext2D" but the
  target was not found.  Perhaps a find_package() call is missing for an
  IMPORTED target, or an ALIAS target is missing?


CMake Error at CMakeLists.txt:27 (add_executable):
  Target "CoProcessingTest" links to target "VTK::ChartsCore" but the target
  was not found.  Perhaps a find_package() call is missing for an IMPORTED
  target, or an ALIAS target is missing?
ParaView version is 5.8.1, I didn't rebuild it. CMake version is 3.16.3 now. ParaView compatibility issue? Should I try other ParaView version? With PV 5.2.0 (worked with Saturne 5.x), it doesn't configure with complain on Catalyst and PythonCatalyst modules (not available), at the same point of configure script. This behaviour is the same for Saturne 6.0.6 and 6.3.0.
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

Hello. Thanks for your support, Ivan. I solved previous problem enabling VTK's ViewsContext2D parameter and one more parameter (something about charts, don't remember exactly) in CCmake settings for Paraview (they can be edited in CMakeCache file). So common way to solve such issues is to search for corresponding parameters in ParaView settings (CMakeCache/CCMake), enable them, then configure/generate/make/install ParaView that Saturne links with(Catalyst version).
Saturne 6.3.0 has configured and built successfully with Catalyst, but the next problem appeared. When I start calculation with MPI, it just aborts execution, so I switched to single-thread for test as usual. And here is approximately what I saw:

Error loading /Programs/.../fvm_catalyst.so: /Programs/ParaView-5.8.1/.../libvtkWrappingPythonCore-pv5.8.so.1: undefined symbol: PyExc_ValueError

Also, I noticed that ParaView's ccmake configure finds Python 3.8 that is current Kubuntu-20 Python version but it fails to build if there is no Python 3.6 in my system (it needs Python3.6m headers).

Looks like I need special ParaView CCMake parameter set for new Saturne/ParaView versions to work with Catalyst (it's clear that older versions will not be supported soon and PV 5.2 is not compatible with modern Saturn already). I tried to find the tutorial on how to configure ParaView in Saturne 6.0 documentation (Install guide) but it seems that it's all different in newer PV versions. There is a new global "Catalyst mode" parameter, while many other related parameters are gone (I checked with my old configure parameter sets for PV 4.3 and 5.2). Would you, please, suggest some solution? May be there are no any Catalyst guides for new versions and the only way for me is to explore myself? Oops, it's getting scary :)
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Error building with Catalyst support

Post by Yvan Fournier »

Hello,

It looks like there is a mix of Python depedencies here, which could explain the library load issue.

Do you have multiple Python installs on your system ? Perhaps though environment modules or Conda or similar packages ?

Also, did you compile ParaView in a fresh (empty) build directory or do you have traces from an older install ? On most of our machines, we have Python 3.6, but I have compiled ParaView for code_saturne on an Arch Linux system with Python 3.8 and no more Python 3.6, so the dependency you observe seems surprising.

The documentation in the code_saturne might not be quite up to date, so here is a recommended option:

Code: Select all

```
$ cmake \
-DCMAKE_INSTALL_PREFIX=${INSTALL_PATH}_osmesa \
-DPARAVIEW_USE_QT=OFF \
-DPARAVIEW_USE_MPI=ON \
-DPARAVIEW_USE_PYTHON=ON \
-DPARAVIEW_INSTALL_DEVELOPMENT_FILES=ON \
-DOSMESA_INCLUDE_DIR=${MESA_INSTALL_PREFIX}/include \
-DOSMESA_LIBRARY=${MESA_INSTALL_PREFIX}/lib/libOSMesa.so \
-DVTK_OPENGL_HAS_OSMESA=ON \
-DVTK_USE_X=OFF \
${PARAVIEW_SRC_PATH}
```
If you are running on a local machine with a display, and not on a cluster, you do not need OSMesa, but can use the standard OpenGL library (I can send another example for that case).

Best regards,

Yvan
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

Hello and thanks for your reply. I tried clean ParaView build, from clean build/install directories but with saved CMakeCache. The same error...

Regarding Python versions. As I know, a bunch of Pythons is normal for an average Linux system. After Kubuntu upgrade that removed old packages (it was needed to continue upgrade) there was 2 Python versions: 2.7 and 3.8 in my system. Then I installed 3.6 because of ParaView make complain (it needed Python 3.6m headers). Is this "version zoo" needs special attention or should Saturne find appropriate version itself? Back i nthe old days, as I remember, I needed LD_PRELOAD particular Python library before starting Saturne. Now it doesn't help (I tried to preload Python3.8.so).

Looks like I need to dig some deeper into this issue. Error message says that PyExc_ValueError function is not found. Usually, unresolved externals arise while linking (building). When pointing to library function, you just have null pointer, you know... Is it a result of loading libvtkWrappingPythonCore-pv5.8.so.1 that is statically linked to Python? How can I trace the problem? Use strace? (I'm at work place now, will be able to check only when I get to my home station)
Also, PyExc_ValueError present in any of my Pythons (2.7, 3.6 and 3.8), I've checked headers. How can it be that the function is not found?
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Error building with Catalyst support

Post by Yvan Fournier »

Hello,

code_saturne should be able to find the correct version but multiplie versions and environnement variables combinations can lead to incorrect detection. For example if you have 3.8 headers in a default path but LD_PRELOAD for another version.

As I noted before, ParaView is ok with only Python 3.8 on my system. Are you sure when it complained about Python 3.6 headers you had installed Python 3.8-devel and not only Python 3.8 ?

Could you also post your config.log ? Are you building from a separate directory now ?

Regards,

Yvan
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

Yes, I build Saturne and Paraview out-of-source now. When I got Python 3.6m headers problem I checked than Python3.8-devel is already installed and latest version. I was sure I will see standard "Do you wan to install [Y/n]" but development package was already there... export LD_PRELOAD=... was set to Python 3.8 before "code_saturne run" and it didn't help.

I'll check for PATH and Python-related Paraview options and write back. Maybe I need some options from this thread: https://discourse.paraview.org/t/undefi ... error/5494
It seems to be Paraview problem and not related to Saturne directly. I think that one of desireable solutions is to include current ParaView sources with custom catalyst settings in Saturne source distribution (just ./configure ... with-catalyst - and voila - it build it's own actual "catalystic" ParaView modules), but it will take time for Saturne team to create this functionality and update with new ParaView versions.
Antech
Posts: 197
Joined: Wed Jun 10, 2015 10:02 am

Re: Error building with Catalyst support

Post by Antech »

Hello. I managed to launch Saturne 6.3.0 with catalyst with ParaView 5.8.1. PyExc_ValueError issue is solved with the following:
1. Set /usr/bin/python symlink to /usr/bin/python3
2. Set /usr/bin/python-config symlink to /usr/bin/python3-config
3. export LD_PRELOAD=$LD_PRELOAD:/usr/lib/x86_64-linux-gnu/libpython3.8.so (before starting Saturne in terminal)

But there is the next issue! When ParaView is ready to accept Catalyst connection and calculation runs, no output is made and an error message is shown in catalyst.log:

Code: Select all

Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "./Catalyst.py", line 117, in RequestDataDescription
    coprocessor.LoadRequestedData(datadescription)
  File "/Programs/ParaView-5.8.1/build-catalyst/lib/python3.8/site-packages/paraview/coprocessing.py", line 172, in LoadRequestedData
    datadescription.GetInputDescriptionByName(key).AllFieldsOn()
AttributeError: 'NoneType' object has no attribute 'AllFieldsOn'
I generetated fresh Catalyst.py with ParaView 8.5, so it should be compatible. What may cause this?
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Error building with Catalyst support

Post by Yvan Fournier »

Hello,

You are getting closer to success. This error typically occurs when the name of the writer and of the input under the Catalyst script do not match.

If you check the documentation on the GitHub wiki https://github.com/code-saturne/code_sa ... w-Catalyst, check step 1.

You can also edit the matching entry (which appears at 2 or 3 places in the catalyst script) directly in the Catalyst Python file instead, but is may be safer to do it from the interface if you have not done it before.

Regards,

Yvan
Post Reply