Library not found with custom compile file

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
fracenvi
Posts: 29
Joined: Wed Jun 19, 2024 9:58 am

Library not found with custom compile file

Post by fracenvi »

Hello,

I'm trying to run a case where i have implemented a Pytorch-based neural network in the cs_user_extra_operations.c file and i need to link the libraries needed for having the full implementation (i.e., Libtorch). I made a custom cs_user_scripts.py where in the domain.compile_libs variables i correctly introduced the path to the libraries i need to include (i also checked with a separate file that those are the one needed for compiling the file that is missing the library). Do you know what else can be the problem in this regards?

The variable in question:
domain.compile_libs = '-lstdc++ -Wall -O3 -g -I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include -L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++'

Screenshot 2024-09-18 alle 10.46.47.png
Yvan Fournier
Posts: 4153
Joined: Mon Feb 20, 2012 3:25 pm

Re: Library not found with custom compile file

Post by Yvan Fournier »

Hello,

I recommend separating compile and link flags:

domain.compile_cflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_cxxflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_fcflags = None
domain.compile_libs = "-L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++"

If this still doesn't work, can you post the compile.log file in the RESU/<run_id> directory.

Of course, do not forget to replace/check the actual /path/to/libtorch and replace it with the real value.

Best regards,

Yvan
fracenvi
Posts: 29
Joined: Wed Jun 19, 2024 9:58 am

Re: Library not found with custom compile file

Post by fracenvi »

Hello,

Thank you very much Yvan, it worked perfectly. However, now i have another issue: it seems to perfectly compile everything as in the compile.log file there are no errors (only warnings on unused variables), yet the calculation doesn't even start with the first iteration and it gives me no error message whatsoever. I attached the files that are produced by the code at the end of the computation.

Do you have any idea of what can possibly cause this behavior?

Thanks,
Francesco
Attachments
RESU.tar
(690 KiB) Downloaded 45 times
Yvan Fournier
Posts: 4153
Joined: Mon Feb 20, 2012 3:25 pm

Re: Library not found with custom compile file

Post by Yvan Fournier »

Hello,

Do you get any info in the terminal in which the computation is launched (if interactive) or in the batch log files ?

Is the cs_solver file created ?

Best regards,

Yvan
fracenvi
Posts: 29
Joined: Wed Jun 19, 2024 9:58 am

Re: Library not found with custom compile file

Post by fracenvi »

Hello,

In my terminal i have no relevant information.

Screenshot 2024-09-20 alle 14.10.32.png

In the only log that i have (compile.log), everything seems to be linked in the right way. In the following is the last line of the terminal:

Code: Select all

/usr/bin/cc -o /home/cenvinzf@coria.fr/workdir/full_coupling/case1/RESU/20240920-1202/cs_solver cs_user_extra_operations.o cs_meg_initialization.o cs_meg_boundary_function.o cs_user_initialization.o cs_user_parameters-base.o predict.o cs_f_user_physical_properties.o cs_user_parameter.o -lcs_solver -L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -lstdc++ -L/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib -L/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib -L/home/cenvinzf@coria.fr/Desktop/med/lib -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib -L/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib -O -Wl,-export-dynamic -fopenmp -lsaturne -lple -lcgns -lmedC -lhdf5 -lmpi -lz -ldl -L/home/cenvinzf@coria.fr/opencv_folder/lib -L/home/cenvinzf@coria.fr/cppflow/lib -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/x86_64-linux-gnu -L/usr/lib -L/lib/x86_64-linux-gnu -L/lib -L. -L/home/cenvinzf@coria.fr/opencv_folder/include -L/home/cenvinzf@coria.fr/cppflow/include -lgfortran -lquadmath -lm -Wl,-rpath -Wl,:/home/cenvinzf@coria.fr/libtorch/lib:/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib:/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib:/home/cenvinzf@coria.fr/Desktop/med/lib:/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/include/../lib:/home/cenvinzf@coria.fr/cppflow/lib/../lib:/home/cenvinzf@coria.fr/cppflow/include/../lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:.:/home/cenvinzf@coria.fr/opencv_folder/lib:/home/cenvinzf@coria.fr/opencv_folder/include:/home/cenvinzf@coria.fr/cppflow/lib:/home/cenvinzf@coria.fr/cppflow/include
When i try to compile first (using code_saturne compile) the cs_solver executable is produced but gives me the error:
./cs_solver: error while loading shared libraries: libhdf5.so.200: cannot open shared object file: No such file or directory
even if the HDF5 library is correctly linked from the moment the code is installed.

Any suggestion?

Regards,
Francesco
Yvan Fournier
Posts: 4153
Joined: Mon Feb 20, 2012 3:25 pm

Re: Library not found with custom compile file

Post by Yvan Fournier »

Hello,

Does the hdf5 file version match the one in the build configure command for code_saturne ? Do you have the same environment settings or loaded modules when installing and trying to run the code ?

Looks like there may beca mismatch here.

Best regards,

Yvan
fracenvi
Posts: 29
Joined: Wed Jun 19, 2024 9:58 am

Re: Library not found with custom compile file

Post by fracenvi »

Hello,

I tried to install once again the code using the same libraries and configuration options to make sure that the hdf5 versions (between configure and compile) match, as well as re-running my case but once again it gives me the "not found library" error.

Here it is the command line i used to run the compile step:

Code: Select all

cs_ann_2 compile --cflags='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu' --cxxflags='-I/home/cenvinzf@coria.fr/libtorch/include -I/home/cenvinzf@coria.fr/libtorch/include/torch/csrc/api/include' --libs='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib -lhdf5 -lstdc++'

Whereas this one is part of the options used in configuration phase:

Code: Select all

--with-hdf5=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2 --with-hdf5-include=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/include -with-hdf5-lib=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
I don't get where the missmatch can happen. I also posted the files needed to reproduce the bug.

Thanks and regards,
Francesco
Attachments
hdf5_bug.tar
(5.02 MiB) Downloaded 40 times
Yvan Fournier
Posts: 4153
Joined: Mon Feb 20, 2012 3:25 pm

Re: Library not found with custom compile file

Post by Yvan Fournier »

Hello,

Could you replace:
-L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
With:
-L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib -Wl,-rpath -Wl,/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
Or add
export LD_LIBRARY_PATH=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
In your launch environment (in you batch header, or in a [compute_prologue] section in your run.cfg file) ?

The "-L" option helps find the library at link time but not at execution time.

Best regards,

Yvan
fracenvi
Posts: 29
Joined: Wed Jun 19, 2024 9:58 am

Re: Library not found with custom compile file

Post by fracenvi »

Hello,

I did manage to sort the issue related to the libhdf5.so.200 file missing, but now it seems to be giving another issue:
./cs_solver: error while loading shared libraries: libple.so.2: cannot open shared object file: No such file or directory
But i checked and it seems to be present in the relative lib folder.
Screenshot 2024-10-02 alle 18.29.52.png
I found a previous post talking about that same issue (viewtopic.php?t=3052&start=20), but i can't manage to find a solution. I have already tried to move to v8.0.3.

Any suggestions?

Regards,
Francesco
Yvan Fournier
Posts: 4153
Joined: Mon Feb 20, 2012 3:25 pm

Re: Library not found with custom compile file

Post by Yvan Fournier »

Hello,

Could you post the run_solver file in your execution directory ? Also, odes the code run normally without the additional libary ?

For PLE, you could also solve the issue in a manner similar to hdf5, but It would be interesting to first understand why you have this issue.

Best regards,

Yvan
Post Reply