Page 1 of 3
Library not found with custom compile file
Posted: Wed Sep 18, 2024 10:44 am
by fracenvi
Hello,
I'm trying to run a case where i have implemented a Pytorch-based neural network in the cs_user_extra_operations.c file and i need to link the libraries needed for having the full implementation (i.e., Libtorch). I made a custom cs_user_scripts.py where in the domain.compile_libs variables i correctly introduced the path to the libraries i need to include (i also checked with a separate file that those are the one needed for compiling the file that is missing the library). Do you know what else can be the problem in this regards?
The variable in question:
domain.compile_libs = '-lstdc++ -Wall -O3 -g -I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include -L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++'
Re: Library not found with custom compile file
Posted: Thu Sep 19, 2024 12:43 am
by Yvan Fournier
Hello,
I recommend separating compile and link flags:
domain.compile_cflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_cxxflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_fcflags = None
domain.compile_libs = "-L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++"
If this still doesn't work, can you post the compile.log file in the RESU/<run_id> directory.
Of course, do not forget to replace/check the actual /path/to/libtorch and replace it with the real value.
Best regards,
Yvan
Re: Library not found with custom compile file
Posted: Thu Sep 19, 2024 10:58 am
by fracenvi
Hello,
Thank you very much Yvan, it worked perfectly. However, now i have another issue: it seems to perfectly compile everything as in the compile.log file there are no errors (only warnings on unused variables), yet the calculation doesn't even start with the first iteration and it gives me no error message whatsoever. I attached the files that are produced by the code at the end of the computation.
Do you have any idea of what can possibly cause this behavior?
Thanks,
Francesco
Re: Library not found with custom compile file
Posted: Thu Sep 19, 2024 6:21 pm
by Yvan Fournier
Hello,
Do you get any info in the terminal in which the computation is launched (if interactive) or in the batch log files ?
Is the cs_solver file created ?
Best regards,
Yvan
Re: Library not found with custom compile file
Posted: Fri Sep 20, 2024 2:10 pm
by fracenvi
Hello,
In my terminal i have no relevant information.
In the only log that i have (compile.log), everything seems to be linked in the right way. In the following is the last line of the terminal:
Code: Select all
/usr/bin/cc -o /home/cenvinzf@coria.fr/workdir/full_coupling/case1/RESU/20240920-1202/cs_solver cs_user_extra_operations.o cs_meg_initialization.o cs_meg_boundary_function.o cs_user_initialization.o cs_user_parameters-base.o predict.o cs_f_user_physical_properties.o cs_user_parameter.o -lcs_solver -L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -lstdc++ -L/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib -L/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib -L/home/cenvinzf@coria.fr/Desktop/med/lib -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib -L/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib -O -Wl,-export-dynamic -fopenmp -lsaturne -lple -lcgns -lmedC -lhdf5 -lmpi -lz -ldl -L/home/cenvinzf@coria.fr/opencv_folder/lib -L/home/cenvinzf@coria.fr/cppflow/lib -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/x86_64-linux-gnu -L/usr/lib -L/lib/x86_64-linux-gnu -L/lib -L. -L/home/cenvinzf@coria.fr/opencv_folder/include -L/home/cenvinzf@coria.fr/cppflow/include -lgfortran -lquadmath -lm -Wl,-rpath -Wl,:/home/cenvinzf@coria.fr/libtorch/lib:/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib:/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib:/home/cenvinzf@coria.fr/Desktop/med/lib:/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/include/../lib:/home/cenvinzf@coria.fr/cppflow/lib/../lib:/home/cenvinzf@coria.fr/cppflow/include/../lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:.:/home/cenvinzf@coria.fr/opencv_folder/lib:/home/cenvinzf@coria.fr/opencv_folder/include:/home/cenvinzf@coria.fr/cppflow/lib:/home/cenvinzf@coria.fr/cppflow/include
When i try to compile first (using code_saturne compile) the cs_solver executable is produced but gives me the error:
./cs_solver: error while loading shared libraries: libhdf5.so.200: cannot open shared object file: No such file or directory
even if the HDF5 library is correctly linked from the moment the code is installed.
Any suggestion?
Regards,
Francesco
Re: Library not found with custom compile file
Posted: Fri Sep 20, 2024 10:08 pm
by Yvan Fournier
Hello,
Does the hdf5 file version match the one in the build configure command for code_saturne ? Do you have the same environment settings or loaded modules when installing and trying to run the code ?
Looks like there may beca mismatch here.
Best regards,
Yvan
Re: Library not found with custom compile file
Posted: Mon Sep 23, 2024 11:01 am
by fracenvi
Hello,
I tried to install once again the code using the same libraries and configuration options to make sure that the hdf5 versions (between configure and compile) match, as well as re-running my case but once again it gives me the "not found library" error.
Here it is the command line i used to run the compile step:
Code: Select all
cs_ann_2 compile --cflags='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu' --cxxflags='-I/home/cenvinzf@coria.fr/libtorch/include -I/home/cenvinzf@coria.fr/libtorch/include/torch/csrc/api/include' --libs='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib -lhdf5 -lstdc++'
Whereas this one is part of the options used in configuration phase:
Code: Select all
--with-hdf5=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2 --with-hdf5-include=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/include -with-hdf5-lib=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
I don't get where the missmatch can happen. I also posted the files needed to reproduce the bug.
Thanks and regards,
Francesco
Re: Library not found with custom compile file
Posted: Wed Oct 02, 2024 1:08 am
by Yvan Fournier
Hello,
Could you replace:
With:
Or add
In your launch environment (in you batch header, or in a [compute_prologue] section in your run.cfg file) ?
The "-L" option helps find the library at link time but not at execution time.
Best regards,
Yvan
Re: Library not found with custom compile file
Posted: Wed Oct 02, 2024 6:32 pm
by fracenvi
Hello,
I did manage to sort the issue related to the libhdf5.so.200 file missing, but now it seems to be giving another issue:
./cs_solver: error while loading shared libraries: libple.so.2: cannot open shared object file: No such file or directory
But i checked and it seems to be present in the relative lib folder.
I found a previous post talking about that same issue (
viewtopic.php?t=3052&start=20), but i can't manage to find a solution. I have already tried to move to v8.0.3.
Any suggestions?
Regards,
Francesco
Re: Library not found with custom compile file
Posted: Thu Oct 03, 2024 12:23 am
by Yvan Fournier
Hello,
Could you post the run_solver file in your execution directory ? Also, odes the code run normally without the additional libary ?
For PLE, you could also solve the issue in a manner similar to hdf5, but It would be interesting to first understand why you have this issue.
Best regards,
Yvan