Library not found with custom compile file
Forum rules
Please read the forum usage recommendations before posting.
Please read the forum usage recommendations before posting.
Library not found with custom compile file
Hello,
I'm trying to run a case where i have implemented a Pytorch-based neural network in the cs_user_extra_operations.c file and i need to link the libraries needed for having the full implementation (i.e., Libtorch). I made a custom cs_user_scripts.py where in the domain.compile_libs variables i correctly introduced the path to the libraries i need to include (i also checked with a separate file that those are the one needed for compiling the file that is missing the library). Do you know what else can be the problem in this regards?
The variable in question:
domain.compile_libs = '-lstdc++ -Wall -O3 -g -I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include -L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++'
I'm trying to run a case where i have implemented a Pytorch-based neural network in the cs_user_extra_operations.c file and i need to link the libraries needed for having the full implementation (i.e., Libtorch). I made a custom cs_user_scripts.py where in the domain.compile_libs variables i correctly introduced the path to the libraries i need to include (i also checked with a separate file that those are the one needed for compiling the file that is missing the library). Do you know what else can be the problem in this regards?
The variable in question:
domain.compile_libs = '-lstdc++ -Wall -O3 -g -I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include -L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++'
-
- Posts: 4153
- Joined: Mon Feb 20, 2012 3:25 pm
Re: Library not found with custom compile file
Hello,
I recommend separating compile and link flags:
domain.compile_cflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_cxxflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_fcflags = None
domain.compile_libs = "-L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++"
If this still doesn't work, can you post the compile.log file in the RESU/<run_id> directory.
Of course, do not forget to replace/check the actual /path/to/libtorch and replace it with the real value.
Best regards,
Yvan
I recommend separating compile and link flags:
domain.compile_cflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_cxxflags = "-I/path/to/libtorch/include -I/path/to/libtorch/include/torch/csrc/api/include"
domain.compile_fcflags = None
domain.compile_libs = "-L/path/to/libtorch/lib -ltorch -ltorch_cpu -lc10 -lstdc++"
If this still doesn't work, can you post the compile.log file in the RESU/<run_id> directory.
Of course, do not forget to replace/check the actual /path/to/libtorch and replace it with the real value.
Best regards,
Yvan
Re: Library not found with custom compile file
Hello,
Thank you very much Yvan, it worked perfectly. However, now i have another issue: it seems to perfectly compile everything as in the compile.log file there are no errors (only warnings on unused variables), yet the calculation doesn't even start with the first iteration and it gives me no error message whatsoever. I attached the files that are produced by the code at the end of the computation.
Do you have any idea of what can possibly cause this behavior?
Thanks,
Francesco
Thank you very much Yvan, it worked perfectly. However, now i have another issue: it seems to perfectly compile everything as in the compile.log file there are no errors (only warnings on unused variables), yet the calculation doesn't even start with the first iteration and it gives me no error message whatsoever. I attached the files that are produced by the code at the end of the computation.
Do you have any idea of what can possibly cause this behavior?
Thanks,
Francesco
- Attachments
-
- RESU.tar
- (690 KiB) Downloaded 49 times
-
- Posts: 4153
- Joined: Mon Feb 20, 2012 3:25 pm
Re: Library not found with custom compile file
Hello,
Do you get any info in the terminal in which the computation is launched (if interactive) or in the batch log files ?
Is the cs_solver file created ?
Best regards,
Yvan
Do you get any info in the terminal in which the computation is launched (if interactive) or in the batch log files ?
Is the cs_solver file created ?
Best regards,
Yvan
Re: Library not found with custom compile file
Hello,
In my terminal i have no relevant information.
In the only log that i have (compile.log), everything seems to be linked in the right way. In the following is the last line of the terminal:
When i try to compile first (using code_saturne compile) the cs_solver executable is produced but gives me the error:
Any suggestion?
Regards,
Francesco
In my terminal i have no relevant information.
In the only log that i have (compile.log), everything seems to be linked in the right way. In the following is the last line of the terminal:
Code: Select all
/usr/bin/cc -o /home/cenvinzf@coria.fr/workdir/full_coupling/case1/RESU/20240920-1202/cs_solver cs_user_extra_operations.o cs_meg_initialization.o cs_meg_boundary_function.o cs_user_initialization.o cs_user_parameters-base.o predict.o cs_f_user_physical_properties.o cs_user_parameter.o -lcs_solver -L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -lstdc++ -L/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib -L/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib -L/home/cenvinzf@coria.fr/Desktop/med/lib -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib -L/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib -O -Wl,-export-dynamic -fopenmp -lsaturne -lple -lcgns -lmedC -lhdf5 -lmpi -lz -ldl -L/home/cenvinzf@coria.fr/opencv_folder/lib -L/home/cenvinzf@coria.fr/cppflow/lib -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/x86_64-linux-gnu -L/usr/lib -L/lib/x86_64-linux-gnu -L/lib -L. -L/home/cenvinzf@coria.fr/opencv_folder/include -L/home/cenvinzf@coria.fr/cppflow/include -lgfortran -lquadmath -lm -Wl,-rpath -Wl,:/home/cenvinzf@coria.fr/libtorch/lib:/home/cenvinzf@coria.fr/CS_ANN_coupling/7.0.0/lib:/home/cenvinzf@coria.fr/Desktop/saturne_lib/cgns/inst/lib:/home/cenvinzf@coria.fr/Desktop/med/lib:/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/lib/../lib:/home/cenvinzf@coria.fr/opencv_folder/include/../lib:/home/cenvinzf@coria.fr/cppflow/lib/../lib:/home/cenvinzf@coria.fr/cppflow/include/../lib:/home/cenvinzf@coria.fr/Desktop/OpenMPI/lib:.:/home/cenvinzf@coria.fr/opencv_folder/lib:/home/cenvinzf@coria.fr/opencv_folder/include:/home/cenvinzf@coria.fr/cppflow/lib:/home/cenvinzf@coria.fr/cppflow/include
even if the HDF5 library is correctly linked from the moment the code is installed../cs_solver: error while loading shared libraries: libhdf5.so.200: cannot open shared object file: No such file or directory
Any suggestion?
Regards,
Francesco
-
- Posts: 4153
- Joined: Mon Feb 20, 2012 3:25 pm
Re: Library not found with custom compile file
Hello,
Does the hdf5 file version match the one in the build configure command for code_saturne ? Do you have the same environment settings or loaded modules when installing and trying to run the code ?
Looks like there may beca mismatch here.
Best regards,
Yvan
Does the hdf5 file version match the one in the build configure command for code_saturne ? Do you have the same environment settings or loaded modules when installing and trying to run the code ?
Looks like there may beca mismatch here.
Best regards,
Yvan
Re: Library not found with custom compile file
Hello,
I tried to install once again the code using the same libraries and configuration options to make sure that the hdf5 versions (between configure and compile) match, as well as re-running my case but once again it gives me the "not found library" error.
Here it is the command line i used to run the compile step:
Whereas this one is part of the options used in configuration phase:
I don't get where the missmatch can happen. I also posted the files needed to reproduce the bug.
Thanks and regards,
Francesco
I tried to install once again the code using the same libraries and configuration options to make sure that the hdf5 versions (between configure and compile) match, as well as re-running my case but once again it gives me the "not found library" error.
Here it is the command line i used to run the compile step:
Code: Select all
cs_ann_2 compile --cflags='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu' --cxxflags='-I/home/cenvinzf@coria.fr/libtorch/include -I/home/cenvinzf@coria.fr/libtorch/include/torch/csrc/api/include' --libs='-L/home/cenvinzf@coria.fr/libtorch/lib -lc10 -ltorch -ltorch_cpu -L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib -lhdf5 -lstdc++'
Whereas this one is part of the options used in configuration phase:
Code: Select all
--with-hdf5=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2 --with-hdf5-include=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/include -with-hdf5-lib=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
Thanks and regards,
Francesco
- Attachments
-
- hdf5_bug.tar
- (5.02 MiB) Downloaded 44 times
-
- Posts: 4153
- Joined: Mon Feb 20, 2012 3:25 pm
Re: Library not found with custom compile file
Hello,
Could you replace:
The "-L" option helps find the library at link time but not at execution time.
Best regards,
Yvan
Could you replace:
With:-L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
Or add-L/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib -Wl,-rpath -Wl,/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
In your launch environment (in you batch header, or in a [compute_prologue] section in your run.cfg file) ?export LD_LIBRARY_PATH=/home/cenvinzf@coria.fr/Desktop/hdf5-200/inst2/lib
The "-L" option helps find the library at link time but not at execution time.
Best regards,
Yvan
Re: Library not found with custom compile file
Hello,
I did manage to sort the issue related to the libhdf5.so.200 file missing, but now it seems to be giving another issue:
Any suggestions?
Regards,
Francesco
I did manage to sort the issue related to the libhdf5.so.200 file missing, but now it seems to be giving another issue:
But i checked and it seems to be present in the relative lib folder. I found a previous post talking about that same issue (viewtopic.php?t=3052&start=20), but i can't manage to find a solution. I have already tried to move to v8.0.3../cs_solver: error while loading shared libraries: libple.so.2: cannot open shared object file: No such file or directory
Any suggestions?
Regards,
Francesco
-
- Posts: 4153
- Joined: Mon Feb 20, 2012 3:25 pm
Re: Library not found with custom compile file
Hello,
Could you post the run_solver file in your execution directory ? Also, odes the code run normally without the additional libary ?
For PLE, you could also solve the issue in a manner similar to hdf5, but It would be interesting to first understand why you have this issue.
Best regards,
Yvan
Could you post the run_solver file in your execution directory ? Also, odes the code run normally without the additional libary ?
For PLE, you could also solve the issue in a manner similar to hdf5, but It would be interesting to first understand why you have this issue.
Best regards,
Yvan