Running the model on HPC

[sophie Lu]

Hi all,

I am running Code_Saturne 8.2 coupled with SSH-Aerosol on an HPC system, and I consistently encounter a runtime error at the last time step of the simulation.

The case runs normally for most of the simulation, but it fails during the final time step. The job terminates with an error (see details below). I have attached the relevant error messages and runtime information from the Slurm output file for reference.

Thank you very much for your time and help.

Best regards,
Sophie

[Yvan Fournier]

Hello,

Do you use a specific tool or setting to obtain backtraces in the SLURM output ? By default, code_saturne installs its own exception handler, which logs backtraces (with less details) to error* files, but some libraries (at least PT-Scotch and ParaView, possibly others) can force their own handlers.

In any case, this seems to be a memory corruption issue. It might be in SSH-Aerosol, but I can't be sure of this. I would recommend running a (much) smaller version of your case for a few time steps on a local workstation with a debuggging tool such as Valgrind or with an AdressSanitizer build to try to locate the issue.

Also, if you have the possibility of upgrading to v9.0, than is recommended, as v8.2 is an intermediate feature release, which has been retired as of the v9.0 release. So although a bug may still occur in v9.0, its fix would be more relevant.

Best regards,

Yvan

[sophie Lu]

Hi Yvan,

Thank you for replying.

I have updated my model to 9.0

I have a question regarding the boundary condition treatment in the new version.

In the older version, after the model read the variables from the chemistry file, it was possible to handle them using iprofc(izone) = 1. However, I noticed that this option no longer exists in the new version.

When I run the new model, an error occurs:

Incoming flow detained for 259 out of 308 outlet faces
Error: incorrect or incomplete boundary conditions
======
At least one open boundary face declared as inlet (or outlet)
with prescribed velocity for which the value of "NO2"
has not been specified (Dirichlet condition).
The calculation will not be run.
Verify the boundary condition definitions.
First face with boundary condition definition error
(out of 106)
has boundary condition type 2, center (-1930, -451.169, 294.5)
Some boundary condition definitions are incomplete or incorrect.


However, after checking the listing file, I can see that the data are actually read successfully. What could be the reason for this behavior? Do all chemical species still need to be explicitly defined in the new version?

Thanks

[Yvan Fournier]

Hello,

The iprofc array was removed in commit 8edbd699(2023-03-29), with the following message:

[Atmo] Chemistry, remove iprofc array.

It was use to specify that we want to use chemistry profile for inlets.
This can already be done by specifying another value in
cs_user_boundary_conditions.

So I guess you need to apply the chemistry profile values directly to the matching boundary "rcodcl1" values for the given fields (which would be consistent with your error message).

Actually, checking the code from the same commit, this should be done automatically if ne boundary condition "rcodcl1" values have not otherwise been initialized in cs_user_boundary_conditions. One possibility is that the GUI does not handle this correctly and places values where it should not. To check this, I would check/print rcodcl1 values at the inlet for at least one of the species (ETOH or CH4 for example). The matching value should be 1E30 if i am not mistaken (to mark an uninitialized value), if the value is lower, than the automatic setting will not work properly.

Can you check this on your side (logging a few values in cs_user_boundary_conditions).

Did you also have the message about detained backflow (incoming flow) in older versions ? This is independent from the error you have, but can indicate similar or higher sensitivity to mesh quality.

Best regards,

Yvan