code_saturne User's Forum

Hello.

I simulate relatively simple "pure aerodynamic" case with Saturne. Initial (preliminary) variant is based on ~5 mln cells tetra mesh generated with NetGen in Salome. Flow algorythm is Static, k-epsilon turbulence model. This case runs OK and shows acceptable argeement with CFX results (taking into account poor mesh resolution and different turbulence models).
But when I switch to another mesh with 20 millions of cells and prism layers the same variant in Saturne fails. I tried single core and 4 cores, the result is "solver failed" after "Computing geometric quantities". I can understand that there are mesh issues e.t.c. but I can't get what is exact problem and so what should I correct/change to make this case run. Listing doesn't contain any error info (only normal messages), "error" file is empty, text output is Saturne GUI only reports that error occured in solver. I didn't find any useful error info in RESU subdirectory both in serial (1 core) and parallel (4 cores) run. How can I find the problem in this case?

Sorry but I can't share the geometry or mesh because I do this work for my company and not allowed to share any confidential info. The domain is a HRSG (steam boiler flow part), but the fluid in this Saturne case is modelled with air (only density is changed with P and T). In CFX I configured "full-featured" gas mixture because CFX solver has built-in mixture thermophysical properties calculations
I attached an archive with logs from case's run directory.

Update
=====
I tried smaller mesh with ~8 millions of cells (tetras) and prism layers. The same behaviour: Saturne crashes after "Computing geometric quantities"
Another experiment was performed with 20-million mesh without layers. The reslts are crash at the beginning of first iteration (after outputting Diffusivity table, also without any particular error messages in error or listing files) on SST turbulence model with default (two-scale) wall treatment and normal calculation start with k-epsilon turbulence. I'm now waiting for k-epsilon case to perform some iterations to check wether it will run and to look at the pressure and velocity maximums (check stability).

OS: CentOS 6.7 (full update with yum).
Saturne: 4.0.2 from this site + Catalyst compilation settings (Catalyst is OK on working cases).

Thanks for you attention and suggestions.

Hello,

In the 20 million cell case, the crash could be due to insufficient memory, especially using Catalyst.

Could you try without Catalyst for this run ? I noticed than when crashes occur in that part of the code (or the Catalyst plug-in), there is less debug info, unless running under a debugger.

I'm more surprised about the 8 million cell mesh also crashing.

Also, try running the code outputting the "listing" directly to screen (advanced run parameters). This might cause you to lose the beginning of the listing, but should avoid the file flushing issues that may lead to some messages in the "listing" not being printed before the crash.

Regards,

Yvan

Yvan Fournier
Thanks for your reply. The "20-million" case without layers is now running and seem to be stable. We are now waiting for OS and Ansys update on a cluster, any freeware "add-ons" are planned later (at best) so I still have to use Saturne only on a workstation (with 32 GB RAM and 4 cores). Therefore I can't apply your suggestions right now but I hope I will do it later.
Running 20-million case suggests that the reason for failure is not a memory (now ~28 GiB of RAM and ~5 GiB of swap space are used). There are also no any slow-downs or huge memory consumptions before the crash (around a half of RAM is free at that moment). IMHO, this problem is 99% related with prism layers and/or SST turbulence, not with memory I tested my home Saturne installation on another case with layers, SST and momentum/energy sources and it was fine but it was another OS (Ubuntu 12.04), another Saturne (4.0.1 without Catalyst), mesh e.t.c.
So, if I get it right, now my primary "Saturne tasks" are to test problematic case without Catalyst redirecting text output to GUI window.

Update
======
I performed an experiment with 20-million cells mesh with prisms and SST model. Saturne 4.0.2 simple standard build (without Catalyst), output directed to GUI. The solver exits with error message about absent turbulence BC! It was just an incorrect setup: I switched to SST and in expression editor I had only k and epsilon but not omega. GUI said nothing about it. Solver also didn't output any useful error info while in Catalyst mode with output to listing. So I had no chances to find where my mistake is. Now I set hydraulic diameter and turbulence intensity 100% (it's a gas turbine outlet) and calculation still OK after ~30 minutes (first iteration in progress, 20 of 32 GiB of RAM is consumed).
Small update. After an hour of calculation it's still alive so I hope this problem is solved, thanks for suggestion.
Is it possible to make the solver more verbose on errors while in Catalyst mode with output to listing?

Hello,

Loss of information using Catalyst is probably due to Catalyst setting up its own exception traps, and overriding Code_Saturne's, but this was only recently noticed, so we have to check.

One solution that might work with both tools would be to save and restore exception handlers before and after Catalyst-related functions, but I'm not 100% sure.

I'll test that in the near future.

Regards,

Yvan

Yvan Fournier
Sorry, I made a mistake launching the calculation yesterday so it was running on a 20-million mesh without layers. I still think that the initial problem of this topic is solved (it was incorrect turbulence BC), but when I started the same variant on a mesh with layers it failed again. It turned out that the reason is a bad mesh. Salome generated the mesh with negative volumes. (IMHO, prism layers feature in Salome Platform is at early experimental stage and cannot be used on more or less realistic geometry for engineering).
The useful outcome from this issue is that Catalyst does not affect error messages in Saturne. I was able to "debug" the case on Catalyst setup just redirecting output to Saturne GUI instead of listing. The only obvious inconvinience of this way is impossibility to apply the "tasty" features like grep "v ||Velocity||".
I will now try to feed Saturne with Ansys mesh (Meshing tool) saved as CGNS. Fortunately, Ansys generate good meshes without thousands of user's interventions The drawback is that I will not verify any mesh-related issues in my CFX cases if I use the same mesh in Saturne.

Hello,

Good, this means the only issue is with "flushing" the log. In theory, this is forced when an error is caught, but between parts written from C and Fortran (actually, everything goes through Fortran after the initial steps, to avoid out-of order issues), flushing is probably not guaranteed.

In any case, due to the reconstruction-based nature of Code_Saturne gradients, the code does not like cells that are both thin and warped, which is often the case with boundary layer cells on hexahedral meshes. Triangular prisms are maybe a bit better here, but the high aspect ratio is still not great. So making sure wall layers are not "too" refined on curved surfaces, or increasing their "tangential" refinement in such areas to have better aspect rations might help.

Regards,

Yvan

Yvan Fournier
Thanks for suggestions about layers.
But in my case this is just an errorneous mesh with negative cell volumes, not aspect ratio or so (BTW, geometry is very simple in layered area). I already posted the question on Salome forum (although, unfortunately, it's unlikely to get a help there for this mesh generation issue because the activity at that forum is quite low).

Now calculation is started on Ansys mesh converted to CGNS in Workbench's Meshing module. It has layers on that CFX solver was stable and produced "good" results.

Update
======
After the weekend I checked Saturne case with Ansys mesh (16 millions of cells) and found out that in these 2 days it performed more than 700 iterations on 4 cores. The mesh is tetra, aspect ratio in first layer of prisms is very high (didn't measure exactly) and there was no any obvious problems with stability (pressure/velocity maximums, velocity field in Catalyst). SST is being used for the turbulence. So, at least in this variant, Saturne behaves stable with small CFL (1.0) on normal mesh (that cannot be said about prism layers algorythm in Salome that produces corrupted mesh, but the author plans to improve it and I hope that in newer versions of Salome layers will be more robust and correct).