PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
Christopher
Posts: 36
Joined: Wed Feb 17, 2021 2:22 pm

Re: PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Post by Christopher »

Hi Yvan,

a good example is the laptop I used to test the calculation with CS 5.0.4. The specs were:

CPU: Intel Core i5-4210U (Dualcore at 1.70 Ghz)
Ram: 8 Gb DDR3

I ran a calculation with 250.000 time steps, a step size of 5e-7 s (variable), CFL <= 0.4. It took a week to finish. This laptop is totally unusable. However, there were no hurry because I was busy in other projects.

Now we set up a desktop PC using Ubuntu 20.04 and CS 7.1. The specs now are:

CPU: AMD Ryzen 9 3950X 16-Core Processor
RAM: 64 Gb DDR4 3200 Mhz

I ran the same calculation for comparison with only one difference. There were only 150.000 time steps. I started yesterday after lunch and expected the calculation to be finished this morning. Somehow strange to me appears the listings file. The time steps listed starts at 1, 2, 3 and so on until 10. Then it jumps to 100! After more than 12 hours running it seems that CS did only 100 steps of 150.000. All CPUs are on 100%. This would be even slower than with my crappy laptop.

Do you have any suggestions? Do I have to edit something in the performance settings? Till now I left everything default.

Kind regards,
Christopher
Attachments
CPU.png
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Post by Yvan Fournier »

Hello,

The jump in logged time step is probably normal : the first 10 time steps are always logged, then the other depend on the selected interval. So if you set the logging interval to 100 time steps you would have that behavior.

If in the comutation directory, you run "touch control_file" (creating an empty control file), the code_saturne should flush buffers within 1 iteration, so you can then check the timer_stats.csv to know at which iteration you are.

I also recommend comparing performance with 4, 8, and 16 CPU's.

Could you also post a "performance.log" of a finished computation (just run a computation over 10 time steps for a sample).

Best regards,

Yvan
Christopher
Posts: 36
Joined: Wed Feb 17, 2021 2:22 pm

Re: PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Post by Christopher »

Hi Yvan,

I have stopped the computation because the calculation stuck at time step 100 and no output under RESU => postprocessing was created. Just the .0001 files.

I wanted the solver to display each 100 steps in the logfiles and create an output each 1000 steps. It seem that there is some kind of dead loop. All CPU's run at 100%, but no output is created. Maybe that's just on my machine. However, no error message is displayed.

Kind regards,
Christopher
Attachments
run_solver.log
(69.02 KiB) Downloaded 116 times
performance.log
(1.37 KiB) Downloaded 119 times
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Post by Yvan Fournier »

Hello,

In any case, you should only be using 1 thread per MPI rank, not 32.

you have about 5000 cells per MPI rank, which is already pretty low. In some cases, using 2 to 4 threads per MPI rank allow pushing the limits on some large cases an high MPI rank counts, but here, you are probably spending more time syncing threads than computing, and you might have some issue with threading (OpenMP is not as well tested in most parts of code_saturne, and usually on a small number of threads).

Since you have 32 cores, you can use 32 MPI ranks (but the mesh is a bit small for that), or 16 MPI ranks, with 1 to 2 threads per rank, or even 8 MPI ranks (also with 1 to 2 threads per rank) for comparison. Depending on the memory bandwidth and cache, you will get anywhere from about 1x to 2x performance increase between 8 and 16 ranks.

Could you try again with these settings ?

Best regards,

Yvan
Christopher
Posts: 36
Joined: Wed Feb 17, 2021 2:22 pm

Re: PROBLEM IN THE BOUNDARY CONDITIONS USING COMPRESSABLE MODEL

Post by Christopher »

Dear Yvan and CS community,

as promised, I would like to share some validation data with you. I ran a simulation and took a measurement with a pitot tube and a thermocouple. Then I matched the two profiles in a plot. Attached to this post are two slides illustrating the simulation and experimental results in terms of velocity c (magnitude) and temperature T. As you can see, the simulation results reproduce the experimental data well. However, this was just a quick shot at lower temperatures than usual, but I am really excited and pleased with the results.

I would like to take this opportunity to thank everyone, especially Yvan Fournier, for their much appreciated help. CS 7.1 is a really good tool that also handles compressible flows.

By the way, I changed the number of threads from 32 to 2 and the computations run blazing fast. The Ryzen 9 with 16 cores is about 15 times faster than the laptop I was using before. So thank you very much for this tip as well.

Kind regards,
Christopher
Attachments
Temperature.pdf
(122.7 KiB) Downloaded 129 times
Velocity.pdf
(121.72 KiB) Downloaded 129 times
Post Reply