Lot of files "error_r****" and simulation stops every N iterations

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
Post Reply
ptessier2
Posts: 2
Joined: Sat Jul 18, 2020 11:35 am

Lot of files "error_r****" and simulation stops every N iterations

Post by ptessier2 »

Hi everyone !

When I run a simulation of a cross flow over a cylinder, it only runs till the 50 000th iteration (out of 500 000 in total) and then stops leaving plenty of files of the type "error_rXXXX". XXXX vary from '0000' to a high number like '1400', it depends on the number of processors. And every of those files are empty

I already succesfully run such simulations but I got this type of error for the last simulations. I have enough storage left. The job.err.log file is ok, and no error message are in the result directory can help me

I can provide any information needed.
Last edited by ptessier2 on Sun Jul 19, 2020 2:59 pm, edited 1 time in total.
Yvan Fournier
Posts: 4221
Joined: Mon Feb 20, 2012 3:25 pm

Re: Lot of files "error_r****" and simulation stops every N iterations

Post by Yvan Fournier »

Hello,

If the error* files are empty (they should contain backtraces) you might have sole info on errors on the batch output or error log file. If a single error_r* file us non-empty that would be interesting.

Do you have user functions ? A specific output at 50000 iterations ? Doea the end of the run_solver.log/listing file contain useful info ?

Regards,

Yvan
ptessier2
Posts: 2
Joined: Sat Jul 18, 2020 11:35 am

Re: Lot of files "error_r****" and simulation stops every N iterations

Post by ptessier2 »

Hello

Tank you for your quick reply !

I have indeed a user function which is the cause of my problem : the simulation runs correctly without it.
I join the user function file and the batch error file which mainly says "segmentation fault".
The user function should print in the listing file the mean velocity of control aeras around a cylindrical geometry.
Attachments
job_4423890.err.log
(2.53 KiB) Downloaded 229 times
cs_user_postprocess_mt.f90
(10.42 KiB) Downloaded 244 times
Yvan Fournier
Posts: 4221
Joined: Mon Feb 20, 2012 3:25 pm

Re: Lot of files "error_r****" and simulation stops every N iterations

Post by Yvan Fournier »

Hello,

Did you test this user subroutine on a smaller case on a workstation ? Ideally using a debug build ? This is the best practice, and usually make debugging much easier than on a large case with a batch system (even with good parallel debuggers such as DDT or TotalView, limiting "cluster debugging" to otherwise hard to reproduce cases is often a better use of resources).

Best regards,

Yvan
Post Reply