Error during calculation

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Error during calculation

Post by frpageot »

Hello,

i did a quick calculation on CS3.0.6 with Unbuntu 14.04, and it ended with a error really quickely (after 13 iterations)

Error report :
Pile d'appels :
1: 0x7f7c7c351810 <ompi_op_base_sum_double+0x20> (libmpi.so.1)
2: 0x7f7c75b0ff18 <ompi_coll_tuned_allreduce_intra_recursivedoubling+0x4a8> (mca_coll_tuned.so)
3: 0x7f7c7c32aeee <PMPI_Allreduce+0x16e> (libmpi.so.1)
4: 0x7f7c7dd7699f <parsom_+0x2f> (libsaturne.so.0)
5: 0x7f7c7df59cf2 <vitens_+0x932> (libsaturne.so.0)
6: 0x7f7c7de33737 <resopv_+0x2247> (libsaturne.so.0)
7: 0x7f7c7de0c983 <navstv_+0x2e03> (libsaturne.so.0)
8: 0x7f7c7de45d80 <tridim_+0x43e0> (libsaturne.so.0)
9: 0x7f7c7dd275d7 <caltri_+0x2e87> (libsaturne.so.0)
10: 0x7f7c7dcfecf5 <cs_run+0xa35> (libsaturne.so.0)
11: 0x7f7c7dcfe1aa <main+0x14a> (libsaturne.so.0)
12: 0x7f7c7d2ecec5 <__libc_start_main+0xf5> (libc.so.6)
13: 0x400d29 <> (cs_solver)
Fin de la pile


and message from the GUI :

MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec has exited due to process rank 0 with PID 2854 on
node 1419-rdlabo exiting improperly. There are two reasons this could occur:

1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.

2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"

This may have caused other processes in the application to be
terminated by signals sent by mpiexec (as reported here).
--------------------------------------------------------------------------

****************************
Saving calculation results
****************************

solver script exited with status 1.

Error running the calculation.

Check code_saturne log (listing) and error* files for details.

Error in calculation stage.



From what I can understand it is linked with parsom and pallel calculations. I am using a simple user boundary conditions file enclosed to calculate a velocity on outlet just by giving a volumic flowrate taking in account the faces orientation.

I used this same file before with success but at this time it was running fine with 2 cores and it was running OK. Recently I upgraded my computer and I now use 16 processes, and it crashed. In a former calculation (not the same mesh), it failed after 1700 iterations.

Can someone help me to understand ?

Thanks.
Attachments
cs_user_boundary_conditions.f90
(14.5 KiB) Downloaded 247 times
Fred
Xubuntu 16.04 / CS5.0
Brian Angel

Re: Error during calculation

Post by Brian Angel »

Hello,

Can you post the listing and error files? There are also several things that I try when seeing this type of problem:

1) Run the calculation mono-processor to see if it goes beyond the point where the simulation stops (if the volume mesh cell count isn't too great). If it does then you might need to do 2) below. If not then the error message in the listing and error files will be more descriptive in terms of stating what the problem is.

2) Specifically select PT-Scotch or another option in the "partitioning type" menu in the "Calc Management > Performance tuning" interface instead of leaving it at default. Run the simulation to see if it goes beyond the point where the simulation stops.

3) If 1) and 2) above do not work, run the simulation for 12 iterations and look at the solution. Check to see if there are excessive values of velocity and / or pressure and / or turbulence. If so, you may need to look at mesh quality, relaxation, boundary conditions, etc.

Regards,

Brian Angel.
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Re: Error during calculation

Post by frpageot »

Hello,

Thanks for your answer Bryan. I just tried to modified the partitionning method and it goes beyond 13 iteration, it seems then that it is linked to this. I have now another problem, but I am used to solve this one, as it is generally linked to a meshing issue, or time step.

Nevertheless, I should have find it by myself.. Sorry for that.
Fred
Xubuntu 16.04 / CS5.0
Brian Angel

Re: Error during calculation

Post by Brian Angel »

Ok, good that it runs.

Regards,

Brian Angel.
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Re: Error during calculation

Post by frpageot »

Hello,

Unfortunately; I still have the problem, just postponed... It ran for 3029 iteration and stropped with the same error message. I enclosed the listing of this last simulation, and as you would notice there is no excessives values, neither on the pressure, nor on velocity. The only differences with the first run are that I change the partiionning method, and that I readuced the time step. Any help would be welcomùe. Thanks.
Attachments
listing.tar.gz
(1023.19 KiB) Downloaded 248 times
Fred
Xubuntu 16.04 / CS5.0
Brian Angel

Re: Error during calculation

Post by Brian Angel »

Hello,

I've just looked through your listing file. I think the problem is now due to the mass flow at the various inlet / outlet boundaries in the flow domain. Just above the error message at the end of the listing file there is a message stating that for the 892 boundary faces of the Free outlet boundary the flow is incoming for all of these faces, which the solver probably does not like. There is also an Undefined boundary with 122 faces which has a mass flow across it. What does this boundary correspond to? Are you expecting a flow through this boundary? If so, can you define this boundary as an inlet with a given mass flow rate? The same for the free outlet, do you know what mass flow rate is coming through this boundary? Hence, can you verify the set up for this case to make sure that it is as expected?

If there are one or more inlet / outlet boundaries for which you do not know the mass flow rate, you might consider using V4.0, which has a Free inlet/outlet boundary that allows flow to pass in our out of this boundary without having to specify anything in particular.

Regards,

Brian Angel.
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Re: Error during calculation

Post by frpageot »

Hello,

I used this configuration because I want to control outlet data and leave some free inlet opening. To do so I use a user boundary file given enclosed. Do you really think this is linked to this configuration ?
Attachments
cs_user_boundary_conditions.f90
(14.5 KiB) Downloaded 256 times
Fred
Xubuntu 16.04 / CS5.0
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Re: Error during calculation

Post by frpageot »

Hello,

Well, I changed my BC's, I switched to other partiionning methods and I still have the problem (still with 3.0.6). I do not think this is linked to my BCs because I made a (long) calculation with only 1 CPU, and it reached the maximum time step given (8000) without any problem. I tried CS4 RC1 but I still have problem to have something running. A lot of errors, not really liked to this problem, but I am not able to find right now what is the reason.

I really think this is related to partitionning. Anyone has an idea ?
Fred
Xubuntu 16.04 / CS5.0
Brian Angel

Re: Error during calculation

Post by Brian Angel »

Hello,

The only thing that I can think of is some incompatibility between v3.0.6 of Code_Saturne and v14.04 of Ubuntu. As previously mentioned, in using v2.0, v3.0.1 and v3.2.1 of Code_Saturne with v13.03 of Ubuntu I sometimes have the problem that you have encounter but can get around it.

Is it possible for you to install a more recent version of Code_Saturne such as v4.0 and try it to see what happens?

Regards,

Brian Angel.
frpageot
Posts: 85
Joined: Wed Mar 21, 2012 1:10 pm

Re: Error during calculation

Post by frpageot »

Hello Sorry for the late answer, I have not been noticed of the message. Yes, I will try with CS 4.0, but I installed RC1 few days agio I will try with this one. Or wait for definitive version. I'll let you know.
Fred
Xubuntu 16.04 / CS5.0
Post Reply