code saturne probe to syrthes source coupling

[filip2109]

Dear all,

I'm currently using CS 4.2 & Syrthes 4.3, on linux.
I'm modelling a coupled system between solid and fluid: depending on the temperature of the outlet, I'm trying to change a heat source in the solid part in syrthes.

So far I've successfully edited the user functions file in syrthes to make the source change with time.

I'm wondering if there is any way I can write a user function that allows me to do this coupling or is it something I have to modify inside the core code of each software?

My other idea (much much much more rough) would be to:
1) have a "tail" on the probe monitor file overwriting a file on the hard drive (or on a ramdisk)
2) read the file from syrthes' user function at each iteration and use the value from it.

Thank you very much for any ideas/insight on this matter
Kind regards
Filip

[Yvan Fournier]

Hello,

You'll probably need to edit user subroutines in both codes.

To avoid coupling through files, you can pass data between codes using the MPI communicators, but to to this, you need either to adapt the _exchange_sync function found in src/base/cs_syr4_coupling.c for Code_Saturne (and its equivalent in Syrthes), or possibly simpler, use a collective communication function such as a MPI_Allreduce with MPI_SUM from both codes, once per time step.

This should not be difficult to implement, but is slightly more advanced than what you will find in most user subroutines.

Regards,

Yvan

[filip2109]

Hi Yvan,

thank you very much for your reply.

I'm a bit at a loss as I've never worked on a project of this size, let alone 2 at the same time.
If you don't mind I'd like to ask you some advise/guidance on this.

First of all, I'm trying to make it right, to have a solution that is in line with the development of CS and might be used by someone else as well (Also, by learning to do it right on this relatively simple case, it should make implementing more complicated things right easier in the future).

So, back to the question.
I've noticed that the function that sends data from CS is cs_syr4_coupling_send_tf_hf, inside cs_syr4_coupling.c.
At the same time, in syrthes: syr_cfd_coupling_face_vars, in syr_cfd_coupling.c
Is this a good place to try implementing the communication? Or should I do it (properly) somewhere else?
Should I use the ple or just a normal MPI_Allreduce?

I've considered implementing it in the user functions such as cs_user_extra_operations.c and user_cond.c (syrthes), but as one is after the iteration and the other is before, I'm afraid of causing a mess with MPI. I guess this is a wrong place, right?

Another question I have is how to get the temperature at a specified probe. Looking at an example for cs_user_extra_operations.c I've thought of using this:

cs_real_t *t = CS_F_(t)->val;
cs_real_t probe_val = *t[cell_id];

I was thinking of getting the cellid from the output probe monitoring file and potentially put it into a user parameter file.
My worry is that all processes might not have access to the whole variable array... so what happens if a process tries to access a cell which is not in its part of interest?
Is there a correct way to access the probe value?
Also, is there a way to access the cell id of let's say probe1 at runtime?

I'm very sorry for the many questions, probably tedious and basic.
I thank you very much for all help,

Kind regards

Filip

[Yvan Fournier]

Hello,

OK, I quoted your various questions to try not to forget parts of the answer:

First of all, I'm trying to make it right, to have a solution that is in line with the development of CS and might be used by someone else as well (Also, by learning to do it right on this relatively simple case, it should make implementing more complicated things right easier in the future).

It is difficult to aim for generality here, as combinations may be very varied, but this might provide ideas for additional "coupling" user functions, and doing things right from the start is indeed a good approach, especially if you plan to do more complex setups in the future.

So, back to the question.
I've noticed that the function that sends data from CS is cs_syr4_coupling_send_tf_hf, inside cs_syr4_coupling.c.
At the same time, in syrthes: syr_cfd_coupling_face_vars, in syr_cfd_coupling.c
Is this a good place to try implementing the communication? Or should I do it (properly) somewhere else?

This should be a good place, depending on your chosen time scheme.
The communication step you mention is done at the beginning of the time step, based on values from the previous time step (so doing your operation here would be consistant with the 1st-order explicit scheme used for coupling).

Should I use the ple or just a normal MPI_Allreduce?

PLE allows for helping split the global MPI communicator and setting up MPI intercommunicators, and provides a synchronization helper function, but for additional communication, a simple MPI_Allreduce on an intracommunicator (or on MPI_COMM_WORLD in the most common case where you have one instance of Code_Saturne and one of Syrthes) is the simplest solution

I've considered implementing it in the user functions such as cs_user_extra_operations.c and user_cond.c (syrthes), but as one is after the iteration and the other is before, I'm afraid of causing a mess with MPI. I guess this is a wrong place, right?

Yes, you need to be careful with this, to avoid interlock.

Another question I have is how to get the temperature at a specified probe. Looking at an example for cs_user_extra_operations.c I've thought of using this:

cs_real_t *t = CS_F_(t)->val;
cs_real_t probe_val = *t[cell_id];

I was thinking of getting the cellid from the output probe monitoring file and potentially put it into a user parameter file.
My worry is that all processes might not have access to the whole variable array... so what happens if a process tries to access a cell which is not in its part of interest?
Is there a correct way to access the probe value?
Also, is there a way to access the cell id of let's say probe1 at runtime?

I definitely don't recommend storing a cell id in a file, or using an id directly in the data setup, as even without parallelism, a simple modification or renumbering of your mesh makes your setup obsolete. Defining a probe by its coordinates and searching for the matching cell at runtime is much safer, and yes, it is possible.

The mechanism for probes is currently being rewritten (to unify it with postprocessing meshes), but for now, it is defined in the Fortran part of the code, and to each probe is associated a (cell_id, rank_id) couple. For output, values are broadcast to rank 0. To see how this works, check src/base/ecrhis.f90 (to locate probes, the findpt function is used. It is horribly ugly and obsolete, locating the cell with closest center to a point, not necessarily exactly the cell containing that point, but still works).

I'm very sorry for the many questions, probably tedious and basic.
I thank you very much for all help,

Don't worry, most user's questions are more basic than this...

Regards,

Yvan