Multi-processes and variable initialization

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
Post Reply
ywan459
Posts: 20
Joined: Thu Nov 25, 2021 9:13 pm

Multi-processes and variable initialization

Post by ywan459 »

Hi,

I am using a user defined subroutine to read in the external spatially varying body force from external file. I was working well with 1 process. I notice that, the at the initialization stage the routine has been called the number of time which exactly equals to the number of processes. I am wondering whether this could destroy the mechanism I introduced here. I understand the model will be partitioned internally during the calculation. But not very sure about the initialization stage.

You may see my code go through all cells one-by-one, and assign an value read from a file to the corresponding cell. My question is: Are external values read multiple times, or the values are mismatched to the cell? I do not mind the data have been read and assigned for multiple times as long as the right value gave to the corresponding cell.

Code: Select all

void
cs_user_initialization(cs_domain_t     *domain)
{
  const cs_mesh_t *m = domain->mesh;
  FILE *fptrx;
  FILE *fptry;
  FILE *fptrz;
  char linex[256], liney[256], linez[256];
  float num_x, num_y, num_z;
  
  /* If this is restarted computation, do not reinitialize values */
  if (domain->time_step->nt_prev > 0)
    return;

  cs_field_t *fx = cs_field_by_name("x");
  cs_field_t *fy = cs_field_by_name("y");
  cs_field_t *fz = cs_field_by_name("z");
  
  if ((fptrx = fopen("fx.txt","r")) == NULL){
       printf("Error! opening file");
       // Program exits if the file pointer returns NULL.
       exit(1);
   }
  
  if ((fptry = fopen("fy.txt","r")) == NULL){
       printf("Error! opening file");
       // Program exits if the file pointer returns NULL.
       exit(1);
   }
  
  if ((fptrz = fopen("fz.txt","r")) == NULL){
       printf("Error! opening file");
       // Program exits if the file pointer returns NULL.
       exit(1);
   }
  
  if (fx != NULL) {
    for (cs_lnum_t cell_id = 0; cell_id < m->n_cells; cell_id++){
      fgets(linex, sizeof(linex), fptrx);
      sscanf(linex,"%f",&num_x);
      fx->val[cell_id] = num_x;
      
	  fgets(liney, sizeof(liney), fptry);
      sscanf(liney,"%f",&num_y);
      fy->val[cell_id] = num_y;
	  
	  fgets(linez, sizeof(linez), fptrz);
      sscanf(linez,"%f",&num_z);
      fz->val[cell_id] = num_z;
    }
  }
  
  fclose(fptrx);
  fclose(fptry);
  fclose(fptrz);
  
  printf("Initialization Done!!! \n");
   
}
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Multi-processes and variable initialization

Post by Yvan Fournier »

Hello,

In this case, the data will be mismatched. You could compare the number of the line read (or number of reads to do this implicitely) with the global cell numbers to avoid the mismatch, but this remains fragile.

How is the data in the file you read generated ?

Best regards,

Yvan
ywan459
Posts: 20
Joined: Thu Nov 25, 2021 9:13 pm

Re: Multi-processes and variable initialization

Post by ywan459 »

Hi Yvan,

Thanks! The data were generated through medcoupling interpolation in salome. I did store all data in the order of cell id, e.g. 1st line corresponds to 1st cell. I cannot use medcoupling with code_saturne.

How to get the global cell id in this subroutine? I guess I have to read the file lines according to the given ids. Actually the med file contains the field data at all cells. I have tried to access directly (without medcoupling), and of course, I have made no success.

I have also found there might be a potential defects in my source term routine. Please refer to the code below. In multi-process case, If my understanding is correct, the parameter *domain should be partitioned domain, and cs_field_by_name lines read the global fields (I have previously created in cs_user_parameters.c). This seems will mismatch data in multi-process cases. According to doc, cs_lnum_t should give us a local mesh/cell id, but will this id apply to field as well? Please note the for loop starts from 0.

My questions is how to access the partitioned domain and corresponding fields. I cannot find the information on the manual. Thanks in advance.

Code: Select all

void
cs_user_source_terms(cs_domain_t  *domain,
                     int           f_id,
                     cs_real_t    *st_exp,
                     cs_real_t    *st_imp)
{
  /* field structure */
  const cs_field_t  *f = cs_field_by_id(f_id);

  /* mesh quantities */
  const cs_lnum_t  n_cells = domain->mesh->n_cells;
  const cs_real_t  *cell_f_vol = domain->mesh_quantities->cell_vol;
  
  /* Density */
  const cs_real_t  *cpro_rom = CS_F_(rho)->val; 
  
  const cs_real_t ro0  = cs_glob_fluid_properties->ro0;
  
  cs_real_t *cvar_svbf_x = cs_field_by_name("x")->val;
  cs_real_t *cvar_svbf_y = cs_field_by_name("y")->val;
  cs_real_t *cvar_svbf_z = cs_field_by_name("z")->val;
  
  if (strcmp(f->name, "velocity") == 0) {
    cs_real_3_t    *_st_exp = (cs_real_3_t *)st_exp;
    cs_real_3_t    *_st_imp = (cs_real_3_t *)st_imp;
    /* apply source terms to all cells */
    for (cs_lnum_t i = 0; i < n_cells; i++) {
		

	  
      _st_exp[i][0] = cell_f_vol[i] * cvar_svbf_x[i];
      _st_exp[i][1] = cell_f_vol[i] * cvar_svbf_y[i];
      _st_exp[i][2] = cell_f_vol[i] * cvar_svbf_z[i];
	  
	  /* printf("calculation of cell = %d \n", i ); */
	  
    }
  }

}

Regards
Yikun Wang
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Multi-processes and variable initialization

Post by Yvan Fournier »

Hello,

If the data was generated using MEDCoupling, you should simply store it using the MED format and initialize the data using code_saturne's MEDCoupling wrappers, which will help you handle the parallel aspects.

There is a user initialization example using MEDCoupling in code_saturne's user examples. You can start from there. You simply need to make sure code_saturne is build using MEDCoupling support.

Best regards,

Yvan
ywan459
Posts: 20
Joined: Thu Nov 25, 2021 9:13 pm

Re: Multi-processes and variable initialization

Post by ywan459 »

Hi Yvan,

I really do not want to bother you about this topic again. But I cannot really find the solution. Following the last post, I did enable the mdecoupling with code_saturne through the configuration.

Command to configure:

Code: Select all

./configure --enable-medcoupling --with-salome=/home/App/salome/SALOME-9.8.0-native-UB20.04-SRC/BINARIES-UB20.04 --with-medcoupling=/home/App/salome/SALOME-9.8.0-native-UB20.04-SRC/BINARIES-UB20.04/MEDCOUPLING/ 
I have the following message:

Code: Select all

Configuration options:
 use debugging code: no
 use malloc hooks: no
 use graphical user interface: yes
 use long global numbers: yes
 Zlib (gzipped file) support: yes
 MPI (Message Passing Interface) support: no
 OpenMP support: yes
 OpenMP Fortran support: yes
 CUDA support: no
 BLAS (Basic Linear Algebra Subprograms) support: no
 ParMETIS (Parallel Graph Partitioning) support: no
 METIS (Graph Partitioning) support: no
 PT-SCOTCH (Parallel Graph Partitioning) support: no
 SCOTCH (Graph Partitioning) support: no
 CCM support: no
 HDF (Hierarchical Data Format) support: no
 CGNS (CFD General Notation System) support: no
 MED (Model for Exchange of Data) support: no
 MEDCoupling support: yes
   MEDLoader support: no
   ParaMEDMEM support: no
   MEDCoupling as plugin: no
 Catalyst (ParaView co-processing) support: no
 Melissa (in-situ statistics) support: no
 EOS support: no
 freesteam support: no
 CoolProp support: no
 PETSc support: no
 MUMPS support: no
 AMGX support: no
 Dynamic loader support: dlopen
 Auto load environment modules: no

 PdfLaTeX (document typesetting) available: no
 Doxygen (document generation) available: yes
   dot (Graphviz) available: yes
   MathJax enabled: no

The package has been configured. Type:
 make
 make install

To generate and install the Code_Saturne kernel
So you can see Medcoupling is enabled. I then did make and make install. However, when I run my test case of medcoupling, it showed the message:

Code: Select all

/home/App/code_saturne/code_saturne-7.0.2/src/base/cs_medcoupling_remapper.cxx:635: Fatal error.

Error: This function cannot be called without MEDCoupling support.
I am totally confused. This is the reason I am saying my medcoupling not working. :oops: :(
Yvan Fournier
Posts: 4070
Joined: Mon Feb 20, 2012 3:25 pm

Re: Multi-processes and variable initialization

Post by Yvan Fournier »

Hello,

Is seems MEDloader support is not present, so either there was an issue detecting this feature (maybe a missing --with-med or --with-hdf5 option, which might be necessary), or your MED build is a "lightweight" version without MED file support, whihc is needed here).

Best regards,

Yvan
Post Reply