Store surface normal vector to array in parallel simulation

[daniele]

Hello,

I am trying to store the surface normal vector components in arrays (nx, ny, nz) and the norm (nn). Through the following code inside the cs_user_extra_operations.f90 I managed to do it, for a serial (1 processor) simulation:

Code: Select all

 call getfbr('WALL1', nlelt1, lstelt)
    do ilelt = 1, nlelt1
       ifac = lstelt(ilelt)
       nx(ilelt)=surfbo(1,ifac)
       ny(ilelt)=surfbo(2,ifac)
       nz(ilelt)=surfbo(3,ifac)
       nn(ilelt)=surfbn(ifac)
    enddo

Nevertheless, this does not work for a parallel simulation. I was wondering if the use of parbcr() is the right way to loop on all processors in this case..?

Thank you very much for your help.
Best regards,
Daniele

[Luciano Garelli]

Hello,

After partitioning the mesh each process will have a portion or not of the boundary WALL1, so each process has to create a local array and then you need to join all these array with a mpi gather.

You can take a look to the file cs_user_extra_operations-parallel_operations.f90 in the example directory and also to the doxygen doc
https://www.code-saturne.org/cms/site ... _8f90.html

I think that you need to do a cs_parall_allgather_r()

https://www.code-saturne.org/cms/sites/ ... ml#details

Regards,
Luciano

[daniele]

Hello,

Thank you very much for your help.

cs_parall_allgather_r() actually seems to be what I was looking for.

Nevertheless, I am a bit confused since cs_parall_allgather_r() requires the size of the local and global array as inlet parameters... but they are not known... And therefore this syntax:

Code: Select all

 call getfbr('WALL1', nlelt1, lstelt)
    do ilelt = 1, nlelt1
       ifac = lstelt(ilelt)
       nx(ilelt)=surfbo(1,ifac)
    enddo
 if (irangp.ge.0) then
 call cs_parall_allgather_r(nlelt1, n_g_elts, nx, nx_par)   
 endif

gives me the following error:

../../../src/base/cs_parall.c:906: Fatal error.
Incorrect arguments to cs_parall_allgather_r:
sum of arg. 1 (n_elts) on ranks is not equal to arg. 2 (n_g_elts).

How can I know the value of n_g_elts, i.e. the size of the global array?

Thanks a lot.
Best regards,
Daniele

[Luciano Garelli]

Hello,

I think that before call cs_parall_allgather_r() you have to do a parallel sum (parcpt) in order to sum "nlelt1" to get the total "n_g_elts".

After you get "n_g_elts" you can call to cs_parall_allgather_r()

Let me know if I'm right, because I never use it in CS, I have used something similar in plain MPI.

Regards,

Luciano

[daniele]

Dear Luciano,

You are right, it works correctly after doing a parcpt!
Thank you very much for your precious help.

Best regards,
Daniele