Store surface normal vector to array in parallel simulation

Questions and remarks about code_saturne usage
Forum rules
Please read the forum usage recommendations before posting.
Post Reply
daniele
Posts: 148
Joined: Wed Feb 01, 2017 11:42 am

Store surface normal vector to array in parallel simulation

Post by daniele »

Hello,

I am trying to store the surface normal vector components in arrays (nx, ny, nz) and the norm (nn). Through the following code inside the cs_user_extra_operations.f90 I managed to do it, for a serial (1 processor) simulation:

Code: Select all

 call getfbr('WALL1', nlelt1, lstelt)
    do ilelt = 1, nlelt1
       ifac = lstelt(ilelt)
       nx(ilelt)=surfbo(1,ifac)
       ny(ilelt)=surfbo(2,ifac)
       nz(ilelt)=surfbo(3,ifac)
       nn(ilelt)=surfbn(ifac)
    enddo
Nevertheless, this does not work for a parallel simulation. I was wondering if the use of parbcr() is the right way to loop on all processors in this case..?

Thank you very much for your help.
Best regards,
Daniele
Luciano Garelli
Posts: 280
Joined: Fri Dec 04, 2015 1:42 pm

Re: Store surface normal vector to array in parallel simulation

Post by Luciano Garelli »

Hello,

After partitioning the mesh each process will have a portion or not of the boundary WALL1, so each process has to create a local array and then you need to join all these array with a mpi gather.

You can take a look to the file cs_user_extra_operations-parallel_operations.f90 in the example directory and also to the doxygen doc
https://www.code-saturne.org/cms/site ... _8f90.html

I think that you need to do a cs_parall_allgather_r()

https://www.code-saturne.org/cms/sites/ ... ml#details

Regards,
Luciano
daniele
Posts: 148
Joined: Wed Feb 01, 2017 11:42 am

Re: Store surface normal vector to array in parallel simulation

Post by daniele »

Hello,

Thank you very much for your help.

cs_parall_allgather_r() actually seems to be what I was looking for.

Nevertheless, I am a bit confused since cs_parall_allgather_r() requires the size of the local and global array as inlet parameters... but they are not known... And therefore this syntax:

Code: Select all

 call getfbr('WALL1', nlelt1, lstelt)
    do ilelt = 1, nlelt1
       ifac = lstelt(ilelt)
       nx(ilelt)=surfbo(1,ifac)
    enddo
 if (irangp.ge.0) then
 call cs_parall_allgather_r(nlelt1, n_g_elts, nx, nx_par)   
 endif
gives me the following error:

../../../src/base/cs_parall.c:906: Fatal error.
Incorrect arguments to cs_parall_allgather_r:
sum of arg. 1 (n_elts) on ranks is not equal to arg. 2 (n_g_elts).

How can I know the value of n_g_elts, i.e. the size of the global array?

Thanks a lot.
Best regards,
Daniele
Luciano Garelli
Posts: 280
Joined: Fri Dec 04, 2015 1:42 pm

Re: Store surface normal vector to array in parallel simulation

Post by Luciano Garelli »

Hello,

I think that before call cs_parall_allgather_r() you have to do a parallel sum (parcpt) in order to sum "nlelt1" to get the total "n_g_elts".

After you get "n_g_elts" you can call to cs_parall_allgather_r()

Let me know if I'm right, because I never use it in CS, I have used something similar in plain MPI.

Regards,

Luciano
daniele
Posts: 148
Joined: Wed Feb 01, 2017 11:42 am

Re: Store surface normal vector to array in parallel simulation

Post by daniele »

Dear Luciano,

You are right, it works correctly after doing a parcpt!
Thank you very much for your precious help.

Best regards,
Daniele
Post Reply