Page 1 of 1
Store surface normal vector to array in parallel simulation
Posted: Mon Jun 08, 2020 2:24 pm
by daniele
Hello,
I am trying to store the surface normal vector components in arrays (nx, ny, nz) and the norm (nn). Through the following code inside the cs_user_extra_operations.f90 I managed to do it, for a serial (1 processor) simulation:
Code: Select all
call getfbr('WALL1', nlelt1, lstelt)
do ilelt = 1, nlelt1
ifac = lstelt(ilelt)
nx(ilelt)=surfbo(1,ifac)
ny(ilelt)=surfbo(2,ifac)
nz(ilelt)=surfbo(3,ifac)
nn(ilelt)=surfbn(ifac)
enddo
Nevertheless, this does not work for a parallel simulation. I was wondering if the use of parbcr() is the right way to loop on all processors in this case..?
Thank you very much for your help.
Best regards,
Daniele
Re: Store surface normal vector to array in parallel simulation
Posted: Mon Jun 08, 2020 9:54 pm
by Luciano Garelli
Hello,
After partitioning the mesh each process will have a portion or not of the boundary WALL1, so each process has to create a local array and then you need to join all these array with a mpi gather.
You can take a look to the file cs_user_extra_operations-parallel_operations.f90 in the example directory and also to the doxygen doc
https://www.code-saturne.org/cms/site ... _8f90.html
I think that you need to do a cs_parall_allgather_r()
https://www.code-saturne.org/cms/sites/ ... ml#details
Regards,
Luciano
Re: Store surface normal vector to array in parallel simulation
Posted: Thu Jun 11, 2020 4:14 pm
by daniele
Hello,
Thank you very much for your help.
cs_parall_allgather_r() actually seems to be what I was looking for.
Nevertheless, I am a bit confused since cs_parall_allgather_r() requires the size of the local and global array as inlet parameters... but they are not known... And therefore this syntax:
Code: Select all
call getfbr('WALL1', nlelt1, lstelt)
do ilelt = 1, nlelt1
ifac = lstelt(ilelt)
nx(ilelt)=surfbo(1,ifac)
enddo
if (irangp.ge.0) then
call cs_parall_allgather_r(nlelt1, n_g_elts, nx, nx_par)
endif
gives me the following error:
../../../src/base/cs_parall.c:906: Fatal error.
Incorrect arguments to cs_parall_allgather_r:
sum of arg. 1 (n_elts) on ranks is not equal to arg. 2 (n_g_elts).
How can I know the value of n_g_elts, i.e. the size of the global array?
Thanks a lot.
Best regards,
Daniele
Re: Store surface normal vector to array in parallel simulation
Posted: Fri Jun 12, 2020 2:33 am
by Luciano Garelli
Hello,
I think that before call cs_parall_allgather_r() you have to do a parallel sum (parcpt) in order to sum "nlelt1" to get the total "n_g_elts".
After you get "n_g_elts" you can call to cs_parall_allgather_r()
Let me know if I'm right, because I never use it in CS, I have used something similar in plain MPI.
Regards,
Luciano
Re: Store surface normal vector to array in parallel simulation
Posted: Wed Jun 17, 2020 5:13 pm
by daniele
Dear Luciano,
You are right, it works correctly after doing a parcpt!
Thank you very much for your precious help.
Best regards,
Daniele