I hope somebody can help me in understanding some specific issues related to parallelization of user routines.
I am trying to store inside an array the face index of the faces of a given mesh boundary (called CYL_WALL).
Things are easy for a serial simulation, become more complex for parallel simulations.
These are the lines of my cs_user_extra_operation.f90 routine concerned:
Code: Select all
write(cond,"(A,1pe12.5,A,1pe12.5)") "CYL_WALL and y < ",0.1," and y > ",0.05
call getfbr(cond, nlelt, lstelt)
if (irangp.ge.0) then
do ilelt = 1, nlelt
ifac_local(ilelt) = lstelt(ilelt)
ifac = lstelt(ilelt)
surf_j(jj) = surf_j(jj)+surfbn(ifac)
enddo
call parrsm(N,surf_j)
nleltg = nlelt
call parcpt(nleltg)
write(nfecra,*) 'nlelt_getfbr', nleltg
ifac_local_r = real(ifac_local)
call cs_parall_allgather_r(nlelt,nleltg,ifac_local_r,ifac_par)
do ilelt = 1, nleltg
ifac = ifac_par(ilelt)
face_dz(jj,ilelt) = ifac
enddo
else !serial
do ilelt = 1, nlelt
ifac = lstelt(ilelt)
ifac_test(ilelt) = lstelt(ilelt)
face_dz(jj,ilelt) = ifac
surf_j(jj) = surf_j(jj)+surfbn(ifac)
enddo
endif
If a print the value of surf_j(jj), I find exactly the same value between the serial and parallel approaches, which seems to suggest that seleceted faces are the same for the approaches, but their index is different. Is that right?
A further question is: once the matrix "face_dz" is built, can I access in the user routine all elements of the matrix which contains face index values (ifac) that are spread on all different ranks, by simply:
Code: Select all
do ii = 1,N
do jj = 1,N
ifac = face_dz(ii,jj)
surf_j(ii) = surf_j(ii)+surfbn(ifac)
enddo
enddo
Or are the "ifac" of other ranks invisible to the rank on which the routine is run?
Hope my questions are clear.
Thank you very much for your help.
Best regards,
Daniele