Local case configuration: Date: Wed 30 Sep 2020 16:41:49 BST System: Linux 3.10.0-957.58.2.el7.csd3.x86_64 ( Machine: knl-e-3 Processor: model name : Intel(R) Xeon Phi(TM) CPU 7210 @ 1.30GHz Memory: 96378 MB Directory: /rds/project/t2_vol5/rds-t2-cs070/tall-3d-forced/couple/RESU/ncouple53 MPI ranks: 1024 OpenMP threads: 1 Processors/node: 256 I/O read method: standard input and output, parallel access I/O write method: standard input and output, serial access I/O rank step: 1 -------------------------------------------------------------------------------- Partitioning: preparing graph: 7.8 s SCOTCH_dgraphPart: 74.8 s wall clock time: 84.4 s -------------------------------------------------------------------------------- Halo creation times summary Interface creation: 0.544 s Extended connectivity creation: 0.533 s Halo creation: 0.294 s Total time for halo creation: 1.37 s -------------------------------------------------------------------------------- Total elapsed time for all gradient computations: 1962.946 s Summary of gradient computations for "Work array" (Iterative reconstruction): Number of calls: 2500 Number of iterations: 3 mean, 0 min., 4 max. Total elapsed time: 456.920 Summary of gradient computations for "pressure" (Iterative reconstruction): Number of calls: 1952 Number of iterations: 3 mean, 3 min., 4 max. Total elapsed time: 313.341 Summary of gradient computations for "temperature" (Iterative reconstruction): Number of calls: 1464 Number of iterations: 3 mean, 3 min., 4 max. Total elapsed time: 309.002 Summary of gradient computations for "velocity" (Iterative reconstruction): Number of calls: 2440 Number of iterations: 4 mean, 4 min., 4 max. Total elapsed time: 883.684 -------------------------------------------------------------------------------- Total elapsed time for linear equation system solvers: 1231.301 s Linear solver performance for fields ------------------------------------ Summary of resolutions for "Velocity" (field id 0) Solver type: Symmetric Gauss-Seidel Number of setups: 488 Number of calls: 488 Minimum number of iterations: 1 Maximum number of iterations: 2 Mean number of iterations: 1 Total setup time: 2.047 Total solution time: 53.094 Summary of resolutions for "Pressure" (field id 1) Solver type: Conjugate Gradient Preconditioning: Multigrid Number of setups: 488 Number of calls: 1036 Minimum number of iterations: 1 Maximum number of iterations: 90 Mean number of iterations: 6 Total setup time: 150.906 Total solution time: 969.339 Multigrid: V-cycle Coarsening: default Smoother: Symmetric Gauss-Seidel Coarsest level solver: Conjugate Gradient mean minimum maximum Number of coarse levels: 11 11 11 Number of cycles: 1 1 2 calls time Construction: 488 150.811 Resolution: 7296 883.086 mean minimum maximum Grid level 0: Number of rows: 79636800 79636800 79636800 Number of active ranks: 1024 1024 1024 Mean local rows: 77516 77516 77516 Mean local columns + ghosts: 85261 85261 85261 Mean local entries: 538653 538653 538653 Rows unbalance: 0.036 0.036 0.036 Columns + ghosts unbalance: 0.062 0.062 0.062 entries unbalance 0.036 0.036 0.036 Descent smoother iterations: 1 1 1 Grid level 1: Number of rows: 26487092 26487092 26487092 Number of active ranks: 1024 1024 1024 Mean local rows: 25796 25796 25796 Mean local columns + ghosts: 29566 29566 29566 Mean local entries: 188342 188342 188342 Rows unbalance: 0.047 0.047 0.047 Columns + ghosts unbalance: 0.101 0.101 0.101 entries unbalance 0.286 0.286 0.286 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 2: Number of rows: 8810852 8810852 8810852 Number of active ranks: 1024 1024 1024 Mean local rows: 8582 8582 8582 Mean local columns + ghosts: 10425 10425 10425 Mean local entries: 75941 75941 75941 Rows unbalance: 0.098 0.098 0.098 Columns + ghosts unbalance: 0.229 0.229 0.229 entries unbalance 0.372 0.372 0.372 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 3: Number of rows: 2940307 2940307 2940307 Number of active ranks: 1024 1024 1024 Mean local rows: 2841 2841 2841 Mean local columns + ghosts: 3821 3821 3821 Mean local entries: 28532 28532 28532 Rows unbalance: 0.229 0.229 0.229 Columns + ghosts unbalance: 0.653 0.653 0.653 entries unbalance 0.368 0.368 0.368 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 4: Number of rows: 973236 973236 973236 Number of active ranks: 1024 1024 1024 Mean local rows: 934 934 934 Mean local columns + ghosts: 1394 1394 1394 Mean local entries: 11968 11968 11968 Rows unbalance: 0.344 0.344 0.344 Columns + ghosts unbalance: 0.596 0.596 0.596 entries unbalance 0.328 0.328 0.328 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 5: Number of rows: 325516 325516 325516 Number of active ranks: 1024 1024 1024 Mean local rows: 310 310 310 Mean local columns + ghosts: 552 552 552 Mean local entries: 4329 4329 4329 Rows unbalance: 0.607 0.607 0.607 Columns + ghosts unbalance: 0.748 0.748 0.748 entries unbalance 0.543 0.543 0.543 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 6: Number of rows: 112170 112170 112170 Number of active ranks: 1024 1024 1024 Mean local rows: 102 102 102 Mean local columns + ghosts: 232 232 232 Mean local entries: 1493 1493 1493 Rows unbalance: 1.739 1.739 1.739 Columns + ghosts unbalance: 1.496 1.496 1.496 entries unbalance 1.052 1.052 1.052 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 7: Number of rows: 42234 42234 42234 Number of active ranks: 1024 1024 1024 Mean local rows: 34 34 34 Mean local columns + ghosts: 113 113 113 Mean local entries: 487 487 487 Rows unbalance: 4.795 4.795 4.795 Columns + ghosts unbalance: 2.955 2.955 2.955 entries unbalance 2.862 2.862 2.862 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 8: Number of rows: 19478 19478 19478 Number of active ranks: 1024 1024 1024 Mean local rows: 11 11 11 Mean local columns + ghosts: 57 57 57 Mean local entries: 160 160 160 Rows unbalance: 10.724 10.724 10.724 Columns + ghosts unbalance: 5.505 5.505 5.505 entries unbalance 6.851 6.851 6.851 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 9: Number of rows: 12225 12225 12225 Number of active ranks: 1024 1024 1024 Mean local rows: 4 4 4 Mean local columns + ghosts: 30 30 30 Mean local entries: 51 51 51 Rows unbalance: 17.177 17.177 17.177 Columns + ghosts unbalance: 8.661 8.661 8.661 entries unbalance 12.608 12.608 12.608 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 10: Number of rows: 9769 9769 9769 Number of active ranks: 1024 1024 1024 Mean local rows: 1 1 1 Mean local columns + ghosts: 16 16 16 Mean local entries: 16 16 16 Rows unbalance: 21.432 21.432 21.432 Columns + ghosts unbalance: 11.245 11.245 11.245 entries unbalance 17.377 17.377 17.377 Descent smoother iterations: 1 1 1 Ascent smoother iterations: 1 1 1 Grid level 11: Number of rows: 9546 9546 9546 Number of active ranks: 1024 1024 1024 Mean local rows: 1 1 1 Mean local columns + ghosts: 16 16 16 Mean local entries: 16 16 16 Rows unbalance: 21.741 21.741 21.741 Columns + ghosts unbalance: 11.269 11.269 11.269 entries unbalance 17.368 17.368 17.368 Iterations for solving: 160 0 500 calls time Grid level 0: build: 488 5.354 descent smoothe: 12737 374.756 restrict: 5441 4.262 prolong: 5441 3.968 Grid level 1: build: 488 76.636 descent smoothe: 5441 39.531 ascent smoothe: 5441 28.545 restrict: 5441 1.298 prolong: 5441 1.361 Grid level 2: build: 488 33.907 descent smoothe: 5441 31.766 ascent smoothe: 5441 13.593 restrict: 5441 0.553 prolong: 5441 0.509 Grid level 3: build: 488 15.332 descent smoothe: 5441 18.508 ascent smoothe: 5441 6.971 restrict: 5441 0.296 prolong: 5441 0.228 Grid level 4: build: 488 7.014 descent smoothe: 5441 12.403 ascent smoothe: 5441 4.042 restrict: 5441 0.189 prolong: 5441 0.126 Grid level 5: build: 488 3.398 descent smoothe: 5441 9.478 ascent smoothe: 5441 2.603 restrict: 5441 0.144 prolong: 5441 0.091 Grid level 6: build: 488 2.019 descent smoothe: 5441 6.978 ascent smoothe: 5441 1.966 restrict: 5441 0.120 prolong: 5441 0.076 Grid level 7: build: 488 1.449 descent smoothe: 5441 5.694 ascent smoothe: 5441 1.573 restrict: 5441 0.112 prolong: 5441 0.069 Grid level 8: build: 488 1.213 descent smoothe: 5441 4.939 ascent smoothe: 5441 1.391 restrict: 5441 0.106 prolong: 5441 0.067 Grid level 9: build: 488 1.132 descent smoothe: 5441 4.167 ascent smoothe: 5441 1.054 restrict: 5441 0.095 prolong: 5441 0.068 Grid level 10: build: 488 1.125 descent smoothe: 5441 3.582 ascent smoothe: 5441 0.659 restrict: 5441 0.089 prolong: 5441 0.096 Grid level 11: build: 488 1.925 solve: 5441 292.731 Summary of resolutions for "TempK" (field id 2) Solver type: BiCGstab Preconditioning: Jacobi Number of setups: 488 Number of calls: 488 Minimum number of iterations: 0 Maximum number of iterations: 1 Mean number of iterations: 0 Total setup time: 0.448 Total solution time: 5.182 -------------------------------------------------------------------------------- Mesh entity selections by criteria statistics: entity type evaluations time ----------------------------------------- cells 488 0.52806 interior faces 0 0.00000 boundary faces 11 0.00314 -------------------------------------------------------------------------------- Writing of "results" (EnSight Gold) summary: CPU time for meshes: 29.247 CPU time for variables: 25.995 CPU time forcing output: 0.000 Elapsed time for meshes: 29.264 Elapsed time for variables: 26.271 Elapsed time forcing output: 0.000 Writing of "" (time_plot) summary: CPU time for meshes: 0.005 CPU time for variables: 2.008 CPU time forcing output: 0.000 Elapsed time for meshes: 0.009 Elapsed time for variables: 2.025 Elapsed time forcing output: 0.000 -------------------------------------------------------------------------------- Instrumented all-to-all operations (using MPI_Alltoall and MPI_Alltoallv): Total: 9.45936 s, 160 calls Metadata exchange: 0.35710 s, 66 calls Data exchange: 4.95340 s, 94 calls -------------------------------------------------------------------------------- Code_Saturne IO files read: restart/auxiliary global: 2.10610 s, 5.891 KiB local: 17.45853 s, 3.946 GiB open: 21.20475 s, 1 open(s) restart/main global: 0.12592 s, 6.152 KiB local: 9.84175 s, 2.373 GiB open: 11.00660 s, 2 open(s) restart/mesh_input global: 21.53318 s, 3.016 KiB local: 33.52327 s, 12.624 GiB open: 33.64383 s, 2 open(s) Code_Saturne IO files written: checkpoint/auxiliary global: 4.30414 s, 4.859 KiB local: 38.89472 s, 8.661 GiB open: 43.47058 s, 1 open(s) checkpoint/main global: 0.19515 s, 3.215 KiB local: 13.75198 s, 2.967 GiB open: 13.99028 s, 1 open(s) partition_output/domain_number_1024 global: 0.00173 s, 0.500 KiB local: 1.62366 s, 303.790 MiB open: 1.63490 s, 1 open(s) -------------------------------------------------------------------------------- Calculation time summary: User CPU time: 4873.748 s System CPU time: 133.543 s Total CPU time: 5157792.754 s Elapsed time: 5046.710 s CPU / elapsed time 0.992 -------------------------------------------------------------------------------- Memory use summary: Total memory used: 922.952 GiB local minimum: 893.418 MiB (rank 364) local maximum: 1.110 GiB (rank 0) --------------------------------------------------------------------------------