cs_matrix_assembler.cpp File Reference

Incremental or general construction of matrix structure. More...

#include "cs_defs.h"
#include <assert.h>
#include <math.h>
#include <stdlib.h>
#include <stdarg.h>
#include <stdio.h>
#include <string.h>
#include "bft_error.h"
#include "bft_mem.h"
#include "bft_mem_usage.h"
#include "cs_log.h"
#include "cs_rank_neighbors.h"
#include "cs_sort.h"
#include "cs_timer.h"
#include "cs_matrix.h"
#include "cs_matrix_assembler_priv.h"
#include "cs_matrix_assembler.h"

Include dependency graph for cs_matrix_assembler.cpp:

Functions
cs_matrix_assembler_t *	cs_matrix_assembler_create (const cs_gnum_t l_range[2], bool separate_diag)
	Create a matrix assembler structure. More...
cs_matrix_assembler_t *	cs_matrix_assembler_create_from_shared (cs_lnum_t n_rows, bool separate_diag, const cs_lnum_t row_idx[], const cs_lnum_t col_id[], const cs_halo_t *halo)
	Create a matrix assembler structure based on a given connectivity and associated halo structure. More...
void	cs_matrix_assembler_destroy (cs_matrix_assembler_t **ma)
	Destroy a matrix assembler structure. More...
void	cs_matrix_assembler_set_options (cs_matrix_assembler_t *ma, int flags)
	Set option flags for a given matrix assembler structure. More...
int	cs_matrix_assembler_get_options (const cs_matrix_assembler_t *ma)
	Return the option flags for a given matrix assembler structure. More...
void	cs_matrix_assembler_add_g_ids (cs_matrix_assembler_t *ma, cs_lnum_t n, const cs_gnum_t row_g_id[], const cs_gnum_t col_g_id[])
	Add entries to a matrix assembler structure. More...
void	cs_matrix_assembler_compute (cs_matrix_assembler_t *ma)
	Compute internal structures required by a matrix assembler. More...
const cs_gnum_t *	cs_matrix_assembler_get_l_range (const cs_matrix_assembler_t *ma)
	Return a pointer to local global row range associated with a matrix assembler. More...
const cs_halo_t *	cs_matrix_assembler_get_halo (const cs_matrix_assembler_t *ma)
	Return a pointer to the halo structure associated with a matrix assembler. More...
bool	cs_matrix_assembler_get_separate_diag (const cs_matrix_assembler_t *ma)
	Indicate if the matrix assembler is based on a separate diagonal. More...
cs_lnum_t	cs_matrix_assembler_get_n_rows (const cs_matrix_assembler_t *ma)
	Return the number of rows associated with a matrix assembler. More...
cs_gnum_t	cs_matrix_assembler_get_n_g_rows (const cs_matrix_assembler_t *ma)
	Return the global number of rows associated with a matrix assembler. More...
cs_lnum_t	cs_matrix_assembler_get_n_columns (const cs_matrix_assembler_t *ma)
	Return the number of columns associated with a matrix assembler. More...
const cs_lnum_t *	cs_matrix_assembler_get_row_index (const cs_matrix_assembler_t *ma)
	Return a row index associated with a matrix assembler. More...
const cs_lnum_t *	cs_matrix_assembler_get_col_ids (const cs_matrix_assembler_t *ma)
	Return a column ids associated with a matrix assembler. More...
void	cs_matrix_assembler_get_rank_counts (const cs_matrix_assembler_t *ma, int rc[4])
	Return info on the number of neighbor ranks a matrix assembler may communicate with. More...
void	cs_matrix_assembler_log_rank_counts (const cs_matrix_assembler_t *ma, cs_log_t log_id, const char *name)
	Log rank counts for a given matrix assembler. More...
cs_matrix_assembler_values_t *	cs_matrix_assembler_values_create (const cs_matrix_assembler_t *ma, bool sep_diag, cs_lnum_t db_size, cs_lnum_t eb_size, void *matrix, cs_matrix_assembler_values_init_t *init, cs_matrix_assembler_values_add_t *add, cs_matrix_assembler_values_add_g_t *add_g, cs_matrix_assembler_values_begin_t *begin, cs_matrix_assembler_values_end_t *end)
	Create and initialize a matrix assembler values structure. More...
void	cs_matrix_assembler_values_finalize (cs_matrix_assembler_values_t **mav)
	Finalize matrix assembler values structure. More...
void	cs_matrix_assembler_values_add (cs_matrix_assembler_values_t *mav, cs_lnum_t n, const cs_lnum_t row_id[], const cs_lnum_t col_id[], const cs_real_t val[])
	Add values to a matrix assembler values structure using local row and column ids. More...
void	cs_matrix_assembler_values_add_g (cs_matrix_assembler_values_t *mav, cs_lnum_t n, const cs_gnum_t row_g_id[], const cs_gnum_t col_g_id[], const cs_real_t val[])
	Add values to a matrix assembler values structure using global row and column ids. More...
void	cs_matrix_assembler_values_done (cs_matrix_assembler_values_t *mav)
	Start assembly of matrix values structure. More...

Detailed Description

Incremental or general construction of matrix structure.

The matrix assembler is intended to assist the building of matrices in general parallel conditions, assuming each parallel rank is assigned a given number of rows, in increasing order. Column elements may refer to rows assigned to other ranks. This allows a good mapping to external libraries such as PETSc and Hypre.

Moreover, in some cases, some global matrix elements may be computed on ranks to whom neither the row nor the column is assigned. This may happen for example when rows are assigned to mesh vertices, and a given cell provides a contribution to two vertices laying on parallel rank boundaries, and assigned to two different ranks, which are different from the cell's rank. Few such contributions are expected with a good partitionning, but they cannot always be avoided, or this would require complex partitioning constraints. libraries such as PETSc do not handle these cases, at least not efficiently (as the recommended preallocation required for local rows cannot be computed without the knowledge of those contributions), so the assembler should help manage these contributions in any case.

The addition of a given non-zero to a matrix structure may be done multiple times (for example when looping on cells with an internal loop on vertices), but for better performance (and memory usage), it is recommended to avoid the same info multiple times, as duplicates may be removed only during the computation stage).

Function Documentation

cs_matrix_assembler_add_g_ids()

void cs_matrix_assembler_add_g_ids	(	cs_matrix_assembler_t *	ma,
		cs_lnum_t	n,
		const cs_gnum_t	row_g_id[],
		const cs_gnum_t	col_g_id[]
	)

Add entries to a matrix assembler structure.

This function should be called by a single thread for a given assembler.

Parameters

[in,out]	ma	pointer to matrix assembler structure
[in]	n	number of entries
[in]	col_g_id	global column ids associated with entries
[in]	row_g_id	global row ids associated with entries

cs_matrix_assembler_compute()

void cs_matrix_assembler_compute

(

cs_matrix_assembler_t *

ma

)

Compute internal structures required by a matrix assembler.

The associated vector halo is also computed at this stage.

Parameters

[in,out]

ma

pointer to matrix assembler structure

This function should be called by a single thread for a given assembler.

cs_matrix_assembler_create()

cs_matrix_assembler_t * cs_matrix_assembler_create	(	const cs_gnum_t	l_range[2],
		bool	separate_diag
	)

Create a matrix assembler structure.

The associated matrix structure data will initially be empty, though the range of rows associated with the current rank must be defined immediately.

Parameters

[in]	l_range	global id range [min, max[ for local rank
[in]	separate_diag	if true, diagonal terms are handled separately

Returns

pointer to created matrix assembler structure

cs_matrix_assembler_create_from_shared()

cs_matrix_assembler_t * cs_matrix_assembler_create_from_shared	(	cs_lnum_t	n_rows,
		bool	separate_diag,
		const cs_lnum_t	row_idx[],
		const cs_lnum_t	col_id[],
		const cs_halo_t *	halo
	)

Create a matrix assembler structure based on a given connectivity and associated halo structure.

The assembler may not be later modified when built this way.

The connectivity arrays and halo will be shared by the caller, so their life cycle must be at least as long as the matrix assembler structure.

Parameters

[in]	n_rows	number fo local rows
[in]	separate_diag	if true, diagonal terms are handled separately
[in]	row_idx	matrix row index
[in]	col_id	matrix column indexes
[in]	halo	associated halo structure

Returns

pointer to created matrix assembler structure

cs_matrix_assembler_destroy()

void cs_matrix_assembler_destroy

(

cs_matrix_assembler_t **

ma

)

Destroy a matrix assembler structure.

Parameters

[in,out]

ma

pointer to matrix assembler structure pointer

cs_matrix_assembler_get_col_ids()

const cs_lnum_t * cs_matrix_assembler_get_col_ids

(

const cs_matrix_assembler_t *

ma

)

Return a column ids associated with a matrix assembler.

These ids relative to all columns of successive rows, with columns ordered by local id, so local columns appear first, distant columns after. Depending on the separate_diag option used when creating the matrix assembler, the columns may include the diagonal or not. The matching index can be obtained using cs_matrix_assembler_get_row_index.

Warning

the returned index is valid only as long as the matrix assembly structure exists.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

pointer to matrix structure row index

cs_matrix_assembler_get_halo()

const cs_halo_t * cs_matrix_assembler_get_halo

(

const cs_matrix_assembler_t *

ma

)

Return a pointer to the halo structure associated with a matrix assembler.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

pointer to halo structure

cs_matrix_assembler_get_l_range()

const cs_gnum_t * cs_matrix_assembler_get_l_range

(

const cs_matrix_assembler_t *

ma

)

Return a pointer to local global row range associated with a matrix assembler.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

pointer to local range

cs_matrix_assembler_get_n_columns()

cs_lnum_t cs_matrix_assembler_get_n_columns

(

const cs_matrix_assembler_t *

ma

)

Return the number of columns associated with a matrix assembler.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

number of columns associated with matrix assembler

cs_matrix_assembler_get_n_g_rows()

cs_gnum_t cs_matrix_assembler_get_n_g_rows

(

const cs_matrix_assembler_t *

ma

)

Return the global number of rows associated with a matrix assembler.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

number of rows associated with matrix assembler

cs_matrix_assembler_get_n_rows()

cs_lnum_t cs_matrix_assembler_get_n_rows

(

const cs_matrix_assembler_t *

ma

)

Return the number of rows associated with a matrix assembler.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

number of rows associated with matrix assembler

cs_matrix_assembler_get_options()

int cs_matrix_assembler_get_options

(

const cs_matrix_assembler_t *

ma

)

Return the option flags for a given matrix assembler structure.

Flags are defined as a sum (bitwise or) of constants, described in cs_matrix_assembler_set_options.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

option flags (sum of integer constants) used y this structure

cs_matrix_assembler_get_rank_counts()

void cs_matrix_assembler_get_rank_counts	(	const cs_matrix_assembler_t *	ma,
		int	rc[4]
	)

Return info on the number of neighbor ranks a matrix assembler may communicate with.

Parameters

[in]	ma	pointer to matrix assembler structure
[out]	rc	rank counts; the 4 values are: 0 number of communicating ranks for vector update (halo) 1 number of communicating ranks for matrix assembly 2 maximum number of communicating ranks for halo construction (assumed ranks algorithm) 3 maximum number of communicating ranks for matrix assembly determination (assumed ranks algorithm)

cs_matrix_assembler_get_row_index()

const cs_lnum_t * cs_matrix_assembler_get_row_index

(

const cs_matrix_assembler_t *

ma

)

Return a row index associated with a matrix assembler.

This index is a CSR type index relative to all columns, including or excluding the diagonal depending on the separate_diag option used when creating the matrix assembler. The matching column ids can be obtained using cs_matrix_assembler_get_col_ids.

Warning

the returned index is valid only as long as the matrix assembly structure exists.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

pointer to matrix structure row index

cs_matrix_assembler_get_separate_diag()

bool cs_matrix_assembler_get_separate_diag

(

const cs_matrix_assembler_t *

ma

)

Indicate if the matrix assembler is based on a separate diagonal.

Parameters

[in]

ma

pointer to matrix assembler structure

Returns

true if the associated structure has a separate diagonal, false otherwise

cs_matrix_assembler_log_rank_counts()

void cs_matrix_assembler_log_rank_counts	(	const cs_matrix_assembler_t *	ma,
		cs_log_t	log_id,
		const char *	name
	)

Log rank counts for a given matrix assembler.

Parameters

[in]	ma	pointer to matrix assembler structure
[in]	log_id	log type
[in]	name	name of this assembler

cs_matrix_assembler_set_options()

void cs_matrix_assembler_set_options	(	cs_matrix_assembler_t *	ma,
		int	flags
	)

Set option flags for a given matrix assembler structure.

When used, this function should be called before defining entries (in case of future option additions) and in any case before computing the final structure.

Flags are defined as a sum (bitwise or) of constants, described below:

• CS_MATRIX_DISTANT_ROW_USE_COL_IDX indicates that column indexes matching distant row data are computed and maintained, so the associated coefficient contributions can be added more efficiently.
• CS_MATRIX_DISTANT_ROW_USE_COL_G_ID indicates that columns global ids matching distant row data are maintained directly, so no lookup to a global id description of matrix columns is needed; this option is useful only when using an external library allowing incremental setting or accumulation of coefficients using global column ids, such as with PETSc's MatSetValues. When building matrices locally first (which includes most cases, whether internal matrices, or using library conversion or import functions such as PETSc's MatCreateMPIAIJWithArrays) this is slightly less efficient, as column ids will need to be matched to each row's columns a second time for those (exchanged) coefficients.
• CS_MATRIX_EXTERNAL_HALO indicates that we do not need to build an associated halo structure, as it will be built externally (i.e. by an external library such as PETSc, HYPRE, ...) using its own equivalent structures.

Parameters

[in,out]	ma	pointer to matrix assembler structure
[in]	flags	sum of matrix assembler flag constants (bitwise or)

cs_matrix_assembler_values_add()

void cs_matrix_assembler_values_add	(	cs_matrix_assembler_values_t *	mav,
		cs_lnum_t	n,
		const cs_lnum_t	row_id[],
		const cs_lnum_t	col_id[],
		const cs_real_t	val[]
	)

Add values to a matrix assembler values structure using local row and column ids.

If the matching matrix coefficients use a block structure, the values passed to this function should use the same block size.

Note also that if the matrix has different diagonal and extradiagonal block sizes, separate calls to this function should be used for both types of coefficients; in addition, in this case, diagonal coefficients should only be provided by the owning rank (this also impacts how the associated matrix assembler structure is defined).

This function may be called by different threads, as long those threads do not add contributions to the same rows (assuming caution is taken in the case of external libraries so that their builder functions have the same property).

Parameters

[in,out]	mav	pointer to matrix assembler values structure
[in]	n	number of entries
[in]	col_id	local column ids associated with entries
[in]	row_id	local row ids associated with entries
[in]	val	values associated with entries

cs_matrix_assembler_values_add_g()

void cs_matrix_assembler_values_add_g	(	cs_matrix_assembler_values_t *	mav,
		cs_lnum_t	n,
		const cs_gnum_t	row_g_id[],
		const cs_gnum_t	col_g_id[],
		const cs_real_t	val[]
	)

Add values to a matrix assembler values structure using global row and column ids.

If the matching matrix coefficients use a block structure, the values passed to this function should use the same block size.

Note also that if the matrix has different diagonal and extradiagonal block sizes, separate calls to this function should be used for both types of coefficients; in addition, in this case, diagonal coefficients should only be provided by the owning rank (this also impacts how the associated matrix assembler structure is defined).

This function may be called by different threads, as long those threads do not add contributions to the same rows (assuming caution is taken in the case of external libraries so that their builder functions have the same property).

Parameters

[in,out]	mav	pointer to matrix assembler values structure
[in]	n	number of entries
[in]	col_g_id	global column ids associated with entries
[in]	row_g_id	global row ids associated with entries
[in]	val	values associated with entries

cs_matrix_assembler_values_create()

cs_matrix_assembler_values_t * cs_matrix_assembler_values_create	(	const cs_matrix_assembler_t *	ma,
		bool	sep_diag,
		cs_lnum_t	db_size,
		cs_lnum_t	eb_size,
		void *	matrix,
		cs_matrix_assembler_values_init_t *	init,
		cs_matrix_assembler_values_add_t *	add,
		cs_matrix_assembler_values_add_g_t *	add_g,
		cs_matrix_assembler_values_begin_t *	begin,
		cs_matrix_assembler_values_end_t *	end
	)

Create and initialize a matrix assembler values structure.

The associated values will initially be set to zero.

This is a low-level function, which should be called by a simpler function (cs_matrix_assembler_values_init) which provides the necessary function pointers.

Warning

The matrix pointer must point to valid data when the selection function is called, so the life cycle of the data pointed to should be at least as long as that of the assembler values structure. In a similar manner, the life cycle of the associated matrix assembler must also be at least as long as that of the assembler values structure.

Parameters

[in]	ma	associated matrix assembly structure
[in]	sep_diag	true if diagonal terms are stored separately
[in]	db_size	diagonal block sizes
[in]	eb_size	extra-diagonal block sizes
[in,out]	matrix	untyped pointer to matrix description structure
[in]	init	pointer to matrix coefficients initialization function
[in]	add	pointer to matrix coefficients addition from local ids function
[in]	add_g	pointer to matrix coefficients addition from global ids function
[in]	begin	pointer to matrix coefficients assembly start function (optional)
[in]	end	pointer to matrix coefficients assembly end function (optional)

Returns

pointer to initialized matrix assembler structure;

cs_matrix_assembler_values_done()

void cs_matrix_assembler_values_done

(

cs_matrix_assembler_values_t *

mav

)

Start assembly of matrix values structure.

The call to this function is always optional, and indicates that assembly may begin. Calling it before cs_matrix_assembler_values_finalize may be useful when the underlying libraries can overlap communication (or other operations such as matrix trasformation) and computation.

Remarks

When values have been added to rows belonging to another rank, communication will occur here. Splitting this function could add another opportunity for overlap of computation and communication, but is not done as of 2016, as it would add complexity, with probably limited returns, as the effective asynchonous progress of MPI libraries is usually limited.

Parameters

[in,out]

mav

pointer to matrix assembler data structure

cs_matrix_assembler_values_finalize()

void cs_matrix_assembler_values_finalize

(

cs_matrix_assembler_values_t **

mav

)

Finalize matrix assembler values structure.

When this function returns, the assembler values structure has been destroyed, and the associated matrix is fully assembled (i.e. ready to use).

Parameters

[in,out]

mav

pointer to matrix assembler values structure pointer