Build Code Saturne using ACML as BLAS source

[edwilkes]

I have been working through the manual installation of Code Saturne 2.0.4 with Syrthes 4.0.0-beta on our HP Blade cluster with Centos 5.2 and Lustre 1.8, using Intel 11.1-059 compiler suite. All modules of CS including ncs have been compiled and installed, but I have been unable to use our ACML installation to add BLAS support. This is the only BLAS source available on this cluster and it works well with other programs which call BLAS.

The libraries for ACML are in /<TOP>/acml/4.4.0/ifort64_mp_int64/lib:

libacml_mp.a, libacml_mv.a, libcblas.a, libifcoremt_pic.a, libimf.a, libiomp5.a, libirc.a, libsvml.a

The includes are in /<TOP>/acml/4.4.0/ifort64_mp_int64/include:

acml.h, acml_mv.h, acml_mv_m128.h

I can easily add these into the configure command for NCS, but since ACML is not recognized as a BLAS type, the file names are not recognized. In particular, the configure script looks for a file called "cblas.h" which we do not have, so it fails at that point.

Can someone explain the proper way to link ACML into Code Saturne?

Thanks, Ed

[Yvan Fournier]

Hello,

I added ACML support in Code_Saturne 2.2 (which will be released in a few days; I am working on ironing out commments and documentation relative to user subroutine file renales, which is the last thing delaying the release). I was only able to test compilation/linking, not execution, as I currently only have direct access to intel processors (and colleagues who work with AMD-based machines have not submitted this issue yet).

So with version 2.2; ACML BLAS should be usable with the unmodified code.

For Code_Saturne 2.0, the only solution would be to adapt src/base/cs_blas.c (possibly based on what you can see for cs_blas.c in http://code-saturne.org/viewvc/saturne/trunk/src/alge) so as to call the correct headers.

I may be ranting here, but there are several BLAS specifications, as described on http://www.netlib.org/blas/blast-forum/. Sadly, none of the most common BLAS libraries follow the latest standard, and ACML does not even follow the "legacy BLAS" variant (relative to the recommended header name). If vendors are not capable of getting their act together (either following standards, or not pretending to follow them, and giving a justification why, which might be perfactly reasonable), it is very difficult for application codes to clean up their mess after them, so we are actually moving towards relying less on BLAS (and slighty more on compilers). Some theoreticians and even specialists might not consider this to be good practice, but for applications to follow standards, vendors have to do their part first...

Rant aside, using BLAS in optimized Code_Saturne builds usually only lead to 10% performance increase or less, as the only BLAS that are used are BLAS 1, which are memory-bandwith bound (rather than maximum FLOPS bound). GMRES may use BLAS 2, but it is a "backup solver" (in the sense that it may be more robust, but slower than Bi-CGstab for example), so if you are not able to get slightly improved performance using a vendor's BLAS, due to that vendor not following standards, tough luck to them if performance of a given code on their hardware reflects badly compared to their competitor's due to their header files not following the recommended standard and thus being recognized....

Reminder: on the bright side, you will have only a very small performance penalty not using BLAS. If you want to play with compiler optimization lavels (more risky, but potentially more interesting), add options to CFLAGS=... or FFLAGS=... when running Code_Saturne's "configure", or edit the congig/cs_auto_flags.sh file for your compiler to choose more optimal flags..

Best regards,

Yvan