Homec4science

optional use of chunks in H5Dumper

Authored by frerot on May 23 2023, 14:03.

Description

optional use of chunks in H5Dumper

chunks=True has a sizeable performance hit when writing in parallel (writing
seems to happen sequentially with locks). Instead it is probably advisable to
write without chunks and use h5repack (hdf5-tools on Debian) to re-chunk files
if needed. Something to try would be chunks=model.shape in MPI, so chunks
correspond to partitionned data

Details

Committed
frerotMay 23 2023, 14:03
Pushed
frerotMay 23 2023, 14:08
Parents
rTAMAAS184b5e7bc082: removed redundant std::move
Branches
Unknown
Tags
Unknown