== BigStitcher fusion noticed issues ==
It may happen that multi-tile images much bigger than RAM are not fused fast enough with BigStitcher.
There are several reasons to that:
* The dataset are loaded with [Soft References](https://www.baeldung.com/java-soft-references) by default, meaning that, as you stream the data from disk, as many cells as possible as kept in the RAM memory. The RAM starts to be freed when it is almost full. There are options in the VM (`-XX:SoftRefLRUPolicyMSPerMB=2500`) that aim at speeding up memory release, but it's not aggressive enough in practice. This leads to performance issues for big dataset because when the RAM is full, it becomes computationally costly to get rid of memory, slowing down the fusion process, and you may even get some out of memory errors, even though Soft Refs are supposed to guarantee that this does not happen.
* The fusion process, at least in the default BigStitcher way, iterates over all tiles for each fused pixel. This becomes particularly bad when many tiles are fused.
* If done with a Hdf5 format, the reading is not parallel -> but that's a minor issue compared to the two other ones.
Note : there is [BigStitcher spark](https://github.com/PreibischLab/BigStitcher-Spark/blob/main/README.md) which exists and which should be faster, but I did not test it. It's not integrated into Fiji (command line tool), and it works with 16 bits images only. I had to develop the next workflow in order to support OME-TIFF (QuPath) export with 8 bits, 16 bits, and rgb pixels, as well as making sure that big 2D planes work as well.
== BigDataViewer-Playground fusion workflow ==
In order to overcome these issues, here is a workflow that adresses the two main issues. Overall, the dataset that needs to be fused is opened with a bounded cache, meaning it will clear memory efficiently before the RAM gets full, and the fusion process occurs by block, where a filtering of the tiles overlapping with the block is first computed before fusing each block.
It is recommended to combine this workflow with the preview features of [tasks in Fiji](https://forum.image.sc/t/demo-and-proposal-new-progress-bars-for-fiji/64956) that you can activate by adding the custom update site `https://biop.epfl.ch/Fiji-Task-Demo/`.
Another thing : you can investigate directly the result of the fusion before you run it completely.
PROBLEM: There is a problem tough in the fuser implementation which can lead to overflow. Bright pixels may overflow in regions when the sumed value is above 65 536 -> that's bad.