Homec4science

Coloc 2: fix flawed bin width calculation

Authored by Johannes Schindelin <johannes.schindelin@gmx.de> on Jan 29 2014, 20:33.

Description

Coloc 2: fix flawed bin width calculation

Math can be wonderful! And fun! And it can also be useful.

For example, when you want to determine a bin width such that, say,

value * binWidth + 0.5 < binCount

for *every* value in a given channel, but optimize such that the bin
width is still as large as possible, mathematical treatment of the
problem will lead to the equation:

maxValue * binWidth + 0.5 = binCount - eps

where eps is a small value (just small enough not to matter much, but
large enough to avoid falling between the machine precision's cracks).
Solving that equation for binWidth yields:

binWidth = (binCount - 0.5 - eps) / maxValue

which is a far cry from the previous

binWidth = binCount / (maxValue + 1)

It is even very easy to construct an example where the latter version
falls on its face: let binCount = 2^8 and maxValue = 2^16 - 1. Then the
flawed formula would yield binWidth = 2^8 and maxValue * binWidth + 0.5
would be exactly 2^-1 - 2^-8 *larger* than binCount.

This bug showed itself in the form of an ArrayOutOfBoundsException
reported by Ginger Pocock in the report at

http://fiji.sc/bugzilla/show_bug.cgi?id=699

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Details

Event Timeline

Johannes Schindelin <johannes.schindelin@gmx.de> committed R6269:8be1045451c8: Coloc 2: fix flawed bin width calculation (authored by Johannes Schindelin <johannes.schindelin@gmx.de>).Feb 11 2014, 02:29