Does anyone have a better solution for reading and writing half-precision float16? As far as I can find Mathematica does not support this at the moment or am I missing something?
Currently, I'm reading the data as "UnsignedInteger16" and then converting it to binary and calculating the floating point number, but this feels a bit cumbersome.
Int16ToFloat16 = Compile[{{int, _Integer, 0}},
Block[{bin, sign, imp, inf, exp, frac, signb, expb, fracb},
bin = IntegerDigits[int, 2, 16];
(*sign bit*)
signb = bin[[1]];
(*exponent bits*)
expb = bin[[2 ;; 6]];
(*fraction/significand bits *)
fracb = bin[[7 ;; 16]];
(*check exponent*)
imp = Max[expb];
inf = Min[expb];
(*Calculate sign exponent and signinficant*)
sign = (-1.)^signb;
exp = ({16, 8, 4, 2, 1} . expb) - 15.;
frac = ({512, 256, 128, 64, 32, 16, 8, 4, 2, 1} . fracb)/1024;
(*calculate the number*)
If[imp === 0(*exp = 00000*), sign (2.^-14.) frac,
If[inf === 1(*exp = 11111*), 0.(*make ininfity and NaN 0*),
sign (2.^exp) (1 + frac)]
]
], RuntimeAttributes -> {Listable}, RuntimeOptions -> "Speed"];
the file can be found here
list = BinaryReadList["positions.3.float16", "UnsignedInteger16"]; // AbsoluteTiming
Dimensions@list
Int16ToFloat16@list[[1 ;; 10000]]; // RepeatedTiming
Int16ToFloat16[list]; // AbsoluteTiming
The list contains around 40M numbers, reading the binary file takes less than a second but converting it to float more than 10.
For storage, I'm just writing it as a float32 but this doubles the file size and the precision is not needed.