Efficient construction of Image[] object

Questions
,
1

Is there a way to import a string of bytes (from disk or memory) directly into an Image[] object, for efficient use of memory during object construction?

Background:

I have an image on disk in flat binary format. It is a sequence of unsigned 16-bit integers, stored row-major.

When I import that data into a Mathematica String, the string length is 968,000,000 with a ByteCount of 1,284,086,240.

When I import "UnsignedInteger16"s from that string, the ByteCount of the result increases to 3,872,000,144.

When I convert that set of UnsignedInteger16’s into an image (first partitioned into rows of 22,000 samples), the ByteCount of the image drops to 968,000,400. That number looks like the original string length, with some additional metadata. Hence my question for a more direct approach to Image construction.

Thanks,

Vince

2

You can use LibraryLink to construct the MImage object of type raw_t_ubit16.

3

I see you mentioning strings, which is not a good idea I think. Have you tried something along the lines of:

data = Import[file,"UnsignedInteger16"];
data = Partition[data, width of your image];
Image[data,"Bit16"]

If it is a color image you might have to use ArrayReshape or nested Partition to get the right dimensions. But using this way it should remain a packed array and therefore memory efficient…

4

Integer packed array needs 8 bytes to store a single value while Image of type “Bit16” needs only 2 bytes.

5

Thank you. That’s an attractive approach. I will pursue this as time permits.

My platform is Windows, so I wonder if there is an equivalent approach in .NET/Link? Perhaps something conveniently compiled from a block of C# in a string . . .

6

Thank you. I am trying to avoid the larger memory footprint implied by this approach.

7

I doubt that you can use C#. LibraryLink requires using C or C++. C and C++ work just fine on Windows. The Windows API is C based too.

You can use MathLink from C#, but transferring the data through MathLink will likely trigger expanding each integer to 64-bits, even if it’s never greater than 255.

But have you tried the RawBitmap import format? RawBitmap—Wolfram Documentation (I have not tested it.)

8

Thank you. That was an obvious miss on my part.

It works on a smaller image (11,000 x 11,000), but quits the kernel on a larger one(22,000 x 22,000) (My machine has 32GB RAM.) Even with the smaller image, it uses an excessive amount of memory. Where MMA used ~ 732MB to create the string containing the bitmap from disk, it used about another 4.8GB to import that string into an Image using “RawBitmap” format. ByteCount of the final Image[] object was ~ 242MB. I used the following syntax, which did create a plausible image.

ImportString[bitmapString, “RawBitmap”, ImageSize → {11000, 11000}, “BitDepth” → 16]