Message Boards Message Boards

0
|
7092 Views
|
7 Replies
|
2 Total Likes
View groups...
Share
Share this post:

Efficient construction of Image[] object

Posted 9 years ago

Is there a way to import a string of bytes (from disk or memory) directly into an Image[] object, for efficient use of memory during object construction?

Background:

I have an image on disk in flat binary format. It is a sequence of unsigned 16-bit integers, stored row-major.

When I import that data into a Mathematica String, the string length is 968,000,000 with a ByteCount of 1,284,086,240.

When I import "UnsignedInteger16"s from that string, the ByteCount of the result increases to 3,872,000,144.

When I convert that set of UnsignedInteger16's into an image (first partitioned into rows of 22,000 samples), the ByteCount of the image drops to 968,000,400. That number looks like the original string length, with some additional metadata. Hence my question for a more direct approach to Image construction.

Thanks,

Vince

POSTED BY: Vincent Virgilio
7 Replies

You can use LibraryLink to construct the MImage object of type raw_t_ubit16.

POSTED BY: Piotr Wendykier

I see you mentioning strings, which is not a good idea I think. Have you tried something along the lines of:

data = Import[file,"UnsignedInteger16"];
data = Partition[data, width of your image];
Image[data,"Bit16"]

If it is a color image you might have to use ArrayReshape or nested Partition to get the right dimensions. But using this way it should remain a packed array and therefore memory efficient...

POSTED BY: Sander Huisman

Integer packed array needs 8 bytes to store a single value while Image of type "Bit16" needs only 2 bytes.

POSTED BY: Piotr Wendykier

Thank you. That's an attractive approach. I will pursue this as time permits.

My platform is Windows, so I wonder if there is an equivalent approach in .NET/Link? Perhaps something conveniently compiled from a block of C# in a string . . .

POSTED BY: Vincent Virgilio

Thank you. I am trying to avoid the larger memory footprint implied by this approach.

POSTED BY: Vincent Virgilio

I doubt that you can use C#. LibraryLink requires using C or C++. C and C++ work just fine on Windows. The Windows API is C based too.

You can use MathLink from C#, but transferring the data through MathLink will likely trigger expanding each integer to 64-bits, even if it's never greater than 255.

But have you tried the RawBitmap import format? http://reference.wolfram.com/language/ref/format/RawBitmap.html (I have not tested it.)

POSTED BY: Szabolcs Horvát

Thank you. That was an obvious miss on my part.

It works on a smaller image (11,000 x 11,000), but quits the kernel on a larger one(22,000 x 22,000) (My machine has 32GB RAM.) Even with the smaller image, it uses an excessive amount of memory. Where MMA used ~ 732MB to create the string containing the bitmap from disk, it used about another 4.8GB to import that string into an Image using "RawBitmap" format. ByteCount of the final Image[] object was ~ 242MB. I used the following syntax, which did create a plausible image.

ImportString[bitmapString, "RawBitmap", ImageSize -> {11000, 11000}, "BitDepth" -> 16]

POSTED BY: Vincent Virgilio
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract