Message Boards Message Boards

[✓] Access an image in a System.Windows.Media.Imaging.CachedBitmap ?

GROUPS:

I am trying to use a camera with a .NET interface. The code below seems like it may be capturing an image into a System.Windows.Media.Imaging.CachedBitmap object.

Any ideas how to access this from within Mathematica? Eventually, I will need a solution that runs quickly. Saving to disk using .NET and then reading the file into Mathematica would functionally give me what I need, but will be too slow.

Needs["NETLink`"];
InstallNET[];
LoadNETAssembly["C:\\XIMEA\\API\\x64"];
LoadNETType["xiApi.NET.xiCam"];
myCam = NETNew["xiApi.NET.xiCam"];
myCam@OpenDevice[0];

(*Start acquisition *)
myCam@StartAcquisition[];

timeouts = 1000;
myCam@GetImage[myImage, timeouts]

myCam@StopAcquisition[];

myImage

Output of last statement is: « NETObject[System.Windows.Media.Imaging.CachedBitmap]»

POSTED BY: Jeff Burns
Answer
5 months ago

The camera manufacturer supplied this MATLAB code that does what I need to do in Mathematica.

disp('Color image format')
        myCam.SetParam(xiApi.NET.PRM.BUFFER_POLICY , xiApi.NET.BUFF_POLICY.UNSAFE);
        bmap=GetImage(myCam,1000);
        BytesPerPixel=bmap.Format.BitsPerPixel/8;
        NetArray=NET.createArray('System.Byte',W*H*BytesPerPixel);
        img_data=zeros(H,W,3,'uint8');
        for id=1:numOfFrames
                bmap=GetImage(myCam,1000);    
                bmap.CopyPixels(NetArray,BytesPerPixel*W,0);  
                img=uint8(NetArray);
                img_data(:,:,1)=transpose(reshape(img(3:BytesPerPixel:end),W,H));
                img_data(:,:,2)=transpose(reshape(img(2:BytesPerPixel:end),W,H));
                img_data(:,:,3)=transpose(reshape(img(1:BytesPerPixel:end),W,H));
                imshow(img_data);
                drawnow;
        end

I can get the image height, width, and bytes per pixel with the code below.

bytesPerPixel = myImage@Format@BitsPerPixel/8;
myCam@GetParam["height", imageHeight];
myCam@GetParam["width", imageWidth];

Still looking for how to read the image from memory.

POSTED BY: Jeff Burns
Answer
4 months ago

I'm not sure at which point you are stuck.

Are you not able to get the image data to Mathematica? It seems the manufacturer has a method to return an image (GetImage function). I suggest writing a simple C# class that would load the manufacturer's class. When you initialize that class, connect to your camera. Have a public function that does the GetImage call, which should return a CachedBitmap. It seems you could use the C# Bitmap method 'CopyPixels' to put the image data inside a .NET array. Have the function return this array. Back in Mathematica, load your new C# class. Then, run your function that gets the [next] image from the camera and return its pixel values as an array.

Once the pixel data is available in Mathematica, you just need to reshape it as is done in the Matlab code (there should be 3 channels, and you already know the image width and height). To create the Image in Mathematica, wrap Image around the data, specifying the ImageType as "Byte",

Image[reshape[-netArrayData-], "Byte"]
POSTED BY: Chad Knutson
Answer
4 months ago

Thanks for the input. I think .NET calls can be made in Mathematica to copy the array. How is a .NET array accessed in Mathematica?

POSTED BY: Jeff Burns
Answer
4 months ago

If the array has a meaningful value, you might try NETObjectToExpression[obj] to convert the NETObject (array) into nested lists of real numbers in Mathematica.

POSTED BY: Kevin Daily
Answer
4 months ago
NETObjectToExpression[myImage] 

returns « NETObject[System.Windows.Media.Imaging.CachedBitmap]»

Clearly I have to do something to convert the CachedBitmap.

MATLAB automatically converts arrays to .NET types as described hear. Does Mathematica do something similar?

This is some code that does not work but outlines what I think is needed.

In[]=  bytesPerPixel = myImage@Format@BitsPerPixel/8
Out[] =  4

In[] = netArray = NET@CreatArray["system.byte", myImage@Height myImage@Width  bytesPerPixel]
Out[] = NET[CreatArray["system.byte", 1.27473*10^7]]

In[] = myImage@CopyPixels[netArray, bytesPerPixel myImage@Width, 0]
NET: Improper arguments supplied for method named CopyPixels. 
Out[] = $Failed

Seems the key is setting up an array to accept the image.

POSTED BY: Jeff Burns
Answer
4 months ago

Thanks to help from Wofram this code works.

Needs["NETLink`"];

InstallNET[];
LoadNETAssembly["C:\\XIMEA\\API\\x64"];
LoadNETType["xiApi.NET.xiCam"];
LoadNETType["System.Runtime.InteropServices.GCHandle"];
LoadNETType["System.Runtime.InteropServices.GCHandleType"];

XIMeaCameraOpen[] := 
 Module[{cam},
  myCam = NETNew["xiApi.NET.xiCam"];
  myCam@OpenDevice[0];
  ]

XIMeaCameraSetup[] :=
  Module[
   {exposure = 100(* Set device exposure in micro seconds *),
    gain = 1 (* gain in decibels *), 
    tempImage,
    bytesPerPixel
    },

   myCam@SetParam["exposure", exposure];


   myCam@SetParam["gain", gain];

   (* Set image output format to RGB 32 bit - 3 / RGB24 - 2 / Mono8 - 
   0 / Mono16 - 1 / RAW8 - 5 / RAW16 - 6 *)
   myCam@SetParam["imgdataformat", 3];     (* RGB32 *)

   (*
   BUFF_POLICY class

   Buffer policy settings (can be safe,data will be copied to user/
   app buffer or unsafe,
   user will get internally allocated buffer without data copy).

   (1) BUFF_POLICYUNSAFE:User gets pointer to internally allocated \
circle buffer and data may be overwritten by device.

   (0) BUFF_POLICY.SAFE:
   Data from device will be copied to user allocated buffer or xiApi \
allocated memory.
   *)
   myCam@SetParam["buffer_policy", 0]; (* BUFF_POLICYSAFE *)

   (*myCam@SetParam["auto_wb", 1];  Automatic white balance *)
   myCam@SetParam["wb_kr", 1]; 
   myCam@SetParam["wb_kg", 1]; 
   myCam@SetParam["wb_kb", 1]; 

   If[ValueQ@buffer ,
     (* true this has been run before do nothing*) ,

    (* Start acquisition to find bytes per pixel*)
     myCam@StartAcquisition[];
     myCam@GetImage[tempImage, 1000];
     bytesPerPixel = tempImage@Format@BitsPerPixel/8;
     (* Stop acquisition *)
     myCam@StopAcquisition[];

     myCam@GetParam["height", imageHeight];
     myCam@GetParam["width", imageWidth];

    (* Create location to receive image data *)

     buffer = 
     NETNew["System.Byte[]", imageHeight imageWidth bytesPerPixel]
    ];

   myCam@SetParam["buffer_policy", 1]; (* BUFF_POLICYUNSAFE *)

   (* Set to tirgered operation *)

   (* 
   Triger Selector
   0, Invalid, Selects a trigger starting the capture of one frame
   1, Selects a trigger controlling the duration of one frame
   2, Selects a trigger starting the capture of the bursts of frames \
in an acquisition
   3, Selects a trigger controlling the duration of the capture of \
the bursts of frames in an acquisition
   4, Invalid, 
   Selects a trigger which when first trigger starts exposure and \
consequent pulses are gating exposure(active HI)
   5, Invalid, 
   Selects a trigger controlling the start of the exposure of one Frame
   6, Invalid, 
   Selects a trigger controlling the multi slope phase in one Frame \
(phase0\[Rule]phase1) or (phase1\[Rule]phase2)
   7, Invalid, Selects a trigger starting acquisition of first frame.
   *)
   myCam@SetParam["gpi_selector", 1]; 

   myCam@SetParam["gpi_mode", 1] ; (* GPI_TRIGGER, Off = 0, On =1 *)

   (*
   Trigger Source
   0, Camera works in free run mode
   1, External trigger (rising edge)
   2, External trigger (falling edge)
   3, Software(manual) trigger
   4, Specifies that the trigger is considered valid as long as the \
level of the source signal is high
   5, Specifies that the trigger is considered valid as long as the \
level of the source signal is low.
   *)
   myCam@SetParam["trigger_source", 1];
   ];


currentImageXIMea[camera_, timeouts_] :=
  Module[
   {
    rawImage
    },

   camera@GetImageByteArray[buffer, timeouts];

   rawImage = buffer // NETObjectToExpression;

   Image[
     ArrayReshape[rawImage, {imageHeight, imageWidth, 4}][[All, 
      All, {3, 2, 1}]],
    "Byte"]
   ];

XIMeaCameraOpen[];

XIMeaCameraSetup[];

myCam@StartAcquisition[];
currentImageXIMea[myCam, 10000]
myCam@StopAcquisition[];
POSTED BY: Jeff Burns
Answer
4 months ago

Group Abstract Group Abstract