Hi Henrik

I looked into out of core processing. I don't think it will do what I need. To give a little more detail these images are stacks of images say 20 to 50 frames per image (intermediate frames in a composite image) with the new cameras now each intermediate frame is 8k 8bit we have went from processing 10s or low hundreds of these stacks (at 2 or 4 k) to 1000's (at 8k). Great for data but presents serious problem in processing ha ha. In the these are filtered and added up to give 3d electron density maps. One of thing that makes it difficult is that the container file is not natively supported by mathematica. You have to read it in as byte code and then separate metadata from the body data and so on. So for instance I use to be able to import the a whole stack into memory no they are 60 or so gigs per stack. So i need good tools for dividing the data into chunks/so i can stream it through. Obviously this no trivial task when talk about this amount of data. Any input or guides to processing this much data wih mathematica would be great. The image processing it self I can do (Cuda C/C++) . However managing this much data takes tom expertise I don't have. I need to know where to look.

Your problem seems to be a problem that will occur more and more. I heard during a wolfram event last year in the Netherlands a presenter say that Wolfram is looking into a streaming technique which would allow to compute on large image files. This would be introduced somewhere in v11. Perhaps if you don't get an answer here you could try the mathematica stackexchange site or log a call @ wolfram support?