Is ArrayReshape intrinsically slow ... ?

Questions
, , , ,
1

In the code attached the Part function is much, much faster than ArrayReshape when converting 1X4 arrays into 2X2 arrays. Clearly, ArrayReshape is a general function with lots of exception handling features, but it’s an order-of-magnitude slower that using Part on even mid-sized problems. Since ArrayReshape is a) exactly what I want, functionally, and b) more readable than using Part, is there anything I can do to remedy its poor performance? Compiling? Explicit parallelism? It’s not just relative to Part that ArrayReshape is slow; it seems slow relative to other operations on the same number of 1X4 or 2X2 arrays. I’ve demonstrated to myself that Part is faster than Transpose on, again, mid-sized problems, but, while noticeable, the difference is no where near as big as I’ve found here.

Context: I’m developing data mining software so converting arrays can be an inner loop of large-scale computations.

Thank you.

2

ArrayFlatten works faster if you can reshape the entire data at once:

In[39]:= testData = RandomInteger[{0, 100}, {176851, 4}];

In[45]:= AbsoluteTiming[
  res = ArrayReshape[testData, {First@Dimensions@testData, 2, 2}]][[1]]

Out[45]= 0.004762

In[46]:= AbsoluteTiming[
  respart = {#[[1 ;; 2]], #[[3 ;; 4]]} & /@ testData][[1]]

Out[46]= 0.094411

In[47]:= res == respart

Out[47]= True

If you must perform the reshape inside an inner loop, then the overhead of arguments checking, etc. has to become a burden—as you found out.

3

Wow!

Yet more loss in readability, but a huge(!) gain in performance. The latter is worth it because the array transformation is an inner loop.

I appreciate this very much.

– Mark