"Updating" -- that is very useful. But I am still finding these waters to be murky.

In[1]:= ds = Dataset[{<|"a" -> 1, "b" -> 2|>, <|"a" -> 2, "b" -> 1|>}];

In[2]:= ds[[1, "a"]]

Out[2]= 1

In[3]:= (* this doesn't work the same way *)
ds[[1, "a"]] = 7

During evaluation of In[3]:= Set::partd: Part specification ds[[1,a]] is longer than depth of object. >>

Out[3]= 7

In[4]:= ds[1]["a"]

Out[4]= 1

In[5]:= ds[1]["a"] = 7

During evaluation of In[5]:= Set::write: Tag Dataset in a   1
b   2
1 level | 2elements \[SpanFromLeft]

[a] is Protected. >>

Out[5]= 7

This is a good example of one of the many things I find confusing in the new Association syntax. I have found it a useful addition, but every time I use it I feel like I'm having to learn it all over again. I rarely accomplish anything without a great deal of re-study and experimentation.

In many ways, the syntax tries to reuse familiar structures, but often up to a point. For example, here is how Part could be used to perform this task with a list structure. Up to a point, it works with a Dataset -- but then fails in the end:

In[1]:= (* With lists *)

In[2]:= list = {{1, 2}, {2, 1}};

In[3]:= list[[1, 2]]

Out[3]= 2

In[4]:= list[[1, 2]] = 7;

In[5]:= list

Out[5]= {{1, 7}, {2, 1}}

In[6]:= (* With a dataset *)

In[7]:= ds = Dataset[{<|"a" -> 1, "b" -> 2|>, <|"a" -> 2, "b" -> 1|>}]

Out[7]= Dataset[{__Association}]

In[8]:= ds[[1, "a"]]

Out[8]= 1

In[9]:= ds[[1, "a"]] = 7

During evaluation of In[9]:= Set::partd: Part specification ds[[1,a]] is longer than depth of object. >>

Out[9]= 7

In[10]:= ds

Out[10]= Dataset[{__Association}]

Dear Brad,

yes, I see what you are saying. I guess that what is even more problematic is this:

ByteCount[dsBig]
(*384000080*)

as compared to

ByteCount[arBig]
(*8000152*)
ByteCount[SparseArray@arBig]
(*8880*)

I am not convinced that Dataset is the best structure for what you want to achieve. I do not know whether ReplacePart actually makes a new copy of the data set, but I doubt it. Arrays or SparseArrays seem to be more useful for your problem. I think that "rule based" programming might be too slow for that.

Best wishes,

Marco

PS: This is a little bit faster still, particularly for large arrays:

arBig = SparseArray@ConstantArray[0, {10, 10}];
AbsoluteTiming[arBig[[2, 2]] = 100;][[1]]
arBig = SparseArray@ConstantArray[0, {100, 100}];
AbsoluteTiming[arBig[[2, 2]] = 100;][[1]]
arBig = SparseArray@ConstantArray[0, {1000, 1000}];
AbsoluteTiming[arBig[[2, 2]] = 100;][[1]]

Marco, thanks.

But I think the following code proves ReplacePart[] is doing something pretty time-consuming, like an underlying copy itself...

buildElement[value_Integer] := <| "A" -> 1, "B" -> value|>

dsBig = buildElement[#] & /@ Range[100]; AbsoluteTiming[ReplacePart[dsBig, {2, "B"} -> 5]] dsBig = buildElement[#] & /@ Range[10000]; AbsoluteTiming[ReplacePart[dsBig, {2, "B"} -> 5]] dsBig = buildElement[#] & /@ Range[1000000]; AbsoluteTiming[ReplacePart[dsBig, {2, "B"} -> 5]]

When I run it, the three returned timings are 0.000014, 0.000275 and 0.032637.

If, in place of a Dataset, I use an Array build with ConstantArray, performance is much better, but still degrades quite a bit with time....

arBig = ConstantArray[0, {10, 10}];
AbsoluteTiming[arBig[[2, 2]] = 100;]
arBig = ConstantArray[0, {100, 100}];
AbsoluteTiming[arBig[[2, 2]] = 100;]
arBig = ConstantArray[0, {1000, 1000}];
AbsoluteTiming[arBig[[2, 2]] = 100;]

Running the above, I get

{9.*10^-6, Null}
{0.000015, Null}
{0.006307, Null}

Respectively.

You should run some experiments and see what happens with Dataset.

If you use an Association then any modification of the association would be done very efficiently while still preserving immutability. You could still query this data structure using the new query syntax.

This video contains may be relevant. http://www.wolfram.com/broadcast/video.php?c=377&p=1&v=1240