Why are empty datasets not the same?

Questions
1

In debugging a program, I investigated how empty lists behave:

l1 = {}; l2 = Take[{}, UpTo[5]]; l3 = Take[{}, UpTo[0]];
Head /@ {l1, l2, l3}
l1 === l2 === l3

This gave the expected results:

{List, List, List}
True

But when I try what I think is analogous with datasets,

d1 = Dataset[{}]; d2 = Take[Dataset[{}], UpTo[5]]; d3 = 
 Take[Dataset[{}], UpTo[0]];
Head /@ {d1, d2, d3}
d1 === d2 === d3

I get

{Dataset, Dataset, Dataset}
False

Investing a bit further,

{SameQ [d1, d2], SameQ [d2, d3], SameQ [d1, d3]}

I got

{False, False, False}

This result was unexpected, especially d2 and d3 not being the same. Normal[] did not shed any extra light on what was happening:

Normal /@ {d1, d2, d3}
{{}, {}, {}}

Can someone explain why these datasets are not the same (it’s the source of the issue with my code)?
I can provide the background on what I’m trying to do with datasets if it’s relevant.

Thanks,

Andrew.

2

When in doubt, always look at the FullForm[]. Here’s an example from my session:

Dataset[{}] // FullForm
   Dataset[List[], TypeSystem`Vector[TypeSystem`UnknownType, TypeSystem`AnyLength],
           Association[Rule["ID", 211651877412011]]]

If you evaluate this in two separate cells, you’ll notice that the number (hash?) associated with "ID" changes with each evaluation.

3

Oh! That would explain it.

Thank you!

4

Hi Andrew,

Datasets have additional metadata and SameQ tests if the expressions are identical. Take a look at the results of

FullForm /@ {l1, l2, l3}

and

FullForm /@ {d1, d2, d3}

You have to compare the Normal

SameQ @@ Normal /@ {d1, d2, d3}
5

Thank you! Applying Normal before doing a test fixed the bug in my program.

6

As I understand it, the ID->something key value pair may be deleted in subsequent versions of the Wolfram Language as (a) users could not make use of it; and (b) it messed things up for users exactly as your question shows.