Message Boards Message Boards

4 Replies
0 Total Likes
View groups...
Share this post:

Undocumented truncation of reals in displaying datasets

Posted 9 years ago
POSTED BY: Michael Helmle
4 Replies

I stumbled on the same. It is indeed a display thing. If you check the "Normal", the underlying value appears:

  {<|"Int" -> 499376, "Real" -> 499376.|>, <|"Int" -> 496899,   "Real" -> 496899.|>, <|"Int" -> 491584, 
  "Real" -> 491584.|>, <|"Int" -> 494629, 
  "Real" -> 494629.|>, <|"Int" -> 490165, "Real" -> 490165.|>}

{True, True, True, True, True}
POSTED BY: Olivier de Barsy

I see what you mean. That is indeed odd. (The correct terminology is rounding, but I probably should have paid closer attention to the output anyways.)

A Trace of the Dataset doesn't show any rounding going on, but I did notice that if you copy and paste the Dataset object, the Reals are represented as strings:

reals represented as strings

I think this issue may have something to do with the machine-number precision of the reals Dataset displays.

POSTED BY: Jesse Friedman
Posted 9 years ago
POSTED BY: Michael Helmle

I tried your code, and the numbers don't seem to be truncated.

numbers not truncated

POSTED BY: Jesse Friedman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
or Discard

Group Abstract Group Abstract