Group Abstract Group Abstract

Message Boards Message Boards

1
|
11.9K Views
|
6 Replies
|
2 Total Likes
View groups...
Share
Share this post:

Retrieve a property for *all* stars with StarData?

Posted 10 years ago

We can easily get properties of individual stars with StarData, i.e. StarData[star, property]. But this is not very useful. I might as well look up the star in Wikipedia or elsewhere.

Such a database becomes useful not as an encylopedia, but as a comprehensive dataset which we can use to calculate various statistics and relationships between properties. For example, plot a Hertzsprung–Russel diagram.

How can I retrieve a property of all stars in the database?

This seems to work in principle, but in practice it takes such a long time that it is plainly unusable. It would probably take more than an hour to finish.

magn = StarData[EntityClass["Star", All], "AbsoluteMagnitude"];

In that time I could google up several easier to use databases, download the data, figure out how to import it to Mathematica, etc. And that's just one property.

Is there a better way then? Is there something wrong with the syntax I am using that causes this to be so slow? If not, then what is the point of all these *Data functions, given that several of them are practically unusable unless we're satisfied with looking at items (stars) one by one? Is there anyone on this forum who is able to make real practical use of this function? If yes, how?

StarData contains about 100,000 stars. That's not a lot. 100,000 floating point number take less than a megabyte of storage and arithmetic with such arrays typically takes less than a millisecond.

POSTED BY: Szabolcs Horvát
6 Replies
Posted 10 years ago
POSTED BY: Martin Lades

You can have that forms returned also with StarData directly, and the syntax is nice. Clearly better than AstronomicalData. It's the performance alone that ruins this.

POSTED BY: Szabolcs Horvát

I like the EntityValue method, since you can have it return a nested Association

EntityValue[StarData[], {"EffectiveTemperature", "AbsoluteMagnitude"}, "EntityPropertyAssociation"];

But it's running unbelievably slow for me today, the same timing as the StarData[.....] method.

POSTED BY: Jason Biggs

There was another like this one that found different behavior for StarData and AstronomicalData:

http://community.wolfram.com/groups/-/m/t/885567

The only result there was to use parallelisation; because each kernel than creates a connection and downloads data...

POSTED BY: Sander Huisman

Yes, that is what happens. I think that first it downloads all the stars, then it downloads the property for each.

Not all *Data functions are so slow. The new, Entity-based ones are much worse than the older ones.

It turns out that AstronomicalData is actually usable here

magn = AstronomicalData["Star", "AbsoluteMagnitude"];

This first downloads a big part (or all?) of the AstronomicalData dataset, which will be kept on the hard drive. The downloads completes in a couple of minutes. Once that is done, it's still not exactly fast, but it is unquestionably usable. It evaluates in about 22 seconds on my computer.

Compare that to StarData, which will re-download for every session and is plainly unusable.

POSTED BY: Szabolcs Horvát

Hi Szabolcs,

If I do:

magn = StarData[All, "AbsoluteMagnitude"];

I get something like this:

enter image description here

Which finishes in a minute or so (steps of 2500). But then it goes again downloading some more stuff but in steps of only 64 which takes indeed forever. Are you seeing the same? I agree, this should be within a second. But all the *Data functions in Mathematica have been notoriously slow since their introduction (V6?).

I agree, steps of 2500 (not to mention 64) seems ridiculously small for today's standards.

I honestly don't know what takes so long, is it the connection method, the protocol, the server, the databases?

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard