Message Boards Message Boards

Computing with Entity classes involving large amounts of data

Posted 10 years ago
POSTED BY: David Reiss
2 Replies

Thanks Sean. This was largely what I thought was the case, but I wanted to make sure that there wasn't something basic and conceptual that I was missing. I don't have a particularly good model in my head yet for how the entire Entity-related landscape works and, for example, what functions in the Wolfram language can take entities as their arguments and return values. Some are documented, but some not yet, I think. And since most of this data resides on Wolfram's cloud servers and is downloaded as needed when requested from within Mathematica, it is not clear to me whether, once downloaded from the servers, it resides then on my local disk for quick subsequent access or is ephemeral in some way. This, in turn, has impact on the question of whether one writes programs that make use of cloud data and how fast those programs will run under different circumstances. I also don't have a sense of how fast data is in fact downloaded from the cloud. An example is the request for the names of all stars as in the example that I had above. In that case there are about 100,000 stars, and therefore, presumably the data for the names of all of those stars is probably not greater than several tens of megabytes. That, under other circumstances than this, one would naively expect to download rather quickly. But the query takes a great deal of time downloading and after waiting a dozen minutes I terminated it. So, this is an additional conceptual puzzle. But of course I realize that these data do not reside on the cloud servers as individual specific files that are downloaded whole cloth.

Just some random thoughts as I begin to experiment with these functionalities....

Thanks again, David

POSTED BY: David Reiss
POSTED BY: Sean Clarke
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract