Group Abstract Group Abstract

Message Boards Message Boards

GeoPosition is slow compared to DateObject

Przemyslaw K.
Posted 1 year ago

Hello,

I have a Dataset with around 42k observation points from various meteorological stations. I run a 

Query[All,DateObject[{#year,#month,#day}]]@ds

and it's around 0.25 seconds.

Now I run

Query[All,GeoPosition[{#latitude,#longitude}]]@ds

and it's a stunning 20 seconds! What's wrong with that? Is it that complex to create a geoposition object? I don't see this mentioned in the docs.

Best,
Przemek
POSTED BY: Przemyslaw K.
4 Replies
Sort By:
Przemyslaw K.
Przemyslaw K.
Posted 1 year ago

Ah, OK, I know, I had the lat/lon elements as Strings, Import from CSV didn't convert these... Thank you for your help!
POSTED BY: Przemyslaw K.

Unfortunately, this is a known bug.

In[23]:= Head@ParallelMap[f, Dataset[Range[5]]]

During evaluation of In[23]:= ParallelCombine::nopar1: Map[f][Dataset [<<5>>]] cannot be parallelized; proceeding with sequential evaluation.

Out[23]= Dataset

There is an internal report about the issue currently, and I've added you to the report so they know more people are concerned about it. Hopefully it will get fixed, and for now you'll apparently have to use normal or no parallelize, sorry...
POSTED BY: Eric Parfitt
POSTED BY: Jose Martin-Garcia
Przemyslaw K.
Przemyslaw K.
Posted 1 year ago

Apparently I can ParallelMap[GeoPosition,ds[All,{"latitude","longitude"}]//Normal but that puts me out of the Dataset world. I can't parallelize without the Normal, why?
POSTED BY: Przemyslaw K.
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard