Message Boards Message Boards

0
|
5100 Views
|
10 Replies
|
4 Total Likes
View groups...
Share
Share this post:

Removing Entities with Missing values

Posted 8 years ago

Hi Very, very, part-time user of Mathematica, and i'm sure it'll show in my question.

i'm trying to get a very simple table of astronomical data together. Let's take a simple example

dwarves := EntityList[EntityClass["MinorPlanet", "DwarfPlanet" ]

I want to only return entries from the list the radius is not missing. This has proven ridiculously hard for me. I have a function i can experiment with:

filteredDwarves := DeleteCases[l, p_ /; shouldFilter[p]]

For shouldFilter, I have tried a million things, for example:

shouldFilter[planet_] := MinorPlanetData[planet, "Radius"] == Missing[_]

shouldFilter[planet_] := MinorPlanetData[planet, "Radius"] == _Missing

shouldFilter[planet_] := MinorPlanetData[planet, "Radius"] >Quantity[0.0,"miles"]

I even tried a ridiculous text thing:

shouldFilter[planet_] := Text[MinorPlanetData[planet, "Radius"] ]=="Missing"NotAvailable""

(pasted from a previous result) + other stuff

This ridiculous contraption WORKED:

shouldFilter[planet_] := Length[DeleteMissing[{MinorPlanetData[planet, "Radius"]}]] == 0

I've spent a long time looking at documentation + internet to no avail. Any help appreciated, what's the best way of doing this

Thanks!

POSTED BY: Steven Parry
10 Replies
filteredSubClass = Keys@DeleteMissing[
   EntityClass["MinorPlanet", "DwarfPlanet"]["Radius", "EntityAssociation"]
   ];

EntityValue[filteredSubClass, {"Entity", "Radius", "OrbitPeriod"}]

In this example filteredSubClass targets only "Radius", does not care if "OrbitPeriod" is Missing.

Posted 8 years ago

Hi

Perhaps i phrased it poorly. What I meant was the DeleteMissing example deletes entities where any one of the properties is missing, but i was trying to do was target 1 property in particular ("Radius"). So in an example like:

EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius", "OrbitPeriod"}]

All of the results have an orbit period, but only 3 have radius. It looks like there's no (simple) arguments i can supply to DeleteMissing that will only remove stuff with Radius missing and not OrbitPeriod. Hence my DeleteCases with a pattern {,Missing,_}. I suppose I could configure the list so that i had the property i was trying to match and then a sublist of other properties and then configure the arguments to DeleteMissing, but it seemed like DeleteCases is easier?

POSTED BY: Steven Parry

I think I understand what you mean, but let's take things slower. Isn't your example redundant because Radius and OrbitPeriod are Missing for the same entries?

EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius", "Density"}] // Column

enter image description here

POSTED BY: Vitaliy Kaurov

BTW you can click "Reply" to a specific comment so it gets indented peoper ly and it is easy to follow the conversation.

POSTED BY: Vitaliy Kaurov
Posted 8 years ago

Thanks for your responses! In your example - you've put Radius and Density, not Radius and OrbitPeriod,

But in general I was using DwarfPlanets as an example because it's a small data set and just using a few properties. My larger goal was wider dataset with more properties where i just wanted to filter out stuff that didn't have a radius.

POSTED BY: Steven Parry

Then I am still confused. OrbitPeriod is know for all entries - what does this change?

EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius", "OrbitPeriod"}] // Column

enter image description here

You do

DeleteMissing[EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius", "OrbitPeriod"}], 1, 2]

and it

filters out stuff that didn't have a radius

as you need. What am I missing?

POSTED BY: Vitaliy Kaurov
Posted 8 years ago

Hi

Thanks to both of you for explaining the error of my ways. In the first example - DeleteMissing, I was totally unaware i could do this with the entities, and also found out i could stack the properties together - which is great. However, I wasn't able to extend DeleteMissing for that scenario:

DeleteMissing[EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius",  "Density"}], 1, 2]

would also filter out missing Densities. However after playing around i found i could use DeleteCases with a pattern so something like:

DeleteCases[ EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius",   "Density"}], {_, _Missing, _}]

Thanks to Christopher for pointing out what i was missing - great stuff, appreciate it.

POSTED BY: Steven Parry

I am happy to see someone is trying to dig deep in Wolfram Language. But I am not sure what you mean.

DeleteMissing[EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius",  "Density"}], 1, 2]

works perfectly fine for me. Perhaps you should compare it to inner part

EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius", "Density"}]

and execute it a few times (to avoid web delays) to see it works as designed.

POSTED BY: Vitaliy Kaurov

Working versions of your filter designs:

shouldFilter1[planet_] := 
 Head[MinorPlanetData[planet, "Radius"]] === Missing
filteredDwarves1 := DeleteCases[dwarves, p_ /; shouldFilter1[p]]

shouldFilter2[planet_] := 
 MatchQ[MinorPlanetData[planet, "Radius"], _Missing]
filteredDwarves2 := DeleteCases[dwarves, p_ /; shouldFilter2[p]]

shouldFilter3[planet_] := 
 MinorPlanetData[planet, "Radius"] > Quantity[0.0, "miles"]
filteredDwarves3 := Cases[dwarves, p_ /; shouldFilter3[p]]

What you where having trouble with were some subtleties. Double = is Equal, like an equation. Triple = is SameQ, a test for lexicographical matching. Two ways to scan for the head Missing, one is to isolate using Head then test it with SameQ; another is to use pattern matching, like MatchQ. Filter #3 worked, but it was a positive value Quantity filter when used with DeleteCases.

Why not to use DeleteMissing directly, or am I missing your point?

DeleteMissing[EntityClass["MinorPlanet", "DwarfPlanet"][{"Entity", "Radius"}], 1, 2]

enter image description here

POSTED BY: Vitaliy Kaurov
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract