Message Boards Message Boards

Downloading StarData that meets a certain qualification?

GROUPS:

In this sample code, I first download subsets, of the names of stars, from multiple StarData classes. A Union is used to combine all the target star names in a single list. Using a parallel request across four CPUs, a certain set of metrics are retrieved for each star. Not all stars have a complete set of parameter data. To simplify plotting, Missing["NotAvailable"] entries are set to zero. The same data is then normalized in relation to the sun, and units are removed. A Cases function is used to remove all stars that have zero for a mass value. This works, but in the real code version, all stars byStarData class are first downloaded once, and saved separately as exports. Then a second program, imports each saved class export, and graphs across classes and particular metrics.

Is there a way of just downloading StarData entries, where the Mass parameter is not equal to Missing["NotAvailable"] ? Something like: Take[StarData[EntityClass["Star", "ClassAStar"]], {1, 100}] where "Mass" != "Missing["NotAvailable"]"

Working sample code:

sunMass = StarData["Sun", "Mass"];
sunLuminosity = StarData["Sun", "Luminosity"];
sunTemperature = StarData["Sun", "EffectiveTemperature"];
sunDiameter = StarData["Sun", "Diameter"];
sunGravity = StarData["Sun", "Gravity"];
sunDensity = StarData["Sun", "Density"];
sunVolume = StarData["Sun", "Volume"];

listDataA = 
  Take[StarData[EntityClass["Star", "ClassAStar"]], {1, 100}];
listDataB = 
  Take[StarData[EntityClass["Star", "ClassBStar"]], {1, 100}];
listDataF = 
  Take[StarData[EntityClass["Star", "ClassFStar"]], {1, 100}];
listDataG = 
  Take[StarData[EntityClass["Star", "ClassGStar"]], {1, 100}];
listDataK = 
  Take[StarData[EntityClass["Star", "ClassKStar"]], {1, 100}];
listDataM = 
  Take[StarData[EntityClass["Star", "ClassMStar"]], {1, 100}];
listDataO = 
  Take[StarData[EntityClass["Star", "ClassOStar"]], {1, 100}];

listDataSuper = 
  Take[StarData[EntityClass["Star", "Supergiant"]], {1, 100}];
listDataGiant = 
  Take[StarData[EntityClass["Star", "NormalGiant"]], {1, 100}];
listDataSubgiant = 
  Take[StarData[EntityClass["Star", "Subgiant"]], {1, 100}];



listData = 
  Union[listDataA, listDataB, listDataF, listDataG, listDataK, 
   listDataM, listDataO, listDataSuper, listDataGiant, 
   listDataSubgiant];

CloseKernels[]; LaunchKernels[4]
AbsoluteTiming[
 Length[
  data =
   Transpose[
    ParallelMap[
     StarData[listData, #] &,
     {"Name", "Metallicity", "SpectralClass", "BVColorIndex", 
      "EffectiveTemperature",
      "Mass", "Luminosity", "AbsoluteMagnitude", "Gravity", "Density",
       "Diameter",
      "DistanceFromEarth", "MainSequenceLifetime", "Parallax",
      "RadialVelocity", "Radius", "StarEndState", "StarType", 
      "SurfaceArea",
      "VariablePeriod", "Volume"}]]]]

zeroData = data /. {Missing["NotAvailable"] -> 0};

noUnitsData = 
  zeroData /. {c1_, c2_, c3_, c4_, c5_, c6_, c7_, c8_, c9_, c10_, 
     c11_, c12_, c13_, c14_, c15_, c16_, c17_, c18_, c19_, c20_, 
     c21_} -> {c1, c2, c3, c4, QuantityMagnitude[c5], 
     QuantityMagnitude[c6/sunMass], 
     QuantityMagnitude[c7/sunLuminosity], c8, 
     QuantityMagnitude[c9/sunGravity], QuantityMagnitude[c10], 
     QuantityMagnitude[c11/sunDiameter], QuantityMagnitude[c12], 
     QuantityMagnitude[c13], QuantityMagnitude[c14]
     , QuantityMagnitude[c15], QuantityMagnitude[c16], c17, c18, 
     QuantityMagnitude[c19], QuantityMagnitude[c20], 
     QuantityMagnitude[c21/sunVolume]};

Length[unionAll =
  Cases[
   Sort[noUnitsData, #1[[6]] > #2[[6]] &],
   {_, _, _, _, _, x_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _} /;
     x != 0]
 ]

topTemp = Take[
   Sort[unionAll, #1[[5]] > #2[[5]] &],
   50];

lowTemp = Take[
   Cases[
    Sort[unionAll, #1[[5]] < #2[[5]] &],
    {_, _, _, _, x_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _} /;
      x != 0]
   , 50];


topMass = Take[
   Sort[unionAll, #1[[6]] > #2[[6]] &]
   , 50];

lowMass = Take[
   Cases[
    Sort[unionAll, #1[[6]] < #2[[6]] &],
    {_, _, _, _, _, x_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _} /;
      x != 0]
   , 50];



topLuminosity = Take[
   Sort[unionAll, #1[[7]] > #2[[7]] &]
   , 50];

lowLuminosity = Take[
   Cases[
    Sort[unionAll, #1[[7]] < #2[[7]] &],
    {_, _, _, _, _, _, x_, _, _, _, _, _, _, _, _, _, _, _, _, _, _} /;
      x != 0]
   , 50];


Manipulate[
 plotType[
  With[{c1 = xAxis, c2 = yAxis, tooltip1 = tooltip},
    Tooltip[{(xAxisReverse )* Slot[c1], (yAxisReverse) * 
        Slot[c2]}, {Slot[tooltip1], Slot[c1], Slot[c2]}] &] @@@ 
   category,
  PlotRange -> Automatic, ImageSize -> 800, 
  AxesLabel -> {xLabel, yLabel}, AxesOrigin -> {xOrigin, yOrigin},
  PlotStyle -> Red],
 {{xAxisReverse, 1}, {1, -1}},
 {{yAxisReverse, 1}, {1, -1}},
 {{xOrigin, 
   0}, {-100000, -60000, -50000, -40000, -30000, -20000, -10000, \
-5000, -1000, -100, -50, -40, -30, -25, -20, -15, -10, -9, -8, -7, \
-6, -5, -4, -3, -2, -1, -.0001, -.001, -.01, -.1, -.2, -.3, -.4, -.5, 
   0, .5, 1, 1.5, 2, 2.5, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 40,
    50}},
 {{yOrigin, 
   0}, {-30, -25, -20, -15, -10, -9, -8, -7, -6, -5, -4, -3, -2, -1, \
-.0001, -.001, -.01, -.1, -.2, -.3, -.4, -.5, 
   0, .1, .001, .0001, .00001, .5, 1, 1.5, 2, 2.5, 3, 4, 5, 6, 7, 8, 
   9, 10, 15, 20, 25, 30, 40, 50, 100, 1000, 5000, 10000, 20000, 
   30000, 40000, 50000, 60000}},
 {{xAxis, 6}, {2 -> "Metallicity", 4 -> "BVColorIndex", 
   5 -> "Temperature", 6 -> "Mass", 7 -> "Luminosity", 
   8 -> "AbsoluteMagnitude", 9 -> "Gravity", 10 -> "Density", 
   11 -> "Diameter", 12 -> "DistanceFromEarth", 
   13 -> "MainSequenceLifetime", 14 -> "Parallax", 
   15 -> "RadialVelocity", 16 -> "Radius", 19 -> "SurfaceArea", 
   20 -> "VariablePeriod", 21 -> "Volume"}},
 {{xLabel, "Mass"},
  {"Metallicity", "BVColorIndex", "Temperature",
   "Mass", "Luminosity", "AbsoluteMagnitude", "Gravity", "Density",
   "Diameter", "DistanceFromEarth", "MainSequenceLifetime", 
   "Parallax", "RadialVelocity",
   "Radius", "SurfaceArea", "VariablePeriod", "Volume"}
  },
 {{yAxis, 7}, {2 -> "Metallicity", 4 -> "BVColorIndex", 
   5 -> "Temperature", 6 -> "Mass", 7 -> "Luminosity", 
   8 -> "AbsoluteMagnitude", 9 -> "Gravity", 10 -> "Density", 
   11 -> "Diameter", 12 -> "DistanceFromEarth", 
   13 -> "MainSequenceLifetime", 14 -> "Parallax", 
   15 -> "RadialVelocity", 16 -> "Radius", 19 -> "SurfaceArea", 
   20 -> "VariablePeriod", 21 -> "Volume"}},
 {{yLabel, "Luminosity"}, {"Metallicity", "BVColorIndex", 
   "Temperature",
   "Mass", "Luminosity", "AbsoluteMagnitude", "Gravity", "Density",
   "Diameter", "DistanceFromEarth", "MainSequenceLifetime", 
   "Parallax", "RadialVelocity",
   "Radius", "SurfaceArea", "VariablePeriod", "Volume"}
  },
 {{tooltip, 3}, {1 -> "Name", 3 -> "SpectralClass", 
   2 -> "Metallicity", 4 -> "BVColorIndex", 5 -> "Temperature", 
   6 -> "Mass", 7 -> "Luminosity", 8 -> "AbsoluteMagnitude", 
   9 -> "Gravity", 10 -> "Density", 11 -> "Diameter", 
   12 -> "DistanceFromEarth", 13 -> "MainSequenceLifetime", 
   14 -> "Parallax", 15 -> "RadialVelocity", 16 -> "Radius", 
   19 -> "SurfaceArea", 20 -> "VariablePeriod", 21 -> "Volume"}},
 {{plotType, ListLogLogPlot}, {ListPlot, ListLogPlot, ListLogLogPlot}},
 {{category, unionAll}, {
   topTemp -> "topTemp",
   lowTemp -> "lowTemp",
   topMass -> "topMass",
   lowMass -> "lowMass",
   topLuminosity -> "topLuminosity",
   lowLuminosity -> "lowLuminosity",
   unionAll -> "unionAll"
   }}]
ListPicker[Dynamic[a],
  {
   topTemp -> "topTemp",
   lowTemp -> "lowTemp",
   topMass -> "topMass",
   lowMass -> "lowMass",
   topLuminosity -> "topLuminosity",
   lowLuminosity -> "lowLuminosity",
   unionAll -> "unionAll"
   }
  ] ;
Attachment

Attachments:
POSTED BY: Joseph Karpinski
Answer
9 months ago

Yes. There is a way to query for specific types of entities. I'm afraid I don't think it has any support for searches like "Not Equal To Missing["NotAvailable"]".

For some examples of how to query entities, see http://community.wolfram.com/groups/-/m/t/889336, which has a problem very similar to yours.

Im afraid the only way to filter out such entities is download them all and filter them.

Select[EntityList@"Star", (#["Mass"] != Missing["NotAvailable"] &)]

That select statement would do it. It's probably very slow. The first argument pulls in all the "Star" entities available and the second is a function that tests whether a particular stars mass is Missing or not.

POSTED BY: Sean Clarke
Answer
9 months ago

Thanks Sean!

Regarding "Im afraid the only way to filter out such entities is download them all and filter them.", in the above sample code I use a Cases statement to do just that, after downloading all targets:

First, replace all missing values with zero

zeroData = data /. {Missing["NotAvailable"] -> 0};

then create a list of all targets, excluding those containing zero in the sixth position, which is for the metric "Mass"

Length[unionAll =
  Cases[
   Sort[noUnitsData, #1[[6]] > #2[[6]] &],
   {_, _, _, _, _, x_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _} /;
     x != 0]
 ]

If I understand the select statement correctly, could this be altered to target one StarData Class, say Class O, and return just those Class O star names that have a value for mass?

Select[EntityList@"Star", (#["Mass"] != Missing["NotAvailable"] &)]

That could work...???

Also, currently reviewing the link you provided of similar problems.

POSTED BY: Joseph Karpinski
Answer
9 months ago

Group Abstract Group Abstract