Message Boards

WOLFRAM COMMUNITY

903 Views

5 Replies

1 Total Likes

View groups...

Follow this post

Share this post:

GROUPS:

Data Science Wolfram Language Statistics and Probability

How to perform analysis on a FlightData call, that is in a time object format?

Ryder Nocks Baldwin

Posted 2 months ago

Hello there! I am trying to get data from FlightData to perform analysis on for a project for college! I am getting as far as getting the info, but I am not sure the best way to get everything into a format or a dataframe that would make this easily analyzable. I am familiar with R, but newer to Wolfram... arrivalDelaysRaw = FlightData[ Entity["Airport", "MEM"] -> Entity["Airport", "LGA"], {"ArrivalDelay"}, {DateObject[{2023, 1, 1}], DateObject[{2023, 1, 10}]}]; I call the data with the above, and try to convert with the below, which was working, but isn't working now for some reason. safeConvertToMinutes[item_] := Module[{minutes}, minutes = Quiet@Check[QuantityMagnitude[QuantityConvert[item, "Minutes"]], None]; If[minutes === None, Nothing, minutes] (If conversion fails, return Nothing to filter out later)]; processArrivalDelay[item_] := If[item === Missing["NotAvailable"], Nothing,(Exclude missing data) safeConvertToMinutes[item] (Attempt to convert to minutes)]; processedArrivalDelays = Map[processArrivalDelay, arrivalDelaysRaw]; cleanedArrivalDelays = DeleteCases[processedArrivalDelays, Nothing]; cleanedArrivalDelays // Short I am trying to perform general descriptive statistics on various flight routes delays, as well as making advanced graphics like stacked line plots or 3D Histograms. Is there anything glaringly obvious I am missing? Thank you in advance!

Hello there! I am trying to get data from FlightData to perform analysis on for a project for college! I am getting as far as getting the info, but I am not sure the best way to get everything into a format or a dataframe that would make this easily analyzable. I am familiar with R, but newer to Wolfram...

arrivalDelaysRaw = 
  FlightData[
   Entity["Airport", "MEM"] -> 
    Entity["Airport", 
     "LGA"], {"ArrivalDelay"}, {DateObject[{2023, 1, 1}], 
    DateObject[{2023, 1, 10}]}];

I call the data with the above, and try to convert with the below, which was working, but isn't working now for some reason.

safeConvertToMinutes[item_] := 
  Module[{minutes}, 
   minutes = 
    Quiet@Check[QuantityMagnitude[QuantityConvert[item, "Minutes"]], 
      None];
   If[minutes === None, Nothing, minutes] (*If conversion fails,
   return Nothing to filter out later*)];

processArrivalDelay[item_] := 
  If[item === Missing["NotAvailable"], Nothing,(*Exclude missing data*)
   safeConvertToMinutes[item] (*Attempt to convert to minutes*)];

processedArrivalDelays = Map[processArrivalDelay, arrivalDelaysRaw];

cleanedArrivalDelays = DeleteCases[processedArrivalDelays, Nothing];

cleanedArrivalDelays // Short

I am trying to perform general descriptive statistics on various flight routes delays, as well as making advanced graphics like stacked line plots or 3D Histograms. Is there anything glaringly obvious I am missing? Thank you in advance!

POSTED BY: Ryder Nocks Baldwin

5 Replies

Sort By:

Henrik Schachner

Henrik Schachner, Radiation Therapy Center, Weilheim, Germany

Posted 2 months ago

Hi Ryder, maybe this helps a bit: arrivalDelaysRaw = FlightData[Entity["Airport", "MEM"] -> Entity["Airport", "LGA"], {"ArrivalDelay"}, {DateObject[{2023, 1, 1}], DateObject[{2023, 1, 10}]}]; arrivalDelays0 = DeleteCases[arrivalDelaysRaw, <\|"ArrivalDelay" -> Missing["NotAvailable"]\|>]; arrivalDelays = KeyValueMap[{#1, UnitConvert[#2["ArrivalDelay"], "Minutes"]} &, arrivalDelays0]

Hi Ryder,

maybe this helps a bit:

arrivalDelaysRaw = FlightData[Entity["Airport", "MEM"] -> Entity["Airport", "LGA"], {"ArrivalDelay"}, {DateObject[{2023, 1, 1}], DateObject[{2023, 1, 10}]}];
arrivalDelays0 = DeleteCases[arrivalDelaysRaw, <|"ArrivalDelay" -> Missing["NotAvailable"]|>];
arrivalDelays = KeyValueMap[{#1, UnitConvert[#2["ArrivalDelay"], "Minutes"]} &, arrivalDelays0]

POSTED BY: Henrik Schachner

Updating Name

Posted 2 months ago

Thank you, I am still having issues. The end goal is to get these datasets for various routes to be able to compare them statistically, but I am having trouble getting the list of numbers into something that is usable. Do you have any general advice for this? I appreciate the help very much!

POSTED BY: Updating Name

Rohit Namjoshi

Posted 2 months ago

Can you provide an example of the data format that you are trying to construct?

POSTED BY: Rohit Namjoshi

Ryder Nocks Baldwin

Posted 2 months ago

So I am trying to do more with this and am still having issues, I originally started another thread, but was told I should just continue it here! So here is the text from that post Hello! I am trying to use data collected from the FlightData[] command to analyze departure delay data for a school project. I am getting as far as using a command to get the data, taking it into a named object, and extracting the values from it, but I am trying to remove the missing values, but DeleteMissing[] doesn't seem to work... I also am trying to get all of my delays to be in units of minutes without hours, but they are pulled in units of minutes and sometimes mixed radix units... <\|Entity["Flight", "202301010000290517"] -> <\|"ArrivalDelay" -> Quantity[MixedMagnitude[{0, 5}], MixedUnit[{"Hours", "Minutes"}]]\|>, Entity["Flight", "202301010000323732"] -> <\|"ArrivalDelay" -> Quantity[-8, "Minutes"]\|>, Entity["Flight", "202301020000152941"] -> <\|"ArrivalDelay" -> Quantity[-12, "Minutes"]\|>, Entity["Flight", "202301020000257894"] -> <\|"ArrivalDelay" -> Quantity[-30, "Minutes"]\|>, Also, I want to do statistical analysis with this data I am collecting, and have plans to track various routes for a year, and would like to make some layered line graphs. I had tried before, but I got a mismatch saying the length of the sets was not the same and this could not be done. Is there a way around this? I think I am also having an issue with the outputs being ArrivalDelay -> 4 min or 1 hr 5 min, and I can't seem to get the raw numbers away from the ArrivalDelay signifier and I'm afraid this is also causing me issues. I think the biggest hump I am trying to overcome is the data collecting and cleaning. I am newer to wolfram, and have most of my experience in R, but am trying to challenge myself to learn a new skill for school. Wolfram seems incredibly intuitive and powerful once the basics are grasped which I am trying to do here! Any help is appreciated! I am sure I will have more questions as I work more, but I appreciate any help with this! I have been speaking with my advisor and we are hoping that this project will get more people in the department interested in exploring other tools for data analysis, such as wolfram, because it really seems to be able to make graphics that are a bit more interesting than R! I have attached a screenshot with a sample output which I hope is helpful. Attachments: Screenshot 2024-...png

POSTED BY: Ryder Nocks Baldwin

Rohit Namjoshi

Posted 2 months ago

Hi Ryder, `QuantityConvert` is not a built-in function. It should be `UnitConvert`. The error checking can be simplified by eliminating `Missing` from the raw data arrivalDelaysRaw // Map[DeleteMissing] // processArrivalDelay There is a handy Resource Function KeyGroupBy that you can use to group the data by airline or any other property of the Flight entity. ResourceFunction["KeyGroupBy"][processedArrivalDelays, #["Airline"] &]

POSTED BY: Rohit Namjoshi

Reply to this discussion

Reply Preview

Attachments

Remove Add a file to this post

Follow this discussion

or Discard

Group Abstract

Feedback