# Filtering Missing from Data

Posted 9 years ago
7817 Views
|
10 Replies
|
9 Total Likes
|
 What am I doing wrong? I am still getting Missing data in my result. Select[FinancialData["GE", {{2012, 5, 1}, {2012, 9, 30}}][[All]], FreeQ[#, Missing] &] 
10 Replies
Sort By:
Posted 9 years ago
 The information that you have provided is excellent and something that I have had trouble getting out of the tutorials. Even the last post even though flawed will be educational to me once I figure out what it means. You have been very kind and more help than I deserve. I am just an old man enjoying the latest technical developments. I consider Mathematica to be revolutionary. I am trying to convince everyone else.Thank you.
Posted 9 years ago
 OK, we did vary from the original subject. But I did learn something. Getting "curated" data is not easy for a beginner. You provided some code that I have not seen in the demonstrations and documentation.Thank You
Posted 9 years ago
 I see your point. On the other hand the problem that you encountered is mostly because the time series is to equidistantly sampled, because there are no data points on the weekends. So getting the data is rather straight forward, but the sampling needs to be taken into consideration. I think that that's more of a data analysis problem, rather than a curated data problem.To fix the sampling issue I separated the "data from the dates" , then did the forecast bit, and then added the dates in again. You could use several alternative strategies, e.g. use the Function TimeSeriesResample you can also fix the problem; and I only have to work with TimeSeries data. The idea is like this: dkfresamp = TimeSeriesResample[dkf]; I then fit a model (which in this case is not a good one!): model = EstimatedProcess[dkfresamp, ARIMAProcess[{a, b, c}, d, {e, f}, g]] I then forecast forecast2 = TimeSeriesForecast[model, dkfresamp, {20}, Method -> "Kalman"] and plot: DateListPlot[{dkfresamp, forecast2}] Of course the fit is quite unsatisfactory in this case, but the program runs through without any error messages and I do not need to split the dates from the values.Best wishes,M.PS: If you want the time series in a standard date, value format you can use: dkfresamp2 = {DateList[#[[1]]], #[[2]]} & /@ dkfresamp; 
Posted 9 years ago
 I get a lot complaints trying to do something like this dkf = FinancialData["GE", {{2012, 5, 1}, {2012, 9, 30}}]; data = TemporalData[ dkf ]["Part", All, {{2012, 5, 1}, {2012, 9, 30}}] epoc = EstimatedProcess[data, ARIMAProcess[{40, 2, 0}]] forecasr = TimeSeriesForecast[epoc, FinancialData["GE", {{2012, 5, 1}, {2012, 9, 30}}], {20}] DateListPlot[{data["Path"], forecasr["Path"]}, Joined -> True, Filling -> Axis, PlotLabel -> "GE", ImageSize -> 350]  Attachments:
Posted 9 years ago
 Hi,I am not sure whether that question is still related to the MissingData issue. There are some problems in that code, for example that there is a change between scalar data and data with dates - as needed for the DateListPlot.The following might work: (*Download the data*) dkf = FinancialData["GE", {{2012, 5, 1}, {2012, 9, 30}}]; (*Make data scalar and fit time series*) data = dkf[[All, 2]]; epoc = EstimatedProcess[data, ARIMAProcess[{a, b, c}, d, {e, f}, g]]; (*Forecast based on the model*) forecasr = TimeSeriesForecast[epoc, data, {20}]; (*Join data and forecast*) datapred = Join[data, Flatten[forecasr["Paths"], 1][[All, 2]]]; (*Generate dates for the DateListPlot; the BusinessDay option requires MMA10.0.2*) dates = DateRange[dkf[[1, 1]], DayPlus[dkf[[-1, 1]], 31], "BusinessDay"]; (*Then Plot*) DateListPlot[{Transpose[{dates, datapred}][[1 ;; Length[dkf]]], Transpose[{dates, datapred}][[Length[dkf] ;;]]}, Joined -> True, Filling -> Axis, PlotLabel -> "GE", ImageSize -> 350] The result looks like this:There is no problem with MissingData, and I am not sure whether this relates to the original post. Cheers,M.
Posted 9 years ago
 Clinton why do you have [[All]] - is it a typo?Anyway generally these functions can be useful for you, in addition to Marco's answer: ?*Missing* 
Posted 9 years ago
 In my stumbling around to see what I was doing wrong, I changed the code from [[All,2]] to [[All]].Thank you.
Posted 9 years ago
 Hi,I cannot reproduce that. If I download the data you say, I do not get any missing data at all. If however, I download this: WeatherData["EEKA", "WindSpeed", {{2004, 7, 12}, {2004, 7, 30}}] // Normal I do get missing data. In that case your function: Select[WeatherData["EEKA", "WindSpeed", {{2004, 5, 1}, {2004, 9, 30}}] // Normal, FreeQ[#, Missing] &] appears to work as expected. If you want you can export the time series that you get and attach it to a post in this thread. I'll try with that then. Cheers,M.
Posted 9 years ago
 You are good at this. I was trying to use some weather data examples to do finance. Thank you.
Posted 9 years ago