Group Abstract Group Abstract

Message Boards Message Boards

5
|
58.5K Views
|
48 Replies
|
34 Total Likes
View groups...
Share
Share this post:

Has anyone encountered problems retrieving financial data of ETF like "SPY"

Posted 6 years ago

I have been trying to retrieve pricing of ETFs and always the output is Missing["NotAvailable"] which is information that FinancialData used to have.

POSTED BY: Eduardo Pollak
48 Replies
Posted 2 years ago

Any update on this? The numebr of exchanges listed is still very poor...

POSTED BY: Giorgio Busoni
Posted 3 years ago

I have always struggled to use the FinancialData interface and encountered many self caused problems. I have been using sample data to compare against but I can never seem to get them to match.

POSTED BY: Jake Bennet

Hello Arnoud and Alan

I'm re writing as I guess the post I wrote some three weeks ago went unnoticed.

As I said before, it has been months since you are saying that the services are going to be restored or something along that line.

Again, the question: the Group "Futures" is still missing from FinancialData... is it ever going to be back?

FinancialData["Groups"]

Returns everything except "Futures"

{"Currencies", "Exchanges", "ExchangeTradedFunds", "IndexComponents", 
"Indices", "MutualFunds", "Sectors", "Stocks"}

Needless to say, Wolfram's own examples do not run...

Find the current price for a troy ounce of gold in US dollars:
https://wolfram.com/xid/0bnh0gq21j3fm-gpsd7


In[1]:= FinancialData["XAU/USD"]    
Out[1]= FinancialData::notent: XAU/USD is not a known entity, class, or tag for FinancialData. Use FinancialData[] for a list of entities.  

I came across Alan's reply on Stack Exchange on the matter saying that in ver12.1 the documentation was going to more accurately reflect the capabilities of the built-in symbol ... but it ended being as vague as before (i.e. the same examples that do not run)

Curated data on StackExchange

I'd really appreciate a straight answer. Thx in advance for your reply.

Posted 5 years ago

The ETFs that I'm interested in have been restored - thank you.

Let me now ask about Mutual Funds. I'm getting nothing back from FinancialData on any of the ones on my list of interest.

Have mutual funds been restored?

If so, is there something tricky in how the call is made in FinanicalData that is tripping me up? An example that works would be helpful.

If not, what's the planned timetable?

Let me just add that I appreciate that Wolfram keeps working on improving this data feed. The support team is getting the brunt of our frustrations for a situation not of your making. Thank you for your continuing support.

POSTED BY: Steve Francis

Did anyone manage to access S&P500?

FinancialData["SP500“] is used in Tutorial Videos on wolfram.com but returns Missing[NotAvailable]!

POSTED BY: Philipp Winkler
Posted 5 years ago

I had a lot of issues using FinancialData and I can't find good docs on the full tickers to use (whether to prepend the exchange or not, it seems rather random). Is there an updated docs on how to query FinancialData? To date, I have only been working off some external sample data.

POSTED BY: Ryan Maxwell
Posted 5 years ago
POSTED BY: Adam S

Data on MBT hasn't gone away:

In[118]:= FinancialData["NYSE:MBT"]
Out[118]= Quantity[8.890000343322754, "USDollars"]

But you're right that this symbol doesn't show up in the list returned by FinancialData[] or through any of the pattern-matching syntax supported by the function. We'll investigate.

POSTED BY: Alan Joyce
Posted 5 years ago
POSTED BY: Adam S
Posted 5 years ago

I forgot, thank you for the help and quick reply!

I hope this gets fixed soon.

POSTED BY: Adam S
Posted 5 years ago
POSTED BY: Adam S

Wolfram could introduce delayed quotes and economic data subscription for individuals, at say USD 40-50 per month.

With the power of the Wolfram language it could be economically feasible to dedicate resources to even aggregate from primary sources.

In the interim, CountryData["UnitedStates", {"GDPRealGrowth", {2015, 2020}}] returns blank.

https://eodhistoricaldata.com - appropriate for quotes but lacks economic data - they promised it soon.

https://intrinio.com/ - good catalogue and becomes very expensive very quickly.

For everything else, there is Bloomberg data feed at USD 25,000 per year.

POSTED BY: Matvei Kruglyak
Posted 5 years ago

Great paid options for data that are reasonably priced: https://eodhistoricaldata.com and https://intrinio.com/

I had to move away from FinancialData years ago. Reliability and quality of the data for serious use was very poor. I've used both of the data providers above and both are excellent for the price.

Best

POSTED BY: Cory Haupt
Posted 5 years ago

Dear Wolfram colleagues,

Would be nice to get an answer to my question on the expected time for data availability from Milan, Amsterdam, Madrid stock exchanges and the historical series range posted almost one month ago.

Thanks.

POSTED BY: Miguel Esteban
Posted 5 years ago

Do I have the NASDAQ index incorrect here?:

FinancialData["NASDAQ"]
Missing["NotAvailable"]

Also how is S&P 500 accessed?

I guess DJI is still unavailable.

Thanks, David

POSTED BY: David Bartley
Posted 6 years ago

Just to complement my previous message:

Checked some of the historical values. Unless a programming mistake, this really looks less enthusiastic. As you can see for the first 12 ETF in the list for which data are available:

Initial historical values before data discontinued in August 2019: {{1, {2017, 1, 24}}, {2, {2015, 11, 10}}, {3, {2005, 3, 2}}, {4, {2008, 10, 13}}, {5, {2009, 3, 4}}, {6, {2008, 10, 6}}, {7, {2013, 9, 10}}, {8, {2016, 5, 10}}, {9, {2014, 9, 16}}, {10, {2010, 6, 23}}, {11, {2009, 3, 4}}, {12, {2010, 6, 16}}

First historical data available now: {{1, {2017, 2, 8}}, {2, {2018, 3, 1}}, {3, {2018, 2, 14}}, {4, {2018, 2, 14}}, {5, {2009, 3, 4}}, {6, {2018, 3, 23}}, {7, {2018, 4, 5}}, {8, {2018, 4, 5}}, {9, {2018, 4, 5}}, {10, {2018, 4, 5}}, {11, {2018, 4, 18}}, {12, {2018, 4, 18}},

In this small example, we're talking of having lost in some cases daily data (i.e. number 3) for over 14 years (!) with all the implied statistical lost. I understand I could nevertheless join the two series from my files...however...

Thanks.

POSTED BY: Miguel Esteban
Posted 6 years ago

Hi Alan,

Many thanks for the update. Much appreciated.

As you may see below, the picture now looks much more promising. Haven't checked historical values yet, but current values for tickers from the exchanges for which access is already granted work 100% fine:

{1, "PA:EGOV", 55.69},
{2, "PA:CBBB", 16.2359},
{3, "PA:C40 ", 89.99},
{4, "PA:C50 ", 85.97},
{5, "PA:CD8 ", 128.58},
{6, "PA:CEU ", 254.38},
{7, "PA:AHYE", 237.372},
{8, "PA:AFLE", 50.3074},
{9, "PA:AFRN", 101.029},
{10, "PA:X1G ", 255.949},
{11, "PA:CV9 ", 236.778},
{12, "PA:ANX ", 97.65},
{13, "XMCE:BBVAE", Missing["NotAvailable"]},
{14, "XMCE:BBVAI", Missing["NotAvailable"]},
{15, "DE:DXSP", 11.172},
{16, "MI:XDAX", Missing["NotAvailable"]},
{17, "DE:DBPD", 2.6875},
{18, "MI:OILB", Missing["NotAvailable"]},
{19, "MI:CORN", Missing["NotAvailable"]},
{20, "MI:BULL", Missing["NotAvailable"]},
{21, "MI:NGAS", Missing["NotAvailable"]},
{22, "MI:PHAU", Missing["NotAvailable"]},
{23, "MI:PHPT", Missing["NotAvailable"]},
{24, "MI:PHAG", Missing["NotAvailable"]},
{25, "MI:AIGP", Missing["NotAvailable"]},
{26, "MI:SBUL", Missing["NotAvailable"]},
{27, "MI:SOIL", Missing["NotAvailable"]},
{28, "MI:WEAT", Missing["NotAvailable"]},
{29, "MI:CRUD", Missing["NotAvailable"]},
{30, "MI:IFFF", Missing["NotAvailable"]},
{31, "MI:AGED", Missing["NotAvailable"]},
{32, "BE:IS0C", Missing["NotAvailable"]},
{33, "DE:2B76", 8.029},
{34, "MI:FXC", Missing["NotAvailable"]},
{35, "DE:EXS1", 117.68},
{36, "AS:CSX5", Missing["NotAvailable"]},
{37, "L:ISF", 7.431},
{38, "MI:SJPA", Missing["NotAvailable"]},
{39, "MI:DGTL", Missing["NotAvailable"]},
{40, "DE:EXI2", 46.79},
{41, "DE:EXI3", 269.3},
{42, "DE:EUNW", 106.15},
{43, "DE:IQQH", 7.076},
{44, "DE:IS0E", 11.398},
{45, "MI:HEAL", Missing["NotAvailable"]},
{46, "DE:IS0R", 95.818},
{47, "DE:IUS7", 106.07},
{48, "DE:IS0Q", 96.16},
{49, "DE:IUSQ", 52.},
{50, "AS:IEER", Missing["NotAvailable"]},
{51, "AS:IEMA", Missing["NotAvailable"]},
{52, "AS:IMEU", Missing["NotAvailable"]},
{53, "MI:IJPE", Missing["NotAvailable"]},
{54, "DE:IBCH", 59.258},
{55, "DE:EXXT", 85.86},
{56, "DE:IS0D", 13.016},
{57, "MI:CINA ", Missing["NotAvailable"]},
{58, "MI:CRB ", Missing["NotAvailable"]},
{59, "MI:AUST", Missing["NotAvailable"]},
{60, "PA:CAC ", 59.84},
{61, "PA:DAX ", 129.68},
{62, "PA:DJE ", 270.6},
{63, "MI:EST ", Missing["NotAvailable"]},
{64, "PA:CRP ", 155.399},
{65, "PA:CSH", 104.632},
{66, "PA:MSED", 186.52},
{67, "PA:MSE ", 37.405},
{68, "PA:BXX ", 2.742},
{69, "PA:LVE ", 33.71},
{70, "PA:BSX ", 14.466},
{71, "MI:LEVMIB", Missing["NotAvailable"]},
{72, "MI:BERMIB", Missing["NotAvailable"]},
{73, "XMCE:LYXIB", Missing["NotAvailable"]},
{74, "XMCE:IBEXA", Missing["NotAvailable"]},
{75, "XMCE:2INVE", Missing["NotAvailable"]},
{76, "XMCE:INVEX", Missing["NotAvailable"]},
{77, "MI:BRA ", Missing["NotAvailable"]},
{78, "PA:JPN ", 137.69},
{79, "MI:LATAM ", Missing["NotAvailable"]},
{80, "PA:LVD ", 121.98},
{81, "PA:LVC ", 25.57},
{82, "PA:ES10", 177.93},
{83, "PA:LEM ", 11.492},
{84, "PA:MEU ", 141.51},
{85, "PA:INR ", 18.006},
{86, "PA:WLD ", 221.19},
{87, "PA:LQQ ", 399.},
{88, "MI:UST ", Missing["NotAvailable"]},
{89, "PA:PAF ", 9.285},
{90, "PA:RUS ", 49.045},
{91, "PA:SP5 ", 31.655},
{92, "PA:SHC ", 16.986},
{93, "PA:SEL ", 18.195},
{94, "MI:BNK ", Missing["NotAvailable"]},
{95, "MI:OIL ", Missing["NotAvailable"]},
{96, "MI:TNO ", Missing["NotAvailable"]},
{97, "MI:TELE ", Missing["NotAvailable"]},
{98, "MI:UTI ", Missing["NotAvailable"]},
{99, "PA:BX4 ", 2.76},
{100, "MI:HK", Missing["NotAvailable"]},
{101, "DE:SYBM", 66.076},
{102, "DE:SPYJ", 36.995},
{103, "DE:SPYF", 64.95},
{104, "DE:SPYA", 63.94},
{105, "DE:SPYM", 53.69},
{106, "PA:ERO", 230.22},
{107, "DE:ZPRS", 72.99},
{108, "DE:SPY4", 56.01},
{109, "DE:SPY5", 310.47},
{110, "DE:ZPDD", 33.19},
{111, "DE:ZPDS", 27.225},
{112, "DE:ZPDF", 30.805},
{113, "DE:ZPDH", 27.365},
{114, "DE:ZPDI", 31.2},
{115, "DE:ZPDM", 25.04},
{116, "DE:ZPDT", 47.51},
{117, "AS:VEUR", Missing["NotAvailable"]},
{118, "AS:VFEM", Missing["NotAvailable"]},
{119, "AS:VUSA", Missing["NotAvailable"]},
{120, "L:VWRD", 0.95935},
{121, "L:VDPX", 0.262087},
{122, "L:VDJP", 0.314737},
{123, "L:VDVA", 0.339}

Would then be possible to know when is it expected to get availability to the remaining exchanges in the 123 sample: Basically Milan, Amsterdam and Madrid.

This would be much appreciated to better plan a way forward.

Thanks again for your support.

POSTED BY: Miguel Esteban
Posted 6 years ago

David,

Nothing in WL is real-time, I was just giving an example of how astronomical data costs are. I can only speak to buy-side so I gave a personal example, though I'm sure sell-side is equally awful. Buying data is not just paying the vendor, you also have to pay exchanges and regulatory fees and the exchanges have carte blanche to raise the license fees as much and as often as they see fit. And indeed, they do. The example I gave regarding non-display is a recent example where monthly costs went up 5x overnight. It started with NYSE and now all exchanges are following suit. The environment has rapidly changed and it's unreasonable to expect WRI or anyone to provide any kind of trade data without the specific intent to directly monetize it. If I recall correctly, DJI is run by S&P Global. I've dealt with them before and wouldn't be surprised if DJI redistribution costs as much as a house.

The best option for free data right now is an Interactive Brokers account with Jonathan Kinlay's MATH-TWS package. You can get real-time there, though I'm sure historical is limited by IB.

POSTED BY: Cary Munves
Posted 6 years ago

I don't think people understand how expensive it is to redistribute or disseminate trade data. The regulatory and exchange costs alone make it prohibitive, to say nothing of the individual vendor. Trade data is a great example of something that increases in cost as it proliferates. I can tell you that if this was real-time, to access this data programatically as you do in WL (what the exchanges call "non-display") would add tens of thousands of dollars to an individual's monthly data costs, regardless of whether you are a professional or non-professional. Furthermore, it's not like WRI is offsetting these costs with ads like yahoo finance or whatever. DJI may be a simple dataset, but you can bet it has a prohibitive and skyrocketing redistribution cost. This is just my opinion but absent a professional datafeed, FinancialData will always be for academic purposes. Nobody can just eat data costs like this.

POSTED BY: Cary Munves
Posted 6 years ago

Cary,

I wonder then what has changed regarding DJI, which is nothing like "real time". I have used the dataset from 2018.

Of course Mathematica programs have skyrocketed, becoming obsolete as operating systems are updated.

David

POSTED BY: David Bartley
Posted 6 years ago

Yet, a seemingly simple dataset as Dow Jones International (DJI), which was useful for "years", is still not available.

POSTED BY: David Bartley

This was very helpful — there was a provisioning problem with some of these ETFs, which should now be solved for the exchanges we have access to (and if not, please report any further problems):

In[58]:= FinancialData[{"PA:EGOV", "PA:CBBB", "PA:C40", "DE:DXSP", 
  "DE:DBPD", "DE:IS0Q", "L:VWRD", "L:VDPX", "L:VDJP"}, {"Symbol", 
  "Close"}]

Out[58]= {{"PA:EGOV", 
  Quantity[55.63600158691406, "Euros"]}, {"PA:CBBB", 
  Quantity[16.23539924621582, "Euros"]}, {"PA:C40", 
  Quantity[89.29000091552734, "Euros"]}, {"DE:DXSP", 
  Quantity[11.251999855041504`, "Euros"]}, {"DE:DBPD", 
  Quantity[2.7335000038146973`, "Euros"]}, {"DE:IS0Q", 
  Quantity[96.00199890136719, "Euros"]}, {"L:VWRD", 
  Quantity[0.9556000232696533, "BritishPounds"]}, {"L:VDPX", 
  Quantity[0.25882500410079956`, "BritishPounds"]}, {"L:VDJP", 
  Quantity[0.31636300683021545`, "BritishPounds"]}}
POSTED BY: Alan Joyce
Posted 6 years ago

Hi again, Alan.

Thanks for the update. I am looking forward to receiving the new data.

The ETF group I used to work with is quoted across the following exchanges: Paris, Xetra, London, Amsterdam, Berlin, Milan and Madrid. However I am encountering problems still when trying to get ETF quotes from Paris, Xetra and London exchanges.

Kindly see below:

enter image description here

As you can see, I only able to get 2 ETF quotes from Paris whilst all the rest are still missing. The same goes for Xetra and London.

Any reason? Could you please help me on this? Any expected date to be able to get the rest?

Many thanks for your support.

POSTED BY: Miguel Esteban

The discrepancy between "theoretically" available quotes and what is actually accessible through FinancialData should be significantly reduced with the next update of that paclet; a version is currently in testing which will update the list of valid ticker symbols and fix various other bugs.

With the exception of some selected indexes for which we are still working to secure redistribution rights, tickers on the exchanges noted above in this thread are what FinancialData is now limited to:

  • Bombay
  • Frankfurt
  • Hong Kong
  • London
  • NASDAQ
  • NYSE
  • Paris
  • Shanghai
  • Shenzhen
  • Tokyo
  • Toronto
  • Xetra

As some people have noted, it's true that an individual may be able to scrape more extensive market data from various websites, but exposure in the Wolfram Language through FinancialData[] is in a different category from simple web display — in the current landscape, bulk redistribution of financial data requires, in most cases, not only licensing from aggregators of financial data, but also direct agreements with individual exchanges. We've had to limit our coverage to the financial instruments that have historically seen the most usage through the Wolfram Language, and for which we are able to satisfy applicable distribution and reporting requirements for each exchange or other provider.

POSTED BY: Alan Joyce
Posted 6 years ago

Hi Alan,

It’s good to see that things are moving ahead and service is progressively returning after got discontinued in August 2019.

I am one of those heavily affected users for which a lot of work has turned completely useless.

I still can see that even if some services are restored, big discrepancies remain between Mathematica catalogue of theoretically available quotes and those effectively available. In my case, I was working with a group of around 125 European ETFs and they’re still not available and this happens since August 2019.

I understand that sometimes commercial issues with providers might come up, but it is less understandable to not receive a precise timeline for which the different pieces of info will be restored. This does not show very open and transparent, in particular as it has not been the case even when reached the company customer service.

Hope we get better news soon.

Thanks for your help.

POSTED BY: Miguel Esteban
Posted 6 years ago

It is interesting that we can download ^GSPC from Yahoo, but can't access it using FinancialData. I really hope these issues can be resolved on a permanent basis.

POSTED BY: David Keith

Extensive historical price data for companies listed on NYSE and NASDAQ remains available. Regarding one specific complaint above, note that L3 Technologies (now "L3Harris Technologies" after a merger earlier this year) is actually LHX, not LXH:

In[81]:= FinancialData["NYSE:LHX"]
Out[81]= Quantity[197.41000366210938`, "USDollars"]

Historical data (at least 10+ years in most cases) for stocks on ten major global exchanges has been restored:

In[103]:= FinancialData[{"DE:BMW", "PA:UBI", "L:LLOY", "HK:03690", 
  "MB:500096", "SS:601857", "SZ:000002", "F:SIE", "TSE:7203", "TO:RY"}]

Out[103]= {Quantity[74.0999984741211, "Euros"], 
 Quantity[60.439998626708984`, "Euros"], 
 Quantity[0.63919997215271, "BritishPounds"], 
 Quantity[102., "HongKongDollars"], 
 Quantity[459.6000061035156, "IndianRupees"], 
 Quantity[5.800000190734863, "ChineseYuan"], 
 Quantity[31.68000030517578, "ChineseYuan"], 
 Quantity[117.05999755859375`, "Euros"], Quantity[7744., "Yen"], 
 Quantity[103.83000183105469`, "CanadianDollars"]}

And as of last Friday, we've restored data for ~2300 ETFs. There were some gaps in historical data that should now be filled in:

In[102]:= FinancialData /@ {"NYSE:SPY", "NYSE:BSV", "NYSE:XLK", 
  "NYSE:SCHB", "NASDAQ:QQQ"}

Out[102]= {Quantity[320.8999938964844, "USDollars"], 
 Quantity[80.62999725341797, "USDollars"], 
 Quantity[90.72000122070312, "USDollars"], 
 Quantity[76.3499984741211, "USDollars"], 
 Quantity[210.86000061035156`, "USDollars"]}

A few remaining pieces noted above in this thread (including access to things like DJIA and IXIC) should be restored within the next week or so, along with historical dividend data.

POSTED BY: Alan Joyce
Posted 6 years ago

FinancialData["DJI"] Missing[NotAvailable]

Does this make sense to you?

POSTED BY: David Bartley
Posted 6 years ago

Yes (speaking to self!). Same problem mentioned above.

POSTED BY: David Bartley

Unfortunately, indices are still not completely resolved — they require a number of additional agreements that we are in the process of trying to secure.

POSTED BY: Alan Joyce
Posted 6 years ago
POSTED BY: Adam S

Yes it seems indeed that stuff is missing. Very annoying. This service is just very unreliable is my experience over the last 4 years.

POSTED BY: l van Veen
Posted 6 years ago

It's a shame but it's true. It's time to look for some other source for Data. I've been reluctant to try Qandl's APIs, but this might be it...

POSTED BY: Enrique Puente
POSTED BY: l van Veen
POSTED BY: Michael Costolo
POSTED BY: Arnoud Buzing
Posted 6 years ago

Arnoud, Thank you for explaining the situation.

I realize a lot of this is outside Wolfram's control, but I would still like to strongly encourage you to bring back full functionality as soon as practical. (For me, FinancialData[] was the primary benefit of moving to version 12. Most of my development work during the last 6 months is now inoperable.)

Can you suggest any alternatives/work arounds? I have a lot of Mathematica code that relies upon FinancialData[] to access ETF, mutual fund, international stocks, and indexes for historical price history.

Regards, Steve

POSTED BY: Steve Francis

Hi Arnoud, Are you also considering of offering this as a paid solution? If the price is good I would not mind.

POSTED BY: l van Veen

We've added metal prices and rates for many currency pairs, but stocks are still limited to historical prices for NASDAQ, NYSE, London, Toronto exchanges. Indices, mutual funds, ETFs have not been restored yet.

POSTED BY: Arnoud Buzing

Is it so?

Metal prices are missing...

TimeSeries[FinancialData["COMEX:HG", {{2019, 9, 1}, Today}]]

Returns

TimeSeries::emptpth: Path is expected to be non-empty.

Note that this line is querying for data only for the last month

Currently available only as properties of a small set of entities, not as listings on a futures exchange:

DateListPlot[
 EntityValue[{Entity["Element", "Gold"], Entity["Element", "Silver"], 
   Entity["Element", "Platinum"], Entity["Element", "Palladium"]}, 
  Dated["Price", 
   Interval[{DateObject[{2019, 9, 1}], DateObject[{2019, 9, 30}]}]]]]
POSTED BY: Arnoud Buzing

Hello Arnoud

Some ten months ago you mentioned that

We've added metal prices and rates for many currency pairs, but stocks are still limited to historical prices for NASDAQ, NYSE, London, Toronto exchanges. Indices, mutual funds, ETFs have not been restored yet.

Would you happen to have any info on when futures will be restored?

So far they're completely missing

FinancialData["Groups"]
{"Currencies", "Exchanges", "ExchangeTradedFunds", "IndexComponents", 
"Indices", "MutualFunds", "Sectors", "Stocks"}

Rgrds,

Posted 6 years ago
POSTED BY: Eduardo Pollak
Posted 6 years ago

Thank you for the answer!!

POSTED BY: Eduardo Pollak

Yes, that has been happening to me too.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard