Message Boards Message Boards

Download multiple tweets using tweet IDs at once?

Posted 8 years ago

Hi,

I have a list of tweetIDs and would like to download them all at once. I tried ServiceConnect["Twitter"], but it limits to one tweet per call and is mighty slow (~3s per call/tweet). Twitter provides APIs for multiple downloads at once but it limits to downloading 100 at one time. I used the Oauth functions in mathematica but it needs verification after every single call as I am using the consumer key instead of access token. I am unable to find the right keywords to specify the access tokens. Can someone help me with this? Here is the call that I am making:

url = "https://api.twitter.com/1.1/statuses/lookup.json?id=" <> StringRiffle[tweetIDs[[;; 100, 1]], ","];
token = OAuthSigning`OAuthToken[
   "ConsumerKey" -> "CONSUMER_KEY",
   "ConsumerSecret" ->"CONSUMER_SECRET",
   "RequestEndpoint" -> "https://api.twitter.com/oauth/request_token",
   "AuthorizeEndpoint" -> "https://api.twitter.com/oauth/authorize",
   "AccessEndpoint" -> "https://api.twitter.com/oauth/access_token"];
URLFetch[url, "OAuthAuthentication" -> token]

This works but needs to be verified for access every single time. I also tried this:

token2 = OAuthSigning`OAuthToken[
       "OAuthVersion" -> "1.0a",
       "ConsumerKey" -> "CONSUMER_KEY",
       "ConsumerSecret" ->"CONSUMER_SECRET",
       "AccessToken" -> "ACCESS_TOKEN",
       "AccessTokenSecret" -> "ACCESS_SECRET"
       "RequestEndpoint" -> "https://api.twitter.com/oauth/request_token",
       "AuthorizeEndpoint" -> "https://api.twitter.com/oauth/authorize",
       "AccessEndpoint" -> "https://api.twitter.com/oauth/access_token"];

But I ended up getting a bunch of errors saying: AccessToken, AccessTokenSecret give values for things that are not valid OAuth 1.0a properties.

In[191]:= $Version

Out[191]= "11.0.1 for Mac OS X x86 (64-bit) (September 21, 2016)"

Any solutions for this?

POSTED BY: Yash Gandhi
3 Replies

Presently Twitter paclet does not support https://api.twitter.com/1.1/statuses/lookup.json end point. Meanwhile one of the oauth developer suggested me a rather simple way of calling this endpoint from mathematica. see attached image for reference: enter image description here

POSTED BY: Damanjit Singh
Posted 8 years ago

This works! Thanks a lot Daman. Hope you are doing fine! :)

POSTED BY: Yash Gandhi

Hi Yash,

The GetTweet request from the Twitter connection only supports one TweetID as you mentioned but we'll soon add support for multiple IDs. I'll keep you informed here.

Thanks

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract