Data Drop: How do I import arbitrary JSON?

Questions
, ,
1

I’d love some advice or a guide to follow for importing JSON into the Data Drop platform. The docs are leading me to creating a Custom API (specifically, APIFunction and CloudDeploy); however, if there was an example to follow, that would be stellar.

2

Hi Neil, this really depends on the type of JSON that you are trying to import. The following is a dataset that I created for Barcelona’s bike sharing system using CityBikes JSON API. Every three minutes the total number of parked bicycles is being added to a Databin:

bin = CreateDatabin[]

databin.jpg

RunScheduledTask[
 DatabinAdd[bin, <| "bicycles" -> 
    Total["bikes" /. Import["http://api.citybik.es/bicing.json", "JSON"]]|>], 180]

Task.jpg

DateListPlot[bin["Data"]]

plot.jpg
plot.jpg450×285 10.8 KB

3

Neil,
You can send json data directly to a databin. Here is an example:

In[1]:= bin = CreateDatabin[]

Out[1]= Databin["....."]

In[2]:= json = 
 ExportString[{"x" -> 11, "y" -> "foo", "z" -> True}, "JSON"]

Out[2]= "{
    \"x\": 11,
    \"y\": \"foo\",
    \"z\": true
}"

In[3]:= URLFetch[
 "https://datadrop.wolframcloud.com/api/v1.0/Add?Bin=" <> bin["UUID"],
  "Body" -> json, "Headers" -> {"Content-Type" -> "application/json"},
  "Method" -> "POST"]

Out[3]= "<|\"Message\" -> \"The data was successfully added.\", \"Bin\
\" -> \"DD56f1e2eac88-e905-4ee5-8b18-7c4bd6072f3c\", \"Data\" -> \
<|\"x\" -> 11, \"y\" -> \"foo\", \"z\" -> True|>, \"Timestamp\" -> \
{2015, 7, 27, 13, 34, 52.135154`8.46970563570117}, \"DataID\" -> \
\"data0bcbcb32-146b-44f5-bcc2-caf713ac9b7d\", \"Information\" -> \
{\"EntryCount\" -> 1, \"LatestTimestamp\" -> 3646992892, \"Size\" -> \
512}|>"

In[4]:= Values[bin]

A couple of things to keep in mind:

  • Make sure the “Content-Type” header in your request is set to “application/json”
  • Data Drop will use the first level of parameters in the json submission. It will not separate nested
4

Is there a good way to separate nested JSON?

5

You can store your data as json in the databin and import it after reading it from the databin.

In[22]:= bin = CreateDatabin[];

In[23]:= DatabinAdd[
  bin, <|"myjson" -> 
    ExportString[<|"a" -> <|"a1" -> 1, "a2" -> RandomInteger[100]|>|>,
      "RawJSON"]|>];
DatabinAdd[
  bin, <|"myjson" -> 
    ExportString[<|"a" -> <|"a1" -> 2, "a2" -> RandomInteger[100]|>|>,
      "RawJSON"]|>];

In[25]:= data = Values[bin]

Out[25]= <|"myjson" -> {"{
   	\"a\":{
   		\"a1\":1,
   		\"a2\":100
   	}
   }", "{
   	\"a\":{
   		\"a1\":2,
   		\"a2\":60
   	}
   }"}|>

In[26]:= ImportString[#, "RawJSON"] & /@ data["myjson"]

Out[26]= {<|"a" -> <|"a1" -> 1, "a2" -> 100|>|>, <|"a" -> <|"a1" -> 2,
     "a2" -> 60|>|>}