Group Abstract Group Abstract

Message Boards Message Boards

Data Drop with images via API?

I have a Python data acquisition application that generates images. I'd like to be able to share those to a DD Databin for subsequent analysis.

It appears that the 'programmatic' interfaces are still 'coming soon' so I presume I would need to use the Web API. I can see how to use HTTP Requests to upload simpler key-value pairs, but images seem to require a bit more heavy lifting than I have experience with.

Does anyone happen to have a simple example of doing this?

POSTED BY: Flip Phillips
7 Replies
POSTED BY: Flip Phillips
POSTED BY: Bob Sandheinrich

So, just to test things, I created a structured databin

foo = CreateDatabin[
  "Interpretation" -> {"Contact" -> "Image", "Object" -> "Image", 
    "Drape" -> "Image", "tContact" -> "Number", "tObject" -> "Number",
     "tDrape" -> "Number", "ID" -> "String", 
    "Experiment" -> "String"}]

I added an entry via its web form located here. Note that each image is c.a. 19Kb for a total of about 60Kb and change.

The resulting databin, with a single entry, is almost 4Mb - which seems like a totally crazy amount of overhead. Notice also that the creation-time is wrong for some reason.

wtf

I have gotten closer sending the data via a POST operation via Python, but, at present, the data is not interpreted correctly (e.g., as an Image ) but is, rather, stored as gigantic pool of binary crud. So... onward!

POSTED BY: Flip Phillips
Posted 10 years ago

Yes, it is. I assume the reasons for the limit are that Data Drop is focused on internet of things data (consisting of small and frequent observations), and that increasing the limit would increase the server load and impact the performance of Data Drop.

POSTED BY: Peter Heemeijer

Do you happen to know if this is a WRI-imposed limit?

POSTED BY: Flip Phillips
Posted 10 years ago

The 25Kb limit of Data Drop can be bypassed by

  1. Using Compress to generate a compressed string of an expression (in this case a figure)

  2. Using StringTake or related function to split the compressed string in substrings smaller than 25Kb

  3. Uploading the substrings to a databin

  4. Downloading the substrings, concatenating them, and using uncompress to reconstruct the original object.

POSTED BY: Peter Heemeijer

I'm getting closer- but

An interesting thing I just discovered- I can upload an image, properly, to the data drop from within Mathematica, but, if I try to use the Web API I receive the message:

"Your databin entry could not be added because it exceeds the 25 kb limit."

from the POST request.

So I'm guessing this is going to make things a little difficult for me, since my images are, generally, larger than 25kb.

POSTED BY: Flip Phillips
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard