Group Abstract Group Abstract

Message Boards Message Boards

1
|
11.6K Views
|
10 Replies
|
4 Total Likes
View groups...
Share
Share this post:

Initializing data structures so application works in the cloud

I have notebooks that start by pulling data in from local spreadsheets and databases (such as PostgreSQL). The notebook then proceeds to perform tasks using this data.

At a point, the data I use becomes stable. I want to publish this notebook to the cloud with data embedded in the notebook so the active parts of the notebook continue to work such as manipulates, and tooltips, without requiring the notebook to continue to import the data from external sources.

I want to be able to access my cloud notebooks from an iPad.

How do I code a notebook to allow this functionality?

POSTED BY: Lawrence Winkler
10 Replies
Posted 4 years ago

Lawrence:

Have you considered deleting the resource objects before you deploy a new one?

Check out DeleteObject[].

POSTED BY: Mike Besso

I started with ResourceObject, ResourceData, and the like. Giving up seems like a cop-out. Using these functions has got to work, doesn't it?

POSTED BY: Lawrence Winkler
POSTED BY: Bob Sandheinrich
POSTED BY: Lawrence Winkler
Posted 4 years ago

Hi Lawrence,

Is there a reason for not using CloudPut and CloudGet as suggested by @Jan Poeschko

POSTED BY: Rohit Namjoshi

Nearing a solution?

Performing

ResourceSearch["ElectionCloseDataset"]

I get a dataset back, now with two rows, and a column heading of "ResourceObject" whose contents seem to be the dataset I assigned to the "Content" label when I created the resource object (see the code above). Why do I have two rows returned? My guess is Ive run the CloudDeploy two times and so I have two copies of my data in the cloud. ResourceUpdate seems to work only for local resources.

Get[ResourceSearch["ElectionCloseDataset", "Objects"] // First]

Works to get my original dataset back. Hooray! Still in Hack mode but perhaps getting closer.

POSTED BY: Lawrence Winkler

So, I'm now trying use ResourceData to push my data into the cloud. Then I want to reference that data in another NB to process it. Seems simple enough.

importSpreadsheet[] := Module[{items, names, data, assoc},
   items = 
    Import["/Users/wkr/Documents/Elections/Closing Graph.csv", 
     "CSV"];
   names = items // First;
   data = items // Rest;
   assoc = AssociationThread[names, #] & /@ data;
  Dataset[assoc]
   ];
buildDataResource[ds_] := Module[{},
   ResourceObject[Association[
     "ResourceType" -> "DataResource",
     "Name" -> "ElectionCloseDataset",
     "Content" -> ds,
     "Description" -> 
      "Dataset created from the Closing Graph Spreadsheet"
     ]]];
storeGraphDataset[] := Module[{ds},
   ds = importSpreadsheet[];
   buildDataResource[ds]
   ];

Now, generate a resource object and deploy to the cloud

ro = storeGraphDataset[];
CloudDeploy[ro, "ElectionCloseDataset"]

The above seems to work.

Now, in order to use this resource, I want get back the dataset portion of the resource, with code in a different NB.

ro = ResourceSearch["ElectionCloseDataset", ResourceObject]

I get back the resource object but none of my attempts to return the dataset that should be in the ResourceObject works.

What is the correct syntax?

POSTED BY: Lawrence Winkler
POSTED BY: Jan Poeschko

I've been playing around with WL ResourceObjects and related functions. I'm guessing I might need a notebook to pull in the data and push it out to the cloud. Then a notebook to find the resource objects in the cloud, pull it in and run the processing.

I think I made a mistake using New -> Repository Item -> Data Repository Item to create a resource object. Seems too complicated for what I wanted. It's a notebook template for documentation of the data, and there is a button to deploy and another to submit. The submit became a request to ask Wolfram to make the data available to others as though my data would be of interest to anyone but me. I really didn't want to do that.

Simple and stupid is all I can muster at the present.

With Mathematica, I'm not yet much of a risk-taker to just try something and see if it works. I don't have the knowledge to know what problems I could be causing when stuff doesn't work and how to debug it when stuff doesn't work. I don't want to clutter up Wolfram's cloud with my poor decisions.

POSTED BY: Lawrence Winkler
Posted 4 years ago

Lawrence:

This is a great question and I will also soon need a solution to this.

I did google "Mathematica store data in notebook" and there are a few suggestions.

And, if I remember correctly, one of the recent upgrades added the ability for a dataset to store its data inside of a notebook.

Check out $NotebookInlineStorageLimit

I am hoping others will chime in with some workflows and best practices.

POSTED BY: Mike Besso
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard