Message Boards Message Boards

Cloud deploying a scheduled task?

Posted 8 years ago

Hi, all

I am trying to deploy a ScheduledTask that monitors the state of a google spread sheet every hour. I want the cloud object to display the contents of the csv file that I am monitoring. Here is a look at my code:

This imports the contents of the google doc:

Clear@url
Clear@csv
(url = "https://path-to-google-doc/export?format=csv") //
Import[#, "CSV", CharacterEncoding -> "UTF8"] & //
Flatten@# & //
(csv = #) & //
Dataset@# &

Which results in an output that takes the form: Dataset[{....}].

To create the ScheduledTask, I have tried both:

 Clear@obj
 obj = ScheduledTask[{Now, csv}, "Hourly"](*creates a scheduled task that monitors the information in the doc every hour*)//
 RunScheduledTask[#]& //
 CloudDeploy[#, Permissions -> "Public"] &

and

Clear@obj
obj = ScheduledTask[{Now, csv}, "Hourly"](*creates a scheduled task that monitors the information in the doc every hour*)//
StartScheduledTask[#]& //
CloudDeploy[#, Permissions -> "Public"] &

The former returns this, and the latter gives me this. I am pretty new to using the Wolfram Cloud so extensively, but it's more and more beginning to feel like a black box, which makes debugging pretty difficult because I am never sure where the issue truly lies; my thought is to wrap the scheduled task in an APIFunction, but I am not sure how to go about doing that, since there is no specific association I am looking to make for the information in the csv file. Is this the right way to go about this? How can I utilize APIFunction if I don't have an explicit set of rules I am looking to make and how is the Cloud processing all of the information I am attempting to send it?

POSTED BY: Jesse Dohmann
6 Replies

Hey Jessica,

ScheduledTasks are meant to just run code on a given schedule, but they do not store a value. What you want is an AutoRefreshed object, like this:

CloudDeploy[
    AutoRefreshed[
        Dataset[Flatten[Import["https://path-to-google-doc/export?format=csv", "CSV", CharacterEncoding -> "UTF8"]]],
        "Hourly",
        "CloudCDF"
    ],
    Permissions->"Public"
]

AutoRefreshed objects will store a value and repeatedly update it according to the given schedule. As far as debugging goes, for ScheduledTasks you can use ScheduledTaskInformation can help show you some information as well as give you access to the task's log file to see more specifically what is going wrong with the task. And in general, GenerateHTTPResponse can help a lot with testing code without actually having to deploy anything to the Wolfram Cloud.

POSTED BY: Dylan Boliske
Posted 8 years ago

I've never seen this GenerateHTTPResponse function before, it's very useful! Can AutoRefreshed be used in conjunction with ScheduledTask? I ask because I have another function I've created:

CheckForChanges[] :=
 If[Length[csv] > 9, 
  SendMail["user@email.com", 
   csv]](*if new elements are added to csv sends email alert to user*)

That I would like to run every time the document is being checked and the value gets consequently updated

POSTED BY: Jesse Dohmann

You can actually do all of it in the AutoRefreshed like this:

getSpreadSheet[] := Dataset[Flatten[Import["https://path-to-google-doc/export?format=csv", "CSV", CharacterEncoding -> "UTF8"]]];
CloudPut[getSpreadSheet[], CloudObject["OldSpreadSheet"]];
CloudDeploy[
    AutoRefreshed[
        Module[{currentCSV = getSpreadSheet[], oldCSV = CloudObject["OldSpreadSheet"]}, 
            If[currentCSV =!= CloudGet[oldCSV], 
                CloudPut[currentCSV, oldCSV];
                SendMail["To" -> "user@email.com", "AttachedExpressions" -> {currentCSV}]
            ];
            currentCSV
        ],
        "Hourly", 
        "CloudCDF"
    ],
Permissions -> "Public"

]

It's a little bit more involved than before since you need to keep track of what the old spread sheet looked like, but it only takes an extra CloudPut to handle that.

POSTED BY: Dylan Boliske
Posted 8 years ago

Dylan this is excellent! Thank you! I'm a big fan personally of PostFix notation and pure functions (sorry, @Andrew de Laix!), so I've been making some small formatting adjustments. However, when I attempt to run it as it, I'm met with server errors in the Cloud; in fact, when breaking down the code, I come to find that any AutoRefreshed function I am attempting to deploy is met with a 500 server error. Strangely enough, wrapping with GenerateHTTPResponse instead of CloudDeploy gives me an a-ok 200 response. I'm not sure why this is (I am not that knowledgeable about the Web, unfortunately, but I'm trying to improve), though I do know there are sometimes general inconsistencies when moving between the Mathematica notebook and the Cloud.

POSTED BY: Jesse Dohmann

Scheduled tasks aren't the easiest thing to understand from the documentation which can make them tricky to get working right. The key thing to remember is that a ScheduledTask is nothing more than some block of code to be run periodically with the result essentially discarded (though it will be written into the task log but not in a way that is easily accessible for computation). To start, you should write code in an interactive session that does exactly what you want. In this case, it is something like import data from an external source, modify it, and finally save that result somewhere. Remember, the result of a scheduled task isn't saved anywhere for you; you have to decide where you want it and CloudPut or CloudExport it. When it works interactively, you can create a ScheduledTask expression that contains that code you want to run with the frequency you want it run. The scheduled task will run exactly the code you give it so be sure it contains all the operations needed. If you want to run that in the cloud, all you need to do is CloudDeploy the expression and it should start working. Note that the highest frequency you can schedule in the cloud is hourly. Also, don't use the RunScheduleTask function which doesn't do what you think when run on cloud objects and I believe is going to be deprecated.

If this doesn't really help you, I suggest just trying to write the code in an interactive session and get the result you really want. If you have questions about just that I'd be happy to help. When you have that part working, try deploying a SheduledTask to the cloud. If you log in to the cloud you can see it listed in your Scheduled Tasks list in the Cloud Files side panel. There's a context menu you can use to run the task immediately to see if it worked the way you expected. I can also help you with this if you are still having difficulties.

And finally, to be honest I find postfix notation very challenging to read. You may want to avoid that except for trivial cases.

POSTED BY: Andrew de Laix
Posted 8 years ago

This is a very helpful description of what exactly ScheduledTask is doing, thank you. Using @Dylan Boliske 's suggestion of AutoRefreshed should let me store information:

AutoRefreshed[csv, "Hourly", "CloudCDF"]

I am not sure what the other differences between AutoRefreshed and ScheduledTask are (they seem almost the same from the documentation description which, as you noted, is not easy to understand), or when one is preferable over another. Could I just store the modified export as a global variable? I've been doing that with the variable I've defined as csv, which is just a level 1 list of the elements in the spreadsheet (this allows me to create the function below, CheckForUpdates, using pattern-matching arguments). Otherwise, I am not sure what you mean when you say an interactive session, and could use some more help!

POSTED BY: Jesse Dohmann
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract