Group Abstract Group Abstract

Message Boards Message Boards

3
|
12.7K Views
|
6 Replies
|
6 Total Likes
View groups...
Share
Share this post:

Clean package update for API/FormFunctions on Wolfram Cloud?

Posted 8 years ago
POSTED BY: Kuba Podkalicki
6 Replies

Kuba, do you have any update on this to share with us? This is a nice topic to WTC2021.

Reading the docs, maybe the IncludeDefinitions option can help. I’m studying CloudAPI deeply.

POSTED BY: Rodrigo Murta

Unfortunately I don't have good news. I went with automatic dependency collection for APIs and Initialization>Get[...] in cloud notebooks. I am fortunate that there isn't anything fancy in my code like working with the LibraryLink, external data/caching or whatever.

While I don't like it didn't cause any problems except I would gladly shave those few milliseconds associated with loading.

And afaik there is no solution. Even with the EPC you'd need to restart it which is hardly a convenient solution.

POSTED BY: Kuba Podkalicki

As you have observed, api kernels (for now) don't clear their symbols after every evaluation. So, yes, your symbols definitions remain as long as that kernel remains active (I don't know how many evaluations are done before a kernel is killed and a new one started). Note that a user that calls an api has no control over which kernel will run the code (again, as your test above demonstrates).

As far as the 'inherently unreliable' comment, consider this case: your nemesis uses the same package name and symbols as you. So when your api gets to the Needs["TestPackage`"], it is possible that your nemesis' package was already loaded, and her symbols will be used in the remainder of your api. I'm not sure that this means apis are 'inherently unreliable'. I think it means you should use Get in your api.

I am unaware of any documentation about the configuration of the cloud. I have learned much of what I know by doing experiments such as the one you showed ab

POSTED BY: Chad Knutson

Thanks for your input Chad. Yes this is what I meant by 'inherently unreliable'. And I still think so, here is why:

Get wouldn't be the core of the solution as old DownValues reordering/overwriting by new ones would be hard to predict, so ClearAll["TestPackage", "TestPackage*"]` is a must I think. Which requires forgetting about Locked+Protected, let's say I can live with this. But as you've noticed my nemesis may not be so flexible and could leave them.

One could say it is highly unlikely but we don't need to talk about custom packages, there are packages widely used by community and since Paclet's are not documented, in case of adjustments, people will just add custom changes to those packages by hand, without any versioning info. It will be not possible to manage this, especially on the public cloud.

The bottom line is, all this is unacceptable in development/deployment/testing/debugging cycle. Sooner or later it will cause hard to debug problems. This should be officially discouraged untill tools to manage API kernels appear or a documentation of how it works is written. I don't mean details but an overview which will help to predict what causes what.

Till then the official guideline should to trust the Language`FullDefinition, and to develop isolated packages for API. Which is still arguable as it is a black box solution but at least one can write reports to support based on CloudDeploy docs:

CloudDeploy[expr,[Ellipsis]] automatically deploys all definitions needed to evaluate expr, much like CloudSave.

p.s. people should not need to do test like this to figure out fundamental aspects of working with cloud.

POSTED BY: Kuba Podkalicki

I would advise you to use Get rather than Needs. This is the public cloud after all; the api kernels are running code for many different users.

So, simply avoid using the Locked attribute. Near the beginning of your package, you can Unprotect all symbols that will be defined and later Protected. If you don't want other users to be able to read DownValues, then ReadProtect them.

POSTED BY: Chad Knutson

So If my API Gets a package from my private $UserBaseDirectory then everyone whose API/Form is attached to the same kernel will have access to my package? Is it account type dependent or is using packages by API/Form on WPC inherently unreliable?

This is the public cloud after all; the api kernels are running code for many different users.

I don't see how is that obvious. Could you point me to the relevant part of the documentation?

All things considered I don't think Get/Needs or fiddling with attributes matter here as sooner or later interference will happen.

So I should probably setup deployment in such a way that Language` will take care about my definitions. And I should trust it will shield it well. Is this package and the flow documented anywhere?

POSTED BY: Kuba Podkalicki
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard