# Clean package update for API/FormFunctions on Wolfram Cloud?

Posted 2 years ago
2405 Views
|
4 Replies
|
2 Total Likes
|

Cross posted on mathematica.stackexchange

# TLDR

There is a pool of kernels/sessions available for each user, you don't have control over the pool, only over specific kernel you currently evaluate in. Re-upload of a package should be followed by quit on every kernel where previous version of a package was used. I failed to find tools for that.

# Background

I'm developing a package which I want to deploy to my Wolfram Public Cloud's account and I want a set of APIFunctions/FormFunctions/etc to be able to use it.

• PacletManager has limited functionality so I'm just uploading package's archive and extracting it to WPC's $UserBaseDirectory / Applications /. • APIFunctions and friends have form:  APIFunction[{}, (Needs["TestPackage"];Symbol["TestPackageapi2"][...]) &]  I use Symbol because otherwise TestPackage definitions are uploaded and I want to avoid that. There should be one code source, the package in $UserBaseDirectory.

# Problem

The problem is that kernels' management on WPC is a closed black box. If you call the api twice, each time it uses one of the kernels in the pool. It also applies to CloudEvaluate etc.

$UserBaseDirectory is shared but Get will find the current version ony for current kernel. If in another kernel TestPackage was loaded earlier, Needs will not load the current version there :-/ I don't wan't to Get in my APIFunctions, a proper way is to call Needs. And there is no way to Quit every available kernel. # Example is worth of 10^3 words: First we will mimic a package upload, 10 times. So it was uploaded, something was fixed it was uploaded again etc: packageTemplate = StringTemplate[" BeginPackage[\"TestPackage<*\"\"*>\"]; myValue = \"\" EndPackage[]; "]; Do[ CloudExport[packageTemplate@RandomReal[], "Text", "TestPackage.m"] ; CloudEvaluate[ CopyFile[ "TestPackage.m" , StringRiffle[ {$UserBaseDirectory, "Applications", "TestPackage.m"}
, "/"
]
, OverwriteTarget -> True
]
; Get @ "TestPackage"
]
, {10}
]


Now let's call code based on the package 25 times:

Table[
CloudEvaluate[
Needs["TestPackage"]
; {$SessionID, Symbol["TestPackagemyValue"]} ] , {25} ] // CountBy[Last] // Normal // Column  7 different results, I expect only one variant, from the last deployment. # The question How to upload a package and clean properly, as we can see Get after the upload only affects one particular kernel/sessopm. CloudEvaluate@Quit[] won't help either. How to reset them all? # Requirements  APIFunction[{}, (Get["TestPackage"];Symbol["TestPackageapi2"][...]) &]  could solve it but if the package contains Protected/Locked symbols you will get a flood of errors. And because of the same reason you can't ClearAll symbols. And obviously you can't Quit in ApiFunction. Answer 4 Replies Sort By: Posted 2 years ago  I would advise you to use Get rather than Needs. This is the public cloud after all; the api kernels are running code for many different users. So, simply avoid using the Locked attribute. Near the beginning of your package, you can Unprotect all symbols that will be defined and later Protected. If you don't want other users to be able to read DownValues, then ReadProtect them. Answer Posted 2 years ago  So If my API Gets a package from my private$UserBaseDirectory then everyone whose API/Form is attached to the same kernel will have access to my package? Is it account type dependent or is using packages by API/Form on WPC inherently unreliable? This is the public cloud after all; the api kernels are running code for many different users. I don't see how is that obvious. Could you point me to the relevant part of the documentation?All things considered I don't think Get/Needs or fiddling with attributes matter here as sooner or later interference will happen.So I should probably setup deployment in such a way that Language will take care about my definitions. And I should trust it will shield it well. Is this package and the flow documented anywhere?
 Thanks for your input Chad. Yes this is what I meant by 'inherently unreliable'. And I still think so, here is why:Get wouldn't be the core of the solution as old DownValues reordering/overwriting by new ones would be hard to predict, so ClearAll["TestPackage", "TestPackage*"] is a must I think. Which requires forgetting about Locked+Protected, let's say I can live with this. But as you've noticed my nemesis may not be so flexible and could leave them. One could say it is highly unlikely but we don't need to talk about custom packages, there are packages widely used by community and since Paclet's are not documented, in case of adjustments, people will just add custom changes to those packages by hand, without any versioning info. It will be not possible to manage this, especially on the public cloud.The bottom line is, all this is unacceptable in development/deployment/testing/debugging cycle. Sooner or later it will cause hard to debug problems. This should be officially discouraged untill tools to manage API kernels appear or a documentation of how it works is written. I don't mean details but an overview which will help to predict what causes what.Till then the official guideline should to trust the LanguageFullDefinition, and to develop isolated packages for API. Which is still arguable as it is a black box solution but at least one can write reports to support based on CloudDeploy docs: CloudDeploy[expr,[Ellipsis]] automatically deploys all definitions needed to evaluate expr, much like CloudSave. p.s. people should not need to do test like this to figure out fundamental aspects of working with cloud.