Message Boards Message Boards

GROUPS:

Announcing the Wolfram Function Repository on Community

Posted 1 month ago
669 Views
|
19 Replies
|
40 Total Likes
|

In his blog Yesterday, Stephen Wolfram announced the launch of the Wolfram Function Repository (WFR). While the blog was the official announcement, this repository has been in development for quite some time, with numerous functions already in use hereabouts. So we are,not surprisingly, quite excited to see this now "go live", and I want to introduce it, informally, in the Community forum. I should mention that several Wolfram Community readers are already familiar with this from our live-streamed meetings, and a few have already had functions published there.

The idea is this. You have a function, say something like MyDailyGoTo[str_String,n_Integer], and you think it is really useful for some purpose. You want it to be available to others, you want it to have a documentation page similar to the ones for functions built into the Wolfram Language, you want to have it when you are visiting someplace and forgot to bring your laptop, etc. You can now have all this. In a notebook go to File > New > Repository Item > Function Repository Item. So with four clicks you are off and running. A templated notebook has appeared, with various fields pre-populated e.g. a name (MyFunction) and a descriptor field. There is a Definition section where you replace the prefilled boilerplate with whatever you need to define your function. Fill in the Usage message section. Provide a few examples. You can have options, applications, show possible issues, fill in keywords and Related Symbols (in the Wolfram Language) for improved discoverability, whatever; all of these have fields that can be added to or left alone. The minimum requirement is that there be a usage message and one example.

If you are new to this, you do not yet know what are the style expectations. No big deal, just hit the Style Guidelines button at the top of the notebook and documentation appears. Want to see what an existing contribution looks like? Hit the Open Sample button. Don't know how to format things like the arguments in the Usage section? Heck, nobody knows how to do that (okay, three people know, but their identities are a corporate secret). Hit the Tools button and new bottons, among them Template Input, appear below the main bar. When you think you have something to submit there is the Check button. Hit that and, well, things happen. An automated set of checks might bring up not just missing item or formatting issues, but also a set of possible actions to redress them. Once the notebook looks right and passes the Check criteria, you can see how it will look once published using the Preview button (with the option to see it in a new notebook or in the Wolfram Cloud). When everything looks just so, hit Submit to Repository. There is also a Deploy button, useful for various purposes when one does not want the work to go into the WFR; see the blog for details.

What happens next? The submission goes into a queue for review at our end. We have over 500 published functions and I have been involved in reviews of more than 300 of them (while not as prolific as some colleagues, I also wrote I think 14 of them). First thing we do is see if the name and description make sense. We then check for formatting issues, assess usability, run the examples, have a look at the code, check for related functions, and myriad other things. We even had a couple of "review" sessions live-streamed. (These were not the ordinary reviews insofar as the boss was in charge, and the functions under consideration were ones that were already published. Also, while I do not think we made this clear, we only went over a random selection from among those we ourselves had authored.)

And following a review? If a function is fine on the first go round (possibly after mild editing), it gets published. If it looks like a good idea but needs work (a common outcome, even for those we ourselves write) then a message is sent to the author requesting revisions. We try to give sufficient detail for this. And rest assured, we do not send to external authors things like a particular 'needs revisions' note I received, which began "Danny you're a moron".

Let me say a bit about what is the expectation for submissions to the WFR. We are not terribly rigid here. Functions you find useful in everyday work, or ones that do very particular tasks that are not covered by existing Wolfram Language functions, or... A function can work in a very narrow area, provided it is something that others in that area might find useful. Existing WFR functions hit areas from basic programming language extensions to specialized graphics to STEM to "just for fun" to user interface extensions to, well, functions for working with WFR functions (e.g. a message formatter specialized to the WFR). If we think a submission is too close to existing functionality, we will tell you that (this has happened with a few in-house submissions, though it seems to be quite infrequent). If we think changes are needed, well, I guess I covered that already. Functions need not be generalized to all manner of inputs; we simply ask that they fail gracefully (e.g. not crash) with unexpected ones. As with any curation effort (e.g. like what was done for the Wolfram Demonstrations Project), we will exercise some judgement as to standards. But we are not reviewing at anything like the standards for the Wolfram Language itself.

So now you have a function in the WFR. Or maybe you are using a function in the WFR, whether yours or written by others. Can it be changed out from under you? We have in place a versioning system, which is still itself undergoing revisions. The idea is that things will be cached on your local machine and/or stored to a Cloud account. Revisions can be obtained when wanted, and avoided when not wanted. We expect we will have a hiccup or two as we proceed, but anticipate a smooth process in the not-distant future. Now suppose a function shows problems, or maybe could benefit from extensions: can this be addressed? Again, there is a mechanism for communicating to us and we are also working on facilitating direct communication to authors (when they opt in for this).

To be clear, we expect there might be growing pains. But the benefits of the WFR are rife: we now have a platform for external contributions of myriad functions, and already they are getting used in day to day workflows. I hope Community members will join in this venture, either by using these functions or by contributing some of your own favorites.

19 Replies

Tracking usage is under discussion. I believe we do count downloads but I'm fuzzy on details. It is not yet determined how we will make this available to contributors but I think we are in consensus that that would be a good thing to do.

Murray,

Thanks for the questions. I don't know what could have caused your initial failure. Perhaps a failed paclet update of the resource function framework code? Regardless, the dead simple use case that you tried is correct and should have worked immediately. There is no configuration or set up needed. With version 12, anywhere that you can CloudConnect, you can just use ResourceFunction["name"] or the gray, pill-shaped blob like you would a normal symbol in Mathematica.

Once you use a published resource function it will be automatically cached on your computer and you should not need to retrieve it from our servers ever again (unless you choose to get an update).

As for the different file paths. You definitely don't need to know anything about them. They are an implementation detail. If you want to know about them, I'm happy to tell you why we have those three paths.

  • Objects/Resources

is where the actual resources objects (ResourceFunction is built using ResourceObject) are stored locally. It contains the full metadata and any content that has been cached along with possible ancillary files. This path is intended to only be used by the built in ResourceObject code. Directly modifying files on this path is a good way to break things in unpredictable ways. However DeleteObject[ResourceObject[...]] should always provide a clean slate.

  • Persistence/ResourceNames

is part of the PersistentValue framework. It is used by ResourceRegister and ResourceObject to provide fast recognition of resource names. This improves the performance of ResourceFunction["name"] from session to session. Again, modifying files directly on this path will probably only give you headaches.

  • DeployedResources/Function

is a default path for local deployments of resources made by LocalCache with no second argument (or with the Deploy button in the definition notebook). This is a path you might interact with by using ResourceFunction[LocalObject["path"]]. Directly modifying the files here is unlikely to cause you any real issues.

I think this thread could use some BirdSay:

ResourceFunction["user:rhennigan/BirdSayCommunityPost"][
    "https://community.wolfram.com/groups/-/m/t/1703373"
]

enter image description here

Thanks Daniel for the nice post. I have already contributed some functions and a bunch are submitted. It is actually also a lot of fun to see new functions appearing every couple days, learn about new stuff and so on!

Just out of curiosity: does Wolfram track the usage of each function? Can contributors get those? I could imagine that functions that are most-used will be considered to be implemented in to the Wolfram Language?

I wish I'd read this about 20 minutes ago when, right before hitting the Submit button on two candidate functions, I said "Oh what the heck" when I could not figure out how to format arguments. The consolation is learning that I am not alone.

Ahem. What it needs, my young friend, is an artistic rendition.

Start by grabbing the full text.

wfrThreadText = 
 Import["https://community.wolfram.com/groups/-/m/t/1703621", 
  "Plaintext"]

I'll spare everyone the long string but it is readily seen that the text of interest is between the second and third appearances of my name.

posns = StringPosition[wfrThreadText, "Daniel Lichtblau"]

(* Out[370]= {{3351, 3366}, {3453, 3468}, {10171, 10186}, {11139, 
  11154}, {11171, 11186}, {11522, 11537}} *)

I have to step in a number of characters to avoid some stuff on the page. Similar issue at the end altthough less stepping needed.

myText = StringTake[wfrThreadText, {posns[[2, 2]] + 160, posns[[3, 1]] - 20}];

And now we make a picture.

ResourceFunction["TextToKaleidoscope"][myText]

enter image description here

The hard part was spelling "kaleidoscope".

I think it would be nice also to have a "What's new section" for new functions for the users visiting frequently the repository. New functions could stay there for a couple of weeks along with their alphabetic or topic sorting.

Yes exactly. I hadn't noticed that. After how long a function is removed from recent ?

A quick note: by "How it works" section, we can access the resource function offline after the first time you retrieve the content from Wolfram Cloud. Information displays the UUID, local cache and persistent path of a particular local object. For example, we have three ways to use BirdSay

localbase

Of course, BirdSayCommunityPost won't work offline because it requires internet connection to retrieve community articles.

If we no longer want the ResourceFunction object to be present on our machine,

DeleteObject[ResourceObject["<UUID>"]]

or

ResourceFunction["BirdSay"] // ResourceRemove

comes in handy to remove the content.

Could you, or somebody, please tell us — with a simple, direct explanation that perhaps a Mathematica user of some 20 years standing — exactly how to use a function already in the Wolfram Function Repository?

For example, suppose I want to use the function SymmetricDifference in my local Mathematica notebook.

  1. How do I retrieve it right now for that notebook?
  2. How do I make it available in subsequent sessions without having to go back to the on-line Wolfram Function Repository each time?

Here's one source of my mystification. Input

    ResourceFunction["SymmetricDifference"]

does return as output some kind of gadget, with a gray background, bearing text like [*] SymmetricDifference (it's a placeholder symbol, actually, where I just showed an asterisk).

Yet strangely,

    Information[%]

gives two ResourceObject errors messages that "The specified ResourceObject could not be found", then followed by a formatted version of:

            ResourceFunction[
            ResourceObject[
            Association[
              "Name" -> "SymmetricDifference", 
               "ShortName" -> "SymmetricDifference", 
               "UUID" -> "4f3f7f00-c70a-455e-8bd4-1eaa2b455ae5", 
               "ResourceType" -> "Function", "Version" -> None, 
               "Description" -> "The complement of the union and intersection of \
            lists with duplicates deleted", 
               "SymbolName" -> "FunctionRepository`$\
            4f3f7f00c70a455e8bd41eaa2b455ae5`SymmetricDifference"], 
              ResourceSystemBase -> "https://www.wolframcloud.com/objects/\
            resourcesystem/api/1.0"]]

Next, if I try, say,

    ResourceFunction["SymmetricDifference"][{1, 2, 3}, {3, 4, 5, 6}]

then again I get a "ResourceObject::notf: The specified ResourceObject could not be found." error message.

So I don't even know the answer to my question 1., let alone the answer to 2.

Somebody needs to write up a simple, clear, direct explanation of how to use the WFR -- and make it prominently available as part of the Mathematica documentation center or, at least, on the WFR site. Without that, all the docs I've seen so far about how to write and submit definitions to the WFR are of little avail except to the insiders who already understand what's going on.

Yes, Mathematica 12.0.0. (macOS)

I just ran CloudConnect[], which apparently worked OK as it returned as output the value of my $WolframID. (Shouldn't that happen automatically, when I start Mathematica? When I do, on the Welcome Screen I do see the signing-in process happening automatically.)

But still the same error as before when I try to use

    ResourceFunction["SymmetricDifference"][{1, 2, 3}, {3, 4, 5, 6}]

namely, the error message "ResourceObject::notf: The specified ResourceObject could not be found." and the output:

    $Failed[{1, 2, 3}, {3, 4, 5, 6}]

Wait... I quit that Mathematica session and opened a new one. Now without evaluating CloudConnect[], the input ResourceFunction["SymmetricDifference"][{1, 2, 3}, {3, 4, 5, 6}] works OK. And so does using the iconized output from ResourceFunction["SymmetricDifference"] in that same expression.

This mysterious failure then success more or less answers my first question.

Now what about an answer to my 2nd question?

And related to that, what exactly are the distinctions between, and relationships among, the following directories?

~/Library/Wolfram/Objects/DeployedResources/Function
~/Library/Wolfram/Objects/Persistence/ResourceNames
~/Library/Wolfram/Objects/Resources

I note that in the first of those three directories I have a subdirectory SymmetricDifference containing two items, namely, object.wl and put.wl, the second of which includes among its associations:

"ResourceLocations " -> {LocalObject["file:///Users/murray/Library/Wolfram/O\
bjects/Resources/4f3/4f3f7f00-c70a-455e-8bd4-1eaa2b455ae5 "]}

Or do I not need to understand the relationships among those directories in order to intelligently (with understanding) use a ResourceFunction?

Nice! The next move is to create twitter box in notebook.

What version of Mathematica do you have? You need 12.

Does CloudConnect[] work on your machine?

Then the input should just be:

ResourceFunction["SymmetricDifference"][{1, 2, 3}, {3, 4, 5, 6}]

and should return: {1, 2, 4, 5, 6}

I have no idea about storage of those functions… Is that important to know?

Greatly clarifies & elucidates; thanks!

It's still surprising that, so far as I can see, the WFR does not tell how to use what's there (except somewhat buried in the linked long blog post by Stephen).

Check out the documentation:

We have a dedicated guide page https://reference.wolfram.com/language/guide/WolframFunctionRepository.html

Here is the page just for ResourceFunction: https://reference.wolfram.com/language/ref/ResourceFunction.html

You're right, it would have been smart to have one or both of these prominently linked in the blog.

Neither of those pages says what is buried in Stephen's blog post: once you've found a particular function on-line in the Repository, by clicking the appropriate spot there, it automatically copies the relevant ResourceFunction expression onto the clipboard, from which you may then paste it into a Mathematica notebook.

Moreover, Stephen refers to it as a "blob", but it's wholly unclear what is meant by "blob" in this context.

Things should not be so hard to figure out. The WFR should have a prominent link to a page that tells one how to do what I just described.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract