Message Boards Message Boards

16
|
10530 Views
|
13 Replies
|
39 Total Likes
View groups...
Share
Share this post:

(Plea for a) Free package repository

Posted 6 years ago

Around 22:15 in this talk a paclet repository is mentioned.

To my mind, this is far-and-away the best repository Wolfram could make, seeing as Mathematica's got no effective way to share packages like python, R, JavaScript, or other modern languages.

I think its introduction would make working with Mathematica exponentially better, and so I'm really hoping WRI puts some effort into getting it off the ground sooner rather than later (especially because I would like to contribute my paclets before I drift away from using Mathematica seriously on a daily basis).

For this to work effectively, though, I think it really has to be a free system. That's not to say that people shouldn't be able to charge money for their packages--it's their prerogative if they don't want anyone to use their code. What I mean is that if it costs money to distribute packages/contribute them to the repository, people will be significantly less likely to share their code, which will keep the Mathematica dev community small and in turn decrease the amount to which Mathematica can penetrate into the broader programming community.

POSTED BY: b3m2a1 ​ 
13 Replies
Posted 6 years ago

Since it wasn't clear when WRI would get to this or even if they would do it well, I tried to roll my own version of a public repository. You can find it here

paclet repo

I talked about it some here and here

POSTED BY: b3m2a1 ​ 
Posted 6 years ago

For those interested I've implemented a GitHub-based paclet server: https://paclets.github.io/PacletServer/

paclet server image

The source repo is linked there, but it's https://github.com/paclets/PacletServer

Hopefully this shows the potential of this type of system, even though I don't expect many people to use it.

POSTED BY: b3m2a1 ​ 
Posted 6 years ago

An exciting development that I had entirely missed. Per this Twitch stream

the first step is to make a documentable version of the paclet system

So as of Dec. 13 this was something of a priority.

Hopefully that means it will be available by Mathematica v22.

POSTED BY: b3m2a1 ​ 
Anonymous User
Anonymous User
Posted 6 years ago

who will "rule" this centralized repository, decide merits additions to it, and who will keep viruses out, like "bleeding heart"? (i deleted the rest of my argument to keep forum moderators more happy). wolfram already has a repository(s) which the official release uses.

I found a url that let's you run your own paclet server:

http://community.wolfram.com/groups/-/m/t/968760

POSTED BY: Anonymous User
Posted 6 years ago

I understand where you're coming from. Safety is a concern with things like these. On the other hand without some form of centralized repository there's no way for the Mathematica developer community to thrive and without that I'd argue Mathematica will continue to lose to things like Jupyter. E.g. this bad omen of the future:

uh oh

or the recent popular Atlantic article.

As for paclet servers I'm running one right now for free in Wolfram's cloud and I have to say I'm generally pretty happy except for one or two issues.

POSTED BY: b3m2a1 ​ 

Hi b3m2a1, how about packagedata.net? They even have a link to the latest version of your development tools. Regards, Ruben

Posted 6 years ago

Alas this method suffers from the fact that PackageData.net is merely a collection of links. It does not make use of the PacletManager system that Mathematica has built-in. It is better than MathSource though.

To my mind anything that doesn’t effectively link into the PacletManager isn’t sufficient. There’s been ample discussion of this over on the StackExchange in the PackageData.net chat room.

POSTED BY: b3m2a1 ​ 
Anonymous User
Anonymous User
Posted 6 years ago

Study: Begin[] and BeginPackage[]

The URL below has several resources, one of which is a public lIbrary (there are also private controlled ones).

http://www.wolfram.com/resources/?source=nav

(http://library.wolfram.com/ is the public library, it is at bottom of page, has no icon. it is an older library that required (requires) communication with WR employees, using older "web technology", is less used but still has important past posts, such as code from certain Mathematica functions that are no longer shipped with new versions)

Mathematica has an excellent Package system, more stable/mature than "linux alternatives". But to know this it must be studied.

https://sourceforge.net is a site that has a few "GPL'ed" (free) Packages and projects for Mathematica.

POSTED BY: Anonymous User
Posted 6 years ago

I am more than well-aware of Begin and BeginPackage: https://github.com/b3m2a1. I even know about paclets.

The point is that these are not centralized in a browsable package repository the way they should have been. The paclet repository (presumably it will live at http://resources.wolframcloud.com/PacletRepository by analogy to the DataRepository, NeuralNetRepository, and the seemingly soon to be coming FunctionRepository).

In terms of what does exist, there's PackageData.net but it doesn't have a link-in to the PacletManager and in any case most people there don't make paclets.

In all honestly WRI should have done this a decade ago, but as long as it comes soon I'm not one to grouse too much.

POSTED BY: b3m2a1 ​ 

Hi b3m2a1,

My suggestion is that you indeed put your material in MathSource, so that it is available and well-documented.

http://library.wolfram.com/contribute/

At some stage, there ought to be more way for others to use the wealth of information there. The first thing to do is to archive the lot in a proper manner. You might also write your own overview report of how things hang together. It remains amazing: software ought to make life easier, but comes with so many complexities of its own. Obviously: use keywords that allow a quick link with the existing Mathematica functionallity, so that familiar terms can also find the new stuff. Discussions in the Community could also link up to MathSource, and vice versa.

Best regards, Thomas

Posted 6 years ago

The primary issue with this is that MathSource is a suboptimal distribution platform. At that point it’s better to just use GitHub as that’s the modern way to distribute source code. You can set up good documentation in GitHub. It just takes effort.

POSTED BY: b3m2a1 ​ 

Hello Thomas,

MathSource doesn't actually exist anymore as it did before. It was merged into the "Wolfram Library Archive", which as b3m2a1 said, is suboptimal ... To be honest, I do not think it does much good to submit there. Most of the hits to my packages do not come from there.

Just to name a few problems:

  • Difficult and slow to update packages
  • Full of old packages, most of which do not work anymore with the latest M version. Part of the problem is of course that it is hard for authors to update them there.
  • Most items are not packages at all. Many of the "packages" are just presentations about packages, or mentions of packages, but the package is nowhere to be found, either on the Library Archive or elsewhere. I can recall even very recent instances where a package was publicly lauded yet it is still not accessible anywhere ...
  • The search is just bad. Usually it doesn't give relevant result.

Some of these faults are remedied in part by http://packagedata.net/

But that's not what b3m2a1 is asking for either.

Have you ever used some of the open source competitors to Mathematica such as Python or R? They have a much better package ecosystem and much much better package management capabilities. E.g. R has a central package repository, with some curation and quality control. Installing a package is as easy as install.packages('somepackage'). It'll be automatically found online, downloaded, compiled if necessary, and installed. Loading it is as easy as library(somepackage). There are standards for packages, so they all work the same way. Most come with documentation, which integrated into the main system. Package authors can find plenty of resources on how to create packages that fit in and follow standards. CRAN will automatically create binaries for several platforms for packages that have binary components. There's a dependency system: when you install a package, its dependencies will also get downloaded. And so on.

Python is even better with its excellent package management systems (pip) things like anaconda, and virtual environments for managing collections of different packages and package versions.

Now compare Mathematica:

  • There are relatively few packages, probably because package authors get little training and resources from Wolfram
  • If you need a package for something, you need to google ... there's no central repository
  • If you were lucky enough to find a package, it will likely have no indication about which M version it works with. Often it won't work with the latest.
  • In many many cases, the "package" follows no standard. It might come as notebooks (why? that's not even a package!), as a plain .m file, or as something else ... The installation procedure will differ from package to package, and sometimes so will the loading procedure. Oftentimes, it won't even come with installation instructions. Package authors who want a better experience for their users will often create their own auto-install mechanisms which sometimes ends up not working so well at all

In the recent two years or so there was some indication from Wolfram that we are welcome to use Mathematica's paclet system for our own packages, even though it is still undocumented. This is a step in the right direction, but there's a lot more work to be done.

To sum up: the situation is currently a mess, and quite literally more than a decade behind the competition.

POSTED BY: Szabolcs Horvát
Posted 6 years ago

I'd upvote this 100 times if I could. Thanks for this, Szabolcs. I feel like this has much more impact coming from someone with more respect in the community.

And just so it doesn't get lost–it seems like something like this is coming–the question is simply when it will come and what it will look like. With luck it will be as at least half as good as as the PyPI.

POSTED BY: b3m2a1 ​ 
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract