Message Boards Message Boards

22
|
29886 Views
|
79 Replies
|
109 Total Likes
View groups...
Share
Share this post:

What is Mathematica For?

I realize that I am possibly the only Mathematica user to suffer from existential angst for the product, but still, its worth pausing to ask the question, what is Mathematica for? Meaning, what is its purpose?

But for those of you pragmatists tempted to roll your eyes and just get on with using it, let me motivate the question with a chart:

enter image description here

You don't need a model to figure out the long term trend in that time series. And someone, somewhere, in WR needs to be asking the question I am posing in this post, if they are seriously hoping to check that trend and reverse it.

Now, let's be fair here. Firstly this is not just a Mathematica issue. For example, here's the comparable chart for Matlab:

enter image description here

So I am not "picking on" Mathematica. Its a general problem for mathematical programming languages. Or perhaps I should say, more accurately, its a challenge for proprietary mathematical programming languages: enter image description here

Now this "analysis" is of course over-simplistic. It could be, for example, that the downward trend in Mathematica interest is a reflection of a decision by WR to increase prices over time, to maximize profits, i.e. it could conceivably be a good thing. But I rather suspect otherwise and instead tend to believe that Mathematica is losing influence and relevance, regardless of whatever I may think of the product.

Another response might be: "Of course we are losing market share to license-free, direct competitor products. What's your point?". To which, I suppose, the answer might be: "Are you quite sure that is inevitable? Are there not examples of for-profit products that are so good they are able to maintain market share in the face of cost-free alternatives?".

The answer to that question is, of course, yes. But in order to maintain your competitive position you have to be crystal clear in your understanding of what the value proposition is, and why customers will (continue to) pay for your product. Which brings me to the subject of this post: what is Mathematica for? Because unless can answer that question, you can't answer the earlier one.

Now I have a theory that the generally held view in WR is that Mathematica is for any kind of programming task you can conceive of. It's a general purpose tool, limited in its capabilities only by your imagination. And at this point we could now queue up a 15 minute informercial featuring examples of applications from Astronomy to Zoology.

But just because you can do something in Mathematica doesn't mean you should: there are often better alternatives. Mathematica has developed into something like a Swiss Army knife, a general purpose tool you can use for almost any purpose. But, often times what you need is a tool with a specific purpose, like a screwdriver, which will make it much easier to complete the task you have in mind.

Let's take a simple example: spreadsheets. I have seen several attempts to get Mathematica to do the kind of thing that Excel can do effortlessly. And the question I always end up asking myself is: "why on God's green earth would anyone in their right mind try to use Mathematica for a task like that?". What is trivial to accomplish in a few seconds or minutes in Excel, typically takes hours of jumping through hoops with Mathematica to achieve a result that is almost always inferior. It's like trying to navigate a freeway in a spaceship, rather than a motor car. Sure, you can do it, buy why??

Another no-no is trying to develop stand-alone software applications in Mathematica. It's extremely hard to do - typically much tougher then using a language designed for that purpose.

On the other hand, there are some things that Mathematica excels at, such as:

  • Mathematics/symbolic logic/symbolic manipulation: superb. Best of breed.
  • Graphics (including animation) ditto
  • Documentation. Also superb. Now as good at Matlab, possibly even superior.

After that, you could pick a bunch of different application areas in which Mathematica is great. But best of breed? I dont know.

Let's take an area I know a little about: time series analysis. prior to 2010, Mathematica was truly awful at handling time series. Basically it didn't. Gradually, over subsequent releases, Mathematica has made enormous strides in this area to the point where it is truly magnificent. For which someone at WL deserves (and I hope has received), several promotions. But despite this, Mathematica is still not best of breed in this area. Why? Because, unaccountably, it is still missing several vital elements of functionality such as Granger causality/ Cointegration and its implementation of important functionality like the Kalman Filter is horribly unintuitive.

This shows two things: (i) it is possible for WR to make Mathematica outstanding in almost any area it chooses to. it's just a question of focus. And (ii) to become best of breed, you need to finish the job! By which I mean, provide something close to 99% of the functionality that a professional practitioner in that area would expect to have available. if it chose to, WR could accomplish that in the time series area in a single release, I would guess.

The problem, I am suggesting, is that in trying to make Mathematica a product that is "all things to all men", it fails to achieve the status of being indispensable to practitioners in a more limited subset of subject areas.

Ok let's go with this conjecture for a moment. Let's wave a magic wand and imagine that we can change Stephen Wolfram's vision about his flagship product, which he has sweated blood over for the last 20-30 years (in other words, this is never going to happen - but its still fun to play the game).

What core capabilities would you have WR focus on with the aim of making Mathematica an indispensable product in that area?

Here are my choices:

(i) Time series & econometrics. Mathematica is so close: just finish the job - it's a no-brainer!

(ii) Machine learning. This also means making sure that the GPU capability works as it should, first time and every time. Mathematica still has a way to go here and its a huge area. But I'm sure WR could do it.

(iii). Meta-Programming. This one is a BIG stretch. But it just happens that WL may have a huge potential competitive advantage, because the language is itself computational. in other words, the building blocks of the language can themselves be manipulated, which is exactly what you need for meta-programming. Try doing that in Python!

On this subject I would say that the starting point would be to create WL functionality capable of generating a single line of WL code that is sufficiently complex and interesting to win the Wolfram One-Liner competition. I realize that's a huge challenge and its not just about the difficulty of navigating the terrain of the Wolfram language. It also involves thinking about what "complex" and "interesting" mean, in this context.

Think of this as a Turing test, for Metaprogramming capability. Once it's passed, it would open up a floodgate of possibilities, both for WL aficionados but also, critically, for those who don't program in WL, or indeed in any language at all. For reference, check out whats going on in the world of "no -code" application environments.

POSTED BY: Jonathan Kinlay
79 Replies

A great discussion, which obviously touched a nerve. For context, I am an ecologist, and one of at most a handful of ecologists who use the WL. I attempt to do almost everything in it, and succeed about 95% of the time.

Some of the more recent posts touched on Wolfram's computational data. It certainly blows people away when you use it in a demo: it seems like magic. But there is an analog to the 'production quality' issue: the built-in data have a black-box vibe to them, whereby tracing those data to a source can be challenging (or at least time-consuming). For this reason I would hesitate to use anything from the Wolfram data repository in an academic publication (my version of 'production'). Which makes it more of a tease than anything.

Overall, the issue seems to be one of breadth. Because the language is so capable in so many areas, Wolfram has built it out almost exponentially into different realms. (Stephen has proudly graphed this on more than one occasion.) It's why it is so fun to play with, and develop ideas. But to make each of those areas state-of-the art you would need equivalent growth in personnel, and that isn't realistic. Which is why so many application areas remain somewhere between 50% and 90% developed. It's a shame because the WL has more potential to be a 'universal tool' than any other I have seen.

The real strength of the platform as it is now lies not in any one application area (except maybe a few really well-developed areas, e.g., symbolic math and physics), but in the ability to move seamlessly between many. This is how I use it: I can ingest multiple large spatial datasets, process them, fit a statistical model, simulate something based in the fitted parameters, then visualize them on a 3D map, all in one notebook. There are tools that would do each part better and/or faster, but I would have to a) install and know them all, and b) keep saving and loading files.

But I suspect most potential users come looking for abilities in just one or two areas, and compare those to their narrow but highly optimized specialist alternatives.

POSTED BY: Gareth Russell

Those "specialist alternatives" often contain modules pieced together by graduate students with no background in numerical analysis, and consequently led to a proliferation of errors spanning decades. Then by demand of specialists, some of the erroneous approaches have found their way into commercial software. Ecology has its share of this, including fantasies such as Bayesian Clustering.

http://wireilla.com/papers/ijcsa/V12N4/12422ijcsa01.pdf

POSTED BY: Richard Frost

Most users of phylogenetic software do not understand what it is actually doing, nor therefore the potential for inferential errors, let alone actual algorithmic or coding mistakes. Which is part of a larger discussion that goes well beyond phylogenetics. But how to solve the problem?

POSTED BY: Gareth Russell

Much of biology has become mathematically inclined over the past 40 years. Early on, Biology departments started teaching their own mathematical analysis courses - in part because math departments refused to add pair-joining and pseudo-metrics to their course curricula. This has not been the case with other math-oriented majors - they require their undergraduates to take the junior level numerical analysis course, and their graduate students to take applied math analysis + numerical analysis II.

POSTED BY: Richard Frost

I'm curious, Gareth, if you feel isolated in your academic arena because so few other ecologists use the WL? Does this mean that others really can't peer review your work (since they don't know how to use WL) and you cannot peer review theirs, when they perhaps use other common ("free") tools like R, Stan & Python?

I invested in WL a long time ago, and resist learning other languages because WL is so productive. Even so, I would be hard pressed to recommend WL to my biology students. After all, what employer really wants WL skills?

POSTED BY: Todd Allen

I invested in WL a long time ago, and resist learning other languages because WL is so productive.

Yeah, I'm in the same boat. I suppose I feel isolated a little, but not in the 'peer review' way. One of the nice things about WL is that I can provide a single notebook that explains all the computational pieces in enough granular detail that anyone with basic skills should be able see that it does what we say it does. And I myself can do basic R, although that is hardly relevant because in my field R is used mainly for stats, via packages. What's needed is that I understand the stat methods and when they are appropriate. I'm not going to unpick the algorithms in, say, a linear mixed effects package.

I actually like coding my own versions of discipline-specific tools, such as those in the VEGAN package. It makes me feel more confident that I know what they are doing, and what the output means.

POSTED BY: Gareth Russell

The "R" package contains both unsound individual modules plus procedures that combine otherwise valid modules into an erroneous computation chain. Somewhere along the line, biology professors started equating "knowing how to program" with "knowing what to program".

POSTED BY: Richard Frost

Want to help me make a new and better version in WL?

POSTED BY: Gareth Russell

Gareth, such an effort would involve eliminating several existing functionalities - so potential users would have to be educated as to why that is the case.

In my perspective, this suite of functions would be invoked by WL and perhaps a WL GUI app for novices; i.e. the R scripting interface would be eliminated.

Further, I'm a fan of commercially supported scientific software so I'd like to see Wolfram involved. This would mean that some of the software development monies went Wolfram instead of graduate students. This would also ensure the package did not become another free-for-all collection of erroneous modules.

Just to be clear - I have no grievances with community-based "non-computational" software. For example: LaTeX.

For the immediate future I'm heavily invested in a genomic ID and ancestry project for fruiting perennials. I'm willing to consider your proposal though. For further discussions feel free to contact me via the email posted on the webpage on my Profile.

POSTED BY: Richard Frost

The functionalities you've listed only scratch the surface. For example, consider the interconnectivity to devices. Mathematica is used as a controller for laboratory apparatus, as a receiver for remote sensing applications, and so on. Take a stroll through the main Language Documentation webpage and you'll find more capabilities you missed.

With regard to LaTeX, it's remarkable that Wolfram provides any support at all to something that is not a product but rather an ever-growing collection of TeX macros. As Peter points out, there is serious support for TeX. You can also use the Mathematica "Copy As" (select, right click) to paste Mathematica output as TeX and paste it into your TeX document. If you need something fancier, then use your programming skills to generate TeX within Mathematica and write it to a TXT file.

POSTED BY: Richard Frost

Mathematica supports LaTeX with TeXForm, and if you press Ctrl+4, you can type TeX into Mathematica. Stephen Wolfram describes in his blog post where he announces the release of Mathematica 12.2 how you can enter TeX into a notebook.

POSTED BY: Peter Burbery

There are two additional items I should have considered in my earlier presentation of what I considered to be the key features of Mathematica/the Wolfram tech stack, but which I neglected to discuss properly.

The first is such an integral part of Mathematica that it is easy to overlook - and yet it is one of its core features and a cornerstone of the technology, i.e., the Notebook. This concept has been copied by others (e.g., Python), but no one has ever surpassed Wolfram in the functionality and flexibility of its notebook technology.

The second feature that is gradually becoming more and more important is computational data. This is something that I believe Stephen has, from the outset, regarded as being integral to the Mathematica product, as demonstrated by the integration of Wolfram Alpha and the development of the Entity Store concept.

My sense is that, at the beginning, when computational data is limited, this feature looks rather unimportant, and only over time does its real significance become apparent to the end user. An analogy might be made with Wikipedia. At the beginning, when articles were scarce and of questionable quality, the enterprise looked doomed to failure. But gradually, over time, more users discovered the site and perceived how useful a universal online encyclopedia might become. This drove the creation of more, higher-quality articles, which attracted more users, and so on. There is a concept in Systems Dynamics that describes this phenomenon, known as the "success to the successful" paradigm. This is what I believe is now beginning to happen with Mathematica/Wolfram Alpha, although the rate of development and user acceptance is slower than for Wikipedia as the technology is proprietary.

POSTED BY: Jonathan Kinlay

Hi David, Thank you for your well organized, thoughtful post. I agree with much of it. Like you, I am happy to see the emergence of useful structures such as AxisObject. However, I have concerns about the reduction of "wrapper" functions such as the ListPlot family you mentioned. One is the effect on legacy codes of substantial length at site-licensed institutions - both public and private. The second concern is for the user base from the non-mathematical arts who have learned just enough WL to be productive. For example, consider applications in comparative semantics. A third concern is with function names that are synonyms. Consider Union and DeleteDuplicates. Should Wolfram attempt to force some users to increase their set theory vocabulary, or conversely force the avid set theory user base to change long standing mathematical terminology? I think not. So while the long list of function names deserves review, I believe there are many criteria to consider.

POSTED BY: Richard Frost

Mathematica started as a computational algebra system; a product with its own proprietary language and unique notebook system to integrate computational code with text and graphics.

It works very well as a sandbox computational environment for e.g. research, analyze, document and develop new models and computational methods (algorithms). It’s strengths are:

  • Symbolic computation combined with numerical methods
  • Functional and rule-based programming (and procedural)
  • Large library of functions so you can write concise code
  • Well-written documentation
  • A notebook environment.

Mathematica is a fun system to work with and it delivers results quickly, even though its learning curve is steep (maybe because some of us came from procedural languages and there are many ways to achieve the same result).

I don’t agree with Jonathan on the use of Excel versus Mathematica. Mathematica is my go-to system to analyze data and models and yes it replaced Excel for me in most cases.

I used Excel for large datasets, with pivot tables, VBA and many worksheets. However, my code to analyze data became opaque or hidden in many worksheet formulas. As such, Excel became unworkable. Now I may use Excel to create a well-formed CSV file, which I import to create a Dataset and analyze it in a Mathematica notebook. I spent a lot of time to master Datasets and associations and now it pays off.

Mathematica's language i.e. the Wolfram Language, has transcended its original product: it has become the vehicle for all things computable. So Jonathan’s question “what is Mathematica for” is relevant.

I think Mathematica should serve a niche market, comprising of mostly researchers, analysts and maybe some system designers. It should become a vehicle of state-of-the-art computational functions and methods. I agree with Jonathan: WR should focus on finishing and polishing e.g. statistics and time series analysis, machine learning and a number of other application areas, so it becomes the go-to tool for research. If you have Mathematica, you shouldn’t need any other specialized application to do your research.

I think the WL should focus on a computational ecosystem with capabilities to develop, test and deploy code, create APIs and GUIs for different OS or distributed computing environments. Its symbolic computational capabilities and extensive library of functions make the WL an ideal language for education or data analysts.

Having said this, I see a number of improvement areas to facilitate the WL/Mathematica adoption:

Transparency of outstanding issues. If Mathematica becomes the indispensable tool for researchers, then more transparency about stability, reliability and accuracy becomes paramount. We need to be able to rely on the tool and hence we need insight in outstanding issues, bugs and workarounds. I hope access to redmine is a first step into more transparency.

Reduction of the number of functions to a select number of unique ones. Mathematica prides itself of its 5000-odd number of functions. However, I see a lot of redundancies i.e. new functions that do things slightly different from similar functions. I like to see a reduction of redundant functions. For example take the ListPlot function: we have ListLinePlot, DateListPlot, StepListPlot, DateStepPlot etc. they are all of the class “ListPlot” and with the new axis capabilities, they can all be integrated into one function. This saves us the effort to remember the name of each specialized function, it makes code maintenance and documentation easier for WR too.

Document implemented methods. I would like to see WR open up the black box of some functions in the documentation i.e. explanation of methods used or references to methods implemented.

Conclusion

I think the purpose of Mathematica should be the go-to tool for researchers and analysts. I agree with Jonathan that WR should focus on delivering a complete set state-of-the-art tools.

I think the Wolfram Language could become the computational ecosystem to develop, test and deploy code for all things computable. We have some miles to go there.

At last, the adoption of Mathematica and WL could be facilitated if:

  • We have more transparency of outstanding issues and bugs.
  • The number of "redundant" functions has been reduced to a number of unique ones.
  • We see more documentation to help us look inside the black box of some functions.
POSTED BY: Dave Middleton

This is a very well thought-out reply with several interesting points.

Now I may use Excel to create a well-formed CSV file, which I import to create a Dataset and analyze it in an Mathematica notebook. I spent a lot of time to master Datasets and associations and now it pays off.

Dave may well be right that the trade-off between using Excel vs Mathematica for pivot-table type analysis comes down to a question of familiarity and dexterity with the WL functionality. Mathematica has a steeper learning curve than Excel, but once you have mastered tools such as datasets and associations you can do much more in MMA than Excel, of course. And there is a bonus in being able to do everything in one application rather than switching between two.

I believe this is what WR is aiming for:

If you have Mathematica, you shouldn’t need any other specialized application to do your research.

which leads logically to Dave's conclusion that:

WR should focus on finishing and polishing e.g. statistics and time series analysis, machine learning and a number of other application areas, so it becomes the go-to tool for research.

I agree in principle that:

If Mathematica becomes the indispensable tool for researchers, then more transparency about stability, reliability and accuracy becomes paramount.

However, that is always going to be something of a challenge for a development platform built with proprietary software. Similar issues may arise w.r.t. Dave's other appeal for more documentation of implemented methods.

I, too,

see a lot of redundancies i.e. new functions that do things slightly different from similar functions.

But I am less convinced than Dave that this may be cluttering up the language unnecessarily - the jury's still out on that question, I think.

I find myself in complete agreement with Dave's final conclusion that:

the Wolfram Language could become the computational ecosystem to develop, test and deploy code for all things computable. We have to some miles to go there.

POSTED BY: Jonathan Kinlay

In my experience, a proprietary solution can be worth having. CDC had a great Algol compiler and the Cray Research Fortran compiler was the best bar none. Every now and then I see a bit of that style in the Mathematica error diagnostics.

I don't enjoy reading requests for a less proprietary WL platform in the same paragraphs with requests for internal details of algortihms. If you want to know what was method was used you can look it up in the documentation. Beyond that seems to be some sort of claim of entitlement that does not exist.

POSTED BY: Richard Frost
Posted 1 year ago

Hi Tom,

I think a technically educated person should be writing with a word processor, doing reports with a spreadsheet, and thinking with a technical computing language as an extension of their thought process.

This strikes me as too prescriptive. Fifty years ago you would have had to write, perhaps: "a technically educated person should be writing with a fountain pen, doing reports using a pocket calculator and hardly thinking at all about technical computing languages, unless they are computer scientists".

The point is: time moves on. If you work at Wolfram Research I imagine you (try to) do everything in WL, whether it be writing text, producing some analysis, or programming an application. Certainly that's my understanding of Stephen Wolfram's expectation, anyway. For myself I do a large proportion of my work in WL, including carrying out analysis, creating presentations, and programming applications. It's very convenient to use a common UI and tech stack for all these varied tasks: it save shifting from one application to another, having to be concerned about file format compatibility, etc.

A technical computing language is not the right tool for producing commercial computer applications.

I understand your point, but I expect that there are legions of Fortran programmers, to say nothing of the good people at WR and Mathworks who would heartily disagree with you. And indeed on the pages of this post several examples are given of commercial applications developed in WL (including one or two of my own).

To your point, however, it's obviously a significant challenge to develop a commercial application in WL, even though it can be and has been done. I certainly wouldn't regard the development of commercial applications as a forte of Mathematica, in they way that, for instance, symbolic programming clearly is. Which goes to the topic of this post - i.e. what is Mathematica for? And I would have to concur that the development of commercial applications is not one of those things - although I am quite sure that Stephen Wolfram and others at WR would strongly disagree.

My answer: “it’s an extension of one’s thought process.”

This is a bold claim for any programming language/technology platform, including Mathematica. Could one not make a similar claim about C++, Java, or Julia? I suspect that, unless you are a professional developer, probably not. Mathematica has certain unique features that make it a natural fit for the rapid exploration of ideas, in a way that certainly isn't true of most other computer languages that I am aware of.

I think that your description encapsulates what Stephen Wolfram may be aiming for: a kind of cognitive sand-pit offering a plethora of computational tools that provide a unique way to explore ideas, whether experimentally, graphically, or analytically, that will assist the process of developing them into functional algorithms.

POSTED BY: Updating Name

Fifty years ago you would have had to write, perhaps: "a technically educated person should be writing with a fountain pen, doing reports using a pocket calculator and hardly thinking at all about technical computing languages, unless they are computer scientists".

Fifty years ago I had a programmable HP calculator, was writing with a word processor on an HP portable, doing reports with Lotus 1-2-3 (spreadsheet) on the portable, and programming in octal, Algol, Macro (assembly language), Basic, and Fortran on time-sharing systems. I was not a computer scientist nor had I taken any C.S. courses in college. These activities were typical among my colleagues.

POSTED BY: Richard Frost

You may want to revise your 'way-back' time. Lotus did not come out until 1983. The HP-35 debuted in 1972, but the programmable HP-65 came out in 1974 -- at a whopping $795 -- equivalent to $4k in today's dollars. HP portables no earlier than 1987.

I started coding in 1972, and it was on a timesharing PDP-5. No personal computers at all -- Altair 8800 debuted in 1974. I got an HP-35 shortly after it came out, which meant that I could ditch my slide rule and log/trig tables. Most of us who learned to code in those days didn't take CS, because most schools didn't offer them. (I was a maths major, so I did have numerical analysis.)

So, 50 years ago (1972), I was writing on a typewriter (an IBM Selectric if I did so at work), doing calculations with my slide rule or walking to the Medical School computer facility -- or by hand, and when I did learn to code, used Assembly and BASIC. C came out in 1972, but I did not have access to a computer the compiler (etc.) would run on until the late 1980s.

SO, no need to make the 1970s 'better' than it was. It was the wild frontier.

I wasn't referring to consumer products. We bought these systems from laboratory suppliers or the build-it yourself versions from Heathkit. Fountain pens were a rarity - we all used ballpoint.

POSTED BY: Richard Frost
Posted 1 year ago

I think a technically educated person should be writing with a word processor, doing reports with a spreadsheet, and thinking with a technical computing language as an extension of their thought process.

I observe that people don’t think about things much. If they do, they use a spreadsheet. And that may be too complicated for some. The market for technical computing language may be limited to what I think of as “practicing PhD’s”. Mathematica makes it practical to think about a problem in terms of differential equations, for example, but one first needs some understanding of calculus and not everyone does.

I have recommended Mathematica to others but so far none have been able to use it well enough for it to be an extension of their thought process. It’s a steep learning curve. I have been using Mathematica since version 1.

A technical computing language is not the right tool for producing commercial computer applications. They are two very different things. This is not good or bad, just two different things. And tools for commercial computer applications are well developed. I had an application where I was writing c-code for a microcomputer. It needed a digital filter. I used Mathematica to simulate the data and to design a digital filter. Expressing the filter as c-code was trivial.

My typical Mathematica notebook often starts with some simple notes on a topic. It extends to some “proof of concept” analysis. Then expands from there. This was the development pattern in a recent activity where I extracted telemetry data from a GoPro MP4 file and used it to analyze sailing performance.

My answer: “it’s an extension of one’s though process.”

Years ago, we may have understood that an analysis might be theoretically doable, but we did not have tools to do it. Now with a technical computer language as an extension of one’s thought process we can. The possibilities for expanding understanding become endless. Perhaps we should put more emphasis on teaching a technical computing language. It might enable people to use, for example, calculus in a meaningful way.

POSTED BY: Tom Compton
Posted 1 year ago

We should leave Wolfram Alpha out of this discussion about programming languages. This is not a programming language at all, but a search or - if you like - answer engine. You can't do anything in it that could be called "programming".

POSTED BY: Werner Geiger

Hi Wener,

I am not sure I agree, for two reasons.

Firstly, the question was "What is Mathematica for", not "What is the Wolfram programming language for". i.e the question was intended to be a little more general. Indeed, as you yourself have pointed out, there is an

impenetrable jumble of products and licenses

i.e it's getting harder to identify the precise boundary lines between each product offering. I believe this is termed "confusion marketing". i.e. it's a deliberate marketing tactic.

Secondly, a major thrust in recent years has been to embed computable data / entities and related functionality within the WL, for example in geographics. I believe Stephen's vision for "Mathematica" is as a blend of computable data and programming capability. Indeed, access to Wolfram Alpha is itself embedded in Mathematica, as you know.

I recently programmed a prototype of a game that makes extensive use of computational geographics data. It would have taken many times longer to program that in e.g. Python, as a great deal of time would have been expended in data retrieval and wrangling. In Mathematica, it was relatively easy. That's a good example of why it makes sense to include computable data in the technology stack.

POSTED BY: Jonathan Kinlay
Posted 1 year ago

You are right. Your question was "what is Mathematica good for".

I mistakenly narrowed that down to programming languages. Perhaps I was misled by the word "Mathematica". After all, I don't really know what the word means. What was obviously meant was "what is the Wolfram System good for"?

Of course, Wolfram Alpha is a very important and valuable part of the Wolfram System.

POSTED BY: Werner Geiger

I think Mathematica is good for writing computational essays. Stephen Wolfram has a blog post on this.

POSTED BY: Peter Burbery

Every language is an algebra.

POSTED BY: Richard Frost

I agree that Wolfram Alpha is becoming indispensable. I think Wolfram Alpha is used by math students, math teachers, science students, and science teachers.

POSTED BY: Peter Burbery

A couple of recent comments led me to review the original post and some of the replies from community members which, I have to say, I find rather interesting (although I suspect Stephen Wolfram would perhaps beg to differ). A thought on this subject that I have been entertaining for some time is the concept of Mathematica as a kind of computational Wikipedia. Perhaps, if it has any relevance at all, the comparison could be better made between Wikipedia and Wolfram Alpha, rather than Mathematica.

In any event, what I find interesting is that for a long time Wikipedia looked like a hopeless quest: in the early days many subjects were barely covered and the quality of the articles was often very poor. But then, amazingly, the whole idea took off and Wikipedia became the kind of indispensable reference that Jimmy Wales imagined it could become.

I think something similar may be happening with Mathematica (or, rather, Wolfram Alpha) as its scope has enlarged over time (and the quality of the reference material has always been excellent). My sense is that it is close to some kind of tipping point, on the verge of becoming the kind of indispensable computational resource that Stephen Wolfram had in mind from the outset.

Do others agree? Or take an altogether different view?

POSTED BY: Jonathan Kinlay

I am a Wikipedia editor and I think Wikipedia is a good reference.

POSTED BY: Peter Burbery

The universities I'm familiar with have site licenses, typically at the department or laboratory level. Also, the Mathematica pricing for students appears in line with textbook prices from my perspective.

The main hindrance I see for students is with department level inbred thinking that community-based free software is better in all regards. However, in biology these projects have become the largest source of research errors in modern history:

http://wireilla.com/papers/ijcsa/V12N4/12422ijcsa01.pdf

POSTED BY: Richard Frost
Posted 1 year ago

Maybe it's different in your country. I am in Germany. And I am not only talking about students in maths or engineering, but social and many other sciences that have at least to do much with statistics and probabilities.

POSTED BY: Werner Geiger

I'm aware of research units in university social sciences and liberal arts with Mathematica installations.

POSTED BY: Richard Frost
Posted 1 year ago

A Wolfram|One Personal licence for Student costs EUR 333/Year (!). This is for one user, two installations. Euros are about the same as US dollars. This is a price never acceptable for students. Unless they have rich parents.

POSTED BY: Werner Geiger

What about the cost of a Mathematica (desktop) student license in your country?

POSTED BY: Richard Frost
Posted 1 year ago

The prices for student Mathematica desktop licenses in Germany:

enter image description here

POSTED BY: Hans Milton

Given the prices of textbooks, I believe that is very reasonable.

POSTED BY: Richard Frost
Posted 1 year ago

I reviewed again the German prices for Student's Wolfram / Mathematica. I found three products/licenses for Students: enter image description here

They range from 104 €/Y to 333 €/Y. The cheapest one does not include the cloud (which probably is not very important, since everybody stores and synchronizes their files within other clouds, namely OneDrive for Windows users).

I could not really tell the difference is between Mathematica Desktop (181€/Y) and Wolfram|One Personal (333 €/Y) (I have the latter, but - strange enough - for 271 €/Y).

I don't even know what the difference is between Mathematica and Wolfram Language. I look at "Mathematica" as just the former name of "Wolfram Language" and abbreviate it as "WL".

Putting it all together, possibly a student could live with Mathematica Desktop for 104 €/Y. "Possibly" because I assume that this does not include any product upgrades which would make it pretty useless. If that is true, he would have to pay at least 181 €/Y, possibly 333 €/Y.

POSTED BY: Werner Geiger
Posted 1 year ago

Much has been said here that I understand and agree with. Especially the uselessness for real production systems. I myself have discovered WL only as about the 20th programming language since I am a pensioner. Before that it was always about large systems in FORTRAN, PL/1, C, C++, Java as a system architect and developer.

Today I program just for fun, and for that WL is great. For me especially because of the symbolic and mathematical capabilities and graphics. And I love notebooks that contain code and its documentation at the same time. Now I do everything simple with EXCEL VBA and everything else with WL.

I would like to add two important points that I think hinder the success of WL:

  • The impenetrable jumble of products and licenses, i.e. the marketing.
  • The price. For people who just want to get to know the product, especially students, a license is way too expensive (the cloud is not a way out, in my opinion you really need a desktop version with local files).

The second point is existential. Students can do all their tasks for free with things like Microsoft Visual Studio, Eclipse, C#, C++, Java, R, Python and the like, but nothing with WL because they can't afford a license.

POSTED BY: Werner Geiger

I think the ability to do symbolic math is underrated. I'm a hobbyist programmer, and I once encountered a problem that really did require to do symbolic math. I really, really didn't want to do the calculations with pen and paper so I bought a Mathematica license. Since then, I've learned to appreciate the capabilities of the language and the elegance of its design.

POSTED BY: Lucien Grondin

Yes, I completely agree. I used MMA to help solve a very challenging problem in stochastic calculus many years ago and find it invaluable for scratch-pad type quant work, or more complex research in mathematical finance.

POSTED BY: Jonathan Kinlay

I think one thing that Mathematica does well is solving differential equations including differential-algebraic equations with DSolve.

POSTED BY: Peter Burbery

Indeed.

POSTED BY: Jonathan Kinlay

Mathematica has been the platform of choice for sensors-to-animation production by the film industry for decades. A colleague of mine is a system administrator at one of the sites.

POSTED BY: Richard Frost

I had no idea! Is there a site you could point me to, in order to get a better understanding of the application?

POSTED BY: Jonathan Kinlay

The famous example are the animations added to the actors in Pirates of the Caribbean.

POSTED BY: Richard Frost
Posted 1 year ago

Do you have any references for that? I'm curious.

POSTED BY: Eric Rimbey

There is a video of the making of Pirates of the Caribbean, although it's been years since I watched it. As for references to specific sites: that is a matter of intellectual property.

POSTED BY: Richard Frost

Some comments from a long time programmer (coming up on my 50th anniversary) and Mathematica user (32 years):

First, Bernard Gress's comment that WL exists so that Stephen can get his projects done is truer than might be expected. Certainly, the best way to get feature 'x' implemented is to get Stephen interested.

Second, I think that there really is confusion about what Mathematica is for. I initially used it solely to solve maths problems, and for that it was, and is, superb. The first edition of the Mathematica book had the subtitle: "A system for doing mathematics by computer". Over the versions, the definition of what maths is has expanded, and a bunch of other stuff has been added.

The notebook interface is great for this type of investigation and interactive exploration.

However, it is rubbish at making stand-alone apps.

It is also far too hard to do computer-science or complicated programming. When I learned c, almost everything I needed to know was in the K&R book, some 125 pages or so, as I recall. Sure, you needed to understand the standard library, but the header files gave the information you needed. This level of documentation is missing in WL, although the general level of documentation otherwise is very good. With c, the step between the 'simple' stuff and a complex program was gradual and the simplicity of the core language made learning what to do relatively straight-forward. With Wolfram language, it is by no means clear what the core language is, and understanding it is not simple. This makes going beyond the basics much more difficult than it should be. At Apple in the early 2000s, they told developers that Xcode (and objective-c) made the regular stuff easy, and the hard stuff possible. With Mathematica, the mathematical functionality for interactive use is made very easy (compare to writing your own), but anything beyond that has been made much more difficult than it needs to be.

To give an example, it is not clear from the documentation how to pass a variable by reference -- if it is there, it is pretty well hidden.

Your comment about losing something when key people leave is very telling, especially for Mac users. When Theo Gray was involved, you could expect that Mathematica would make use of Apple technologies -- it was largely due to him that Mathematica remained a Mac program through the dark days. However, I am no longer feeling the love, specifically in the areas of UI and GPU utilization. As far as I can tell, Apple's neural engine is at least as good as anything based oil NVIDIA, yet it is not used at all for machine learning in Mathematica. Embracing other new OS features has been very slow -- it took far too long to move the front end to 64 bit, for example.

There is also the tendency to leave stuff at the 99% done level and move on to the next shiny object. There are simply too many bugs. There are too many similar functions that do pretty much the same thing, but which have subtle differences in the API or performance, and the documentation does not make that clear at all.

When I retired, I decided that I would let my skills in c (etc....) lapse in favor of doing all my coding in WL. I have come to regret this decision. When I use WL as part of my mathematical studies, I have (mostly) no problems. For anything else, it is a constant slog to achieve even moderate success.

I do have proposals for a solution:

  1. Fix the bugs.

  2. Market Mathematica (Wolfram|One, etc.) for what it does best -- interactive computation.

  3. Fully implement the Wolfram engine so that it can be used as a resource when developing stand-alone code in pretty much any language for any platform. Specifically, this means making the engine work with iOS and iPadOS so that other developers can develop apps with a state-of-the-art UI and functionality, yet making use of the strengths of WL. (A similar case can be made for Android, as well.)

  4. Make full use of the native capabilities of each hardware platform. This means making use of all the technologies in macOS (and iPad/iOS), Windows and Linux, although the most obvious mismatch is with macOS at the moment.

  5. Institute a developer program (with nominal costs attached), so that developers will have access to essentially whatever or whoever tech support gets their undocumented answers from. I remember that the Wolfram Technology conference was originally called the Mathematica Developer's Conference.

  6. As a corollary, provide some really state-of-the art developer tools. I tried using Eclipse, but compared to the tools that I used when developing Mac apps (Lightspeed c, Code Warrior, Xcode), the tool is clumsy and out-of-date. I have no experience in Java development or development on Linux or Windows; perhaps Eclipse is perfectly usable for these platforms. I know that discussing tools is likely to lead to a religious war -- after all there are those who think c++ is the best of all possible languages. However, if Wolfram Research wants to break out if its ghetto, it will need to integrate with modern IDEs, and Eclipse is not going to help with this (in my opinion, of course).

  7. Consider making a new language. All older languages have had re-thinks about what the language means. C has had several successors, with different levels of success: c++, Objective-c, and Swift. While I am not fluent in Swift (yet) it seems to have captured the essence of what was good about c without the excesses of c++, and is easy to learn. A similar rethink of WL would be a really good idea. While many of the design objective of WL were good, not all were, and some of them are real howlers -- especially in the area of UI design and implementation. Done right, no old code would be obsolete (you can still use ANSI c in Swift), but things would be a lot easier both for typical users and experts. Of course, if Wolfram Engine were available for macOS and iPadOS, some enterprising person or group could re-implement WL from within Swift, so this might not be so far out after all. (Remember Swift is not proprietary -- and I am using that only as an example.)


When I first used Mathematica, I was developing (in-house) real time process control software for a large clinical lab. I would solve the mathematical problems and prototype graphs, etc., using Mathematica, then translate the results into c on a Macintosh. This combination worked extraordinarily well. If my suggestions were implemented, and I were doing the same thing today, I would do the prototyping in WL, then use the Wolfram Engine in a native program I would write in Swift for macOS and iPadOS that would make native use of Apple's Neural engine. This would result is a more useful and stable application than I could ever hope to do using WL alone.

So, to answer your question: what is Mathematica for?

In its current state, Mathematica is for interactive problem solving of primarily mathematical problems using the notebook interface.

I do not think it is effective when used for anything else (certainly by not anyone outside of Wolfram Research) : web deployment, etc. except as a way to publish notebooks interactively, seems to be a toy project.

With proper focus, as I outlined above, Wolfram Language could be a significant tool in the development and implementation of basically any type of scientific application.

Hi George, thanks for taking the time to comment in detail. You have plenty of relevant experience with WL and have come to some clear views as to its strengths and limitations.

First, I agree with this comment:

There is also the tendency to leave stuff at the 99% done level and move on to the next shiny object. There are simply too many bugs. There are too many similar functions that do pretty much the same thing, but which have subtle differences in the API or performance, and the documentation does not make that clear at all.

I also largely agree that:

The notebook interface is great for this type of investigation and interactive exploration. However, it is rubbish at making stand-alone apps.

But WL is not completely rubbish at production systems. I am aware of at least one product, which I believe was developed 100% in WL that works extremely well, even by the standard of a production environment. I, too, have produced a small WL application that has shown itself to be very robust (although it's bit of a cheat as its just WL wrapper for a C++ system, really).

I also struggle with this view:

It is also far too hard to do computer-science or complicated programming.

I've written some pretty complicated programs in WL - not in the sense that developers typically mean, which often just refers to tens of thousands of lines of code related to a UI for example, but in terms of core mathematical, statistical and econometric concepts, which (in my view) is much more important (and complicated) stuff.

Yes, WL is a large ocean and for the new swimmer it can appear confusing and intimidating. But is not that hard, because of WL's nature as an interactive, functional programming language. The trick is simply to start by formulating some small component of the system you want to build, test it interactively to ensure it works as intended and then move on to the next component. Then you string all the components together, one by one.

I think the difficulty is that in languages like C, Fortran, Python, etc, the structure of the language often dictates a single, obvious way to tackle a problem. WL's diversity and flexibility means that there are often multiple plausible approaches one could take. In WL, I often spend ages thinking about which method might be best and experimenting with alternatives in order to arrive at the best solution. Sometimes, as I do that, I feel I am wasting time - but this is usually more than made up during the implementation stage, which tends to progress much faster if I have spent time at the outset thinking more deeply about the problem I am trying to solve.

Finally:

  1. Fix the bugs.
  2. Market Mathematica (Wolfram|One, etc.) for what it does best -- interactive computation.
  3. Fully implement the Wolfram engine

Perhaps as you say the solution to the challenge facing WR is not to try to "fix" MMA to enable it to do production work, but to focus on the Wolfram engine instead...

POSTED BY: Jonathan Kinlay

Perhaps as you say the solution to the challenge facing WR is not to try to "fix" MMA to enable it to do production work, but to focus on the Wolfram engine instead...

Exactly. Of course, the Wolfram engine, as I understand it, is mostly the Kernel, plus some bits that ended up in the front end for historical reasons. So, it really isn't focusing on the Wolfram Engine as a separate project, but making sure that there is functional parity -- as well as full support for native hardware.

An interesting test case will be whether, as part of the port of the Wolfram Engine to Apple Silicon, it will also be available for iPadOS. If this turns out to be the case, then I can see a lot of apps that use the Wolfram Engine, but which use a custom UI. One of the issues (among many others) with porting Mathematica to iPad is the need to make the UI for the iPad version as close to the desktop as possible, a formidable task given two fundamentally different design principles at work.

I want to re-iterate that Mathematica and its notebook interface is perfect for a wide range of applications and probably is the best in class for the vast majority of Mathematica users. But it should not try to be all things to all people: the best hammer in the world is not much use if you need a wrench.

Hi George,

as far as I know the Wolfram Engine is already ported to iOS for the App "Wolfram Player" but not embeddable in a custom application.

https://apps.apple.com/us/app/wolfram-player/id1059014516

POSTED BY: Philipp Winkler
Posted 1 year ago

In fact Wolfram has an embeddable solution for iOS: It's called CDFKit and it was presented many times in Wolfram Technology Conferences by Rob Raguet-Schofield and Jason Harris. CDFKit can be integrated as an iOS-Framework in Objective-C or Swift projects.

The availability of CDFKit seems to be bound to a purchase of an Enterprise Version of Mathematica (which is costly) and maybe a special contractual arrangement (to publish/distribute a certain number of apps).

This is of cause way too complicated: availability and usage should be as easy as free Wolfram Engine.

I agree what was said earlier: Apple Silicon (CPU, GPU, Neural-Engine) should be exploited and optimized much more for special tasks as AI, ML and -- if possible -- general compute. Maybe CDFKit could be adapted also as a "lightweight" Wolfram Engine for macOS.

It would be quite easy for Wolfram Research to develop a similiar "educational app" like Apple's Swift Playgrounds for iPadOS. A mini-REPL on the left with WL syntax coloring and CDF-Output incl. DynamicModule/Manipulate on the right (using CDFKit). This would be a much better experience than the Wolfram Cloud App.

The ultimate question remains: Where is Mathematica for the iPad?

POSTED BY: S. M. Wirker

I think the complaints about cost are misplaced. Wolfram products are in high demand and so I don't believe the price is going to come down.

POSTED BY: Richard Frost

Having said that, the Wolfram Engine would not be regarded as a complete solution by many:

(i) production systems developed using the WE require a production license and distribution of the WE runtime module

(ii) it entails development of the system in another programming language (which calls the necessary functionality in the WE).

In other words, in this mode of usage the WL alone is insufficient to create a production system. It is necessary to be proficient in another programming language such as C++ or Python to create the complete application. From that perspective, the WE is just another library (albeit a powerful one).

POSTED BY: Jonathan Kinlay

My thoughts on this topic, besides that it should recieve more public discussion, are twofold.

First, I am not sure whether I will buy Mathematica once I exit academia. As long as I have a 'free' license though, I'm hooked. I think both mathematics and computer science students have a hard time approaching MMA even though it has tremendous potential to benefit both kinds of people†. I like to tell them that MMA is the worlds best software rapid prototyping tool; or when I'm being cynical, that MMA is both good for nothing and good at literally everything††. Frivolous yet powerful things are often the most compelling and fun...

Second, can MMA ever have a practical use†††? I think besides small niches, there is exactly one possibility. I haven't played with it's C code generation very much, but it seems like it's mainly limited to producing 'zombie routines' that must hook into a kernel. The symbolic nature of WL admits, for instance, a search over all possible algorithms to achieve a task. I think a richer library of code generation tools/examples plus an expanded code generation interface (i.e. x11 calls, various windows dlls; think of existing gpu and ffmpeg functionality) could start to make WL useful for 'actual code development'.

†In many lower division math classes, the entirety of homework can be automated in MMA. Document scanning, equation parsing, equation solving with steps, typesetting and printing. Literally the whole process: it's a thing of beauty. Applied computer science students, once relieved of worrying about the nontrivial trivialities of modern software, can appreciate the beauty of theory.

††Good for nothing in that it's useless for producing anything dynamic (i.e. production software), and good for everything in that it's capabilities are wiiiidespread and richly interconnected and as a result can produce amazing static visualizations, sounds, videos etc. Most academic publications are entirely static--most of scholarship is just prototyping ideas--such that MMA is the perfect scholarly workbench.

†††Again, practical in the sense of providing a directly valuable service--there's some economics word I'm looking for. One might say I'm asking for the well crafted code that goes into the kernel to be freely given up. Instead, I'm suggesting that the symbolic power of WL could be suited for automatedly writing lots of code--I would only be able to give examples if I were to embark on this as a project.

POSTED BY: Adam Mendenhall

Adam,

I think my view is quite well aligned with yours: Mathematica is unparalleled in its facility for doing mathematics and its interactive nature makes it an outstanding choice for doing research, experimentation and prototyping. Which thereby also makes it an excellent pedagogical tool.

If we were to draw a line here and say: "this is what Mathematica is for", would that be so bad? Does its current inadequacy as a platform for developing production systems render it useless? No of course not, not at all. Why should it? After all, I have never heard anyone complain about R's shortcomings in the same area - why should Mathematica be held to a higher standard?

Well, I think there may be a couple of reasons (in addition to cost, of course):

Firstly, if we dispense with the notion that Mathematica can ever become a platform for doing series development work in a production environment, it means that you have to become highly proficient in at least one other language. And given the scope and complexity of WL, and of the other language one may be obliged to learn (e.g. Python, Java, C# or C++ ), a developer can be forgiven for thinking that they may as well save some time and effort and just focus on the second language, which can do everything (although perhaps not as elegantly as WL, in some areas).

Second, Stephen Wolfram's boundless enthusiasm and ambition for the WL leads one to hope that perhaps, eventually, a way will be found to remedy the WL's deficiencies and make it suitable for development of production systems. This is, after all, what Mathworks appears to have achieved with Matlab/Simulink, enabling Matlab code to be translated into production-suitable C++.

However, I suspect this is just a pipe dream: there are all kinds of structural reasons why a similar translations capability would be next to impossible to achieve in WL (beyond the very limited extant ability to compile simple functions).

POSTED BY: Jonathan Kinlay

You make some good points but maybe your premise is wrong. Is Facebook dying? enter image description here

Google trends might be deceiving. As a frequent WL user I never search for WL!
I think the WR is addressing many of the isssues you mentioned: creating standalone software - Wolfram cloud and WebMathematica -- WL add-on for Excel -- etc. I do agree that some applications need more finishing and polishing. As an Engineer I appreciate the addition of FEA to Mathemetica but it still needs a little more. The statistics capabilities are tops but why not add design of experiment capability. The presenter notebooks are fantastic if only I could use my custom theme background images like power point can. Nevertheless Mathematica remains to be my preferred tool to solve complex problems symbolically and numerically, simulate systems and present results.

POSTED BY: Kay Herbert

Hi Kay, yes indeed it might that the premise is wrong (and in my original post I gave one possible explanation for the downtrend that is innocuous). In this case, however, I think that perhaps the chart for Facebook accurately reflects the slowdown in usage growth of the platform:

Source: Statista

Source: Statista

So, while it certainly doesn't mean Facebook is dying yet, it tells us that growth is slowing, which may ultimately mean the same thing.

In term of your other points, I think we are saying the same thing: Mathematica has come so far as to be considered excellent in many fields, but seems to lack the last x% of functionality to render it truly outstanding (like design of experiments in statistics, in your example). Perhaps that is by design. A rational argument might be presented along the following lines:

Usage growth Mathematica is slowing. We (i.e WR) can't reverse that trend by providing more functionality to existing users who are already committed to our product. We have to find new users and bring them on board. To do that we have to provide new functionality that will appeal to potential users who haven't tried Mathematica before. That argues in favor of adding more breadth to the product scope, rather than more depth, beyond a certain level.

If this argument were to be accepted, it would mean that WR would be constantly developing functionality in a field up to a certain level - typically below what is required by a professional at the forefront of the field - before abandoning it to focus on developing new functionality in other areas.

It seems to me that WR needs a "finish the job" Tzar, whose function would be to identify the critical functionalities missing from key subject matter areas and with the authority to prioritize them for development.

In many cases the gaps in necessary functionality could be filled with a handful of new functions. But for some fields (e.g. machine learning) the list would be very long indeed.

POSTED BY: Jonathan Kinlay

I agree Jonathan and I am also worried about the long term future.

In my opinion the following issues have to be addressed first:

1.) Polishing and bug fixing of existing features

  • Inconsistencies in TimeSeries
  • Bugs and missing features in Machine Learning (Batch Prediction, GPU-support, Support for Apple Silicon and ML engine, …)
  • Performance of Dataset, …
  • Improve Developer Tools (update Workbench, support for other editors - might be coming as far as I know)
  • Fixing long standing issues in various kinds of functions. Very often when I try something new, I stumble over bugs which give an unpolished impression of the product and the features run out of gas after they are 90% finished.

2.) Generally „Big Data“ and Data Processing has to be handled better

  • fast converter / importing routine for numbers, datetime, …
  • Out of core storage
  • Supporting CSV - Files with „;“ deliminator
  • support for secure web sockets wss - Protocol
  • => focus on performance for processing huge amount of data

It is really surprising what basic features are missing in this area. Python is far ahead.

3.) Deployment for Production

  • as far as I know efforts are made in this area (Application Server) but nothing really useable in an easy way has been released so far
  • Licensing has to be pain free, no cloud credits or something like this but permanent license
  • Deployment / packaging of an app has to be easy and stable
  • Deployment on own infrastructure
  • support long running services (eg. Bot)
  • Automatic error recovery, restart, logging, …

4.) Predictable bug fixes and releases

  • when a bug is reported it is simple unpredictable for a customer if it gets fixed at all and when it gets fixed
  • This is especially annoying for regressions. Fixes have to be pushed out quicker.
  • I wouldn‘t suggest WL for a critical app in an enterprise because of this
  • as long as this is not worked out, WL is only useable for research and experimentation in my opinion

5.) Create and promote examples for real world use cases in enterprises (IoT, Predicitive Maintenance, Big Data, ML Recommendation engine, Trading Bot, …) outside of research and development. Whenever I talk to somebody about MMA nobody thinks about production usage of it. This is probably the hardest part to build up awareness.

That said I really like the concept and idea of Mathematica and I want it to last for a long time and be as stable as possible. I enjoy discovering new things and areas where I had no idea about before.

POSTED BY: Philipp Winkler

Philipp,

Thanks for your thoughtful input. One can only hope that someone at WR is paying attention...

Perhaps the single biggest shortcoming in Mathematica, compared to, say, Matlab or Python is in regard to deployment in production. I have developed a couple of production systems IN WL, but my sense in each case was that I was pushing the envelope in terms of Mathematica's core capabilities. I always had to be ready to handle bug fixes and crash-recovery scenarios. Unlike other languages, which also had their fair share of bugs and crashes along the route to deployment, there never really seemed to come a time when Mathematica could be expected to work seamlessly - there always seemed to be another problem waiting in the wings. Similar systems developed in C#, C++, Python or Matlab would always (eventually) get to a point where they were reliable enough to deploy in a production setting, with a reasonable expectation of reliable performance. But I can't say the same for systems I have developed in Mathematica.
Now, it could be that this is just my lack of experience/expertise in developing production ready systems in WL. But I also think that it is perhaps one of the key limitations of the language. It's not to do with the language being interpreted - so are Matlab and Python - but some other aspect of the language architecture (e.g. memory management) that renders it unsuitable for deployment in production. I wish it were otherwise; but I can live with that limitation, providing I know it exists.

In any case, while the title of the post is about what Mathematica is For, it is perhaps also useful to produce a list of things that Mathematica is NOT suitable for, which from my perspective would include:

1) Anything that can be done more easily in Excel

2) Anything requiring deployment in production

What about other users' experience? Can anyone else offer an alternative view of Mathematica's capabilities in relation to development and deployment in a production setting?

Are there any other limitations you see Mathematica as having?

POSTED BY: Jonathan Kinlay
Posted 3 years ago

Jonathan, could you please elaborate on what you mean by:

Anything requiring deployment in production

Does it mean the possibility to convert a Mathematica application so it can be used by a non-MMA user? Or a close to bug free MMA application for a MMA user (a "bug" may reside in the applications algorithm, not necessarily in MMA itself)? Or something else.

POSTED BY: Hans Milton

Yes, it would include standalone applications designed for general use and not requiring user familiarity with Mathematica (or possibly even a Mathematica runtime module). In Matlab, for example, applications developed in Matlab can be compiled as standalone C++ applications - a capability that Mathworks has worked long and hard on to produce.

But I also mean WL applications that are designed for use in the WL by Mathematica users (e.g. in package form, in a Mathematica notebook). Even here, as I said, it is challenging to produce an application for use in production, where application errors and bugs are very infrequent and remedied relatively easily. In most other languages bugs tends to get ironed out eventually, whereas my experience is that a Mathematica application is much harder to stabilize and often continues to surface problems. This is not always the case: for example my MATH-TWS application (which connects Mathematica to IB TWS) has proved very reliable. However, the great majority of that application is written in C++ and interfaces to the IB TWS api. So it doesn't really count as a true Mathematica application in the sense I mean here.

POSTED BY: Jonathan Kinlay

I recently spent almost two weeks to install keras(the most star count deep learning framework on github) to my gcp compute engine linux debian machine, keras now bundled with tensorflow2, only can run inside docker container, docker itself is cloud native technology, and I also spent great portion of time to install the gpu support in docker, now everything seems ok to go. The Capability to package app inside docker to run seems the way to go in cloud era.

POSTED BY: vincent feng
Posted 3 years ago

Hi Philipp,

Wolfram Application Server was announced today. Perhaps it will address some of the issues we face with deploying and supporting WL-based applications in production environments. There is no mention of pricing / licensing but I suspect it will likely be out of reach for most startups / small companies.

POSTED BY: Rohit Namjoshi

Hi Rohit,

Thank you for the information. I will take a look at it as soon as there is a trial version available. There was also a presentation about WAS at WWDC 2020. Hopefully it will be easy accessible => pricing information available online without contacting sales, trial license for download, …

POSTED BY: Philipp Winkler

@Philipp Winkler, about bugs, the recently released Wolfram Research Issue Tracker:

https://redmine.wolfram.com

is a good sign for the path forward on this.

POSTED BY: Kapio Letto

Hi Kapio,

I know about redmine. Actually I am a heavy user :) and it is definitely a step in the right direction. Direct contact with QA and sometimes with the developers helps a lot. Still what I would really appreciate is that development cycles are dedicated for bug fix only releases like version 12.1.1. Just polishing the existing stuff. There are also a lot of things fixed in regular releases but because of the new features almost always some regressions end up in the release.

POSTED BY: Philipp Winkler
Posted 3 years ago

I think Mathematica is far behind Python. Since I have used Python, I seldom use Mathematica. Python is much more powerful than Mathematica in data processing.

POSTED BY: Wenguang Wang

Wenguang,

I happen to agree with you. But what would you say are the specific data processing capabilities that Mathematica lacks, or is inferior to Python, from your experience? Can you give some examples?

POSTED BY: Jonathan Kinlay
Posted 3 years ago

@Wenguang Wang, maybe it is because you did not have a chance to get to the Mathematica's full potential. I heard a different story. My friend uses Mathematica extensively, data science, analytics, general coding, etc. Then she had to use python for some of her new work tasks. She got very frustrated with how not-smooth the experience was with some workflows and the front end in python. To me hybrid numeric-symbolics and fundamental symbolic nature of Mathematica and its functional style coding architecture seems far superior than python. Some similar user opinions here: https://mathematica.stackexchange.com/questions/86058

POSTED BY: Updating Name

I always enjoy using Mathematica for serious and less serious work. Don't want to spend all time on finding the right python libraries and check there vitality etc when most absolutely necessary. I think Mathematica is fantastic and thoroughly enjoy listening to Stephen wolfram's discussions with his outstanding team. I mean I don't know any other cto doing that. I also believe that Wolfram is really looking for "stuff" that is actually useful for his customers. Other languages have also great features and sometimes better syntax but that doesn't matter. The flexibility is incredible. I also enjoy the documentation and all the crosslink to other fields that might be useful. You always learn from it. No other languages has given me more pleasure programming than this Wolfram language. I once played with low code software mendix in combination with sap software. I didn't really like it. Low code for sure but many many many clicks and different windows.. Not for me.

POSTED BY: l van Veen

100% agree with your comments about Mathematica being enjoyable .... one of the great things about it is that it so easily allows you to "play" with concepts and ideas, and even pick up new ones from completely different fields. In that regard WL is unlike any other language I have programmed in (with the possible exception of APL). There is a great deal of tedium involved in programming in most other languages (although I quite enjoy working in R).

That said, I suspect that some of this is because I am not usually trying to build production systems in WL, a process that inevitably involves a lot of humdrum, laborious, house-keeping code. I use Mathematica for R&D, and in that context it's an ideal environment - fast, powerful, broad in scope and highly interactive. And, like many users, I love to dabble in all kinds of interesting topic areas that Mathematica opens up.

POSTED BY: Jonathan Kinlay
Posted 3 years ago

Jonathan, the answer is very simple:

"Mathematica is for Stephen Wolfram to do his NKS-stuff on."

Other features are put in from time-to-time and as is deemed necessary to make Mathematica sufficiently saleable to support Stephen so he doesn't have to get some other less enjoyable occupation while he pursues NKS.

;-)

B

POSTED BY: Bernard Gress

Your op is conversational/general (i like the content and agree, so i upvoted it) and i wouldn't wonder if it gets deleted. Nothing will come out of it (the discussion, if there is any) other than us writers/readers who agree with you feeling better because we said our piece and got the sore topic off our chest, hopefully without ranting. Anyway…

I am in the same boat, having expressed my existential angst for the product from the very start, at the time of registration see the About Me page.

"Success is the sum of right decisions." — If the popularity continues to trend downward, more and more senior key figures "turn their backs" ( @Michael, @Paul, @Roman, et al. because of age, capacity, retirement, or ultimately, expiration), and in 15yrs the product is no more than a relic, then that's fate. To me, $\text{Wolfram L}$ will still be the one and only programming language I am willing to accept; i never gave FREE $\text{Python}$ a serious or fair chance, because i am stubborn and, erh, i am no programmer (and have no interest in becoming one)!

I am no programmer (by profession, by heart, by need, by desire, by whatever) and have no interest in becoming one, also because of my very restricted talents hehe, yet i can write some simple functional code, create updated tables/graphs with a press of a key, and i do enjoy this ability of mine! All thanks to Mathematica. Honestly, I do miss the GUI features of M$ Word, Excel, Corel, Maple but i put up with these shortcomings because my enjoyment of the language is so much stronger. No software package or programming language is perfect or complete but imho Mathematica is the #1 contender in the all-rounder category. Nothing wrong with knowing how to draw maximum use out of a SAK Swiss Army Knife! And best of all, i have access to it all the time:

  1. on the road: through the Android Wolfram Cloud App, on my phone
  2. on the road: through the Wolfram Cloud web browser access (phone, computer, inet cafe, etc)
  3. on the road: through the VNC Cloud access (e.g. on my phone) to my Raspberry Pi at home -> my preferred mobile access
  4. at home: through the VNC Wi-Fi access (e.g. on my phone) to my Raspberry Pi at home
  5. at home: through the VNC Wi-Fi access (on my PC) to my Raspberry Pi at home
  6. at home: through eventual local installations on PC at home

The VNC access is possible only because my Raspberry Pi is non-stop powered by mains 24/7*365, a 2.0 Watts device.

Q: $\text{"So …, What is Mathematica For?"}$

A: $\text{"Well …, it is For Me! Basta ya."}$ — ©2021 Raspi. hh

POSTED BY: Raspi Rascal

Hi Raspi, in some ways I think you are an ideal Mathematica user - a total convert: MMA with everything! And I get it - it's a wonderful sandpit environment for every kind of experimentation.

But in broad terms, my thinking is that this is what Mathematica really is: a means of doing fast research, experimentation and prototyping - but not necessarily for production systems.

If you are not a professional developer that probably doesn't matter at all - everything is an experiment, or recreation! But for a professional developer, getting systems into production and making sure they are battle-ready is a critical part of what you do for a living. And its in this context that I see the greatest challenge for Mathematica - making WL applications that are sufficiently bullet-proof to hand over to an end user (rather than, say, a research co-worker).

BTW what do you use Raspberry Pi for?

POSTED BY: Jonathan Kinlay

My use of Raspberry Pi? Mainly for having the alternative mobile access to Mathematica (for mobile coding, or as calculator) and to my personal locally saved Mathematica notebook files. The web browser access and the Android app access are convenient and very good to have but they are not full versions of a Mathematica installation. So through the VNC access (does everyone here know what a VNC access is, oh please?) you're on the road and have access to a full local Mathematica installation, namely the one on your Raspberry Pi sitting at home near the Wi-Fi router. Seriously, this is reason enough and reason alone to buy and setup a Raspberry Pi, and shame on whomever that there is no youtube video showcasing how Mathematica can be used this way on a phone while you're on the road, apart from browser access and Android app access! Shame. The youtube title should be "How To Use Mathematica On Your Phone Through VNC Access" and it should demo both the VNC Wi-Fi access and the VNC Cloud access. No, i am not gonna do such a video, not my job. It's Wolfram marketing department's job :P

I also used the Raspberry Pi for logging electronic devices, like a logging multimeter, a logging charger/discharger, or other similar eletronic devices which offer a USB connection (or serial connection) for logging purposes. You can let the Pi do the logging for days and weeks, non-stop! There's logging software for Linux/Raspbian, wonderful stuff.

And i don't mind letting the Pi stream music non-stop from a streaming music subscription website, looping an online playlist infinitely. I only need to turn on the speakers and there's music!

Of course, the Pi is not a replacement of a full PC. One can watch a youtube or a HD video with VLC media player but you can't do both at the same time. In summary, my personal main uses are: 1) having access to a full local Mathematica installation (granted the Wi-Fi or mobile internet is working), 2) logging electronic devices, 3) 24/7 streaming of music. There is so much more, purposeful things, one could do with a Pi, but as you can see, I keep it simple, basic, boring.

POSTED BY: Raspi Rascal

You mention using your RPi to access Mathematica remotely. But cant that be done just as easily and more effectively by accessing MMA on your PC via remote desktop when you are travelling)? Is there something specific about the RPi implementation of MMA that makes it more accessible for remote work?

POSTED BY: Jonathan Kinlay

The only difference is that a 2.0W RPi is constantly powered on, in its booted state. A 400W PC usually is turned off for the night or when you're travelling. Other than that, there is no difference between VNC access to RPi or VNC access to home PC, you're right. A full-fledged PC is more powerful than a small RPi, of course. Do you have your PC (or Mac) turned on for 24 7 365, see? With a RPi, one does so, for the sake of nonstop remote access to Wolfram L full installation, at slow performance.

POSTED BY: Raspi Rascal
Posted 3 years ago

I have found Mathematica essential in production settings for at least 20 years. Most of those were before the Wolfram Engine (and the more recent cloud offereings). I have re-analyzed some of the applications I mention below, and we could have cost-effectively used the Wolfram Engine if it had been available. Stability of the kernel has also dramatically improved.

I have used Mathematica in at least three different companies. In a consulting environment, we did all our in-house analyses using Mathematica. Notebooks would call large databases, issue SQL commands, and bring in the data for further processing and visualizations. Most algorithms we needed had to be developed from scratch, and Mathematica made it quick to go from reading a paper or a pencil and paper calculation, to something tested and working on data. The graphics would usually get some tweaking in Illustrator, but would often be close to camera ready. We also used Mathematica to create reference algorithms to be implemented in C++ or Java. Many of these where not numerical algorithms.

At my next company we used Mathematica directly as a compute engine. We had developed a very performant cyber security system that identified executing malicious code by observing the stream of system calls. All the machine learning steps were done in Mathematica (this was before the free Wolfram Engine). It allowed us to have version 1 of our system up way faster than if we had implemented them directly. We experienced memory leaks and random hangs that forced us to periodically kill and restart the compute engine, but we were able to engineer it so that those issues remained invisible to the user. Over time, because of costs, we replaced the Mathematica subsystem with Java code.

In my current work we still develop most of the algorithms in Mathematica. We are building a system that use Mathematica for some core data generation; reference algorithms are still in Mathematica. We also use it as a thinking tool (see my blog post). Having all that curated data at one's fingertips is great.

I feel costs have prevented a wider adoption. Wolfram Research now offers low cost versions for non-commercial applications, which should encourage wider adoption, but I sometimes worry that its market may be shrinking.

POSTED BY: Ronnie M
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract