Message Boards Message Boards

GROUPS:

What is Mathematica For?

Posted 22 days ago
1421 Views
|
29 Replies
|
46 Total Likes
|

I realize that I am possibly the only Mathematica user to suffer from existential angst for the product, but still, its worth pausing to ask the question, what is Mathematica for? Meaning, what is its purpose?

But for those of you pragmatists tempted to roll your eyes and just get on with using it, let me motivate the question with a chart:

enter image description here

You don't need a model to figure out the long term trend in that time series. And someone, somewhere, in WR needs to be asking the question I am posing in this post, if they are seriously hoping to check that trend and reverse it.

Now, let's be fair here. Firstly this is not just a Mathematica issue. For example, here's the comparable chart for Matlab:

enter image description here

So I am not "picking on" Mathematica. Its a general problem for mathematical programming languages. Or perhaps I should say, more accurately, its a challenge for proprietary mathematical programming languages: enter image description here

Now this "analysis" is of course over-simplistic. It could be, for example, that the downward trend in Mathematica interest is a reflection of a decision by WR to increase prices over time, to maximize profits, i.e. it could conceivably be a good thing. But I rather suspect otherwise and instead tend to believe that Mathematica is losing influence and relevance, regardless of whatever I may think of the product.

Another response might be: "Of course we are losing market share to license-free, direct competitor products. What's your point?". To which, I suppose, the answer might be: "Are you quite sure that is inevitable? Are there not examples of for-profit products that are so good they are able to maintain market share in the face of cost-free alternatives?".

The answer to that question is, of course, yes. But in order to maintain your competitive position you have to be crystal clear in your understanding of what the value proposition is, and why customers will (continue to) pay for your product. Which brings me to the subject of this post: what is Mathematica for? Because unless can answer that question, you can't answer the earlier one.

Now I have a theory that the generally held view in WR is that Mathematica is for any kind of programming task you can conceive of. It's a general purpose tool, limited in its capabilities only by your imagination. And at this point we could now queue up a 15 minute informercial featuring examples of applications from Astronomy to Zoology.

But just because you can do something in Mathematica doesn't mean you should: there are often better alternatives. Mathematica has developed into something like a Swiss Army knife, a general purpose tool you can use for almost any purpose. But, often times what you need is a tool with a specific purpose, like a screwdriver, which will make it much easier to complete the task you have in mind.

Let's take a simple example: spreadsheets. I have seen several attempts to get Mathematica to do the kind of thing that Excel can do effortlessly. And the question I always end up asking myself is: "why on God's green earth would anyone in their right mind try to use Mathematica for a task like that?". What is trivial to accomplish in a few seconds or minutes in Excel, typically takes hours of jumping through hoops with Mathematica to achieve a result that is almost always inferior. It's like trying to navigate a freeway in a spaceship, rather than a motor car. Sure, you can do it, buy why??

Another no-no is trying to develop stand-alone software applications in Mathematica. It's extremely hard to do - typically much tougher then using a language designed for that purpose.

On the other hand, there are some things that Mathematica excels at, such as:

  • Mathematics/symbolic logic/symbolic manipulation: superb. Best of breed.
  • Graphics (including animation) ditto
  • Documentation. Also superb. Now as good at Matlab, possibly even superior.

After that, you could pick a bunch of different application areas in which Mathematica is great. But best of breed? I dont know.

Let's take an area I know a little about: time series analysis. prior to 2010, Mathematica was truly awful at handling time series. Basically it didn't. Gradually, over subsequent releases, Mathematica has made enormous strides in this area to the point where it is truly magnificent. For which someone at WL deserves (and I hope has received), several promotions. But despite this, Mathematica is still not best of breed in this area. Why? Because, unaccountably, it is still missing several vital elements of functionality such as Granger causality/ Cointegration and its implementation of important functionality like the Kalman Filter is horribly unintuitive.

This shows two things: (i) it is possible for WR to make Mathematica outstanding in almost any area it chooses to. it's just a question of focus. And (ii) to become best of breed, you need to finish the job! By which I mean, provide something close to 99% of the functionality that a professional practitioner in that area would expect to have available. if it chose to, WR could accomplish that in the time series area in a single release, I would guess.

The problem, I am suggesting, is that in trying to make Mathematica a product that is "all things to all men", it fails to achieve the status of being indispensable to practitioners in a more limited subset of subject areas.

Ok let's go with this conjecture for a moment. Let's wave a magic wand and imagine that we can change Stephen Wolfram's vision about his flagship product, which he has sweated blood over for the last 20-30 years (in other words, this is never going to happen - but its still fun to play the game).

What core capabilities would you have WR focus on with the aim of making Mathematica an indispensable product in that area?

Here are my choices:

(i) Time series & econometrics. Mathematica is so close: just finish the job - it's a no-brainer!

(ii) Machine learning. This also means making sure that the GPU capability works as it should, first time and every time. Mathematica still has a way to go here and its a huge area. But I'm sure WR could do it.

(iii). Meta-Programming. This one is a BIG stretch. But it just happens that WL may have a huge potential competitive advantage, because the language is itself computational. in other words, the building blocks of the language can themselves be manipulated, which is exactly what you need for meta-programming. Try doing that in Python!

On this subject I would say that the starting point would be to create WL functionality capable of generating a single line of WL code that is sufficiently complex and interesting to win the Wolfram One-Liner competition. I realize that's a huge challenge and its not just about the difficulty of navigating the terrain of the Wolfram language. It also involves thinking about what "complex" and "interesting" mean, in this context.

Think of this as a Turing test, for Metaprogramming capability. Once it's passed, it would open up a floodgate of possibilities, both for WL aficionados but also, critically, for those who don't program in WL, or indeed in any language at all. For reference, check out whats going on in the world of "no -code" application environments.

29 Replies

Your op is conversational/general (i like the content and agree, so i upvoted it) and i wouldn't wonder if it gets deleted. Nothing will come out of it (the discussion, if there is any) other than us writers/readers who agree with you feeling better because we said our piece and got the sore topic off our chest, hopefully without ranting. Anyway…

I am in the same boat, having expressed my existential angst for the product from the very start, at the time of registration see the About Me page.

"Success is the sum of right decisions." — If the popularity continues to trend downward, more and more senior key figures "turn their backs" ( @Michael, @Paul, @Roman, et al. because of age, capacity, retirement, or ultimately, expiration), and in 15yrs the product is no more than a relic, then that's fate. To me, $\text{Wolfram L}$ will still be the one and only programming language I am willing to accept; i never gave FREE $\text{Python}$ a serious or fair chance, because i am stubborn and, erh, i am no programmer (and have no interest in becoming one)!

I am no programmer (by profession, by heart, by need, by desire, by whatever) and have no interest in becoming one, also because of my very restricted talents hehe, yet i can write some simple functional code, create updated tables/graphs with a press of a key, and i do enjoy this ability of mine! All thanks to Mathematica. Honestly, I do miss the GUI features of M$ Word, Excel, Corel, Maple but i put up with these shortcomings because my enjoyment of the language is so much stronger. No software package or programming language is perfect or complete but imho Mathematica is the #1 contender in the all-rounder category. Nothing wrong with knowing how to draw maximum use out of a SAK Swiss Army Knife! And best of all, i have access to it all the time:

  1. on the road: through the Android Wolfram Cloud App, on my phone
  2. on the road: through the Wolfram Cloud web browser access (phone, computer, inet cafe, etc)
  3. on the road: through the VNC Cloud access (e.g. on my phone) to my Raspberry Pi at home -> my preferred mobile access
  4. at home: through the VNC Wi-Fi access (e.g. on my phone) to my Raspberry Pi at home
  5. at home: through the VNC Wi-Fi access (on my PC) to my Raspberry Pi at home
  6. at home: through eventual local installations on PC at home

The VNC access is possible only because my Raspberry Pi is non-stop powered by mains 24/7*365, a 2.0 Watts device.

Q: $\text{"So …, What is Mathematica For?"}$

A: $\text{"Well …, it is For Me! Basta ya."}$ — ©2021 Raspi. hh

Hi Raspi, in some ways I think you are an ideal Mathematica user - a total convert: MMA with everything! And I get it - it's a wonderful sandpit environment for every kind of experimentation.

But in broad terms, my thinking is that this is what Mathematica really is: a means of doing fast research, experimentation and prototyping - but not necessarily for production systems.

If you are not a professional developer that probably doesn't matter at all - everything is an experiment, or recreation! But for a professional developer, getting systems into production and making sure they are battle-ready is a critical part of what you do for a living. And its in this context that I see the greatest challenge for Mathematica - making WL applications that are sufficiently bullet-proof to hand over to an end user (rather than, say, a research co-worker).

BTW what do you use Raspberry Pi for?

My use of Raspberry Pi? Mainly for having the alternative mobile access to Mathematica (for mobile coding, or as calculator) and to my personal locally saved Mathematica notebook files. The web browser access and the Android app access are convenient and very good to have but they are not full versions of a Mathematica installation. So through the VNC access (does everyone here know what a VNC access is, oh please?) you're on the road and have access to a full local Mathematica installation, namely the one on your Raspberry Pi sitting at home near the Wi-Fi router. Seriously, this is reason enough and reason alone to buy and setup a Raspberry Pi, and shame on whomever that there is no youtube video showcasing how Mathematica can be used this way on a phone while you're on the road, apart from browser access and Android app access! Shame. The youtube title should be "How To Use Mathematica On Your Phone Through VNC Access" and it should demo both the VNC Wi-Fi access and the VNC Cloud access. No, i am not gonna do such a video, not my job. It's Wolfram marketing department's job :P

I also used the Raspberry Pi for logging electronic devices, like a logging multimeter, a logging charger/discharger, or other similar eletronic devices which offer a USB connection (or serial connection) for logging purposes. You can let the Pi do the logging for days and weeks, non-stop! There's logging software for Linux/Raspbian, wonderful stuff.

And i don't mind letting the Pi stream music non-stop from a streaming music subscription website, looping an online playlist infinitely. I only need to turn on the speakers and there's music!

Of course, the Pi is not a replacement of a full PC. One can watch a youtube or a HD video with VLC media player but you can't do both at the same time. In summary, my personal main uses are: 1) having access to a full local Mathematica installation (granted the Wi-Fi or mobile internet is working), 2) logging electronic devices, 3) 24/7 streaming of music. There is so much more, purposeful things, one could do with a Pi, but as you can see, I keep it simple, basic, boring.

You mention using your RPi to access Mathematica remotely. But cant that be done just as easily and more effectively by accessing MMA on your PC via remote desktop when you are travelling)? Is there something specific about the RPi implementation of MMA that makes it more accessible for remote work?

The only difference is that a 2.0W RPi is constantly powered on, in its booted state. A 400W PC usually is turned off for the night or when you're travelling. Other than that, there is no difference between VNC access to RPi or VNC access to home PC, you're right. A full-fledged PC is more powerful than a small RPi, of course. Do you have your PC (or Mac) turned on for 24 7 365, see? With a RPi, one does so, for the sake of nonstop remote access to Wolfram L full installation, at slow performance.

Posted 21 days ago

Jonathan, the answer is very simple:

"Mathematica is for Stephen Wolfram to do his NKS-stuff on."

Other features are put in from time-to-time and as is deemed necessary to make Mathematica sufficiently saleable to support Stephen so he doesn't have to get some other less enjoyable occupation while he pursues NKS.

;-)

B

Posted 21 days ago

I always enjoy using Mathematica for serious and less serious work. Don't want to spend all time on finding the right python libraries and check there vitality etc when most absolutely necessary. I think Mathematica is fantastic and thoroughly enjoy listening to Stephen wolfram's discussions with his outstanding team. I mean I don't know any other cto doing that. I also believe that Wolfram is really looking for "stuff" that is actually useful for his customers. Other languages have also great features and sometimes better syntax but that doesn't matter. The flexibility is incredible. I also enjoy the documentation and all the crosslink to other fields that might be useful. You always learn from it. No other languages has given me more pleasure programming than this Wolfram language. I once played with low code software mendix in combination with sap software. I didn't really like it. Low code for sure but many many many clicks and different windows.. Not for me.

100% agree with your comments about Mathematica being enjoyable .... one of the great things about it is that it so easily allows you to "play" with concepts and ideas, and even pick up new ones from completely different fields. In that regard WL is unlike any other language I have programmed in (with the possible exception of APL). There is a great deal of tedium involved in programming in most other languages (although I quite enjoy working in R).

That said, I suspect that some of this is because I am not usually trying to build production systems in WL, a process that inevitably involves a lot of humdrum, laborious, house-keeping code. I use Mathematica for R&D, and in that context it's an ideal environment - fast, powerful, broad in scope and highly interactive. And, like many users, I love to dabble in all kinds of interesting topic areas that Mathematica opens up.

Posted 20 days ago

I think Mathematica is far behind Python. Since I have used Python, I seldom use Mathematica. Python is much more powerful than Mathematica in data processing.

Wenguang,

I happen to agree with you. But what would you say are the specific data processing capabilities that Mathematica lacks, or is inferior to Python, from your experience? Can you give some examples?

Posted 20 days ago

@Wenguang Wang, maybe it is because you did not have a chance to get to the Mathematica's full potential. I heard a different story. My friend uses Mathematica extensively, data science, analytics, general coding, etc. Then she had to use python for some of her new work tasks. She got very frustrated with how not-smooth the experience was with some workflows and the front end in python. To me hybrid numeric-symbolics and fundamental symbolic nature of Mathematica and its functional style coding architecture seems far superior than python. Some similar user opinions here: https://mathematica.stackexchange.com/questions/86058

I agree Jonathan and I am also worried about the long term future.

In my opinion the following issues have to be addressed first:

1.) Polishing and bug fixing of existing features

  • Inconsistencies in TimeSeries
  • Bugs and missing features in Machine Learning (Batch Prediction, GPU-support, Support for Apple Silicon and ML engine, …)
  • Performance of Dataset, …
  • Improve Developer Tools (update Workbench, support for other editors - might be coming as far as I know)
  • Fixing long standing issues in various kinds of functions. Very often when I try something new, I stumble over bugs which give an unpolished impression of the product and the features run out of gas after they are 90% finished.

2.) Generally „Big Data“ and Data Processing has to be handled better

  • fast converter / importing routine for numbers, datetime, …
  • Out of core storage
  • Supporting CSV - Files with „;“ deliminator
  • support for secure web sockets wss - Protocol
  • => focus on performance for processing huge amount of data

It is really surprising what basic features are missing in this area. Python is far ahead.

3.) Deployment for Production

  • as far as I know efforts are made in this area (Application Server) but nothing really useable in an easy way has been released so far
  • Licensing has to be pain free, no cloud credits or something like this but permanent license
  • Deployment / packaging of an app has to be easy and stable
  • Deployment on own infrastructure
  • support long running services (eg. Bot)
  • Automatic error recovery, restart, logging, …

4.) Predictable bug fixes and releases

  • when a bug is reported it is simple unpredictable for a customer if it gets fixed at all and when it gets fixed
  • This is especially annoying for regressions. Fixes have to be pushed out quicker.
  • I wouldn‘t suggest WL for a critical app in an enterprise because of this
  • as long as this is not worked out, WL is only useable for research and experimentation in my opinion

5.) Create and promote examples for real world use cases in enterprises (IoT, Predicitive Maintenance, Big Data, ML Recommendation engine, Trading Bot, …) outside of research and development. Whenever I talk to somebody about MMA nobody thinks about production usage of it. This is probably the hardest part to build up awareness.

That said I really like the concept and idea of Mathematica and I want it to last for a long time and be as stable as possible. I enjoy discovering new things and areas where I had no idea about before.

Philipp,

Thanks for your thoughtful input. One can only hope that someone at WR is paying attention...

Perhaps the single biggest shortcoming in Mathematica, compared to, say, Matlab or Python is in regard to deployment in production. I have developed a couple of production systems IN WL, but my sense in each case was that I was pushing the envelope in terms of Mathematica's core capabilities. I always had to be ready to handle bug fixes and crash-recovery scenarios. Unlike other languages, which also had their fair share of bugs and crashes along the route to deployment, there never really seemed to come a time when Mathematica could be expected to work seamlessly - there always seemed to be another problem waiting in the wings. Similar systems developed in C#, C++, Python or Matlab would always (eventually) get to a point where they were reliable enough to deploy in a production setting, with a reasonable expectation of reliable performance. But I can't say the same for systems I have developed in Mathematica.
Now, it could be that this is just my lack of experience/expertise in developing production ready systems in WL. But I also think that it is perhaps one of the key limitations of the language. It's not to do with the language being interpreted - so are Matlab and Python - but some other aspect of the language architecture (e.g. memory management) that renders it unsuitable for deployment in production. I wish it were otherwise; but I can live with that limitation, providing I know it exists.

In any case, while the title of the post is about what Mathematica is For, it is perhaps also useful to produce a list of things that Mathematica is NOT suitable for, which from my perspective would include:

1) Anything that can be done more easily in Excel

2) Anything requiring deployment in production

What about other users' experience? Can anyone else offer an alternative view of Mathematica's capabilities in relation to development and deployment in a production setting?

Are there any other limitations you see Mathematica as having?

Posted 20 days ago

Jonathan, could you please elaborate on what you mean by:

Anything requiring deployment in production

Does it mean the possibility to convert a Mathematica application so it can be used by a non-MMA user? Or a close to bug free MMA application for a MMA user (a "bug" may reside in the applications algorithm, not necessarily in MMA itself)? Or something else.

Yes, it would include standalone applications designed for general use and not requiring user familiarity with Mathematica (or possibly even a Mathematica runtime module). In Matlab, for example, applications developed in Matlab can be compiled as standalone C++ applications - a capability that Mathworks has worked long and hard on to produce.

But I also mean WL applications that are designed for use in the WL by Mathematica users (e.g. in package form, in a Mathematica notebook). Even here, as I said, it is challenging to produce an application for use in production, where application errors and bugs are very infrequent and remedied relatively easily. In most other languages bugs tends to get ironed out eventually, whereas my experience is that a Mathematica application is much harder to stabilize and often continues to surface problems. This is not always the case: for example my MATH-TWS application (which connects Mathematica to IB TWS) has proved very reliable. However, the great majority of that application is written in C++ and interfaces to the IB TWS api. So it doesn't really count as a true Mathematica application in the sense I mean here.

I recently spent almost two weeks to install keras(the most star count deep learning framework on github) to my gcp compute engine linux debian machine, keras now bundled with tensorflow2, only can run inside docker container, docker itself is cloud native technology, and I also spent great portion of time to install the gpu support in docker, now everything seems ok to go. The Capability to package app inside docker to run seems the way to go in cloud era.

Posted 18 days ago

Hi Philipp,

Wolfram Application Server was announced today. Perhaps it will address some of the issues we face with deploying and supporting WL-based applications in production environments. There is no mention of pricing / licensing but I suspect it will likely be out of reach for most startups / small companies.

Hi Rohit,

Thank you for the information. I will take a look at it as soon as there is a trial version available. There was also a presentation about WAS at WWDC 2020. Hopefully it will be easy accessible => pricing information available online without contacting sales, trial license for download, …

@Philipp Winkler, about bugs, the recently released Wolfram Research Issue Tracker:

https://redmine.wolfram.com

is a good sign for the path forward on this.

Hi Kapio,

I know about redmine. Actually I am a heavy user :) and it is definitely a step in the right direction. Direct contact with QA and sometimes with the developers helps a lot. Still what I would really appreciate is that development cycles are dedicated for bug fix only releases like version 12.1.1. Just polishing the existing stuff. There are also a lot of things fixed in regular releases but because of the new features almost always some regressions end up in the release.

You make some good points but maybe your premise is wrong. Is Facebook dying? enter image description here

Google trends might be deceiving. As a frequent WL user I never search for WL!
I think the WR is addressing many of the isssues you mentioned: creating standalone software - Wolfram cloud and WebMathematica -- WL add-on for Excel -- etc. I do agree that some applications need more finishing and polishing. As an Engineer I appreciate the addition of FEA to Mathemetica but it still needs a little more. The statistics capabilities are tops but why not add design of experiment capability. The presenter notebooks are fantastic if only I could use my custom theme background images like power point can. Nevertheless Mathematica remains to be my preferred tool to solve complex problems symbolically and numerically, simulate systems and present results.

Hi Kay, yes indeed it might that the premise is wrong (and in my original post I gave one possible explanation for the downtrend that is innocuous). In this case, however, I think that perhaps the chart for Facebook accurately reflects the slowdown in usage growth of the platform:

Source: Statista

Source: Statista

So, while it certainly doesn't mean Facebook is dying yet, it tells us that growth is slowing, which may ultimately mean the same thing.

In term of your other points, I think we are saying the same thing: Mathematica has come so far as to be considered excellent in many fields, but seems to lack the last x% of functionality to render it truly outstanding (like design of experiments in statistics, in your example). Perhaps that is by design. A rational argument might be presented along the following lines:

Usage growth Mathematica is slowing. We (i.e WR) can't reverse that trend by providing more functionality to existing users who are already committed to our product. We have to find new users and bring them on board. To do that we have to provide new functionality that will appeal to potential users who haven't tried Mathematica before. That argues in favor of adding more breadth to the product scope, rather than more depth, beyond a certain level.

If this argument were to be accepted, it would mean that WR would be constantly developing functionality in a field up to a certain level - typically below what is required by a professional at the forefront of the field - before abandoning it to focus on developing new functionality in other areas.

It seems to me that WR needs a "finish the job" Tzar, whose function would be to identify the critical functionalities missing from key subject matter areas and with the authority to prioritize them for development.

In many cases the gaps in necessary functionality could be filled with a handful of new functions. But for some fields (e.g. machine learning) the list would be very long indeed.

My thoughts on this topic, besides that it should recieve more public discussion, are twofold.

First, I am not sure whether I will buy Mathematica once I exit academia. As long as I have a 'free' license though, I'm hooked. I think both mathematics and computer science students have a hard time approaching MMA even though it has tremendous potential to benefit both kinds of people†. I like to tell them that MMA is the worlds best software rapid prototyping tool; or when I'm being cynical, that MMA is both good for nothing and good at literally everything††. Frivolous yet powerful things are often the most compelling and fun...

Second, can MMA ever have a practical use†††? I think besides small niches, there is exactly one possibility. I haven't played with it's C code generation very much, but it seems like it's mainly limited to producing 'zombie routines' that must hook into a kernel. The symbolic nature of WL admits, for instance, a search over all possible algorithms to achieve a task. I think a richer library of code generation tools/examples plus an expanded code generation interface (i.e. x11 calls, various windows dlls; think of existing gpu and ffmpeg functionality) could start to make WL useful for 'actual code development'.

†In many lower division math classes, the entirety of homework can be automated in MMA. Document scanning, equation parsing, equation solving with steps, typesetting and printing. Literally the whole process: it's a thing of beauty. Applied computer science students, once relieved of worrying about the nontrivial trivialities of modern software, can appreciate the beauty of theory.

††Good for nothing in that it's useless for producing anything dynamic (i.e. production software), and good for everything in that it's capabilities are wiiiidespread and richly interconnected and as a result can produce amazing static visualizations, sounds, videos etc. Most academic publications are entirely static--most of scholarship is just prototyping ideas--such that MMA is the perfect scholarly workbench.

†††Again, practical in the sense of providing a directly valuable service--there's some economics word I'm looking for. One might say I'm asking for the well crafted code that goes into the kernel to be freely given up. Instead, I'm suggesting that the symbolic power of WL could be suited for automatedly writing lots of code--I would only be able to give examples if I were to embark on this as a project.

Adam,

I think my view is quite well aligned with yours: Mathematica is unparalleled in its facility for doing mathematics and its interactive nature makes it an outstanding choice for doing research, experimentation and prototyping. Which thereby also makes it an excellent pedagogical tool.

If we were to draw a line here and say: "this is what Mathematica is for", would that be so bad? Does its current inadequacy as a platform for developing production systems render it useless? No of course not, not at all. Why should it? After all, I have never heard anyone complain about R's shortcomings in the same area - why should Mathematica be held to a higher standard?

Well, I think there may be a couple of reasons (in addition to cost, of course):

Firstly, if we dispense with the notion that Mathematica can ever become a platform for doing series development work in a production environment, it means that you have to become highly proficient in at least one other language. And given the scope and complexity of WL, and of the other language one may be obliged to learn (e.g. Python, Java, C# or C++ ), a developer can be forgiven for thinking that they may as well save some time and effort and just focus on the second language, which can do everything (although perhaps not as elegantly as WL, in some areas).

Second, Stephen Wolfram's boundless enthusiasm and ambition for the WL leads one to hope that perhaps, eventually, a way will be found to remedy the WL's deficiencies and make it suitable for development of production systems. This is, after all, what Mathworks appears to have achieved with Matlab/Simulink, enabling Matlab code to be translated into production-suitable C++.

However, I suspect this is just a pipe dream: there are all kinds of structural reasons why a similar translations capability would be next to impossible to achieve in WL (beyond the very limited extant ability to compile simple functions).

Some comments from a long time programmer (coming up on my 50th anniversary) and Mathematica user (32 years):

First, Bernard Gress's comment that WL exists so that Stephen can get his projects done is truer than might be expected. Certainly, the best way to get feature 'x' implemented is to get Stephen interested.

Second, I think that there really is confusion about what Mathematica is for. I initially used it solely to solve maths problems, and for that it was, and is, superb. The first edition of the Mathematica book had the subtitle: "A system for doing mathematics by computer". Over the versions, the definition of what maths is has expanded, and a bunch of other stuff has been added.

The notebook interface is great for this type of investigation and interactive exploration.

However, it is rubbish at making stand-alone apps.

It is also far too hard to do computer-science or complicated programming. When I learned c, almost everything I needed to know was in the K&R book, some 125 pages or so, as I recall. Sure, you needed to understand the standard library, but the header files gave the information you needed. This level of documentation is missing in WL, although the general level of documentation otherwise is very good. With c, the step between the 'simple' stuff and a complex program was gradual and the simplicity of the core language made learning what to do relatively straight-forward. With Wolfram language, it is by no means clear what the core language is, and understanding it is not simple. This makes going beyond the basics much more difficult than it should be. At Apple in the early 2000s, they told developers that Xcode (and objective-c) made the regular stuff easy, and the hard stuff possible. With Mathematica, the mathematical functionality for interactive use is made very easy (compare to writing your own), but anything beyond that has been made much more difficult than it needs to be.

To give an example, it is not clear from the documentation how to pass a variable by reference -- if it is there, it is pretty well hidden.

Your comment about losing something when key people leave is very telling, especially for Mac users. When Theo Gray was involved, you could expect that Mathematica would make use of Apple technologies -- it was largely due to him that Mathematica remained a Mac program through the dark days. However, I am no longer feeling the love, specifically in the areas of UI and GPU utilization. As far as I can tell, Apple's neural engine is at least as good as anything based oil NVIDIA, yet it is not used at all for machine learning in Mathematica. Embracing other new OS features has been very slow -- it took far too long to move the front end to 64 bit, for example.

There is also the tendency to leave stuff at the 99% done level and move on to the next shiny object. There are simply too many bugs. There are too many similar functions that do pretty much the same thing, but which have subtle differences in the API or performance, and the documentation does not make that clear at all.

When I retired, I decided that I would let my skills in c (etc....) lapse in favor of doing all my coding in WL. I have come to regret this decision. When I use WL as part of my mathematical studies, I have (mostly) no problems. For anything else, it is a constant slog to achieve even moderate success.

I do have proposals for a solution:

  1. Fix the bugs.

  2. Market Mathematica (Wolfram|One, etc.) for what it does best -- interactive computation.

  3. Fully implement the Wolfram engine so that it can be used as a resource when developing stand-alone code in pretty much any language for any platform. Specifically, this means making the engine work with iOS and iPadOS so that other developers can develop apps with a state-of-the-art UI and functionality, yet making use of the strengths of WL. (A similar case can be made for Android, as well.)

  4. Make full use of the native capabilities of each hardware platform. This means making use of all the technologies in macOS (and iPad/iOS), Windows and Linux, although the most obvious mismatch is with macOS at the moment.

  5. Institute a developer program (with nominal costs attached), so that developers will have access to essentially whatever or whoever tech support gets their undocumented answers from. I remember that the Wolfram Technology conference was originally called the Mathematica Developer's Conference.

  6. As a corollary, provide some really state-of-the art developer tools. I tried using Eclipse, but compared to the tools that I used when developing Mac apps (Lightspeed c, Code Warrior, Xcode), the tool is clumsy and out-of-date. I have no experience in Java development or development on Linux or Windows; perhaps Eclipse is perfectly usable for these platforms. I know that discussing tools is likely to lead to a religious war -- after all there are those who think c++ is the best of all possible languages. However, if Wolfram Research wants to break out if its ghetto, it will need to integrate with modern IDEs, and Eclipse is not going to help with this (in my opinion, of course).

  7. Consider making a new language. All older languages have had re-thinks about what the language means. C has had several successors, with different levels of success: c++, Objective-c, and Swift. While I am not fluent in Swift (yet) it seems to have captured the essence of what was good about c without the excesses of c++, and is easy to learn. A similar rethink of WL would be a really good idea. While many of the design objective of WL were good, not all were, and some of them are real howlers -- especially in the area of UI design and implementation. Done right, no old code would be obsolete (you can still use ANSI c in Swift), but things would be a lot easier both for typical users and experts. Of course, if Wolfram Engine were available for macOS and iPadOS, some enterprising person or group could re-implement WL from within Swift, so this might not be so far out after all. (Remember Swift is not proprietary -- and I am using that only as an example.)


When I first used Mathematica, I was developing (in-house) real time process control software for a large clinical lab. I would solve the mathematical problems and prototype graphs, etc., using Mathematica, then translate the results into c on a Macintosh. This combination worked extraordinarily well. If my suggestions were implemented, and I were doing the same thing today, I would do the prototyping in WL, then use the Wolfram Engine in a native program I would write in Swift for macOS and iPadOS that would make native use of Apple's Neural engine. This would result is a more useful and stable application than I could ever hope to do using WL alone.

So, to answer your question: what is Mathematica for?

In its current state, Mathematica is for interactive problem solving of primarily mathematical problems using the notebook interface.

I do not think it is effective when used for anything else (certainly by not anyone outside of Wolfram Research) : web deployment, etc. except as a way to publish notebooks interactively, seems to be a toy project.

With proper focus, as I outlined above, Wolfram Language could be a significant tool in the development and implementation of basically any type of scientific application.

Hi George, thanks for taking the time to comment in detail. You have plenty of relevant experience with WL and have come to some clear views as to its strengths and limitations.

First, I agree with this comment:

There is also the tendency to leave stuff at the 99% done level and move on to the next shiny object. There are simply too many bugs. There are too many similar functions that do pretty much the same thing, but which have subtle differences in the API or performance, and the documentation does not make that clear at all.

I also largely agree that:

The notebook interface is great for this type of investigation and interactive exploration. However, it is rubbish at making stand-alone apps.

But WL is not completely rubbish at production systems. I am aware of at least one product, which I believe was developed 100% in WL that works extremely well, even by the standard of a production environment. I, too, have produced a small WL application that has shown itself to be very robust (although it's bit of a cheat as its just WL wrapper for a C++ system, really).

I also struggle with this view:

It is also far too hard to do computer-science or complicated programming.

I've written some pretty complicated programs in WL - not in the sense that developers typically mean, which often just refers to tens of thousands of lines of code related to a UI for example, but in terms of core mathematical, statistical and econometric concepts, which (in my view) is much more important (and complicated) stuff.

Yes, WL is a large ocean and for the new swimmer it can appear confusing and intimidating. But is not that hard, because of WL's nature as an interactive, functional programming language. The trick is simply to start by formulating some small component of the system you want to build, test it interactively to ensure it works as intended and then move on to the next component. Then you string all the components together, one by one.

I think the difficulty is that in languages like C, Fortran, Python, etc, the structure of the language often dictates a single, obvious way to tackle a problem. WL's diversity and flexibility means that there are often multiple plausible approaches one could take. In WL, I often spend ages thinking about which method might be best and experimenting with alternatives in order to arrive at the best solution. Sometimes, as I do that, I feel I am wasting time - but this is usually more than made up during the implementation stage, which tends to progress much faster if I have spent time at the outset thinking more deeply about the problem I am trying to solve.

Finally:

  1. Fix the bugs.
  2. Market Mathematica (Wolfram|One, etc.) for what it does best -- interactive computation.
  3. Fully implement the Wolfram engine

Perhaps as you say the solution to the challenge facing WR is not to try to "fix" MMA to enable it to do production work, but to focus on the Wolfram engine instead...

Perhaps as you say the solution to the challenge facing WR is not to try to "fix" MMA to enable it to do production work, but to focus on the Wolfram engine instead...

Exactly. Of course, the Wolfram engine, as I understand it, is mostly the Kernel, plus some bits that ended up in the front end for historical reasons. So, it really isn't focusing on the Wolfram Engine as a separate project, but making sure that there is functional parity -- as well as full support for native hardware.

An interesting test case will be whether, as part of the port of the Wolfram Engine to Apple Silicon, it will also be available for iPadOS. If this turns out to be the case, then I can see a lot of apps that use the Wolfram Engine, but which use a custom UI. One of the issues (among many others) with porting Mathematica to iPad is the need to make the UI for the iPad version as close to the desktop as possible, a formidable task given two fundamentally different design principles at work.

I want to re-iterate that Mathematica and its notebook interface is perfect for a wide range of applications and probably is the best in class for the vast majority of Mathematica users. But it should not try to be all things to all people: the best hammer in the world is not much use if you need a wrench.

Hi George,

as far as I know the Wolfram Engine is already ported to iOS for the App "Wolfram Player" but not embeddable in a custom application.

https://apps.apple.com/us/app/wolfram-player/id1059014516

Having said that, the Wolfram Engine would not be regarded as a complete solution by many:

(i) production systems developed using the WE require a production license and distribution of the WE runtime module

(ii) it entails development of the system in another programming language (which calls the necessary functionality in the WE).

In other words, in this mode of usage the WL alone is insufficient to create a production system. It is necessary to be proficient in another programming language such as C++ or Python to create the complete application. From that perspective, the WE is just another library (albeit a powerful one).

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract