Group Abstract Group Abstract

Message Boards Message Boards

22
|
51.9K Views
|
79 Replies
|
109 Total Likes
View groups...
Share
Share this post:

What is Mathematica For?

POSTED BY: Jonathan Kinlay
79 Replies
POSTED BY: Gareth Russell
POSTED BY: Richard Frost

Most users of phylogenetic software do not understand what it is actually doing, nor therefore the potential for inferential errors, let alone actual algorithmic or coding mistakes. Which is part of a larger discussion that goes well beyond phylogenetics. But how to solve the problem?

POSTED BY: Gareth Russell

Much of biology has become mathematically inclined over the past 40 years. Early on, Biology departments started teaching their own mathematical analysis courses - in part because math departments refused to add pair-joining and pseudo-metrics to their course curricula. This has not been the case with other math-oriented majors - they require their undergraduates to take the junior level numerical analysis course, and their graduate students to take applied math analysis + numerical analysis II.

POSTED BY: Richard Frost

I'm curious, Gareth, if you feel isolated in your academic arena because so few other ecologists use the WL? Does this mean that others really can't peer review your work (since they don't know how to use WL) and you cannot peer review theirs, when they perhaps use other common ("free") tools like R, Stan & Python?

I invested in WL a long time ago, and resist learning other languages because WL is so productive. Even so, I would be hard pressed to recommend WL to my biology students. After all, what employer really wants WL skills?

POSTED BY: Todd Allen

I invested in WL a long time ago, and resist learning other languages because WL is so productive.

Yeah, I'm in the same boat. I suppose I feel isolated a little, but not in the 'peer review' way. One of the nice things about WL is that I can provide a single notebook that explains all the computational pieces in enough granular detail that anyone with basic skills should be able see that it does what we say it does. And I myself can do basic R, although that is hardly relevant because in my field R is used mainly for stats, via packages. What's needed is that I understand the stat methods and when they are appropriate. I'm not going to unpick the algorithms in, say, a linear mixed effects package.

I actually like coding my own versions of discipline-specific tools, such as those in the VEGAN package. It makes me feel more confident that I know what they are doing, and what the output means.

POSTED BY: Gareth Russell

The "R" package contains both unsound individual modules plus procedures that combine otherwise valid modules into an erroneous computation chain. Somewhere along the line, biology professors started equating "knowing how to program" with "knowing what to program".

POSTED BY: Richard Frost

Want to help me make a new and better version in WL?

POSTED BY: Gareth Russell
POSTED BY: Richard Frost

The functionalities you've listed only scratch the surface. For example, consider the interconnectivity to devices. Mathematica is used as a controller for laboratory apparatus, as a receiver for remote sensing applications, and so on. Take a stroll through the main Language Documentation webpage and you'll find more capabilities you missed.

With regard to LaTeX, it's remarkable that Wolfram provides any support at all to something that is not a product but rather an ever-growing collection of TeX macros. As Peter points out, there is serious support for TeX. You can also use the Mathematica "Copy As" (select, right click) to paste Mathematica output as TeX and paste it into your TeX document. If you need something fancier, then use your programming skills to generate TeX within Mathematica and write it to a TXT file.

POSTED BY: Richard Frost
POSTED BY: Peter Burbery
POSTED BY: Jonathan Kinlay

Hi David, Thank you for your well organized, thoughtful post. I agree with much of it. Like you, I am happy to see the emergence of useful structures such as AxisObject. However, I have concerns about the reduction of "wrapper" functions such as the ListPlot family you mentioned. One is the effect on legacy codes of substantial length at site-licensed institutions - both public and private. The second concern is for the user base from the non-mathematical arts who have learned just enough WL to be productive. For example, consider applications in comparative semantics. A third concern is with function names that are synonyms. Consider Union and DeleteDuplicates. Should Wolfram attempt to force some users to increase their set theory vocabulary, or conversely force the avid set theory user base to change long standing mathematical terminology? I think not. So while the long list of function names deserves review, I believe there are many criteria to consider.

POSTED BY: Richard Frost
Posted 3 years ago

Mathematica started as a computational algebra system; a product with its own proprietary language and unique notebook system to integrate computational code with text and graphics.

It works very well as a sandbox computational environment for e.g. research, analyze, document and develop new models and computational methods (algorithms). It’s strengths are:

  • Symbolic computation combined with numerical methods
  • Functional and rule-based programming (and procedural)
  • Large library of functions so you can write concise code
  • Well-written documentation
  • A notebook environment.

Mathematica is a fun system to work with and it delivers results quickly, even though its learning curve is steep (maybe because some of us came from procedural languages and there are many ways to achieve the same result).

I don’t agree with Jonathan on the use of Excel versus Mathematica. Mathematica is my go-to system to analyze data and models and yes it replaced Excel for me in most cases.

I used Excel for large datasets, with pivot tables, VBA and many worksheets. However, my code to analyze data became opaque or hidden in many worksheet formulas. As such, Excel became unworkable. Now I may use Excel to create a well-formed CSV file, which I import to create a Dataset and analyze it in a Mathematica notebook. I spent a lot of time to master Datasets and associations and now it pays off.

Mathematica's language i.e. the Wolfram Language, has transcended its original product: it has become the vehicle for all things computable. So Jonathan’s question “what is Mathematica for” is relevant.

I think Mathematica should serve a niche market, comprising of mostly researchers, analysts and maybe some system designers. It should become a vehicle of state-of-the-art computational functions and methods. I agree with Jonathan: WR should focus on finishing and polishing e.g. statistics and time series analysis, machine learning and a number of other application areas, so it becomes the go-to tool for research. If you have Mathematica, you shouldn’t need any other specialized application to do your research.

I think the WL should focus on a computational ecosystem with capabilities to develop, test and deploy code, create APIs and GUIs for different OS or distributed computing environments. Its symbolic computational capabilities and extensive library of functions make the WL an ideal language for education or data analysts.

Having said this, I see a number of improvement areas to facilitate the WL/Mathematica adoption:

Transparency of outstanding issues. If Mathematica becomes the indispensable tool for researchers, then more transparency about stability, reliability and accuracy becomes paramount. We need to be able to rely on the tool and hence we need insight in outstanding issues, bugs and workarounds. I hope access to redmine is a first step into more transparency.

Reduction of the number of functions to a select number of unique ones. Mathematica prides itself of its 5000-odd number of functions. However, I see a lot of redundancies i.e. new functions that do things slightly different from similar functions. I like to see a reduction of redundant functions. For example take the ListPlot function: we have ListLinePlot, DateListPlot, StepListPlot, DateStepPlot etc. they are all of the class “ListPlot” and with the new axis capabilities, they can all be integrated into one function. This saves us the effort to remember the name of each specialized function, it makes code maintenance and documentation easier for WR too.

Document implemented methods. I would like to see WR open up the black box of some functions in the documentation i.e. explanation of methods used or references to methods implemented.

Conclusion

I think the purpose of Mathematica should be the go-to tool for researchers and analysts. I agree with Jonathan that WR should focus on delivering a complete set state-of-the-art tools.

I think the Wolfram Language could become the computational ecosystem to develop, test and deploy code for all things computable. We have some miles to go there.

At last, the adoption of Mathematica and WL could be facilitated if:

  • We have more transparency of outstanding issues and bugs.
  • The number of "redundant" functions has been reduced to a number of unique ones.
  • We see more documentation to help us look inside the black box of some functions.
POSTED BY: Dave Middleton
POSTED BY: Jonathan Kinlay

In my experience, a proprietary solution can be worth having. CDC had a great Algol compiler and the Cray Research Fortran compiler was the best bar none. Every now and then I see a bit of that style in the Mathematica error diagnostics.

I don't enjoy reading requests for a less proprietary WL platform in the same paragraphs with requests for internal details of algortihms. If you want to know what was method was used you can look it up in the documentation. Beyond that seems to be some sort of claim of entitlement that does not exist.

POSTED BY: Richard Frost
Posted 3 years ago

Hi Tom,

I think a technically educated person should be writing with a word processor, doing reports with a spreadsheet, and thinking with a technical computing language as an extension of their thought process.

This strikes me as too prescriptive. Fifty years ago you would have had to write, perhaps: "a technically educated person should be writing with a fountain pen, doing reports using a pocket calculator and hardly thinking at all about technical computing languages, unless they are computer scientists".

The point is: time moves on. If you work at Wolfram Research I imagine you (try to) do everything in WL, whether it be writing text, producing some analysis, or programming an application. Certainly that's my understanding of Stephen Wolfram's expectation, anyway. For myself I do a large proportion of my work in WL, including carrying out analysis, creating presentations, and programming applications. It's very convenient to use a common UI and tech stack for all these varied tasks: it save shifting from one application to another, having to be concerned about file format compatibility, etc.

A technical computing language is not the right tool for producing commercial computer applications.

I understand your point, but I expect that there are legions of Fortran programmers, to say nothing of the good people at WR and Mathworks who would heartily disagree with you. And indeed on the pages of this post several examples are given of commercial applications developed in WL (including one or two of my own).

To your point, however, it's obviously a significant challenge to develop a commercial application in WL, even though it can be and has been done. I certainly wouldn't regard the development of commercial applications as a forte of Mathematica, in they way that, for instance, symbolic programming clearly is. Which goes to the topic of this post - i.e. what is Mathematica for? And I would have to concur that the development of commercial applications is not one of those things - although I am quite sure that Stephen Wolfram and others at WR would strongly disagree.

My answer: “it’s an extension of one’s thought process.”

This is a bold claim for any programming language/technology platform, including Mathematica. Could one not make a similar claim about C++, Java, or Julia? I suspect that, unless you are a professional developer, probably not. Mathematica has certain unique features that make it a natural fit for the rapid exploration of ideas, in a way that certainly isn't true of most other computer languages that I am aware of.

I think that your description encapsulates what Stephen Wolfram may be aiming for: a kind of cognitive sand-pit offering a plethora of computational tools that provide a unique way to explore ideas, whether experimentally, graphically, or analytically, that will assist the process of developing them into functional algorithms.

POSTED BY: Updating Name

Fifty years ago you would have had to write, perhaps: "a technically educated person should be writing with a fountain pen, doing reports using a pocket calculator and hardly thinking at all about technical computing languages, unless they are computer scientists".

Fifty years ago I had a programmable HP calculator, was writing with a word processor on an HP portable, doing reports with Lotus 1-2-3 (spreadsheet) on the portable, and programming in octal, Algol, Macro (assembly language), Basic, and Fortran on time-sharing systems. I was not a computer scientist nor had I taken any C.S. courses in college. These activities were typical among my colleagues.

POSTED BY: Richard Frost

You may want to revise your 'way-back' time. Lotus did not come out until 1983. The HP-35 debuted in 1972, but the programmable HP-65 came out in 1974 -- at a whopping $795 -- equivalent to $4k in today's dollars. HP portables no earlier than 1987.

I started coding in 1972, and it was on a timesharing PDP-5. No personal computers at all -- Altair 8800 debuted in 1974. I got an HP-35 shortly after it came out, which meant that I could ditch my slide rule and log/trig tables. Most of us who learned to code in those days didn't take CS, because most schools didn't offer them. (I was a maths major, so I did have numerical analysis.)

So, 50 years ago (1972), I was writing on a typewriter (an IBM Selectric if I did so at work), doing calculations with my slide rule or walking to the Medical School computer facility -- or by hand, and when I did learn to code, used Assembly and BASIC. C came out in 1972, but I did not have access to a computer the compiler (etc.) would run on until the late 1980s.

SO, no need to make the 1970s 'better' than it was. It was the wild frontier.

I wasn't referring to consumer products. We bought these systems from laboratory suppliers or the build-it yourself versions from Heathkit. Fountain pens were a rarity - we all used ballpoint.

POSTED BY: Richard Frost
Posted 3 years ago
POSTED BY: Tom Compton
Posted 3 years ago

We should leave Wolfram Alpha out of this discussion about programming languages. This is not a programming language at all, but a search or - if you like - answer engine. You can't do anything in it that could be called "programming".

POSTED BY: Werner Geiger
POSTED BY: Jonathan Kinlay
Posted 3 years ago
POSTED BY: Werner Geiger

I think Mathematica is good for writing computational essays. Stephen Wolfram has a blog post on this.

POSTED BY: Peter Burbery

Every language is an algebra.

POSTED BY: Richard Frost

I agree that Wolfram Alpha is becoming indispensable. I think Wolfram Alpha is used by math students, math teachers, science students, and science teachers.

POSTED BY: Peter Burbery

A couple of recent comments led me to review the original post and some of the replies from community members which, I have to say, I find rather interesting (although I suspect Stephen Wolfram would perhaps beg to differ). A thought on this subject that I have been entertaining for some time is the concept of Mathematica as a kind of computational Wikipedia. Perhaps, if it has any relevance at all, the comparison could be better made between Wikipedia and Wolfram Alpha, rather than Mathematica.

In any event, what I find interesting is that for a long time Wikipedia looked like a hopeless quest: in the early days many subjects were barely covered and the quality of the articles was often very poor. But then, amazingly, the whole idea took off and Wikipedia became the kind of indispensable reference that Jimmy Wales imagined it could become.

I think something similar may be happening with Mathematica (or, rather, Wolfram Alpha) as its scope has enlarged over time (and the quality of the reference material has always been excellent). My sense is that it is close to some kind of tipping point, on the verge of becoming the kind of indispensable computational resource that Stephen Wolfram had in mind from the outset.

Do others agree? Or take an altogether different view?

POSTED BY: Jonathan Kinlay

I am a Wikipedia editor and I think Wikipedia is a good reference.

POSTED BY: Peter Burbery

The universities I'm familiar with have site licenses, typically at the department or laboratory level. Also, the Mathematica pricing for students appears in line with textbook prices from my perspective.

The main hindrance I see for students is with department level inbred thinking that community-based free software is better in all regards. However, in biology these projects have become the largest source of research errors in modern history:

http://wireilla.com/papers/ijcsa/V12N4/12422ijcsa01.pdf

POSTED BY: Richard Frost
Posted 3 years ago
POSTED BY: Werner Geiger

I'm aware of research units in university social sciences and liberal arts with Mathematica installations.

POSTED BY: Richard Frost
Posted 3 years ago

A Wolfram|One Personal licence for Student costs EUR 333/Year (!). This is for one user, two installations. Euros are about the same as US dollars. This is a price never acceptable for students. Unless they have rich parents.

POSTED BY: Werner Geiger
POSTED BY: Richard Frost
Posted 3 years ago

The prices for student Mathematica desktop licenses in Germany:

enter image description here

POSTED BY: Hans Milton

Given the prices of textbooks, I believe that is very reasonable.

POSTED BY: Richard Frost
Posted 3 years ago

I reviewed again the German prices for Student's Wolfram / Mathematica. I found three products/licenses for Students: enter image description here

They range from 104 €/Y to 333 €/Y. The cheapest one does not include the cloud (which probably is not very important, since everybody stores and synchronizes their files within other clouds, namely OneDrive for Windows users).

I could not really tell the difference is between Mathematica Desktop (181€/Y) and Wolfram|One Personal (333 €/Y) (I have the latter, but - strange enough - for 271 €/Y).

I don't even know what the difference is between Mathematica and Wolfram Language. I look at "Mathematica" as just the former name of "Wolfram Language" and abbreviate it as "WL".

Putting it all together, possibly a student could live with Mathematica Desktop for 104 €/Y. "Possibly" because I assume that this does not include any product upgrades which would make it pretty useless. If that is true, he would have to pay at least 181 €/Y, possibly 333 €/Y.

POSTED BY: Werner Geiger
Posted 3 years ago

Much has been said here that I understand and agree with. Especially the uselessness for real production systems. I myself have discovered WL only as about the 20th programming language since I am a pensioner. Before that it was always about large systems in FORTRAN, PL/1, C, C++, Java as a system architect and developer.

Today I program just for fun, and for that WL is great. For me especially because of the symbolic and mathematical capabilities and graphics. And I love notebooks that contain code and its documentation at the same time. Now I do everything simple with EXCEL VBA and everything else with WL.

I would like to add two important points that I think hinder the success of WL:

  • The impenetrable jumble of products and licenses, i.e. the marketing.
  • The price. For people who just want to get to know the product, especially students, a license is way too expensive (the cloud is not a way out, in my opinion you really need a desktop version with local files).

The second point is existential. Students can do all their tasks for free with things like Microsoft Visual Studio, Eclipse, C#, C++, Java, R, Python and the like, but nothing with WL because they can't afford a license.

POSTED BY: Werner Geiger

I think the ability to do symbolic math is underrated. I'm a hobbyist programmer, and I once encountered a problem that really did require to do symbolic math. I really, really didn't want to do the calculations with pen and paper so I bought a Mathematica license. Since then, I've learned to appreciate the capabilities of the language and the elegance of its design.

POSTED BY: Lucien Grondin
POSTED BY: Jonathan Kinlay

I think one thing that Mathematica does well is solving differential equations including differential-algebraic equations with DSolve.

POSTED BY: Peter Burbery

Indeed.

POSTED BY: Jonathan Kinlay

Mathematica has been the platform of choice for sensors-to-animation production by the film industry for decades. A colleague of mine is a system administrator at one of the sites.

POSTED BY: Richard Frost

I had no idea! Is there a site you could point me to, in order to get a better understanding of the application?

POSTED BY: Jonathan Kinlay

The famous example are the animations added to the actors in Pirates of the Caribbean.

POSTED BY: Richard Frost
Posted 3 years ago

Do you have any references for that? I'm curious.

POSTED BY: Eric Rimbey
POSTED BY: Richard Frost

Hi George, thanks for taking the time to comment in detail. You have plenty of relevant experience with WL and have come to some clear views as to its strengths and limitations.

First, I agree with this comment:

There is also the tendency to leave stuff at the 99% done level and move on to the next shiny object. There are simply too many bugs. There are too many similar functions that do pretty much the same thing, but which have subtle differences in the API or performance, and the documentation does not make that clear at all.

I also largely agree that:

The notebook interface is great for this type of investigation and interactive exploration. However, it is rubbish at making stand-alone apps.

But WL is not completely rubbish at production systems. I am aware of at least one product, which I believe was developed 100% in WL that works extremely well, even by the standard of a production environment. I, too, have produced a small WL application that has shown itself to be very robust (although it's bit of a cheat as its just WL wrapper for a C++ system, really).

I also struggle with this view:

It is also far too hard to do computer-science or complicated programming.

I've written some pretty complicated programs in WL - not in the sense that developers typically mean, which often just refers to tens of thousands of lines of code related to a UI for example, but in terms of core mathematical, statistical and econometric concepts, which (in my view) is much more important (and complicated) stuff.

Yes, WL is a large ocean and for the new swimmer it can appear confusing and intimidating. But is not that hard, because of WL's nature as an interactive, functional programming language. The trick is simply to start by formulating some small component of the system you want to build, test it interactively to ensure it works as intended and then move on to the next component. Then you string all the components together, one by one.

I think the difficulty is that in languages like C, Fortran, Python, etc, the structure of the language often dictates a single, obvious way to tackle a problem. WL's diversity and flexibility means that there are often multiple plausible approaches one could take. In WL, I often spend ages thinking about which method might be best and experimenting with alternatives in order to arrive at the best solution. Sometimes, as I do that, I feel I am wasting time - but this is usually more than made up during the implementation stage, which tends to progress much faster if I have spent time at the outset thinking more deeply about the problem I am trying to solve.

Finally:

  1. Fix the bugs.
  2. Market Mathematica (Wolfram|One, etc.) for what it does best -- interactive computation.
  3. Fully implement the Wolfram engine

Perhaps as you say the solution to the challenge facing WR is not to try to "fix" MMA to enable it to do production work, but to focus on the Wolfram engine instead...

POSTED BY: Jonathan Kinlay

Hi George,

as far as I know the Wolfram Engine is already ported to iOS for the App "Wolfram Player" but not embeddable in a custom application.

https://apps.apple.com/us/app/wolfram-player/id1059014516

POSTED BY: Philipp Winkler
Posted 3 years ago

In fact Wolfram has an embeddable solution for iOS: It's called CDFKit and it was presented many times in Wolfram Technology Conferences by Rob Raguet-Schofield and Jason Harris. CDFKit can be integrated as an iOS-Framework in Objective-C or Swift projects.

The availability of CDFKit seems to be bound to a purchase of an Enterprise Version of Mathematica (which is costly) and maybe a special contractual arrangement (to publish/distribute a certain number of apps).

This is of cause way too complicated: availability and usage should be as easy as free Wolfram Engine.

I agree what was said earlier: Apple Silicon (CPU, GPU, Neural-Engine) should be exploited and optimized much more for special tasks as AI, ML and -- if possible -- general compute. Maybe CDFKit could be adapted also as a "lightweight" Wolfram Engine for macOS.

It would be quite easy for Wolfram Research to develop a similiar "educational app" like Apple's Swift Playgrounds for iPadOS. A mini-REPL on the left with WL syntax coloring and CDF-Output incl. DynamicModule/Manipulate on the right (using CDFKit). This would be a much better experience than the Wolfram Cloud App.

The ultimate question remains: Where is Mathematica for the iPad?

POSTED BY: S. M. Wirker

I think the complaints about cost are misplaced. Wolfram products are in high demand and so I don't believe the price is going to come down.

POSTED BY: Richard Frost

Having said that, the Wolfram Engine would not be regarded as a complete solution by many:

(i) production systems developed using the WE require a production license and distribution of the WE runtime module

(ii) it entails development of the system in another programming language (which calls the necessary functionality in the WE).

In other words, in this mode of usage the WL alone is insufficient to create a production system. It is necessary to be proficient in another programming language such as C++ or Python to create the complete application. From that perspective, the WE is just another library (albeit a powerful one).

POSTED BY: Jonathan Kinlay
POSTED BY: Adam Mendenhall
POSTED BY: Jonathan Kinlay
POSTED BY: Kay Herbert

Hi Kay, yes indeed it might that the premise is wrong (and in my original post I gave one possible explanation for the downtrend that is innocuous). In this case, however, I think that perhaps the chart for Facebook accurately reflects the slowdown in usage growth of the platform:

Source: Statista

Source: Statista

So, while it certainly doesn't mean Facebook is dying yet, it tells us that growth is slowing, which may ultimately mean the same thing.

In term of your other points, I think we are saying the same thing: Mathematica has come so far as to be considered excellent in many fields, but seems to lack the last x% of functionality to render it truly outstanding (like design of experiments in statistics, in your example). Perhaps that is by design. A rational argument might be presented along the following lines:

Usage growth Mathematica is slowing. We (i.e WR) can't reverse that trend by providing more functionality to existing users who are already committed to our product. We have to find new users and bring them on board. To do that we have to provide new functionality that will appeal to potential users who haven't tried Mathematica before. That argues in favor of adding more breadth to the product scope, rather than more depth, beyond a certain level.

If this argument were to be accepted, it would mean that WR would be constantly developing functionality in a field up to a certain level - typically below what is required by a professional at the forefront of the field - before abandoning it to focus on developing new functionality in other areas.

It seems to me that WR needs a "finish the job" Tzar, whose function would be to identify the critical functionalities missing from key subject matter areas and with the authority to prioritize them for development.

In many cases the gaps in necessary functionality could be filled with a handful of new functions. But for some fields (e.g. machine learning) the list would be very long indeed.

POSTED BY: Jonathan Kinlay

I agree Jonathan and I am also worried about the long term future.

In my opinion the following issues have to be addressed first:

1.) Polishing and bug fixing of existing features

  • Inconsistencies in TimeSeries
  • Bugs and missing features in Machine Learning (Batch Prediction, GPU-support, Support for Apple Silicon and ML engine, …)
  • Performance of Dataset, …
  • Improve Developer Tools (update Workbench, support for other editors - might be coming as far as I know)
  • Fixing long standing issues in various kinds of functions. Very often when I try something new, I stumble over bugs which give an unpolished impression of the product and the features run out of gas after they are 90% finished.

2.) Generally „Big Data“ and Data Processing has to be handled better

  • fast converter / importing routine for numbers, datetime, …
  • Out of core storage
  • Supporting CSV - Files with „;“ deliminator
  • support for secure web sockets wss - Protocol
  • => focus on performance for processing huge amount of data

It is really surprising what basic features are missing in this area. Python is far ahead.

3.) Deployment for Production

  • as far as I know efforts are made in this area (Application Server) but nothing really useable in an easy way has been released so far
  • Licensing has to be pain free, no cloud credits or something like this but permanent license
  • Deployment / packaging of an app has to be easy and stable
  • Deployment on own infrastructure
  • support long running services (eg. Bot)
  • Automatic error recovery, restart, logging, …

4.) Predictable bug fixes and releases

  • when a bug is reported it is simple unpredictable for a customer if it gets fixed at all and when it gets fixed
  • This is especially annoying for regressions. Fixes have to be pushed out quicker.
  • I wouldn‘t suggest WL for a critical app in an enterprise because of this
  • as long as this is not worked out, WL is only useable for research and experimentation in my opinion

5.) Create and promote examples for real world use cases in enterprises (IoT, Predicitive Maintenance, Big Data, ML Recommendation engine, Trading Bot, …) outside of research and development. Whenever I talk to somebody about MMA nobody thinks about production usage of it. This is probably the hardest part to build up awareness.

That said I really like the concept and idea of Mathematica and I want it to last for a long time and be as stable as possible. I enjoy discovering new things and areas where I had no idea about before.

POSTED BY: Philipp Winkler

Philipp,

Thanks for your thoughtful input. One can only hope that someone at WR is paying attention...

Perhaps the single biggest shortcoming in Mathematica, compared to, say, Matlab or Python is in regard to deployment in production. I have developed a couple of production systems IN WL, but my sense in each case was that I was pushing the envelope in terms of Mathematica's core capabilities. I always had to be ready to handle bug fixes and crash-recovery scenarios. Unlike other languages, which also had their fair share of bugs and crashes along the route to deployment, there never really seemed to come a time when Mathematica could be expected to work seamlessly - there always seemed to be another problem waiting in the wings. Similar systems developed in C#, C++, Python or Matlab would always (eventually) get to a point where they were reliable enough to deploy in a production setting, with a reasonable expectation of reliable performance. But I can't say the same for systems I have developed in Mathematica.
Now, it could be that this is just my lack of experience/expertise in developing production ready systems in WL. But I also think that it is perhaps one of the key limitations of the language. It's not to do with the language being interpreted - so are Matlab and Python - but some other aspect of the language architecture (e.g. memory management) that renders it unsuitable for deployment in production. I wish it were otherwise; but I can live with that limitation, providing I know it exists.

In any case, while the title of the post is about what Mathematica is For, it is perhaps also useful to produce a list of things that Mathematica is NOT suitable for, which from my perspective would include:

1) Anything that can be done more easily in Excel

2) Anything requiring deployment in production

What about other users' experience? Can anyone else offer an alternative view of Mathematica's capabilities in relation to development and deployment in a production setting?

Are there any other limitations you see Mathematica as having?

POSTED BY: Jonathan Kinlay
Posted 5 years ago
POSTED BY: Hans Milton

Yes, it would include standalone applications designed for general use and not requiring user familiarity with Mathematica (or possibly even a Mathematica runtime module). In Matlab, for example, applications developed in Matlab can be compiled as standalone C++ applications - a capability that Mathworks has worked long and hard on to produce.

But I also mean WL applications that are designed for use in the WL by Mathematica users (e.g. in package form, in a Mathematica notebook). Even here, as I said, it is challenging to produce an application for use in production, where application errors and bugs are very infrequent and remedied relatively easily. In most other languages bugs tends to get ironed out eventually, whereas my experience is that a Mathematica application is much harder to stabilize and often continues to surface problems. This is not always the case: for example my MATH-TWS application (which connects Mathematica to IB TWS) has proved very reliable. However, the great majority of that application is written in C++ and interfaces to the IB TWS api. So it doesn't really count as a true Mathematica application in the sense I mean here.

POSTED BY: Jonathan Kinlay

I recently spent almost two weeks to install keras(the most star count deep learning framework on github) to my gcp compute engine linux debian machine, keras now bundled with tensorflow2, only can run inside docker container, docker itself is cloud native technology, and I also spent great portion of time to install the gpu support in docker, now everything seems ok to go. The Capability to package app inside docker to run seems the way to go in cloud era.

POSTED BY: vincent feng
Posted 5 years ago

Hi Philipp,

Wolfram Application Server was announced today. Perhaps it will address some of the issues we face with deploying and supporting WL-based applications in production environments. There is no mention of pricing / licensing but I suspect it will likely be out of reach for most startups / small companies.

POSTED BY: Rohit Namjoshi

Hi Rohit,

Thank you for the information. I will take a look at it as soon as there is a trial version available. There was also a presentation about WAS at WWDC 2020. Hopefully it will be easy accessible => pricing information available online without contacting sales, trial license for download, …

POSTED BY: Philipp Winkler

@Philipp Winkler, about bugs, the recently released Wolfram Research Issue Tracker:

https://redmine.wolfram.com

is a good sign for the path forward on this.

POSTED BY: Kapio Letto

Hi Kapio,

I know about redmine. Actually I am a heavy user :) and it is definitely a step in the right direction. Direct contact with QA and sometimes with the developers helps a lot. Still what I would really appreciate is that development cycles are dedicated for bug fix only releases like version 12.1.1. Just polishing the existing stuff. There are also a lot of things fixed in regular releases but because of the new features almost always some regressions end up in the release.

POSTED BY: Philipp Winkler
Posted 5 years ago

I think Mathematica is far behind Python. Since I have used Python, I seldom use Mathematica. Python is much more powerful than Mathematica in data processing.

POSTED BY: Wenguang Wang

Wenguang,

I happen to agree with you. But what would you say are the specific data processing capabilities that Mathematica lacks, or is inferior to Python, from your experience? Can you give some examples?

POSTED BY: Jonathan Kinlay
Posted 5 years ago
POSTED BY: Updating Name
POSTED BY: l van Veen
POSTED BY: Jonathan Kinlay
Posted 5 years ago

Jonathan, the answer is very simple:

"Mathematica is for Stephen Wolfram to do his NKS-stuff on."

Other features are put in from time-to-time and as is deemed necessary to make Mathematica sufficiently saleable to support Stephen so he doesn't have to get some other less enjoyable occupation while he pursues NKS.

;-)

B

POSTED BY: Bernard Gress

Your op is conversational/general (i like the content and agree, so i upvoted it) and i wouldn't wonder if it gets deleted. Nothing will come out of it (the discussion, if there is any) other than us writers/readers who agree with you feeling better because we said our piece and got the sore topic off our chest, hopefully without ranting. Anyway…

I am in the same boat, having expressed my existential angst for the product from the very start, at the time of registration see the About Me page.

"Success is the sum of right decisions." — If the popularity continues to trend downward, more and more senior key figures "turn their backs" ( @Michael, @Paul, @Roman, et al. because of age, capacity, retirement, or ultimately, expiration), and in 15yrs the product is no more than a relic, then that's fate. To me, $\text{Wolfram L}$ will still be the one and only programming language I am willing to accept; i never gave FREE $\text{Python}$ a serious or fair chance, because i am stubborn and, erh, i am no programmer (and have no interest in becoming one)!

I am no programmer (by profession, by heart, by need, by desire, by whatever) and have no interest in becoming one, also because of my very restricted talents hehe, yet i can write some simple functional code, create updated tables/graphs with a press of a key, and i do enjoy this ability of mine! All thanks to Mathematica. Honestly, I do miss the GUI features of M$ Word, Excel, Corel, Maple but i put up with these shortcomings because my enjoyment of the language is so much stronger. No software package or programming language is perfect or complete but imho Mathematica is the #1 contender in the all-rounder category. Nothing wrong with knowing how to draw maximum use out of a SAK Swiss Army Knife! And best of all, i have access to it all the time:

  1. on the road: through the Android Wolfram Cloud App, on my phone
  2. on the road: through the Wolfram Cloud web browser access (phone, computer, inet cafe, etc)
  3. on the road: through the VNC Cloud access (e.g. on my phone) to my Raspberry Pi at home -> my preferred mobile access
  4. at home: through the VNC Wi-Fi access (e.g. on my phone) to my Raspberry Pi at home
  5. at home: through the VNC Wi-Fi access (on my PC) to my Raspberry Pi at home
  6. at home: through eventual local installations on PC at home

The VNC access is possible only because my Raspberry Pi is non-stop powered by mains 24/7*365, a 2.0 Watts device.

Q: $\text{"So …, What is Mathematica For?"}$

A: $\text{"Well …, it is For Me! Basta ya."}$ — ©2021 Raspi. hh

POSTED BY: Raspi Rascal

Hi Raspi, in some ways I think you are an ideal Mathematica user - a total convert: MMA with everything! And I get it - it's a wonderful sandpit environment for every kind of experimentation.

But in broad terms, my thinking is that this is what Mathematica really is: a means of doing fast research, experimentation and prototyping - but not necessarily for production systems.

If you are not a professional developer that probably doesn't matter at all - everything is an experiment, or recreation! But for a professional developer, getting systems into production and making sure they are battle-ready is a critical part of what you do for a living. And its in this context that I see the greatest challenge for Mathematica - making WL applications that are sufficiently bullet-proof to hand over to an end user (rather than, say, a research co-worker).

BTW what do you use Raspberry Pi for?

POSTED BY: Jonathan Kinlay

My use of Raspberry Pi? Mainly for having the alternative mobile access to Mathematica (for mobile coding, or as calculator) and to my personal locally saved Mathematica notebook files. The web browser access and the Android app access are convenient and very good to have but they are not full versions of a Mathematica installation. So through the VNC access (does everyone here know what a VNC access is, oh please?) you're on the road and have access to a full local Mathematica installation, namely the one on your Raspberry Pi sitting at home near the Wi-Fi router. Seriously, this is reason enough and reason alone to buy and setup a Raspberry Pi, and shame on whomever that there is no youtube video showcasing how Mathematica can be used this way on a phone while you're on the road, apart from browser access and Android app access! Shame. The youtube title should be "How To Use Mathematica On Your Phone Through VNC Access" and it should demo both the VNC Wi-Fi access and the VNC Cloud access. No, i am not gonna do such a video, not my job. It's Wolfram marketing department's job :P

I also used the Raspberry Pi for logging electronic devices, like a logging multimeter, a logging charger/discharger, or other similar eletronic devices which offer a USB connection (or serial connection) for logging purposes. You can let the Pi do the logging for days and weeks, non-stop! There's logging software for Linux/Raspbian, wonderful stuff.

And i don't mind letting the Pi stream music non-stop from a streaming music subscription website, looping an online playlist infinitely. I only need to turn on the speakers and there's music!

Of course, the Pi is not a replacement of a full PC. One can watch a youtube or a HD video with VLC media player but you can't do both at the same time. In summary, my personal main uses are: 1) having access to a full local Mathematica installation (granted the Wi-Fi or mobile internet is working), 2) logging electronic devices, 3) 24/7 streaming of music. There is so much more, purposeful things, one could do with a Pi, but as you can see, I keep it simple, basic, boring.

POSTED BY: Raspi Rascal

You mention using your RPi to access Mathematica remotely. But cant that be done just as easily and more effectively by accessing MMA on your PC via remote desktop when you are travelling)? Is there something specific about the RPi implementation of MMA that makes it more accessible for remote work?

POSTED BY: Jonathan Kinlay
POSTED BY: Raspi Rascal
Posted 5 years ago

I have found Mathematica essential in production settings for at least 20 years. Most of those were before the Wolfram Engine (and the more recent cloud offereings). I have re-analyzed some of the applications I mention below, and we could have cost-effectively used the Wolfram Engine if it had been available. Stability of the kernel has also dramatically improved.

I have used Mathematica in at least three different companies. In a consulting environment, we did all our in-house analyses using Mathematica. Notebooks would call large databases, issue SQL commands, and bring in the data for further processing and visualizations. Most algorithms we needed had to be developed from scratch, and Mathematica made it quick to go from reading a paper or a pencil and paper calculation, to something tested and working on data. The graphics would usually get some tweaking in Illustrator, but would often be close to camera ready. We also used Mathematica to create reference algorithms to be implemented in C++ or Java. Many of these where not numerical algorithms.

At my next company we used Mathematica directly as a compute engine. We had developed a very performant cyber security system that identified executing malicious code by observing the stream of system calls. All the machine learning steps were done in Mathematica (this was before the free Wolfram Engine). It allowed us to have version 1 of our system up way faster than if we had implemented them directly. We experienced memory leaks and random hangs that forced us to periodically kill and restart the compute engine, but we were able to engineer it so that those issues remained invisible to the user. Over time, because of costs, we replaced the Mathematica subsystem with Java code.

In my current work we still develop most of the algorithms in Mathematica. We are building a system that use Mathematica for some core data generation; reference algorithms are still in Mathematica. We also use it as a thinking tool (see my blog post). Having all that curated data at one's fingertips is great.

I feel costs have prevented a wider adoption. Wolfram Research now offers low cost versions for non-commercial applications, which should encourage wider adoption, but I sometimes worry that its market may be shrinking.

POSTED BY: Ronnie M
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard