If results had error bars, and regression analyses showed the correlation coefficient, a lot of what passes for "knowledge" in Sociology, psychology, and medicine would be seen as being very tenuous at best.

I think that this is a major cultural problem. No one gets through Physics 1 or any decent engineering course without learning that any measurement that lacks an estimate of error is useless. I used to work for a clinical lab, and the entire notion knowing what the error bars for a result were was antithetical to the thinking both of the management of the company and the clients.

As I recall, Tufte does discuss error, and I believe that there is a section where he shows radically different sets of data give identical least squares fits. However, this is not the main thrust of his discussion, as you point out.

This whole topic is overdue for a treatment that can be understood by people who just want to use software, and not have a deep understanding of numerical analysis. I'm pretty sure that Mathematica is a good tool to use for this.

The quintessential example that comes to mind for me is the Fleischmann–Pons claim of cold fusion.

http://en.wikipedia.org/wiki/Cold_fusion

Bill, Bravo. Very well articulated. Thanks.

Sadly, careful error propagation, null hypotheses, standard error and sample sizes, p values, etc are disappearing from the canonical topics of a physical science education, and many papers appear in press with no discussion of error and uncertainty.

The typical excuse for omission is "curriculum pressure". I think we are just not teaching effectively.

"Just playing with Mathematica is a good strategy for staying young"

I tried that Craig and it doesn't work! Running very very fast might work in a relative sense.

More accurately, playing with Mathematica is a good strategy for maximizing the use of your time.

Clearly, that is something I have to start doing.

I agree with your assessment, though I would not have thought that I could or should write a document wholly in Mathematica. That is a novel idea that I will try out. I am guilty like everyone else of importing the results from Mathematica into PDF documents. The irony to me is that I write everything as a first draft document, including insertion of equations and rationale for their use via Mathtype within Word, as I develop a thesis and solutions, along with reams of paper scratching and associated Mathematica notebook pages. If I don't write it down, it is lost forever, since my notes are never indexed except by various page number styles.

That said, I use Mathematica at its most rudimentary levels compared to the things you discussed, but I get your drift. All of the manuals on how to use Mathematica have been written by people who know too much. Their work should be edited by someone like me who is trying to learn what the developers already know. In commercial areas, you find that software invariably suffers from this issue: developers know too much and don't realize that they need to make their programs and documentation user friendly...where the user can be totally ignorant of the underlying algorithms and could care less. The irony is that when the users have a problem that in turn is becoming an issue, the developers run in a panic to the human factor and user interface groups to have them fix the garbage the developers have created. Wolfram unwittingly operates with this same paradigm, along with most other software product developers.

In my early years at Miami, I had Prof Arfken as an instructor during the times he was contemplating his book. I am sure we were guinea pigs for some of his efforts. He was a pleasant man and fun to listen to. In a symposium at one time he stated an object lessen I still adhere to. He said he had spent weeks trying to solve some problem with a methodology with which he was unfamiliar. He finally made arrangements to visit with a distant heavy weight in mathematical physics to try and resolve what he was trying to do. In his words he said after a day's travel, he went into the heavy weights office, sat down, presented his problem, to which within 5 minutes the "great man," as Prof Arfken referred to him with a twinkle in his eye, laid out the approach and solution. The lesson: when all else fails, ask, and don't wait forever to do so.

With the central claim for Mathematica being that it is the single tool needed for everything from jotting down mathematical notes to doing initial exploratory calculations to creating graphics to implementing entire sets of tools for a problem domain to writing and publishing papers and textbooks, I am surprised that I have not seen more published on how to carry out that last step. From the style of it, I am assuming that Ruskeepa's "Mathematica Navigator" was written almost entirely within Mathematica.

There was a thin little book published perhaps twenty years ago showing some of how, entirely within Mathematica, to create documents for publishing. I held a copy in my hands for a minute, but didn't buy it and have never been able to track down that title again. If you could find that book, even though it is old now, then it might help you get started. Perhaps someone else can recall that title.

On the larger question of reception, culture and attitude is far more powerful and persuasive than most might imagine. If you want to do another literature search, then look at the reception that automated theorem proving or even computer proof checking has received from the vast majority of the mathematics community over the last fifty years. Then you can compare and constrast that with the reception that Mathematica in particular and other computer algebra systems in general have received. Perhaps that would help you better understand what you are seeing.

Some have claimed that ideas which are rejected now will be embraced and adopted when the previous generation dies. But culture and attitude may be more persistent than that. If you are creating something new then choose your culture and attitude carefully because you will pay the price and live with the consequences of that forever. For example, if you happened to be around and watching during 1980s, when the culture and attitude was instilled into Mathematica, you might be better able to notice, and possibly even understand, some of the behavior of people inside and outside of Wolfram Inc. thirty years ago and, in some cases, unchanged today.

Lastly, I try to sense your attitude from what you have written above. You might contemplate that. It has a way of leaking out and there almost always seems to be a price to pay for that. I do realize my some of attitude has leaked out in what I have written here.

Good points and I plead guilty to the leakage observation. I was in the "business" at one time and have seen a variety of cultures at work. My knee jerk was a recollection of the times I was burned as a "customer". Within software developers is an undercurrent of certainty that they know best or, as with many technical people, they come across a better way of doing things while working on some project...so they shift gears without letting anyone know. This is the bane of project managers.

I recall meeting a past graduate school colleague at a conference. He had set up a company that was successful and one of its lines of business was developing specialized scientific analysis software for the DoD in the areas of spectroscopy. We chatted and he discovered I was associating with the same types of customers and developers as he was, so he asked me my opinion on what it would take to commercialize his products for a wider audience. One of my own staff had developed a different type of specialized analysis software and we were wrestling with the same question: how to expand the market for that intellectual property. I had no answer for my friend or myself. Some fifteen or so years later, the software my group had developed did show up as a commercial product developed independently elsewhere, and their secret was the user interfaces and easy of use, which our intellectual property never possessed....nor did we have the capacity to address that issue. Plus, the market for the new product had never occurred to us.

Mathematica's documentation is as good as it has to be for Wolfram not to suffer any commercial consequences. Wolfram is blessed by an outstanding product and a user community that is so smart that it overcome the deficiencies in the documentation...plus, to be crass about it, Wolfram sells support services. I think where Wolfram may be somewhat short sighted is that such issues as are being raised in this thread indicates that they could promote a ground swell of broader acceptance and use by "commoditizing" their product. This is just the businessman in me speaking. I would reject the idea of a slimmed down or crippled version just to get a cheaper product out, but too many bells and hidden whistles is pretty frustrating in any product. As an example, I just bought a new phone that uses Android and I tried to set up my voice mail account. I had to go on-line to find someone to tell me where the set up was hidden and it was definitely hidden. The documentation did not even address voice mail but it supplied overkill (from my perspective) in the areas of applications and data.

Mathematica is what it is and from a commercial perspective, Wolfram is doing what it needs to do to stay on top of their game...they have a business strategy that does not depend on better transparent documentation. Mathematica is the best mousetrap around from my perspective. Could it be better? Probably, but define better.

I concur with your assessments and those of Park based on a more general exposure to technologists with particular preferences. I started with a blank slate in terms of exposure and preferences, since I had avoided programming at any level after my early pre-PC experiences with Fortran and Basic and more punch cards than any one graduate student should have to carry. I was never required to do much programming, starting with my first job at Michigan (we used a Wang with punch cards!!!) to Battelle, where I was fortunate to manage a talented group of scientists and technologists who loved programming. Now as I close in on my dotage, I am on my own.

I had to find a muscular program to work out some problems that had never been properly solved because all work on those problems stopped a hundred years ago. A reading of a basic paper showed that the author had not actually done what he said he was going to do and did something else because there were no tools like Mathematica available then.The author did what he could. I reread the paper and identified an asyllogistic conclusion and set out to test the author's methodology. I got results that contradicted the text books. However, I did not know if the fault was mine (mis-interpretation or mis-statement of the real problem) or whether the issue was Mathematica (again, was it my use of mathematica or something within Mathematica), or were the results true? So, after research that led to no resolution on any of the points, I asked.

The results of my consulting with several faculty were disappointing. Other than criticizing Mathematica and my methodology, I simply could not get anyone to actually read the statement of the problem as originally presented nor my interpretation and subsequent modeling using Mathematica. I had expected push back but not total out of hand rejection of my thesis, and the criticism of Mathematica seemed to be a way out for actually having to address the issues and possible consequences that the problem presented. I am, at this point, working down the list of criticism eliminate them. Hard slog.

Hello Danny and Luther, That makes three of us who started with punch cards.

A bizarre as it might seem, I think punch cards were a beneficial way to learn to program. I spent so much more time thinking about the program and doing thoughtful debugging because submission of a job was so painful. I confess that I often do debugging now with less reflective thought because I can do dozens of haphazard experiments in minutes--this probably speeds the debugging process as often as it slows it down.

I was sold on symbolic computation (pre-wolfram-language) when I was able to solve classical mechanics homeworks faster and more accurately than my classmates. I remember that just being able to do a taylor expansion around a point and copy down the results and redraw crt-rendered plots into my homework felt like cheating.

I also remember my own "inertia" of switching from "symbolic-program-x" which was free at Berkeley (and had opaque syntax that I had learned moderately well) to Mathematica 1.

Craig