Message Boards Message Boards

Lost a Notebook with 11.0.1. How to get it again?

GROUPS:

I use Wolfram Programming Lab (Standard License). Normally from the Desktop version on an up-to-date Win10 PC. Last Thursday I updated to version 11.0.1 from 11.0.0. This worked with no problem.

I was developing a pretty large and tricky notebook (.nb) for two weeks. It was stored within the Wolfram cloud and is still there. I worked with it last Friday after the update and saved it to the cloud. No problem.

Since Saturday I cannot reopen that notebook. The ProgLab-Desktop says: "There was a syntax error in the file being read. Do you want to open it as plain text? ... If you are able to fix the error, save the file and open it again".

Let aside that my notebook had no syntax error when last saved on Friday evening, it's somewhat ridiculous that a program with a syntax error cannot be reopened.

When I answer YES to open the file as plain text, I get another error message: "There was a syntax error on line ... The error was: <lots of zeros>". Then the file opens empty as white space. No notebook, no plain text. Just one header line "Notebook[{},...".

When I try to open that notebook directly from the website (https://lab.wolframcloud.com/app/. Not the desktop version), it opens completely empty with no error message at all.

I'm afraid, I lost that program and have no idea, why. It is still in the cloud, 1MB large, saved last Friday evening, which is correct.

Does anyone have an idea how I could regain that notebook-file?

POSTED BY: Werner Geiger
Answer
1 year ago

BTW: I think this idea of storing things within internet-clouds instead of locally and without a local copy in 1:1 structure is some of the biggest nonsense that IT-people invented during the last 10 years or so. It started on smartphones with nonsense-terms like "photos, music, videos, documents, hubs" etc. and propagated to PCs meanwhile. Nobody knows anymore since then what is stored where and when and under which exact name and which file attributes. And what is replicated where and if at all. Actually all those are just files within some directory-structure. Vendors try everything to hide that from customers since then.

This is particularly true for a product like Wolfram Programming Lab Desktop, which is aimed to allow for local development, but cannot live an hour without an online connection to the Wolfram cloud. And then - to make things a desaster - just destroys files in their cloud.

If I imagine, I would have to develop a huge program (as I did for tenths of years in many different IDEs) with such an IDE with hundreds of files and some tricky directory structure, I think it would be a real nightmare.

POSTED BY: Werner Geiger
Answer
1 year ago

Please contact support directly for such issues: http://www.wolfram.com/support/contact

Forum members cannot provide help in this matter.

POSTED BY: Moderation Team
Answer
1 year ago

@Moderation Team: Wolfram support could not help. I come back with that problem to help others avoiding this kind of disaster.

Now I have exactly the same problem again with another notebook. I think I understand now what happens. The notebook produces some large amount of data - say a list or array with some millions of elements - and outputs this in Short-form. For example Print[Short[list]].

I do always a ClearAll["Global`*"] before saving the notebook into the cloud. Hoping to reduce its size. But this is not sufficient, since the output stays within the notebook and when saved, it is not saved in Short-form but in full-form, which makes the save huge (say some 20 MB or larger, not really huge). It takes some time, and then - very strange - the notebook can not be reopened since Wolfram claims the syntax error described above.

For me this is a horrible bug.

To get around it I now always introduce a parameter controlling the size of the problem. In this example the list-Length. Before saving I reduce that parameter from said some millions to 10 or so, run the complete code, do the ClearAll and save. Then the save gets really small and can be reopened.

(Probably the ClearAll is not necessary at all. One has just to avoid large outputs within the notebook, even if displayed in Short-form only).

POSTED BY: Werner Geiger
Answer
10 months ago

It is possible to push the limits of Notebooks as data containers. I had similar problems years ago, but have instinctually learned to avoid them. It can be very distressing when the only copy of an important notebook becomes unreadable. Backups are your particular responsibility, since the front end does not have file recovery, for example, like MS Office.

Also, you might avoid similar problems in the future by deleting all output before saving your notebooks at the end of a session, to the cloud or elsewhere.

There are threads elsewhere in the old Mathematica mailing list, on Stack Exchange, and probably in this Community that discuss methods for persistent notebook data.

Vince

POSTED BY: Vincent Virgilio
Answer
10 months ago

Hmm, for me this is quite strange thinking, since it passes the responsibilty for trivial save-operations to me instead of the IDE.

If the IDE thinks, it cannot save a notebook/program (But why? Can somebody think on any reason under a working Internet-connection?), it should tell that together with the reason and stop in an ordered manor. But not just destroy the program.

That the IDE tries to save a multiple of the data in use, i.e. fullform-output instead of shortform, and then kills everything thereon is ridiculous.

The error-messages and recovery-procedure (see above: Syntax error, plain text opening) are all wrong and do not work.

All of those are nothing but bugs, which render the Wolfram-PL-IDE very dangerous to use. Practically unusable.

You say, I should delete all output before saving. OK, but how? Manually? Tens or Hundreds of outputs, one after the other by manually deleting all those output cells? Or ist there a command for that? I could not find something like that.

All in all there remain two facts: 1) The Wolfram IDE Programming Lab IDE is dangerous since it destroys programs during save into the Wolfram cloud. 2) Basically the problem comes from not storing programs locally on my PC, but this cloud-nonsense instead.

POSTED BY: Werner Geiger
Answer
10 months ago

You say, I should delete all output before saving. OK, but how? Manually? Tens or Hundreds of outputs, one after the other by manually deleting all those output cells? Or ist there a command for that? I could not find something like that.

Desktop FrontEnd Menu "Cell > Delete All Output". Keyboard shortcut Alt+c,l ('el')

POSTED BY: Vincent Virgilio
Answer
10 months ago

Great! Thank you. I did not know that.

For the PL-desktop that option is in menu Advanced > Cell > Delete All Output

POSTED BY: Werner Geiger
Answer
10 months ago

BTW: There are more things, which are very strange and prove that the Wolfram programmers are unable to handle their IDE: (I am still talking about WL-Programming Lab-Desktop).

The whole startup is a joke. Nonsense-windows that have to be opened until I get to my program (described elsewhere).

In each notebook I have File-Options "Save" and "Save to Cloud" which obviously do the same. Or not? Who knows?

I save a notebook and then close it. Then I am asked, if I want to save it again. Horrible! What happens? But I have no chance but saying Yes and pray, that the IDE will not kill my program.

I have no real idea what is saved where.

I assume that nothing is stored or replicated to my own PC. They keep me fully dependent on their cloud. And like to kill my programs thereby - see above. Although they claim to give me with that heavily payed desktop-license the possibility to develop locally.

Sometimes the UI just freezes. No idea when and why.


I love the Wolfram Language. This is really impressive and very intelligent what can be done with it.

Strange enough the IDE is anything else than intelligent. It is stupid and filled with bugs.

POSTED BY: Werner Geiger
Answer
10 months ago

BTW: There are more things, which are very strange and prove that the Wolfram programmers are unable to handle their IDE: (I am still talking about WL-Programming Lab-Desktop).

The whole startup is a joke. Nonsense-windows that have to be opened until I get to my program (described elsewhere).

In each notebook I have File-Options "Save" and "Save to Cloud" which obviously do the same. Or not? Who knows?

I save a notebook and then close it. Then I am asked, if I want to save it again. Horrible! What happens? But I have no chance but saying Yes and pray, that the IDE will not kill my program.

I have no real idea what is saved where.

I assume that nothing is stored or replicated to my own PC. They keep me fully dependent on their cloud. And like to kill my programs thereby - see above. Although they claim to give me with that heavily payed desktop-license the possibility to develop locally.

Sometimes the UI just freezes. No idea when and why.


I love the Wolfram Language. This is really impressive and very intelligent what can be done with it.

Strange enough the IDE is anything else than intelligent. It is stupid and filled with bugs.

POSTED BY: Werner Geiger
Answer
10 months ago

Within another thread "How can I clear Mathematica data stored in the Virtual Memory? " I just found a hint from Sander concerning $HistoryLength which controls the stored amount of previous output.

I think this could really solve my problem.

POSTED BY: Werner Geiger
Answer
10 months ago

You are probably much better off using the desktop version only for data-intensive calculations, no cloud at all. Years ago I did professional consulting project with Gigabytes of data and all went fine, locally. I have no idea how I would do some real project using (public) cloud. Maybe with the Wolfram Enterprise private cloud that would be possible. But that seems still somewhat expensive.

POSTED BY: Rolf Mertig
Answer
10 months ago

There is no way of using PL locally. With my standard license it insists in storing everything within the Wolfram cloud.

This standard license costs are 240 EUR per year, it has a local desktop, but still forces me into the cloud for storage. Without any chance to escape from that.

I could update to a premium license for the double cost. This license claims to allow for local files. This is horribly expensive for a programming-hobby and I don't believe that I could get rid of that useless Wolfram cloud with such a license.

POSTED BY: Werner Geiger
Answer
10 months ago

Why not Mathematica Home edition? Just call the german dealer. I do not sell Mathematica, but if you want you can also call me.

POSTED BY: Rolf Mertig
Answer
10 months ago

What is Mathematica Home Edition? I cannot find that on the Wolfram website.

Anyway. What I want from Wolfram is a locally running WL-IDE that stores my programs and data locally. And does not destroy them.

That's all. A pretty simple requirement.

POSTED BY: Werner Geiger
Answer
10 months ago

Home Edition is the "true" standalone desktop version, not Cloud depending. For private use, and at a substantial discount. Searching for it on this site gives:

enter image description here

Edit: I checked the pricing. Mathematica Home Edition is actually cheaper than Programming Lab Standard. For one year, respectively:

  • 240 EUR for Programming Lab Standard
  • 155 EUR for Mathematica Home Edition

So if you are not using WL in a commercial context, it looks like the Home Edition is the better bet. With regards to both performance and price.

POSTED BY: Hans Milton
Answer
10 months ago

Thanks very much, Hans, for that hint. I think I have seen that home edition some time ago but did not notice that it can work locally.

But there is no official way to upgrade/downgrade from my current PL-license and I am not sure, if I can still use my cloud notebooks with that home edition. Probably downloading from the cloud to local before changing the license.

I think I will have to call wolfram support to find out.

POSTED BY: Werner Geiger
Answer
10 months ago

All in all for me the Wolfram product structure is very confusing and "over-marketed". There are so many different products with different options each (online, desktop, local, cloud, credits, API-calls, storage, kernels, computing time, ....).

It's really difficult to get to reasonable work, instead of investigating what is what.

POSTED BY: Werner Geiger
Answer
10 months ago

Yes, it is confusing. There is some elucidation here. Otherwise: Fell free to visit Kuba and me in my office in Berlin. Or call the german reseller.

POSTED BY: Rolf Mertig
Answer
10 months ago

To get back to my original problem: How to get notebooks corrupted by the Wolfram IDE back again?

I think I have found a manual procedure to recover from the problem described above. It is tedious but worked twice in my cases. If you have such a corrupted notebook within the Wolfram cloud, which can no longer be opened, e.g. "xmp.nb", then:

  1. Make a copy of it within the cloud. E.g to "xmp(1).nb".

  2. Download this copy to your local PC.

  3. Open it there with some standard text-editor, e.g. Windows Editor. Fortunatly these ".nb"-notebooks are ASCII text and they are readable.

  4. Find large regions of cryptic output. Sometimes (normaly?/always?) they start with "CompressedData[" and consist of an endless sequence like "/…", where "…" are some cryptic ASCII-strings of different lengths. All this sequence is surrounded by apostrophs.

  5. Find the next closing bracket "]".

  6. Delete everything between "CompressedData[" until the next closing bracket "]". But keep the opening and closing apostrophes in between. This is tedious and may take some time. And it drives you angious since you delete 90 or even 99.9% of your file thereby. If you do that in several manual Select/Delete-actions (which I recommend), then save your file in between, to be sure.

  7. When you have finished, save your file and quit the editor. The size of your local file "xmp(1).nb" should now have decreased dramatically. Say from some tens of megabytes to some tens or hundreds of kilobytes.

  8. Upload that file ("xmp(1).nb") back into the Wolfram cloud. This gives "xmp(2).nb" in my case. But you can override the previous useless, corrupted version as well if your IDE allows it. Mine doesn't, strange enough.

  9. Try to open that file within web and/or desktop.

  10. If that works: Check your notebook, if you lost something. Maybe you have to correct things manually. Thereafter just rename that working notebook to "xmp.nb" and kill all those copies in between (cloud and locally).

  11. If that does not work: Sorry for you. But you still have your original corrupted "xmp.nb" and can try to do other things with it. Let me know.

POSTED BY: Werner Geiger
Answer
10 months ago

I have another case now with a lost notebook. This ist just a simple but large notebook where I collect different, independent tasks, each in its own section.

Two days ago after some work on my newest task I could save it without any problem into the cloud (I'm still working with Programming Lab Desktop). But since then I cannot reopen it. The PL desktop claims the file to be corrupted. My procedure from above does not help.

If I look at the end of that notebook with some standard text editor, it ends right in the middle of some lengthy encoded ASCII-sequence. Without closing strings, brackets and open function calls. I think, it's in the middle of an encoded image as part of a cell.

I am not sure, but I think, there is some size-limit during save, but which is unknown to the user. The program should tell me during save something like: "Your notebook is too large. Please remove some parts and move them into a new notebook". But the program just saves quietly up to it's internal unknown limit and truncates the file instead. Which would be pretty stupid and a horrible bug.

I lost my work from many months thereby. I am still trying to get some parts back of that corrupted notebook by editing it with a normal text editor. Really difficult and some kind of blind flight.

POSTED BY: Updating Name
Answer
11 days ago

I forgot: Of course I work with the newest version 11.2.0.0 (Windows 10, 64-bit)

POSTED BY: Werner Geiger
Answer
11 days ago

BTW: That notebook in question is tiny, only 2 MB large. Isn't it ridiculous, that Wolfram has problems with such mini-files?

And BTW: Why does the Wolfram cloud not save versions of notebooks? Or does it and I was not aware of that until now?

To summarize: The problem is that Programming Lab stores files only within the Wolfram cloud and not locally (or the latter at least only temporarily outside of my control). If my files were local, i could use all versioning features of Win10 itself, the Microsoft (OneDrive) cloud or some true versioning-system. Or at least some private backup procedures.

Each and every day I am sweating from fear, knowing that all my work lies within that unreliable Wolfram cloud and I can lose it at every time without any reason.

And that for a product (Programming Lab Standard) which costs $250 per year.

POSTED BY: Werner Geiger
Answer
11 days ago

Group Abstract Group Abstract