Group Abstract Group Abstract

Message Boards Message Boards

Do I need WorkBench or can I achieve results with a notebook?

GROUPS:
Hello,

I need to import time series data. Then run a routine that will generate a 2.5*10^6 X 13 X 10 grid or table (per series) for visual and computational analysis.
The full system will run 100's of instances of my above description. With the exception of the analysis piece, I have this code written and collecting data in other software, but after some research I think migrating most of the code to mathematica will greatly improve the speed and efficiency of the final product.

Before I commit time and funds to a solution, does anyone have an opinion about the best path to get this done in Mathematica?

Thank you in advance.

August 
POSTED BY: August Voegeli
Answer
5 months ago
Hi August,

You can do that with notebooks. You can download a trial version of Mathematica here. The performance will of course depend on what exactly you are doing, but here's a quick example to give you a feel for it. This generates a random dataset of the size you describe, totals the columns of the 2.5 million matrices, splits them into groups of high and low variance, and then makes histograms from the first column total for the first hundred results from each group. That takes about 20 seconds on my 3 GHz processor without any parallelization or compilation optimizations.
PairedHistogram @@
  GatherBy[Total /@ RandomReal[10, {2500000, 13, 10}],
    StandardDeviation@# > 10 &][[All, ;; 100, 1]] // AbsoluteTiming

There are many more interesting examples in the documentation.
POSTED BY: Michael Hale
Answer
5 months ago
Workbench is for building Mathematica packages.  It enables you to edit the package and retest all in one environment (based on Eclipse).  As Michael posted, you don't need it for your application.
POSTED BY: Frank Kampas
Answer
5 months ago
Thank you again.
When I get it up and running I'll post an example.
POSTED BY: August Voegeli
Answer
5 months ago
Learning this system is pretty fun. Everytime I write a function with regards to testing and building lists, I find it already written somewhere later on. Clearly familiarity is the key. And putting in the time to learn it will pay off in the long run.
POSTED BY: August Voegeli
Answer
5 months ago
[color=#000000][/color]
                      a = SortBy[Table[RandomReal[6, {200, 3}]], first];
b = Table[a[[i,3]], {i, Length[Select[a[[1;;-1,1, # < 3 &]]}]
After much reading, listening, and searching I can manipulate and analyse data in a table. And I am ditching my program that generates the big tables that I refered to earlier and putting everything into Wolfram. This will be much more work, but it will be worth it in performance and research.
Have a great weekend.
POSTED BY: August Voegeli
Answer
5 months ago
The following code creates a table:
a = Table[{i,j,k,l,m,n}, {i,3}, {j,3},{k,3},{l,3},{m,3},{n,3}]
However when I use Part:
a[[1,3]]
I don't get anything close to the correct cell value. I get a list of the whole table.

However when I use RandomReal to build a Table instead of the built in iterators, Part works as expected.

Does anyone know why part works for one Table and not the other?

Thank you in advance.

August
POSTED BY: August Voegeli
Answer
5 months ago
The following code creates a table:
a = Table[{i,j,k,l,m,n}, {i,3}, {j,3},{k,3},{l,3},{m,3},{n,3}]
a is a 3 x 3 x 3 x 3 x 3 x 3 matrix, not just a 3 x 3 table, each entry is a list of length 6 and each entry in that list  is from Range[3]
However when I use Part:
a[[1,3]]
I don't get anything close to the correct cell value. I get a list of the whole table.
You don't get a list of the whole table, you get the 3 x 3 x 3 x 3 matrix at position i = 1 and j = 3
POSTED BY: Udo Krause
Answer
5 months ago
Udo, thank you. I did not realize it was a 3x3x3x3x3x3  matrix. My intent was a 297 x 6 matrix and I was trying to do it with the use of nested loops. That makes perfect sense.

Ariel, thank you. This is a big project. ~700 lines of procedural code so I expect to trim that back considerably by adapting to Mathmatica. And then adding more code for the analysis piece. So I will begin reading.....

August
POSTED BY: August Voegeli
Answer
4 months ago
The Workbench is never required as a platform for creating Mathematica applications.  If your application is going to be only a few pages long I recommend that you don't even think of learning to use the Workbench for such a small project.  However, although it will take you a fairly large amount of time to learn using the Workbench efficiently, if your project ever becomes big enough I would strongly recommend learning to use it.  At this point in my "Mathematica career" I only use Mathematica's front end for applications that are smaller than ~50 lines of code.  The navigation, editing tools and capabilities of the Workbench will quickly pay back when you are working on big projects.  In summary, the more time you dedicate to create Mathematica applications, the more likely the Workbench will help you use your time more efficiently.
POSTED BY: Ariel Sepulveda
Answer
4 months ago