Train a neural networks with this format of data?

Questions
, ,
1

Is is possible to train this format of data?

{{21, 30, 39, 45, 46, 44}->{1, 5, 20, 28, 52, 24}
{31, 52, 36, 34, 26, 32}->{9, 47, 40, 34, 7, 11}
{48, 35, 44, 8, 45, 4}->{25, 30, 4, 28, 21, 35}
.
.
.
{15, 42, 49, 17, 44, 30}->{22, 11, 19, 23, 15, 18}}

Will a Neural Network learn these associations?

Thank You Michel

2

There’s no reason why a neural network wouldn’t be able to learn a vector-to-vector mapping. Here’s an example:

data = {
 {21, 30, 39, 45, 46, 44} -> {1, 5, 20, 28, 52, 24},
 {31, 52,  36, 34, 26, 32} -> {9, 47, 40, 34, 7, 11},
 {48, 35, 44, 8, 45,  4} -> {25, 30, 4, 28, 21, 35},
 {15, 42, 49, 17, 44, 30} -> {22, 11, 19, 23, 15, 18}
};

trainedNet = NetTrain[
  NetChain[{
    LinearLayer[10],
    Ramp,
    LinearLayer[]
    }
   ],
  data,
  TimeGoal -> 30
]

Note that by specifying the final layer as LinearLayer[], the network will figure out automatically from the data what shape the output should be (in this case, a length-6 vector).

3

Thank You
Would training be learned on whole Vector representation to whole Vector representation
Or component to component .

Michel

4

If you use only regular LinearLayers (often called “dense” layers) and elementwise layers (like Ramp) such as I did in my example, the learned representation will be vector-to-vector. A linear layer is nothing more than a matrix multiplication with trainable matrix elements.

5

Thank You
I need to do more in-depth reading on layers.

Have an Excellent day

6

We have some excellent videos and tutorials on Machine Learning!

Here are some of the resources you can use:

(there are many many more … just search online for ‘wolfram machine learning’)

7

How would I deal with a Data file of this type that is 66 GB?

Michel

8

Training on a dataset like that is going to require some thinking, but you should start by reading the tutorial about this subject in the documentation:

Basically, you need to write a function that can randomly sample part of your big data file without having to load all of it into memory at once. You may want to consider loading the data into some sort of database to make this easier, but you can try using low-level file reading functionality as well: