Message Boards Message Boards

2
|
5952 Views
|
6 Replies
|
5 Total Likes
View groups...
Share
Share this post:

Change of hidden Layers in neural networks

Yeso Alde
Posted 4 years ago

Hello,

I am trying to do some machine learning with Mathematica. I loved the ease of use of variety of methods which allow me to test some models on my data. Unfortunately I also find it very limiting when it comes customizing the methods.

For NeuralNetwork, I couldn't find too much information about how to setup the number of hidden layers, number of nodes per layer, etc. The only information I found in the help file was NeuralDepth. I am not quite sure what it refers to.

Can you help me on how to customize the hidden layers of NN?

Thank you.
POSTED BY: Yeso Alde
6 Replies
Sort By:
Rohit Namjoshi
Rohit Namjoshi
Posted 4 years ago

Hi Yesim,

That post from MSE is very old. The "HiddenLayers" sub-option for Predict has been removed. In that example "HiddenLayers" -> {4, 3, 3} means 3 fully connected linear layers with output vector sizes of 4, 3, 3.

If you want complete control over the NN then I suggest not using Predict with Method -> "NeuralNetwork". Instead build the network using specific layers in NetChain and NetGraph and then NetTrain. Each layer type has parameters that can be controlled and there are a lot of customization options for NetTrain. You will have to study the documentation.

NetChain[{4, 3, 3}] (* Equivalent to "HiddenLayers" -> {4, 3, 3} *)

You have not provided any details about the problem you are trying to solve so it is hard to provide more specific advice.
POSTED BY: Rohit Namjoshi
Rohit Namjoshi
Rohit Namjoshi
Posted 4 years ago

Hi Yesim,

the layers given in Information["*Layer"] are layers trained for a specific task

They are not trained for a specific task, they perform a specific operation/function. This is no different from layers in TensorFlow / Keras or PyTorch. As far as replicating results using a different framework, you may be able to export the network from WL to a standard format such as MXNet or ONNX and import into another framework. If not, the network will have to be re-created using the corresponding layer types available in the other framework. It is relatively easy to see the correspondence between WL layer names and the names in Keras or PyTorch.

Many popular NN models have been implement in WL, take a look at the Wolfram Neural Net Repository.
POSTED BY: Rohit Namjoshi
Rohit Namjoshi
Rohit Namjoshi
Posted 4 years ago

Hi Yesim,

Hidden layer is not a vey useful concept. Every layer between the input and the output is a "hidden layer". WL supports many different layer types that can be used as "hidden layers"

Information["*Layer"]

They each have different attributes/parameters that can be customized.

Take a look at this overview document.
POSTED BY: Rohit Namjoshi
Updating Name
Updating Name
Posted 4 years ago

Hello Rohit,

As far as I understand, the layers given in Information["*Layer"] are layers trained for a specific task. What I am doing is training my own NN. MMA is "magically" generating me results (more or less correct) with parameters I neither know, nor have control over.

Information about hidden layers, activation functions, etc. being used in NN are important if one needs other people to repreduce the results by other means, such as TensorFlow, sklearn, etc.

POSTED BY: Updating Name
Updating Name
Updating Name
Posted 4 years ago

Dear Rohit,

Thank you for the link. they are truly wonderful net models which can easily be incorporated into many projects. It certainly motivated me for some other projects. However, the one at hand is a research project, so the details I am looking for are important to me.

I found something on the net regarding hidden layers https://mathematica.stackexchange.com/questions/64785/how-to-change-neuralnetwork-options Have these parameters been removed from Mathematica?

I bet MMA has some internal variables that keep this information.
POSTED BY: Updating Name
Yeso Alde
Yeso Alde
Posted 4 years ago

Dear Rohit,

I think NetChain and relevant functions will work for me. I am really pleased to find that this didn't turn into a blackbox of blackmagic kind of thing:)

Your help is greatly appreciated.
POSTED BY: Yeso Alde
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract