Message Boards Message Boards

GROUPS:

Arbitrary length vector of strings into Net

Posted 4 months ago
542 Views
|
2 Replies
|
0 Total Likes
|

Hi, I've been struggling with an input problem for a while now. I'm hoping someone here could point me in the right direction.

net=NetChain[{EmbeddingLayer[50,39265]},"Input"->{"Varying",NetEncoder["Tokens"]}]
net=NetInitialize[net] 
net[{"hi","my"}]

I'm expecting this code to output a 2x2x50 tensor but I get these errors instead: enter image description here

If it helps, what I'm trying to do is replicate the following net from https://arxiv.org/pdf/1803.09337.pdf:

enter image description here

Of course I haven't built the rest of the NetModel, but I wanted to get the Input right first.

2 Replies
Posted 4 months ago

Hi Alec,

Isn't "Tokens" implicitly "Varying"? This works

net = NetInitialize@
 NetChain[{EmbeddingLayer[50, 39265]},  "Input" -> NetEncoder["Tokens"]]

Hi Rohit,

Thanks for the reply, yeah you're right. What I have so far is something like this:

net = NetChain[{NetMapOperator[{EmbeddingLayer[50, 39265]}, 
    "Input" -> NetEncoder["Tokens"]],
   NetBidirectionalOperator[LongShortTermMemoryLayer[50]],
   NetBidirectionalOperator[LongShortTermMemoryLayer[50]],
   PoolingLayer[{100}]
   }]

I believe, this defines the sentence-level neural network. I now want to modify net so that it takes an arbitrary sequence of strings. I thought that the way to do this is to do

net = NetMapOperator[net]

But this throws "NetMapOperator::invvardim: Varying dimension in port "Output" that isn't the first dimension".

I'm looking for the ultimate behaviour to be something like this

net[{"Segment 1.", "blah blah", "Segment 2.", "blah blah"}] === {0, 1,
   0, 1}

where the binary number indicates whether the corresponding string ends a segment taking the context of all the given strings. Attempting to do this I incorrectly used {"Varying", NetEncoder["Tokens"]} at the start.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract