Message Boards Message Boards

How to Use GPT2 Transformer for FeatureExtraction

Posted 1 year ago

I've been reading the documentation about GPT2 Transformer Trained on WebText Data. I noticed that it can perform 2 tasks and one of then is FeatureExtraction.

lm = NetModel[{"GPT2 Transformer Trained on WebText Data", "Task" -> "FeatureExtraction"}]

However, there are no examples on how to use it for Feature Extraction. I will appreciate an example or hints on how to use it for this purpose.

2 Replies

I did this

input = "Dalila went to the market"
features = NetExtract[lm, "Input" -> input]

I got the following error

NetExtract::pspec1: Part specification Input->Dalila went to the market is not applicable.

Thank you for presenting the example of how to use GPT2 for feature Extraction.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract