How to Use GPT2 Transformer for FeatureExtraction

Questions
, , ,
1

I’ve been reading the documentation about GPT2 Transformer Trained on WebText Data.
I noticed that it can perform 2 tasks and one of then is FeatureExtraction.

lm = NetModel[{"GPT2 Transformer Trained on WebText Data", "Task" -> "FeatureExtraction"}]

However, there are no examples on how to use it for Feature Extraction. I will appreciate an example or hints on how to use it for this purpose.

2

Thank you for presenting the example of how to use GPT2 for feature Extraction.

3

I did this

input = "Dalila went to the market"
features = NetExtract[lm, "Input" -> input]

I got the following error

NetExtract::pspec1: Part specification Input->Dalila went to the market is not applicable.