The Encoder-Decoder Transformer Neural Network Architecture

[jofreep (Jofre Espigule-Pons)]

Open in Wolfram Cloud ↗

[wolframcommunity (EDITORIAL BOARD)]

– you have earned Featured Contributor Badge
Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!

[sangdonlee]

Hi, Jofre,
I am working through your notebook and stuck at the following command. Could you please let me know how to modify the command?

vocabulary = NetExtract[ResourceFunction["GPTTokenizer"][], "Tokens"]

(* this creates an error, and it is modified as follows*)

netDecoder = NetDecoder[{"Class", NetExtract[ResourceFunction["GPTTokenizer"][], "Tokens"]}]
netDecoderModified = NetDecoder[{"Class", {netEncoder, "Tokens"}}]

[okazakikotaro (Kotaro Okazaki)]

Sangdon-san,

It seems that the problem is caused by a difference in the GPTTokenizer version.At the time this post was made, the GetTokenizer version was 1.0.0. Currently, it is 1.2.0, and there seems to be an incompatibility issue.
It may work well by specifying the version in GetTorganizer as follows.

NetDecoder[{"Class",
  NetExtract[ResourceFunction["GPTTokenizer", "Version" -> "1.0.0"][],
    "Tokens"]}]

[sangdonlee]

Kotaro-san,
Thank you very. I appreciate.