Unfortunately, training transformers is slightly more complicated. You would need to add code for the loss function.
You could instead train a simple (non-transformer) text classifier in Mathematica using the Classify function:
classifier = Classify[{"This is a happy example"->1, "This is a negative or bad example" -> 0}]
If you really wanted to train transformers (and do so without implementing a lot of code yourself), you could check out the HiggingFace transformer github repo - which is a Python library implementing training of transformers.
