Message Boards Message Boards

Unable to get same outputs of BERT examples

Posted 2 years ago

enter image description here

Why can't I have the same result as the 9th and 10 the cells on the website below?

https://resources.wolframcloud.com/NeuralNetRepository/resources/BERT-Trained-on-BookCorpus-and-English-Wikipedia-Data

POSTED BY: Pei Hsuan Shen
5 Replies

There was a bug found in the pre-processing BERT when the front-end language is not English (for instance when it is Chinese).

We will update BERT to fix it. (and people who already downloaded the model will have to re-download it after having called ResourceRemove@ResourceObject["BERT Trained on BookCorpus and English Wikipedia Data"])

In the meantime the workaround for this bug is:

net = NetModel[{"BERT Trained on BookCorpus and English Wikipedia Data", "InputType" -> "ListOfStrings"}]

net2 = NetReplacePart[net, {"Input", "Function"} ->
   ReplaceAll[NetExtract[net, {"Input", "Function"}],
    RemoveDiacritics -> Function @ RemoveDiacritics[#, Language -> "English"]]]

then use net2 in place of net.

Posted 2 years ago

What Mathematica version are you using?

POSTED BY: Rohit Namjoshi
Posted 2 years ago

All methods in the page I have tried. But none of them can work.

POSTED BY: Pei Hsuan Shen
Posted 2 years ago

Hi Pei,

I cannot reproduce it either on 13.0.0 for Mac OS X ARM (64-bit) (December 3, 2021). Similar issues have been reported on V12, you could try some of the suggestions in this thread.

POSTED BY: Rohit Namjoshi

It might be a bug, I cannot reproduce it in my version. You should contact the support team using https://www.wolfram.com/support/contact/email/?topic=feedback

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract