Message Boards Message Boards

Unable to get same outputs of BERT examples

Posted 2 years ago

enter image description here

Why can't I have the same result as the 9th and 10 the cells on the website below?

https://resources.wolframcloud.com/NeuralNetRepository/resources/BERT-Trained-on-BookCorpus-and-English-Wikipedia-Data

POSTED BY: Pei Hsuan Shen
5 Replies

It might be a bug, I cannot reproduce it in my version. You should contact the support team using https://www.wolfram.com/support/contact/email/?topic=feedback

Posted 2 years ago

Hi Pei,

I cannot reproduce it either on 13.0.0 for Mac OS X ARM (64-bit) (December 3, 2021). Similar issues have been reported on V12, you could try some of the suggestions in this thread.

POSTED BY: Rohit Namjoshi
Posted 2 years ago

All methods in the page I have tried. But none of them can work.

POSTED BY: Pei Hsuan Shen
Posted 2 years ago

What Mathematica version are you using?

POSTED BY: Rohit Namjoshi

There was a bug found in the pre-processing BERT when the front-end language is not English (for instance when it is Chinese).

We will update BERT to fix it. (and people who already downloaded the model will have to re-download it after having called ResourceRemove@ResourceObject["BERT Trained on BookCorpus and English Wikipedia Data"])

In the meantime the workaround for this bug is:

net = NetModel[{"BERT Trained on BookCorpus and English Wikipedia Data", "InputType" -> "ListOfStrings"}]

net2 = NetReplacePart[net, {"Input", "Function"} ->
   ReplaceAll[NetExtract[net, {"Input", "Function"}],
    RemoveDiacritics -> Function @ RemoveDiacritics[#, Language -> "English"]]]

then use net2 in place of net.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract