A framework for generating subword vocabulary from a tensorflow dataset and building custom BERT tokenizer models.
machine-learning deep-learning tensorflow machine-translation vocabulary-builder bert subword wordpiece berttokenizer tensorflow-text
-
Updated
Jul 6, 2021 - Python