Uh oh!
There was an error while loading. Please reload this page.
Created cardinal Semiotic class for Kannada TN - #448
Conversation
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
…iles Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
9149d5a to
53b3447Comparefor more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
is there any solution that not listing 10 - 99? (eg 20 - 29 share same rule ಮೂ + digit?)
There was a problem hiding this comment.
is there any solution that not listing 10 - 99? (eg 20 - 29 share same rule ಮೂ + digit?)
| pynini.string_file(get_abs_path("data/numbers/teens_and_ties.tsv")), | ||
| pynini.string_file(get_abs_path("data/numbers/teens_and_ties_en.tsv")), | ||
| ) | ||
| teens_and_ties = pynutil.add_weight(teens_ties, -0.1) |
| hundreds = pynini.string_file(get_abs_path("data/numbers/hundreds.tsv")) | ||
| self.digit = digit | ||
| self.zero = zero |
There was a problem hiding this comment.
only set self.X = X if it's reusable on next semiotic class
| EMPTY = pynini.accep("") # epsilon, used when no zeros are deleted | ||
| # suffixes (each defined once) | ||
| suf_thousand = pynutil.insert(" ಸಾವಿರ") |
There was a problem hiding this comment.
can we add the suffix word to a tsv file? try not to hardcode on .py file
| NEMO_CHAR = utf8.VALID_UTF8_CHAR | ||
| NEMO_DIGIT = byte.DIGIT | ||
| NEMO_KN_DIGIT = pynini.union("೦", "೧", "೨", "೩", "೪", "೫", "೬", "೭", "೮", "೯").optimize() |
There was a problem hiding this comment.
we have kn digit tsv file, why we need it in here?
| if ( | ||
| self.lang in ["en", "hi", "kn", "vi"] | ||
| and hasattr(self, 'post_processor') | ||
| and self.post_processor is not None |
There was a problem hiding this comment.
do not add (), let it only:
if self.lang in ["en", "hi", "kn", "vi"] and hasattr(self, 'post_processor') and self.post_processor is not None:
| # print("Loading training data: " + file_path) | ||
| # to_lower = True if args.input_case == "lower_cased" else False | ||
| # training_data = load_files([file_path], to_lower=to_lower) |
There was a problem hiding this comment.
pls make sure both pytest and sparrowhawk test all pass
| @pytest.mark.unit | ||
| def test_norm(self, test_input, expected): | ||
| pred = self.normalizer.normalize(test_input, verbose=False) | ||
| assert pred.strip() == expected.strip() |
There was a problem hiding this comment.
we need to make sure it pass without .strip()
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.