Uh oh!
There was an error while loading. Please reload this page.
Tamil TN Cardinal Semiotic Class - #449
Conversation
b06d8e8 to
6b431c2CompareSigned-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
for more information, see https://pre-commit.ci Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
231dd35 to
eaf2223CompareSigned-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
Signed-off-by: surendran-246 <surendrans@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: surendran-246 <surendrans@nvidia.com>
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: surendran-246 <surendrans@nvidia.com>
0e62120 to
2907744CompareSigned-off-by: surendran-246 <surendrans@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: surendran-246 <surendrans@nvidia.com>
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: surendran-246 <surendrans@nvidia.com>
for more information, see https://pre-commit.ci
folivoramanh
left a comment
There was a problem hiding this comment.
make sure both pytest and sparrrowhawk test pass
There was a problem hiding this comment.
is there any solution that not listing 10 - 99? (eg 20 - 29 share same rule ಮೂ + digit?)
There was a problem hiding this comment.
same question: is there any solution that not listing 10 - 99? (eg 20 - 29 share same rule ಮೂ + digit?)
| class CardinalFst(GraphFst): | ||
| """ | ||
| Finite state transducer for classifying cardinals, e.g. |
There was a problem hiding this comment.
please check other language's format and refractor the docstring
| digit = sf("digit") | ||
| zero = sf("zero") | ||
| teens_ties = pynini.union(sf("teens_and_ties"), sf("teens_and_ties_en")) | ||
| teens_and_ties = pynutil.add_weight(teens_ties, -0.1) |
There was a problem hiding this comment.
try not to add custom weight
| def __init__(self, deterministic: bool = True, lm: bool = False): | ||
| super().__init__(name="cardinal", kind="classify", deterministic=deterministic) | ||
| def sf(name): |
| super().__init__(name="word", kind="classify", deterministic=deterministic) | ||
| # Define Tamil characters and symbols using pynini.union | ||
| TAMIL_CHAR = pynini.union( |
There was a problem hiding this comment.
if dont use post processing, is there any redundant blank space? if yes, try to figure out the root cause from your implementation (punct, word, etc...), do not abuse post processing
| output = SPACE_DUP.sub(' ', output[1:]) | ||
| if self.lang in ["en", "hi", "vi"] and hasattr(self, 'post_processor') and self.post_processor is not None: | ||
| if ( |
There was a problem hiding this comment.
if self.lang in ["en", "hi", "ta", "vi"] and hasattr(self, 'post_processor') and self.post_processor is not None:
| to_lower = args.input_case == "lower_cased" | ||
| training_data = load_files([file_path], to_lower=to_lower) | ||
| # print("Loading training data: " + file_path) |
| @pytest.mark.unit | ||
| def test_norm(self, test_input, expected): | ||
| pred = self.normalizer.normalize(test_input, verbose=False) | ||
| assert pred.strip() == expected.strip() |
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.