Created cardinal Semiotic class for Kannada TN - #448
Conversation
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
…iles Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
9149d5a to
53b3447
Compare
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Additionally, I removed the unnecessary weights from cardinal.py file from tagger module. Testing |
Signed-off-by: richa-2002 <richa@nvidia.com>
| class CardinalFst(GraphFst): | ||
| """ | ||
| Finite state transducer for classifying cardinals, e.g. | ||
| 23 -> cardinal { negative: "true" integer: "ಇಪ್ಪತ್ತಮೂರು" } |
There was a problem hiding this comment.
let's document what is the max magnitude for large numbers that the grammar covers
| class CardinalFst(GraphFst): | ||
| """ | ||
| Finite state transducer for classifying cardinals, e.g. | ||
| 23 -> cardinal { negative: "true" integer: "ಇಪ್ಪತ್ತಮೂರು" } |
There was a problem hiding this comment.
-23, done. Replaced it with the maximum magnitude covered by the grammar.
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Testing
|
| @@ -0,0 +1,20 @@ | |||
| ನಿದ್ರೆ~ನಿದ್ರೆ | |||
| ಯಾಹೂ!~ಯಾಹೂ! | |||
There was a problem hiding this comment.
please follow the style of English word test cases
There was a problem hiding this comment.
Updated the word test cases to align with the English test cases style, covering whitespace, leading/trailing spaces, punctuation, hyphenated words, and digit–punctuation spacing.
Measure, money, decimal grammar scenarios and English-specific letter–digit splits are not included yet, as Kn does not currently support those cases.
| @@ -0,0 +1,74 @@ | |||
| .~. | |||
There was a problem hiding this comment.
please follow the style of English punctuation test cases
There was a problem hiding this comment.
Updated the Kn punctuation test cases based on the English test case format.Added test_cases_punctuation_match_input.txt to cover punct_post_process=True, where punctuation spacing is adjusted to match the original input similar to EN.
Measure, money, decimal grammar scenarios and English-specific letter–digit splits are not included yet, as Kn does not currently support those cases.
| ಡಾ. ರಾಮು~ಡಾ. ರಾಮು | ||
| ಎ.ಪಿ.~ಎ.ಪಿ. | ||
| ರೂ. ನೂರು~ರೂ. ನೂರು | ||
| 2!~ಎರಡು ! |
There was a problem hiding this comment.
why is a spaced introduced after TN?
There was a problem hiding this comment.
The extra spaces occur when punct_post_process=False. When punct_post_process=True, those spaces are removed.
| import pynini | ||
| from pynini import Far | ||
| from pynini.export import export | ||
| from pynini.lib import byte, pynutil, utf8 |
There was a problem hiding this comment.
please clean up any unused imports in all scripts (byte is not used here)
| """ | ||
| Finite state transducer for verbalizing cardinals, e.g. | ||
| cardinal { integer: "ಐದು" } -> ಐದು | ||
| cardinal { negative: "true" integer: "ಇಪ್ಪತ್ತ್ಮೂರು" } -> ಮೈನಸ್ ಇಪ್ಪತ್ತ್ಮೂರು |
There was a problem hiding this comment.
is this the same spelling as in TSV?
Signed-off-by: richa-2002 <richa@nvidia.com>
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Testing Pytest: All test cases passed — Cardinal, Word, and Punctuation. |
3 participants
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.