Implement system `alalc-ara-arab-latn-1997` (ALA-LC Romanization Table -- Arabic (1997)

[ronaldtse]

This issue is to implement the transliteration system of alalc-ara-arab-latn-1997.

This system is referred in the GeoNames database as ara_Arab2Latn_ALA_1997, with the system title 'ALA-LC Romanization Table -- Arabic (1997)'.

Tests should rely on the data extracted for the ara_Arab2Latn_ALA_1997 system in https://github.com/riboseinc/geonames-transliteration-data .

[ronaldtse]

@manuel489 this one is next. Spec here: http://catdir.loc.gov/catdir/cpso/romanization/arabic.pdf

Thanks!

Ping @nbh707

[manuelfuenmayor]

Hi guys,

I'd like to share some situation that I have recently noticed.

When I copy and paste an example given in the specs (for the tests), I realize that the Unicode values that compose that example are not included in the map list given by the same specs, even though its rendering looks exactly the same. Ok, let me explain...

To give an example, consider this Arabic word taken from the specs: وضع (waḍ‘ in Latin)

If we look at the Unicode values, these are: \u0648\u0636\u0639

I mean, وضع = \u0648\u0636\u0639

The thing is, I realized that these Unicode values are not included in map list given by the specs.

Maybe it's an error that comes from the action of copying and pasting. But I think this could affect the result of the tests.

So, to propose a solution:

• maybe I could include these Unicode values in the map list (even though they are not included in the map list provided by the specs). Or,

• I could reproduce the same Arabic word using the Unicode values that are actually included in the map list and use them in the tests. Following the above example, it would be something like:

ﻭﻀﻊ = \ufeed\ufec0\ufeca

• Or, if you want, I could keep copying and pasting the Arabic words into the test section just as they are.

Well, this message was just for information purposes,
Thanks!

[nbh707]

What do you mean by "the map list provided by the specs"?

[manuelfuenmayor]

Oh, yes, I mean the romanization list. In this case, this one:

[manuelfuenmayor]

What I do is copy every Arabic and Latin character and paste it into the map section (map:) of the yaml file. So I called it "map list".

[manuelfuenmayor]

After copying and pasting all the characters of the above list, part of the structure of the YAML file results in this code:

map:
 characters:

 # Initial
 '\u0627': '' # ا -- omit (see Note 1)
 '\ufe91': 'b' # ﺑ
 '\ufe97': 't' # ﺗ
 '\ufe9b': 'th' # ﺛ
 '\ufe9f': 'j' # ﺟ
 '\ufea3': 'ḥ' # ﺣ
 '\ufea7': 'kh' # ﺧ
 '\ufea9': 'd' # ﺩ
 '\ufeab': 'dh' # ﺫ
 '\ufead': 'r' # ﺭ 
 '\ufeaf': 'z' # ﺯ
 '\ufeb3': 's' # ﺳ
 '\ufeb7': 'sh' # ﺷ
 '\ufebb': 'ṣ' # ﺻ
 '\ufebf': 'ḍ' # ﺿ
 '\ufec3': 'ṭ' # ﻃ
 '\ufec7': 'ẓ' # ﻇ
 '\ufecb': '‘' # ﻋ
 '\ufecf': 'gh' # ﻏ
 '\ufed3': 'f' # ﻓ -- see Note 2
 '\ufed7': 'q' # ﻗ -- see Note 2
 '\ufedb': 'k' # ﻛ
 '\ufedf': 'l' # ﻟ
 '\ufee3': 'm' # ﻣ
 '\ufee7': 'n' # ﻧ
 '\ufeeb': 'h' # ﻫ
 '\ufeed': 'w' # ﻭ
 '\ufef3': 'y' # ﻳ
 
 
 # Medial
 '\ufe8e': '' # ﺎ -- omit (see Note 1)
 '\ufe92': 'b' # ﺒ
 '\ufe98': 't' # ﺘ
 '\ufe9c': 'th' # ﺜ
 '\ufea0': 'j' # ﺠ
 '\ufea4': 'ḥ' # ﺤ
 '\ufea8': 'kh' # ﺨ
 '\ufeaa': 'd' # ﺪ
 '\ufeac': 'dh' # ﺬ
 '\ufeae': 'r' # ﺮ
 '\ufeb0': 'z' # ﺰ
 '\ufeb4': 's' # ﺴ
 '\ufeb8': 'sh' # ﺸ
 '\ufebc': 'ṣ' # ﺼ
 '\ufec0': 'ḍ' # ﻀ
 '\ufec4': 'ṭ' # ﻄ
 '\ufec8': 'ẓ' # ﻈ
 '\ufecc': '‘' # ﻌ
 '\ufed0': 'gh' # ﻐ
 '\ufed4': 'f' # ﻔ -- see Note 2
 '\ufed8': 'q' # ﻘ -- see Note 2
 '\ufedc': 'k' # ﻜ
 '\ufee0': 'l' # ﻠ
 '\ufee4': 'm' # ﻤ
 '\ufee8': 'n' # ﻨ
 '\ufeec': 'h' # ﻬ -- see Note 3
 '\ufeee': 'w' # ﻮ
 '\ufef4': 'y' # ﻴ
 
 # Final
 '\ufe8e': '' # ﺎ -- omit (see Note 1)
 '\ufe90': 'b' # ﺐ
 '\ufe96': 't' # ﺖ
 '\ufe9a': 'th' # ﺚ
 '\ufe9e': 'j' # ﺞ
 '\ufea2': 'ḥ' # ﺢ
 '\ufea6': 'kh' # ﺦ
 '\ufeaa': 'd' # ﺪ
 '\ufeac': 'dh' # ﺬ
 '\ufeae': 'r' # ﺮ
 '\ufeb0': 'z' # ﺰ
 '\ufeb1': 's' # ﺱ
 '\ufeb6': 'sh' # ﺶ
 '\ufeba': 'ṣ' # ﺺ
 '\ufebe': 'ḍ' # ﺾ
 '\ufec2': 'ṭ' # ﻂ
 '\ufec6': 'ẓ' # ﻆ
 '\ufeca': '‘' # ﻊ
 '\ufece': 'gh' # ﻎ
 '\ufed2': 'f' # ﻒ -- see Note 2
 '\ufed6': 'q' # ﻖ -- see Note 2
 '\ufeda': 'k' # ﻚ
 '\ufede': 'l' # ﻞ
 '\ufee2': 'm' # ﻢ
 '\ufee6': 'n' # ﻦ
 '\ufeea': 'h' # ﻪ -- see Note 3
 '\u0629': 'h' # ة -- see Note 3
 '\ufeee': 'w' # ﻮ
 '\u064a': 'y' # ي


 # Alone
# '\u0627': '' # ا -- omit (see Note 1)
 '\ufe8f': 'b' # ﺏ
 '\ufe95': 't' # ﺕ
 '\ufe99': 'th' # ﺙ
 '\ufe9d': 'j' # ﺝ
 '\ufea1': 'ḥ' # ﺡ
 '\ufea5': 'kh' # ﺥ
 '\ufea9': 'd' # ﺩ
 '\ufeab': 'dh' # ﺫ
 '\ufead': 'r' # ﺭ
 '\ufeaf': 'z' # ﺯ
 '\ufeb1': 's' # ﺱ
 '\ufeb5': 'sh' # ﺵ
 '\ufeb9': 'ṣ' # ﺹ
 '\ufebd': 'ḍ' # ﺽ
 '\ufec1': 'ṭ' # ﻁ
 '\ufec5': 'ẓ' # ﻅ
 '\ufec9': '‘' # ﻉ (ayn)
 '\ufecd': 'gh' # ﻍ
 '\ufed1': 'f' # ﻑ -- see Note 2
 '\ufed5': 'q' # ﻕ -- see Note 2
 '\ufed9': 'k' # ﻙ
 '\ufedd': 'l' # ﻝ
 '\ufee1': 'm' # ﻡ
 '\ufee5': 'n' # ﻥ
 '\u0647': 'h' # ه
 '\u0629': 'h' # ة
 '\ufeed': 'w' # ﻭ
# '\u064a': 'y' # ي
 
 # Vowels and Diphthongs
 '\u064e': 'a'
 '\u064f': 'u'
 '\u0650': 'i'
 '\u064e\u0627': 'ā' # see Rule 5
 '\ufeef \u064e': 'á' # see Rule 6(a)
 '\ufeed \u064f': 'ū'
 '\ufeef \u0650': 'ī'
 '\ufeed\u0652 \u064e': 'aw'
 '\ufeef\u0652 \u064e': 'ay'

[chaaklau]

I agree with @manuel489 , and let me provide more details.

maybe I could include these Unicode values in the map list (even though they are not included in the map list provided by the specs).

I believe only codepoints between U+0600 to U+06FF under the Arabic block are used in actual Arabic text. Looks like these codepoints are missing in the above YAML.

The example above with the codepoints \ufeed\ufec0\ufeca are from the Arabic Presentation Forms-B block. These characters give you the required glyph even in isolation, e.g. for the same letter ب , in addition to the basic codepoint U+0628, there are additional codepoints that give you the alone (U+FE8F), initial (U+FE91), medial (U+FE92), final (U+FE90) forms respectively.

[manuelfuenmayor]

Ah ok, thanks @chaaklau . I will include the codepoints between U+0600 to U+06FF as well.

[manuelfuenmayor]

YAML file added in #244

[nbh707]

Hey Manuel I have been working on this already but I have not uploaded my work yet as I have been working on it offline.

[nbh707]

I don't think we need the initial, medial, final and alone forms. Any computer system can interpret what form to use from the basic Unicode characters depending on whether there is a space before, after or none at all. These forms are only used if someone wants to refer to that specific form in isolation as a letter. Since we are not dealing with letters, but full words, this is irrelevant.

Please hold off on anymore until I have a chance to upload what I have done already. I am not familiar with how to make updates via github so if you could help me out there I can get you what I have done so far.

[nbh707]

Please see: #219