Thanks for URIEL+ — the expanded coverage is exactly what I need. I work on Nheengatu (yrl / nhen1239), a Tupi-Guarani language of the Brazilian Amazon, and Grambank gives it 183 observed syntactic features where the original URIEL had none.
Environment: urielplus 1.2, Python 3.10, macOS on Apple silicon, 24 GB RAM.
Three questions.
1. Precomputed distances. lang2vec distributes a precomputed distances.zip, so l2v.distance("syntactic", "por", "yrl") is instant. URIEL+ seems to require running integrate_databases() plus imputation locally. Here integrate_glottolog has been running for over 30 minutes at ~5.5 GB RSS: it adds the language and dialect codes from dialects.csv (26,879 distinct codes across the two code columns) to the 7,970 already present, then recomputes the phylogeny, geocoord and scriptural vectors — all before softimpute even starts.
Is there a precomputed distance file we could download instead? If not, would you consider publishing one?
2. Two data files missing from the PyPI package.pip install urielplus (v1.2) installs without duplicate_feature_sets.json and resource_languages.json, so integrate_databases() fails with:
FileNotFoundError: .../urielplus/database/urielplus_csvs/duplicate_feature_sets.json
Copying both from the repo fixes it. Looks like a MANIFEST.in issue.
3. Genetic distance for sparse family vectors. In the original URIEL, genetic_distance("grn", "gug") returns 1.0, though both are Guarani (macrolanguage and Paraguayan variety). grn–yrl is also 1.0, while tpn–yrl (Tupinamba–Nheengatu) is a sensible 0.25. The grn family vector has only 2 active branches out of 3,718. Does URIEL+ correct this? I ask because I plan to use a Tupi-Guarani language as a genealogical control.
Happy to send a PR for the missing data files if useful.
Thanks for URIEL+ — the expanded coverage is exactly what I need. I work on Nheengatu (yrl / nhen1239), a Tupi-Guarani language of the Brazilian Amazon, and Grambank gives it 183 observed syntactic features where the original URIEL had none.
Environment: urielplus 1.2, Python 3.10, macOS on Apple silicon, 24 GB RAM.
Three questions.
1. Precomputed distances. lang2vec distributes a precomputed
distances.zip, sol2v.distance("syntactic", "por", "yrl")is instant. URIEL+ seems to require runningintegrate_databases()plus imputation locally. Hereintegrate_glottologhas been running for over 30 minutes at ~5.5 GB RSS: it adds the language and dialect codes fromdialects.csv(26,879 distinct codes across the two code columns) to the 7,970 already present, then recomputes the phylogeny, geocoord and scriptural vectors — all before softimpute even starts.Is there a precomputed distance file we could download instead? If not, would you consider publishing one?
2. Two data files missing from the PyPI package.
pip install urielplus(v1.2) installs withoutduplicate_feature_sets.jsonandresource_languages.json, sointegrate_databases()fails with:Copying both from the repo fixes it. Looks like a MANIFEST.in issue.
3. Genetic distance for sparse family vectors. In the original URIEL,
genetic_distance("grn", "gug")returns 1.0, though both are Guarani (macrolanguage and Paraguayan variety).grn–yrlis also 1.0, whiletpn–yrl(Tupinamba–Nheengatu) is a sensible 0.25. Thegrnfamily vector has only 2 active branches out of 3,718. Does URIEL+ correct this? I ask because I plan to use a Tupi-Guarani language as a genealogical control.Happy to send a PR for the missing data files if useful.