Prevert iterator

To use the prevert parser, copy the file prevert.py in your directory.

Use

# import librariesfromprevertimportdatasetimportpandasaspd

If you are using the MaCoCu corpora in the XML format, the method dataset() needs only the path of the file as the argument:

# Open the dataset with the prevert parser dset=dataset("/data/monolingual/mk.xml")

dset consists of docs where you can access the metadata by doc.meta['attribute_name']. Docs consist of paragraphs where you can access the metadata by par.meta['attribute_name'].

Basic use:

fordocindset: # iterating through documents of a datasetprint(doc.meta) # all attributesprint(eval(doc.meta['lang_distr'])[0][0]) # most prominent language in the documentprint(str(doc)) # whole document textforparindoc: # iterating through paragraphs of a documentprint(par.meta['id']) # specific attributeprint(str(par)) # whole paragraph textprint(doc.to_prevert()) # obtaining the original format