This program is utility to download pascal sentence dataset.
You can install by "git clone" command.
git clone https://github.com/rupy/PascalSentenceDataset.git
You must install some python libraries. Use pip command.
pyquery 1.2.9
requests 2.6.0
To download dataset, just run program as follow:
python pascal_sentence_dataset.py
You can also write code like this:
# importfrompascal_sentence_datasetimportPascalSentenceDataSet# create instancedataset=PascalSentenceDataSet()
# download imagesdataset.download_images()
# download sentencesdataset.download_sentences()
# create correspondence data by datasetdataset.create_correspondence_data()That's it!
Correspondence data is the csv data to correspond data id to image data.
Our lab created Japanese translation of Pascal Sentence Dataset. Translation class is the utility to use parallel translation data, "pascal_sentence_numbers.csv". You can get text files of two languages by the class. To use the class, you have to install depencent libraries as follow:
mecab-python 0.996
To use mecab-python, you have to install MeCab in addition.
To create Japanese & English parallel translation data, just run program as follow:
python pascal_sentence_dataset.py
You can also write code like this:
# importfromtranslationimportTranslation# put parallel translation data somewhere in advancecsv_file='translations/pascal_sentence_numbers.csv'# initialize instanceps=Translation(csv_file)
# create text data from csv fileps.read_csv_and_save_as_txt()
# create wakati-gaki text data (Japanese text data separated by space between each word)ps.wakati()