A multi-threaded GitHub scraper to collect Python code with docstrings from public repositories, creating a well-documented dataset for the JaraConverse LLM model.
nlpscraperscriptpython3datasetdataset-generationnlp-machine-learningdata-scrapinggithub-scraperpython-codedocstring-generatorllmcausal-language-modelingdataset-scriptspython-datasetllm-trainingdocst
-
Updated
Jul 21, 2024 - Python