From 53eff3ad6d4f845e1eb93a80a94afab28af85fd2 Mon Sep 17 00:00:00 2001 From: Fatima Jawadwala Date: Tue, 8 Mar 2022 22:46:20 +0530 Subject: [PATCH] archeived gsoc and gsod --- docs/source/gsoc/gsoc19_final_report.rst | 69 --- docs/source/gsoc/gsoc_2017.rst | 422 -------------- docs/source/gsoc/gsoc_2018.rst | 4 - docs/source/gsoc/gsoc_2019.rst | 712 ----------------------- docs/source/gsod/gsod-2020.rst | 265 --------- docs/source/gsod/gsod-2021.rst | 234 -------- docs/source/gsod/gsod_2019.rst | 383 ------------ docs/source/gsod/gsod_2019_report.rst | 103 ---- 8 files changed, 2192 deletions(-) delete mode 100644 docs/source/gsoc/gsoc19_final_report.rst delete mode 100644 docs/source/gsoc/gsoc_2017.rst delete mode 100644 docs/source/gsoc/gsoc_2018.rst delete mode 100644 docs/source/gsoc/gsoc_2019.rst delete mode 100644 docs/source/gsod/gsod-2020.rst delete mode 100644 docs/source/gsod/gsod-2021.rst delete mode 100644 docs/source/gsod/gsod_2019.rst delete mode 100644 docs/source/gsod/gsod_2019_report.rst diff --git a/docs/source/gsoc/gsoc19_final_report.rst b/docs/source/gsoc/gsoc19_final_report.rst deleted file mode 100644 index 0b4e3111..00000000 --- a/docs/source/gsoc/gsoc19_final_report.rst +++ /dev/null @@ -1,69 +0,0 @@ -Google Summer of Code 2019 - Final report -========================================= - -Project: Approximately similar file detection in DeltaCode ----------------------------------------------------------- - -**Arnav Mandal ** - -Project Overview ----------------- - -DeltaCode is a tool to compare and report scan differences. It takes JSON files as an input which -is the output of ScanCode-toolkit as well. When comparing files, it only uses the exact comparison. -By exact comparison, I mean it compares the hash value of the files. The output of DeltaCode is a -JSON/CSV file which includes the details of the scan such as delta score, delta count, etc. The -goal of this project is to improve the usefulness of the delta by also finding files that are -mostly the same (e.g. quasi or near duplicates) vs. files that are completely different. After this -project, DeltaCode would be able to detect similar files in a directory approximately. - -Requirements of the project ---------------------------- - -- Provided two files using ScanCode-toolkit, the new near-duplicate detection should return the - distance between the two files. -- The code should be seamlessly integrated with ScanCode-toolkit. It should be highly configurable - by the maintainers. -- The strictness of near-duplicates should be noted and adjusted by a threshold variable. - -The Project ------------ - -- Addition of new fingerprint plugin in the ScanCode Toolkit. -- Implementation and integration of the fingerprint generation algorithm in the ScanCode Toolkit - codebase. -- Implementation of distance finding algorithm between the files and process them further in the - DeltaCode codebase. -- Integration of fingerprint field in the JSON file to compare the deltas and provide them with - appropriate scores. -- Make changes to old unit tests and addition of new unit tests in ScanCode Toolkit as well as - DeltaCode. - -I have completed all the tasks that were in the scope of this GSoC project. - -Pull Requests -------------- - -- https://github.com/nexB/scancode-toolkit/pull/1576 [Closed] (something went wrong while rebasing) -- https://github.com/nexB/scancode-toolkit/pull/1651 [Merged] -- https://github.com/nexB/deltacode/pull/128 [Merged] - -Links ------ - -.. - [Org Link] https://summerofcode.withgoogle.com/organizations/6118953540124672/ - [Project Link] https://summerofcode.withgoogle.com/projects/#6422961651712000 - -- `Project Details `_ -- `Proposal `_ -- `ScanCode Toolkit `_ -- `DeltaCode `_ - ------------- - -I’ve had a wonderful time during these three months and have learned plenty of things. I would -really like to thank `@pombredanne `_, -`@steven-esser `_, and `@JonoYang `_ for their -constant support throughout the journey. From good job claps to nit-picky constructive -code-reviews, I enjoyed every bit of this GSoC project. diff --git a/docs/source/gsoc/gsoc_2017.rst b/docs/source/gsoc/gsoc_2017.rst deleted file mode 100644 index d3d383c2..00000000 --- a/docs/source/gsoc/gsoc_2017.rst +++ /dev/null @@ -1,422 +0,0 @@ -Google Summer of Code 2017 -========================== - - -.. image:: https://cdn.rawgit.com/wiki/nexB/aboutcode/aboutcode_logo.svg - :target: http://www.aboutcode.org/ - :alt: AboutCode Logo - - -Welcome to AboutCode! -This year AboutCode is a mentoring Organization for the Google Summer of Code 2017 edition. - -`AboutCode `_ is a project to uncover data ... about software code: - - -* where does it come from? -* what is its license? copyright? -* is it secure, maintained, well coded? - -All these are questions that are important to find answers to when there are million of free and -open source software components available on the web. - -Where software comes from and what is its license should be a problem of the past, such that -everyone can safely consume more free and open source software. Come and join us to make it so! - -Our tools are used to help detect and report the origin and license of source code, packages and -binaries, as well as discover software and package dependencies, track vulnerabilities, bugs and -other important software component attributes. - -Contact -------- - -Subscribe to the mailing list at https://lists.sourceforge.net/lists/listinfo/aboutcode-discuss -and introduce yourself and start the discussion! The mailing list is usually the better option -to avoid timezone gaps. - -The list archive have also plenty of interesting information. Someone may have asked your question -before. Search and browse the archives at -https://sourceforge.net/p/aboutcode/mailman/aboutcode-discuss/ ! - -For short chats, you can also join the #aboutcode IRC channel on Freenode or the Gitter channel at -https://gitter.im/aboutcode-org/discuss - -For personal issues, you can contact the org admin directly: @pombredanne and pombredanne@gmail.com - -Please ask questions the smart way: http://www.catb.org/~esr/faqs/smart-questions.html - -Technology ----------- - -Discovering the origin of code is a vast topic. We primarily use Python for this and some C/C++ and -JavaScript, but we are open to using any other language within reason. - -Our domain includes text analysis and processing (for instance for copyrights and licenses), -parsing (for package manifest formats), binary analysis (to detect the origin and license of -binaries, which source code they come from, etc) as well as web based tools and APIs (to expose the -tools and libraries as web services). - -About your project application ------------------------------- - -We expect your application to be in the range of 1000 words. Anything less than that will probably -not contain enough information for us to determine whether you are the right person for the job. -Your proposal should contain at least the following information, plus anything you think is -relevant: - - -* Your name -* Title of your proposal -* Abstract of your proposal -* Detailed description of your idea including explanation on why is it innovative and what it will - contribute - - * hint: explain your data structures and the main processing flows in details. - -* Description of previous work, existing solutions (links to prototypes, bibliography are more - than welcome) -* Mention the details of your academic studies, any previous work, internships -* Relevant skills that will help you to achieve the goal (programming languages, frameworks)? -* Any previous open-source projects (or even previous GSoC) you have contributed to and links. -* Do you plan to have any other commitments during GSoC that may affect your work? Any - vacations/holidays? Will you be available full time to work on your project? (Hint: do not bother - applying if this is not a serious full time commitment) - -Subscribe to the mailing list at https://lists.sourceforge.net/lists/listinfo/aboutcode-discuss or -join the #aboutcode IRC channel on Freenode and introduce yourself and start the discussion! - -You need to understand something about open source licensing or package managers or code and -binaries static analysis. The best way to demonstrate your capability would be to submit a small -patch ahead of the project selection for an existing issue or a new issue. - -Project ideas -------------- - -**ScanCode live scan server** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This project is to use ScanCode as a library in a web and REST API application that allows you to -scan code on demand by entering a URL and then store the scan results. It could also be made -available as a Travis or Github integration to scan on commit with webhooks. Bonus feature is to -scan based on a received tweet of similar IRC or IM integration. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit - -- **Mentors** : - - * @steven-esser https://github.com/steven-esser - * @tdruez https://github.com/tdruez - -**Package security vulnerability data feed (and scanner)** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The end goal for this project is to build on existing projects to match packages identified by -ScanCode to existing vulnerability alerts. This is not trivial as there are several gaps in the CVE -data and how they relate to packages as they are detected by ScanCode or else. This is a green -field project. - -The key points to tackle are: - - -#. create the tools to build a free and open source structured and curate security feed - - - * the aggregation of packages vulnerabilities feeds in a common and structured model (CVE, - distro trackers, etc), - * the aggregation of additional security data (CWE, CPE, and more) in that model, - * the correlation of the different data items, creating accurate relationships and matching of - actual package identifiers to vulnerabilities, - * an architecture for community curation of vulnerabilities, correlation and enhancement of the - data. - -#. as a side bonus, build the tools in ScanCode to match detected packages to this feed. - Note there is no FOSS tool and DB that does all of this today (only proprietary solutions such - as vfeed or vulndb). - - -- **Some Related URLS for other projects in the same realm** : - - * https://github.com/cve-search/cve-search - * https://github.com/jeremylong/DependencyCheck/ - * https://github.com/victims/victims-cve-db - * https://github.com/rubysec/ruby-advisory-db - * https://github.com/future-architect/vuls - * https://github.com/coreos/clair - * https://github.com/anchore/anchore/ - * https://github.com/pyupio/safety-db - * https://github.com/RetireJS/retire.js - * and many more including Linux distro feeds - -- **Mentors** : - - * @steven-esser https://github.com/steven-esser - * @JonoYang https://github.com/JonoYang - * @pombredanne https://github.com/pombredanne - -**Port the Python license expression library to JScript and prepare and publish an NPM package**: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Use automated code translation (for JS) for the port. Add license expression support to -AboutCodeMgr with this library. As a bonus, create a web server app and API service to parse and -normalize ScanCode and SPDX license expressions either in Python or JavaScript. - - -- **URLS** : - - * https://github.com/nexB/license-expression - * https://github.com/bastikr/boolean.py - * https://github.com/nexB/aboutcode-manager - * https://github.com/jashkenas/coffeescript/wiki/list-of-languages-that-compile-to-js - -- **Mentors** : - - * @JonoYang https://github.com/JonoYang - * @steven-esser https://github.com/steven-esser - -**MatchCode** : -^^^^^^^^^^^^^^^^^^ - -Create a system for matching code using checksums and fingerprints -against a repository of checksums and fingerprints. Create a basic -repository for storing these fingerprints and expose a basic API. Create -a client that can collect fingerprints on code and get matches using API -calls to this repository or package manager APIs (Maven, Pypi, etc), or -search engines APIs such as searchcode.com, debsources, or Github or -Bitbucket commit hash searches/API or the SoftwareHeritage.org API. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit-contrib for samecode fingerprints drafts. - * https://github.com/nexB/scancode-toolkit for commoncode hashes - -- **Mentors** : - - * @pombredanne https://github.com/pombredanne - -**ScanCode scan deduction** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The goal of this project is to take existing scan and match results and infer summaries and -deduction at a higher level, such as the licensing of a whole directory tree. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit/issues/426 - * https://github.com/nexB/scancode-toolkit/issues/377 - -- **Mentors** : - - * @pombredanne https://github.com/pombredanne - * @JonoYang https://github.com/JonoYang - -**DeltaCode** : -^^^^^^^^^^^^^^^^^^ - -A new tool to help determine at a high level if the licensing for two codebases or versions of -code has changed, and if so how. This is NOT a generic diff tool that identifies all codebase -differences, rather it focuses on changes in licensing based on differences between ScanCode files. - - -- **Mentor** : - - * @steven-esser https://github.com/steven-esser - -**License and copyright detection benchmark** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Compare ScanCode runtimes with Fossology, licensee, LicenseFinder, license-check, ninka, slic, LiD -and others. This project is to create a comprehensive test suite and a benchmark for several FOSS -open source license and copyright detection engines, establish mappings between the different -conventions they use for license identification and evaluate and publish the results of detection -accuracy and precision. - -Note that this not about the speed of scanning: the performance and time taken is accessory and a -nice to have result only. What matters is the accuracy of the license detection: - - -#. is the right license detected and how correct is this detection? -#. when a license is detected is the correct exact text matched and returned? - -So what is needed is a (large) test set of files. - -Then establishing a ground truth for reference e.g. detecting then reviewing manually possibly with -scancode to set up the baseline that will be used to compare all the scanners. - -Then run the other tools and scancode to see how well they perform and of course establish a -mapping of license identifiers: each tool may use different license ids so we need to map these -to the ids used in the test baseline (e.g. the scancode license keys): all this has to be built, -possibly reusing some or all of the scancode tests and lacing in all the tests from the other tools -and adding more ass needed. - - -- **Mentors** : - - * @mjherzog https://github.com/mjherzog - * @pombredanne https://github.com/pombredanne - -**Improved copyright parsing in ScanCode** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -by keeping track of line numbers and offsets where copyrights are found. This would likely require -either replacing or enhancing NLTK which is used as a natural language parser to add support for -tracking where a copyright has been detected in a scanned text. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit/tree/develop/src/cluecode - -- **Mentor** : - - * @JonoYang https://github.com/JonoYang - -**Support full JSON and ABCD formats in AttributeCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - -- **URLS** : - - * https://github.com/nexB/attributecode/issues/277 - -- **Mentor** : - - * @chinyeungli https://github.com/chinyeungli - -**Transparent archive extraction in ScanCode** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -ScanCode archive extraction is currently done with a separate command line invocation. The goal -of this project is to integrate archive extraction transparently into the ScanCode scan loop. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit/issues/14 - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**Automated docker and VM images static package analysis** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -to determine which packages are installed in Docker layers for RPMs, Debian or Alpine Linux. This -is for the conan Docker image analysis tool. - - -- **URLS** : - - * https://github.com/pombredanne/conan - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**Plugin architecture for ScanCode** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Create ScanCode plugins for outputs to multiple formats (CSV, JSON, SPDX, Debian Copyright) - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit/issues/552 - * https://github.com/nexB/scancode-toolkit/issues/381 - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**Static analysis of binaries for build tracing in TraceCode** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -TraceCode does system call tracing. The goal of this project is to do the same using symbol, debug -symbol or string matching to accomplish something similar, - - -- **URLS** : - - * https://github.com/nexB/tracecode-build for the existing non-static tool - * https://github.com/nexB/scancode-toolkit-contrib for the work in progress on binaries/symbols - parsers/extractors - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**Better support tracing the lifecycle of file descriptors in TraceCode build** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -TraceCode does system call tracing. The goal of this project is to improve the way we track -open/close file descriptors in the trace to reconstruct the life of a file. - - -- **URLS** : - - * https://github.com/nexB/tracecode-build - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**Create Debian and RPM packages for ScanCode, AttributeCode and TraceCode**. -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Consider also including an AppImage.org package. If you think this may not fill in a full three -months project, consider also adding some extras such as submitting the packages to Debian and -Fedora. - - -- **URLS** : - - * https://github.com/nexB/scancode-toolkit/issues/487 - * https://github.com/nexB/scancode-toolkit/issues/469 - -- **Mentor** : - - * @pombredanne https://github.com/pombredanne - -**AboutCode Manager test suite and Ci** : -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Create an extensive test suite for the Electron app and setup the CI to run unit, integration and -smoke tests on Ci for Windows, Linux and Mac. - - -- **URLS** : - - * https://github.com/nexB/aboutcode-manager - -- **Mentors** : - - * @jdaguil https://github.com/jdaguil - * @pombredanne https://github.com/pombredanne - -**DependentCode** : -^^^^^^^^^^^^^^^^^^^^^^ - -Create a tool for mostly universal package dependencies resolution. - - -- **URLS** : - - * https://github.com/nexB/dependentcode - -- **Mentors** : - - * @pombredanne https://github.com/pombredanne - -**FetchCode** : -^^^^^^^^^^^^^^^^^^ - -Create a tool for mostly universal package and code download from VCS, web, ftp, etc. - - -- **Mentors** : - - * @pombredanne https://github.com/pombredanne diff --git a/docs/source/gsoc/gsoc_2018.rst b/docs/source/gsoc/gsoc_2018.rst deleted file mode 100644 index 26c3a426..00000000 --- a/docs/source/gsoc/gsoc_2018.rst +++ /dev/null @@ -1,4 +0,0 @@ -Google Summer of Code 2018 -========================== - -See :ref:`contributor_project_ideas`. diff --git a/docs/source/gsoc/gsoc_2019.rst b/docs/source/gsoc/gsoc_2019.rst deleted file mode 100644 index b82a9a9a..00000000 --- a/docs/source/gsoc/gsoc_2019.rst +++ /dev/null @@ -1,712 +0,0 @@ -.. _GSoC2019: - -Google Summer of Code 2019 -========================== - -AboutCode is participating in the Google Summer of Code in 2019 as a mentoring org. This page -contain all the information for students and anyone else interested in helping. - -AboutCode is a family of FOSS projects to uncover data ... about software code: - - -- where does the code come from? which software package? -- what is its license? copyright? -- is the code secure, maintained, well coded? - -All these are questions that are important to answer: there are million -of free and open source software components available on the web for reuse. - -Knowing where a software package comes from, what is its license and if it is -vulnerable and what's its licensing should be a problem of the past such that -everyone can safely consume more free and open source software. - -*Join us to make it so!* - -Our tools are used to help detect and report the origin and license of source -code, packages and binaries as well as discover software and package -dependencies, and in the future track security vulnerabilities, bugs and other -important software package attributes. This is a suite of command line tools, -web-based and API servers and desktop applications. - -Table of Contents ------------------ - -- :ref:`aboutcode_projects_are_gsoc19` -- :ref:`contact_gsoc19` -- :ref:`technology_gsoc19` -- :ref:`skills_gsoc19` -- :ref:`about_your_project_application_gsoc19` -- :ref:`our_project_ideas_gsoc19` - - * :ref:`improve_copyright_detection_accuracy_and_speed_in_scancode` - * :ref:`port_scancode_to_python_3_gsoc_2019` - * :ref:`improve_programming_language_detection_and_classification_in_scancode` - * :ref:`improve_license_detection_accuracy_and_speed_in_scancode` - * :ref:`improve_scancode_scan_summarization_and_deduction` - * :ref:`create_linux_distros_and_freebsd_packages_for_scancode` - * :ref:`deltacode_projects` - * :ref:`tracecode_projects` - * :ref:`conan_and_other_projects` - -- :ref:`mentoring_gsoc19` - -.. _aboutcode_projects_are_gsoc19: - -AboutCode projects are... -------------------------- - -- `ScanCode Toolkit `_ is a popular - command line tool to scan code for licenses, copyrights and packages, used by - many organizations and FOSS projects, small and large. - -- `Scancode Workbench `_ - (formerly AboutCode Manager) is a JavaScript, Electron-based desktop - application to review scan results and document your origin and license - conclusions. - -- `AboutCode Toolkit `_ is a - command line tool to document and inventory known packages and licenses - and generate attribution docs, typically using the results of analyzed - and reviewed scans. - -- `TraceCode Toolkit `_ is - a command line tool to find which source code file is used to create a - compiled binary and trace and graph builds. - -- `DeltaCode `_ is a command line - tool to compare scans and determine if and where there are material - differences that affect licensing. - -- `ConAn `_ : a command line tool to analyze - the code in Docker and container images - -- `VulnerableCode `_ : an emerging - server-side application to collect and track known package vulnerabilities. - -- `license-expression `_ : a library - to parse, analyze, simplify and render boolean license expression (such as SPDX) - -We also work closely, contribute and co-started several other orgs and projects: - - -- - `Package URL `_ which is an emerging - standard to reference software packages of all types with simple, readable and - concise URLs. - -- - `SPDX `_ aka. Software Package Data Exchange, a spec to - document the origin and licensing of packages. - -- - `ClearlyDefined `_ to review and help FOSS - projects improve their licensing and documentation clarity. - -.. _contact_gsoc19: - -Contact -------- - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/discuss -Introduce yourself and start the discussion! - -For personal issues, you can contact the primary org admin directly: -@pombredanne and pombredanne@gmail.com - -Please ask questions the smart way: http://www.catb.org/~esr/faqs/smart-questions.html - -.. _technology_gsoc19: - -Technology ----------- - -Discovering the origin of code is a vast topic. We primarily use Python for this -and some C/C++ (and eventually some Rust and Go) for performance sensitive code and -Electron/JavaScript for GUI. - -Our domain includes text analysis and processing (for instance for copyrights -and licenses detection), parsing (for package manifest formats), binary analysis -(to detect the origin and license of binaries, which source code they come from, -etc.) as well as web based tools and APIs (to expose the tools and libraries as -web services) and low-level data structures for efficient matching (such as Aho- -Corasick and other automata). - -.. _skills_gsoc19: - -Skills ------- - -Incoming students will need the following skills: - - -- Intermediate to strong Python programming. For some projects, strong C/C++ and/or Rust is needed - too. -- Familiarity with git as a version control system -- Ability to set up your own development environment -- An interest in FOSS licensing and software code and origin analysis - -We are happy to help you get up to speed, but the more you are able to demonstrate ability and -skills in advance, the more likely we are to choose your application! - -.. _about_your_project_application_gsoc19: - -About your project application ------------------------------- - -We expect your application to be in the range of 1000 words. Anything less than -that will probably not contain enough information for us to determine whether -you are the right person for the job. Your proposal should contain at least the -following information, plus anything you think is relevant: - - -- Your name -- Title of your proposal -- Abstract of your proposal - -- Detailed description of your idea including explanation on why is it - innovative and what it will contribute to the project - - - * hint: explain your data structures and you planned main processing flows in - details. - -- Description of previous work, existing solutions (links to prototypes, - bibliography are more than welcome) - -- Mention the details of your academic studies, any previous work, internships - -- Relevant skills that will help you to achieve the goal (programming languages, - frameworks)? - -- Any previous open-source projects (or even previous GSoC) you have contributed - to and links. - -- Do you plan to have any other commitments during GSoC that may affect your - work? Any vacations/holidays? Will you be available full time to work on your - project? (Hint: do not bother applying if this is not a serious full time - commitment during the GSoC time frame) - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/discuss -introduce yourself and start the discussion! - -The best way to demonstrate your capability would be to submit a small patch -ahead of the project selection for an existing issue or a new issue. - -We will **always** consider and prefer a project submissions where you have -submitted a patch over any other submission without a patch. - -You can pick any project idea from the list below. If you have other ideas that -are not in this list, contact the team first to make sure it makes sense. - -.. _our_project_ideas_gsoc19: - -Our Project ideas ------------------ - -Here is a list of candidate project ideas for your consideration. Your own ideas -are welcomed too! Please chat about them to increase your chances of success! - -ScanCode ideas -############## - -.. _improve_copyright_detection_accuracy_and_speed_in_scancode: - -**Improve Copyright detection accuracy and speed in ScanCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Copyright detection is reasonably good by the slowest scanner in ScanCode. -It is based on NLTK part of speech (PoS) tagging and a copyright grammar. -The exact start and end lines where a copyright is found are approximate. - -The goal of this project is to refactor Copyright detection for speed and -simplicity possibly implementing a new parser (PEG?, etc.) or re-implementing -core elements in Rust with a Python binding for speed or using a fork of NLTK -or any other tool to be faster and more accurate. - -This would include also keeping track of line numbers and offsets where -copyrights are found. - -Also we detect copyrights that are part of a standard license text (e.g. FSF -copyright in a GPL text) and we should be able to filter these out. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, Rust, Go? - -- **URLS** - - * https://github.com/nexB/scancode-toolkit/tree/develop/src/cluecode - -- **Mentors** - - * @JonoYang https://github.com/JonoYang - -.. _port_scancode_to_python_3_gsoc_2019: - -**Port ScanCode to Python 3** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -ScanCode runs only on Python 2.7 today. The goal of this project is to port -ScanCode to support both Python 2 and Python 3. - - -- **Level** - - * Intermediate to Advanced - -- **Tech** - - * Python, C/C++, Go (for native code) - -- **URLS** - - * https://github.com/nexB/scancode-toolkit/issues/295 - -- **Mentors** - - * @steven-esser https://github.com/steven-esser - -.. _improve_programming_language_detection_and_classification_in_scancode: - -**Improve Programming language detection and classification in ScanCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -ScanCode programming language detection is not as accurate as it could be and -this is important to get this right to drive further automation. We also need to -automatically classify each file in facets when possible. - -The goal of this project is to improve the quality of programming language -detection (which is using only Pygments today and could use another tool, e.g. -some Bayesian classifier like Github linguist, enry ?). And to create and -implement a flexible framework of rules to automate assigning files to facets -which could use some machine learning and classifier. - - -- **Level** - - * Intermediate to Advanced - -- **Tech** - - * Python - -- **URLS** - - * https://github.com/nexB/scancode-toolkit/issues/426 - * https://github.com/nexB/scancode-toolkit/issues/1012 - * https://github.com/nexB/scancode-toolkit/issues/1036 - -- **Mentors** - - * @pombredanne https://github.com/pombredanne - -.. _improve_license_detection_accuracy_and_speed_in_scancode: - -**Improve License detection accuracy and speed in ScanCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -ScanCode license detection is using a sophisticated set of techniques -base on automatons, inverted indexes and sequence matching. There are some cases -where license detection accuracy could be improved (such as when scanning long -notices). Other improvements would be welcomed to ensure the proper detected -license text is collected in an improved way. Dealing with large files sometimes -trigger a timeout and handling these cases would be needed too (by breaking files -in chunks). The detection speed could also be improved possibly by porting -some critical code sections to C or Rust and that would need extensive profiling. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, C/C++, Rust, Go - -- **Mentors** - - * @mjherzog https://github.com/mjherzog - * @pombredanne https://github.com/pombredanne - -.. _improve_scancode_scan_summarization_and_deduction: - -**Improve ScanCode scan summarization and deduction** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The goal of this project is to take existing scan results and infer -summaries and perform some deduction of license and origin at a higher level, -such as the licensing or origin of a whole directory tree. The ultimate goal is -to automate the conclusion of a license and origin based on scans. -This could include using statistics and machine learning techniques such as -classifiers where relevant and efficient. - -This should be implemented as a set of ScanCode plugins and further the -summarycode module plugins. - - -- **Level** - - * Advanced - -- **Tech** - - * Python (Rust and Go welcomed too) - -- **URLS** - - * https://github.com/nexB/scancode-toolkit/issues/426 - * https://github.com/nexB/scancode-toolkit/issues/377 - -- **Mentors** - - * @pombredanne https://github.com/pombredanne - * @JonoYang https://github.com/JonoYang - -.. _create_linux_distros_and_freebsd_packages_for_scancode: - -**Create Linux distros and FreeBSD packages for ScanCode**. -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The goal of this project is to ensure that we have proper packages for Linux -distros and FreeBSD for ScanCode. - -The first step is to debundle pre-built binaries that exist in ScanCode such -that they come either from system-packages or pre-built Python wheels. This -covers libarchive, libmagic and a few other native libraries and has been -recently completed. - -The next step is to ensure that all the dependencies from ScanCode are also -available as distro packages. - -The last step is to create proper distro packages for RPM, Debian, FreeBSD and -as many other distros such as Nix and GUIX, Alpine, Arch and Gentoo (and possibly -also AppImage.org packages and Docker images) and submit these package to the distros. - -As a bonus, the same could then be done for AboutCode toolkit and TraceCode. - -This requires a good understanding of packaging and Python. - - -- **Level** - - * Intermediate to Advanced - -- **Tech** - - * Python, Linux, C/C++ for native code - -- **URLS** - - * https://github.com/nexB/scancode-toolkit/issues/487 - * https://github.com/nexB/scancode-toolkit/issues/469 - -- **Mentor** - - * @pombredanne https://github.com/pombredanne - -.. _deltacode_projects: - -DeltaCode projects -################## - -**Approximately Similar file detection in DeltaCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -DeltaCode is a tool to compare and report scan differences. -When comparing files, it only uses exact comparison. -The goal of this project is to improve the usefulness of the delta by also -finding files that are mostly the same (e.g. quasi or nrea duplicates) vs. files -that are completely different. Then the DeltaCode comparison core should be -updated accordingly to detect and report material changes to scans (such as -new, update or removed licenses, origins and packages) when changes are also -meterial in the code files (e.g. such that small changes may be ignored) - - -- **Level** - - * Intermediate to Advanced - -- **Tech** - - * Python - -- **URLS** - - * https://github.com/nexB/deltacode/ - -- **Mentors** - - * @steven-esser https://github.com/steven-esser - * @johnmhoran https://github.com/johnmhoran - -.. _tracecode_projects: - -TraceCode projects -################## - -**Static analysis of binaries for build tracing in TraceCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -TraceCode does system call tracing only today. The primary goal of this project -is to create a tool that provides the same results as the strace-based tracing -but would be using using ELF symbols, DWARF debug symbols, signatures or string -matching to determine when and how a source code file is built in a binary -using only a static analysis. The primary target should be Linux executables, -though the code should be designed to be extensible to Windows PE and macOS Dylib and exes. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, Linux, ELFs, DWARFs, symbols, reversing - -- **URLS** - - * https://github.com/nexB/tracecode-toolkit for the existing non-static tool - * https://github.com/nexB/scancode-toolkit-contrib for some work in progress on binaries/symbols - parsers/extractors - -- **Mentor** - - * @pombredanne https://github.com/pombredanne - -**Improve dynamic build tracing in TraceCode** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -TraceCode does system call tracing and relies on kernel-space system calls and -in particular tracing file descriptors. This project should improve the tracing -of the lifecycle of file descriptors when tracing a build with strace. We need -to improve how TraceCode does system call tracing by improving the way we track -open/close file descriptors in the trace to reconstruct the lifecycle of a traced -file. This requires to understand and dive if the essence of system calls and -file lifecycle from a kernel point of view and build datastructure and code -to reconstruct user-space file activity from the kernel traces along a timeline. - -This project also would cover updating TraceCode to use the Click command -line toolkit (like for ScanCode). - - -- **Level** - - * Advanced - -- **Tech** - - * Python, Linux kernel, system calls - -- **URLS** - - * https://github.com/nexB/tracecode-toolkit for the existing non-static tool - * https://github.com/nexB/scancode-toolkit-contrib for the work in progress on binaries/symbols - parsers/extractors - -- **Mentor** - - * @pombredanne https://github.com/pombredanne - -.. _conan_and_other_projects: - -Conan and Other projects -######################## - -**Containers and VM images static package analysis** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The goal of this project is to further the Conan container static analysis tool -to effectively support proper inventory of installed packages without running -the containers. - -This includes determining which packages are installed in Docker layers for -RPMs, Debian or Alpine Linux in a static way. And this may eventually require -the integration with ScanCode. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, Go, containers, distro package managers, RPM, Debian, Alpine - -- **URLS** - - * https://github.com/nexB/conan - -- **Mentor** - - * @JonoYang https://github.com/JonoYang - -**DependentCode: a mostly universal Package dependencies resolver** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The goal of this project is to create a tool for a universal package -dependencies resolution using a SAT solver that should leverage the detected -packages from ScanCode and the Package URLs and could provide a good enough way -to resolve package dependencies for many system and application package formats. -This is a green field project. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, C/C++, Rust, SAT - -- **URLS** - - * https://github.com/package-url - * https://fosdem.org/2018/schedule/event/purl/ - * https://github.com/heremaps/oss-review-toolkit - -- **Mentors** - - * @pombredanne https://github.com/pombredanne - -**VulnerableCode Package security vulnerability correlated data feed** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This project is to futher and evolve the VulnerableCode server and software -package vulnerabilities data aggregator. - -VulnerableCode was started as a GSoC project in 2017. Its goal is to -collect, aggregate and correlate vulnerabilities data and provide semi-automatic -correlation. In the end it should provide the basis to report vulnerabilities -alerts found in packages identified by ScanCode. - -This is not trivial as there are several gaps in the CVE data and how they -relate to packages as they are detected by ScanCode or else. - -The features and TODO for this updated server would be: - - -- Aggregate more and new packages vulnerabilities feeds, - -- Automating correlation: add smart relationship detection to infer new - relatiosnhips between available packages and vulnerabilities from mining the - graph of existing relations. - -- Create a ScanCode plugin to report vulnerabilities with detected packages - using this data. - -- Integrate API lookup on the server withe the AboutCode Manager UI - -- Create a UI and model for community curation of vulnerability to package - mappings, correlations and enhancements. - - -- **Level** - - * Advanced - -- **Tech** - - * Python, Django - -- **URLS** - - * https://github.com/nexB/vulnerablecode - * https://github.com/nexB/aboutcode-manager - * https://github.com/nexB/scancode-toolkit - * Other interesting pointers: - - * https://github.com/cve-search/cve-search - * https://github.com/jeremylong/DependencyCheck/ - * https://github.com/victims/victims-cve-db - * https://github.com/rubysec/ruby-advisory-db - * https://github.com/future-architect/vuls - * https://github.com/coreos/clair - * https://github.com/anchore/anchore/ - * https://github.com/pyupio/safety-db - * https://github.com/RetireJS/retire.js - * and many more including Linux distro feeds - -- **Mentors** - - * @steven-esser https://github.com/steven-esser - * @JonoYang https://github.com/JonoYang - -**High volume matching automatons and data structures** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Finding similar code is a way to detect the origin of code against an index of -open source code. - -To enable this, we need to research and create efficient and compact data -structures that are specialized for the type of data we lookup. Given the volume -to consider (typically multi billion values indexed) there are special -considerations to have compact and memory efficient dedicated structures (rather -than using a general purpose DB or Key/value pair store) that includes looking -at automata, and memory mapping. This types of data structures should be -implemented in Rust as a preference (though C/C++ is OK) and include Python -bindings. - -There are several areas to research and prototype such as: - - -- A data structure to match efficiently a batch of fix-width checksums (e.g. - SHA1) against a large index of such checksums, where each checksum points to - one or more files or packages. A possible direction is to use finite state - transducers, specialized B-tree indexes, blomm-like filters. Since when a - codebase is being matched there can be millions of lookups to do, the batch - matching is preferred. - -- A data structure to match efficiently a batch of fix-width byte strings (e.g. - LSH) against a large index of such LSH within a fixed hamming distance, where - each points to one or more files or packages. A possible direction is to use - finite state transducers (possibly weighted), specialized B-tree indexes or - multiple hash-like on-disk tables. - -- A memory-mapped Aho-Corasick automaton to build large batch tree matchers. - Available Aho-Corasick automatons may not have a Python binding or may not allow - memory-mapping (like pyahocorasick we use in ScanCode). The volume of files we - want to handle requires to reuse, extend or create specialized tree/paths - matching automatons that can handle eventually billions of nodes and are larger - than the available memory. A possible direction is to use finite state - transducers (possibly weighted). - -- Feature hashing research: we deal with many "features" and hashing to limit - the number and size of the each features seems to be a valuable thing. The goal - is to research the validaty of feature hashing with short hashes (15, 16 and - 32 bits) and evaluate if this leads to acceptable false-positive and loss of - accuracy in the context of the data structures mentioned above. - -Then using these data structures, the project should create a system for -matching code as a Python-based server exposing a simple API. -This is a green field project. - - -- **Level** - - * Advanced - -- **Tech** - - * Rust, Python - -- **URLS** - - * https://github.com/nexB/scancode-toolkit-contrib for samecode fingerprints drafts. - * https://github.com/nexB/scancode-toolkit for commoncode hashes - -- **Mentors** - - * @pombredanne https://github.com/pombredanne - -.. _mentoring_gsoc19: - -Mentoring ---------- - -We welcome new mentors to help with the program and require some good unerstanding of the project -codebase and domain to join as a mentor. Contact the team on Gitter. diff --git a/docs/source/gsod/gsod-2020.rst b/docs/source/gsod/gsod-2020.rst deleted file mode 100644 index 0b5eed0a..00000000 --- a/docs/source/gsod/gsod-2020.rst +++ /dev/null @@ -1,265 +0,0 @@ -.. _GSoD2020: - -=================================== -:index:`Google Season of Docs 2020` -=================================== - -AboutCode.org is applying to be a mentoring organization for the `Google Season of Docs (GSoD) 2020 `_. - -In the event that our application is accepted, we'll be looking for fellow members of the FOSS -community with technical writing skills and an interest in helping. If you're interested, keep -reading to get a sense of what we have in mind for GSoD 2020. - ----- - -.. contents:: Contents - :local: - :depth: 2 - ----- - -AboutCode Overview ------------------- - -AboutCode is a family of FOSS projects for Software Composition Analysis (SCA) and FOSS Compliance: - -* Where does code come from? -* What is its license? -* What is its origin and copyright? -* Is the code secure, maintained, well coded? - -All of these questions are important, and are relevant to millions of free and open source software -components available on the web for reuse. The answers are critical to ensure that anyone and -everyone can safely consume free and open source software. - -*Join us to make it so!* - -Our tools are used to detect and report the origin and license of source code, packages and -binaries as well as to discover software and package dependencies. We are also working on projects -to track security vulnerabilities, bugs and other important software package attributes. - -AboutCode Projects -~~~~~~~~~~~~~~~~~~ - -Our focus for GSoD 2020 is on `ScanCode Toolkit `_ -and `Scancode Workbench `_, -but proposals to improve the documentation for other AboutCode projects are welcome. - -See :ref:`aboutcode-project-overview` for a complete list of AboutCode projects. - -Technology -~~~~~~~~~~ - -We primarily use Python for AboutCode software. There is also some code in C/C++ or other -languages. We use Electron/JavaScript for the ScanCode Workbench desktop application. - -Our domain includes text analysis and processing (e.g., for copyright and license -detection), parsing (for package manifest formats), binary analysis (to detect the origin and -license of binaries), web-based tools and APIs (to expose the tools and libraries as web -services) and low-level data structures for efficient matching (such as Aho-Corasick and -other automata). - -For documentation we primarily use reStructured Text files with Sphinx and ReadTheDocs. -For command line tools we use text files for help documentation. -There is still some project documentation in project wikis, but our goal is to migrate -that to ReadTheDocs. -For each project there will also be a README and a few other files in the corresponding -GitHub repository. - -Contact Info -~~~~~~~~~~~~ - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/gsod-season-of-docs. -Introduce yourself and start the discussion! - -You can also contact: - -- the primary org admin directly: @pombredanne and pombredanne@gmail.com -- the GSoD coordinator directly: jmhoran@nexb.com - -Technical Writing Skills Needed -------------------------------- - -Incoming technical writers will need the following skills: - -- Ability to install and configure open source code from GitHub. -- Ability to understand and run programs from the command line in a terminal window. -- An interest in FOSS licensing and software code and origin analysis. - -During the application process and during a GSoD project with us you will learn how to: - -- Design and create documentation as Tutorials, HowTo Guides, Reference or Discussions. - - - See https://documentation.divio.com/introduction/ for more information about this approach. - -- Create and edit documentation files in reStructured Text. -- Manage documentation files with Sphinx. -- Publish documentation to ReadTheDocs. - -We are happy to help you get up to speed. The more you are able to demonstrate ability and -skills in advance, the more likely we are to choose your application! - -About Your Project Application ------------------------------- - -Your application should be in the range of 1,000 words, and should contain the following -information, plus anything else that you think is relevant: - -- Your name and contact details. - -- Title of your proposal. - -- Abstract of your proposal. - -- Description of your idea including an explanation of what it will contribute to the project. - -- Description of previous work, existing solutions, open-source projects, preferably with links. - -- Details of your academic studies and any previous internships. - -- Description of your relevant skills. - -- Do you plan to have any other commitments during GSoD that may affect your work? Any - vacations/holidays? Will you be available full time to work on your project? Please apply - only if this is a serious full time commitment during the GSoD time frame. - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/gsod-season-of-docs. -Introduce yourself and start a discussion! - -An excellent, competitive way to demonstrate your capability would be to submit a documentation -improvement to an AboutCode project, especially to ScanCode Toolkit or ScanCode Workbench. - -You can pick a project idea from the list below. You can also submit -`Your Documentation Project Ideas <#your-documentation-project-ideas-2020>`_. - -Our Documentation Project Ideas -------------------------------- - -Note that the AboutCode focus for GSoD 2020 is on ScanCode Toolkit and ScanCode Workbench, -although proposals to improve the documentation for other AboutCode projects are welcome. - -`Your Documentation Project Ideas <#your-documentation-project-ideas-2020>`_ are welcome too! Please -chat about them to increase your chances of success! - ----- - -Improve ScanCode Toolkit (SCTK) Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Upgrade the ScanCode Toolkit documentation on ReadTheDocs by streamlining and re-organizing it. -* Apply the Tutorial, HowTo, Reference and Discussions framework (or propose alternative). -* Implement process to sync documentation changes with releases. - -Note: Filter current SCTK GitHub Issues by Label 'documentation' to see many examples of specific -areas for improvement. - -- **Level** - - - Intermediate - -- **Tech** - - - Command line processing in a Linux-compatible terminal window - - GitHub, reStructured Text and Sphinx - -- **URLs** - - - https://scancode-toolkit.readthedocs.io/en/latest/index.html - - https://github.com/nexB/scancode-toolkit - - https://gitter.im/aboutcode-org/scancode - -- **Mentors** - - - https://github.com/johnmhoran - - https://github.com/mjherzog - - -Improve ScanCode Workbench (SCWB) Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Upgrade the ScanCode Workbench documentation on ReadTheDocs with a focus on Tutorials and - HowTo Guides because this is a GUI application. -* Apply the Tutorial, HowTo, Reference and Discussions framework (or propose alternative). -* Implement process to sync documentation changes with releases. - -Note: Filter current SCWB GitHub Issues by Label 'documentation' to see many examples of specific -areas for improvement. - -- **Level** - - - Intermediate - -- **Tech** - - - GUI design - - GitHub, reStructured Text and Sphinx - -- **URLs** - - - https://scancode-workbench.readthedocs.io/en/develop/index.html - - https://github.com/nexB/scancode-workbench - - https://gitter.im/aboutcode-org/scancode-workbench - -- **Mentors** - - - https://github.com/johnmhoran - - https://github.com/steven-esser - -Automate Code Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Identify tools to generate code documentation from Python files. -* Define coding standards necessary to support automated generation of code - documentation across AboutCode projects. -* Implement process and tools to automate publication of code documentation - for ScanCode Toolkit to ReadTheDocs using Sphinx and related extensions. - -- **Level** - - - Intermediate - -- **Tech** - - - Python programming - - GitHub, reStructured Text and Sphinx - -- **URLs** - - - https://scancode-workbench.readthedocs.io/en/develop/index.html - - https://github.com/nexB/scancode-workbench - - https://gitter.im/aboutcode-org/scancode-workbench - -- **Mentors** - - - https://github.com/johnmhoran - - https://github.com/steven-esser - ----- - -.. _your_documentation_project_ideas-2020: - -Your Documentation Project Ideas --------------------------------- - -Download and install `ScanCode Toolkit `_ -and `Scancode Workbench `_ and try them out. -For example, you -may try scanning an open source software package in a technology with which you are familiar. -What are the documentation weak points? - - -* Is it difficult to get started? A **Tutorial** documentation opportunity. -* Is it difficult to accomplish a specific objective? A **How-To** documentation opportunity. -* Are the capabilities of the tool too mysterious? Do you want to know more about what you can do - with it? A **Reference** documentation opportunity. -* Do you feel that you need to understand its concepts better in order to use it and trust it? Do - you want to know more about how the code scanning actually works? A **Discussion** documentation - opportunity. - -Feel free to propose and describe your own documentation ideas. - -Mentoring ---------- - -We welcome new mentors to help with the program. We require some understanding of the project -domain to join as a mentor. Contact the team on Gitter at https://gitter.im/aboutcode-org/gsod-season-of-docs diff --git a/docs/source/gsod/gsod-2021.rst b/docs/source/gsod/gsod-2021.rst deleted file mode 100644 index 10022ec6..00000000 --- a/docs/source/gsod/gsod-2021.rst +++ /dev/null @@ -1,234 +0,0 @@ -.. _GSoD2021: - -=================================== -:index:`Google Season of Docs 2021` -=================================== - -AboutCode.org is applying to be a mentoring organization for the `Google Season of Docs (GSoD) 2021 `_. - -In the event that our proposal is accepted, we'll be looking for fellow members of the FOSS -community with technical writing skills and an interest in helping. If you're interested, keep -reading to get a sense of what we have in mind for GSoD 2021. - ----- - -.. contents:: Contents - :local: - :depth: 2 - ----- - -AboutCode Overview ------------------- - -AboutCode.org is a community of developers behind a suite of Software Composition Analysis tools -(command line tools, web-based and API servers and desktop applications) and data for -license, origin and security: - -* Where does the code come from? -* What software packages are present? -* What is its license? copyright? -* Is the code secure, maintained, well coded? -* Are there any known vulnerabilities? - -All these questions are important, and are relevant to millions of free and open source -software components available on the web for reuse. The answers are critical to ensure that -everyone can safely consume free and open source software. - - -*Join us to make it so!* - - -AboutCode Projects -~~~~~~~~~~~~~~~~~~ - -Our focus for GSoD 2021 is to improve documentation on our flagship projects: - -- `ScanCode Toolkit `_ -- `ScanCode.io `_ -- `Vulnerablecode `_ - -And specifically to create/update developement and user guides. - -See :ref:`aboutcode-project-overview` for a complete list of AboutCode projects. - - -Technology -~~~~~~~~~~ - -We use Python for programming AboutCode software. Our projects are a combination of -command line tools, libraries, and web based applications based on Django and JavaScript. - -Our command line tools are designed to run on Linux, macOS and Windows, and -our web-based applications run only on Linux. - -We write our documentation using reStructuredText with Sphinx and ReadTheDocs. - -We use git and GitHub to store our code and track tasks and issues. - - -Technical Skills Needed ------------------------ - -Incoming technical writers will need the following skills: - -- Basic understanding of reStructuredText, Sphinx and ReadTheDocs-based documentation - toolchain including git. -- Ability to understand and run programs from the command line in a terminal. -- Ability to read well-structured Python code, and to update Python docstrings. -- Ability to manage Windows and Linux based installation of our tools if need be in - virtual machines. -- Ability to write technical documentation such as Tutorials, How-To Guides, and - API References. - -We do not expect a working knowledge of our tools as this will be picked from mentors and -community guidance. An interest in FOSS licensing and software code and origin -analysis would be welcome but is not a hard requirement. - -Some of these skills could be picked up during the project with help from your mentors, for -candidates with strong writing skills. - - -About Your Project Application ------------------------------- - -Check out the `GSoD Tech Writer Guide `_ - and `Statement Template `_. - -Your statement of interest should be in the range of 1,000 words, -and should contain the following information, plus anything else that -you think is relevant: - -- Personal information, i.e. Your name and contact details. - -- A Project statement. - -- A detailed description of your project. - -- Description of your relevant skills. - -- Professional information: Description of previous work, existing solutions, - open-source projects, preferably with links. - -- Proposed Timeline - -- Proposed total budget - -- Do you plan to have any other commitments during GSoD that may affect your work? Any - vacations/holidays? Will you be available full time to work on your project? Please apply - only if this is a serious full time commitment during the GSoD time frame. - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/gsod-season-of-docs. -Introduce yourself and start a discussion! - -An excellent, competitive way to demonstrate your capability would be to submit a documentation -improvement (small changes/typos are welcome but do not demonstrate your ability to write -documentation) to an AboutCode project, especially to ScanCode Toolkit, Scancode.io. - -Please select a project from the list below. - ----- - -Create and update scancode.io user and developer guides -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Create an HowTo guide for integrating third-party libraries into a ScanCode.io Pipeline. -* Add a tutorial for adding a new Pipeline with a third-party library. -* Extend the HowTo Guides to cover Software Composition Analysis workflows based on ScanCode.io. -* Upgrade the scancode.io Web UI documentation. -* Create an introductory `video `_ - to teach how the web UI is used. -* Update and improve the existing Pipe libraries reference API documentation (which is generated - from code documentation "docstrings"). -* Sync the new documentation set with the code to support continuous integration with code changes. - -- **Level** - - - Intermediate - -- **Tech** - - - Command line processing in a Linux-compatible terminal window - - Python [Django] - - GitHub, reStructured Text and Sphinx - -- **URLs** - - - https://scancodeio.readthedocs.io - - https://github.com/nexB/scancode.io - - https://gitter.im/aboutcode-org/discuss - -- **Mentors** - - - https://github.com/johnmhoran - - https://github.com/mjherzog - - https://github.com/tdruez - - https://github.com/AyanSinhaMahapatra - - -Create ScanCode-Toolkit Plugin Author Guide -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The goal of this guide is to provide an end-to-end tutorial and instruction -set on how to create scancode-toolkit plugins, with concrete examples and to -generate a reference API documentation for modules, classes and functions used to -create plugins. - -Some of the specific supported activities would be: - -* Explain the Plugin Architechture used in scancode-toolkit -* Explain the types of plugins used in scancode-toolkit - (pre-installed/seperately-installed) and (post-scan, pre-scan, scanner) -* An exhaustive list of all plugins and what they do, with links to their code. -* How to create a minimal plugin adding some functionality to scancode-toolkit. -* Useful examples of different types of plugins -* Using `nexb/skeleton `_ as a template for plugins. - -Note: Filter current scancode-toolkit GitHub Issues by Label 'documentation' to see -many examples of specific areas for improvement. - -- **Level** - - - Intermediate - -- **Tech** - - - Some lightweight python programming to create example plugins and - understand how they are constructed, the mentors will help here. - - Command line processing in a Linux-compatible terminal window - - GitHub, reStructured Text and Sphinx - -- **URLs** - - - https://scancode-toolkit.readthedocs.io - - https://github.com/nexB/scancode-toolkit - - https://gitter.im/aboutcode-org/scancode - -- **Mentors** - - - https://github.com/johnmhoran - - https://github.com/steven-esser - - https://github.com/AyanSinhaMahapatra - - ----- - -Mentoring ---------- - -We have an established team of contributors that are willing to mentor the selected -Tech Writers, and they have extensive experience from mentoring in the open, including -a successful previous participation in Google Season of Docs 2019. We have a developed -a gentle and efficient way to mentor our contributors to success. - - -Contact Info -~~~~~~~~~~~~ - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/gsod-season-of-docs. -Introduce yourself and start the discussion! - -You can also contact: - -- the primary org admin directly: @pombredanne and pombredanne@gmail.com -- the GSoD coordinator directly: jmhoran@nexb.com diff --git a/docs/source/gsod/gsod_2019.rst b/docs/source/gsod/gsod_2019.rst deleted file mode 100644 index 7f819f96..00000000 --- a/docs/source/gsod/gsod_2019.rst +++ /dev/null @@ -1,383 +0,0 @@ -.. _GSoD2019: - -Google Season of Docs 2019 -========================== - -AboutCode has been accepted as a participant in the Google Season of Documents in 2019 as a -mentoring org, and is looking for people with technical writing skills. This page contains -information for technical writers and anyone else interested in helping. - -AboutCode is a family of FOSS projects to uncover data about software code: - - -* Where does the code come from? which software package? -* What is its license? copyright? -* Is the code secure, maintained, well coded? - -All these questions are important, and are relevant to millions of free and open source software -components available on the web for reuse. The answers are critical to ensure that everyone can -safely consume free and open source software. - -*Join us to make it so!* - -Our tools are used to help detect and report the origin and license of source code, packages and -binaries as well as to discover software and package dependencies, and in the future track security -vulnerabilities, bugs and other important software package attributes. This is a suite of command -line tools, web-based and API servers and desktop applications. - -Table of Contents ------------------ - - -- :ref:`list_projects_aboutcode` -- :ref:`contact_gsod` -- :ref:`tech_gsod` -- :ref:`writing_skills_gsod` -- :ref:`application_gsod` -- :ref:`doc_project_ideas_gsod` -- :ref:`tutorial_ideas_gsod` - - * :ref:`scan_a_codebase_and_analyze_the_results` - -- :ref:`how_to_ideas_gsod` - - * :ref:`how_to_get_the_license_clarity_score_of_a_package` - * :ref:`how_to_discover_licensing_issues_in_a_software_project` - -- :ref:`referance_ideas_gsod` - - * :ref:`scancode_output_formats` - -- :ref:`discussion_ideas_gsod` - - * :ref:`integrating_scancode_into_a_software_development_lifecycle` - -- :ref:`your_documentation_project_ideas` -- :ref:`mentoring_gsod` - -.. _list_projects_aboutcode: - -List of AboutCode projects --------------------------- - -Note that the AboutCode focus for GSOD 2019 is on **ScanCode Toolkit** and **ScanCode Workbench**\, -although proposals to improve the documents of other AboutCode projects are welcome. - - -- `ScanCode Toolkit `_ is a popular command line tool to - scan code for licenses, copyrights and packages, used by many organizations and FOSS projects, - small and large. - -- `Scancode Workbench `_ (formerly AboutCode Manager) - is a JavaScript, Electron-based desktop application to review scan results and document your - origin and license conclusions. - -- Other AboutCode projects are described at https://www.aboutcode.org and - https://github.com/nexB/aboutcode - - We also work closely with, contribute to and have co-started several other orgs and projects: - -- `Package URL `_ is an emerging standard to reference software - packages of all types with simple, readable and concise URLs. - -- `SPDX `_ is the Software Package Data Exchange, a specification to document - the origin and licensing of software packages. - -- `ClearlyDefined `_ is a project to review FOSS software and help FOSS projects - to improve their licensing and documentation clarity. - -.. _contact_gsod: - -Contact -------- - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/discuss -Introduce yourself and start the discussion! - -For personal issues, you can contact the primary org admin directly: -@pombredanne and pombredanne@gmail.com - -or the GSOD coordinator directly at dmclark@nexb.com - -Please ask questions the smart way: http://www.catb.org/~esr/faqs/smart-questions.html - -.. _tech_gsod: - -Technology ----------- - -We primarily use Python (and some C/C++) for code analysis. We use Electron/JavaScript for GUI. - -Our domain includes text analysis and processing (for instance for copyright and license -detection), parsing (for package manifest formats), binary analysis (to detect the origin and -license of binaries, which source code they come from, etc.) as well as web based tools and APIs -(to expose the tools and libraries as web services) and low-level data structures for efficient -matching (such as Aho-Corasick and other automata). - -Our documentation is provided in text files that support the help function of our command line -tools. We also have begun to provide documentation in the Wiki section of some AboutCode projects. - -.. _writing_skills_gsod: - -Technical Writing Skills Needed -------------------------------- - -Incoming technical writers will need the following skills: - - -* Ability to install and configure open source code from GitHub. -* Ability to understand and run programs from the command line in a terminal window. -* Familiarity with the four document functions described at - https://www.divio.com/blog/documentation/ -* Ability to create and edit wiki pages with multiple markdown languages. -* An interest in FOSS licensing and software code and origin analysis. - -We are happy to help you get up to speed, and the more you are able to demonstrate ability and -skills in advance, the more likely we are to choose your application! - -.. _application_gsod: - -About your project application ------------------------------- - -Your application should be in the range of 1000 words, and should contain the following -information, plus anything else that you think is relevant: - - -- Your name and contact details - -- Title of your proposal - -- Abstract of your proposal - -- Description of your idea including an explanation of what it will contribute to the project, - such as the software development life cycle requirements that you expect to help with the - documentation improvements. - -- Description of previous work, existing solutions, open-source projects, preferably with links. - -- Details of your academic studies and any previous internships. - -- Descriptions of your relevant skills. - -- Do you plan to have any other commitments during GSOD that may affect your work? Any - vacations/holidays? Will you be available full time to work on your project? Please apply - only if this is a serious full time commitment during the GSOD time frame. - -Join the chat online or by IRC at https://gitter.im/aboutcode-org/discuss -Introduce yourself and start the discussion! - -An excellent, competitive way to demonstrate your capability would be to submit a documentation -improvement to an AboutCode project, especially to ScanCode Toolkit or ScanCode Workbench. - -You can pick any project idea from the list below. You can also submit -`Your Documentation Project ideas <#your-documentation-project-ideas>`_. - -.. _doc_project_ideas_gsod: - -Our Documentation Project ideas -------------------------------- - -Here is a list of candidate project ideas for your consideration, organized by documentation -function: **Tutorial** , **How-To** , **Reference** , **Discussion**. - -Note that the AboutCode focus for GSOD 2019 is on ScanCode Toolkit and ScanCode Workbench, -although proposals to improve the documents of other AboutCode projects are welcome. - -`Your Documentation Project ideas <#your-documentation-project-ideas>`_ are welcome too! Please -chat about them to increase your chances of success! - -.. _tutorial_ideas_gsod: - -Tutorial ideas --------------- - -.. _scan_a_codebase_and_analyze_the_results: - -**Scan a Codebase and Analyze the Results** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Provide specific instructions to guide a new user to: - - -* Scan a somewhat complex sample codebase using scancode-toolkit. -* Import the results into ScanCode Workbench. -* Analyze the scan results. - - - * **Level** - - * Intermediate - - * **Tech** - - * Command line processing in a Linux-compatible terminal window - - * **URLS** - - * https://github.com/nexB/scancode-toolkit/blob/develop/README.rst - * https://github.com/nexB/scancode-toolkit/wiki - * https://github.com/nexB/scancode-workbench/blob/develop/README.md - * https://github.com/nexB/scancode-workbench/wiki - - * **Mentors** - - * https://github.com/DennisClark - -.. _how_to_ideas_gsod: - -How-To ideas ------------- - -.. _how_to_get_the_license_clarity_score_of_a_package: - -**How To Get the License Clarity Score of a Package** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Explain the recommended scancode-toolkit options to get a license clarity score. - - -* **Level** - - * Intermediate - -* **Tech** - - * Command line processing in a Linux-compatible terminal window - -* **URLS** - - * https://github.com/nexB/scancode-toolkit/blob/develop/README.rst - * https://github.com/nexB/scancode-toolkit/wiki - * https://github.com/nexB/scancode-workbench/blob/develop/README.md - * https://github.com/nexB/scancode-workbench/wiki - -* **Mentors** - - * https://github.com/DennisClark - -.. _how_to_discover_licensing_issues_in_a_software_project: - -**How To Discover Licensing Issues in a Software Project** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - -* Explain the recommended scancode-toolkit options to discover licenses. -* Explain how to take advantage of license policy support. - - * **Level** - - * Intermediate - - * **Tech** - - * Command line processing in a Linux-compatible terminal window - - * **URLS** - - * https://github.com/nexB/scancode-toolkit/blob/develop/README.rst - * https://github.com/nexB/scancode-toolkit/wiki - * https://github.com/nexB/scancode-workbench/blob/develop/README.md - * https://github.com/nexB/scancode-workbench/wiki - - * **Mentors** - - * https://github.com/DennisClark - -.. _referance_ideas_gsod: - -Reference ideas ---------------- - -.. _scancode_output_formats: - -**ScanCode Output Formats** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Explain the various ScanCode output formats and their business purposes. - - -* **Level** - - * Intermediate - -* **Tech** - - * Command line processing in a Linux-compatible terminal window - -* **URLS** - - * https://github.com/nexB/scancode-toolkit/blob/develop/README.rst - * https://github.com/nexB/scancode-toolkit/wiki - * https://github.com/nexB/scancode-workbench/blob/develop/README.md - * https://github.com/nexB/scancode-workbench/wiki - -* **Mentors** - - * https://github.com/DennisClark - -.. _discussion_ideas_gsod: - -Discussion ideas ----------------- - -.. _integrating_scancode_into_a_software_development_lifecycle: - -**Integrating ScanCode into a Software Development Lifecycle** -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Discuss options and techniques to integrate ScanCode into a software development lifecycle -workflow: - - -* During software creation and maintenance. -* During software check-out/check-in. -* During sofware build and test. - - - * **Level** - - * Intermediate - - * **Tech** - - * Command line processing in a Linux-compatible terminal window - - * **URLS** - - * https://github.com/nexB/scancode-toolkit/blob/develop/README.rst - * https://github.com/nexB/scancode-toolkit/wiki - * https://github.com/nexB/scancode-workbench/blob/develop/README.md - * https://github.com/nexB/scancode-workbench/wiki - - * **Mentors** - - * https://github.com/DennisClark - -.. _your_documentation_project_ideas: - -Your Documentation Project ideas --------------------------------- - -Download and install ScanCode Toolkit and ScanCode Workbench and try them out. For example, you -may try scanning an open source software package in a technology with which you are familiar. -What are the documentation weak points? - - -* Is it difficult to get started? A **Tutorial** document opportunity. -* Is it difficult to accomplish a specific objective? A **How-To** document opportunity. -* Are the capabilities of the tool too mysterious? Do you want to know more about what you can do - with it? A **Reference** document opportunity. -* Do you feel that you need to understand its concepts better in order to use it and trust it? Do - you want to know more about how the code scanning actually works? A **Discussion** document - opportunity. - -Feel free to propose and describe your own documentation ideas. - -.. _mentoring_gsod: - -Mentoring ---------- - -We welcome new mentors to help with the program. We require some understanding of the project -domain to join as a mentor. Contact the team on Gitter at https://gitter.im/aboutcode-org/discuss diff --git a/docs/source/gsod/gsod_2019_report.rst b/docs/source/gsod/gsod_2019_report.rst deleted file mode 100644 index 1f189756..00000000 --- a/docs/source/gsod/gsod_2019_report.rst +++ /dev/null @@ -1,103 +0,0 @@ -Google Season of Docs 2019 Report -================================= - -Project Name ------------- - -The Project Name was **"Reference for Command Line Options in scancode-toolkit and Reorganize the -structure of AboutCode documentation at aboutcode.readthedocs.io"** - -About Me --------- - -| Name - Ayan Sinha Mahapatra -| Email : ayansmahapatra@gmail.com -| Country : India -| Links : - -- `Github `_ -- `Twitter `_ -- `Website `_ -- `Resume `_ - -Important Links for this GSoD Project -------------------------------------- - -- `GSoD Report `_ -- `GSoD Proposal `_ -- `ScanCode Toolkit Docs `_ -- `Aboutcode Docs `_ - ----- - -Project Overview ----------------- - -#. **Restructured Documentation hosted at ReadTheDocs:** Outdated Documentation from GitHub Wikis - was updated, improved and converted to .RST, to be hosted at aboutcode.readthedocs.io. - -#. **Documentation Tests and Roadmaps:** Documentation tests were added to check Documentation - Builds, Internal/External Links, and Style Standards. These tests were added in both aboutcode - and scancode-toolkit and already existing/new documentation was made compliant of these tests. - -#. **Command Line Options Referance:** The entire Command Line Options were listed, explained, and - their application elaborated from scratch. This is the main goal and the most important addition - to the scancode-toolkit documentation. - -#. **Migration of Scancode-Toolkit docs into its own Repository:** After adding the scancode toolkit - docs to aboutcode and reviews, this section was migrated to its own repository, to be - distributed with scancode-toolkit and with local builds enabled. - -#. **[WIP] Improve/Add to Existing Tutorials/How-Tos:** Added two major Tutorials and other - Tutorials/How-Tos were also updated to be consistent with the latest scancode versions and more - Tutorials/How-Tos can be added later to better help the users. - -#. **[WIP] Add Plugin Support:** A cookiecutter based plugin creation system has to be added, and - relevant docs for this process and for the plugin architecture has to be added. - -#. **[WIP] Solve Other Documentation Issues:** Solving multiple issues listed under the tag - "Documentation". - ----- - -Pull Requests -------------- - -PRs before GSoD Started @ nexB/aboutcode -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -- `[#17] `_ Adds Travis-CI for sphinx-build and linkcheck -- `[#15] `_ Adds Documentation from Wiki -- `[#13] `_ Adds GitHub wiki of Scancode-Toolkit and Scancode-Workbench - -GSoD PRs @ nexB/scancode-toolkit -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -- `[#1784] `_ Add Travis CI Test For Docs -- `[#1827] `_ Add New Issue and Pull Request Templates -- `[#1812] `_ Add Documentation from Aboutcode -- [WIP] Add Cookiecutter Plugin and Plugin Docs -- [WIP] `[#1847] `_ Fix remaining documentation Issues - -GSoD PRs @ nexB/aboutcode -^^^^^^^^^^^^^^^^^^^^^^^^^ - -- `[#26] `_ Adds Documentation Versioning, Command Line Options, Changelog -- `[#27] `_ CLI Options Reference for v3.1.1 -- `[#29] `_ Update Docs with changes in Wiki -- `[#30] `_ Add doc8 style checks to Travis-CI -- `[#31] `_ Fix template error -- `[#33] `_ Add final Changes/Improvements before Migrating -- `[#34] `_ Delete scancode-toolkit documentation -- `[#35] `_ Add GSoD 2019 Report - ----- - -Acknowledgments ---------------- - -I would like to thank `Dennis Clark `_, `Philippe Ombredanne `_ and `Steven Esser `_ -for their help throughout my GSoD project, from helping me prepare my proposal and project to -constructive criticism, nit-picky reviews, conference calls to discuss project roadmaps and to -explain to me when I was stuck in any problem. Their support and positivity helped me through -this project and made me enjoy every bit of it.