Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
[DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); [DISCARDED] Add OTL education CSV reader script by Goziee-git · Pull Request #168 · creativecommons/quantifying · GitHub
Skip to content

[DISCARDED] Add OTL education CSV reader script - #168

Closed
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education
Closed

[DISCARDED] Add OTL education CSV reader script#168
Goziee-git wants to merge 2 commits into
creativecommons:mainfrom
Goziee-git:feature/OTL-education

Conversation

@Goziee-git

@Goziee-gitGoziee-git commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Fixes

Description

the OTL_education_read_csv.pyscript reads the pre-automation OTL.csv file and copies it to the current quarter's data directory for processing.

The script:
• Reads from pre-automation/education/datasets/OTL.csv
• Copies the data to data/{quarter}/1-fetch/otl_raw_data.csv
• Includes proper error handling and logging
• Uses the --enable-save flag to control whether data is actually saved
• Follows the project's established patterns for fetch phase scripts

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

Checklist

  • I have read and understood the Developer Certificate of Origin (DCO), below, which covers the contents of this pull request (PR).
  • My pull request doesn't include code or content generated with AI.
  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main or master).
  • My commit messages follow [best practices][best_practices].
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

@Goziee-git
Goziee-git requested review from a team as code ownersOctober 10, 2025 05:32
@Goziee-git
Goziee-git requested review from TimidRobot and possumbilities and removed request for a teamOctober 10, 2025 05:32
@cc-open-source-botcc-open-source-bot moved this to In review in TimidRobotOct 10, 2025

@TimidRobotTimidRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

@TimidRobotTimidRobot self-assigned this Oct 10, 2025
@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Thank you. I'ld refocus on the task on fetching data from sources using API since the sources in pre-automation/ are largely not valid data sources currently.

@TimidRobot

TimidRobot commented Oct 10, 2025

Copy link
Copy Markdown
Member

Also:

Technical Note: Script Execution Requirements

Issue: The OTL_education_read_csv.py script fails with ModuleNotFoundError: No module named 'shared' when run directly from the 1-fetch directory, necessitating the need for adjusting the need import structure. if import is adjusted, the pre-commit fails.

Root Cause: The script imports the shared module from the parent scripts directory, but Python cannot locate it without proper path configuration.

The scripts should be run from the root of the repository using pipenv run ./SCRIPT_PATH. For example:

pipenv run ./scripts/1-fetch/github_fetch.py -h

When run this way, the shared library (scripts/shared.py) provides easy access to all of the necessary paths and all of the modules managed by pipenv are available.

Thank you for highlighting this issue. I've addressed the missing documentation by adding the Running the scripts section to the repository README.

@Goziee-git

Copy link
Copy Markdown
ContributorAuthor

Fix PR description

Please correct the Checklist in the pull request (PR) description (use [x] where appropriate).

Relatedly it isn't appropriate to modify the template. The following change isn't welcome:

- <!-- Replace the [ ] with [x] to check the boxes (there is no space between x and square brackets). -->+ <!-- Replace the [ ] with [y/n] to check the boxes (there is no space between x and square brackets). -->

Query API

The goal of this project is to regularly query APIs. The pre-automation/education/datasets/OTL.csv file is not a valid fetch source.

Lowercase file names

Please use lowercase files names.

Hello @TimidRobot, I apologise for the delay here, i've had to read through your reviews to pin down exactly what the issue here was and i have made changes as requested for this PR. As regards the lower case for file names, I have adhered to that pattern for file naming in subsequent scripts submitted for your review other than the otl_fetch.py script because you mentioned that the OTL.csv data source is not a valid source. I'ld like to kindly ask that you please reopen the closed PR submitted as to give me a good chance to contributing to the Quantifying Creative Commons Project. I acknowledge my errors in this PR and i'll pay attention to it going forward. Thank You🙏🏼🙏🏼

@TimidRobotTimidRobot added the ⛔️ status: discarded Will not be worked on label Oct 17, 2025
@TimidRobotTimidRobot changed the title Add OTL education CSV reader script[Discarded] Add OTL education CSV reader scriptOct 17, 2025
@github-project-automationgithub-project-automationBot moved this from In review to Done in TimidRobotOct 17, 2025
@TimidRobotTimidRobot changed the title [Discarded] Add OTL education CSV reader script[DISCARDED] Add OTL education CSV reader scriptNov 4, 2025
@Goziee-git
Goziee-git deleted the feature/OTL-education branch November 5, 2025 07:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⛔️ status: discardedWill not be worked on

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Add OTL (Open Texbook Library) Education data source fetch script

3 participants

@Goziee-git@TimidRobot@cc-open-source-bot