BranchesTags

Latest commit

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py

Repository files navigation

Multidocument Summarization using SumBasic

Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.

The SumBasic algorithm is outlined in the following paper: Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research, Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf

SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document, determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences correspondingly.

The variations implemented were:

  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.

Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an positive integer corresponding to the cluster number, and B is another positive integer corresponding to the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international committee meeting dedicated to privacy in the UK.

Usage

python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>* is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.

About

Multidocument summarization using the SumBasic implementation

Topics

automatic-summarizationnatural-language-processingnltkpython3sumbasic

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Latest commit
History
7 Commits
Folders and files
NameName
Last commit message
Last commit date
docs
docs
README.md
README.md
sumbasic.py
sumbasic.py
Repository files navigation
Multidocument Summarization using SumBasic
Multidocument summarization using the SumBasic implementation. Uses Python3 and nltk.
The SumBasic algorithm is outlined in the following paper:
Ani Nenkova and Lucy Vanderwende. The Impact of Frequency on Summarization. Microsoft Research,
Redmond, Washington, Tech. Rep. MSR-TR-2005-101. 2005. https://www.cs.bgu.ac.il/~elhadad/nlp09/sumbasic.pdf
SumBasic is a multidocument summarization method that determines a frequency distribution of words over every document,
determines the average probability of each sentence based on its word composition, and takes the best scoring sentence of
the most frequent word until the desired summary length has been reached. A nonredundancy step every iteration updates the
probability of the most frequent word by multiplying it by itself and updating the assigned weights of the sentences
correspondingly.
The variations implemented were:
  1. orig: The original version, including the non-redundancy update of the word scores.
  2. best_avg: A version of the system that picks the sentence that has the highest average probability
in Step 2, skipping Step 3.
  3. simplified: A simplified version of the system that holds the word scores constant and does not
incorporate the non-redundancy update.
  4. leading, which takes the leading sentences of one of the articles, up until the word length limit is reached.
Each cluster of testing articles is stored in the docs/ folder. Each file follows docA-B.txt convention, where A is an
positive integer corresponding to the cluster number, and B is another positive integer corresponding to
the article number within that cluster. The testing clusters used were on articles about GM's plant closure in Oshawa, NASA InSight's
first photo taken on Mars, Paul Manafort's alleged secret meeting with Julian Assange, and Zuckerberg's no-show at an international
committee meeting dedicated to privacy in the UK.
Usage
python3 sumbasic.py <method_name> <file_n>* where <method_name> is one of orig, best_avg, simplified, or leading, and <file_n>*
is a regex expression of the file path of the cluster, e.g. docs/doc1-*.txt.
About
Multidocument summarization using the SumBasic implementation
Topics
automatic-summarizationnatural-language-processingnltkpython3sumbasic
Resources
Readme
Activity
Stars
0 stars
Watchers
1 watching
Forks
0 forks
Report repository
Releases
Packages
Contributors
Languages