Simple utility for scraping data from html tables on a given website into a list of javascript objects.
npm install --save table-scraper
Returns a promise that resolves to a list of tables found on the input website. HTML table rows are converted to javascript objects
For example: suppose the website at http://www.some-fake-url.com consisted of the following:
<html><head></head><body><table><thead><tr><th>State</th><th>Capital City</th><th>Pop.<th></tr></thead><tbody><tr><td>Minnesota</td><td>Saint Paul</td><td>3</td></tr><tr><td>New York</td><td>Albany</td><td>Eight Million</td></tr></tbody></table></body></html>The following code would result in the array displayed below:
varscraper=require('table-scraper');scraper.get('http://www.some-fake-url.com').then(function(tableData){/* tableData === [ [ { State: 'Minnesota', 'Capital City': 'Saint Paul', 'Pop.': '3' }, { State: 'New York', 'Capital City': 'Albany', 'Pop.': 'Eight Million' } ] ] */});Important to note: the tableData returned is a list of lists. So, if some-fake-url.com
contained three tables, the structure of the response would look like
[[/* list of data from the first table */],[/* list of data from the second table */],[/* list of data from the third table */]]If a table has NO headings (no <th> elements), the object keys are simply the column index:
[{'0': <firstcolumndataoffirstrow>, '1': <secondcolumndataoffirstrow>, .... }]Feedback/PRs welcome! Please include tests around any new functionality, and make sure existing tests pass:
npm test
The following node libraries make this utility super easy: