This is a
playground to test code. It runs a full
Node.js environment and already has all of
npm’s 400,000 packages pre-installed, including
aviation-scraper with all
npm packages installed. Try it out:
This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.
This project is going to be composed of few packages that are going to be scraping from wikipedia the following information:
This is intended for use of an updated database of information for a map application.
Before testing this application you have to run
make update-local-pages to download a local copy of the html pages required for tests.
Then run tests with
displays this help:
Usage: aviation-scraper [options] Retrieve airlines destinations and airports data. Multiple options are not allowed, only '-b' and '-s' options can be combined with the rest. Example for saving individual files to airports: $ aviation-scraper -a -s true Example for saving file to an specific directory: $./aviation-scraper -a -b './my_new_directory' Options: -h, --help output usage information -V, --version output the version number -l, --list List of airports with the link to the wikipedia page. (Saved to a single file) -d, --destinations Destinations of all the airlines listed on the wikipedia. (saved each airline with an individual JSON file) -a, --airports Saves the important data for each airport (saved in a single JSON file and optional for each airport) -c, --companies Saves all the airline links and all the important information for each airline. -b, --base <basedir> the base directory where to save the files generated (default: tmp) -s, --save <save> this options allows you to save individual files for each airline, airport or destination (default:false)
if you wish to contribute create your branch and create a pull request on the dev branch or you can create an issue.