Sign Up for Free

RunKit +

Try any Node.js package right in your browser

This is a playground to test code. It runs a full Node.js environment and already has all of npm’s 400,000 packages pre-installed, including parallel-wget with all npm packages installed. Try it out:

var parallelWget = require("parallel-wget")

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.

parallel-wget v0.1.0

Parallel wget (network retriever)

parallel-wget

Install

npm install -g git://github.com/chbrown/parallel-wget.git

Use

First, customize the url transform function:

var wget = require('parallel-wget');
wget.url2Filename = function(urlStr) {
  return urlStr.replace(/\//g, '-');
}

wget.downloadUrls('.', 50, urls_array, function() {
  logger.info('Done');
});

Or use from the command line like:

<allurls.txt parallel-wget -d tmp --concurrency 50

License

Copyright © 2011–2013 Christopher Brown. MIT Licensed.

RunKit is a free, in-browser JavaScript dev environment for prototyping Node.js code, with every npm package installed. Sign up to share your code.
Sign Up for Free