Sign Up for Free

RunKit +

Try any Node.js package right in your browser

This is a playground to test code. It runs a full Node.js environment and already has all of npm’s 400,000 packages pre-installed, including nspider with all npm packages installed. Try it out:

var nspider = require("nspider")

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.

nspider v0.0.3

A Node.js Web Spider

nspider

Build Status

A simple Web Spider/Crawler Node.js package.

var nspider = require('nspider')();

/* create spider object */
var spider = new nspider.Spider();

/* fetch the content of 'http://nodejs.org' */
spider.fetch('http://nodejs.org', function (err, data) {
  if (err) {
    /* fetch error */
    console.log('fetchError: %s', err.toString());
    return;
  }

  if (data.contentType.type !== 'text' ||
      data.contentType.subtype !== 'html') {
    /* Not a HTML page */
    console.log('Invalid contentType: %j', data.contentType);
    return;
  }

  /* Print all <a> href attr */
  data.$('a').each(function (idx,  item) {
    console.log('Link[%d]: %s', idx, data.$(item).attr('href'));
  });
});

Installation

npm install nspider

Examples

  • examples/list_all_links.js - List all links.
  • examples/save_all_img.js - Save all images.

Build

git clone https://github.com/xiongjia/nspider.git
npm run build

Metadata

RunKit is a free, in-browser JavaScript dev environment for prototyping Node.js code, with every npm package installed. Sign up to share your code.
Sign Up for Free