Sign Up for Free

RunKit +

Try any Node.js package right in your browser

This is a playground to test code. It runs a full Node.js environment and already has all of npm’s 400,000 packages pre-installed, including es6-crawler-detect with all npm packages installed. Try it out:

var es6CrawlerDetect = require("es6-crawler-detect")

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.

es6-crawler-detect v3.1.1

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.



crawlerdetect.io

npm version contributions welcome

Version 3

The new version now do include a middleware for expressjs, please read the documentation for the latest updates.

About ES6-CrawlerDetect

This Library is an ES6 version of the original PHP class @CrawlerDetect, it helps you detect bots/crawlers and spiders only by scanning the user-agent string or from the global request.headers.

Installation

npm install es6-crawler-detect

Usage

ECMAScript 6 (ES6)

'use strict';

const express = require('express')
const {Crawler, middleware} = require('es6-crawler-detect/src')

const app = express()

app.get('your/route', function async (request, response) {
  // create a new Crawler instance
  var CrawlerDetector = new Crawler(request)
  
  // check the current visitor's useragent
  if ( CrawlerDetector.isCrawler() )
  {
    // true if crawler user agent detected
  }
  
  // or check a user agent string
  if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )
  {
    // true if crawler user agent detected
  }
  
  // Output the name of the bot that matched (if any)
  response.send(CrawlerDetector.getMatches())
})

/**
 * Or by using the middleware
*/
app.use(middleware)

app.get('/crawler', function async (request, response) {
  // or check a user agent string
  request.Crawler.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')
  
  // Output the name of the bot that matched (if any)
  response.send(request.Crawler.getMatches())
})

ECMAScript 5 (ES5)

<head>
    <!-- Metas -->
    <meta name="title" content="ES6-CrawlerDetect">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta name="description" content="">
    <meta name="keywords" content="">
    <title>CrawlerDetect - the web crawler detection library</title>
    <!-- Scripts -->
    <script type="text/javascript" src="./your/path/to/main.bundle.js"></script>
</head>
  // create a new Crawler instance
  var CrawlerDetector = new Crawler.Crawler();
  var userAgentString = navigator.userAgent;
  
  // check the current visitor's useragent
  if ( CrawlerDetector.isCrawler(userAgentString) )
  {
    // true if crawler user agent detected
  }
  
  // Output the name of the bot that matched (if any)
  console.debug(CrawlerDetector.getMatches());

Contributing

If you find a bot/spider/crawler user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the data array in ./crawler/crawlers.js.

RunKit is a free, in-browser JavaScript dev environment for prototyping Node.js code, with every npm package installed. Sign up to share your code.
Sign Up for Free