Sign Up for Free

RunKit +

Try any Node.js package right in your browser

This is a playground to test code. It runs a full Node.js environment and already has all of npm’s 400,000 packages pre-installed, including og-crawler with all npm packages installed. Try it out:

var ogCrawler = require("og-crawler")

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.

og-crawler v0.0.1

Crawls/scrapes a website for og data, so Facebook will show current images and descriptions when sharing.

#og-crawler

Often when sharing a link on Facebook, the provided image/description is outdated and not what you actually want to share. One method for refreshing this information is to take the URL and pass it through Facebook's official URL Debugger.

og-crawler will crawl your website and scrape it for og data using Facebook's Open Graph API so you don't have to manually run all of your site's URLs through the Debugger.

You can crawl your entire website, or just tell og-crawler to scrape a single page.

Installation

<sudo> npm i -g og-crawler

Usage

CLI

og_crawl http://google.com

Node.js

var OgCrawler = require('og-crawler');

new OgCrawler({
    depth: 3,
    maxParallel: 5,
    silent: false,
    url: 'http://google.com'
}).crawl();

Options

-d, --depth This will determine how deep the crawling will go. The larger the number, the more pages will be scraped. Set this to 1 to scrape only the url provided.

Default value: 3

-s, --silent Setting this to true will prevent any logging. It will always prevent og-crawler from asking if you'd like to create a log file at the end of the process.

Default value: false

-m, --max-parallel If your website is large, and it's taking a long time to scrape the og data, increase this number to make more parallel requests.

Default value: 5

Metadata

RunKit is a free, in-browser JavaScript dev environment for prototyping Node.js code, with every npm package installed. Sign up to share your code.
Sign Up for Free