53

GitHub - transitive-bullshit/check-links: Robustly checks an array of URLs for l...

 6 years ago
source link: https://github.com/transitive-bullshit/check-links
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

readme.md

check-links

Robustly checks an array of URLs for liveness.

NPM Build Status JavaScript Style Guide

For each URL, it first attempts an HTTP HEAD request, and if that fails it will attempt an HTTP GET request, retrying several times by default with exponential falloff.

This module handles concurrency and retry logic so you can check the status of thousands of links quickly and robustly.

Install

This module requires node >= 4.

npm install --save check-links

Usage

const checkLinks = require('check-links')

const results = await checkLinks([
  'https://foo.com',
  'https://404.com',
])

results['https://foo.com'] // { status: 'alive', statusCode: 200 }
results['https://404.com'] // { status: 'dead', statusCode: 404 }

// example using a custom timeout and retry count
const results2 = await checkLinks([
  'https://foo.com',
  'https://404.com',
], {
  timeout: 30000,
  retry: 1
})
  • Supports HTTP and HTTPS urls.
  • Defaults to a 10 second timeout per HTTP request with 2 retries.
  • Defaults to a Mac OS Chrome user-agent.
  • Defaults to following redirects.

API

checkLinks

Robustly checks an array of URLs for liveness.

For each URL, it first attempts an HTTP HEAD request, and if that fails it will attempt an HTTP GET request, retrying several times by default with exponential falloff.

Returns a Map<String, LivenessResult> that maps each input URL to an object containing status and possibly statusCode.

LivenessResult.status will be one of the following:

  • alive if the URL is reachable (2XX status code)
  • dead if the URL is not reachable
  • invalid if the URL was parsed as invalid or used an unsupported protocol

LivenessResult.statusCode will contain an integer HTTP status code if that URL resolved properly.

Type: function (urls, opts)

  • urls array<string> Array of urls to test
  • opts object? Optional configuration options (any extra options are passed to got)
    • opts.concurrency number Maximum number of urls to resolve concurrently (optional, default 8)

Related

License

MIT © Travis Fischer


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK