class BrokenLinkChecker

A concurrent website crawler that finds broken links.

Examples

Example 1

const { successes } = await BrokenLinkChecker.create()
  .withConcurrency(20)
  .withTimeout(10_000)
  .keepBroken((r) => r.reason !== "Forbidden")
  .check(["https://example.com", "https://example.com/sitemap.xml"])
  .partition();

Constructors

Static Methods

Creates a new BrokenLinkChecker with all defaults.

Properties

private
readonly
config: ResolvedConfig

Methods

check(startUrls: (string | URL)[]): Stream<CheckResult, Error>

Start crawling from one or more entrypoints. Returns a stream of CheckResult. Use partition() to separate ok/broken links (successes) from stream errors (errors).

filterUrls(fn: (url: URL) => boolean): BrokenLinkChecker

Returns a new BrokenLinkChecker that filters URLs before checking them.

keepBroken(fn: (result: CheckResult) => boolean): BrokenLinkChecker

Returns a new BrokenLinkChecker that keeps broken links matching the predicate in results.

Returns a new BrokenLinkChecker with the given concurrency limit.

withFetch(fetch: globalThis.fetch): BrokenLinkChecker

Returns a new BrokenLinkChecker using the given fetch function.

Returns a new BrokenLinkChecker with the given log level.

Returns a new BrokenLinkChecker with retry options merged into existing ones.

Returns a new BrokenLinkChecker with the given request timeout in milliseconds.

Returns a new BrokenLinkChecker with the given User-Agent header.