crawlerdetect.io

About CrawlerDetect

CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent and http_from header. Currently able to detect 1,000's of bots/spiders/crawlers.

Installation

composer require jaybizzle/crawler-detect

Usage

use Jaybizzle\CrawlerDetect\CrawlerDetect;

$CrawlerDetect = new CrawlerDetect;

// Check the user agent of the current 'visitor'
if($CrawlerDetect->isCrawler()) {
    // true if crawler user agent detected
}

// Pass a user agent as a string
if($CrawlerDetect->isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')) {
    // true if crawler user agent detected
}

// Output the name of the bot that matched (if any)
echo $CrawlerDetect->getMatches();

Contributing

If you find a bot/spider/crawler user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the $data array in Fixtures/Crawlers.php and add the failing user agent to tests/crawlers.txt.

Failing that, just create an issue with the user agent you have found, and we'll take it from there :)

Laravel Package

If you would like to use this with Laravel, please see Laravel-Crawler-Detect

Symfony Bundle

To use this library with Symfony 2/3/4, check out the CrawlerDetectBundle.

YII2 Extension

To use this library with the YII2 framework, check out yii2-crawler-detect.

ES6 Library

To use this library with NodeJS or any ES6 application based, check out es6-crawler-detect.

Python Library

To use this library in a Python project, check out crawlerdetect.

JVM Library (written in Java)

To use this library in a JVM project (including Java, Scala, Kotlin, etc.), check out CrawlerDetect.

.NET Library

To use this library in a .net standard (including .net core) based project, check out NetCrawlerDetect.

Ruby Gem

To use this library with Ruby on Rails or any Ruby-based application, check out crawler_detect gem.

Go Module

To use this library with Go, check out the crawlerdetect module.

Parts of this class are based on the brilliant MobileDetect