A PHP Laravel Library for ScrapingBee
A PHP Laravel Package for ScrapingBee
You can install the package via composer:
composer require ziming/laravel-scrapingbeeYou can publish the config file with:
php artisan vendor:publish --provider="Ziming\LaravelScrapingBee\LaravelScrapingBeeServiceProvider" --tag="laravel-scrapingbee-config"This is the contents of the published config file:
return [
'api_key' => env('SCRAPINGBEE_API_KEY'),
'base_url' => env('SCRAPINGBEE_BASE_URL', 'https://app.scrapingbee.com/api/v1/'),
'timeout' => env('SCRAPINGBEE_TIMEOUT', 120),
];$scrapingBeeClient = Ziming\LaravelScrapingBee\LaravelScrapingBee::make();
$response = $scrapingBeeClient->blockAds()
->when(now()->weekOfMonth === 4, function (LaravelScrapingBee $scrapingBeeClient): LaravelScrapingBee {
// if it is last week of the month, spam premium proxy to use up credits!
return $scrapingBeeClient->premiumProxy();
})
->jsonResponse()
->jsScenario([
['click' => '#button_id'],
['wait' => 1000],
['wait_for' => '#slow_div'],
['scroll_x' => 1000],
['scroll_y' => 1000],
['fill' => ['#input_1','value_1']],
['evaluate' => 'console.log(window);'],
])->get('https://www.scrapingbee.com')Look at the source code of src/LaravelScrapingBee.php for the other methods (link below). Methods that return $this are chainable. An example is the blockAds() method you saw above. Meanwhile methods such as get(), post(), usageStatistics() returns you an Illuminate\Http\Client\Response object if no exceptions are thrown.
If for some reason you prefer to set all parameters at once you may wish to use the setParams() or addParams() method. Take note that these methods simply takes in an array and sets the parameters as is. So for the methods that does something extra before setting the parameter you would have to do them yourselves now if you chose this path.
An example is shown below:
$scrapingBeeClient = Ziming\LaravelScrapingBee\LaravelScrapingBee::make();
$response = $scrapingBeeClient->setParams([
'js_scenario' => json_encode([
'instructions' => [
['click' => '#button_id'],
['wait' => 1000],
['wait_for' => '#slow_div'],
['scroll_x' => 1000],
['scroll_y' => 1000],
['fill' => ['#input_1','value_1']],
['evaluate' => 'console.log(window);']
]
]),
'block_ads' => true,
'json_response' => true,
])->get('https://www.scrapingbee.com')This class is never tested by me. Feel free to let me know how it goes.
$googleSearchScrapingBeeClient = Ziming\LaravelScrapingBee\LaravelScrapingBeeGoogleSearch::make();
$response = $googleSearchScrapingBeeClient
->nbResults(8)
->page(1)
->search('scrapingbee')
->get();Look at the source code of src/LaravelScrapingBeeGoogleSearch.php for the other methods (link below).
LaravelScrapingBeeGoogleSearch.php
Currently there are no tests. But if there are tests in the future, you can run the command below to execute the testcases.
composer test1 way to contribute is to donate. The other alternative is to use my referral link to Scrapingbee & be a paying customer
As for contributing to this codebase, please see CONTRIBUTING for details.
Please review our security policy on how to report security vulnerabilities.
The MIT License (MIT). Please see License File for more information.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent