Giter Club home page Giter Club logo

laravel-slower's Introduction

Laravel Slower: Optimize Your DB Queries with AI

Latest Version on Packagist GitHub Tests Action Status GitHub Code Style Action Status Total Downloads

halilcosdu%2Flaravel-slower | Trendshift

Laravel Slower is a powerful package designed for Laravel developers who want to enhance the performance of their applications. It intelligently identifies slow database queries and leverages AI to suggest optimal indexing strategies and other performance improvements. Whether you're debugging or routinely monitoring your application, Laravel Slower provides actionable insights to streamline database interactions.

Installation

You can install the package via composer:

composer require halilcosdu/laravel-slower

You can publish the config file with:

php artisan vendor:publish --tag="slower-config"

This is the contents of the published config file:

You can disable AI recommendations by setting the ai_recommendation key to false in the config file. If you disable AI recommendations, the package will not make any API requests to OpenAI.

<?php

// config for HalilCosdu/Slower

use HalilCosdu\Slower\Models\SlowLog;

return [
    'enabled' => env('SLOWER_ENABLED', true),
    'threshold' => env('SLOWER_THRESHOLD', 10000),// Set null to capture all queries
    'resources' => [
        'table_name' => (new SlowLog)->getTable(),
        'model' => SlowLog::class,
    ],
    'ai_recommendation' => env('SLOWER_AI_RECOMMENDATION', true), // Set to false to disable AI recommendations
    'recommendation_model' => env('SLOWER_AI_RECOMMENDATION_MODEL', 'gpt-4'),
    'open_ai' => [
        'api_key' => env('OPENAI_API_KEY'),
        'organization' => env('OPENAI_ORGANIZATION'),
        'request_timeout' => env('OPENAI_TIMEOUT'),
    ],
    'prompt' => env('SLOWER_PROMPT', 'As a distinguished database optimization expert, your expertise is invaluable for refining SQL queries to achieve maximum efficiency. Please examine the SQL statement provided below. Based on your analysis, could you recommend sophisticated indexing techniques or query modifications that could significantly improve performance and scalability?'),
];

You can publish and run the migrations with:

php artisan vendor:publish --tag="slower-migrations"
php artisan migrate
public function up()
{
    Schema::create(config('slower.resources.table_name'), function (Blueprint $table) {
        $table->id();
        $table->boolean('is_analyzed')->default(false)->index();
        $table->longtext('bindings');
        $table->longtext('sql');
        $table->float('time')->nullable()->index();
        $table->string('connection');
        $table->string('connection_name')->nullable();
        $table->longtext('raw_sql');
        $table->longtext('recommendation')->nullable();

        $table->timestamps();
    });
}

public function down(): void
{
    Schema::dropIfExists(config('slower.resources.table_name'));
}

Usage

You can register the commands with your scheduler.

php artisan slower:clean /*{days=15}  Delete records older than 15 days.*/
php artisan slower:analyze /*Analyze the records where is_analyzed=false*/
    use HalilCosdu\Slower\Commands\AnalyzeQuery;
    use HalilCosdu\Slower\Commands\SlowLogCleaner;

    protected $commands = [
        AnalyzeQuery::class,
        SlowLogCleaner::class,
    ];

    /**
     * Define the application's command schedule.
     */
    protected function schedule(Schedule $schedule): void
    {
        $schedule->command(AnalyzeQuery::class)->runInBackground()->daily();
        $schedule->command(SlowLogCleaner::class)->runInBackground()->daily();
    }
$model = \HalilCosdu\Slower\Models\SlowLog::first();

\HalilCosdu\Slower\Facades\Slower::analyze($model): Model;

dd($model->raw_sql); /*select count(*) as aggregate from "product_prices" where "product_id" = '1' and "price" = '0' and "discount_total" > '0'*/

dd($model->recommendation);

Example Screen

Screenshot

Example Recommendation

In order to improve database performance and scalability, here are some suggestions below:

  1. Indexing: Effective database indexing can significantly speed up query performance. For your query, consider adding a combined (composite) index on product_id, price, and discount_total. This index would work well because the where clause covers all these columns.
CREATE INDEX idx_product_prices
ON product_prices (product_id, price, discount_total);

(Note: The order of the columns in the index might depend on the selectivity of the columns and the data distribution. Therefore, you might have to reorder them depending on your specific situation.)

  1. Data Types: Ensure that the values being compared are of appropriate data types. Comparing or converting inappropriate data types at run time will slow down the search. It appears that you're using string comparisons ('1') for product_id, price, and discount_total which are likely numerical columns. Remove the quotes for these where clause conditions.

Updated Query:

SELECT COUNT(*) AS aggregate
FROM product_prices
WHERE product_id = 1
AND price = 0
AND discount_total > 0;
  1. ANALYZE: Another practice to improve query performance could be running the ANALYZE command. This command collects statistics about the contents of tables in the database, and stores the results in the pg_statistic system catalog. Subsequently, the query planner uses these statistics to help determine the most efficient execution plans for queries.
ANALYZE product_prices;

Remember to periodically maintain your index to keep up with the CRUD operations that could lead to index fragmentation. Depending on your DBMS, you might want to REBUILD or REORGANIZE your indices.

Testing

composer test

Roadmap

  • Create a documentation page.
  • Begin development of version 2.
  • Auto Indexer (Premium Feature)
  • Create a FilamentPHP plugin.

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

License

The MIT License (MIT). Please see License File for more information.

laravel-slower's People

Contributors

dependabot[bot] avatar github-actions[bot] avatar halilcosdu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

laravel-slower's Issues

[Bug]: Small size of "bindings" column of the "slow_logs" table

What happened?

If you have a lot of bindings on your slow query, the bindings column 255 length it will not be enough and it will generates an error when trying to save the entry.

How to reproduce the bug

Execute a slow query with a lot of (or long) bindings.

Package Version

1.0.5

PHP Version

8.2.0

Laravel Version

11

Which operating systems does with happen with?

Linux

Notes

I have solved the problem changing the bindings column type in https://github.com/halilcosdu/laravel-slower/blob/main/database/migrations/create_slower_table.php.stub#L14

$table->longtext('bindings');

[Bug]: Error when creating slow_logs entries

What happened?

When the package tries to save the slow query entry, it produces the following error:

{
"message": "Object of class Illuminate\\Database\\MySqlConnection could not be converted to string",
"details": "Error in Connection.php line 707: Object of class Illuminate\\Database\\MySqlConnection could not be converted to string"
}

The problem is in this line:
https://github.com/halilcosdu/laravel-slower/blob/main/src/SlowerServiceProvider.php#L77

I'm not sure if you are trying to save the connection class name or another thing.

How to reproduce the bug

Use package in Laravel 11 app.

Package Version

1.0.5

PHP Version

8.2.0

Laravel Version

11

Which operating systems does with happen with?

Linux

Notes

I have temporary solved it updating this line to:

'connection' => get_class($event->connection),

[Bug]: Loop when slow logs entries creation are slow too

What happened?

Sometimes, if the slow logs entries creation is slow (below threshold), it enters an infinite loop saving the slow log entry of the previous slow log entry saving, and again in a recursive infinite loop that hangs the execution.

How to reproduce the bug

Lower the SLOWER_THRESHOLD to 100

Package Version

1.0.5

PHP Version

8.2.0

Laravel Version

11

Which operating systems does with happen with?

Linux

Notes

The problem can be fixed skipping the events which sql contains the slow_logs table name, for example, inserting the following code in https://github.com/halilcosdu/laravel-slower/blob/main/src/SlowerServiceProvider.php#L72

// Skip logging if the query is related to the slow log table
if (Str::contains($event->sql, [(new $model)->getTable()])) {
    return;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.