Giter Club home page Giter Club logo

schema_plus_indexes's Introduction

Gem Version Build Status Coverage Status

SchemaPlus::Indexes

SchemaPlus::Indexes adds various convenient capabilities to ActiveRecord's index handling.

SchemaPlus::Indexes is part of the SchemaPlus family of Ruby on Rails extension gems.

Installation

As usual:

gem "schema_plus_indexes"                # in a Gemfile
gem.add_dependency "schema_plus_indexes" # in a .gemspec

Features

Migrations:

Shorthand to define a column with an index:

t.string :role,             index: true     # shorthand for index: {}

Shorthand to define a column with a unique index:

t.string :product_code,     index: :unique  # shorthand for index: { unique: true }

Create multi-column indexes as part of column definition

Adds an option to include other columns in the index:

t.string :first_name
t.string :last_name,        index: { with: :first_name }

t.string :country_code
t.string :area_code
t.string :local_number,     index: { with: [:country_code, :area_code] }

Create indexes with add_column, change_table

ActiveRecord supports the index: option to column definitions when creating table. SchemaPlus::Indexes extends that to work also with add_column and in change_table

add_column "tablename", "columnname", index: { ... }

change_table :tablename do |t|
  t.integer :column,    index: true
end

These of course accept the shorthands and with: option described above.

Remove index :if_exists

remove_index "tablename", "columnname", if_exists: true

Models

SchemaPlus::Indexes lets you easily get the indexes of a model:

Model.indexes  # shorthand for `connection.indexes(Model.table_name)`

The value gets cached until the next time Model.reset_column_information is called.

Other things...

  • Provides consistent behavior regarding attempted duplicate index creation: Ignore and log a warning. Different versions of Rails with different db adapters otherwise behave inconsistently: some ignore the attempt, some raise an error.

  • In the schema dump schema.rb, index definitions are included within the create_table statements rather than added afterwards

  • When using SQLite3, makes sure that the definitions returned by connection.indexes properly include the column orders (:asc or :desc)

  • For the ActiveRecord::ConnectionAdapters::IndexDefinition class (the object that's returned by connection.indexes), SchemaPlus::Indexes:

    • Provides an == operator to compare if two objects refer to an equivalent index
    • Allows calling new with a signature that matches add_index: IndexDefinition.new(table_name, column_names, options)
    • Fleshes out the :orders attribute, listing :asc for a column instead of leaving it undefined.
    • Prevents errors from a down :change migration attempting to remove an index that wasn't previously added (this can arise, e.g. with auto-indexing plugins).

Compatibility

schema_plus_indexes is tested on

  • ruby 2.5 with activerecord 5.2, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.5 with activerecord 6.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.5 with activerecord 6.1, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.7 with activerecord 5.2, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.7 with activerecord 6.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.7 with activerecord 6.1, using mysql2, sqlite3 or postgresql:9.6
  • ruby 2.7 with activerecord 7.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.0 with activerecord 6.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.0 with activerecord 6.1, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.0 with activerecord 7.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.1 with activerecord 6.0, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.1 with activerecord 6.1, using mysql2, sqlite3 or postgresql:9.6
  • ruby 3.1 with activerecord 7.0, using mysql2, sqlite3 or postgresql:9.6

Release Notes

v1.0.1

  • Add AR 6.1 and 7.0 support
  • Add Ruby 3.1 support

v1.0.0

  • Drop AR < 5.2 and add 6.0 support
  • Drop Ruby < 2.5 and add Ruby 3.0 support
  • Remove its-it GEM

v0.3.1

v0.3.0

v0.2.4

  • Supports AR 5.0. Thanks to @myabc

v0.2.3

  • Missing require

v0.2.2

  • Explicit gem dependencies

v0.2.1

  • Upgrade to schema_plus_core 1.0 and conform

v0.2.0

  • Prevent down :change migrations from failing due to trying to remove non-existent indexes

v0.1.0

  • Initial release, extracted from schema_plus 1.x

Development & Testing

Are you interested in contributing to schema_plus_indexes? Thanks! Please follow the standard protocol: fork, feature branch, develop, push, and issue pull request.

Some things to know about to help you develop and test:

  • schema_dev: SchemaPlus::Indexes uses schema_dev to facilitate running rspec tests on the matrix of ruby, activerecord, and database versions that the gem supports, both locally and on github actions

    To to run rspec locally on the full matrix, do:

      $ schema_dev bundle install
      $ schema_dev rspec
    

    You can also run on just one configuration at a time; For info, see schema_dev --help or the schema_dev README.

    The matrix of configurations is specified in schema_dev.yml in the project root.

  • schema_plus_core: SchemaPlus::Indexes uses the SchemaPlus::Core API that provides middleware callback stacks to make it easy to extend ActiveRecord's behavior. If that API is missing something you need for your contribution, please head over to schema_plus_core and open an issue or pull request.
  • schema_monkey: SchemaPlus::Indexes is implemented as a schema_monkey client, using schema_monkey's convention-based protocols for extending ActiveRecord and using middleware stacks.

schema_plus_indexes's People

Contributors

felipemaier avatar fj avatar iagopiimenta avatar myabc avatar nbouscal avatar ronen avatar urkle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

schema_plus_indexes's Issues

SchemaPlus::Indexes break AR index expressions

ActiveRecord 5 added the ability to use functional/expression indexes with PostgreSQ, i.e.

t.index 'lower(name)'

SchemaPlus::Indexes appears to break this functionality and requires you to fall back to the old-style execute statements directly. SchemaPlus::Indexes raises a PG::UndefinedColumn exception, with the error message reading ERROR: column "lower(name)" does not exist, as it tries to simply escape the full expression as text rather than allowing the expression through to create the index.

Rails 5.1 support

Currently with Rails 5.1.1, getting:

    schema_plus_pg_indexes was resolved to 0.2.1, which depends on
      schema_plus_indexes (>= 0.2.4, ~> 0.2) was resolved to 0.2.4, which depends on
        activerecord (< 5.1, >= 4.2)

undefined local variable or method `its' in 0.2.2

Looks like removing the its-it dependency in SchemaPlus/modware#2 has broken schema_plus_pg_indexes for me on https://github.com/andrew/contribulator, see this travis build: https://github.com/andrew/contribulator

Adding its-it back to the Gemfile file gets the tests passing again: 24pullrequests/contribulator#295

== 20160421143043 AddIndexesToProjects: migrating =============================
-- change_table(:projects)
   -> 0.0090s
== 20160421143043 AddIndexesToProjects: migrated (0.0091s) ====================
rake aborted!
NameError: undefined local variable or method `its' for #<Modware::Stack::Middleware:0x00000005cce7f0>
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_indexes-0.2.2/lib/schema_plus/indexes/middleware/dumper.rb:14:in `block in after'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_indexes-0.2.2/lib/schema_plus/indexes/middleware/dumper.rb:11:in `each'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_indexes-0.2.2/lib/schema_plus/indexes/middleware/dumper.rb:11:in `after'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:46:in `block in execute'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:45:in `each'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:45:in `execute'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:19:in `start'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_monkey-2.1.5/lib/schema_monkey/stack.rb:34:in `start'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_core-1.0.1/lib/schema_plus/core/active_record/schema_dumper.rb:48:in `table'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/schema_dumper.rb:97:in `block in tables'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/schema_dumper.rb:96:in `each'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/schema_dumper.rb:96:in `tables'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_core-1.0.1/lib/schema_plus/core/active_record/schema_dumper.rb:43:in `block in tables'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:54:in `call_implementation'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:84:in `_modware_continue'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:76:in `_modware_call'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:43:in `execute'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/modware-0.1.3/lib/modware/stack.rb:19:in `start'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_monkey-2.1.5/lib/schema_monkey/stack.rb:34:in `start'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_core-1.0.1/lib/schema_plus/core/active_record/schema_dumper.rb:42:in `tables'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/schema_dumper.rb:38:in `dump'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/schema_plus_core-1.0.1/lib/schema_plus/core/active_record/schema_dumper.rb:17:in `dump'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/schema_dumper.rb:22:in `dump'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/railties/databases.rake:240:in `block (4 levels) in <top (required)>'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/railties/databases.rake:239:in `open'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/railties/databases.rake:239:in `block (3 levels) in <top (required)>'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/railties/databases.rake:52:in `block (2 levels) in <top (required)>'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.6/lib/active_record/railties/databases.rake:45:in `block (2 levels) in <top (required)>'
/home/travis/build/andrew/contribulator/vendor/bundle/ruby/2.3.0/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/cli/exec.rb:63:in `load'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/cli/exec.rb:63:in `kernel_load'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/cli/exec.rb:24:in `run'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/cli.rb:304:in `exec'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/cli.rb:11:in `start'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/exe/bundle:27:in `block in <top (required)>'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
/home/travis/.rvm/gems/ruby-2.3.1@global/gems/bundler-1.12.5/exe/bundle:19:in `<top (required)>'
/home/travis/.rvm/gems/ruby-2.3.1/bin/bundle:23:in `load'
/home/travis/.rvm/gems/ruby-2.3.1/bin/bundle:23:in `<main>'
/home/travis/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
/home/travis/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => db:schema:dump
(See full trace by running task with --trace)

PG: migration failure when re-attempting concurrent index addition - ArgumentError: unknown keyword: :algorithm

Summary

schema_plus_indexes raises ArgumentError: unknown keyword: :algorithm when encountering an invalid index from a previous (concurrent) index creation attempt, rather than raising the underlying PG::DuplicateTable error (or a similar error).

Background

Postgres indexes can be added concurrently to prevent reads/writes from being blocked while the index is created.

If the index can't be added, for example if it is a unique index and the table data is not unique, then the index is left in an "invalid" state that must be cleaned up before re-attempting the index creation.

Reproduction

The following tests exhibits the issue:

diff --git a/spec/migration_spec.rb b/spec/migration_spec.rb
index c51a6a6..eecfb27 100644
--- a/spec/migration_spec.rb
+++ b/spec/migration_spec.rb
@@ -124,6 +124,32 @@ describe ActiveRecord::Migration do
     end
   end
 
+  context "when a PostgreSQL concurrent index creation fails", postgresql: :only do
+    before(:each) do
+      @model = Comment
+      # Create rows so that user_id: 1 is not unique...
+      @model.create!(user_id: 1)
+      @model.create!(user_id: 1)
+    end
+
+    def add_unique_index
+      change_table(@model) do |t|
+        t.index :user_id, unique: true, algorithm: :concurrently
+      end
+    end
+
+    it "raises if an invalid index already exists" do
+      # Simulate a migration that attempts to (concurrently) add a unique index. The first time it
+      # runs, the index add fails because the user_id is not unique. The second time, the migration
+      # fails because the previous migration attempt left an index that is in an invalid state and
+      # thus requires manual attention to clean up (to delete the index)
+      expect { add_unique_index }.to raise_error(ActiveRecord::RecordNotUnique)
+
+      # Try again... we expect that we should be alerted to the already-present (but invalid) index
+      expect { add_unique_index }.to raise_error(/already exists/)
+    end
+  end
+
   context "when column is added", :sqlite3 => :skip do
 
     before(:each) do

The issue seems to stem from the functionality to not raise duplicate index errors (I can't tell why this functionality is desirable - surely a duplicate index indicates some kind of logic error and is thus unexpected. Perhaps it would be sensible to deprecate/remove it):

attempted = ::ActiveRecord::ConnectionAdapters::IndexDefinition.new(
env.table_name, name, env.options[:unique], env.column_names,
**env.options.except(:unique)
)
raise if attempted != existing
::ActiveRecord::Base.logger.warn "[schema_plus_indexes] Index name #{name.inspect}' on table #{env.table_name.inspect} already exists. Skipping."

The ArgumentError is raised since ::ActiveRecord::ConnectionAdapters::IndexDefinition does not have an algorithm: keyword. I think the class in question is meant to represent an index that has already been created and thus the algorithm that was used to create that index is irrelevant.

Related

This Rails PR looks useful, but is not (as far as I can tell) available in a released version of Rails: rails/rails#45160

Dumper overrides change order of schema dump

Since this gem overrides SchemaDumper, and generates table schemas in a different place to ActiveRecord, it interferes with other gems that hook into SchemaDumper, e.g. Scenic. All of e.g. Scenic's dump output will come first, and the normal ActiveRecord output after, which doesn't work if the e.g. Scenic output depends on the ActiveRecord tables existing first (which is almost always the case).

Now, since this gem is a dependency of schema_validations, it interferes with schema generation even when that's completely unrelated to what the user actually wants, and there's not even a good way to disable it. I'm having to do this to turn off the schema dump features:

module SchemaPlus::Core::ActiveRecord::SchemaDumper
  instance_methods.each do |name|
    define_method(name) { |*args| super *args }
  end
end

To fix this, the SchemaDumper should dump tables in the same place as ActiveRecord, rather than after the rest of the dump is done.

Thanks for an otherwise great library.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.