Giter Club home page Giter Club logo

sequenced's Introduction

Sequenced

.github/workflows/ci.yml Code Climate Gem Version

Sequenced is a simple gem that generates scoped sequential IDs for ActiveRecord models. This gem provides an acts_as_sequenced macro that automatically assigns a unique, sequential ID to each record. The sequential ID is not a replacement for the database primary key, but rather adds another way to retrieve the object without exposing the primary key.

Purpose

It's generally a bad practice to expose your primary keys to the world in your URLs. However, it is often appropriate to number objects in sequence (in the context of a parent object).

For example, given a Question model that has many Answers, it makes sense to number answers sequentially for each individual question. You can achieve this with Sequenced in one line of code:

class Question < ActiveRecord::Base
  has_many :answers
end

class Answer < ActiveRecord::Base
  belongs_to :question
  acts_as_sequenced scope: :question_id
end

Installation

Add the gem to your Gemfile:

gem 'sequenced'

Install the gem with bundler:

bundle install

Usage

To add a sequential ID to a model, first add an integer column called sequential_id to the model (or you many name the column anything you like and override the default). For example:

rails generate migration add_sequential_id_to_answers sequential_id:integer
rake db:migrate

Then, call the acts_as_sequenced macro in your model class:

class Answer < ActiveRecord::Base
  belongs_to :question
  acts_as_sequenced scope: :question_id
end

The scope option can be any attribute, but will typically be the foreign key of an associated parent object. You can even scope by multiple columns for polymorphic relationships:

class Answer < ActiveRecord::Base
  belongs_to :questionable, :polymorphic => true
  acts_as_sequenced scope: [:questionable_id, :questionable_type]
end

Multiple sequences can be defined by using the macro multiple times:

class Answer < ActiveRecord::Base
  belongs_to :account
  belongs_to :question

  acts_as_sequenced column: :question_answer_number, scope: :question_id
  acts_as_sequenced column: :account_answer_number, scope: :account_id
end

Schema and data integrity

This gem is only concurrent-safe for PostgreSQL databases. For other database systems, unexpected behavior may occur if you attempt to create records concurrently.

You can mitigate this somewhat by applying a unique index to your sequential ID column (or a multicolumn unique index on sequential ID and scope columns, if you are using scopes). This will ensure that you can never have duplicate sequential IDs within a scope, causing concurrent updates to instead raise a uniqueness error at the database-level.

It is also a good idea to apply a not-null constraint to your sequential ID column as well if you never intend to skip it.

Here is an example migration for a model that has a sequential_id scoped to a burrow_id:

# app/db/migrations/20151120190645_create_badgers.rb
class CreateBadgers < ActiveRecord::Migration
  def change
    create_table :badgers do |t|
      t.integer :sequential_id, null: false
      t.integer :burrow_id
    end

    add_index :badgers, [:sequential_id, :burrow_id], unique: true
  end
end

If you are adding a sequenced column to an existing table, you need to account for that in your migration.

Here is an example migration that adds and sets the sequential_id column based on the current database records:

# app/db/migrations/20151120190645_add_sequental_id_to_badgers.rb
class AddSequentalIdToBadgers < ActiveRecord::Migration
  add_column :badgers, :sequential_id, :integer

  execute <<~SQL
    UPDATE badgers
    SET sequential_id = old_badgers.next_sequential_id
    FROM (
      SELECT id, ROW_NUMBER()
      OVER(
        PARTITION BY burrow_id
        ORDER BY id
      ) AS next_sequential_id
      FROM badgers
    ) old_badgers
    WHERE badgers.id = old_badgers.id
  SQL

  change_column :badgers, :sequential_id, :integer, null: false
  add_index :badgers, [:sequential_id, :burrow_id], unique: true
end

Configuration

Overriding the default sequential ID column

By default, Sequenced uses the sequential_id column and assumes it already exists. If you wish to store the sequential ID in different integer column, simply specify the column name with the column option:

acts_as_sequenced scope: :question_id, column: :my_sequential_id

Starting the sequence at a specific number

By default, Sequenced begins sequences with 1. To start at a different integer, simply set the start_at option:

acts_as_sequenced start_at: 1000

You may also pass a lambda to the start_at option:

acts_as_sequenced start_at: lambda { |r| r.computed_start_value }

Indexing the sequential ID column

For optimal performance, it's a good idea to index the sequential ID column on sequenced models.

Skipping sequential ID generation

If you'd like to skip generating a sequential ID under certain conditions, you may pass a lambda to the skip option:

acts_as_sequenced skip: lambda { |r| r.score == 0 }

Example

Suppose you have a question model that has many answers. This example demonstrates how to use Sequenced to enable access to the nested answer resource via its sequential ID.

# app/models/question.rb
class Question < ActiveRecord::Base
  has_many :answers
end

# app/models/answer.rb
class Answer < ActiveRecord::Base
  belongs_to :question
  acts_as_sequenced scope: :question_id

  # Automatically use the sequential ID in URLs
  def to_param
    self.sequential_id.to_s
  end
end

# config/routes.rb
resources :questions do
  resources :answers
end

# app/controllers/answers_controller.rb
class AnswersController < ApplicationController
  def show
    @question = Question.find(params[:question_id])
    @answer = @question.answers.find_by(sequential_id: params[:id])
  end
end

Now, answers are accessible via their sequential IDs:

http://example.com/questions/5/answers/1  # Good

instead of by their primary keys:

http://example.com/questions/5/answer/32454  # Bad

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

sequenced's People

Contributors

ajb avatar clupprich avatar derrickreimer avatar excid3 avatar febeling avatar gabetax avatar gavinballard avatar hashtegner avatar istanful avatar louim avatar mkilling avatar mmartinson avatar olleolleolle avatar rmm5t avatar ryush00 avatar shaer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

sequenced's Issues

Race-Condition Proof?

Yes this is a hard problem to solve...However I have noted when performing imports or bulk model creation this library generates non-unique sequences.

class Person < ActiveRecord::Base
  validates_presence_of :account_id
   acts_as_sequenced scope: :account_id, column: :badge_number
end

In 2 consoles run the following command at the same time

 1000.times{ |i| Person.new(account_id: 1) }

I have a couple of strategies/ideas, but I thought it was worth asking. It is a problem that will come up in any bulk creation at scale or during concurrent processing which we do in the Rails world quite often through Resque or DJ or Sidekiq.

Strategies

  • db locking
  • In memory or in key-value store allocate the sequence and then use it. Similar to queue based strategies.
  • Retry mechanism

I don't know how much of this is in the scope of a gem. But I wondered what we could do here to minimize this issue, or what is an approach that is used at scale that doesn't break sequence generation?

Add migration path demonstration for existing tables

Add migration path demonstration for existing application

I just imported this gem to an existing application. I wanted to add an article_number column to my offers that is scoped to their respective organization.

In order to add the article_number column with a not null constraint, I had to create a migration with some custom SQL.

# Add the column without a constraint
add_column :offers, :article_number, :integer

# Generate the supposed values
execute <<~SQL
    UPDATE offers AS affected_offers
    SET article_number = (
      SELECT COUNT(id)
      FROM offers
      WHERE offers.organization_id = affected_offers.organization_id
      AND offers.id < affected_offers.id
    ) + 1
SQL

# Add not null constraint
change_column :offers, :article_number, :integer, null: false

# Add index
add_index :offers, %i[article_number organization_id], unique: true

How about adding an example for this in the README? Or even better create a generator?
If you think it's reasonable I could open a PR. :)

skip some numbers for sequential_id [feature]

My case is we have some number already taken by the user. So when sequenced generates a sequential_id, the numbers have to be skipped. I have basic logic but not successful in implementing in gem (a noob).

module Sequenced
  class Generator
    attr_reader :record, :scope, :column, :start_at, :skip, :forget

    def initialize(record, options = {})
      @record = record
      @scope = options[:scope]
      @column = options[:column].to_sym
      @start_at = options[:start_at]
      @skip = options[:skip]
      @forget = options[:forget]
    end
...
    def next_id      
      next_id_in_sequence.tap do |id|
        id += 1 until ( ! self.forget.map{ |d| d === id }.any? && unique?(id) )
      end
    end

forget option accepts flat array with number & range.

Is it possible to include this feature in the gem, please?

Suggestions/Questions

I had an idea for a similar gem in my TODO list, but after this week's RubyWeekly I noticed there is one already. So I will invest some effort into improving it, if you don't mind ๐Ÿ˜„

  1. Wdyt on supporting scopes as belongs_to association names:
class Answer < ActiveRecord::Base
  belongs_to :question
  acts_as_sequenced scope: :question
end

It looks more natural to me, than specifying column names and simplifies the setup when belongs_to is polymorphic.

  1. Support sharing scopes. For example, like on GitHub: both issues and pull requests are scoped to a repository.
  2. Why testing of MySQL was removed from this gem?
  3. Currently, when calculating the next sequential id, the whole table is locked (PostgreSQL only ๐Ÿค”?)
    def lock_table
    if postgresql?
    record.class.connection.execute("LOCK TABLE #{record.class.table_name} IN EXCLUSIVE MODE")
    end
    end

    Whole table locks are very heavyweight and dangerous (see lock queueing in PostgreSQL) and are really not needed. I suggest locking only the parent record (if :scope is for parent record) using SELECT ... FOR UPDATE (https://api.rubyonrails.org/v5.2/classes/ActiveRecord/Locking/Pessimistic.html), this should be enough.
  4. README states that the scope option can be any attribute (not only the foreign key of an associated parent object). But what is the use case for this?

If these changes are OK, I will start implementing them one by one.

Bug: primary key names other than 'id' are not working

If the model uses a non-default primary key name (anything other than 'id'), then this error comes up, when a new sequence ID is being generated:

ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR:  column "id" does not exist
LINE 1: ...UNT(*) FROM "with_custom_primary_keys" WHERE (NOT id = 4) AN...
                                                             ^
: SELECT COUNT(*) FROM "with_custom_primary_keys" WHERE (NOT id = 4) AND "with_custom_primary_keys"."sequential_id" = $1 AND "with_custom_primary_keys"."account_id" = $2

This happens only under the rare condition that the record is already persisted.

undefined method `sequenced_options' for #<Class:0xde76744>

Hi,nice work with this gem!

i'm having an issue when trying to save a new sequenced object.

undefined method 'sequenced_options' for #<Class:0xde76744>

Not sure what is going on, in my models got something like

class Question < ActiveRecord::Base
  has_many :answers
end

class Answer < ActiveRecord::Base
  belongs_to :question
  acts_as_sequenced scope: :question_id
end

And in some point in the controller i'm doing

new_a = question.answers.new(answer_params)
if (new_a.save)  <-- here the error is thrown
  ...
end

Any help will much appreciated.
Thanks,

Using on a base class with STI doesn't scope to base class

For example if I do something like this:

class Monster < ActiveRecord::Base
  acts_as_sequenced
end

class Zombie < Monster
end

class Werewolf < Monster
end

Zombie.create!
Werewolf.create!

the two monsters created will have a sequential_id of 1. I'd like my zombie to be 1 and my werewolf to be 2.

Using this as-is in Rails 5.2.1 doesnt work

Using a basic install of this doesn't seem to work with 5.2.1.

All of the sequential columns are set to 0 and aren't incremented on save. I also noticed it doesn't seem to do any queries at all to fetch the next ID in my logs.

Allow multiple sequences per model

Use case:

class Order < ActiveRecord::Base
  belongs :cart 
  acts_as_sequenced column: :sequence
  acts_as_sequenced column: :per_cart_sequence, scope: :cart_id
end

Issue:

per_cart_sequence will overwrite the sequence options, and we and having only one sequence triggered

Collisions under high load

Hi,

It appears like the current implementation will experience collisions under high load. How do you feel about relying on the DBMS to sequence the id?

Cheers

Failing tests due to changed migration API

The gemspec specifies rails version >= 3.1 which resolves to rails
5.2.1. In this version of rails the migration API had changed, resulting
in our test helper to fail.

Typo on Readme

Sorry if it's kind of stupid to open an issue to report that but...

https://github.com/djreimer/sequenced#example

On

# config/routes.rb
resources :questions
  resources :answers
end

You forgot the 'do'

# config/routes.rb
resources :questions do
  resources :answers
end

That's it :)

Thanks

Building scopes doesn't work with ActiveRecord enums

Thanks for the great gem. It's been working perfectly for me, up until when I tried to use enums as a sequence scope.

The issue is here:
https://github.com/djreimer/sequenced/blob/master/lib/sequenced/generator.rb#L74

rel.where(c => record.send(c.to_sym))

Querying by enum value requires an integer, not a string value, and Rails' automatic typecasting is causing the query to return nothing.

Calling record.send(c.to_sym)) with an enum gives the string value of the enum, not the integer. Then the where(c => 'my_string_value') gets typecast to 0 before the query runs and nothing is returned (so all sequence_ids end up being 1).

Adding sequential_id to has_one :through relation

Good Day :),

Relations:

class Bank < ApplicationRecord
  has_many :accounts
  has_many :customers, through: :accounts
end

class Customer < ApplicationRecord
  has_one :account
  has_one :bank, through: :account
end

class Account < ApplicationRecord
  belongs_to :bank
  belongs_to :customer
end

Problem

Since, there is no bank_id in customers table, how can I add sequential_id to Customer model and scope it the Bank model?

Start at not work

I am trying use the attribute start_at, but not works. I am using Rails 5.2 and PostgreSQL 10.1.

Deadlock when running concurrent transactions

I am running into an issue where I am getting an ActiveRecord::Deadlocked error

PG::TRDeadlockDetected: ERROR:  deadlock detected
DETAIL:  Process 15304 waits for ExclusiveLock on relation 87680265 of database 87679929; blocked by process 16055.
Process 16055 waits for ExclusiveLock on relation 87680265 of database 87679929; blocked by process 15304.

This looks to be because there is an exclusive lock being put on the entire table. I understand the idea here, but when using a scope, it prevents distributed jobs running in different scopes. Is there any way around this? Is it possible to limit the lock based on scope?

Promotability/Demotability/Destruction sanitization

We guys are using this acts_as_sequenced a little differently to sequentially index stuff in scoped constraints . Like in dependent priority orders and stuff. And so I wrote code for promote(up/down) and sanitize sequence upon destruction(get all the records and assigns seq_id in asc order).

Will it be useful for you?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.