Art Vandelay is an importer/exporter for Rails 6.0 and higher.
Have you ever been on a project where, out of nowhere, someone asks you to send them a CSV of data? You think to yourself, βOk, cool. No big deal. Just gimme five minutesβ, but then that five minutes turns into a few hours. Art Vandelay can help.
At a high level, hereβs what Art Vandelay can do:
- πΆ Automatically filters out sensitive information.
- π Export data in batches.
- π§ Email exported data.
- π₯ Import data from a CSV.
Add this line to your application's Gemfile:
gem "art_vandelay"
And then execute:
$ bundle
# config/initializers/art_vandelay.rb
ArtVandelay.setup do |config|
config.filtered_attributes = [:credit_card, :birthday]
config.from_address = "[email protected]"
config.in_batches_of = 5000
end
Attribute | Value | Description |
---|---|---|
filtered_attributes |
[:passw, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn] |
Attributes that will be automatically filtered when exported |
from_address |
nil |
The email address used when sending an email of exports |
in_batches_of |
10000 |
The number of records that will be exported into each CSV |
ArtVandelay::Export.new(records, export_sensitive_data: false, attributes: [], in_batches_of: ArtVandelay.in_batches_of)
Argument | Description |
---|---|
records |
An Active Record Relation or an instance of an Active Record. E.g. User.all , User.first , User.where(...) , User.find_by |
export_sensitive_data |
Export sensitive data. Defaults to false . Can be configured with ArtVandelay.filtered_attributes . |
attributes |
An array attributes to export. Default to all. |
in_batches_of |
The number of records that will be exported into each CSV. Defaults to 10,000. Can be configured with ArtVandelay.in_batches_of |
Returns an instance of ArtVandelay::Export::Result
.
result = ArtVandelay::Export.new(User.all).csv
# => #<ArtVandelay::Export::Result>
csv_exports = result.result.csv_exports
# => [#<CSV::Table>, #<CSV::Table>, ...]
csv = csv_exports.first.to_a
# => [["id", "email", "password", "created_at", "updated_at"], ["1", "[email protected]", "[FILTERED]", "2022-10-25 09:20:28 UTC", "2022-10-25 09:20:28 UTC"]]
result = ArtVandelay::Export.new(User.all, export_sensitive_data: true).csv
# => #<ArtVandelay::Export::Result>
password = result.csv_exports.first["password"]
# => ["bosco"]
result = ArtVandelay::Export.new(User.all, attributes: [:email]).csv
# => #<ArtVandelay::Export::Result>
csv = result.csv_exports.first.to_a
# => [["email"], ["george@vandelay_industries.com"]]
result = ArtVandelay::Export.new(User.all, in_batches_of: 100).csv
# => #<ArtVandelay::Export::Result>
csv_size = result.csv_exports.first.size
# => 100
Emails the recipient(s) CSV exports as attachments.
email_csv(to:, from: ArtVandelay.from_address, subject: "#{model_name} export", body: "#{model_name} export")
Argument | Description |
---|---|
to |
An array of email addresses representing who should receive the email. |
from |
The email address of the sender. |
subject |
The email subject. Defaults to the following pattern: "User export" |
body |
The email body. Defaults to the following pattern: "User export" |
ArtVandelay::Export
.new(User.where.not(confirmed: nil))
.email_csv(
to: ["george@vandelay_industries.com", "kel_varnsen@vandelay_industries.com"],
from: "noreply@vandelay_industries.com",
subject: "List of confirmed users",
body: "Here's an export of all confirmed users in our database."
)
# => ActionMailer::Base#mail: processed outbound mail in...
ArtVandelay::Import.new(model_name, **options)
Argument | Description |
---|---|
model_name |
The name of the model being imported. E.g. :users , :user , "users" or "user" |
**options |
A hash of options. Available options are rollback: |
Option | Description |
---|---|
rollback: |
Whether the import should rollback if any of the records fails to save. |
Imports records from the supplied CSV. Returns an instance of ArtVandelay::Import::Result
.
csv_string = CSV.generate do |csv|
csv << ["email", "password"]
csv << ["george@vandelay_industries.com", "bosco"]
csv << ["kel_varnsen@vandelay_industries.com", nil]
end
result = ArtVandelay::Import.new(:users).csv(csv_string)
# => #<ArtVandelay::Import::Result>
result.rows_accepted
# => [{:row=>["george@vandelay_industries.com", "bosco"], :id=>1}]
result.rows_rejected
# => [{:row=>["kel_varnsen@vandelay_industries.com", nil], :errors=>{:password=>["can't be blank"]}}]
csv(csv_string, **options)
Argument | Description |
---|---|
csv_string |
Data in the form of a CSV string. |
**options |
A hash of options. Available options are headers: and attributes: |
Option | Description |
---|---|
headers: |
The CSV headers. Use when the supplied CSV string does not have headers. |
attributes: |
The attributes the headers should map to. Useful if the headers do not match the model's attributes. |
csv_string = CSV.generate do |csv|
csv << ["email", "password"]
csv << ["george@vandelay_industries.com", "bosco"]
csv << ["kel_varnsen@vandelay_industries.com", nil]
end
result = ArtVandelay::Import.new(:users, rollback: true).csv(csv_string)
# => rollback transaction
csv_string = CSV.generate do |csv|
csv << ["george@vandelay_industries.com", "bosco"]
end
result = ArtVandelay::Import.new(:users).csv(csv_string, headers: [:email, :password])
# => #<ArtVandelay::Import::Result>
csv_string = CSV.generate do |csv|
csv << ["email_address", "passcode"]
csv << ["george@vandelay_industries.com", "bosco"]
end
result = ArtVandelay::Import.new(:users).csv(csv_string, attributes: {email_address: :email, passcode: :password})
# => #<ArtVandelay::Import::Result>
- Run
./bin/setup
. - Make your changes.
- Ensure
./bin/ci
passes. - Create a pull request.
The gem is available as open source under the terms of the MIT License.