A rails 6 boilerplate to get things started quickly on future pojects.
The project will bootstrap
- user models with non-social register, login and logout apis (logout for removing mobile notification token since jwt technically dont need a logout function in backend)
- admin user models with non-social login web pages at the front of a CMS, following the SB2 Admin template.
- a default post model for demo purpose, which should be removed on start up.
These steps are taken when setting up the project. It affects only the first commit of the project. They are run only once and is noted here for documentation purpose only.
- Run
rvm get stable
to get latest version ofrvm
- Run
rvm install ruby-2.6.6
to get latest version ofruby
at time of making this project - Run
rvm use 2.6.6 && rvm gemset create rails6boilerplate
in the parent folder of the app folder that will be created - Run
gem install rails -v 6
to get the rails binaries - Run
brew install yarn
to install yarn which is required by webpacker, the new front end management tool for rails 6 - Run
rails new rails6boilerplate --database=mysql --webpack=react
to setup project - Run
brew install [email protected]
to install latest version of mysql-5.7.mysql-8
will not be used.
The api responses will contain response_code
and response_message
.
The message will rely on I18n translation as much as possible, with response_code
representing the key and response_message
representing the message.
Each API controller will inherit from Api::BaseController
and their actions will have @response_code
and @response_message
variables added in a before_action
call. To change the response_code
or response_message
in the response json, overwrite the @response_code
or @response_message
variables.
Run the command below to generate example responses and request using apipie and rspec
APIPIE_RECORD=examples rspec
All cms controllers should inherit from a Cms::BaseController
following the design as the API portion.
Cms controllers will render html views like a original Ruby on Rails app. The Cms::BaseController
set the default layout for all cms routes to the cms.html.slim
template.
For admin users' devise related views and controllers, the views follow the path of the original devise controllers, scoped under admin_users
. The individual views still yield
from the devise.html.slim
template, which is tweaked to fit the single pages in the SB2 Admin template. Only the registrations
pages will be under the cms.html.slim
template as they occur within an authenticated session and should be viewing the pages inside the cms panel.
after_sign_in_path_for
and after_sign_out_path_for
are set in ApplicationController
and they will affect the behavior for devise controller on admin_users
only.
User model use devise
with OAuth provider doorkeeper
and doorkeeper-jwt
to allow refresh token and for users stay logged in. Consider devise-jwt
if you want to expire your users' sessions.
The decision to use doorkeeper
instead of devise-jwt
is due to the requirement for permanent logged in session in most of the applications that I need to build and this comment from the owner of devise-jwt
gem.
These are the steps taken:
- Run
rails generate devise:install
- Run
rails generate devise user
- Follow this guide to setup doorkeeper with devise
- Follow this guide to add the jwt support for doorkeeper
Admin user will authenticate without using neither devise-jwt
nor doorkeeper
. The only interaction admin users will have with this app is via a browser to work on the CMS. That implies the use of cookies instead of jwt, as well as just the old school devise
.
Sample model for showing sample codes for associations, active storage integration etc.
This should be deleted before starting work on the application.
Read the Usage section for the procedure to do so.
Change the gemset name and ruby version to be used in .ruby-version
file.
Run bundle
to install the files.
Add password to database.yml
for your root user to authenticate with the database.
Run EDITOR=vim rails credentials:edit
to generate config/master.key
file and config/credentials.yml.enc
file. Make sure to add the key secret_key_base
. It is used to create secrets.
NOTE: In the event the master.key
is lost, go to the aws management console of the application and get a copy form the environment variables configurations.
Run rails g rename:into <YOUR_PROJECT_NAME
to rename the application. Note that this will rename the repository you are in as well. You will need to run cd
commands to switch directories.
Remove sample
related tools by:
- run
rails d model sample
- run
rails d scaffold cms::samples
- run
rails d scaffold_controller api::v1::samples
- delete
has_many :samples
in user model - delete
db/seeds/1_samples.rb
- delete
spec/factories/sample.rb
- remove
samples
related routes inroutes.rb
- drop, create and migrate database
- run APIPIE_RECORD=examples rspec
- run
annotate
Run EDITOR=vim rails credentials:edit
to generate config/master.key
- Remove db migration file with
rm db/migrate/*_create_doorkeeper_tables.rb
- Remove API related files with
rm \
app/concerns/api_rescues.rb \
app/controllers/api/v1/tokens_controller.rb \
spec/support/token_helpers.rb \
spec/support/api_helpers.rb \
config/initializers/apipie.rb \
config/initializers/doorkeeper.rb \
db/migrate/20190905013830_create_doorkeeper_tables.rb \
config/locales/doorkeeper.en.yml
rm -rf \
doc \
app/controllers/api \
spec/requests \
app/views/api
- Make changes at these files:
spec/spec_helper.rb
config/application.rb
config/routes.rb
config/locales/custom.en.yml
app/views/layouts/_sidebar.html.slim
app/controllers/api/base_controller.rb
app/models/user.rb
- Remove gems
bundle remove apipie-rails doorkeeper doorkeeper-jwt rack-cors
With reference to this guide, the oauth_applications
table and all its associated indices and associations are removed. The t.references :application, null: false
is also changed to t.integer :application_id
. previous_refresh_token
column is also removed. access_token
and refresh_token
are set to text
data type and have their indices removed to prevent being too long to save in database column.
In the config/routes.rb
file, token_info
controller is skipped.
API mode is established and authorization request are removed and doorkeeper
applications views are not rendered.
This setup will remove the authorization server that is doorkeeper, leaving only the refresh token and access token mechanism still in place.
The Api::V1::TokensController
controller inherits from Doorkeeper::TokensController
.
refresh
route will handle the refresh mechanism while the login
route will handle the login mechanism. Both share the parent controller's create
method by the methodology of doorkeeper
.
login
routes will use application/json
content-type
instead of application/x-www-form-urlencoded
according to spec.
Tokens will be revoked in a logout
api. Revoked tokens will have impact on posts
APIs. handle_auth_errors
is set to :raise
in doorkeeper.rb
, so the Doorkeeper::Errors
will be triggered via the before_action :doorkeeper_authorize!
in the API::BaseController
, which should be inherited by most of, if not all, the custom controllers. Each of the Doorkeeper::Errors
will return their specific errors.
Provisioning of cloud resources will be done using Terraform
.
Terraform
commands will be run using terraform
and packer
docker
images.
An AWS S3
backend will hold the tfstate
file for Terraform
. The s3 bucket is created via the terraform:init
rake task.
A private and its corresponding ssh key pair will be generated using ssh-keygen
command. The ssh keys serve 2 purposes:
- For creating the
aws_key_pair
for your ec2 instance(s) - For ssh authentication with your project on private git repository if any
Application will be deployed using AWS Elastic Beanstalk
.
- Do this once. This ensures the official net-ssh-gateway gem is downloaded and not a tamerped version.
# Add the public key as a trusted certificate
# (You only need to do this once)
$ curl -O https://raw.githubusercontent.com/net-ssh/net-ssh-gateway/master/net-ssh-public_cert.pem
$ gem cert --add net-ssh-public_cert.pem
$ rm -f net-ssh-public_cert.pem
- Install
docker
Run a rake task to get prompt to enter details. This step is necessary to prevent erroneously change environments.
rake ebs:init
This command will require you to input aws_profile
, env
and region
, and whether your want to setup a single instance or not.
It will save the tfstate
file in a newly created S3 bucket storing the tf_state.
There is a multiple_instances
and single_instance
module for different setup required.
The rake function will create the terraform
files in the terraform/<ENV>
directory. These terraform
files will include the relevant modules depending on what you have selected.
TODO single_instance with ssl.
After deploying the infrastructure, the eb-user
access key id and access secret key will be shown on the terminal. Use it to deploy your application to Elastic Beanstalk
.
Requires the Elastic Beanstalk
cli.
eb init --region <REGION>--profile <YOUR_AWS_NAMED_PROFILE>
eb deploy # OR eb deploy --staged
Note that eb deploy
deploys only committed files to the server, or at the very least, staged files but that will require the --staged
option.
To get the Elastic Beanstalk logs, run:
eb logs --all
The logs for all the components of Elastic beanstalk will be downloaded into .elasticbeanstalk/logs
folder
Sometimes the application fails to deploy right at the start. You will have to ssh into the instance and do a tail of the all the logs:
tail -f /var/log/**/*log* /var/log/*log*
The database of single_instance
is be made publicly/remotely accessible.
To connect to it via rails console from the developer's local machine, run this command.
DATABASE_URL=mysql2://<RDS_ENDPOINT>/<DB_NAME> RAILS_ENV=<ENV> rails console
The RDS endpoint can be obtain be running in the rake ebs:output
to make trivial changes to the terraform state and show the output of the various resources in the infrastructure.
The database of multiple_instances
will not be made publicly/remotely accessible. As such, to communicate with the instances which are in the private subnets, a bastion server is required with the use of ssh agent forwarding.
This will bring up the bastion server.
rake ebs:bastion:up
- create bastion server AMI
- setup bastion server in one of the public subnets that were created in the custom VPC
- outputs bastion server public ip address
This will bring up the bastion server.
rake ebs:bastion:down
This will remove the bastion AMI (to save on S3 storage cost for storing the image which may be negligible)
rake ebs:bastion:unpack
Some common rails commands that can be executed on the instances/database conveniently.
rake ebs:rails:console
rake ebs:rails:seed
rake ebs:rails:reseed
These tasks involves tunneling through the bastion server, which means the bastion server has to be setup before hand.
If the environment created is a single instance, its RDS should be publicly accessible. Hence, you can connect from your local machine and run the commands locally instead of having to use these commands.
Rails logger is an instance of cloudwatchlogger. Log stream name is using the default generated by the gem. Setup is under the config/environments/production.rb
file. Credentials and region uses the secrets in the credentials.yml.enc
file.
Refer to this gist.
Order is in descending order using created_at
attribute of ActiveStorage::Attachment
model (since updated_at
is not present by default). The created_at
is artificially tweaked when admin changes the order in the cms to maintain psuedo order.
Which means any new image will be the latest created.
- use https://registry.terraform.io/modules/trussworks/logs/aws/3.0.0 to add logs bucket instead of aws cli
- deployment rake task should check for
config/<ENV>.rb
and allow user to choose, instead of asking - Use packer instead of provisioner scripts
- add taggable
- update ckeditor version when latest version, which contain support for ActiveStorgae, is released (galetahub/ckeditor#853)
To install SSL on single instance, do these things
- create this file
.ebextensions/00_ssl_certificates.config
container_commands:
copy_combined_crt:
command: cp .ebextensions/ssl/<CRT_FILE_NAME> /home/ec2-user/<CRT_FILE_NAME>
copy_csr_key:
command: cp .ebextensions/ssl/<KEY_FILE_NAME> /home/ec2-user/<KEY_FILE_NAME>
- Adjust
nginx.conf
to fit your needs - Install the ssl files into the folder
.ebextensions/ssl
- Update the
asset_host
,host
,protocol
etc in the credential file - Open up port 443 in
rds.tf
to allow request to come in from that port.
resource "aws_security_group_rule" "https-web_server-single_instance" {
type = "ingress"
from_port = 443
to_port = 443
protocol = "tcp"
security_group_id = aws_security_group.web_server-single_instance.id
cidr_blocks = ["0.0.0.0/0"]
}
Create the dump file in one of the private instance
rake ebs:bastion:ssh
mysqldump -h <RDS_ENDPOINT> -u <DB_USER> -p <DATABASE_NAME > dump.sql
# Determine the private instance with the dump file
curl http://169.254.169.254/latest/meta-data/local-ipv4
Download to bastion
ssh -tt -A -i <SSH_KEY> ec2-user@<BASTION_PUBLIC_IP> -o 'UserKnownHostsFile /dev/null' -o StrictHostKeyChecking=no "scp -o 'UserKnownHostsFile /dev/null' -o StrictHostKeyChecking=no ec2-user@<INSTANCE_PRIVATE_IP>:~/dump.sql ."
Download to local computer
scp -i <SSH_KEY> ec2-user@<BASTION_PUBLIC_IP>:~/dump.sql ./dump.sql