Giter Club home page Giter Club logo

voyage-api-java's Introduction

Overview

A foundational set of web services that implement industry standard guidelines, common best practices, and the experienced insights afforded to Lighthouse Software thru decades of enterprise business software development.

Created and supported by Lighthouse Software @ https://LighthouseSoftware.com

Topics

5 Minute Test

Run the Voyage API and execute a JSON API request within 5 minutes

  1. Download the source locally

    • Ensure you have Git installed locally or within your IDE
    • This example will assume you are at a terminal console command prompt.
    • NOTE: If you choose to use your own IDE, then follow the git clone instructions for your particular IDE

    git clone https://github.com/lssinc/voyage-api-java.git

  2. Start the app

    • Gradle build system is used to build, test, and run the application
    • Using the "gradew" commands in the root of the project will automatically download and install a local version of Gradle

    Linux/OSX: ./gradlew bootRun Windows: gradlew.bat

  3. Verify the app is running

    • Once the app is done starting up and loading the in-memory database with seed data, the following log statement will display in the console:

      INFO 23204 --- [ main] voyage.App : Started App in 27.519 seconds (JVM running for 28.708)

    • Open a browser and access the general "status" web service at http://localhost:8080/api/status. A JSON response should appear like the following:

      {"status":"alive","datetime":"2018-05-15T13:52:12-05:00"}

  4. Get an OAuth2 Authorization token to execute requests on the API

    • On the terminal, run the following cURL command (OSX, Linux already has cURL. Download for Windows)

      curl -u client-super:secret -X POST -d "client_id=client-super&client_secret=secret&grant_type=client_credentials" http://localhost:8080/oauth/token

    • The return should be a JSON result like the following:

        "token_type":"bearer",
        "expires_in":7199,
        "scope":"Read Data Write Data",
        "created":1527473447843,
        "jti":"77a008f4-0820-4619-a50a-acb11224853a"}```
        
      
    • Copy the "access_token" value from the JSON response eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJjcmVhdGVkIjoxNTI3NDczNDQ3ODQzLCJzY29wZSI6WyJSZWFkIERhdGEiLCJXcml0ZSBEYXRhIl0sImV4cCI6MTUyNzQ4MDY0NywiYXV0aG9yaXRpZXMiOlsiYXBpLnBlcm1pc3Npb25zLmRlbGV0ZSIsImFwaS5yb2xlcy5kZWxldGUiLCJhcGkucm9sZXMudXBkYXRlIiwiYXBpLnBlcm1pc3Npb25zLmxpc3QiLCJhcGkucGVybWlzc2lvbnMudXBkYXRlIiwiYXBpLnVzZXJzLmNyZWF0ZSIsImFwaS51c2Vycy5nZXQiLCJhcGkudXNlcnMubGlzdCIsImFwaS5wZXJtaXNzaW9ucy5nZXQiLCJhcGkucm9sZXMuZ2V0IiwiYXBpLnVzZXJzLnVwZGF0ZSIsImFwaS5wcm9maWxlcy5tZSIsImFwaS5yb2xlcy5jcmVhdGUiLCJhcGkudXNlcnMuZGVsZXRlIiwiYXBpLnBlcm1pc3Npb25zLmNyZWF0ZSIsImFwaS5yb2xlcy5saXN0Il0sImp0aSI6Ijc3YTAwOGY0LTA4MjAtNDYxOS1hNTBhLWFjYjExMjI0ODUzYSIsImNsaWVudF9pZCI6ImNsaWVudC1zdXBlciJ9.QyIgq2HmFDXUT-nKLXJLudA3yQP0bz_wi6Ru9lPizs4UbPOawxlLHQauapsdpN2yXd611pYILRUwGP9F8C_H-fJ28uBs7bRpCUN8dslwHovk-1K8Gs5wjyuibQ-YZQqfi_LxsE7-cNS2el7j86Pqfw_RAFnsr4A9wRi9oAu8tSJ611PGBSUimMssZRULzcbh7zHUbPhDHOPY2NR2YapETclyPQap5qmFcqJE1BpGt0ZtdpUb8qX_Rjw7Y8lHxjg5ux583sdZFlENZF1hf1SvJw0XaNgs2f9R-n0F8Y4zBJwqEA-8-OdrLFBdmYUpwR5xA83ohoZNi1QEhr-UedViBg

  5. Execute a GET request for a list of users

    • cURL command:

    curl --header "Authorization: Bearer PUT_ACCESS_TOKEN_HERE" http://localhost:8080/api/v1/users

    • cURL example:

    curl --header "Authorization: Bearer eyJhbGciOiJSUzI1NiIsIImV4cCI6MTUyNzQ4MDY0NywiYXV0aG9yaXRpZXMiOlsiYXBpLnBlcm1pc3Npb25zLmRlbGV0ZSIsImFwaS5yb2xlcy5kZWxldGUiLCJhcGkucm9sZXMudXBkYXRlIiwiYXBpLnBlcm1pc3Npb25zLmxpc3QiLCJhcGkucGVybWlzc2lvbnMudXBkYXRlIiwiYXBpLnVzZXJzLmNyZWF0ZSIsImFwaS51c2Vycy5nZXQiLCJhcGkudXNlcnMubGlzdCIsImFwaS5wZXJtaXNzaW9ucy5nZXQiLCJhcGkucm9sZXMuZ2V0IiwiYXBpLnVzZXJzLnVwZGF0ZSIsImFwaS5wcm9maWxlcy5tZSIsImFwaS5yb2xlcy5jcmVhdGUiLCJhcGkudXNlcnMuZGVsZXRlIiwiYXBpLnBlcm1pc3Npb25zLmNyZWF0ZSIsImFwaS5yb2xlcy5saXN0Il0sImp0aSI6Ijc3YTAwOGY0LTA4MjAtNDYxOS1hNTBhLWFjYjExMjI0ODUzYSIsImNsaWVudF9pZCI6ImNsaWVudC1zdXBlciJ9.QyIgq2HmFDXUT-nKLXJLudA3yQP0bz_wi6Ru9lPizs4UbPOawxlLHQauapsdpN2yXd611pYILRUwGP9F8C_H-fJ28uBs7bRpCUN8dslwHovk-1K8Gs5wjyuibQ-YZQqfi_LxsE7-cNS2el7j86Pqfw_RAFnsr4A9wRi9oAu8tSJ611PGBSUimMssZRULzcbh7zHUbPhDHOPY2NR2YapETclyPQap5qmFcqJE1BpGt0ZtdpUb8qX_Rjw7Y8lHxjg5ux583sdZFlENZF1hf1SvJw0XaNgs2f9R-n0F8Y4zBJwqEA-8-OdrLFBdmYUpwR5xA83ohoZNi1QEhr-UedViBg" http://localhost:8080/api/v1/users

  6. View API documentation

  7. Read Developer Documentation for more detailed instructions

Features

Web Services

  • HTTP Compliant RESTful API
    • Follows HTTP protocols for RESTful web services
    • Lightweight JSON requests and responses
    • See our Web Service Standards
  • Public API Status Service
    • Web service that provides general status of the API to the public
    • Helpful endpiont for automated monitoring
  • User Administration Services
    • Full suite of user administration web services (list, get, create, update, delete)
    • Secured access through role based security
  • Account Management Services
    • Users can update their account information themselves
    • Manage account settings
    • Password reset
  • API Documentation
    • Complete documentation for web services consumers
    • Includes detailed descriptions and example to quickly interact with the API

Security

  • OWASP Hacker Proof
    • Tested nightly against OWASP common hacks (particularly the top 10)
    • Tested nightly using 3rd party penetration testing services to ensure entperprise grade security!
  • OAuth2 Authentication
    • Bearer Token authentication configuraiton
    • SHA2 hash encrypted user password (when authenticating using the database)
    • Supports other authentication methods
  • Active Directory / LDAP Authentication
    • Extends OAuth2 to support authentication with an AD/LDAP system
    • Supports Enterprise SSO environments using AD/LDAP
  • Role Based Authorization
    • Custom role definitions to suit any situation
    • Supports granular security permissions
    • Full suite of role administration web services (list, get, create, update, delete)
  • Forgot Username / Password Support
    • Web services that allow users to reset their username and/or password
    • Validates a user via their email address
  • Auditing
    • Complete enterprise access and data auditing to meet compliance requirements
    • HTTP Request / Response logging to track user activity (anonymous and authenticated users)
    • Database change logging to track manipulation of data over time (anonymous and authenticated users)

Tech Stack

  • JSON RESTful Web Services
  • Spring Boot
    • Spring MVC / REST
    • Groovy
    • Spring Security
    • Hibernate
    • (auditing, logging, ...)
  • Database Neutral
    • Capable of integrating with any major database vendor (SQL Server, Oracle, DB2, MySQL, etc)
    • Database interactions follow SQL99 standards with no vendor specific database features relied upon
    • Liquibase database migrations produce on-demand SQL specific to the integrated database
  • Integrated Test Suite
    • Automated test coverage using Spock testing framework
    • Tests executed during every build to ensure high quality code coverage
  • Continuous Integration (CI)
    • Jenkins CI jobs able to invoke Gradle and apiDoc commands to build, test, and package
    • Jenkins jobs included with API source
    • Supports other CI environments like Team Foundation Server (TFS)

Developers

  • Team Protocols
    • Fast learning curve through clear documentation
    • Easy values, standards, best practices that most developers will agree to follow
  • Core Values
    • Documented core values that we believe will resonate with most development teams
    • Unifies teams and promotes healthy communication
    • See our Core Values documentation
  • Coding Standards
    • Industry accepted language coding standards
    • Best practices when developing within the code base
    • Standard enforced using static code analysis at build time (CodeNarc)
    • See our Development Team Standards

System Administrators

  • Deploy Instructions
    • Full instructions on how to properly build, test, and package the API app for deploy
    • Continuous Integration job templates for QA, UAT, and PROD
  • Docker Support
    • Preconfigured Dockerfile for deployment within Amazon Web Services environment
    • Generate a Docker bundle for distribution using built-in tasks
    • Customize to fit any environment
  • Amazon Web Services (AWS) - Elastic Beanstalk
    • Supports AWS Elastic Beanstalk using a Docker image
    • Run a build task to generate an AWS EB compatible .zip file
  • API Monitoring
    • Configure automated web uptime monitoring to use the Status Web Service
  • DevOps Ready
    • Ansible scripts for deploying the API Docker image to the Amazon Web Service (AWS) environment
    • Customize scripts to support any environment

voyage-api-java's People

Contributors

christianlawler avatar clawler21 avatar dependabot[bot] avatar harttamale avatar laxman513 avatar mannejkumar avatar roblouie avatar sbennett-lssinc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

voyage-api-java's Issues

Security: Password Policy Enforcement

Need to define the following customizable password policy:

  • Password Expires Days
    • xx days
    • 0 days means never expires
  • Password Policy
    • Interface class
    • Implement a few policies (At least 8 characters, One upper, One number, One Special)
    • Note: using a class instead of just a simple regex so that complex password policies can be created
  • Implement password policy on user.saveDetatched()
    • Return exception if the password policy doesn't match
    • Include the password policy requirements in the error message ?
    • On password change:
      • Reset user.isCredentialsExpired = false
      • Update user.passwordCreated to today's date
  • User Fields
    • passwordCreated - Date that the password was last created/changed
  • Password Expired Servlet Filter
    • Copy the pattern established with the User Verification Servlet Filter
    • Detect if the user has an expired password. Either of the following constitute an expired password.
      • user.isUserCredentialsExpired - manual override to force a user to reset their password
      • (user.passwordCreated + passwordResetDays) >= today - real-time monitoring of the expiration
    • Return exception and do not let any actions be completed until they reset password
    • Have an exclusion list on the filter so that /account/password can be updated
  • /account/password PUT
    • Update the currently logged in user's password
    • Fetch the user out of the database
    • Apply the password and call user.saveDetatched()

User Verification

Implement an API user verification process as outlined within the Security documentation at https://github.com/lssinc/voyage-api-java/blob/master/readme_docs/SECURITY.md#user-verification

API Endpoints

  • /v1/verify/send GET
    • sends a unique code to up to 5 mobile phones from the authenticated user profile
    • no parameters
    • requires the user to be authenticated
    • no permissions besides being authenticated
  • /v1/verify POST
    • receives a single body param: "code"
    • looks up the "code" against the authenticated user's mobile phone codes. If one matches, then the phone is "validated" and the user account is considered "validated"
      • user.is_verify_required = true/false
      • user_phone.verify_code = code value
      • user_phone.verify_code_expires_on = date
      • user_phone.is_validated = true/false
    • JSON encoded {"code":"234324"}
    • requires the user to be authenticated
    • no permissions besides being authenticated

Brute Force Attack: Lock OAuth2 Client Account After X Attempts

A brute force attack (BFA) is when an attacker makes thousands of attempts on an HTTP resource in an attempt to guess common values for an input. For example, guessing a username and password on a login page. In most cases there are many combinations of usernames and passwords, so the attacker needs to be able to execute as many combinations as possible per second to find matches (i.e. 3-5k+).

For certain apps, the OAuth2 client account should be locked after a number of failed attempts so that the user's data is protected if the failures are from an attacker. There are app scenarios where locking an OAuth2 client account would disrupt a legitimate client's ability to facilitate access to the API, so it's up to the app owner if they wish to enable this feature. Either way, it's important that the API provide the means to lock OAuth2 client accounts after a specific number of failed login attempts.

Requirements:

  • Store a 'enabled' true/false property in an external property config so that this feature can be turned on/off for any given server and any time.
  • Store a 'max login attempts' property in an external property config so that each app can determine the count that is appropriate
  • Create an event listener or http filter that monitors failed FORM login attempts
  • Create an event listener or http filter that monitors failed BASIC AUTH login attempts
  • Lock OAuth2 client feature approach
    • If the feature is enabled (based on external config property), then execute
    • Lookup the OAuth2 Client by the client identifier provided
    • If the Client exists, then increment the number of failed login attempts (the Client should have a "failedLoginAttempts" property stored on the Client object
    • Check if the client's new failedLoginAttempts > max failed login attempts (from property config), then lock the Client account (client.isLocked = true or client.isEnabled = false)
      • The client should have a locking or disable feature that would prevent further login
      • There is NO forgot password for clients, so the only way to unlock/enable a client account is for the administrator or DBA to manage this change.

Voyage App: User Services Sync

/src/app/core/user/user.service.ts

  • /profiles/me
    • GET User object
    • PUT User object
  • /users
    • GET list of User objects
  • /users/${userId}/account-status
    • PUT a UserStatus object

User

  • email: string;
  • firstName: string;
  • lastName: string;
  • username: string;
  • id: string;
  • isActive: boolean;
  • isVerifyRequired: boolean;
  • roles: Array;
  • phones: Array;
  • profileImage: string;

UserStatus

  • isActive: boolean;
  • isVerifyRequired: boolean;

Phone

  • phoneNumber: string;
  • phoneType: number;

Password Policy: No Reuse of Past Password

Enforce that users cannot reuse their current password when changing their password.

FUTURE: Consider storing past passwords to ensure that the user cannot use any of the past X passwords. This would require storing past passwords encrypted and comparing the new password hash with the old password hashes. At the moment, it might be too much to store old passwords and could be some sort of a liability? Perhaps worth researching on OWASP or other security forums.

/account POST: require at least 1 mobile phone when creating a new account

  1. Change the /account/register endpoint to simply be /account
    • There is no need to call out that it is a register method
    • This method should already be a POST
    • Only allow the creation of new users from this web service (not updates to existing)
    • Public method
  2. Require at least 1 mobile phone in order for the account to be created
    • Enforce international phone number E.164 (+165423422234)
  3. All new user accounts should have the "isVerifyRequired" flag to true so that they validate their phone number or email.

Account: Support adding/updating/deleting phone numbers

The proposed architecture for managing phone numbers for the currently logged in user is through the /account API endpoint. The /account should mirror the functionality and reuse the services created by the /users/:userId/phones endpoint, except that the current user should be used instead of providing a userId on the URL.

List: /account/phones GET
Get: /account/phones/:phoneId GET
Create: /account/phones POST {body}
Update: /account/phones/:phoneId PUT {body}
Delete: /account/phones/:phoneId DELETE

Requirements:

  • Implement the web service endpoints defined above
  • Restrict access to authenticated users (no other permissions required)
  • All of the Phone IDs passed in MUST be validated as belonging to the current user!
  • Reuse the UserService methods created for the /users/:userId/phones endpoints, but override the methods to retrieve the current user and pass that user into the method.

Notifications

GET /notifications

  • Get all notifications for the current user
  • No input
  • Body Output
    ** id: number;
    ** subject: string;
    ** description: string;
    ** isRead: boolean;
    ** createdBy: string;
    ** createdDate: Date;

PUT /notifications/${id}

  • Mark the given notification as "read"
  • Restrict to notifications for the current user
  • No input
  • No body output

PUT /notifications

  • Mark all notifications of the current user as "read"
  • No input
  • No body output

OAuth2: Client redirect URL error message is displaying all valid redirect URLs for the client

When a client passes in an invalid redirect URL in an OAuth2 Implicit Authorization request, eventually in the process the OAuth2 service will validate the URL to see if it matches what is in the client's profile. If the given client redirect URL does not match a URL in the client's profile in the database, then an error page is displayed. In the error page, the error text currently displays the valid redirect URLs for the client, which is revealing information to a potential hacker on which URLs will get them past the validation check.

Task:

  • Update the invalid client redirect URL error message to NOT display the valid client URLs.
  • Only display that the client redirect URL was not valid
  • Update the error message to NOT echo back the client redirect URL in the error message as the URL might contain malicious coding that would be executed on the user's browser.
  • Display instructions for the user on what to do in order to resolve the problem.

Security: Password Attempts Restriction

Create a Servlet Filter that tracks when an incorrect password was provided. When more than X passwords have been entered within Y minutes, then ban access to the account for Z minutes. Allow the variables to be configured within the application.yaml file.

SPIKE: How to detect when an incorrect password occurs?

  • Do we log these in the action logs?
  • Can we extend a Spring Security class to intercept the request and detect when a bad password is entered?
  • Can we catch an exception in the DefaultExceptionHandler and update the user record with a count increment?
  • Need to investigate these options to determine how best to detect a bad password.

User: Support adding/updating/removing phone numbers for a user profile

The proposed architecture for managing phone numbers for a user profile is:

List: /users/:userId/phones GET
Get: /users/:userId/phones/:phoneId GET
Create: /users/:userId/phones POST {body}
Update: /users/:userId/phones/:phoneId PUT {body}
Delete: /users/:userId/phones/:phoneId DELETE

Requirements:

  • Restrict to authenticated users
  • Add a permission name and register within the database.
  • Default to only the Super User role having access to this endpoint
  • All of the Phone IDs passed in MUST be validated as belonging to the given user!
    • Be mindful that an attacker could give a phoneId that belongs to a different user
    • Throw an exception when the phoneId doesn't belong to the given user

Docs: Security

  1. Complete the notes within the SECURITY.md file
  2. Link within the main README.md
  3. Describe how to test OAUth2 implicit pattern (Postman doesn't support)
  4. Describe how you can create a user with the same username as a client and it will load both the user and the client?

Activity Throttle Log

Create a Service that can monitor activities within the code and prevent them from occurring too many times within a period of time. For example, if a user tries to reset their password more than 1x per minute or 30 times in an hour, then throw an exception blocking that action from happening until the time duration threshold is met. This will reduce the efficacy of brute force attacks (BFA) on activities that are sensitive in nature.

Areas of Voyage that need this feature are:

  • Password reset request /password/forgot
    • do not allow an email to be requested for reset more than ____ in ___ minutes
  • Password reset submit /password/submit
    • do not allow an email to be requested for reset more than ____ in ___ minutes

Tech Notes

  • Create a base abstract class in security.bfa package called something like "ActivityThrottleLog"
    • This would be a hibernate domain object
    • fields:
      • TYPE - a unique type code
      • VALUE - the user provided value to monitor
      • MAX COUNT - the maximum number of instances allowed for the TYPE + VALUE
      • MAX MINUTES - the maximum number of minutes for the MAX COUNT to occur (ie 30 minutes for 100 password reset requests for email [email protected])
      • CREATED - date the record was inserted
      • CREATED BY - user Id that created the record
      • Method to validate if the limits have been met based on MAX COUNT & MAX MINUTES
    • Any app area that needs this functionality would extend this class and implement:
      • TYPE: unique name of the subclass activity
      • VALUE: constructor property when creating new instances of this type
      • MAX COUNT: a static value for the maximum occurrences
      • MAX MINUTES: a static value for the maximum allowed time
  • Create a ActivityThrottleLogService that would insert new records and validate the limits

Docs: Security > Brute Force Attack

Brute Force Attack (BFA) is a new set of features to Voyage and requires a new section that describes BFA techniques employed.

New Security sections:

  • Brute Force Attack (BFA) Overview
  • User Account Locking after X attempts
  • Client Account Locking after X attempts
  • Sleep X Seconds after failed requests

Brute Force Attack: Sleep random seconds after a failed request

A brute force attack (BFA) is when an attacker makes thousands of attempts on an HTTP resource in an attempt to guess common values for an input. For example, guessing a username and password on a login page. In most cases there are many combinations of usernames and passwords, so the attacker needs to be able to execute as many combinations as possible per second to find matches (i.e. 3-5k+).

To make the API a less desirable target for BFAs, after specific HTTP status codes are returned from the API, pause the response for a random number of seconds. Randomly slowing down the API will prevent the number of requests that an attacker can send and still get a result. If we can slow down an attacker by 3-8 seconds per request, then the API can potentially extend the time it would take to break into an account by decades of time.

Requirements:

  • Store a 'enabled' true/false property in an external property config so that this filter can be turned on/off for any given server and any time.
  • Store a list of http status codes in an external property config that should be paused if they are returned by the API
  • Store a minimum and maximum seconds to pause configuration property in an external property config
  • Create a filter that watches for outbound response HTTP status codes and matches the code against a list of valid 'pause' http status codes
    • If the filter is enabled, then execute the filter
    • If the status code matches, then pause the thread for a random number of seconds
    • don't allow the thread to sleep less than the minimum or more than the maximum

Security: Forgot Password Web Services

  • Can be disabled within the application.yaml file for internal applications with a centralized Password Recovery process.

Workflow

  1. User initiates the password recovery process from a link within the web app
  2. User is required to enter their username and mobile phone used to create the account
  3. Upon successful verification of the username and mobile phone number, the user is presented with a set of security questions
  4. The user must answer each question exactly as they had entered it within their account profile and submit
  5. If the answers are correct, then the user is displayed a

Security Questions

  • Security questions are presented at random

Technical Notes:

  • Need to do this without authenticating a user. If we authenticate a user, then they would get a token and have free reign to attempt other attacks
  1. Once a user enters a valid username and phone number, then return a Recovery Verify Token
    • POST /recover/password { username: blah, password: ***** }
    • Anonymous access web service
    • 3 attempts from for a given username (existing or not) will disable attempts for 10 minutes

    • 6 attempts from for a given username (existing or not) within a 60 minute period will disable attempts for the username for 24-hours

    • Track all attempts within the action_log so that non-existing usernames can be tracked as well
    • The Recovery Verify Token is only good for 20 minutes (shorter?) and should be stored on the user profile for date expiration
    • Use a long hash token that wont likely be replicated between multiple users ("account recovery" + userid + datetime)
    • Send verification code to mobile phone on record
      • Sets user.is_verify_required with code and expiration
      • Next time user logs in with their username and password, they will be required to verify
  2. Accept verification code
    • POST /recover/verify { recovery_verify_token: ANDJDUS*#*, code: 3423432 }
    • anonymous access web service
    • 3 attempts from for a given recovery code or IP address (existing or not) will disable attempts for 10 minutes

    • Check that the recovery token has not expired
    • Follows User Verification process
    • Upon successful verification, return a Recovery Questions Token
  3. Request security questions
    • POST /recover/questions { recovery_questions_token: EOIUWORJSDFN#373432 }
    • Requires the account recovery token
    • Requires user.is_verify_required = false (otherwise returns an error status code with message)
    • 3 attempts with an invalid account recovery token from a given IP address will ban the IP Address for 60 minutes (configurable)

    • Returns 3 of the 5 security questions (2 canned, 1 custom / out of 3 canned and 2 custom) rotated
    • Each question should have a question ID hash key that is generated using the question_code or the question text (and stored in the database or on-the-fly?)
      [
         {"question_id": "LKSJFSDJ*#&SAAHANDL*", "question": "What is the avg airspeed velocity of an unladen swallow?", "question_code": "avg_airspeed_swallow"},
         {"question_id": "&#$DSDJ356", "question": "What was your favorite teacher's last name?", "question_code": "favorite_teacher"},
         {"question_id": ")(SKSDFJLH#$", "question": "When is my favorite day of the year?", "question_code": "favorite_day_of_year"}
      ]
      
  4. Verify security questions
    • POST /recover/questions/answers { recover_questions_token: EOIUWORJSDFN#373432, questions: [ {question_id: &#$DSDJ356, answer: "my answer", question_id: ... }]
    • Requires the account recovery token
    • Requires user.is_verify_required = false (otherwise returns an error status code with message)
    • 3 attempts with an invalid account recovery token from a given IP address will ban the IP Address for 60 minutes (configurable)

    • All questions must have exact answers that match one-way hashed answers in the database
    • 3 attempts with an invalid security answers will ban the recovery token for 10 minutes

    • RETURNS a Recovery Change Password Token
  5. Change Password
    • POST /recover/password/change { recovery_change_password_token: 34987DHKWJHWERNAHQH, new_password: "changeme" }
    • sets user.is_credentials_expired = false (if it was set to true)
  6. Redirect user to the login page
    • Should be able to login fine without any sort of user verification since this was handled during the extensive password recovery process.

NOTE: UserPasswordExpiredServletFilter will intercept this and force password reset process. Unauthenticated users will use this path, authenticated users will be able to change their password at any time if they provider their current password first)

References

Action Logging: Exclude resources from action logging

Provide the ability to exclude certain URL resources from being logged. These resources might include unrestricted images, stylesheets, javascript files, or other static content that is not pertinent to user data or activity.

Requirements

  • Store a list of resource paths in an external property config that should be EXCLUDED from action logging.
    • Example using AntPath matcher: ['/resources/images/', '/static/css/']
  • Examine the incoming request URL and match the URL against the list of excluded resources
    • If the URL matches an excluded resource address, then skip the action logging.
    • If no match, then log the incoming request/response

Docs: AWS Configuration

  1. Update Developer workspace setup
  2. Describe overriding AWS configuration in the environment variables

Security: On startup, generate a new password for users using the default password

The API source comes bundled with migration scripts that include users with hardcoded passwords. If the API was immediately put into production without changing the password of the 'super user' account, then anyone could login with full privileges to the app. NOT GOOD!

To ensure that the default accounts never have the default password on start-up, create a bootstrap process that does the following:

  1. Finds all users with
    • Default role of 'role.super'
    • Default users created with the app: 'super', 'standard'
  2. Examines the passwords of each of the users to make sure they are not using the default password
    • Default password is password
  3. For users that have this default password, generate a new password
    • Generate a highly secure password with low likelihood of guessing
    • Update the user record with the new password (encrypted using the one-way hash algorithm in the app)
  4. Write the username + new password out to a log file for each user that has their password changed
    • Display the password in plaintext in the log file so that the admin can view the log file to know how to get into the system.

Example log file output:

Restricted Users found with default password. Generating new passwords:
User: super, Password: u(73Jdl;83jsEp38-997@!_
User: standard, Password: Ueje780-#123Iuyey

apiDoc: review all controller endpoints and update to meet clear documentation guidelines

  1. Review every Controller to make sure that they have apiDoc comments
  2. Read through all of the apiDoc documentation and make sure they have the following annotations in the order displayed below:
    • @api - correct - HTTP method, URL, and short name (i.e. {post} /v1/register Create account)
    • @apiversion - the current version of the endpoint (i.e. 1.0.0)
    • @apiName - a unique name for the endpoint (i.e. AccountCreate)
    • @APIGroup - the correct group name (i.e. Account)
    • @apiDescription - A full and complete description of the web service endpoint with as many helpful tips about how to use the endpoint, constraints, and other tips.
    • @apiPermission - the name of the permission that the user is required to have in order to access the web service endpoint.
      • Use "none" if no permissions are required and the service is public
      • Use "authenticated" if the user only needs to be authenticated
    • @apiHeader - REQUEST header parameters required (if any)
    • @apiHeaderExample - REQUEST example header parameters
    • @apiParam - input parameters supported by the web service endpoint
    • @apiexample - an example body post or get request
    • @apiHeader - RESPONSE header parameters required (if any)
    • @apiHeaderExample - RESPONSE example header parameters

Validate User Phone Number

When a user phone number is submitted on the POST /users endpoint, the user phones should be validated to be E.164 international phone format, like +16123366715.

Also document the APIDoc for the endpoint to explain that the phoneNumber must be in this format.

Write unit and integration tests that validate the various formats that are acceptable for phoneNumber.

If no AWS credentials are found, then write a warning log at startup and log SMS messages

By default, the Voyage API will not come bundled with AWS credentials. In order to support a 5-minute test project where a casual developer can get to know Voyage quickly, we want to provide a way to bypass AWS features. To date, the only AWS feature supported is SMS messaging via the Simple Notification Service (SNS).

Requirements:

  • At startup, if no AWS credentials are found, then write a WARN message to the logs that states that AWS is not configured and that all content sent through AWS services will be written to the logs.
  • When SMS messages are sent through to AWS, detect if the AWS credentials are available, if not, then write the SMS message to the logs as a WARN.

Docs: Action Logging Update

Update existing Action Logging documentation with new additions:

  • Excluding resources from action logging
  • Masking sensitive data
  • Enabling request body storage
  • Enabling response body storage
  • Detecting usernames for storing in the "username" field

Default database configuration is embedded H2 database

To support a 5-minute setup configuration for exploratory users, we want to default the database configuration to use an H2 database. It's optional as to whether the H2 database is in-memory or writes to disk.

Requirements

  • Default the database configuration to use an H2 database
    • H2 should already be bundled into the dependencies since we use it during automated tests
  • Use an in-memory database initially
    • The downside is that it will remove all data on restart
    • The upside is that we won't have to explain to users where to delete the H2 database file if they want to start over.

Security: Enforce unique mobile phone numbers database-wide

Voyage API is moving towards the pattern where mobile phones will be the primary means of validating a user and communicating sensitive information. In the past e-mail was the primary method of sending verification codes, but e-mail has proven insecure. A mobile phone is always with a user, and once validated, can be a more trusted source for security verification codes.

  1. Add a new field to UserPhone called "isVerified"
  2. When a mobile phone is successfully verified via the User Verification process, then mark the UserPhone with "isVerified" = true
  3. Ensure that all UserPhone records being added to the database have a unique phone number
    • Before inserting the UserPhone into the database, do a database lookup of the phone number
    • The phone number must be in the e.164 international format to ensure we can match it consistently
    • Only compare the phone number against UserPhone records that are "isVerified" = true and "isDeleted" = false. (The first to successfully verify the phone number 'owns' that phone, but not before).
  4. Immediately after a phone number has been validated, then mark any UserPhone records with the same phone number and isVerified = false to be logically deleted (isDeleted = true).
    • This will ensure that other users who entered the phone number in the past (and didn't validate the number) cannot attempt to complete the user verification process with that number.
    • Users who has this phone number before will not be able to add the phone number to their profile because we will be enforcing unique phone numbers in the database.

Brute Force Attack: Lock User Account After X Attempts

A brute force attack (BFA) is when an attacker makes thousands of attempts on an HTTP resource in an attempt to guess common values for an input. For example, guessing a username and password on a login page. In most cases there are many combinations of usernames and passwords, so the attacker needs to be able to execute as many combinations as possible per second to find matches (i.e. 3-5k+).

For certain apps, the user account should be locked after a number of failed attempts so that the user's data is protected if the failures are from an attacker. There are app scenarios where locking a user account would disrupt a legitimate user's ability to use their own account, so it's up to the app owner if they wish to enable this feature. Either way, it's important that the API provide the means to lock user accounts after a specific number of failed login attempts.

Requirements:

  • Store a 'enabled' true/false property in an external property config so that this feature can be turned on/off for any given server and any time.
  • Store a 'max login attempts' property in an external property config so that each app can determine the count that is appropriate
  • Create an event listener or http filter that monitors failed FORM login attempts
  • Create an event listener or http filter that monitors failed BASIC AUTH login attempts
  • Lock user feature approach
    • If the feature is enabled (based on external config property), then execute
    • Lookup the User by the username provided
    • If the User exists, then increment the number of failed login attempts (the User should have a "failedLoginAttempts" property stored on the User object
    • Check if the user's new failedLoginAttempts > max failed login attempts (from property config), then lock the user account (user.isLocked = true or user.isEnabled = false)
      • The user should have a locking or disable feature that would prevent further login
      • The user should be able to complete the forgot password process to unlock/enable their account

Forgot Password

Workflow

NOTE: The entire workflow should be completed outside of an authenticated session. Do not allow the user to ever be authenticated during this process because that would give them access to probe the API and other areas of the server! Once the password reset process is complete, then force the user to authenticate with their new credentials.

  1. User clicks "Forgot Password" from the Login page
  2. A form is displayed with the email address of the user on file
    • The form is always submitted successfully and a message is returned saying "an email with instructions has been sent..."
  3. An email appears in the user's inbox with a link to reset the password
    • The link has a token that is only good for X minutes
    • a password reset page is displayed if the token is good
    • a token timeout page is displayed if the token expired
  4. Password reset page displays a New Password and Confirm Password entry
    • Submit will update the password
    • Send an email with a password change notification
    • Send an SMS message with a password change notification
    • -- coinbase auto authenticates the user ---
    • Force SMS User Verification process for the user
  5. Display a User Verification screen
    • Ask the user to click a link to send the SMS code
    • ask the user to enter the code
    • Verify the user and redirect to the dashboard

OAuth2: Multiple client_redirect records are not being recognized

Spring Security OAuth2 appears to support multiple redirect urls for a client. The current database is structured to allow for multiple redirect urls in client_redirect table. Investigation needs to be done on how the lookup occurs within the database to determine if the given redirect url within the request matches a record within the client_redirect table for the current client.

Currently, seems like only the first record in the database is compared against the redirect_url given in the request.

User Registration

POST /profiles/register

  • Creates a new user account
  • Input is a Register object
    • email: string;
    • firstName: string;
    • lastName: string;
    • username: string;
    • password: string;
    • confirmPassword: string;
    • phones: Array;
  • Output appears to be a status confirmation ?

Login: Apply Voyage UI layout to the login page

The current login page is a generic gray background using a bootstrap theme. Update the visual look of the app to match the Angular mobile app. Include a Voyage logo and a simple banner.

Voyage App: Confirm Verification process

GET /verify/send

  • Send a code to the currently logged in user
  • No input
  • No body output

POST /verify

  • Post the verification code given from the user
  • No body output

Action Logging: Mask sensitive fields (i.e. passwords) before saving to the database

When saving headers, request body, response body, or even query parameters, it's possible that the consumer is sending sensitive information that should only ever be stored securely in the database. These data pieces cannot be stored in the action logs in clear text as it would be a violation of the security policy for the organization.

Within the Action Logging filter, create a parsing process that will scan the inbound content for parameter names, and if matches, will replace the parameter value with asterisks like *******.

Masking locations:

  • Request headers
    • Authorization:*********
  • Request Parameter
    • username=john&password=*********
  • Request Body FORM
    • username=john&password=*********
  • Request Body JSON (any depth)
    • {"username":"john", "password":"", "alternatePassword":[{"password":""}]

Enable / Disable storing Request Body to the database

  • In some cases the app might not want the request body to be stored to the database due to security or size of the request body.
  • Add an application property external to the source that will allow for the request body to be skipped during the logging.

Enable / Disable storing Response Body to the database

  • In some cases the app might not want the response body to be stored to the database due to security or size of the response body.
  • Add an application property external to the source that will allow for the response body to be skipped during the logging.

Write unit & integration tests that validate failed and success attempts.

User: Enforce international phone number format E.164

All phone numbers that are given to the API must be in E.164. This will set a standard within the database and allow for simplification of parsing for the API. The consumers will have the responsibility of following the E.164 format.

This might be unreasonable and too rigid for API consumers, especially if the app in question is a national app and the consumer would like some leniency to not be so formal. Consider adding the ability to pass in the locale to the API to do local parsing of the phone number and translation into an E.164 format.

The database should ALWAYS store in an international format by default.

Use the Google phone parser lib: libphonenumber
Source code at:
https://github.com/googlei18n/libphonenumber/blob/master/java/libphonenumber/src/com/google/i18n/phonenumbers/PhoneNumberUtil.java#L3027

Gradle include

compile group: 'com.googlecode.libphonenumber', name: 'libphonenumber', version: '8.4.0'

Example:

PhoneNumberUtil.parse('333-333-3333', 'US')

user roles and permissions endpoints are missing

the following endpoints need to be created in the UserController:

get /api/v1/users/{userId}/roles

post /api/v1/users/{userId}/roles

get /api/v1/users/{userId}/permissions

delete /api/v1/users/{userId}/roles/{roleId}

get /api/v1/users/{userId}/roles/{roleId}

Security: Create a common CryptoService that will hash encode as well as encrypt/decrypt

Provide a common service called CryptoService that can be injected into any class for the purpose of hashing or encrypting/decrypting data. This feature will allow developers to properly handle secure data with a simple service call, abstracting away the complexities of cryptography.

Requirements:

  • Hash encode plaintext using no less than bCrypt
  • Verify if a plaintext string matches a given hash encoded value
  • Encrypt plaintext using RSA 2048-bit public/private key
  • Decrypt plaintext using RSA 2048-bit public/private key

Security: Set a JWT token expiration date by user

JWT tokens are a strongly signed string of JSON text that contain the user ID, a token expiration date, and a few other bits that are helpful for authentication.

A deficiency of JWT is that the tokens cannot be expired faster than what is embedded into the token. The purpose of this issue ticket is to add in a Servlet filter that will intercept the authentication information embedded into the JWT token, check the associated user profile for a "tokens valid after date" value. Any token that is generated after the given date are good. Any token before the date are invalidated.

Tasks:

  1. Create a servlet filter that is ordered before Spring Security
  2. Get the Authentication header value from the HTTP request
  3. Validate the bearer token "Bearer SK8df87s..." token using the JWT built in services/utilities
    • the validation should check the signature of the token using the asymmetric keys defined in the Voyage API
  4. If token is valid, get the user_name and fetch the User from the database
  5. CHECK: User account is disabled or deleted
    • isEnabled == false
    • isAccountExpired == true
    • isAccountLocked == true
    • isDeleted == true
  6. CHECK: User token generation date is still valid
    • Requires adding a "created" value to the JWT
    • Requires adding a new User column called "access_tokens_valid_after_date"
    • Check that the "created" JWT token date is AFTER user.accessTokensValidAfterDate
  7. IF any of the CHECK situations fail, then return a generic Unauthorized error
    • 401 Unauthorized
    • Use the same error response as if someone tried to access a secured endpoint anonymously
    • Don't display any specific reason so that attackers cannot gain any additional information
  8. Test that the Servlet Filter exits the filter chain and an HTTP Request is immediately returned
    • A failure should completely abort any other operations!

Example JWT Token (current):

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE0OTExOTMyOTAsInVzZXJfbmFtZSI6InN1cGVyIiwiYXV0aG9yaXRpZXMiOlsiYXBpLnBlcm1pc3Npb25zLmRlbGV0ZSIsImFwaS5yb2xlcy5kZWxldGUiLCJhcGkucm9sZXMudXBkYXRlIiwiYXBpLnBlcm1pc3Npb25zLmxpc3QiLCJhcGkucGVybWlzc2lvbnMudXBkYXRlIiwiYXBpLnVzZXJzLmNyZWF0ZSIsImFwaS51c2Vycy5nZXQiLCJhcGkudXNlcnMubGlzdCIsImFwaS5wZXJtaXNzaW9ucy5nZXQiLCJhcGkucm9sZXMuZ2V0IiwiYXBpLnVzZXJzLnVwZGF0ZSIsImFwaS5yb2xlcy5jcmVhdGUiLCJhcGkudXNlcnMuZGVsZXRlIiwiYXBpLnBlcm1pc3Npb25zLmNyZWF0ZSIsImFwaS5yb2xlcy5saXN0Il0sImp0aSI6Ijk4MDZhMmJiLTdjYjgtNDE1ZC05YWFmLTJiOWY4ZTFiOTU3OCIsImNsaWVudF9pZCI6ImNsaWVudC1zdXBlciIsInNjb3BlIjpbIlJlYWQgRGF0YSIsIldyaXRlIERhdGEiXX0.gkobmaVCx6k0o_-vhnPYGtN3dv-lFgnLw7xhO3xYrUvHX8bAjDBpMsPW6ye18hpxFlI3DSjClC4uPMaB2HoREyRubJmsOo7ZL5scOA7IQt4BORczyBDAfKnAwMjUMQeGfNv1N-W92D5xUHnsln0r8GBn_gS3fvhep5SOcK_ujVo592ypz8o3Kcl3FMN3l4OKHQvfCma7ElMPWK-erJEuofpZLeTe9Wvx7Qo6EH1M0R0PVzu0Px3Dy23FuwMxmfJMEEh7iPxe0rkOdJHq21umtFQEYIo4PidAvuX9poM4_S3no6dLpsxSIhVXlpkANBdCzfOF62Pl8var_IBFIHmnpgeyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

Example JWT Payload (current):

{
  "exp": 1491193290,
  "user_name": "super",
  "authorities": [
    "api.permissions.delete",
    "api.roles.delete",
    "api.roles.update",
    "api.permissions.list",
    "api.permissions.update",
    "api.users.create",
    "api.users.get",
    "api.users.list",
    "api.permissions.get",
    "api.roles.get",
    "api.users.update",
    "api.roles.create",
    "api.users.delete",
    "api.permissions.create",
    "api.roles.list"
  ],
  "jti": "9806a2bb-7cb8-415d-9aaf-2b9f8e1b9578",
  "client_id": "client-super",
  "scope": [
    "Read Data",
    "Write Data"
  ]
}

Notifications API is missing from from java api

Create notifications resource

Notifications has the following properties:

  • id: number;
  • subject: string;
  • description: string;
  • isRead: boolean;
  • createdBy: string;
  • createdDate: Date;

Actions:

  1. GET notifications: returns all notifications for the logged in user
  2. PUT notifications/ : updates notification with id's read flag as true
  3. PUT notifications : sets all notifications for current user with read flag as true
  4. POST notifications: creates a notification for current user

Upgrade Path: Create separate projects for core voyage modules

In order for Voyage to be upgradable in the future, the app must follow the Spring Boot + Maven pattern where components are segmented into their own GitHub projects and published to Maven repositories for inclusion into the build.gradle file if necessary. Each project must be independent and not have Voyage circular dependencies.

When someone wishes to implement Voyage API, the base project will be very bare. The only files that should be included in the project are the build.gradle, liquibase migration, configuration files, application.properties, and anything else that is necessary to kickstart a project.

Tasks

  • Liquibase
    • How do the liquibase files get kicked off from the dependencies?
    • Should there be manual entries put into the main project?
    • What if we built something that would auto-detect liquibase main files 'db.changelog_master.yaml' and auto-include them. These would have to not be dependent on the order of execution... so it might be better to just have some way to manually include them... for visibility sake too
  • Configuration
    • Documentation should be good in each project on how to completely configure the module into an app
  • Create a Voyage website that lists out all of the projects, like Spring home page
    • There will need to be a Java API project
    • Within the Java API project will need to be links to the modules available. These should be GitHub pages that include everything necessary to configure the module
  • Projects
    • Common
    • Security
      • Depends on Common
    • Security OAuth2
      • depends on Security
      • Not sure OAuth2 can be separated out
    • Status
    • Account
      • depends on Security (user service)
  • Create a version number convention
  • Add a build job in Jenkins for each project
  • Publish to Maven for each project
  • Create GitHub Projects for each project

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.