Giter Club home page Giter Club logo

pacbot's Introduction

Latest release Build Status GitHub license GitHub contributors Gitter

Introduction

Policy as Code Bot (PacBot) is a platform for continuous compliance monitoring, compliance reporting and security automation for the cloud. In PacBot, security and compliance policies are implemented as code. All resources discovered by PacBot are evaluated against these policies to gauge policy conformance. The PacBot auto-fix framework provides the ability to automatically respond to policy violations by taking predefined actions. PacBot packs in powerful visualization features, giving a simplified view of compliance and making it easy to analyze and remediate policy violations. PacBot is more than a tool to manage cloud misconfiguration, it is a generic platform that can be used to do continuous compliance monitoring and reporting for any domain.

More Than Cloud Compliance Assessment

PacBot's plugin-based data ingestion architecture allows ingesting data from multiple sources. We have built plugins to pull data from Qualys Vulnerability Assessment Platform, Bitbucket, TrendMicro Deep Security, Tripwire, Venafi Certificate Management, Redhat Satellite, Spacewalk, Active Directory and several other custom-built internal solutions. We are working to open source these plugins and other tools as well. You could write rules based on data collected by these plugins to get a complete picture of your ecosystem and not just cloud misconfigurations. For example, within T-Mobile we have implemented a policy to mark all EC2 instances having one or more severity 5 (CVSS score > 7) vulnerabilities as non-compliant.

Quick Demo

How Does It Work?

Assess -> Report -> Remediate -> Repeat

Assess -> Report -> Remediate -> Repeat is PacBot's philosophy. PacBot discovers resources and assesses them against the policies implemented as code. All policy violations are recorded as an issue. Whenever an Auto-Fix hook is available with the policies, those auto-fixes are executed when the resources fail the evaluation. Policy violations cannot be closed manually, the issue has to be fixed at the source and PacBot will mark it closed in the next scan. Exceptions can be added to policy violations. Sticky exceptions (Exception based on resource attribute matching criteria) can be added to exempt similar resources that may be created in future.

PacBot's Asset Groups are a powerful way to visualize compliance. Asset Groups are created by defining one or more target resource's attribute matching criteria. For example, you could create an Asset Group of all running assets by defining criteria to match all EC2 instances with attribute instancestate.name=running. Any new EC2 instance launched after the creation of the Asset Group will be automatically included in the group. In PacBot UI you can select the scope of the portal to a specific asset group. All the data points shown in the PacBot portal will be confined to the selected Asset Group. Teams using cloud can set the scope of the portal to their application or org and focus only on their policy violations. This reduces noise and provides a clear picture to cloud users. At T-Mobile, we create an Asset Groups per stakeholder, per application, per AWS account, per Environment etc.

Asset groups can also be used to define the scope of rule executions as well. PacBot policies are implemented as one or more rules. These rules can be configured to run against all resources or a specific Asset Group. The rules will evaluate all resources in the asset group configured as the scope for the rule. This provides an opportunity to write policies which are very specific to an application or org. For example, some teams would like to enforce additional tagging standards apart from the global standards set for all of the cloud. They can implement such policies with custom rules and configure these rules to run only on their assets.

PacBot Key Capabilities

  • Continuous compliance assessment.
  • Detailed compliance reporting.
  • Auto-Fix for policy violations.
  • Omni Search - Ability to search all discovered resources.
  • Simplified policy violation tracking.
  • Self-Service portal.
  • Custom policies and custom auto-fix actions.
  • Dynamic asset grouping to view compliance.
  • Ability to create multiple compliance domains.
  • Exception management.
  • Email Digests.
  • Supports multiple AWS accounts.
  • Completely automated installer.
  • Customizable dashboards.
  • OAuth Support.
  • Azure AD integration for login.
  • Role-based access control.
  • Asset 360 degree.

Technology Stack

  • Front End - Angular
  • Backend End APIs, Jobs, Rules - Java
  • Installer - Python and Terraform

Deployment Stack

  • AWS ECS & ECR - For hosting UI and APIs
  • AWS Batch - For rules and resource collection jobs
  • AWS CloudWatch Rules - For rule trigger, scheduler
  • AWS Redshift - Data warehouse for all the inventory collected from multiple sources
  • AWS Elastic Search - Primary data store used by the web application
  • AWS RDS - For admin CRUD functionalities
  • AWS S3 - For storing inventory files and persistent storage of historical data
  • AWS Lambda - For gluing few components of PacBot

PacBot installer automatically launches all of these services and configures them. For detailed instruction on installation look at the installation documentation.

PacBot UI Dashboards & Widgets

  • Asset Group Selection Widget
  • Compliance Dashboard
  • Policy Compliance Page - S3 buckets public read access
  • Policy Compliance Trend Over Time
  • Asset Dashboard
  • Asset Dashboard - With Recommendations
  • Asset 360 / Asset Details Page

  • Linux Server Quarterly Patch Compliance
  • Omni-Search Page
  • Search Results Page With Results filtering
  • Tagging Compliance Summary Widget

Installation

Detailed installation instructions are available here

Usage

The installer will launch required AWS services listed in the installation instructions. After successful installation, open the UI load balancer URL. Log into the application using the credentials supplied during the installation. The results from the policy evaluation will start getting populated within an hour. Trendline widgets will be populated when there are at least two data points.

When you install PacBot, the AWS account where you install is the base account. PacBot installed on the base account can monitor other target AWS accounts. Refer to the instructions here to add new accounts to PacBot. By default base account will be monitored by PacBot.

Login as Admin user and go to the Admin page from the top menu. In the Admin section, you can

  1. Create/Manage Policies
  2. Create/Manage Rules and associate Rules with Policies
  3. Create/Manage Asset Groups
  4. Create/Manage Sticky Exception
  5. Manage Jobs
  6. Create/Manage Access Roles
  7. Manage PacBot Configurations

See detailed instruction with screenshots on how to use the admin feature here

User Guide / Wiki

Wiki is here.

Announcement Blog Post

Introducing PacBot

License

PacBot is open-sourced under the terms of section 7 of the Apache 2.0 license and is released AS-IS WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND.

pacbot's People

Contributors

0xflotus avatar adiagrwl94 avatar adiagrwl94mr avatar anilcs81 avatar anoopc444 avatar bobrich avatar codingalyona avatar dipesh1903 avatar dmpl1 avatar erichetti avatar gabel7r avatar hsdhott avatar janitha09 avatar johnakash avatar johnrex-j avatar johnrexj avatar kanchanaaradhya avatar kaykumar avatar mayurkr avatar nidhishkrishnan avatar pbaser avatar ritesh74 avatar robinjosephkurian avatar robint-mobile avatar sajeer-nooh avatar santhoshigorle avatar sonawanesangram avatar sukesh-ss avatar suryajak avatar varadh10 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pacbot's Issues

Integration with AWS Config as a Data Source

๐Ÿ™‹ feature request

AWS Configuration has some pretty powerful rules. Such as their Zelkova Ruleset for Public bucket reasoning

๐Ÿค” Expected Behavior

Would like the Compliance Stats from Config to be integrated into Pacbot, in order to see a holistic view of Compliance.

๐Ÿ˜ฏ Current Behavior

Unable to see any AWS Config Information

๐Ÿ’ Possible Solution

Since Config looks into the Resources it, may need to cross reference the items that are already being collected with the config result.

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-311

Remove Sun dependencies

Micro services

  • Spring boot Applications:

    • Admin Service
    • Auth Service
    • Asset Service
    • Compliance Service
    • Statistics Service
    • Notification Service

Rule Engine

Rules

ETL

Webapp

  • OS Type: Windows/Linux/MacOS

  • Java version:

    • 1.9
    • 1.8
    • 1.7

Summary


/bin/sh: -c: line 0: syntax error near unexpected token `>' /bin/sh: -c: line 0:`mvn clean &>>/var/log/pacbot/pacbot-install.log' /bin/sh: -c: line 0: syntax error near unexpected token `>' /bin/sh: -c: line 0:`mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V &>>/var/log/pacbot/pacbot-install.log' Traceback (most recent call last): File "setup_pacman.py", line 213, in _access_validation(assignedpolicieslist, user_name, user_arn) File "setup_pacman.py", line 190, in _access_validation awsterraformbuild._create_aws_resources(accessKey, secretKey, region) File "/Users/gabelevasseur/Repos/pacbot/installer/awsterraformbuild.py", line 28, in _create_aws_resources jsonRead._build_ui_apps(aws_access_key, aws_secret_key, region) File "/Users/gabelevasseur/Repos/pacbot/installer/jsonRead.py", line 760, in _build_ui_apps bucket=_get_s3_bucket_name() + '-' + _get_base_accountid() File "/Users/gabelevasseur/Repos/pacbot/installer/ui/build_apps.py", line 37, in build_api_and_ui_apps self.build_jar_and_ui_from_code() File "/Users/gabelevasseur/Repos/pacbot/installer/ui/build_apps.py", line 84, in build_jar_and_ui_from_code self.copy_jars_to_upload_dir(self.codebase_root_dir) File "/Users/gabelevasseur/Repos/pacbot/installer/ui/build_apps.py", line 109, in copy_jars_to_upload_dir shutil.copy2(copy_file_from, self.upload_dir) File "/Users/gabelevasseur/.pyenv/versions/2.7.5/lib/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/Users/gabelevasseur/.pyenv/versions/2.7.5/lib/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: IOError: [Errno 2] No such file or directory: '/Users/gabelevasseur/Repos/pacbot/dist/api/pacman-api-admin.jar' ``` I get this error during install as a result of using sun dependencies, as best practice they shouldn't be used anyways [https://www.oracle.com/technetwork/java/faq-sun-packages-142232.html](https://www.oracle.com/technetwork/java/faq-sun-packages-142232.html)

[https://github.com/tmobile/pacbot/pull/48](https://github.com/tmobile/pacbot/pull/48) created to address issue

## Reproduce steps

## Expected Results

## Actual Results


Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-309

Issues with IAM permissions

Upon trying to deploy, i have created an iam user with full admin access, generated keys, and started the install. I continuously receive:
Nececssary permissions are NOT avaiable!!!
the pacman-install.log logs only:
"Starting pacman installation"

[docs] How to add a new plugin ?

โ” Question

Any good docs on the code structure on how to add new providers there ? Code is slightly convoluted and it's quite hard to see what changes are required to add one - ideally they'd be plugin-like structure bundled in one dir, aren't they yet ?

๐Ÿ”ฆ Context

Just trying to add some more providers

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-315

Errors Experienced During Install on CentOS 7

โ” Question

Does anyone know how to resolve the issue below and can help me?

๐Ÿ”ฆ Context

Currently, I'm attempting to install pacbot but it currently fails during the install and quits. It states that "rule-engine.jar" is not available. I've attached both the printed return statement as well as the error components of the log file.

The printed error statement returned:

WARNING:python_terraform:error:
Error: Error applying plan:

1 error(s) occurred:

* null_resource.load_mysql_schema: Error running command 'mysql -u pacmanuser -ppacman123 -h pacmandbinstance.cqwfxlgk2owe.us-west-2.rds.amazonaws.com < DB_With_Values.sql': exit status 1. Output: ERROR 1064 (42000) at line 846: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO cf_AssetGroupDetails (groupId,groupName,dataSource,displayName,group' at line 5


Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.



oss-api  creation completed
Creating  lambda-submit
WARNING:python_terraform:error:
lambda-submit  creation completed
Execution started...............

Cloning the repository...............

Cloned repository to path: /tmp/pacman_cloned_dir_1541619661.6

Uploading Email templates to S3...............

Email templates upload to S3 completed...............

Started building the jar...............

Traceback (most recent call last):
  File "setup_pacman.py", line 213, in <module>
    _access_validation(assignedpolicieslist, user_name, user_arn)
  File "setup_pacman.py", line 190, in _access_validation
    awsterraformbuild._create_aws_resources(accessKey, secretKey, region)
  File "/home/centos/pacbot/installer/awsterraformbuild.py", line 27, in _create_aws_resources
    jsonRead._build_ui_apps(aws_access_key, aws_secret_key, region)
  File "/home/centos/pacbot/installer/jsonRead.py", line 596, in _build_ui_apps
    bucket=_get_s3_bucket_name() + '-' + _get_base_accountid()
  File "/home/centos/pacbot/installer/ui/build_apps.py", line 38, in build_api_and_ui_apps
    self.build_jar_and_ui_from_code(cloned_repo)
  File "/home/centos/pacbot/installer/ui/build_apps.py", line 101, in build_jar_and_ui_from_code
    self.copy_jars_to_upload_dir(working_dir)
  File "/home/centos/pacbot/installer/ui/build_apps.py", line 129, in copy_jars_to_upload_dir
    shutil.copy2(copy_file_from, self.upload_dir)
  File "/usr/lib64/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib64/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/tmp/pacman_cloned_dir_1541619661.6/dist/jobs/rule-engine.jar'

The contents of pacman-install.log are below:

_[INFO] Building jar: /tmp/pacman_cloned_dir_1541619661.6/jobs/pacman-rule-engine-2.0/target/rule-engine.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] api-commons ....................................... SUCCESS [25.295s]
[INFO] commons ........................................... SUCCESS [5.738s]
[INFO] PacMan Commons Projects ........................... SUCCESS [0.003s]
[INFO] admin-service ..................................... SUCCESS [31.389s]
[INFO] asset-service ..................................... SUCCESS [41.810s]
[INFO] config ............................................ SUCCESS [10.981s]
[INFO] compliance-service ................................ SUCCESS [15.418s]
[INFO] notification-service .............................. SUCCESS [7.346s]
[INFO] statistics-service ................................ SUCCESS [4.544s]
[INFO] auth-service ...................................... SUCCESS [5.614s]
[INFO] PacMan Api Projects ............................... SUCCESS [0.615s]
[INFO] inventoryFetch .................................... SUCCESS [1:19.782s]
[INFO] data-shipper ...................................... SUCCESS [7.865s]
[INFO] awsrules .......................................... SUCCESS [20.870s]
[INFO] pacman-rule-engine ................................ FAILURE [31.102s]
[INFO] PacMan Jobs Projects .............................. SKIPPED
[INFO] PacBot UI ......................................... SKIPPED
[INFO] PacMan ............................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:59.194s
[INFO] Finished at: Wed Nov 07 19:46:14 UTC 2018
Cleaning up unclosed ZipFile for archive /root/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar
[INFO] Final Memory: 237M/239M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.6:single (build-a) on project rule-engine: Failed to create assembly: Error creating assembly archive jar-with-dependencies: Problem creating jar: Execution exception: Java heap space -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :rule-engine
-es-data-shipper"
target_id: "" => "pacman-submitBatchjob"
aws_lambda_permission.lambda_invoke2: Creating...
action: "" => "lambda:InvokeFunction"
function_name: "" => "pacman-submitBatchjob"
principal: "" => "events.amazonaws.com"
source_arn: "" => "arn:aws:events:us-west-2:245459132561:rule/aws-redshift-es-data-shipper"
statement_id: "" => "AllowExecutionFromCloudWatch2"
aws_lambda_permission.lambda_invoke1: Creation complete after 0s (ID: AllowExecutionFromCloudWatch1)
aws_cloudwatch_event_target.target_for_cloudwatch: Creation complete after 0s (ID: AWS-Data-Collector-pacman-submitBatchjob)
aws_cloudwatch_event_target.pacman_redshift_for_cloudwatch: Creation complete after 0s (ID: aws-redshift-es-data-shipper-pacman-submitBatchjob)
aws_lambda_permission.lambda_invoke2: Creation complete after 0s (ID: AllowExecutionFromCloudWatch2)

Apply complete! Resources: 7 added, 0 changed, 0 destroyed._

๐Ÿ’ป Code Sample

Syntax error in installer/terraform/oss-api/DB.sql

Line 846 is missing a semi-colon, causes a syntax error on build

INSERT  INTO `cf_Datasource`(`dataSourceId`,`dataSourceName`,`dataSourceDesc`,`config`,`createdDate`,`modifiedDate`) VALUES (1,'aws','Amazon WebService','N/A','2017-08-01','2018-03-09')

should be

INSERT  INTO `cf_Datasource`(`dataSourceId`,`dataSourceName`,`dataSourceDesc`,`config`,`createdDate`,`modifiedDate`) VALUES (1,'aws','Amazon WebService','N/A','2017-08-01','2018-03-09');

mvn can't build

When running the python installer the java project is not built correctly causing it to fail. When i try to run mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V -e manually I get this error output

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project data-shipper: Compilation failure: Compilation failure: 
[ERROR] /Users/gabelevasseur/Repos/pacbot/jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/Util.java:[17,42] package javax.xml.bind.annotation.adapters does not exist
[ERROR] /Users/gabelevasseur/Repos/pacbot/jobs/pacman-data-shipper/src/main/java/com/tmobile/cso/pacman/datashipper/util/Util.java:[101,25] cannot find symbol
[ERROR]   symbol:   class HexBinaryAdapter
[ERROR]   location: class com.tmobile.cso.pacman.datashipper.util.Util
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project data-shipper: Compilation failure
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1029)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:137)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :data-shipper

I've also updated the pom file to contain the dependency as well:

	<!-- https://mvnrepository.com/artifact/javax.xml.bind/jaxb-api -->
    <dependencies>
        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>2.1</version>
        </dependency>
    </dependencies>

Something went wrong while we were fetching your data.

After installation, I dont see any data on the dashboard. Instead i see the below error message. Can you please help me out here.

Something went wrong while we were fetching your data.
There are no assets in this asset group

Thank you in advance.

Asset Dashboard not showing untagged resources

Micro services

-- Asset Service?

Java Version

  • 1.8

Summary

Navigating to the instance dashboard assets/asset-dashboard?ag=aws&domain=Infra%20%26%20Platforms

I see the list of resource types across the middle ENI (111) - EC2 (23) When i click on one on the left i see Assets By Application and on the right i see a graph. Currently I am not see any graph or at least the graph is blank Expected Behavior: A Graph of instances is shown

On the EC2 Screen i implemented the required tags, Application and Environment I can click on the number and it takes me to the list of instances that are tagged. But for instances that are not tagged "Without Application Tagged" when I click on the number i see nothing. It takes me to a page with filters on top, and even when i remove the filters i still do not see anything. Expected Behavior: I would see a list of the untagged instances

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-312

Issue Tracking Strategy

Noticed in a couple issues there was mention that it is already in your pipeline. i get that this is pretty new, so you would have a pretty large backlog in your internal tracking system. Wondering if you plan on keeping track of all of the issues in this repo along with having releases defined, etc. Would really help a lot for folks wanting to contribute.

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-314

Source Code Double Download

โ” Build Process Question

My understanding of the the Build Process steps are as follows.

  • Setup Prerequisites
  • Clone Source (Manually)
  • Modify the Resource.json
  • Start the install
    • Enter the Access Keys
    • Run the terraform to setup the core stuff
    • Clone the Source . (Through Python using ssh)
    • Run the maven build
    • Create more infra
    • upload the payloads

In this flow the code is downloaded twice in two places.
Seems like just setting a BUILD_DIR that points to the original cloned repo, if someone wants to.

PacBot as a zero-day attack informant

Disclaimer: Not my original idea, but idea is too good to be forgotten. I'll credit the person once i found out who. This was during the Q&A session in "Cloud Security with T-Mobile" last January 29, 2019.

๐Ÿ™‹ feature request

Whenever PacBot, detects an attack or vulnerability that might not have a fix yet. Let PacBot be the platform to spread the information regarding the attack/vulnerability.

๐Ÿค” Expected Behavior

Small scale: All PacBot users/subscribers can get immediate information on the discovery. Large scale: All Cloud Service users (meaning the public) will receive information on the recent discovery.

๐Ÿ˜ฏ Current Behavior

N/A

๐Ÿ’ Possible Solution

N/A

๐Ÿ”ฆ Context

This can be really useful as a unifying approach to all cloud service providers to combat attacks in the cloud. As PacBot is already open source, this new feature could further enhance PacBot as a foundational technology.

๐Ÿ’ป Examples

N/A

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-303

Adding client accounts

I have created the roles in both base and client account and made sure the trust is configured in the client account and modified the cloud watch event role with the client account number.

With that said, the UI isn't showing data from the client account where should I look for errors to troubleshoot this misery?

Cannot see the client account details on the Dashboard

Created the Roles on both account and added base account role ARN to Client account(As per the document) , but I still cannot able to see any data for client account on the dashboard. Lambda function ran without any errors.

Thank You

Terraform Plan / Terraform Apply Workflow -- Automated Build for Infra

Thanks for the great project

Would love to be able to just run terraform plan and terraform apply and set some variables
Seems like the folder could be modules with a just a main.tf that calls the modules.

Right now it is difficult to see what is being created.
Also because each folder has a state file, a build process would need to store them in separate folders.
Which would be unique to this particular build.

The idea would be to have an automated build, and as part of that approve the infra changes.

Batch jobs not getting executed

Hello Team,

I have Pacbot operational and I see 96 pacman-batch-jobs in runnable state and 1000+ jobs in pacman-rule-engine but they are not getting executed. I can see the ECS cluster pacman-rule-engine-env_Batch_890b6ec2-2e9a-3364-ac05-a8e3ce496abe and there are no any running tasks in it. The lambda functions are getting executed every hour and I can see a bunch of EC2 instances (all healthy with proper instance profile roles) but since the jobs are not getting executed, I cannot see any data of AWS accounts.

Can you please help to fix this or provide any guidelines? I have added 100+ AWS accounts as per installation guidelines and the setup is running for more than 2 days now.

Thanks

--

UnboundLocalError: local variable 'is_deleted' referenced before assignment

I am using Amazon Linux 2 and get this error:


*********************************** pacbot Installation Started! ***********************************


Creating baserole
Waiting for the role to refresh
Creating baserole
baserole creation completed
Creating security
-- Skipping security group creation as it already exists.
Creating s3
Creating es
WARNING:python_terraform:error:
Creating rds
WARNING:python_terraform:error:
Creating redshift
Traceback (most recent call last):
File "setup_pacman.py", line 213, in
_access_validation(assignedpolicieslist, user_name, user_arn)
File "setup_pacman.py", line 190, in _access_validation
awsterraformbuild._create_aws_resources(accessKey, secretKey, region)
File "/home/ec2-user/pacbot/installer/awsterraformbuild.py", line 47, in _create_aws_resources
if count != 2 and checkresources._check_resource(aws_access_key, aws_secret_key, region, resource, terraform) is True:
File "/home/ec2-user/pacbot/installer/checkresources.py", line 12, in _check_resource
return getattr(checkresources, 'check%s' % methodname)(accessKey, secretKey, region, resource, terraform)
File "/home/ec2-user/pacbot/installer/checkresources.py", line 245, in _check_redshift
_remove(resource, is_deleted)
UnboundLocalError: local variable 'is_deleted' referenced before assignment

"Download" failing under Policy Violations

Micro services

  • Spring boot Applications:
  • Admin Service
  • Auth Service
  • Asset Service
  • Compliance Service
  • Statistics Service
  • Notification Service

Rule Engine

Rules

ETL

Webapp

  • OS Type: Windows/Linux/MacOS
  • Java version:
  • 1.9
  • 1.8
  • 1.7

Summary

"Download" failing under Policy Violations

Reproduce steps

Compliance --> Policy Violations--> click Download

Expected Results

All violations should be downloaded to my PC

Actual Results

Download Failed. Please try later

Pac-bot is not compatible with java9

Micro services

  • Spring boot Applications:
  • Admin Service
  • Auth Service
  • Asset Service
  • Compliance Service
  • Statistics Service
  • Notification Service

Rule Engine

Rules

ETL

Webapp

  • OS Type: Windows/Linux/MacOS
  • Java version: 9
  • 1.9

Summary

Reproduce steps

Expected Results

<>

Actual Results

After some research, looks like Java9 is breaking the whole script.
screen shot 2018-11-20 at 11 11 18 am

Has anyone experience this and found a work around?

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project api-commons: Error creating properties files for forking; nested exception is java.io.IOException: No such file or directory

when i try to manually execute mvn install i get already a failure:
mvn install "-DskipTests=true -Dmaven.javadoc.skip=true -B -V"

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project api-commons: Error creating properties files for forking; nested exception is java.io.IOException: No such file or directory

Compliance Rules failing -

Micro services

  • Spring boot Applications:
  • Compliance Service

Rule Engine

Rules

ETL

Webapp

  • OS Type: Windows/Linux/MacOS
  • 1.8
  • 1.7

Summary

I noticed that in this environment i was 100% compliant.
Which surprised me, especially since it is sandbox.
So i tried creating a new Elastic IP and not using it to validate a failed rule state.
PacMan_UnusedElasticIpRule_version-1_UnusedElasticIpRule_elasticip-job

Reproduce steps

Added a new EIP and did not use it.

Expected Results

A Failed test based on the unused rule

Actual Results

All of the tests are passing.
I am bit surprised that things seem to be failing and i get a dashboard saying everything is passing.
The Pacman Rule Engine Job Stats are 1000+ Failed and 105 Succeeded.

Would seem like there is something broken with the deployment, but not sure what specifically i would need to fix.

Here are the logs
23:08:22,926 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@27ddd392 - Registering current configuration as safe fallback point
23:08:22,926 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@27ddd392 - Registering current configuration as safe fallback point
23:08:22
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
23:08:23
2018-10-29 23:08:23 [main] DEBUG c.t.pacman.executor.RuleExecutor - rule Param String {"autofix": false, "ruleType": "ManageRule", "alexaKeyword": "UnusedElasticIpRule", "ruleId": "PacMan_UnusedElasticIpRule_version-1_UnusedElasticIpRule_elasticip", "ruleRestUrl": "", "environmentVariables": [], "targetType": "elasticip", "ruleUUID": "09159bf1-a452-4746-bccf-6f9b162824ab", "params": [{"encrypt":
23:08:23
2018-10-29 23:08:23 [main] DEBUG c.t.pacman.executor.RuleExecutor - target Type :elasticip
23:08:23
2018-10-29 23:08:23 [main] DEBUG c.t.pacman.executor.RuleExecutor - rule Key : check-for-unused-elastic-ip
23:08:23
2018-10-29 23:08:23 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] DEBUG c.t.pacman.executor.RuleExecutor - uncaught exception handler engaged.
23:08:23
2018-10-29 23:08:23 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] DEBUG c.t.pacman.executor.RuleExecutor - shutdown hook engaged.
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] DEBUG com.tmobile.pacman.util.ESUtils - querying ES for target type:elasticip
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.CommonUtils - {"query":{"bool":{"must":[{"term":{"latest":"true"}}]}}}
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.CommonUtils - error closing issueunable to execute post request because Not Found
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.ESUtils - error getting total documents
23:08:24
java.lang.Exception: unable to execute post request because Not Found
23:08:24
at com.tmobile.pacman.util.CommonUtils.doHttpPost(CommonUtils.java:201)
23:08:24
at com.tmobile.pacman.util.ESUtils.getTotalDocumentCountForIndexAndType(ESUtils.java:143)
23:08:24
at com.tmobile.pacman.util.ESUtils.getResourcesFromEs(ESUtils.java:90)
23:08:24
at com.tmobile.pacman.executor.RuleExecutor.run(RuleExecutor.java:182)
23:08:24
at com.tmobile.pacman.executor.RuleExecutor.main(RuleExecutor.java:91)
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] DEBUG com.tmobile.pacman.util.ESUtils - total resource count-1
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] DEBUG com.tmobile.pacman.util.ESUtils - inventory query{"size":10000,"query":{"bool":{"must":[{"term":{"latest":"true"}}]}}}
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.CommonUtils - {"size":10000,"query":{"bool":{"must":[{"term":{"latest":"true"}}]}}}
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.CommonUtils - error closing issueunable to execute post request because Not Found
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR com.tmobile.pacman.util.ESUtils - error retrieving inventory from ES
23:08:24
java.lang.Exception: unable to execute post request because Not Found
23:08:24
at com.tmobile.pacman.util.CommonUtils.doHttpPost(CommonUtils.java:201)
23:08:24
at com.tmobile.pacman.util.ESUtils.getDataFromES(ESUtils.java:373)
23:08:24
at com.tmobile.pacman.util.ESUtils.getResourcesFromEs(ESUtils.java:92)
23:08:24
at com.tmobile.pacman.executor.RuleExecutor.run(RuleExecutor.java:182)
23:08:24
at com.tmobile.pacman.executor.RuleExecutor.main(RuleExecutor.java:91)
23:08:24
2018-10-29 23:08:24 2b571590-2e58-4ee3-a690-8f3e382c7283 [main] ERROR c.t.pacman.executor.RuleExecutor - unable to get inventory for aws-all--elasticip
23:08:24
java.lang.Exception: unable to execute post request because Not Found
23:08:24
at com.tmobile.pacman.util.CommonUtils.doHttpPost(CommonUtils.java:201)
23:08:24
at com.tmobile.pacman.util.ESUtils.getDataFromES(ESUtils.java:373)
23:08:24
at com.tmobile.pacman.util.ESUtils.getResourcesFromEs(ESUtils.java:92)
23:08:24
at com.tmobile.pacman.executor.RuleExecutor.run(RuleExecutor.java:182)

Error Building admin-service on Mac

Running into an error installing pacbot from a Mac
Not sure if the Mac part is relevant

The install fails when it is looking for the admin-service jar file
Upon further inspection i found this info in the log.

Error Message
[INFO] Reactor Summary:
[INFO]
[INFO] api-commons ........................................ SUCCESS [ 12.785 s]
[INFO] commons ............................................ SUCCESS [ 8.594 s]
[INFO] PacMan Commons Projects ............................ SUCCESS [ 0.343 s]
[INFO] admin-service ...................................... FAILURE [ 14.127 s]
[INFO] asset-service ...................................... SKIPPED
[INFO] config ............................................. SKIPPED
[INFO] compliance-service ................................. SKIPPED
[INFO] notification-service ............................... SKIPPED
[INFO] statistics-service ................................. SKIPPED
[INFO] auth-service ....................................... SKIPPED
[INFO] PacMan Api Projects ................................ SKIPPED
[INFO] inventoryFetch ..................................... SKIPPED
[INFO] data-shipper ....................................... SKIPPED
[INFO] awsrules ........................................... SKIPPED
[INFO] pacman-rule-engine ................................. SKIPPED
[INFO] PacMan Jobs Projects ............................... SKIPPED
[INFO] PacBot UI .......................................... SKIPPED
[INFO] PacMan 1.0.0-SNAPSHOT .............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 39.298 s
[INFO] Finished at: 2018-10-15T21:34:44-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:2.6:jar (default-jar) on project admin-service: Execution default-jar of goal org.apache.maven.plugins:maven-jar-plugin:2.6:jar failed: An API incompatibility was encountered while executing org.apache.maven.plugins:maven-jar-plugin:2.6:jar: java.lang.ExceptionInInitializerError: null

Environment Info
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T13:33:14-05:00)
Maven home: /usr/local/Cellar/maven/3.5.4/libexec
Java version: 11, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.14", arch: "x86_64", family: "mac"

Installation fails

Summary

When trying to install I get this error

Traceback (most recent call last):
  File "setup_pacman.py", line 213, in <module>
    _access_validation(assignedpolicieslist, user_name, user_arn)
  File "setup_pacman.py", line 190, in _access_validation
    awsterraformbuild._create_aws_resources(accessKey, secretKey, region)
  File "/Users/gabelevasseur/Repos/pacbot/installer/awsterraformbuild.py", line 30, in _create_aws_resources
    varsmap = varsdata._get_terraform_map(resource, "")
  File "/Users/gabelevasseur/Repos/pacbot/installer/varsdata.py", line 27, in _get_terraform_map
    if len(methodname)>1: value=getattr(jsonRead, methodname)()   
  File "/Users/gabelevasseur/Repos/pacbot/installer/jsonRead.py", line 336, in _get_base_accountid
    data = json.load(data_file)
  File "/Users/gabelevasseur/.pyenv/versions/2.7.5/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/Users/gabelevasseur/.pyenv/versions/2.7.5/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/Users/gabelevasseur/.pyenv/versions/2.7.5/lib/python2.7/json/decoder.py", line 368, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))```
## Reproduce steps
<!--steps to reproduce the issue-->


## Expected Results
<!--What did you expect to happen when running the steps above-->


## Actual Results
<!--What actually happened-->

Installer script possible sets up RDS incorrectly

Install Script

Summary

The install script seems to setup the RDS 'Roles' table with an incorrect type. roleId is set to an BIGINIT when i believe roleId is actually created as an UUID.

db.sql:
image

UserRolesServiceImpl.createUserRole:
image

Reproduce steps

Run the install script creating a fresh environment and see what the type of roleId is.
You can see a failure if you attempt to create a new Role in the web application.

Expected Results

I'm assuming it should be a VARCHAR() but maybe I'm missing something.

Actual Results

roleId is set to a BIGINT

Parameter validation failed:

I have cloned the pacbot project in my aws instance with ubuntu 16.04 LTS image. I successfully installed all the dependencies mentioned in the instructions. Created a vpc and 2 subnets in different availability zones as well. Entered the vpc id,cidr and subnets in resource.json.
But when I run the "python setup_pacman.py install" I enter the access key,security key and region, It exits with the below error:


Traceback (most recent call last):
File "setup_pacman.py", line 204, in
_get_user_managed_policies(iam, user_name)
File "setup_pacman.py", line 114, in _get_user_managed_policies
response = iam.list_attached_user_policies(UserName=user_name)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 586, in _make_api_call
api_params, operation_model, context=request_context)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 621, in _convert_to_request_dict
api_params, operation_model)
File "/usr/local/lib/python2.7/dist-packages/botocore/validate.py", line 291, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter UserName, value: None, type: <type 'NoneType'>, valid types: <type 'basestring'>


Kindly point me if I have done any mistake in the configuration part.

Terraform Error Handling

There are times when it makes sense to modify the terraform in order to customize certain parts.
An example is i want to make the alb public or something.
The current way the terraform is ran it will just roll over a syntax error and just keep creating everything. This creates a lot of problems and many times ends up in a full destroy recreate cycle.
I referenced the terraform plan/apply workflow in another issues, and this is related but a little different.

Python3 for installer

Today the installer is written in Python2.
Wondering if you have plans to migrate to Python3 or there are environment/library reasons that you did not.
I tried doing the conversion and i think it worked. Need to do some more testing.
If no problems i can submit a pr.

Easier way to create AGs based on accountIds

๐Ÿ™‹ feature request

Easier way to create AGs based on accountIds. A common use case for larger organizations with multiple accounts is to create dashboards/AGs per account/team.

๐Ÿค” Expected Behavior

Have the possibility to configure attributes such as accountId across all AWS resources simultaneously.

๐Ÿ˜ฏ Current Behavior

The user has to configure the accountId attribute under each individual resource (41 supported ones currently).

๐Ÿ’ Possible Solution

Have the possibility to configure the accountId attribute on all supported resources simultaneously.

Another approach could be to create a role on a consolidated billing account, have the base pacbot account assume that role and read all registered accounts under the billing account to later display all accounts in "Account management"- where the user can checkbox which accounts to collect data from.

๐Ÿ”ฆ Context

We want dashboards/AGs per account basis to better visualize compliance on team/account basis. Not being able to do this blocks users from onboarding/enabling accounts on pacbot.

๐Ÿ’ป Examples

It takes too long time to add the accountId attribute for each individual resource, lets say we have 10 accounts and growing. With the supported resources today its 410 individual configurations. Current workaround is to use the API - which may or may not slow down adoption of pacbot.

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-308

Npm Package install issue

When executing this command on amazon linux 2

sudo yum -y install java-1.8.0-openjdk npm docker maven wget unzip mysql

I get this returned

No package npm available.

When i run npm i get not found.

Asset Groups are not getting deleted, Is there a way to delete unwanted asset groups from UI

Micro services

  • Spring boot Applications:
  • Admin Service
  • Auth Service
  • Asset Service
  • Compliance Service
  • Statistics Service
  • Notification Service

Rule Engine

Rules

ETL

Webapp

  • OS Type: Windows/Linux/MacOS
  • Java version:
  • 1.9
  • 1.8
  • 1.7

Summary

Asset Groups are not getting deleted, Is there a way to delete unwanted asset groups from UI

Reproduce steps

  1. Create Asset Group, try deleting the same from UI

Expected Results

When Deleted, Asset Group should go away from my Dashboard

Actual Results

Saying, Deleted successfully but still showing up in UI

KMS Secret Management/ Parameter Store for Database Credentials

๐Ÿ™‹ feature request

When i look into the ECS Console for each container The RDS Password and Redshift Password are in plain text Not a super big deal since the environment is built into an internal vpc. But anyone who has AWS Access and access to the internal network could get at the credentials. Best Practices would be to store this in Parameter Store

๐Ÿค” Expected Behavior

These values should be stored in a KMS / Parameter Store and looked up via key.

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-310

Tagging Compliance Documentation

Not sure where/how to set the tags that are checked for tagging compliance.
It also seems like there are some key tags that pacbot uses in order to group resources.
Application and Environment seem to be two of them.
Would be good to document this process, also the resources that are created in the installer should be compliant out of the box. So they should have the key tags that Pacbot is looking in order to demonstrate tagging compliance

IOError: [Errno 2] No such file or directory: '/home/ec2-user/pacbot/dist/jobs/rule-engine.jar'

Started building the jar...............

Traceback (most recent call last):
File "setup_pacman.py", line 213, in
_access_validation(assignedpolicieslist, user_name, user_arn)
File "setup_pacman.py", line 190, in _access_validation
awsterraformbuild._create_aws_resources(accessKey, secretKey, region)
File "/home/ec2-user/pacbot/installer/awsterraformbuild.py", line 28, in _create_aws_resources
jsonRead._build_ui_apps(aws_access_key, aws_secret_key, region)
File "/home/ec2-user/pacbot/installer/jsonRead.py", line 760, in _build_ui_apps
bucket=_get_s3_bucket_name() + '-' + _get_base_accountid()
File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 37, in build_api_and_ui_apps
self.build_jar_and_ui_from_code()
File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 84, in build_jar_and_ui_from_code
self.copy_jars_to_upload_dir(self.codebase_root_dir)
File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 113, in copy_jars_to_upload_dir
shutil.copy2(copy_file_from, self.upload_dir)
File "/usr/lib64/python2.7/shutil.py", line 144, in copy2
copyfile(src, dst)
File "/usr/lib64/python2.7/shutil.py", line 96, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/home/ec2-user/pacbot/dist/jobs/rule-engine.jar'

Few errors

Hello Guys, Seems like a great product thanks for making it available for us.

I have a few questions for you, hopefully, you have answers for me.

1- I keep getting this error even though I can see the log file in var/log directory :

$ python setup_pacman.py destroy
Traceback (most recent call last):
File "setup_pacman.py", line 25, in
import awsterraformbuild
File "/home/ec2-user/pacbot/installer/awsterraformbuild.py", line 15, in
pacman_installation = filecreator.create_pacman_log_file_handler()
File "/home/ec2-user/pacbot/installer/filecreator.py", line 64, in create_pacman_log_file_handler
return open(get_logfile_path(), "a+")
IOError: [Errno 13] Permission denied: '/var/log/pacman/pacman-destroy.log'

2- Resources we provisions based on the log file since I got this message: "Apply complete! Resources: 7 added, 0 changed, 0 destroyed." and I can also see the resources in my account but auto-scaling keep terminating the EC2 instances.

3- I can clone the git repo manually but the program keeps erroring about authentication

3- Would stale resources prevent the destroy command from running I only get the error: "System existed"

Run installer without sudo

๐Ÿ™‹ feature request

This install script should be able to be run without using sudo permissions

๐Ÿค” Expected Behavior

๐Ÿ˜ฏ Current Behavior

๐Ÿ’ Possible Solution

Don't write to files that require elevated privileges, I think right now it's just the log files, put them in the directory of the project

๐Ÿ”ฆ Context

๐Ÿ’ป Examples

Pacbot login issue

Hi,
I was able to install pacbot and am able to access the internal ALB through port forwarding. When I use the default credentials either for admin or for user I get "Authentication Failed. Please check your credentials".

Looking at the network traffic for the login request I see this response.

{"success":false,"message":"Client Validation Failed!!!"}

Can you please help with this.

Aha! Link: https://t-mobile1t-mobile.aha.io/features/PM-523

IOError: [Errno 2] No such file or directory: '/home/ec2-user/pacbot/dist/api/pacman-api-admin.jar'

Pacbot stops at the below point when doing an install

Started building the jar...............

Traceback (most recent call last):
  File "setup_pacman.py", line 213, in <module>
    _access_validation(assignedpolicieslist, user_name, user_arn)
  File "setup_pacman.py", line 190, in _access_validation
    awsterraformbuild._create_aws_resources(accessKey, secretKey, region)
  File "/home/ec2-user/pacbot/installer/awsterraformbuild.py", line 28, in _create_aws_resources
    jsonRead._build_ui_apps(aws_access_key, aws_secret_key, region)
  File "/home/ec2-user/pacbot/installer/jsonRead.py", line 760, in _build_ui_apps
    bucket=_get_s3_bucket_name() + '-' + _get_base_accountid()
  File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 37, in build_api_and_ui_apps
    self.build_jar_and_ui_from_code()
  File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 84, in build_jar_and_ui_from_code
    self.copy_jars_to_upload_dir(self.codebase_root_dir)
  File "/home/ec2-user/pacbot/installer/ui/build_apps.py", line 109, in copy_jars_to_upload_dir
    shutil.copy2(copy_file_from, self.upload_dir)
  File "/usr/lib64/python2.7/shutil.py", line 144, in copy2
    copyfile(src, dst)
  File "/usr/lib64/python2.7/shutil.py", line 96, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/home/ec2-user/pacbot/dist/api/pacman-api-admin.jar'

unable to connect to the portal

I was able to install all resources successfully with no errors but I am unable to connect to the UI, anything I need to look at?


********** Login Domain: internal-pacman-alb-uijobs-012345678.us-east-1.elb.amazonaws.com
***************** Admin: [email protected] / pacman
****************** User: [email protected] / user


S3 Bucket Archive Folder Empty

Looking through the S3 bucket i noticed that there are folders for each deployment
backup/20181030-070210
But when i look in the folder there is nothing in there.
I think it is failing when gathering up the zip file to send up to S3.
Not a big deal, since the intent should be to have a pipeline archiving the bits.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.