shinesolutions / packer-aem Goto Github PK

Currently certs are retrieved from S3, which is not as convenient to manage compared to AWS Certificate Manager.

Need to add support to retrieve cert from certificate manager as an alternative to S3.

Create a sample project for using packer-aem with custom configuration and custom tags.
It should also make clear which custom configurations are mandatory to be overridden, which ones are optional.
Update configuration matrix accordingly.

As part of author and publish provisioning, the value of AEM Healthcheck package version should be configurable for easy future upgrades.

Packer template generic.json currently only allow source_ami override. It should also support instance type override by component.

Author and Publish baking require more powerful and expensive instance types, but the others should be on a cheaper one.

Since Inspec superseded serverspec's featureset, we need to replace serverspec with Inspec (or any other better tool really).

One benefit of moving to Inspec is the ability to publish AEM Inspec Profile to Chef marketplace, this allows multiple projects to use the same AEM spec checks, and that allows the automation code to deep AEM inspection rather than just relying on the limited checks that serverspec provides.
E.g. check whether replication agent exists or not.

Currently users need to override dependency configs, e.g. via timezone::, cloudwatchlogs::, and aem_curator:: .
This will break when we switch to new Puppet modules and users will have to keep updating their config. Which is already the case with AEM-related configs.

Improve the user experience here by abstracting the common user configs to use config::params:: , and modify the common hiera config to then do a lookup for those user configs.

Dispatcher AMI currently has Apache docroot on the root volume.
Considering it contains content, consistent to author and publish AMIs' repository, it should be moved to a separate volume which can be configured to have larger size.

The dispatcher docroot can also be considered a state to replicate during scaling process, or to restore to during recovery.

In order to avoid AEM reinstalling the same package in subsequent phase of environment runtime, we need to ensure that the install directory is empty at the end of image baking.

Need to add another check to inspec test script which checks the emptiness of the install directory.

The tomcat package including dependencies needs to be installed on the java component. Furthermore the service should be stopped and disabled.

Move temporary files on packer-launched instance to /tmp/shinesolutions/ in order to easily identify the files generated by packer-aem-bootstrap, and to clean up later on.

This is also consistent with namespacing stack building provisioning files in /opt/shinesolutions/

Currently both author and publish instances produces the same service name aem-aem .
The two have to be distinguished, e.g. author to have aem-author and publish to have aem-publish.

This is to allow a combination of author and publish on the same instance.

Even though the logs at /var/logs/shinesolutions/ have a very tiny growth rate, it's better to set a default log rotation against all log files within that directory.

Setting volume sizes is currently global.

Like to set these independently to make more efficient use of resources.

eg:

Dispatchers: 20g root, 50g data
Publish: 50g root, 200g data
Author 50g root, 400g data

etc.

newrelic-infra package is currently provisioned in init.pp .
NewRelic is user-specific and if user wants to use it, then it should be baked into user's base AMI (and any provisioning during stack creation time can live in custom stack provisioner).

packer-aem's scope is to support AWS and potentially other cloud providers in the future, but not various other external services.

Currently there are only two AEM setup options:

1. AEM 6.2 with SP1 + CFP3 + mandatory hotfixes
2. AEM 6.2 with SP1 + CFP5 + mandatory hotfixes

Some users requested to have the option to install 6.2 without hotfixes in order to test baseline AEM version.

Tags like Cost Center, Availability, Owner, Application ID, and Application Role are not used by the platform, they are user-specific sample tags, not relevant for everyone, and should be moved to examples/user-config/sandpit-global-tags.yaml .

It should be up to the user to define their own tags following their own naming convention.

cloudwatch_writer.py within Packer AEM https://github.com/shinesolutions/packer-aem/blob/master/provisioners/puppet/modules/config/files/cloudwatch_writer.py should be removed and the provisioning of CloudWatch Collectd plugin config should no longer use that standalone python file. Instead, it should use the file bundled within awslabs/collectd-cloudwatch , which is provisioned at

This is where the standalone cloudwatch_writer.py is used as collectd cloudwatch plugin python module

Extend platform test to check if the correct library versions are installed.

A number of users need to support multiple AEM environments at the same time, which means they'll need to manage AMIs with a combination of AEM profiles.

Need to add AEM Profile to AMI tags so users can filter which AMIs they need when they want to create an environment with specific AEM versions.

Currently OS-specific logic is part of a bash provisioner that's symlinked to the current location.
Rather then making user manage this symlink, simply expose the OS type as a configuration value so userland doesn't need to care about the underlying implementation.

Another reason for this ticket is that packer-aem is keeping a list of symlinks of AMI IDs that are mapped to specific OS. This is not manageable for two reasons:

1. source AMI IDs are often not public on userland, e.g. their own private SOE AMIs, whereas packer-aem is a public repo that can't have any knowledge of private userland info
2. how many AMI IDs do we want to keep there? we should simply support the OS, and not the AMI IDs, otherwise we'll end up with lots of symlinks there.

Userland should have the freedom to specify OS type without waiting for a source AMI ID to be available on packer-aem .
This will also remove AWS-specific mechanism (AMI ID) in order to support OS type across other cloud providers.

Currently Makefile package target enforces users to also lint and validate.
However, not all users are able to run this step on an environment where they can install the required tools (yes, really! :) ).

So in order to make it possible for users who can't install the required tools, and to those who can, we need to remove the lint and validate dependencies.
This allows the users who can't install the tools to still be able to package, while those who can install the tools can then explicitly call make lint validate package .

Development Tools package group installation was added in order to satisfy nokogiri dependencies for its native compilation, however, it installs some unnecessary OS packages.

We need to remove 'Development Tools' and replace only the necessary packages in

Every AMI creation currently involves yum upgrade in bash provisioner.
While it serves a value by updating packages to latest and greatest, it also serves a problem by having a different (newer) version of packages compared to the ones defined in the source AMI, specially when it's a security requirement to control the package versions inside an SOE AMI.
e.g. SOE AMI might contain package xyz to have version 123, but yum upgrade might then upgrade the version to 124 .

In this kind of environment, xyz package is expected to be upgraded inside the SOE AMI first, before then consumed by the child AMIs.

So rather than enforcing everyone to upgrade when they create the AMI, its better to provide a configuration value to control whether to upgrade the packages or not.

Root volume and data volume are currently set to a specific value, enough for installing AEM.
However, it doesn't work for scenarios where the source AMI already requires a larger volume size.

Volume sizes need to be made configurable, with a default that works for installing AEM on vanilla OSes.

Instead of using S3 as the storage for Author, Publish, and Dispatcher TLS cert's private key, investigate the use of AWS Secret Manager to store the key.

Creating the AMI with httpd running leaves a pid file lying around and generates this warning during stack init:
/etc/httpd/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?

Puppet manifests that are used within packer-aem should be extracted out to its own module and published to Puppet Forge.

In order to let the running SSM Agent daemon on linux contact the AWS API behind a proxy server, we need to add a Manifest which configured the proxy setting for the SSM Agent Daemon.

As described here:

https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-proxy-with-ssm-agent.html

The first test shows, that the configuration file for systemd seems to be the best solution.

Because hotfix installation that requires restart can intermittently fail, some users would like to simply install AEM without the hotfixes for non-production-like environments.

Even though most users would run config once from the base CF template, it's possible that some users would run config against a pre-configured state, and it currently ends up adding a tag even if it already exists.

This is also a common scenario during development. When we're testing the configuration for various permutations of aem versions and os types, config ends up getting executed multiple times.

Need to add a check so a tag won't be added if it already exists.

Currently lint, validate, and AMI baking targets forces dependencies download.
This needs to be made optional due to variety of user environments.

Some users sit behind unstable proxy and wouldn't want to always resolve dependencies.
Some users purposely decide to resolve dependencies one off only at the base AMI, while other components (author, publish, dispatcher, java, etc) must not need to resolve those dependencies by design.

Downloading dependencies should be isolated into its own target, and we're leaving it up to the user to choose whether they want to download the dependencies or not as part of lint, validate, and AMI baking targets.

packer-aem-bootstrap currently only supports RHEL7.
RHEL7-specific setup needs to be identified and extracted to a configuration.
Then introduce another set of configuration for Amazon Linux.

AEM installation intermittently fails with execution expired during AEM author/publish provisioning.

ami_author-publish-dispatcher: Notice: /Stage[main]/Config::Aem/Aem::Instance[publish]/Aem::Config[publish]/File[/opt/aem/publish/crx-quickstart/install]/ensure: created ami_author-publish-dispatcher: Info: Aem::Config[publish]: Scheduling refresh of Aem::Service[publish] ami_author-publish-dispatcher: Error: Could not set 'present' on ensure: execution expired at /tmp/packer-puppet-masterless-b/module-0/aem/manifests/instance.pp:137 ami_author-publish-dispatcher: Error: Could not set 'present' on ensure: execution expired at /tmp/packer-puppet-masterless-b/module-0/aem/manifests/instance.pp:137 ami_author-publish-dispatcher: Wrapped exception: ami_author-publish-dispatcher: execution expired ami_author-publish-dispatcher: Error: /Stage[main]/Config::Aem/Aem::Instance[publish]/Aem_installer[publish]/ensure: change from absent to present failed: Could not set 'present' on ensure: execution expired at /tmp/packer-puppet-masterless-b/module-0/aem/manifests/instance.pp:137

Due to some changes in the

cloudwatch.modules.flusher

module. flusher accepts now 3 arguments and not 2 anymore:

Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: [AmazonCloudWatchPlugin][cloudwatch.modules.configuration.confighelper] Fetched asg name as NONE
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Unhandled python exception in init callback: TypeError: init() takes exactly 3 arguments (2 given)
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Traceback (most recent call last):
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: File "/opt/collectd-cloudwatch/src/cloudwatch_writer.py", line 20, in aws_init#012 flusher = Flusher(config)
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: TypeError: init() takes exactly 3 arguments (2 given)
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: plugin_dispatch_values: No write callback has been registered. Please load at least one output plugin, if you want the collected data to be stored.
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Available write targets: [none]
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status 2 (ENOENT). Most likely this means you didn't load any write plugins.
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Available write targets: [none]
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Initialization complete, entering read-loop.
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Available write targets: [none]
Jan 19 12:08:56 ip-10-0-13-85 collectd[14688]: Available write targets: [none]

Since packer-aem already moved to generic JSON template for almost all components, the terminology repository_volume is not accurate because it's specific to AEM repository.

It should be renamed to data_volume because it can be the data of any application other than AEM components. It should also be renamed to data_volume for consistency with the terminology used in aem-aws-stack-builder.

Need to allow user to specify environment variables to be passed to Puppet command execution.

Sample use case scenario: user might have to bake the AMIs from behind proxy.

author-publish-dispatcher component runs out of disk space. This is due to baking being done against the root volume. This component should introduce sdb1 and sdb2 devices and mount them to author and publish repositories accordingly.

ami_author-publish-dispatcher: Error: Could not write report for ip-172-31-2-42.ap-southeast-2.compute.internal at /opt/puppetlabs/puppet/cache/reports/ip-172-31-2-42.ap-southeast-2.compute.internal/201710250525.yaml: No space left on device @ dir_s_mkdir - /opt/puppetlabs/puppet/cache/reports/ip-172-31-2-42.ap-southeast-2.compute.internal/201710250525.yaml20171025-109974-1u0qqel.lock ami_author-publish-dispatcher: Error: Could not send report: No space left on device @ dir_s_mkdir - /opt/puppetlabs/puppet/cache/state/last_run_report.yaml20171025-109974-zkvpui.lock

Dispatcher artifact download (up to version 4.2.3) used to be distributed via adobeaemcloud.com.
List: https://www.adobeaemcloud.com/content/companies/public/adobe/dispatcher/dispatcher.html Artifact: https://www.adobeaemcloud.com/content/companies/public/adobe/dispatcher/dispatcher/_jcr_content/top/download_9/file.res/dispatcher-apache2.4-linux-x86-64-ssl-4.2.3.tar.gz

However, the latest version 4.3.1 is not available from the same location.
The release notes https://helpx.adobe.com/experience-manager/dispatcher/release-notes.html now lists the download from macromedia.com http://download.macromedia.com/dispatcher/download/dispatcher-apache2.4-linux-x86_64-ssl-4.3.1.tar.gz

Packer AEM needs to switch the default download location to macromedia.com .

At the moment the Keystore test introduced with #95 will print out the keystore password in the results. To avoid this in the future we should use the option redact_regex. This option is available from InSpec release 2.2.36 on.

the aem keystore is located at /opt/aem/{author/publish}/crx-quickstart/ssl/aem.ks

We need to verify that the key has been added to the keystore with the alias cqse

describe command("keytool -list -keystore #{aem_keystore_path} -alias cqse -storepass {aem_keystore_password} ", redact_regex: /-storepass .*/) do
  its('exit_status') { should eq 0 }
end

The above command should work but was returning an exit status of 1.

In order to verify that Custom Image Provisioner feature works as expected, we need to add AEM Hello World Custom Image Provisioner package to integration test runs.

At the moment we have to rely on users who use custom stack provisioning feature to identify any potential issue.

Inventory files currently have a lot of sandpit environment config.
They should be replaced with placeholder values, and the sandpit config values should be moved to examples/ .

As of packer-aem v2.1.0, user configuration files include Hieradata and Packer variables configuration, and this leads to users having to keep modifying their config files as they upgrade Packer AEM.

In order to improve user experience with packer-aem upgrades, we need to introduce a generic config file, e.g. a simple YAML file that allows users to configure the most common config values, which will then be used to generate the Hieradata and Packer variables config (e.g. from templates).
This way users won't need to keep modifying their config when we change the underlying tech stack (e.g. changing Puppet modules to do a particular task).

Having said that, we should also leave the ability for power user to override the less common configs if need be using the Hieradata and Packer variables.

Also consider using Ansible to replace/manage the existing scripts, this provides consistency with aem-aws-stack-builder, and also provide a single interface for the users, opening up the possibility for users to have a single plain YAML config for using both packer-aem and aem-aws-stack-builder.

Currently only the running instance and resulting AMI + snapshots are tagged, while the runtime volumes that are used by the running instance is not.

Volumes need to be tagged as well, often used for billing purpose.

AWS resources CF template currently only contains S3 bucket and IAM instance profile + roles.
We need to add a blank parameter store secret as part of this CloudFormation template, for the user to then populate it with the AEM license.

Here's the aws-resources CF template https://github.com/shinesolutions/packer-aem/blob/master/templates/cloudformation/aws-resources.yaml

Naming convention wise, please keep them consistent to AEM AWS Stack Builder's CF templates https://github.com/shinesolutions/aem-aws-stack-builder/tree/master/templates/cloudformation

amazon-linux bash provisioner is currently just a placeholder https://github.com/shinesolutions/packer-aem/blob/master/provisioners/bash/amazon-linux.sh .

This needs to be made on-par with RHEL7. https://github.com/shinesolutions/packer-aem/blob/master/provisioners/bash/rhel7.sh

Amazon Linux 2 has been released, the PR that Ramses provided attempts to use Amazon Linux 2 but not yet done. The aim is to install Puppet5 using systemd, which was first introduced in Amazon Linux 2.

In order to help users with getting started, it would be handy to build a Docker image that contains all the requirements needed for running packer-aem .

As a (cheaper) alternative to RHEL7, some users would like to use CentOS 7 support, specially for development / non-production AEM environments.

Image provisioning (e.g. for AWS AMIs) currently only provides a single customisation point via the source image. It was assumed that users would place all customisations on the source image, which will then be consumed by packer-aem component AMIs provisioning.

However, there are use cases where the users need to provision some custom steps during the image provisioning process, so we need to introduce customisation points.

This customisation should follow the same pattern as AEM AWS Stack Builder:

• user can provide aem-custom-image-provisioner.tar.gz, place it under stage/custom/ directory (note: this needs to be a directory resource in order to allow non-customisation when the artifact doesn't exist, Packer does not allow provisioning with a file name if the file doesn't exist), and then introduce file provisioner on the Packer templates, before and after the AEM provisioning
• this artifact must contain a pre-common.sh and a post-common.sh scripts which will be executed before and after AEM provisioning
• the provisioning process should handle both scenario when custom provisioner is supplied and when it's not. this means if the custom provisioner pre and post scripts are executed from Packer template shell definition, then it should not break when there is no custom provisioner artifact, Executing the pre and post scripts from Puppet manifests is not preferred because it will enforce all steps to be chained within the manifests
• these shellscripts are generic, and we're not going to make any assumption whether it will run Puppet or shell or other tools like Ansible. If the user needs to execute Ansible, then it's up to the users either to install Ansible on the pre script, or on the source image.

We need to add a new configuration platform_type with default value of aws.
This is to anticipate future movement of adding support to non-aws-platform and the need to build non-ami machine images.