I'm trying to deploy the autoscale templates with DNS. The F5 devices look to be properly provisioned, licensed, and clustered. However, they never show up as GSLB pool members on my BIG-IP DNS box. When I login to the new F5 autoscale devices via CLI, I checked logs.

[admin@ip-10-0-11-18:Active:Standalone] ~ # cat /var/log/cloud/aws/custom-config.log | grep error
2018-10-28T22:04:44.074Z error: [pid: 25199] [scripts/autoscale.js] autoscaling error: providerOptions.poolName is required

When comparing the template, there is no parameter called provderOptions.poolName. There however a parameter called dnsProviderPool. Is the autoscale.js script looking for the wrong parameter name?

I know my GTM box is working fine because I can successfully run the Azure autoscale DNS templates, point to the same autoscale_pool name, target GTM boxes, etc. Something is throwing the autoscale.js error in the AWS templates.

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/autoscale/waf/via-dns/1nic/existing-stack/bigiq

same issue for ltm templates too

Description

Describe the problem you're having or the enhancement you'd like to request.

The present supported CFTs default to a fixed volume size of (I think) 30Gb. For some applications, this is not sufficient. Provide an option to make VolumeSize a parameter. We've manually altered the template with a static value, but would like to see it in the supported template.

Template

For bugs, enter the template with which you are experiencing issues below.

f5-existing-stack-across-az-cluster-byol-3nic-bigip.template

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: <Fill in level: 1 through 5>
3
Severity level definitions:

1. Severity 1 (Critical) : Defect is causing systems to be offline and/or nonfunctional. immediate attention is required.
2. Severity 2 (High) : Defect is causing major obstruction of system operations.
3. Severity 3 (Medium) : Defect is causing intermittent errors in system operations.
4. Severity 4 (Low) : Defect is causing infrequent interuptions in system operations.
5. Severity 5 (Trival) : Defect is not causing any interuptions to system operations, but none-the-less is a bug.

There are many outdated instance types included in the CFTs.

...
   "AllowedValues": [
    "t2.medium",
    "t2.large",
    "m3.large",
    "m3.xlarge",
    "m3.2xlarge",
    "m4.large",
    "m4.xlarge",
    "m4.2xlarge",
    "m4.4xlarge",
    "m4.10xlarge",
    "c3.xlarge",
    "c3.2xlarge",
    "c3.4xlarge",
    "c3.8xlarge",
    "c4.xlarge",
    "c4.2xlarge",
    "c4.4xlarge",
    "c4.8xlarge"
   ],
   "ConstraintDescription": "Must be a valid EC2 instance type for BIG-IP",
   "Default": "m3.2xlarge",
   "Description": "Size of the F5 BIG-IP Virtual Instance",
   "Type": "String"
  },
...

m3.* are no longer available in most regions. At the very least 'm3.*' should no longer be the default instance size.

Description

Some of customer want to use Auto Scale WAF solution with BIG-IQ license MGR(BYOL) so could you support us for satisfying this requirement thru CFT?

Template

Auto scaling the BIG-IP VE Web Application Firewall in AWS

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 2(not a bug but customer POC will be failed so i set 2)

Severity level definitions:

1. Severity 1 (Critical) : Defect is causing systems to be offline and/or nonfunctional. immediate attention is required.
2. Severity 2 (High) : Defect is causing major obstruction of system operations.
3. Severity 3 (Medium) : Defect is causing intermittent errors in system operations.
4. Severity 4 (Low) : Defect is causing infrequent interuptions in system operations.
5. Severity 5 (Trival) : Defect is not causing any interuptions to system operations, but none-the-less is a bug.

Hi,

Would it be possible to add an optional, boolean parameter to the CFTs - to install the latest AS3 version on the instances after stack deployment?

I'm currently using this: https://github.com/ArtiomL/f5networks/blob/master/scripts/instas3.sh

This would help with automating the actual BIG-IP config post-deployment, without the need to manually install AS3.

Thanks!

404:
https://github.com/F5Networks/f5-aws-cloudformation/blob/master/experimental/full-stack-byol-1nic-bigip.template

perhaps the correct URL?
https://github.com/F5Networks/f5-aws-cloudformation/blob/master/experimental/standalone/1nic/learning-stack/byol/f5-full-stack-byol-1nic-bigip.template

Description

After deploying the below template, log in and confirm the HA iApp settings. The "AZ1 VIP:" field is "/Common/" , which is missing the actual IP of the secondary IP used for the VIP on BIG-IP1 in AZ1.

Failover issues can arise if user is not aware that this field needs to be "reconfigured" in the iApp.

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/cluster/2nic/across-az-ha

Severity Level

Severity: 4

Troubleshooting:

I am looking here:
https://github.com/F5Networks/f5-aws-cloudformation/blob/d04fcc4ed7982410840cd456682a7dea068cab99/supported/cluster/2nic/across-az-ha/f5-existing-stack-across-az-cluster-byol-2nic-bigip.template

line 891
"\"tmsh create /sys application service HA_Across_AZs template f5.aws_advanced_ha.v1.4.0rc1 tables add { eip_mappings__mappings { column-names { eip az1_vip az2_vip } rows { { row { ${VIPEIP} /Common/${EXTPRIVIP} /Common/${PEER_EXTPRIVIP} } } } } } variables add { eip_mappings__inbound { value yes } }\"\n",

It should be grabbing the value of /Common/${EXTPRIVIP} , but I don't see it defined anywhere.
I think should have something like this somewhere:

"EXTPRIVIP='",
           {
            "Fn::Select": [
             "0",
             {
              "Fn::GetAtt": [
               "Bigip1subnet1Az1Interface",
               "SecondaryPrivateIpAddresses"
              ]
             }
            ]
},

Attempt to failover to the BIG-IP1 unit without this properly configured, and you will get below error, adn the EIP remap will not succeed despite the log saying "EIP takeover completed"....:

/var/log/ltm

Feb 18 21:49:28 ip-10-0-0-197 info aws_advanced_failover: EIP takeover started.
Feb 18 21:49:28 ip-10-0-0-197 notice logger: /usr/libexec/aws/aws-failover-tgactive.sh (traffic-group-1): Started.
Feb 18 21:49:28 ip-10-0-0-197 info aws_advanced_failover: Setting Environmental Variables.
Feb 18 21:49:28 ip-10-0-0-197 info aws_advanced_failover: Environmental Variables Set.
Feb 18 21:49:28 ip-10-0-0-197 info aws_advanced_failover: No Secret and Key found, attempting to use IAM
Feb 18 21:49:31 ip-10-0-0-197 err tmsh[2945]: 01420006:3: a single IP address must be specified.
Feb 18 21:49:31 ip-10-0-0-197 debug aws_advanced_failover: command: tmsh show net route lookup ended with status 1 had stdout: and stderr: Data Input Error: a single IP address must be specified.
Feb 18 21:49:31 ip-10-0-0-197 err aws_advanced_failover: Failed command: "tmsh show net route lookup "
Feb 18 21:49:31 ip-10-0-0-197 err aws_advanced_failover: There is no good matching VIP for EIP x.x.x.x <----!!!!!!!!!!!!!
Feb 18 21:49:31 ip-10-0-0-197 info aws_advanced_failover: No VIP to EIP mapping exists.
Feb 18 21:49:31 ip-10-0-0-197 info aws_advanced_failover: No reconfiguration of AWS routes was requested.
Feb 18 21:49:31 ip-10-0-0-197 info aws_advanced_failover: EIP takeover completed.

Description

The template allows the user to change the virtual server and application pool member ports to 443 or other (rather than 80). When you do this, you can't access the app server through the ELB unless you make manual changes.

Either:

• You should only allow 80
• You should only allow 80 and 443 (and it should work correctly for 443)

If you leave this as a free-form field, the user can enter any number, and I assume a number like 666 wouldn't work. (Although I haven't tested. : )

Here are the changes required to make 443 work:

1. The pool is using an http monitor. You must change it to https.
2. The virtual server needs an SSL Profile (Client) and SSL Profile (Server).
3. I think but am not sure that you should be able to upload a cert.

Template

F5Networks/f5-aws-cloudformation

Severity Level

Severity: 2 ? It doesn't work without this manual workaround.

The project should be configured to use sphinx to build documentation.

I deployed F5 across 2 AZs using your cloud formation template of 3 nics. However, when it comes to create HA, I can't assign the HA IP for the 2nd F5 in different AZ. What's the workaround?

Description

Running existing or new stack to license with BIG-IQ requires the BIG-IQ password to sit in a file in S3. If the S3 file/location is wrong, then there should be a more descriptive error. Currently, the onboard.log file says something like unable to get credentials. Instead, it should say something like S3 ARN path not found. Or better yet, the AWS CFT stack magic should check that the S3 ARN path exists first prior to kicking off the stack. This would avoid the whole boot up process then inevitable failure if S3 ARN path is wrong for the BIG-IQ password resulting in lack of password knowledge.

If S3 ARN path and permissions are correct, no issues...works great. Can you give a more detailed log other than basically saying that you can't grab the credentials.

To test, launch the stack with a purposely incorrect S3 ARN path. When the VE fails to license and onboarding is done, check onboard.log and you'll see the "can't get credentials" type message. I would like to see that message if indeed it can't get credentials of a valid path/file location. If however the path/file is completely wrong, then it should say something different like "wrong path/file".

Template

https://github.com/F5Networks/f5-aws-cloudformation/blob/master/supported/standalone/3nic/f5-existing-stack-bigiq-3nic-bigip.template

Other templates using BIG-IQ licensing would probably be affected too.

Severity Level

4. Severity 4 (Low) : Defect is causing infrequent interuptions in system operations.

Description

When using the cluster configuration it attempts to resolve the DNS name. If the VPC does not have DNS hostnames enabled this will fail.

2017-02-13T15:40:23.845Z verbose: tryUntil: getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443 tries left:
2017-02-13T15:40:23.846Z debug:  {"message":"getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443","stack":"Error: getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443\n    at errnoException (dns.js:26:10)\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)","code":"ENOTFOUND","errno":"ENOTFOUND","syscall":"getaddrinfo","hostname":"ip-10-1-1-230.ec2.internal","host":"ip-10-1-1-230.ec2.internal","port":443}
2017-02-13T15:40:23.847Z verbose: Max tries reached.
2017-02-13T15:40:23.847Z verbose: tryUntil: getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443 tries left:
2017-02-13T15:40:23.847Z debug:  {"message":"getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443","stack":"Error: getaddrinfo ENOTFOUND ip-10-1-1-230.ec2.internal ip-10-1-1-230.ec2.internal:443\n    at errnoException (dns.js:26:10)\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)","code":"ENOTFOUND","errno":"ENOTFOUND","syscall":"getaddrinfo","hostname":"ip-10-1-1-230.ec2.internal","host":"ip-10-1-1-230.ec2.internal","port":443}

aws ec2 describe-vpc-attribute --vpc-id "vpc-12d52674" --attribute enableDnsHostnames

{
    "VpcId": "vpc-12d52674",
    "EnableDnsHostnames": {
        "Value": false
    }
}

Template

For bugs, enter the template with which you are experiencing issues below.

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 4

Please update the referenced AMI. They refer to 12.1.0 HF2 vs. 12.1.1 HF1

Old AMIs. New Good BYOL AMI is ami-5e077c49

            "us-east-1": {
                "Best": "ami-83cab794",
                "Better": "ami-3fc4b928",
                "Good": "ami-e2c9b4f5"
},

Description

Templates referencing getNameServer.sh in latest revision are calling it from /config. This fails since file location is not found. One in particular is listed below. I didn't check others. I discovered this when deploying a CFT to have the VE auto license to BIG-IQ. After a long wait, I checked my VE and it was still unlicensed. I scrubbed various logs and here's what I found.

/var/log/cfn-init.log
2017-03-22 13:57:04,934 [DEBUG] Running command 003-onboard-BIG-IP
2017-03-22 13:57:04,934 [DEBUG] No test for command 003-onboard-BIG-IP
2017-03-22 13:57:04,977 [INFO] Command 003-onboard-BIG-IP succeeded
2017-03-22 13:57:04,977 [DEBUG] Command 003-onboard-BIG-IP output: /bin/sh: /config/getNameServer.sh: No such file or directory

As a result, nameserver never gets set. Licensing never happens. I also don't see any /var/log/onboard.log files as I use to see in previous CFT deployments on the VE. This makes sense seeing how the -o output command to /var/log/onboard.log is first called in part of the 003-onboard-BIG-IP section. If it fails finding getNameServer.sh, then it probably fails the rest.

In looking through my archive templates, the getNameServer.sh was called from the following location.
"NAME_SERVER=/config/cloud/aws/f5-cloud-libs/scripts/aws/getNameServer.sh eth1;"

I just checked my new VE but do not see a folder path for /config/cloud/aws/f5-cloud-libs/....

Template

https://github.com/F5Networks/f5-aws-cloudformation/blob/master/experimental/reference/2nic/bigiq/f5-existing-stack-bigiq-license-pool-2nic-bigip.template

branch: master
Github modification date shows Jan 23 2017

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 3

Description

Describe the problem you're having or the enhancement you'd like to request.

The AMI ids listed on https://github.com/F5Networks/f5-aws-cloudformation/blob/master/supported/standalone/2nic/f5-existing-stack-byol-2nic-bigip.template seem to have been discontinued. We were unable to launch this stack until we manually updated the ami ID for the ami we are using (we entered an AMI id found on the AWS marketplace for the BYOL image we were trying to use).

Template

For bugs, enter the template with which you are experiencing issues below.

https://github.com/F5Networks/f5-aws-cloudformation/blob/master/supported/standalone/2nic/f5-existing-stack-byol-2nic-bigip.template

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 5

I was not able to utilize an ALB when specifying my external load balancer for the WAF hourly template. Is this supported or can it be added as a feature in the future?

Description

Deployed BYOL CFT but stalls at creation of the autoscale group but eventually fails to create with error:

"Group did not stabilize... Failed Scaling Activity: In order to use this AWS Marketplace product you need to accept terms and subscribe.

So something to do with the EULA acceptance, but this normally is automated...Any hints?

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/solutions/autoscale/ltm

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 5

In a lab setup I am using:
/supported/solutions/autoscale/ltm/f5-bigiq-autoscale-bigip-ltm.template

The assignment of license is working. When a BIG-IP is removed from the auto scaling group its license is not revoked from the BIG-IQ purchased license pool. Based on the BIG-IQ logs, there does not seem to be any failed attempts to remove. On the BIG-IQ, in resstjavad.0.log and restjavad-audit.0.log, there are no logs relating to the removal of the instance.

As a time reference, this is where the S3 bucket size changed from 2 to 1 in the aws-autoscale.log on the master BIG-IP:

2018-03-28T14:04:04.150Z silly: [pid: 12474] [lib/awsAutoscaleProvider.js] getInstancesFromDb: S3 bucket size: 1

Here is that log file
aws-autoscale.log

I am not exactly clear on how this works, could you give a high level of the events that take place to revoke a license from a BIG-IQ pool?

Let me know if there is some other relevant data I should be looking at in the setup or any other data you would like to see.

i.e. similar to https://github.com/f5devcentral/f5-aws-autoscale
"Mappings": {
"BigipRegionMap": {
"ap-northeast-1": {
"Best1000": "ami-e15eac80",
"Best200": "ami-5f5daf3e",
"Best25": "ami-f25daf93",
"Better1000": "ami-e25eac83",
"Better200": "ami-f05eac91",
"Better25": "ami-815ba9e0",
"Good1000": "ami-8d58aaec",
"Good200": "ami-165cae77",
"Good25": "ami-d95fadb8"
},

Description

When the below template is run, we wre able to create instances and LTM has created a pool and instances are registered to the pools. but auto-scaling sync group is not creating with the below error
We tried to ssh to the instance and execute the same command, still the same message.

Error while creating device group with error "The requested device (ip-10-114-5-129.ec2.internal) was not found.".

Detailed Error message in custom-config.log
2017-09-19T11:45:47.471Z info: /config/cloud/aws/node_modules/f5-cloud-libs/scripts/autoscale.js called with /usr/bin/f5-rest-node /config/cloud/aws/node_modules/f5-cloud-libs/scripts/autoscale.js --cloud aws --provider-options s3Bucket:athena-bigip-poc-s3bucket-u0fd7pcspc35,mgmtPort:8443 --host localhost --port 8443 --user cluster-admin --password-url file:///config/cloud/aws/.adminPassword --device-group autoscale-group --block-sync -c join --log-level info --output /var/log/aws-autoscale.log
2017-09-19T11:45:47.741Z info: Initializing BIG-IP.
2017-09-19T11:45:48.172Z info: Getting this instance ID.
2017-09-19T11:45:48.173Z info: Getting info on all instances.
2017-09-19T11:45:48.443Z info: Determining master instance id.
2017-09-19T11:45:48.444Z info: Possible master ID: i-058db7e10ec725e82
2017-09-19T11:45:48.444Z info: Valid master ID: i-058db7e10ec725e82
2017-09-19T11:45:48.444Z info: Using master ID: i-058db7e10ec725e82
2017-09-19T11:45:48.444Z info: This instance is master
2017-09-19T11:45:48.445Z info: Cluster action JOIN
2017-09-19T11:45:48.558Z info: Storing master credentials.
2017-09-19T11:45:48.644Z info: Not seting config sync IP because we're master and block-sync is specified.
2017-09-19T11:45:48.644Z info: Creating device group.
2017-09-19T12:00:59.379Z error: 01020036:3: The requested device (ip-10-114-5-129.ec2.internal) was not found.
2017-09-19T12:00:59.382Z info: Autoscale finished.

Template

For bugs, enter the template with which you are experiencing issues below.

https://github.com/F5Networks/f5-aws-cloudformation/blob/13ad36ee81e1070a8e080a305acb0596cbb6f4a5/supported/solutions/autoscale/ltm/f5-autoscale-bigip-ltm.template

Parameters to the template

1. Changed template assigning from pubic ip to private ip
2. Restricted Address is given 0.0.0.0/0
3. Selected private subnets of our VPC

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 1

Description

Issue with template failing to run completely. I had a build fail twice before the third run took.

2017-04-10T13:37:18.546Z info: /config/bigip.conf
2017-04-10T13:37:18.550Z info: /config/bigip_base.conf
2017-04-10T13:37:18.553Z info: /config/bigip_script.conf
2017-04-10T13:37:18.556Z info: /config/bigip_user.conf
2017-04-10T13:37:18.559Z info: /config/partitions/LOCAL_ONLY/bigip.conf
2017-04-10T13:37:18.642Z info: Saving Ethernet mapping...
2017-04-10T13:37:18.654Z info: done
2017-04-10T13:37:19.040Z info: Loading
2017-04-10T13:37:19.041Z info: configuration...
2017-04-10T13:37:19.046Z info: /config/cloud/f5.http.v1.2.0rc4.tmpl
2017-04-10T13:37:23.555Z error: Unexpected Error: Thrift exception (probably ASM config server is not running): No more data to read.
2017-04-10T13:37:23.721Z error: 01020036:3: The requested ASM policy (/Common/linux-low) was not found.
2017-04-10T13:37:24.986Z error: 01020036:3: The requested policy action (/Common/linux-low) was not found.
2017-04-10T13:37:25.493Z error: 0107172d:3: Policy '/Common/app-ltm-policy' can't be applied to virtual server '/Common/erchen-waf.app/erchen-waf_vs' because it has no rules.
2017-04-10T13:37:27.343Z info: Saving running configuration...

Template

f5-autoscale-bigip.template

Severity Level

Severity: 3

I tried to deploy the 3-nic template with existing vpc and subnets:
https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/standalone/3nic

Due to following error a rollback was done:
In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit https://aws.amazon.com/marketplace/pp?sku=8esk90vx7v713sa0muq2skw3j (Service: AmazonEC2; Status Code: 401; Error Code: OptInRequired; Request ID: 42042966-8757-4e1e-a4fe-4b7942b36816)

Bigip1Instance AWS::EC2::Instance

From the error message it seems that the Bigip Instance needs to ack the license.
Is there a way to ack such a problem with in the cloudformation template ?

Hi,

The new f5-existing-stack-byol-3nic-bigip.template version v3.3.0 doesn't mention anything about restrictedSrcAddressApp in the docs, however:

* aws_cloudformation_stack.bigip: ValidationError: Parameters: [restrictedSrcAddressApp] must have values

Should probably default to 0.0.0.0/0.

Thanks.

Hi,

The EC2 action DescribeNetworkInterfaceAttributes is unrecognized. It should be DescribeNetworkInterfaceAttribute (singular).

Thanks,
John

Hey guys. Thanks for creating these templates and documentation. They are extremely helpful!

FWIW, here are a couple of details that tripped me up when using these from scratch for the first time. I think some tweaks to the prompts/docs could prevent others for having the same issues.

(I was using the 1 NIC/Good/1 GB Hourly template - literally the first yellow button on the template page. I'm not sure if these issues occur with other templates, but it seems like they might.)

1.) The first time through, after a long confusing delay, creating the stack failed because I needed to subscribe to the specific VE I wanted in the AWS Marketplace. If would have been nice if that info was in the hourly stack descriptions (maybe with a link to facilitate subscribing).

2.) Obviously, F5 can't control the confusing "Region" selecting issue with CFTs, but maybe you can add a big message on the second page that says "Make sure you've selected the proper region before continuing." I'm guessing many VEs are accidentally created in Virginia by mistake.

3.) "Source Address(es) for Management Access" confused me for a while. Specifically the description line that ended with "instances" (plural). That caused me to think about the instances (plural) behind the load balancer, not the instance that contains the load balancer. The description in the docs weren't too helpful either. I know this is wordsmithing, but maybe something like: "IP Address(es) for Management Access: _______ - For SSH and web management access. Often this is your public IP address (/32)."

4.) "Source Address(es) for Web Application Access (80/443)" is still somewhat confusing to me. Should it be "0.0.0.0/0" if the load balancer is fronting a public website? I see that these settings are used in the security group that gets setup. Maybe make that more clear in the description (i.e., mention that fact above both of these fields in the template)?

Looking back now, I think the term "Source" confused me.

5.) After the template runs, it is very unclear what to do next. The docs state "If you want access to the BIG-IP web-based Configuration utility, you must first SSH into the BIG-IP VE using the SSH key you provided in the template. You can then create a user account with admin-level permissions on the BIG-IP VE to allow access if necessary."

The problem is that I don't setup BIG-IP LTMs from scratch every day (which is possibly my main message here in this post). "Create a user account with admin-level permissions" needs to be clearly defined. Eventually, I found a DevCentral article that showed someone logging in with ssh to the "admin" account (good to know) and entering "modify auth admin password" to change the admin password (also good to know).

6.) Something else that had slipped my mind was the default address for the configuration utility. http://xx.xx.xx.xx didn't work. https://xx.xx.xx.xx didn't work either. Looking at the security group for the BIG-IP made me think that one of those two should have worked. I wasted a couple of hours in frustration until discovering/remembering that it was https://xx.xx.xx.xx:8443 (doh!)

Again, these are simple things that - for a harried administrator - can really cause a ton of frustration. Adding super-clear steps to the setup instructions (especially at the end) would really help.

• Chip

Hi,

The subnet documentation for f5-existing-stack-across-az-cluster-byol-3nic-bigip.template is wrong:

Should probably be:

Also it's unclear what's the different between Subnet1 and Subnet2 in each AZ as they have the same description.

Thanks.

Description

Additional autoscaled/secondary BIG-IP fails to join the cluster.
Seems that by default only "restrictedSrcAddress" is allowed inbound 8443 access in the CFT created security group. The value I input for the restrictedSrcAddress is a /32, but as it is now, only a 0.0.0.0/0 would seem to work.

I am thinking the options are to add another field to input/select the internal subnet for allowing cluster join to occur, OR somehow adding the internal subnet by default in the scripts that creates the secgroup.

Is this just my environment?

Repro Steps:

1. Deploy CFT
2. Trigger autoscale
3. Confirm that additional BIG-IP comes up but does not join the cluster

Log of affected BIG-IP
/var/log/custom-config.log

2017-07-18T10:28:22.227Z info: /config/cloud/aws/node_modules/f5-cloud-libs/scripts/runScript.js called with /usr/bin/f5-rest-node /config/cloud/aws/node_modules/f5-cloud-libs/scripts/runScript.js --log-level info --file /config/cloud/aws/custom-config.sh --cwd /config/cloud/aws -o /var/log/custom-config.log --wait-for ONBOARD_DONE
2017-07-18T10:28:22.236Z info: Waiting for ONBOARD_DONE
2017-07-18T10:32:56.027Z info: /config/cloud/aws/custom-config.sh starting.
2017-07-18T10:32:56.045Z info: Tue Jul 18 19:32:56 JST 2017
2017-07-18T10:32:56.061Z error: % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2017-07-18T10:32:56.061Z error: Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0^M  0    44    0    44    0     0   4564      0 --:--:-- --:--:-- --:--:-- 44000
2017-07-18T10:32:57.608Z info: Attempting to Join or Initiate Autoscale Cluster
2017-07-18T10:32:57.759Z info: 2017-07-18T10:32:57.757Z info: /config/cloud/aws/node_modules/f5-cloud-libs/scripts/autoscale.js called with /usr/bin/f5-rest-node /config/cloud/aws/node_modules/f5-cloud-libs/scripts/autoscale.js --cloud aws --provider-options s3Bucket:awsmpautoscalingwafviacft-s3bucket-11zlr5b2w9mjt,mgmtPort:8443 --host localhost --port 8443 --user awsadmin --password-url file:///config/cloud/aws/.adminPassword --device-group autoscale-group --block-sync -c join --log-level info --output /var/log/aws-autoscale.log
2017-07-18T10:32:58.004Z info: 2017-07-18T10:32:58.004Z info: Initializing BIG-IP.
2017-07-18T10:32:58.386Z info: 2017-07-18T10:32:58.386Z info: Getting this instance ID.
2017-07-18T10:32:58.387Z info: 2017-07-18T10:32:58.387Z info: Getting info on all instances.
2017-07-18T10:32:58.812Z info: 2017-07-18T10:32:58.812Z info: Determining master instance id.
2017-07-18T10:32:58.813Z info: 2017-07-18T10:32:58.812Z info: Possible master ID: i-03a409857dd4e
2017-07-18T10:32:58.813Z info: 2017-07-18T10:32:58.813Z info: Valid master ID: i-03a409857dd4e
2017-07-18T10:32:58.814Z info: 2017-07-18T10:32:58.813Z info: Using master ID: i-03a409857dd4e
2017-07-18T10:32:58.814Z info: This instance is not master
2017-07-18T10:32:58.815Z info: 2017-07-18T10:32:58.815Z info: Cluster action JOIN
2017-07-18T10:32:58.886Z info: 2017-07-18T10:32:58.886Z info: Setting config sync IP.
2017-07-18T10:32:59.001Z info: 2017-07-18T10:32:59.000Z info: Joining cluster.
2017-07-18T11:06:17.971Z error: 2017-07-18T11:06:17.970Z error: connect ECONNREFUSED 10.0.11.77:8443   <---!!!!!
2017-07-18T11:06:17.972Z info: 2017-07-18T11:06:17.972Z info: Autoscale finished.
2017-07-18T11:06:17.978Z info: Tue Jul 18 20:06:17 JST 2017
2017-07-18T11:06:17.979Z info: custom-config.sh complete

This log told me it is a security group related issue so I modified the CFT created security group to allow inbound 8443 from 10.0.0.0/16, redeployed the BIG-IP and confirmed it succeeded.

----snip----
2017-07-18T13:30:05.227Z info: 2017-07-18T13:30:05.227Z info: Initializing BIG-IP.
2017-07-18T13:30:05.761Z info: 2017-07-18T13:30:05.760Z info: Getting this instance ID.
2017-07-18T13:30:05.761Z info: 2017-07-18T13:30:05.761Z info: Getting info on all instances.
2017-07-18T13:30:06.349Z info: 2017-07-18T13:30:06.348Z info: Determining master instance id.
2017-07-18T13:30:06.356Z info: 2017-07-18T13:30:06.349Z info: Possible master ID: i-03a409857dd4e
2017-07-18T13:30:06.349Z info: Valid master ID: i-03a409857dd4e
2017-07-18T13:30:06.360Z info: 2017-07-18T13:30:06.360Z info: Using master ID: i-03a409857dd4e
2017-07-18T13:30:06.360Z info: 2017-07-18T13:30:06.360Z info: This instance is not master
2017-07-18T13:30:06.361Z info: 2017-07-18T13:30:06.360Z info: Cluster action JOIN
2017-07-18T13:30:06.428Z info: 2017-07-18T13:30:06.428Z info: Setting config sync IP.
2017-07-18T13:30:06.593Z info: 2017-07-18T13:30:06.592Z info: Joining cluster.
2017-07-18T13:30:07.014Z info: 2017-07-18T13:30:07.014Z info: Checking remote host for cluster readiness.
2017-07-18T13:30:07.191Z info: 2017-07-18T13:30:07.142Z info: Getting local hostname for trust.
2017-07-18T13:30:07.191Z info: 2017-07-18T13:30:07.155Z info: Getting local management address.
2017-07-18T13:30:07.163Z info: Adding to remote trust.
2017-07-18T13:30:16.278Z info: 2017-07-18T13:30:16.278Z info: Adding to remote device group.
2017-07-18T13:30:16.725Z info: 2017-07-18T13:30:16.721Z info: Checking for datasync-global-dg.
2017-07-18T13:30:17.137Z info: 2017-07-18T13:30:17.137Z info: Telling remote to sync.
2017-07-18T13:30:47.227Z info: 2017-07-18T13:30:47.226Z info: Telling remote to sync datasync-global-dg request.
2017-07-18T13:30:47.634Z info: 2017-07-18T13:30:47.633Z info: Waiting for sync to complete.
2017-07-18T13:31:18.102Z info: 2017-07-18T13:31:18.101Z info: Sync not yet complete.
2017-07-18T13:31:18.102Z info: 2017-07-18T13:31:18.102Z info: Following recommended action. Syncing group autoscale-group
2017-07-18T13:31:48.189Z info: 2017-07-18T13:31:48.188Z info: Telling remote to sync.
2017-07-18T13:32:18.524Z info: 2017-07-18T13:32:18.523Z info: Telling remote to sync datasync-global-dg request.
2017-07-18T13:32:18.652Z info: 2017-07-18T13:32:18.651Z info: Waiting for sync to complete.
2017-07-18T13:32:29.114Z info: 2017-07-18T13:32:29.113Z info: Sync complete.
2017-07-18T13:32:29.115Z info: 2017-07-18T13:32:29.115Z info: Autoscale finished.
2017-07-18T13:32:29.121Z info: Tue Jul 18 22:32:29 JST 2017

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/solutions/autoscale/waf

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 2

Hi,

Trying to delete a CF stack based on a successfully deployed f5-existing-stack-across-az-cluster-byol-3nic-bigip.template fails with the following error message:

* aws_cloudformation_stack.bigip (destroy): 1 error(s) occurred:

* aws_cloudformation_stack.bigip: DELETE_FAILED: ["The following resource(s) failed to delete: [S3Bucket]. " "The bucket you tried to delete is not empty (Service: Amazon S3; Status Code: 409; Error Code: BucketNotEmpty; 

Not sure if this is an expected behaviour.

Deleting the bucket's contents manually - solves the issue, and stack delete succeeds.

Thanks!

Is it possible for the SSL Cert field to accept the ACM Cert instead of the IAM cert format?

Description

BIG-IQ v5.1
BIG-IP 12.1.1 Build 1.0.196

Summary: CFT not completing licensing via BIG-IQ license pool. Need help to get this running.

• Nothing gets written to /var/log/cloud-init.log (is anything supposed to get written here?)
  Supplying boot.log to show you that cloud-init runs.
• Entered in the license pool UUID as obtained from below api: curl -sku admin:*** -X GET https://IP/mgmt/cm/shared/licensing/pools (This is the API for purchased pools, which btw is different from regkey pools)

boot.txt

• Note: The BIG-IQ IP used is the local management IP, on the primary NIC.

Template

https://github.com/F5Networks/f5-aws-cloudformation/blob/master/experimental/reference/2nic/bigiq/f5-existing-stack-bigiq-license-pool-2nic-bigip.template

Severity Level

2 - need to do POC for cloud tenant licensing automated by BIG-IQ license pools

Severity: <Fill in level: 1 through 5>

Severity level definitions:

1. Severity 1 (Critical) : Defect is causing systems to be offline and/or nonfunctional. immediate attention is required.
2. Severity 2 (High) : Defect is causing major obstruction of system operations.
3. Severity 3 (Medium) : Defect is causing intermittent errors in system operations.
4. Severity 4 (Low) : Defect is causing infrequent interuptions in system operations.
5. Severity 5 (Trival) : Defect is not causing any interuptions to system operations, but none-the-less is a bug.

seems like same-az failover isn't working when the virtual servers are configured under a different partition than common.
please add support or specify in the documentation that it isn't supported.

also, please specify in the configuration that a virtual server has to be configured for the failover to work.
Thanks

S3 backup script is not running successfully. This is the error when run:

18-09-21T15:25:34.031Z info: This instance is master
2018-09-21T15:25:34.393Z warn: Unknown cluster action backup-ucs--log-level
2018-09-21T15:25:34.394Z error: autoscaling error: Unknown cluster action backup-ucs--log-level

There is "space" missing between the option that specifies backup-ucs and the next option log-level.

This is from the script on the F5 ASM after deployment:

--cluster-action backup-ucs--log-level silly

I am using the latest f5-bigiq-autoscale-bigip-ltm.template with a regKey pool that contains only two regKeys. BIG-IQ version 5.3.0-final.

My autoscaling group only has two members. It is working well to handout and revoke licenses if I terminate the non-master BIG-IPs. But if I terminate the master BIG-IP, it fails to revoke the license on the BIG-IQ. I do not see any logs about an attempt on the BIG-IQ.

Here is a run through what I am doing:

At the start the Master BIG-IP is "i-09f3a3736d449b6dc".

Below is the s3 instances files contents showing who is master before I terminate the master:

{
"privateIp": "10.0.2.5", <-- i-0114d988efc79a4c5 non-master
"mgmtIp": "10.0.2.5",
"hostname": "ip-10-0-2-5.us-west-2.compute.internal",
"isMaster": false,
"providerVisible": true,
"status": "OK",
"lastUpdate": "2018-04-06T22:50:05.308Z",
"masterStatus": {
"instanceId": "i-09f3a3736d449b6dc",
"status": "OK",
"lastUpdate": "2018-04-06T22:50:05.118Z",
"lastStatusChange": "2018-04-06T22:14:27.219Z"
},
"lastJoinRequest": "2018-04-06T22:50:05.305Z"
}
{
"privateIp": "10.0.2.153", <- i-09f3a3736d449b6dc Master
"mgmtIp": "10.0.2.153",
"hostname": "ip-10-0-2-153.us-west-2.compute.internal",
"isMaster": true,
"providerVisible": true,
"status": "OK",
"lastUpdate": "2018-04-06T22:50:04.157Z",
"masterStatus": {
"instanceId": "i-09f3a3736d449b6dc",
"status": "OK",
"lastUpdate": "2018-04-06T22:50:04.156Z",
"lastStatusChange": "2018-04-06T22:02:04.192Z"
}
}

Then I terminate i-09f3a3736d449b6dc.
For reference in the /var/log/cloud/aws/aws-autoscale.log I did this about 2018-04-06T22:57:25

On remaining instance:
2018-04-06T23:02:04.159Z info: [pid: 19086] [scripts/autoscale.js] No master ID found.

The new instance finished launching before the license was not revoked, so there was no license for it since pool only had two.
I mention this so you can understand the time of BIG-IQ log below in case it is useful.

On BIG-IQ:
INFO][06 Apr 2018 16:03:55 PDT][/shared/authn/login AuthnWorker] User admin successfully logged in from 10.0.2.213 using the local authentication provider.
[WARN][06 Apr 2018 16:03:55 PDT][/cm/device/tasks/licensing/pool/member-management/cb1b3ba9-c727-4e37-a85e-0f695616edf4/worker DeviceLicensingAssignmentTaskWorker] Task failed at step VALIDATE_LICENSE with message: 2 offerings found for regkey pool license 'mcnair_regkey' and the given search criteria (none), but none are available for use

At this point the former non-master, i-0114d988efc79a4c5, is master but the old master that I terminated did not get it's license revoked. The new instance that spun up could not get a license because there were none available.

Here is the current status of the s3 instance files:

{
"privateIp": "10.0.2.5", <- new master i-0114d988efc79a4c5
"mgmtIp": "10.0.2.5",
"hostname": "ip-10-0-2-5.us-west-2.compute.internal",
"isMaster": true,
"providerVisible": true,
"status": "OK",
"lastUpdate": "2018-04-06T23:04:05.209Z",
"masterStatus": {
"instanceId": "i-0114d988efc79a4c5",
"status": "OK",
"lastUpdate": "2018-04-06T23:04:04.465Z",
"lastStatusChange": "2018-04-06T23:04:04.465Z"
},
"lastJoinRequest": "2018-04-06T22:56:04.470Z"
}
{
"privateIp": "10.0.2.213", <- new instance, no license
"mgmtIp": "10.0.2.213",
"hostname": "ip-10-0-2-213.us-west-2.compute.internal",
"isMaster": false,
"providerVisible": true,
"status": "OK",
"lastUpdate": "2018-04-06T23:04:11.360Z",
"masterStatus": {
"instanceId": "i-0114d988efc79a4c5",
"status": "OK",
"lastUpdate": "2018-04-06T23:04:07.930Z",
"lastStatusChange": "2018-04-06T23:04:07.930Z"
},
"lastJoinRequest": "2018-04-06T23:04:11.359Z"
}

So the issue is that if the master BIG-IP is terminated, the license for the master does not get revoked.
There were no further logs on the BIG-IQ after the ones I posted above. It's been about 30 minutes.

I am attaching the log aws-autoscale.log from the new master BIG-IP.
aws-autoscale.log

Description

My resident Network Engineer told me that if we have to re-deploy our BYOL F5 virtual edition instance in AWS, it will need to be re-registered. And if we want to use the same registration key, we will have to de-register the old instance first. He also said that this is a manual process which cannot be automated.

What options exist to solve this problem?

Can I ...

1. back up the license file after the instance has been licensed, re-create the instance, and restore the license file?
2. Snapshot the instance, stop it, and boot a new image from the snapshot AMI ?
3. Does F5 provide an API for product deregistration / registration ?

Template

https://github.com/F5Networks/f5-aws-cloudformation/blob/master/supported/standalone/2nic/f5-existing-stack-byol-2nic-bigip.template

Severity Level

5

A per the title all the templates under the following branch create a failover group with a confusing name:
f5-aws-cloudformation/supported/failover/same-net/via-api/

The failover group is created as 'across_az_failover_group' even though this is a 'same az' deployment

Description

After deploying the template, the "Log Profile" setting is not set, thus nothing gets logged to security event logs.

Logged into BIG-IP GUI and checked Virtual Server > Security > Policies > Log Profile > set to "Disabled"

The Azure template has logging enabled by default, so thought this should be as well.

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/solutions/autoscale/waf

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 5

Description

When attempting to use the templates, i get the following error:

Template validation error: TemplateURL must reference a valid S3 object to which you have access.

Template

https://s3.amazonaws.com/f5-cft/f5-existing-stack-byol-2nic-bigip.template

Severity Level

Severity: 2

Hi,

It's somewhat unclear which exact values are legal for the timezone parameter in templates.
The default UTC is confusing.

Moreover, the template accepts illegal values like UTC+3 or IL, and the deployment succeeds, however the BIG-IP is unlicensed due to:

2018-08-28T19:51:02.659Z error:  Onboard failed: 01070920:3: Application error for confpp: UTC+3 is not a valid time zone.

We could potentially link to:
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
referencing the TZ* column and list some examples like US/Eastern or Europe/Bratislava.

Thanks!

Hi,

The restrictedSrcAddress and restrictedSrcAddressApp template parameters don't seem to accept lists:

Error: aws_cloudformation_stack.main: parameters (restrictedSrcAddress): '' expected type 'string', got unconvertible type '[]interface {}'

Error: aws_cloudformation_stack.main: parameters (restrictedSrcAddressApp): '' expected type 'string', got unconvertible type '[]interface {}'

So it's unclear if (and how) we can specify multiple ranges of allowed source IP addresses.

I used f5-existing-stack-byol-3nic-bigip.template, but I imagine other templates have the same issue.

Thanks!

Description

I'm deploying using the https://github.com/F5Networks/f5-aws-cloudformation/tree/master/experimental/bigiq/licenseManagement/standalone/2nic/existing-stack/byol
template.

after the template completes, i login to the BIGIQ and under "this device" --> "general properties" , no parameters are populated.
on restjavad.0.log i get the following messages:

[INFO][11 Dec 2018 13:32:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured
[INFO][11 Dec 2018 13:33:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured
[INFO][11 Dec 2018 13:34:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured
[INFO][11 Dec 2018 13:35:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured
[INFO][11 Dec 2018 13:35:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured
[INFO][11 Dec 2018 13:36:34 PST][/cm/shared/event/analyzer EventAnalyzer] The machineIdResolver failed to find the localhost, this usually happens before the discovery address has been set or right after a restart. Event Analysis cannot start until it has been configured

if i manually go to /ui/setup and click 'next' without changing any of the values the log messages stop and the properties show up on the device information.

Template

f5-existing-stack-byol-2nic-bigiq-licmgmt.template

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

2

as below screenshot, it can not be ran in AWS beijing region. could you please double check if there is some thing wrong?
f5-aws-cloudformation/supported/cluster/2nic/across-az-ha/f5-existing-stack-across-az-cluster-byol-2nic-bigip.template

I would like to deploy F5-WAF HA cross two AZ in AWS CN-BeiJing region, would you please give me a instruction and guide docoument? thanks a lot.

[DOC] correct config example image

Description

In the "Configuration Example" section of the readme, the web server IPs should be in the 10.0.1.0/24 subnet.
Currently they are in the AZ2 subnet for application traffic.

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/cluster/2nic/across-az-ha

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: 5

I tested a few templates and noticed failures. When I checked logs, I get this...

2016-12-10T01:05:39.720Z - info: start install biqiq license
2016-12-10T01:05:39.720Z - error: /config/cloud/aws/custom-config.sh: line 11: /config/cloud/aws/license_from_bigiq.sh: No such file o r directory

However, the CFT places that shell file in /tmp like below...

                        "/tmp/license_from_bigiq.sh": {
                            "group": "root",
                            "mode": "000755",
                            "owner": "root",
                            "source": "http://cdn.f5.com/product/templates/utils/license_from_bigiq_v5.0.sh"

As a result, the following license line fails because the script is called from the wrong directory.

                                        "echo 'start install biqiq license'\n",
                                        ". /config/cloud/aws/license_from_bigiq.sh\n",

Restore from backup fails to execute:

Restore from backup UCS is still failing:

2018-09-21T16:29:20.630Z debug: [pid: 27906] [lib/bigIp.js] {"kind":"tm:sys:mcp-state:mcp-statestats","selfLink":"https://localhost/mgmt/tm/sys/mcp-state?ver=13.1.1","entries":{"https://localhost/mgmt/tm/s
ys/mcp-state/0":{"nestedStats":{"entries":{"endPlatformIdReceived":{"description":"true"},"lastLoad":{"description":"high-config-load-succeed"},"phase":{"description":"running"}}}}}}
2018-09-21T16:29:21.471Z warn: [pid: 27906] [scripts/autoscale.js] /config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/updateAutoScaleUcs failed: Error: Command failed: /config/cloud/aws/node
_modules/@f5devcentral/f5-cloud-libs/scripts/updateAutoScaleUcs --original-ucs /config/ucsOriginal_1537547346673.ucs --updated-ucs /config/ucsUpdated_1537547346673.ucs --cloud-provider aws --extract-directo
ry /config/cloud/ucsRestore
Traceback (most recent call last):
File "/config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/updateAutoScaleUcs", line 263, in
main()
File "/config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/updateAutoScaleUcs", line 243, in main
removeFiles(extract_ucs_dir + '/config/cloud/keys', 'cloudLocal*')
File "/config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/updateAutoScaleUcs", line 127, in removeFiles
for f in os.listdir(dir):
OSError: [Errno 2] No such file or directory: '/config/cloud/ucsRestore/config/cloud/keys'

Description

Describe the problem you're having or the enhancement you'd like to request.

From AWS Console-> Template validation error: Template may not exceed 460800 bytes in size.

Template

For bugs, enter the template with which you are experiencing issues below.
f5-full-stack-across-az-cluster-hourly-2nic-bigip
f5-full-stack-across-az-cluster-byol-2nic-bigip

Severity Level

For bugs, enter the bug severity level. Do not set any labels.

Severity: <Fill in level: 1 through 5>
3

Severity level definitions:

1. Severity 1 (Critical) : Defect is causing systems to be offline and/or nonfunctional. immediate attention is required.
2. Severity 2 (High) : Defect is causing major obstruction of system operations.
3. Severity 3 (Medium) : Defect is causing intermittent errors in system operations.
4. Severity 4 (Low) : Defect is causing infrequent interuptions in system operations.
5. Severity 5 (Trival) : Defect is not causing any interuptions to system operations, but none-the-less is a bug.

I deployed the CFT autoscaling template and it was successful with no issues. But when I create a classical ELB with BIGIP instances as targets, the ELB health check(configured to ping TCP:443) shows the instances are out of service.
Is there anything else, I should be changing in the template or on the BIG IP itself ??

I don't see a supported CFT that builds a new stack.

For example, supported/standalone/1nic has only "existing-stack" or "production-stack" but no "learning-stack" or "full-stack".

I think it would be nice to add a README.md in supported/standalone/1nic that explains that there is an experimental template in experimental/standalone/1nic for "full-stack" deployment.

Hi. I need your assistance about application persistence with client cert auth.

I am interested in using the following template:
Auto scaling the BIG-IP VE Local Traffic Manager (LTM) in AWS: Existing Stack with BIG-IQ Licensing (Frontend via ELB)

If using TCP listener, sticky session is not available. It means that BIG-IP's persistence is not available as well. However, client cert auth is available.

If using HTTPS listener, sticky session is available, but CLB has to re-encrypt the traffic for client cert auth with BIG-IP, so server cert/key has to be registered in CLB setting. However, I cannot find any server key setting in CLB.

I was wondering what is the best way to use client cert auth using the above template?

Thank you.