The “pks get-credentials” is no longer a recognized command though is documented.

Used the second instructions in the docs to obtain the credentials via the APIs. This is a lengthy and a very inconvenient way to obtain the creds.

· export CLUSTER_NAME="pks-kube01"
· export PKS_API_ENDPOINT="pks.core.hypervizor.com"
· Get the APIs creds.
· export PKS_API_CREDS="CREDS HERE"
· export CREDENTIALS="$(curl -k -u $PKS_API_CREDS -s -X POST --header 'Content-Type: application/json' https://$PKS_API_ENDPOINT:9021/v1/clusters/$CLUSTER_NAME/binds)"
· echo $CREDENTIALS
· Save the output into the config file under the .kube folder of the client machine.

The docs refer to the user to the k8s docs to access the dashboard. This is confusing as the referenced docks will change over time it was not specifying a version of k8s in the link. Also the linked docs don't explain how to get the creds to access to the dashboard. It would much clearer if we provided a step by stop instructions that were tested with the version of pks that is included in the docs.

To avoid confusion, PKS upgrade with NSX-T (https://docs.pivotal.io/runtimes/pks/1-1/upgrade-pks.html#upgrade-nsxt) needs a note to tell customers to use a different CIDR range for the IP Block being used for Node Networks from the ones they used for the Service Networks.

Also, this same section has some formatting errors. Some steps are munged together.

RBAC is the only mode

The two pages https://docs.pivotal.io/runtimes/pks/1-0/gcp-requirements.html and https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-gcp.html link to each other circularly and each imply that the other will tell you more but neither tells you the permissions the service account needs.

We should add the list of permissions needed, probably on the gcp-requirements page.

We need to add a section on Preparing to Install PKS on vSphere with NSX-T. This section could include all content on this page up to Step 2 (https://docs.vmware.com/en/VMware-Pivotal-Container-Service/1.1/vmware-pks-11/GUID-PKS11-installing-nsx-t.html), as well as additional information we are gathering from field experience and feedback. Targeting PKS release when NSX-T 2.2 is supported.

In the doc, https://docs.pivotal.io/runtimes/pks/1-1/vsphere-om-config.html#dir-config

16. Leave the Director Hostname field blank.

Is this mandatory?

With it, you can't bosh cli to bosh director on NAT topology with NSX-T from external.

I've been following this manual https://docs.pivotal.io/runtimes/pks/1-1/installing-pks.html

But I've reached the end of that part and I'm not able to apply the changes because the Pivotal Container Services it's still in orange after all the configuration with a message that says "Missing stemcell"

When I click on it, it asks me to upload a stemcell library (which it's not documented in the link I was following) and the version of the library it's not available in the pivotal network: ubuntu-trusty 3586.24

But it seems that has been recently released https://docs.pivotal.io/pivotalcf/2-1/stemcells/stemcells.html#3586-line

I've tried to upload version 3596.16 but it says me that:
"The currently staged and deployed products cannot uselight-bosh-stemcell-3586.16-google-kvm-ubuntu-trusty"

I'm trying to install version v1.1.0-build.311 of Pivotal Container Service over Ops Manager v2.1-build.326 in GCP

I hope you can help me to unblock this situation thanks!

It is not clear in the docs what values the PKS Tile > Networking > vSphere Cluster Names option expects. The inline help in the UI says "Enter the list of vSphere cluster names (comma separated) where kubernetes clusters will be deployed." The documentation says (https://docs.pivotal.io/runtimes/pks/1-1/installing-nsx-t.html#networking) "For vSphere Cluster Names, enter the name of the vSphere cluster that corresponds to the AZ where you deployed the PKS control plane VM."

These statements appear to be conflicting, and do not accurately reflect what this field is used for. This field is used for validation by nsx-t precheck errand, which checks if all the hosts from the clusters specified in the field are onboarded in NSX. You can specify clusters in the format cluster1,cluster2,cluster3.

Source: https://pivotal.slack.com/archives/C6JEFDLT0/p1532372535000404
VMW Jira: https://jira.eng.vmware.com/browse/PKS-919

I am referring to URL: https://docs.pivotal.io/runtimes/pks/1-1/vsphere-om-deploy.html

Please find below list of issues that need to be addressed on this page.

1. This page does not tell which type of folder needs to be created in the datacenter at step# 5. There are 4 different types of folders that can be created at this step. But, the instruction does not reflect that.

2. There is no file menu as described at step# 7 to select .ova file to upload

3. There is no license agreement step as mentioned at step# 10 in the actual flow

Almost all steps are different from actual flow on vSphere. So, not listing everything here. But this is completely off the actual experience and not very helpful. Please correct this page.

The instructions for deploying Ops Manager to VMware vSphere has the following note

Note: With vSphere 6.5 and NSX-T 2.1, when initially deploying the Operations Manager OVF, you 
cannot connect directly to an NSX-T logical switch. You must first connect to a vSphere Standard (vSS) 
or vSphere Distributed Switch (vDS). A suggested approach is to connect to a VSS or VDS when 
deploying the OVF, but do not power the VM on. After the OVF deployment has completed, you can 
then connect the network interface to the appropriate NSX-T logical switch and power the VM on to 
proceed with the install.

This KB describes why this fails, the workaround and which version this problem is addressed in https://kb.vmware.com/kb/54142. It will be great if we have this information presented to reader.

There is no pdf version of document for version 1.1. 1.0 is outdated.

Following the steps at https://docs.pivotal.io/runtimes/pks/1-1/gcp-api-load-balancer.html Step 3 is asking that the PKS controller VM be tagger, but the PKS tile has not been imported yet so the VM has not yet been created.

Documentation states the following
pks create-cluster CLUSTER-NAME --external-hostname HOSTNAME --plan PLAN-NAME [--num-nodes WORKER-NODES]
But returns Error: unknown flag: --plan
Can the document be fixed to what is currently supported?

https://docs.pivotal.io/runtimes/pks/1-1/installing-pks.html#install

The usage data section needs to go all the way at the bottom.

On page: docs-pks/custom-workloads.html.md.erb

We do not give any examples of what can go here. It might be clearer if we provide an example.

Regarding the step for "NSX Manager Super User Principal Identify Certificate" under Networking:
The referenced step for creating this Cert, Step 6: Generate and Register Certificates is empty.

What would be the procedure for this?

Deploying the concourse chart will fail if privileged containers is not enabled. The statefulset will display this event which is not particularly descriptive.

:> kubectl describe statefulset my-concourse-worker
Warning  FailedCreate      1m (x36 over 11m)  statefulset-controller  create Pod my-concourse-worker-0 in StatefulSet my-concourse-worker failed error: Pod "my-concourse-worker-0" is invalid: spec.containers[0].securityContext.privileged: Forbidden: disallowed by cluster policy

A note like this in the procedure will be helpful

<p class="note"><strong>Note</strong>: Privileged containers must be enabled in the PKS plan for Concourse to function.  Refer to configuring [plans](https://docs.pivotal.io/runtimes/pks/1-1/installing-pks.html#plans) in the install guide for more information on enabling privileged containers.</p>

The instructions at https://docs.pivotal.io/runtimes/pks/1-1/gcp-cluster-load-balancer.html are very confusing for example the step titled "Creating GCP Load Balancers for PKS Clusters" in substep 8 wants the selection of worker nodes to picked but the tile has not been installed at this point there is no point. Seems like this whole page is in the wrong spot or has wrong instructions.

Using pcf-vsphere-2.1-build.326.ova for opsmanger deployment

https://docs.pivotal.io/runtimes/pks/1-1/vsphere-om-config.html
Step 5: Create Networks Page

Doc says - Service Network Leave Service Network unchecked for network creation
but with the above mentioned build I don't see the option of "service network"

Feedback from user: "I looked around in https://docs.vmware.com/en/VMware-Pivotal-Container-Service/1.1/vmware-pks-11/GUID-PKS11-index.html, and I can’t find the doc for Wavefront and LogInsight."

The Release Notes (https://docs.vmware.com/en/VMware-Pivotal-Container-Service/1.1/rn/VMware-Pivotal-Container-Service-11-Release-Notes.html) contain links to the following sections:

• Adds support for PKS integration with VMware Wavefrontto capture metrics for clusters and pods. For more information, see (Optional) Logging.
• Adds support for PKS integration with VMware vRealize Log Insight (vRLi)for tagged logging of the control plane, clusters, and pods. For more information, see (Optional) Monitoring.

Need to incorporate these products into the documentation:

• Reference to LogInsight can go here: https://docs.pivotal.io/runtimes/pks/1-1/installing-pks.html#syslog
• Description of what we provide out-of-the-box via Wavefront and reference to Wavefront docs can go here: https://docs.pivotal.io/runtimes/pks/1-1/installing-pks.html#monitoring

https://docs-pks.cfapps.io/pks/architecture.html

Image is out of sync with the text (objects called out are not in image). Image is stale

Google Cloud APIs doesn't seem to contain Compute Engine API any longer. It appears under a generic Compute category. Might be best to just use the search box, as in step 6.

Under Installing and Configuring PKS on GCP -> Create a Firewall Rule for the Load Balancer we recommended that a Target tag of pks-api is assigned to the rule. This tag isn't assigned to the PKS VM so the firewall rule will never route correctly.

A unique value that is generated by Ops Manager during tile installation is assigned to the VM as a tag. This is the product uuid from Ops Manager. That might be a better value to assign to the tag.

We're writing to let you know that the group you tried to contact (pks-feedback) may not exist, or you may not have permission to post messages to the group. A few more details on why you weren't able to post:

Near-duplicate wording/garbled sentence: tags tag apply

Upgrade steps 2 and 3 should be merged under the upgrade instructions https://docs.pivotal.io/runtimes/pks/1-1/upgrade-pks.html. The reason behind that is NSX-T properties are configured as part of PKS tile. Step 2 requires the reader to Upgrade the PKS Tile and hit apply changes. However before hitting Apply Changes all the NSX-T properties should also be configured in order for the upgrade to be successful in NSX-T environment.

the screenshot here - https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-gcp.html#azs-networks

the Network and Services Network are shown as pcf-management-network and pcf-services-network, it should probably show them as infrastructure and pks-services as described in https://docs.pivotal.io/runtimes/pks/1-0/gcp-om-config.html#network for consistency.

Network names need to be lower-case. If you type in "MY-PKS-virt-net" as instructed it errors with Name must be lowercase letters, numbers, and hyphens

Hi,

In the roles mentioned Compute Viewer is missing. Earlier it was there for v1.0.2 where there was only one role for master as well as worker.

When the role is missing for Master, k8s cluster could be created, but k8s services like "kube-dns" has issue with bootstrapping hence throws error in creation.

Looking at the kube-controller-manager logs, the missing google cloud service account privileges error is seen.

Thanks
Darshit Sanghavi

On the https://docs.pivotal.io/runtimes/pks/1-0/manage-users.html page, the Retrieve UAA Admin Credentials step comes after Log in as an Admin.

However, the steps should be reversed as we need the credentials before we can login. step 5 of Log in as an Admin actually references that fact.

Upgrade of NSX-T tile is not required as PKS never uses NSX-T tile. All the NSX-T configurations are embedded into PKS tile and configured there. https://docs-pcf-staging.cfapps.io/runtimes/pks/1-1/upgrade-pks.html

Currently in many of the files we have a section repeated, it seems unnecessary to repeat it in so many places. We could link to another page which contains this info, or put it as a note. It doesnt add much value to each page, and makes there longer than needed.

On the command line, run the following command to log in:

pks login -a PKS_API -u USERNAME -p PASSWORD

Replace the placeholder values in the command as follows:
PKS_API is your PKS API hostname. PKS_API is the domain name you entered in Ops Manager > Pivotal Container Service > UAA > UAA URL and includes the port number for the PKS API. For example, https://api.pks.example.com:9021. The PKS API uses port 9021 by default.
USERNAME is your PKS API username. The USERNAME is the user created as a result of the Grant Cluster Access to a User step in the Manage Users in UAA topic.
PASSWORD is your PKS API password. The PASSWORD is the password created as a result of the Grant Cluster Access to a User step in the Manage Users in UAA topic.

• https://docs.pivotal.io/runtimes/pks/1-0/view-cluster-list.html
• https://docs.pivotal.io/runtimes/pks/1-0/cluster-credentials.html
• https://docs.pivotal.io/runtimes/pks/1-0/view-cluster-list.html
• https://docs.pivotal.io/runtimes/pks/1-0/view-cluster-details.html
• https://docs.pivotal.io/runtimes/pks/1-0/view-cluster-plans.html
• https://docs.pivotal.io/runtimes/pks/1-0/scale-clusters.html
• https://docs.pivotal.io/runtimes/pks/1-0/delete-cluster.html

Pivotal provides the Gitbot service to synchronize issues and pull requests made against public GitHub repos with Pivotal Tracker projects.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request.

Steps:

• Fork this repo: cfgitbot-config (an ask+cf@ ticket is the fastest way to get read access if you get a 404)
• Add the Toolsmiths-Bots team to have admin access to your repo
• Add the cf-gitbot ([email protected]) user to have owner access to your Pivotal Tracker project
• Add your new repo and or project to config-production.yml file
• Submit a PR, which will get auto-merged if you've done it right

If you are not a pivotal employee, you can request that [email protected] set up the integration for you.

You might also be interested in configuring GitHub's Service Hook for Tracker on your repo so you can link your commits to Tracker stories. You can do this yourself by following the directions at:

https://www.pivotaltracker.com/blog/guide-githubs-service-hook-tracker/

If you do not want to use Pivotal Tracker to manage this GitHub repo, Please add this repo to the Ignored repositories list

If there are any questions, please reach out to [email protected].

installing-pks-vsphere.html.md.erb refers to
To modify the resource usage of PKS, click Resource Config and edit the PKS on-demand broker job. If you created a PKS API load balancer, enter a name that begins with tcp: in the Load Balancers column. For example, tcp:pks-api.

but I see no column Load Balancers in Ops Manager 2.0 with Pivotal Container Service with vSphere.

The role mentioned in the document has been deprecated by GCP

Ref: https://cloud.google.com/iam/docs/service-accounts#the_service_account_actor_role

In the release notes documented here: "https://docs.pivotal.io/runtimes/pks/1-0/release-notes.html#v1.0.2-upgrade"
Please add the following step before step 6 ("6. Follow the procedures in Upgrade PKS.")
6. Download the latest 3468.x stemcell and configure the PKS tile with the stem cell.

This has been relayed from a filed testing-resource:

https://docs.pivotal.io/runtimes/pks/1-1/vsphere-requirements.html

from William Lam

Customer: VVD Team (Internal)

Desired Outcome: We clearly document the minimum CPU/MEM/STORAGE reservations required to host PKS Management VMs in the scenario where they are running alongside the K8S Workload and having a vSphere Resource Pool to isolate between Mgmt and Workload components.

Why: Today, the documentation are quite limiting and only state that Resource Pools can be used where both the Management and Compute Cluster is consolidated. We do not state that RPs must be created in advanced compared to other parts of the installation, where PKS will create the objects if they do not exists. More importantly, we should document the expected resource consumption for the PKS Mgmt Cluster. I know we document the the latter three VMs, but we don't explicitly call out the Compilation VMs that are dynamically spun up during initial K8S provisioning as well as for BOSH configuration/deployment. If this is miss-sized, then we could impose an artificial limit and starve resources to the PKS Mgmt components. We also want to make sure we size the RPs so that under contention, K8S Workloads will not impact PKS Mgmt Clusters which is the whole point of using RPs.

From my limited investigation, I have observed the following 7 VMs that could be spun up at a given point in time which we should account for when configuring the RP.

4 x Compliation VMs (4 vCPU and 4 GB memory)
1 x BOSH VM (2 vCPU and 8 GB memory)
1 x PKS Control Plane VM (1 vCPU and 1 GB memory)
1 x Harbor VM (2 vCPU and 8 GB memory)
There maybe more VMs, but ideally we clearly document the worse case scenario for each release and customers can then size RPs accordingly.

The Google Cloud Config section of the OpsManager Director tile is the configuration to be passed to Bosh to talk to the GCP API. We should use the BOSH / Ops Manager Service Account for this.
These two lines should be edited:

• https://github.com/pivotal-cf/docs-pks/blob/master/gcp-om-config.html.md.erb#L84
• https://github.com/pivotal-cf/docs-pks/blob/master/gcp-om-config.html.md.erb#L142

Where it says master node service account it should read BOSH / Ops Manager Service Account

This should be

bosh -e MY-ENVIRONMENT task TASK-NUMBER

For example:

$ bosh -e pks task 23

task and not tasks.

https://docs.pivotal.io/runtimes/pks/1-0/gcp-prepare-env.html
https://github.com/pivotal-cf/docs-pks/blob/master/gcp-prepare-env.html.md.erb

Step 1 is for setting up an IAM Service Account and Step 2 is for enabling Google Cloud API.

The order of this steps shouldn't bother if the GCP account/project is not newly created. In case of a newly created GCP account, the Compute Engine roles don't show up unless the API is enabled under Step 2.

Step 2 should be swapped with Step 1

The stemcell version specified at https://github.com/pivotal-cf/docs-pks/blob/master/release-notes.html.md.erb#L63 is 3451.25, it should be 3541.25

Working through some of the Pivotal docs, specifically at how to Deploy Ops Manager to vSphere (https://docs.pivotal.io/runtimes/pks/1-1/vsphere-om-deploy.html) all the screenshots are from the vSphere C# client and not the web client! Should we ask to get this updated? Whilst the deployment might work now, future versions may not expose all the configuration settings when deploying the OVA through the C# client?

On the https://docs.pivotal.io/runtimes/pks/1-0/manage-users.html page, under the Retrieve UAA Admin Credentials heading, step 3 points to Uaa Admin Secret from the tile to get the admin client credentials.

Today in the tile for 1.0 it is actually called Pks Uaa Management Admin Client

Firewall Rules also appears under VPC network. There is no Networking pane.

I just tried with PKS 1.0 tile and followed the official document on https://docs.pivotal.io/runtimes/pks/1-0/index.html

I notice that in the document https://docs.pivotal.io/runtimes/pks/1-0/using-prerequisites.html is very brief, it does not mention anything about setting up UAA users.

Maybe at https://docs.pivotal.io/runtimes/pks/1-0/create-cluster.html, add a link to https://docs.pivotal.io/runtimes/pks/1-0/manage-users.html ? So that for someone who follows the steps to install PKS would quickly find the steps to set up UAA users.

Hi Team,

While trying to add a new user then providing access Getting Error like below

CF::UAA::NotFound: CF::UAA::NotFound

ubuntu@pksopsmgr:$ uaac user add alana --emails [email protected] -p password
user account successfully added
ubuntu@pksopsmgr:$ uaac member add pks.clusters.admin alana

CF::UAA::NotFound: CF::UAA::NotFound

Can you please suggest how to resolve this and

Is there is a simple way to login to pks

I am trying to follow doc
https://docs.pivotal.io/runtimes/pks/1-0/create-cluster.html#cluster-access

But information provided there is very tidious.

Please Share some simple steps I can do to Login :

pks login -a PKS_API -u USERNAME -p PASSWORD --ca-cert CERT-PATH

Best Regards
Ganesh Kumar

• PKS 1.1.0 section is still there but the release is not available on Pivnet
• Minor Typo

VMware NSX-T | 2.1 - Advanced Edition | NST-T versions supported for Pivotal Container Service (PKS)

• There is a single component version table for all three version PKS 1.1.0, 1.1.1 and 1.1.2 and the component version still states that the supported Opsmgr version is 2.1.0-2.1.6 which is not the case for publicly available releases as per Pivnet
• VMware Harbor Registry version compatible is 1.5.0 whereas there is no 1.5.0 release on Pivnet. I only see three versions publicly available 1.4.2, 1.5.1 and 1.5.2 out of which only 1.5.x is compatible with PKS 1.1.x

In the section Preparing to Deploy PKS on GCP (Step 2: Create Service Accounts) you are advised to create two Service Accounts (for Kubernetes Master and Worker nodes). But you also need a Service Account for OpsManager and Bosh to use.

Preparing OpsManager in GCP
https://github.com/pivotal-cf/docs-pks/blob/master/gcp-om-deploy.html.md.erb#L84
It links back to Preparing to Deploy PKS on GCP, but the link is broken. It should link to Step 2: Create Service Accounts and specify that the Service Account to use is the Master one.

Configuring Ops Manager in GCP
https://github.com/pivotal-cf/docs-pks/blob/master/gcp-om-config.html.md.erb#L84
It links back to Preparing to Deploy PKS on GCP (Step 2: Create Service Accounts) but it doesn't specify which Service account to use from the two that you are requested to create (Master and Worker). It should specify to use the Master one.

https://docs.pivotal.io/runtimes/pks/1-1/installing-nsx-t.html#certificates-nsx-manager

Step 6.2.2: Register the Certificate with NSX Manager

Import the certificate by copying nsx.crt and nsx.key. For instructions, see Import a CA Certificate in the NSX-T documentation.

It should be "Import a Certificate".