Describe security policy handling in v4 about doc-caasp HOT 4 CLOSED

Extracted from SUSE/avant-garde#15:

We are currently deploying by default (there's no optional way) two PSP policies:

• privileged (full access everywhere)
• unprivileged (pretty much no access apart from the very basic stuff)

This is a binary situation -- all or nothing scenario. This what happens by default:

• All the containers running kube-system namespace are using the privileged policy
• All the new containers (from the customer) are using the unprivileged policy.

This means that if the customer wants to use some extra capabilities he can't -- unless he creates his own PSP. That is a wanted behavior. See:

Yes, they are encouraged to provide their own PSP's and they can create bindings as required. We can also provide a set of extra PSP's if we want with specific use cases, but for now our privileged PSP for kube-system namespace and a general unprivileged pod security policy looks fine to me.

Also maybe we need to add some links to our documentation pointing to the k8s documentation related to PSPs. A starting point might be: kubectl explain podsecuritypolicy

Last but not least I have written some stuff on the commit message of the PR https://github.com/SUSE/caaspctl/pull/96/commits

* The caaspctl cluster init command should create the deployment
  files of v3 pod security policies and later apply them during
  cluster  bootstrap. Also include the 'clusterrolebinding' as part
  of the PSP manifests.
* Run kube-system pods only in privileged mode:
  Setups a ClusterRole which uses the privileged policy. Then a
  a RoleBinding references this ClusterRole and pairs with all
  the (authenticated users and all ServiceAccounts) in kube-system
  namespace. So it grants usage of privileged PSP for pods.
  being run into kube-system namespace.
* Any other pods running at other namespaces are using the
  unprivileged PSP by default.
* Replace apiVersion extensions/v1beta1 to policy/v1beta1.
* Remove Apparmor annotation.
* Enable PSPs via admission controller.

If anything is not clear or you need more clarification, don't hesitate pinging me :)

from doc-caasp.

Relates to: https://github.com/SUSE/avant-garde/issues/156

from doc-caasp.

Relates to: https://github.com/SUSE/avant-garde/issues/221

from doc-caasp.

First pass uploaded to PR. Closing issue, tracking PR from now on.

from doc-caasp.