I don't get why are we using the weird notion of kubernetes- that breaks Go Dep sem-version matching logic... was there a reason behind it or just evolved that way?

We (Pulumi) are using client-go and apimachinery directly to manage Kubernetes resources.

We've found that calling *unstructured.Unstructured#MarshalJSON on any Unstructured that contains an object with types apimachinery does not expect will immediately cause a panic. The relevant code seems to be here.

Needless to say, this is very unfriendly for a library to do. :) It would be nice if one of the following was true

1. MarshalJSON simply returned an error in this case, or
2. there was clear guidance about how to validate objects going into Unstructured so that at least we can check them before calling them—perhaps as a constructor.

Perhaps I'm missing something?

In the latest version of kubernetes/apimachinery，the includeUninitialized parameter is deprecated，how can my initializer controller list and watch for uninitialized objects now?
Thanks!

We are using the StreamWatcher and seeing extremely high memory allocations. Attached is a memory profile:

pprof.pilot-discovery.alloc_objects.alloc_space.inuse_objects.inuse_space.033.pb.gz

From this we have 9.6 GB of memory allocated by the StreamWatcher.

To get to this state I had a deployment with 200 replicas, and every 30 seconds I redeployed all of the pods (kubectl rollout restart). I left this running overnight and saw this memory usage.

With pretty much the latest of everything, I'm getting:

+ GOPATH=/builddir/build/BUILD/apimachinery-kubernetes-1.9.6/_build:/usr/share/gocode
+ go test -buildmode pie -compiler gc -ldflags '-extldflags '\''-Wl,-z,relro  '\'''
--- FAIL: TestMicroTimeMarshalYAML (0.00s)
        micro_time_test.go:49: Failed to marshal input: '{1998-05-05 01:05:05.00000005 -0400 test}': expected t: 1998-05-05T05:05:05.000000Z
                , got "t: \"1998-05-05T05:05:05.000000Z\"\n"
        micro_time_test.go:49: Failed to marshal input: '{1998-05-05 05:05:05 +0000 UTC}': expected t: 1998-05-05T05:05:05.000000Z
                , got "t: \"1998-05-05T05:05:05.000000Z\"\n"
--- FAIL: TestTimeMarshalYAML (0.00s)
        time_test.go:49: Failed to marshal input: '{1998-05-05 01:05:05.00000005 -0400 test}': expected t: 1998-05-05T05:05:05Z
                , got "t: \"1998-05-05T05:05:05Z\"\n"
        time_test.go:49: Failed to marshal input: '{1998-05-05 05:05:05 +0000 UTC}': expected t: 1998-05-05T05:05:05Z
                , got "t: \"1998-05-05T05:05:05Z\"\n"
FAIL
exit status 1
FAIL    k8s.io/apimachinery/pkg/apis/meta/v1    0.068s

...to help dep users get the correct version of dependencies (e.g., k8s.io/apimachinery).

I am using the following version:

require (
        k8s.io/api v0.17.0
        k8s.io/apimachinery v0.17.0
        k8s.io/client-go v0.17.0
)

In the documentation it says:

NewSerializerWithOptions creates a JSON/YAML serializer that handles encoding versioned objects into the proper JSON/YAML pfadmission form. If typer is not nil, the object has the group, version, and kind fields set. Options are copied into the Serializer and are immutable.

In my test I add the AdmissionReview to the scheme via the schemebuilder. With s.ObjectKinds it seems that the scheme knows about the object, but in the test, I get the following result, even with the typer set:

{
  "response": {
    "uid": "",
    "allowed": true
  }
}

package api

import (
	"bytes"
	"fmt"
	"testing"

	"github.com/stretchr/testify/require"
	admissionv1 "k8s.io/api/admission/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/serializer/json"
)

func TestSerialize(t *testing.T) {
	buf := bytes.NewBufferString("")
	s := runtime.NewScheme()
	builder := runtime.SchemeBuilder{
		admissionv1.AddToScheme,
	}
	builder.AddToScheme(s)
	serializer := json.NewSerializerWithOptions(json.DefaultMetaFactory, s, s, json.SerializerOptions{
		Pretty: true,
		Yaml:   false,
		Strict: true,
	})
	ar := &admissionv1.AdmissionReview{
		Response: &admissionv1.AdmissionResponse{
			Allowed: true,
		},
	}
	_, _, err := s.ObjectKinds(ar) 
	require.NoError(t, err) // no error, so scheme knows the AdmissionReview object
	err = serializer.Encode(ar, buf)
	require.NoError(t, err)
	fmt.Println(buf.String())
}

Without the TypeMeta set, I get the following error from the configured admission controller:

Error from server (InternalError): Internal error occurred: failed calling webhook "validation.pfadmission.example.com": expected webhook response of admission.k8s.io/v1, Kind=AdmissionReview, got /, Kind=

Hi

I have a project which need this repo as external dependence.

My environment is bazel 0.11.0, Docker engine 17.12 docker compose 1.18 .

After adding it to my WORKSPACE and try to build it, it throw an error information like below:
ERROR: /root/cache/bazel/21cdec581d01b82fb2aa0eb823f1cae8/external/com_k8s_io_apimachinery/pkg/apis/meta/v1/BUILD.bazel:9:1: @com_k8s_io_apimachinery//pkg/apis/meta/v1:go_default_test: no such attribute 'embed' in 'go_test' rule

Is the bazel version issue?

Does anyone know this error?

The pluralization logic (here)[https://github.com/kubernetes/apimachinery/blob/master/pkg/api/meta/restmapper.go#L141] caused us issues with a custom resource definition that we were testing. The resource name that we were using was a FabricGateway which when pluralized should have been FabricGateways, however it was incorrectly being pluralized as FabricGatewaies.

Hi,

I'm working on getting an Istio VirtualService client working, and am running into the following error:

admission webhook "pilot.validation.istio.io" denied the request: error decoding configuration: YAML decoding error: gateways:
- test-gateway
hosts:
- '*'
http:
- match:
  - uri:
      MatchType:
        Prefix: /
  route:
  - destination:
      host: test-service
      port:
        Port:
          Number: 8080
 unknown field "MatchType" in v1alpha3.StringMatch 

This is because the match should encode to:

- match:
  - uri:
    -prefix: /

Ultimately, the encoding/json package is being used to encode the object, which includes a Protobuf for the VirtualService:

The VirtualService is defined here: https://github.com/istio/api/blob/master/networking/v1alpha3/virtual_service.proto

Is there a non-obvious way for me to control the encoding of this object?

I can reproduce the error here: michaelkipper/istio-client-go#6

Thanks!
Michael

I understand the sync-from-kubernetes.sh can match up your client library to the kubernetes version, but for vendor tools like godeps, this tracks the commit checksum, so its not reproducible when rewriting the history.

I'm not sure how most users are using this library, but it would be great to be able to checkout k8s tags directly here instead of using the sync script.

It should return also error, not hide it and return selector matching everything. We used .AsSelector in our reaper which ended up deleting everything because of it. Sure, it was unvalidated input (selector value was "_aaaa") but not returning an error and ignoring it is a bug. Pretty sure this is widely used and error prone.

• The Struct is PytorchJob. Frist we convert the PytorchJob to Unstructed with json(inline mode).When we use function FromUnstructured convert Unstructed to PytorchJob,it cannot convert anonymous field "common.JobStatus" success.

  type PytorchJob struct {
      ...
      Status PytorchStatus `json:"status,omitempty"`
  }
  type PytorchStatus struct {
      common.JobStatus
      StatusForAdjust string `json:"statusForAdjust,omitempty"`
      ....
  }
  

• I printed the Unstructed and converted PytorchJob,PytorchJob does not have the Success status after the conversion is performed.And the Unstructured Status shows that the common.JobStatus part of the json is the inline attribute (excluding JobStatus Struct), but the PytorchJob receives it not through the inline form (including JobStatus Struct, from "{{[] map [] } 0 0}} ”can be seen)

  func (c *unstructuredConverter) FromUnstructured(u map[string]interface{}, obj interface{}) error {
      t := reflect.TypeOf(obj)
      value := reflect.ValueOf(obj)
      ....
      fmt.Println(reflect.ValueOf(u))
      err := fromUnstructured(reflect.ValueOf(u), value.Elem())
      fmt.Println(value.Elem())
      ...
  }
  

  map[kind:PytorchJob metadata:map[creationTimestamp:<nil> name:test-pytorchjob namespace:default] spec:map[block:<nil> cleanPodPolicy:All maxGPU:4 minGPU:4 pytorchReplicaSpecs:map[Master:map[replicas:1 template:map[metadata:map[creationTimestamp:<nil>] spec:map[containers:[map[args:[Fake Fake] image:test-image-for-kubeflow-pytorch-operator:latest name:pytorch ports:[map[containerPort:23456 name:pytorchjob-port]] resources:map[]]]]]] Worker:map[replicas:4 template:map[metadata:map[creationTimestamp:<nil>] spec:map[containers:[map[args:[Fake Fake] image:test-image-for-kubeflow-pytorch-operator:latest name:pytorch ports:[map[containerPort:23456 name:pytorchjob-port]] resources:map[]]]]]]] topologyGPU:<nil>] status:map[conditions:[map[lastTransitionTime:2019-12-14T09:02:53Z lastUpdateTime:2019-12-14T09:02:53Z reason:PytorchJobSucceeded status:True type:Succeeded]] replicaStatuses:<nil>]]
  
  {{PytorchJob } {test-pytorchjob  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] nil [] } {<nil> <nil> 0xc0002bb0d0 <nil> 0xc000608cf8 0xc000608d08 <nil> <nil> map[Master:0xc000139600 Worker:0xc000139b80]} {{[] map[] <nil> <nil> <nil>}  0 0}}
  

• When the conversion to common.JobStatus is performed, the else part of the code will be executed (I have Printed fieldInfo.name is JobStatus), and the function will take the value of the key "JobStatus" from the sv map according to JobStatus, which is obviously not found. Therefore, a zero value is set for the JobStatus

  func structFromUnstructured(sv, dv reflect.Value) error {
      st, dt := sv.Type(), dv.Type()
      if st.Kind() != reflect.Map {
          return fmt.Errorf("cannot restore struct from: %v", st.Kind())
      }
  
      for i := 0; i < dt.NumField(); i++ {
          fieldInfo := fieldInfoFromField(dt, i)
          fv := dv.Field(i)
          if len(fieldInfo.name) == 0 {
              // This field is inlined.
              if err := fromUnstructured(sv, fv); err != nil {
                  return err
              }
          } else {
              value := unwrapInterface(sv.MapIndex(fieldInfo.nameValue))
              if value.IsValid() {
                  if err := fromUnstructured(value, fv); err != nil {
                      return err
                  }
              } else {
                  fv.Set(reflect.Zero(fv.Type()))
              }
          }
      }
      return nil
  }
  

  status:map[conditions:[map[lastTransitionTime:2019-12-14T09:02:53Z lastUpdateTime:2019-12-14T09:02:53Z reason:PytorchJobSucceeded status:True type:Succeeded]] replicaStatuses:<nil>]]
  

• We need to handle the case where anonymous filed is included in Struct and the field is converted to Json with inline mode .After modification, we successfully converted it.

      for i := 0; i < dt.NumField(); i++ {
          fieldInfo := fieldInfoFromField(dt, i)
          fv := dv.Field(i)
          // if we cannot found fieldInfo.name of Anonymous field in json,the json is inline mode for this Anonymous field
          _, ok := sv.Interface().(map[string]interface{})[fieldInfo.name]
          if len(fieldInfo.name) == 0 || dt.Field(i).Anonymous && !ok {
              // if len(fieldInfo.name) == 0 {
              // This field is inlined.
              if err := fromUnstructured(sv, fv); err != nil {
                  return err
              }
          } else {
              ...
          }
      }
  

It would be nice to provide support for label selector comparisons of major version numbers and other Float values possibly for other use cases.

[{"key": "version", "operator": "Gt", "values": ["1.5"]}]

The current NewRequirement and Matches function only support comparison of Integers.

Any concern with updating this to convert and compare Floats instead of Integers?
Would it need to support both and only convert and compare Floats if a decimal character is found or is that overkill?

My problem:

Changing any client configuration through command-line arguments prevents me from using the in-cluster configuration. It falls back to default configuration which is clearly surprising for the user.

Steps to reproduce:

# Run a random container in the cluster
kubectl run --restart=Never --image=nginx test
# Copy a kubectl binary in there
kubectl cp "$(which kubectl)" test:/tmp/kubectl

# WORKING CASE:

# Try to get pods, using in-cluster config
kubectl exec test -- /tmp/kubectl get pods -v7
# Gets the pods, yay
I0801 21:14:50.499145      14 merged_client_builder.go:123] Using in-cluster configuration
I0801 21:14:50.499496      14 round_trippers.go:395] GET https://10.67.240.1:443/api
...

# BROKEN CASE:

# Try to get pods, using in-cluster config, but change a random configuration flag
kubectl exec test -- /tmp/kubectl get pods -v7 --request-timeout=5
I0801 21:14:58.051543      27 round_trippers.go:395] GET http://localhost:8080/api
# It doesn't use in-cluster configuration anymore, and tries to connect to localhost ..

When running Helm 2.9.0-rcX in Bash for Windows, the entire application crashes.
This version of Helm was the first with support for K8S 1.10.
Below error does not appear in Helm 2.8.x or earlier.
Got advice from Helm core maintainer in helm/helm#3833 to report it here.

runtime: epollwait on fd 7 failed with 38
fatal error: runtime: netpoll failed

runtime stack:
runtime.throw(0x1619830, 0x17)
/usr/local/go/src/runtime/panic.go:616 +0x81
runtime.netpoll(0x5d207926fc500, 0xc400000000)
/usr/local/go/src/runtime/netpoll_epoll.go:75 +0x216
runtime.sysmon()
/usr/local/go/src/runtime/proc.go:4265 +0x424
runtime.mstart1(0x0)
/usr/local/go/src/runtime/proc.go:1227 +0xe7
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1193 +0x76

goroutine 1 [runnable, locked to thread]:
k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypeWithName(0xc420320c40, 0x1614fc4, 0x14, 0x1600cac, 0x2, 0x12c31f7, 0xd, 0x17486a0, 0xc4203218c0)
/go/src/k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime/scheme.go:199 +0x498
k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime.(*Scheme).AddKnownTypes(0xc420320c40, 0x1614fc4, 0x14, 0x1600cac, 0x2, 0xc4204dc7b0, 0x4, 0x4)
/go/src/k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime/scheme.go:167 +0x168
k8s.io/helm/vendor/k8s.io/apimachinery/pkg/apis/meta/v1.AddToGroupVersion(0xc420320c40, 0x1614fc4, 0x14, 0x1600cac, 0x2)
/go/src/k8s.io/helm/vendor/k8s.io/apimachinery/pkg/apis/meta/v1/register.go:50 +0x39d
k8s.io/helm/vendor/k8s.io/api/authorization/v1.addKnownTypes(0xc420320c40, 0x0, 0x0)
/go/src/k8s.io/helm/vendor/k8s.io/api/authorization/v1/register.go:53 +0x1a6
k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme(0x20a9d70, 0xc420320c40, 0x0, 0x0)
/go/src/k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime/scheme_builder.go:29 +0x6d
k8s.io/helm/vendor/k8s.io/apimachinery/pkg/runtime.(*SchemeBuilder).AddToScheme-fm(0xc420320c40, 0x0, 0x0)
/go/src/k8s.io/helm/vendor/k8s.io/api/core/v1/register.go:41 +0x34
k8s.io/helm/vendor/k8s.io/client-go/kubernetes/scheme.AddToScheme(0xc420320c40)
/go/src/k8s.io/helm/vendor/k8s.io/client-go/kubernetes/scheme/register.go:87 +0xc9
k8s.io/helm/vendor/k8s.io/kubernetes/pkg/kubectl/scheme.init.0()
/go/src/k8s.io/helm/vendor/k8s.io/kubernetes/pkg/kubectl/scheme/install.go:60 +0x85

goroutine 5 [chan receive]:
k8s.io/helm/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x20abb40)
/go/src/k8s.io/helm/vendor/github.com/golang/glog/glog.go:879 +0x8b
created by k8s.io/helm/vendor/github.com/golang/glog.init.0
/go/src/k8s.io/helm/vendor/github.com/golang/glog/glog.go:410 +0x203

goroutine 7 [syscall]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:139 +0xa6
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:28 +0x41

There is no added value by that conversion, even worse it can be misused.

@kubernetes/sig-api-machinery-bugs

ObjectMeta.CreationTimestamp's json tag says omitempty, but it's a struct so as far as I understand it can never satisfy the "empty" check (zero value or nil).
https://github.com/kubernetes/apimachinery/blob/master//pkg/apis/meta/v1/types.go#L199

Every other field in ObjectMeta is of type that can be zero or nil (slices and primitives). Also, we can take example from ObjectMeta.DeletionTimestamp which is a pointer to struct.

Hello,

I began getting the following error when importing the k8s.io/client-go/rest package.

go/pkg/mod/k8s.io/[email protected]+incompatible/rest/request.go:598:31: not enough arguments in call to watch.NewStreamWatcher
	have (*versioned.Decoder)
	want (watch.Decoder, watch.Reporter)

Upon doing some investigation it seems that the NewStreamWatcher function in was changed fairly recently:
6bfc472#diff-13d4c1aef2b5ff5ac3d279505679acc0L52

This change though wasn't reflected in the call to NewStreamWatcher in request.go:
https://github.com/kubernetes/client-go/blob/master/rest/request.go#L598

It seems that only a single decoder struct gets returned from the NewDecoder function:
https://github.com/kubernetes/client-go/blob/master/rest/watch/decoder.go#L39

Is this something that's planned to be fixed?

Hi Team,

Can someone pls share the version and revision details that can be used in the go.mod file for Release 1.10.0-beta.2?

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md

I'm currently relying on k8s.io/apimachinery/pkg/apis/meta/v1 package in a Go project, as well as sigs.k8s.io/yaml.

However, it seems like k8s.io/apimachinery/pkg/apis/meta/v1.test is using github.com/ghodss/yaml instead:

go mod why github.com/ghodss/yaml
# github.com/ghodss/yaml
<<my project>>
k8s.io/apimachinery/pkg/apis/meta/v1
k8s.io/apimachinery/pkg/apis/meta/v1.test
github.com/ghodss/yaml

When I look at the code (https://sourcegraph.com/search?q=repo:%5Egithub%5C.com/kubernetes/apimachinery%24+ghodss) I only see the package listed in go.mod and Godeps.json, but it's actually not used in *.go files:

Godeps/Godeps.json
46:			"ImportPath": "github.com/ghodss/yaml",

pkg/api/meta/OWNERS
27:- ghodss

go.sum
17:github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680 h1:ZktWZesgun21uEDrwW7iEV1zPCGQldM2atlJZ3TdvVM=
18:github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=

I am trying to unmarshal CR with type PersistentVolumeClaimSpec struct and go the following error (for both YAML and JSON):

cannot unmarshal !!str `2Gi` into resource.Quantity

for

yaml.Unmarshal(str)

where yaml is "gopkg.in/yaml.v2" and str is either YAML or JSON:

apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "default-volumeclaimtemplate"
spec:
  defaults:
    templates:
      volumeClaimTemplate: default-volumeclaim-template
  templates:
    volumeClaimTemplates:
      - name: default-volumeclaim-template
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 2Gi

or

{
  "apiVersion": "clickhouse.altinity.com/v1",
  "kind": "ClickHouseInstallation",
  "metadata": {
    "name": "default-volumeclaimtemplate"
  },
  "spec": {
    "defaults":{
      "templates": {
        "volumeClaimTemplate": "default-volumeclaim-template"
      }
    },
    "templates": {
      "volumeClaimTemplates": [
        {
          "name": "default-volumeclaim-template",
          "spec": {
            "accessModes": [
              "ReadWriteOnce"
            ],
            "resources": {
              "requests": {
                "storage": "2Gi"
              }
            }
          }
        }
      ]
    }
  }
}

Any ideas how to solve? Thanks in advance!

I use client-go to watch k8s service and endpoints like this:
clientset.CoreV1().Services("").Watch(metav1.ListOptions{})

But every 30~50 minittues，the StreamWatcher will receive the io.EOF.
"(sw *StreamWatcher) receive()" receive io.EOF and stop.
And I have to watch again.

I try to use the k8s.io/client-go v7.0.0, and i've got this error :

vendor/k8s.io/apimachinery/pkg/runtime/serializer/json/json.go:115:16: unknown field 'CaseSensitive' in struct literal of type jsoniter.Config

Same error as in operator-framework/getting-started#16 i think.

Comparing apimachinery/pkg/api/meta/restmapper.go and gengo/blob/master/namer/plural_namer.go, I find that apimachinery and gengo have a different algorithm of pluralize name, it's fine at most time, but it becomes tricky when the name ends with capital 'S'.

In apimachinery add "es" will pass the test:

{Kind: "MongoRS", Plural: "mongorses", Singular: "mongors"}

But in gengo add 's' will pass the test:

{
    "MongoRS",
    "mongorss",
    "MongoRSs",
},

It seems that in apimachinery, ToLower() first and then add 'es' after lowercase 's', but in gengo add 's' after Capital 'S' first then ToLower().

the package k8s.io/gengo/namer was used in k8s.io/code-generator and I use it to generate CRD client and it's different from apimachinery generated rest api.

Maybe apimachinery may use funcs in gengo to stay the same.

Hi all.

Given that the API types are actually protobufs, I would expect to use jsonpb to Marshal/Unmarshal, not the standard json package.

Here was my setup:

package clienttest

import (
	"bytes"
	"encoding/json"
	"testing"

	corev1 "k8s.io/api/core/v1"
	"k8s.io/apimachinery/pkg/api/resource"

	"github.com/gogo/protobuf/jsonpb"
	"github.com/stretchr/testify/assert"

	log "github.com/sirupsen/logrus"
)

func quantityOrDie(s string) resource.Quantity {
	q, err := resource.ParseQuantity(s)
	if err != nil {
		log.Fatal(err)
	}
	return q
}

func getContainer() corev1.Container {
	return corev1.Container{
		Resources: corev1.ResourceRequirements{
			Limits: corev1.ResourceList{
				"cpu": quantityOrDie("0.2"),
				"ram": quantityOrDie("2000000000"),
			},
		},
	}
}

func TestMarshalContainerJsonpb(t *testing.T) {
	c := getContainer()
	m := jsonpb.Marshaler{}
	s, err := m.MarshalToString(&c)
	assert.NoError(t, err, "Could not marshal Container")
	log.WithField("buffer", s).Info("Marshalled Container")

	cc := corev1.Container{}
	u := jsonpb.Unmarshaler{}
	err = u.Unmarshal(bytes.NewBufferString(s), &cc)
	assert.NoError(t, err, "Could not unmarshal Container")
	log.WithField("container", cc.String()).Info("Unmarshalled Container")
}

func TestMarshalContainerJson(t *testing.T) {
	c := getContainer()
	b, err := json.Marshal(c)
	assert.NoError(t, err, "Could not marshal Container")
	log.WithField("buffer", string(b)).Info("Marshalled Container")

	cc := corev1.Container{}
	err = json.Unmarshal(b, &cc)
	assert.NoError(t, err, "Could not unmarshal Container")
	log.WithField("container", cc.String()).Info("Unmarshalled Container")
}

The error message is:

--- FAIL: TestMarshalContainerJsonpb (0.00s)
    resource_requirements_test.go:47: 
        	Error Trace:	resource_requirements_test.go:47
        	Error:      	Received unexpected error:
        	            	json: cannot unmarshal string into Go value of type map[string]json.RawMessage
        	Test:       	TestMarshalContainerJsonpb
        	Messages:   	Could not unmarshal Container

Also, when I switched from github.com/gogo/protobuf/jsonpb to github.com/golang/protobuf/jsonpb things got even worse:

--- FAIL: TestMarshalContainerJsonpb (0.00s)
panic: reflect.Value.Addr of unaddressable value [recovered]
	panic: reflect.Value.Addr of unaddressable value

Should I be using the json package with these types?

Thanks,
-mk

We had a strange bug with kubebuilder 2.0 and Quantity.

We are building ResourceQuotas and comparing them to the existing. Sometimes, they do not compare correctly (false change) as we do

requestsCPU, _  := resource.ParseQuantity(resourceQuota.Requests.CPU)

type operations to build the desired state.
if the code

default:
    if scale%3 == 0 && !strings.HasSuffix(shifted, "000") && shifted[0] != '0' {
        return Quantity{i: int64Amount{value: result, scale: Scale(scale)}, Format: format, s: str}, nil
    }
}

is triggered, e.g. changing the scale, then DeepEqual will fail as attribute s will be set, but not in the one from kube as that will already be converted.

There is a fix to rebuild the quotas before comparing them but it is a gotcha

.
.
.
cq := make(map[corev1.ResourceName]string,6)
for k, v := range foundResourceQuota.Spec.Hard { 
    cq[k] = v.String()
}
if !reflect.DeepEqual(rq, cq) {
.
.
.

https://github.com/kubernetes/apimachinery/blob/master/pkg/api/resource/quantity.go#L324

Hi, I'd like to use the metav1.ObjectMeta type in my custom gRPC server so I imports k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto (https://github.com/kubernetes/apimachinery/blob/release-1.15/pkg/apis/meta/v1/generated.proto) to the custom proto file.
I have generated the *.pb.go files by proto-gen-gogo (v1.3.1). However, when I call the api which message proto have k8s type, the server panics.
I could not get some useful information from the panic messages. Am I make some mistakes in my proto file? Or this is a bug of protobuf tool chains?

I1112 14:40:12.719555       1 log.go:172] http: panic serving 10.0.2.10:46862: runtime error: slice bounds out of range
goroutine 134 [running]:
net/http.(*conn).serve.func1(0xc000262820)
        /usr/local/go/src/net/http/server.go:1769 +0x139
panic(0x1318ca0, 0x2309ea0)
        /usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.typeUnmarshaler(0x16e6fe0, 0x12ab8c0, 0x0, 0x0, 0x0)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:480 +0x2973
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.makeUnmarshalMap(0xc0000f7a40, 0x15)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:1934 +0x3c3
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.fieldUnmarshaler(0xc0000f7a40, 0x149ca4e)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:464 +0x51
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc00017e6e0)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:356 +0x58f
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc00017e6e0, 0xc0003d6b00, 0xc00047f506, 0x325, 0x327, 0x13e4d80, 0xc0007302d0)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:138 +0xe2e
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.makeUnmarshalMessagePtr.func1(0xc00047f506, 0x327, 0x329, 0xc000730ab0, 0x2, 0x0, 0x0, 0xc000149680, 0x198, 0x22c8640)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:1826 +0x135
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc00017e640, 0xc000730ab0, 0xc00047f503, 0x32a, 0x32a, 0xc0004fcea8, 0x10b2294)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:175 +0x7ae
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.makeUnmarshalMessagePtr.func1(0xc00047f503, 0x32a, 0x32c, 0xc000730960, 0x2, 0xc0004fcf68, 0x4702ca, 0xc000738060, 0xc0003d2c00, 0xc00073c840)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:1826 +0x135
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc00017e5a0, 0xc000730960, 0xc00047f500, 0x32d, 0x32d, 0x40a30c, 0xc000032000)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:175 +0x7ae
github.com/repo/proj/vendor/github.com/gogo/protobuf/proto.(*InternalMessageInfo).Unmarshal(0x2320540, 0x169bb00, 0xc000730960, 0xc00047f500, 0x32d, 0x32d, 0x470601, 0x7f12f500bbe0)
        /go/src/github.com/repo/proj/vendor/github.com/gogo/protobuf/proto/table_unmarshal.go:63 +0x68
github.com/repo/proj/pkg/proxy/api/v1.(*GetMatrixStatusResponse).XXX_Unmarshal(0xc000730960, 0xc00047f500, 0x32d, 0x32d, 0xc000730960, 0xc000730301)
        /go/src/github.com/repo/proj/pkg/proxy/api/v1/api.pb.go:458 +0x61
github.com/repo/proj/vendor/github.com/golang/protobuf/proto.(*Buffer).Unmarshal(0xc000730308, 0x169bb40, 0xc000730960, 0x0, 0x0)
        /go/src/github.com/repo/proj/vendor/github.com/golang/protobuf/proto/decode.go:399 +0xa3
github.com/repo/proj/vendor/google.golang.org/grpc/encoding/proto.codec.Unmarshal(0xc00047f500, 0x32d, 0x32d, 0x13e4e80, 0xc000730960, 0xc0004fd268, 0x4dcb5c)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/encoding/proto/proto.go:93 +0x12f
github.com/repo/proj/vendor/google.golang.org/grpc.recv(0xc000732680, 0x7f12f4f76b78, 0x2340588, 0xc0003d6900, 0x0, 0x0, 0x13e4e80, 0xc000730960, 0x400000, 0x0, ...)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/rpc_util.go:688 +0x10f
github.com/repo/proj/vendor/google.golang.org/grpc.(*csAttempt).recvMsg(0xc00073e480, 0x13e4e80, 0xc000730960, 0x0, 0x0, 0x0)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/stream.go:885 +0xed
github.com/repo/proj/vendor/google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc00073e480, 0xc00047f180, 0x7)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/stream.go:736 +0x46
github.com/repo/proj/vendor/google.golang.org/grpc.(*clientStream).withRetry(0xc000149560, 0xc0000d9518, 0xc0000d9508, 0xc00047f180, 0xc0006d2f70)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/stream.go:594 +0x7c
github.com/repo/proj/vendor/google.golang.org/grpc.(*clientStream).RecvMsg(0xc000149560, 0x13e4e80, 0xc000730960, 0x0, 0x0)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/stream.go:735 +0x100
github.com/repo/proj/vendor/google.golang.org/grpc.invoke(0x16b3b40, 0xc000730900, 0x14ee3f0, 0x49, 0x13e4d80, 0xc000730930, 0x13e4e80, 0xc000730960, 0xc000091180, 0xc000732560, ...)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/call.go:73 +0x141
github.com/repo/proj/vendor/google.golang.org/grpc.(*ClientConn).Invoke(0xc000091180, 0x16b3b40, 0xc000730900, 0x14ee3f0, 0x49, 0x13e4d80, 0xc000730930, 0x13e4e80, 0xc000730960, 0xc000732560, ...)
        /go/src/github.com/repo/proj/vendor/google.golang.org/grpc/call.go:37 +0x1b4
github.com/repo/proj/pkg/proxy/api/v1.(*matrixServiceClient).GetMatrixStatus(0xc0000fe020, 0x16b3b40, 0xc000730900, 0xc000730930, 0xc000732560, 0x2, 0x2, 0x0, 0x16b3a80, 0xc000350840)
        /go/src/github.com/repo/proj/pkg/proxy/api/v1/api.pb.go:1447 +0xd2
github.com/repo/proj/pkg/proxy/api/v1.request_MatrixService_GetMatrixStatus_0(0x16b3b40, 0xc000730900, 0x16b9000, 0x230dbc0, 0x16c98a0, 0xc0000fe020, 0xc0003d6800, 0xc000730870, 0xc00075e011, 0x5, ...)
        /go/src/github.com/repo/proj/pkg/proxy/api/v1/api.pb.gw.go:250 +0x164
github.com/repo/proj/pkg/proxy/api/v1.RegisterMatrixServiceHandlerClient.func6(0x16aa700, 0xc00014e0e0, 0xc0003d6800, 0xc000730870)
        /go/src/github.com/repo/proj/pkg/proxy/api/v1/api.pb.gw.go:727 +0x1ca
github.com/repo/proj/vendor/github.com/grpc-ecosystem/grpc-gateway/runtime.(*ServeMux).ServeHTTP(0xc000338310, 0x16aa700, 0xc00014e0e0, 0xc0003d6800)
        /go/src/github.com/repo/proj/vendor/github.com/grpc-ecosystem/grpc-gateway/runtime/mux.go:243 +0xbea
net/http.serverHandler.ServeHTTP(0xc000330680, 0x16aa700, 0xc00014e0e0, 0xc0003d6800)
        /usr/local/go/src/net/http/server.go:2774 +0xa8
net/http.(*conn).serve(0xc000262820, 0x16b3a80, 0xc000350680)
        /usr/local/go/src/net/http/server.go:1878 +0x851
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2884 +0x2f4

I try to downgrade proto version from 3 to 2, to use proto-gen-go instead of proto-gen-gogo, but it neither works.

(When use proto-gen-go, the panic message is:)

panic: message/group field v1.Time:bytes without pointer

Here is a part of my custom proto file. I've just used ObjectMeta type in k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto:

syntax = "proto3";
package api.v1;

import "google/api/annotations.proto";
import "k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto";

service {
  rpc GetMatrixStatus(GetMatrixStatusRequest)
    returns (GetMatrixStatusResponse) {
    option (google.api.http) = {
      get: "/v1/matrices/{name}/status"
      response_body: "status"
    };
  }
}

message GetMatrixStatusRequest {
  string name = 1;
}

message GetMatrixStatusResponse {
  MatrixStatus status = 1;
}

message MatrixStatus {
  k8s.io.apimachinery.pkg.apis.meta.v1.ObjectMeta metadata = 1;
  MatrixStatusDetail status = 2;
}

if I remove the imported type and define ObjectMeta by myself, it won't panic again.

message ObjectMeta {
  string name = 1;
  string namespace = 2;
  string uid = 3;
  map<string, string> labels = 4;
  map<string, string> annotations = 5;
}

message MatrixStatus {
  ObjectMeta metadata = 1;
  MatrixStatusDetail status = 2;
}

I'm not sure I'm framing this issue correctly, but I'm trying to pass multiple LabelSelectors to the metav1.ListOptions construct (currently I have them as a map[string]string), but the LabelSelector on that struct only takes a string.

Are multiple selectors intended to be passed as a single concatenated string? Is it even possible to specify multiple selectors for ListOptions? I found a possibly related conversion function, but I'm not sure that covers the correct case.

Apologies if this ended up in the wrong place. The reference is ksync/ksync#216 if that helps.

I encountered strange issue in the external resources added in /pkg/apis/... while logging resource events. The error message while getting the resources is

error: unknown type &ext.Group{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"****", GenerateName:"", Namespace:"", SelfLink:"/apis/***/v1alpha1/***", UID:"994f56a4-6f16-11e7-856b-74dbd1a04845", ResourceVersion:"55349629", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{sec:63636349651, nsec:0, loc:(*time.Location)(0x2b2e9e0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:""}, ***.GroupSpec{GroupDescription:"", Owner:"****", Users:[]string(nil)}, ***.GroupStatus{Phase:"Inactive", Conditions:[]***.GroupCondition(nil), StartTime:(*v1.Time)(nil)}}

After debugging the source code, and I found strange code in kubernetes between decode and versioning.

In direct decoder:

The apiVersion and Kind is override to null values in versioning.go: https://github.com/kubernetes/apimachinery/blob/master/pkg/runtime/serializer/versioning/versioning.go#L267-#L271.

// Decode does not do conversion. It removes the gvk during deserialization.
func (d DirectDecoder) Decode(data []byte, defaults *schema.GroupVersionKind, into runtime.Object) (runtime.Object, *schema.GroupVersionKind, error) {
	obj, gvk, err := d.Decoder.Decode(data, defaults, into)
	if obj != nil {
		kind := obj.GetObjectKind()
		// clearing the gvk is just a convention of a codec
		kind.SetGroupVersionKind(schema.GroupVersionKind{})
	}
	return obj, gvk, err
}

However, the code is using something which marked wrong in ref.go : https://github.com/kubernetes/kubernetes/blob/v1.6.3/staging/src/k8s.io/client-go/pkg/api/v1/ref.go#L55-#L62

	// if the object referenced is actually persisted, we can just get kind from meta
	// if we are building an object reference to something not yet persisted, we should fallback to scheme
	kind := gvk.Kind
	if len(kind) == 0 {
		// TODO: this is wrong
		gvks, _, err := scheme.ObjectKinds(obj)
		if err != nil {
			return nil, err
		}
		kind = gvks[0].Kind
	}

And after commenting out the reset GVK part(func (d DirectDecoder) Decode(data []byte, defaults *schema.GroupVersionKind, into runtime.Object) (runtime.Object, *schema.GroupVersionKind, error)), everything works as expected.

I have some type that looks like

type EmptyDiskSource struct {
	// Capacity of the sparse disk.
	Capacity resource.Quantity `json:"capacity"`
}

when I run openapi-gen on it, I have

"kubevirt.io/kubevirt/pkg/api/v1.EmptyDiskSource": {
			Schema: spec.Schema{
				SchemaProps: spec.SchemaProps{
					Description: "EmptyDisk represents a temporary disk which shares the vmis lifecycle.",
					Properties: map[string]spec.Schema{
						"capacity": {
							SchemaProps: spec.SchemaProps{
								Description: "Capacity of the sparse disk.",
								Ref:         ref("k8s.io/apimachinery/pkg/api/resource.Quantity"),
							},
						},
					},
					Required: []string{"capacity"},
				},
			},
			Dependencies: []string{
				"k8s.io/apimachinery/pkg/api/resource.Quantity"},
		},
...
"k8s.io/apimachinery/pkg/api/resource.Quantity": {
			Schema: spec.Schema{
				SchemaProps: spec.SchemaProps{
					Description: "Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and Int64() accessors.\n\nThe serialization format is:\n\n<quantity>        ::= <signedNumber><suffix>\n  (Note that <suffix> may be empty, from the \"\" case in <decimalSI>.)\n<digit>           ::= 0 | 1 | ... | 9 <digits>          ::= <digit> | <digit><digits> <number>          ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign>            ::= \"+\" | \"-\" <signedNumber>    ::= <number> | <sign><number> <suffix>          ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI>        ::= Ki | Mi | Gi | Ti | Pi | Ei\n  (International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)\n<decimalSI>       ::= m | \"\" | k | M | G | T | P | E\n  (Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)\n<decimalExponent> ::= \"e\" <signedNumber> | \"E\" <signedNumber>\n\nNo matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.\n\nWhen a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.\n\nBefore serializing, Quantity will be put in \"canonical form\". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:\n  a. No precision is lost\n  b. No fractional digits will be emitted\n  c. The exponent (or suffix) is as large as possible.\nThe sign will be omitted unless the number is negative.\n\nExamples:\n  1.5 will be serialized as \"1500m\"\n  1.5Gi will be serialized as \"1536Mi\"\n\nNOTE: We reserve the right to amend this canonical format, perhaps to\n  allow 1.5 to be canonical.\n  or after March 2015.\n\nNote that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.\n\nNon-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)\n\nThis format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation.",
					Properties: map[string]spec.Schema{
						"i": {
							SchemaProps: spec.SchemaProps{
								Description: "i is the quantity in int64 scaled form, if d.Dec == nil",
								Ref:         ref("k8s.io/apimachinery/pkg/api/resource.int64Amount"),
							},
						},
						"d": {
							SchemaProps: spec.SchemaProps{
								Description: "d is the quantity in inf.Dec form if d.Dec != nil",
								Ref:         ref("k8s.io/apimachinery/pkg/api/resource.infDecAmount"),
							},
						},
						"s": {
							SchemaProps: spec.SchemaProps{
								Description: "s is the generated value of this quantity to avoid recalculation",
								Type:        []string{"string"},
								Format:      "",
							},
						},
						"Format": {
							SchemaProps: spec.SchemaProps{
								Description: "Change Format at will. See the comment for Canonicalize for more details.",
								Type:        []string{"string"},
								Format:      "",
							},
						},
					},
					Required: []string{"i", "d", "s", "Format"},
				},
			},
			Dependencies: []string{
				"k8s.io/apimachinery/pkg/api/resource.infDecAmount", "k8s.io/apimachinery/pkg/api/resource.int64Amount"},
		},
...
"k8s.io/apimachinery/pkg/api/resource.int64Amount": {
			Schema: spec.Schema{
				SchemaProps: spec.SchemaProps{
					Description: "int64Amount represents a fixed precision numerator and arbitrary scale exponent. It is faster than operations on inf.Dec for values that can be represented as int64.",
					Properties: map[string]spec.Schema{
						"value": {
							SchemaProps: spec.SchemaProps{
								Type:   []string{"integer"},
								Format: "int64",
							},
						},
						"scale": {
							SchemaProps: spec.SchemaProps{
								Type:   []string{"integer"},
								Format: "int32",
							},
						},
					},
					Required: []string{"value", "scale"},
				},
			},
			Dependencies: []string{},
		},
...

but I do not have entry for k8s.io/apimachinery/pkg/api/resource.infDecAmount, I think it because that infDecAmount type includes type from go standard library and they can not be auto-generated by openapi-gen. I can set // +k8s:openapi-gen=false on problematic field

// d is the quantity in inf.Dec form if d.Dec != nil
// +k8s:openapi-gen=false
d infDecAmount

, but I do not sure if it good solution.

Assigning to @kubernetes/sig-api-machinery-misc leads:
/assign @deads2k @lavalamp

Please give @sttts @caesarxuchao write access to the following repos:

• k8s.io/code-generator
• k8s.io/sample-apiserver
• k8s.io/kube-aggregator
• k8s.io/apiserver
• k8s.io/client-go
• k8s.io/api master
• k8s.io/apimachinery
• k8s.io/metrics
• k8s.io/apiextensions-apiserver

This is basically a repetition of kubernetes/kubernetes#51643, the same situation 2 weeks ago.

Status Quo: The github publisher bot https://github.com/kubernetes/test-infra/tree/master/mungegithub/publisher is not pushing the repos right now because it cannot cope with a number of commits (merges) and branch structures (fast-forward master merges) and falls over. Fixes are being reviewed #4134, but it might take another 1-2 weeks until they are ready and maybe more days to build up the confidence again that we can let it push nightly automatically.

To unblock people with components depending on 1.8 another soon'ish push is necessary. For that the bot is run in a private GKE cluster now and pushes to private forks (https://github.com/sttts/client-go, https://github.com/sttts/apiserver, ...). The actual manual push is a sequence of fast-forward pushes. These are codified in

https://gist.github.com/anonymous/2fd6e66e9364dd87e66b12cdb9bd92dc for the pushes, all fast-forward (no force-pushes!)
https://gist.github.com/anonymous/62e661ea6dde3cb7615023f2e7fd69fb for the latest repo heads for a possible roll-back.
Only somebody with push permissions to the repos can execute them. Neither I nor @caesarxuchao have these permisssions. PRs against the repos do not work either because they linearize the branches and destroy the structure we need to consistently create tags.

What we need:

somebody to give us (@sttts and @caesarxuchao) push permissions (or some special user for that we have access to) for those repos to do this push and maybe some more until the bot is running again.
As the situation can happen easily again that the bot falls over (or even pushes wrong content such that we have to react reasonably fast), I would feel much better with (2). Last time it took 3.5 days until we were able to do (1) after asking a number of people for help, not a good experience.

/cc @kubernetes/sig-api-machinery-misc @fejta @lavalamp @thockin @wojtek-t @smarterclayton @eparis

Further context: kubernetes/kubernetes#51643 kubernetes/test-infra#4499

The following commit in kubernetes is now blocking our ability to move up to fixes we (kubernetes-incubator/cri-containerd) need in kubernetes:

kubernetes/kubernetes@1b3836d

Specifically the above commit moved "List" from "k8s.io/api/types.go" to the following location in their staging version of apimachinery "k8s.io/apimachinery/pkg/apis/meta/internalversion/types.go"

@Random-Liu

Release 1.7.15 does not pass unit tests with Go 1.10. At least:

+ GOPATH=/builddir/build/BUILD/apimachinery-kubernetes-1.7.15/_build:/usr/share/gocode
+ go test -buildmode pie -compiler gc -ldflags '-extldflags '\''-Wl,-z,relro  '\'''
# k8s.io/apimachinery/pkg/fields
./selector_test.go:390: Errorf format %t arg result.Empty is a func value, not called
FAIL	k8s.io/apimachinery/pkg/fields [build failed]

Hi, In using the meta info API I had needed to add my own labels. I had to do a get, add labels to what got retrieved and then use the SetLabels() api. Any reason why we do not have a "AddLabels()" api with the semantics of adding onto the labels map ? I am assuming this must have come up before but could not locate an applicable thread. Thanks.

I would like to have the ability to sort objects server side by a particular field on the object being returned when issuing a get request that returns a list of objects.

I am wondering about the impact of potentially adding a Sort field to ListOptions so that a sort field can be specified as part of a get request.

An example of where I would like to use such a field can be found as part of the Tekton project. The idea here would be to sort pipelineruns by their start times and then returned to the requestor.

If there is already an option available, I would like to know where I can find an example.

While refactoring a unit-test (moved the pakage) I suddenly get the error :

	// decode config, empty if no bytes
	config, err := decode(data)
	if err != nil {
		return nil, errors.Errorf("could not read config: %v", err)
	}

no kind "Config" is registered for version "v1" in scheme, nothing in terms of dependencies changed ! I was wondering if you know what is the issue ?

    context_test.go:129: Error not expected but got could not read config: Error decoding config from data: apiVersion: v1
        clusters: []
        contexts: []
        current-context: "minikube"
        kind: Config
        preferences: {}
        users: []
        : no kind "Config" is registered for version "v1" in scheme "k8s.io/apimachinery@v0.0.0-20190801232353-f6278300bebbb750328ac16ee6dd3aa7d3549568/pkg/runtime/scheme.go:101"

e.g. This label points to a dependency that is not vendored.

Trying to print a provider config (all of it using glog.Infof("%+v", foo)) will not work. It will print an empty blob. However printing actual fields in it work just fine.

I0530 18:07:42.608732    3635 machineactuator.go:158] clusterConfig = &TypeMeta{Kind:,APIVersion:,}
I0530 18:07:42.608737    3635 machineactuator.go:159] clusterConfig.VsphereUser = [email protected]

I don't have a smaller example (we run into this with sigs.k8s.io/cluster-api) but if needed, I can make one.

when resource (or limit) is set to kibibyte, mebibyte or gibibyte as decimal number, it is not parsed correctly. For example setting memory to 2.1Gi will be converted to 2254857830400m. What is happening is that the number is padded to 10 places 2100000000 and then converted.

Hi,

We're using the watch.Event type in our internal code base extensively. Some of the places encode it as well. After upgrading to latest client-go it stopped compiling because of the missing method DeepCopyObject. Seems like the following line is missing (https://github.com/kubernetes/apimachinery/blob/master/pkg/watch/watch.go#L54):

// k8s:deepcopy-gen:interfaces=k8s.io/kubernetes/runtime.Object

Can we generate this method for this type as well? I can open a PR but wasn't sure if that was intentionally like this.

../../vendor/k8s.io/client-go/discovery/restmapper.go:42:75: undefined: meta.VersionInterfacesFunc
../../vendor/k8s.io/client-go/discovery/restmapper.go:176:19: undefined: meta.VersionInterfacesFunc

client-go version is 7.0.0
apimachinery version kubernetes-1.9.4 works fine, but the latest broke

Hi,

Is there any plan to support i18n in error message?

KubeVirt is aiming to become an add-on for running virtual machines on Kube,
This includes that KubeVirt is getting installed into the kube-system (or some other predefined namespace) and will handle VM objects in other namespaces.

This is at least how add-ons could be imagined to work.

But - As a good citizen KubeVirt's controllers should add OwnerReferences to the objects it creates and manages, and here is the issue: The current OwnerReference API does work with the assumption that owner and owned object live in different namespaces.

What is the expectation here? Should KubeVirt behave different or should the API be extended?

Releases 1.9.5 1.8.10 and 1.7.15 do not pass unit tests with Go 1.10. At least:

+ GOPATH=/builddir/build/BUILD/apimachinery-kubernetes-1.9.5/_build:/usr/share/gocode
+ go test -buildmode pie -compiler gc -ldflags '-extldflags '\''-Wl,-z,relro  '\'''
# k8s.io/apimachinery/pkg/util/httpstream/spdy
./roundtripper_test.go:402: direct assignment to atomic value
FAIL	k8s.io/apimachinery/pkg/util/httpstream/spdy [build failed]

I discovered a weird behaviour when working with the Quantity type. It's String() implementation tries to be smart by caching the calculated string representation into the private property s. String() is actually meant to be a getter, read only if you like. At least this is my intuition on it. Modifying internal state on such a getter is considered an anti-pattern IMO. I discovered this when unit tests failed because you either call String() or not and the result of the tests change in case you use reflect.DeepEqual as comparator. I would like to know what you think. IMO if something should be cached, and I do not really see the need for that from my naive point of view, then it should happen on instance construction (as soon as all necessary information are present), and not during runtime (especially not to change internal state auto magically).

See

The proto for metadata.creationTimeStamp

In k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto creationTimestamp is defined as

// CreationTimestamp is a timestamp representing the server time when this object was
  // created. It is not guaranteed to be set in happens-before order across separate operations.
  // Clients may not set this value. It is represented in RFC3339 form and is in UTC.
  // 
  // Populated by the system.
  // Read-only.
  // Null for lists.
  // More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata
  // +optional
  optional Time creationTimestamp = 8;

where Time is

// Time is a wrapper around time.Time which supports correct
// marshaling to YAML and JSON.  Wrappers are provided for many
// of the factory methods that the time package offers.
// 
// +protobuf.options.marshal=false
// +protobuf.as=Timestamp
// +protobuf.options.(gogoproto.goproto_stringer)=false
message Time {
  // Represents seconds of UTC time since Unix epoch
  // 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
  // 9999-12-31T23:59:59Z inclusive.
  optional int64 seconds = 1;

  // Non-negative fractions of a second at nanosecond resolution. Negative
  // second values with fractions must still have non-negative nanos values
  // that count forward in time. Must be from 0 to 999,999,999
  // inclusive. This field may be limited in precision depending on context.
  optional int32 nanos = 2;
}

If receiving a message from the kube API with metadata the creationTimestamp will be serialized as a string such as : "2017-10-31T08:17:56Z"
If you try to use in Java the JsonFormat proto buffer class using the k8s protos to parse such as resource response you will get an error like:

com.google.protobuf.InvalidProtocolBufferException: Expect message object but got: "2017-10-31T08:17:56Z" as JsonFormat can't handle the time string in a custom manner.

In the Java protobuffers library JSONFormat has special handlers for normal timestamps, e.g.

 // Special-case Timestamp.
      parsers.put(
          Timestamp.getDescriptor().getFullName(),
          new WellKnownTypeParser() {
            @Override
            public void merge(ParserImpl parser, JsonElement json, Message.Builder builder)
                throws InvalidProtocolBufferException {
              parser.mergeTimestamp(json, builder);
            }
          });

But the parsers are not available to be extended:

 private static final Map<String, WellKnownTypeParser> wellKnownTypeParsers =
        buildWellKnownTypeParsers();

deletionTimestamp has the same issue.

Hi,

In the following function:

There's no retry to connect back after a failure. Not sure if should be handled here though. However my proposal is to use an exponential backoff retry (https://godoc.org/github.com/cenkalti/backoff).

If it should be handled here, please let me know, and I will be happy to supply a PR.

Regards,
Christian