tektoncd / results Goto Github PK
View Code? Open in Web Editor NEWLong term storage of execution results.
License: Apache License 2.0
Long term storage of execution results.
License: Apache License 2.0
Tracking issue for porting Update Record (formerly known as Executions) behavior from v1alpha1 to v1alpha2.
Pipelineruns fails for various reasons e.g (git_related, build_related). I know results give top level success/failure status. I want to be able to further classify failures to provide more context.
This would help easily identify & surface underlying root causes of taskrun failures.
git_related
failures in the last week.Tracking issue for porting Get Record (formerly known as Executions) behavior from v1alpha1 to v1alpha2.
Watcher should be able to patch the annotations for all TaskRun/PipelineRun objects.
Reconcile fails with an opaque error:
{
"level": "error",
"ts": "2021-02-02T15:43:36.046Z",
"logger": "watcher.TaskRunResultsWatcher",
"caller": "taskrun/taskrun.go:84",
"msg": "TaskRun.Patch: the server rejected our request due to an error in our request",
"knative.dev/traceid": "9ff62d71-d1a7-441e-a0ce-5248a1095ae3",
"knative.dev/key": "mario/mario-comment-github-pdh9h-vqbj5",
"stacktrace": "github.com/tektoncd/results/pkg/watcher/reconciler/taskrun.(*Reconciler).Reconcile\n\tgithub.com/tektoncd/results/pkg/watcher/reconciler/taskrun/taskrun.go:84\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tknative.dev/[email protected]/controller/controller.go:511\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\tknative.dev/[email protected]/controller/controller.go:449"
}
This only seems to be affecting mario tasks in dogfooding.
For TaskRuns that are created as a result of a PipelineRun, we should store them as Records within the same Result
This aggregates related Records within the same result, which we want for querying.
tkn results --help
describes all required args and available flags
The required arg for all tkn results
commands is not specified:
$ tkn results list -l 10
Error: accepts 1 arg(s), received 0
Usage:
tkn-results list [flags]
Flags:
-f, --filter string CEL Filter
-h, --help help for list
-l, --limit int32 number of items to return. Response may be truncated due to server limits.
-o, --output string output format. Valid values: textproto|json (default "textproto")
-p, --page string pagination token to use for next page
Global Flags:
-a, --addr string Result API server address (default "localhost:50051")
-t, --authtoken string authorization bearer token to use for authenticated requests
Kubernetes version:
Output of kubectl version
:
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T21:15:16Z", GoVersion:"go1.16.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.17-gke.4900", GitCommit:"2812f9fb0003709fc44fc34166701b377020f1c9", GitTreeState:"clean", BuildDate:"2021-03-19T09:19:27Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"} ```
$ tkn version
Client version: 0.18.0
Pipeline version: v0.23.0
Triggers version: v0.13.0
Dashboard version: v0.16.1
Tracking issue for adding ListRecords to the v1alpha2 API.
At this moment api and watcher images are running only on linux/amd64
platform. However it is already included into Tekton operator, which has multi-arch support. Is it possible to publish multi-arch versions of api and watch images used for Tekton operator with linux/amd64
, linux/s390x
, linux/ppc64le
and linux/arm64
support?
In case if ko
tool was used to build initial version, generate the multi-arch image is possible using --platform=linux/amd64,linux/s390x,linux/ppc64le,linux/arm64
extra parameter.
Also for the future is it possible to use multi-arch version of mysql
image for the Tekton Result, for instance https://hub.docker.com/r/ubuntu/mysql
When deploying to a real kubernetes cluster, the MySQL deployment fails with the following error:
Warning Failed 10s (x4 over 49s) kubelet, gke-dogfooding-default-pool-f62aa79c-cipp Error: failed to start container "mysql": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt: read-only file system
https://discuss.kubernetes.io/t/running-tutorial-mysql-example-gives-me-read-only-file-system-error/3197 appears to describe what we're running into. We need to tweak how we're provisioning the persistent disk so that it's not read-only.
I should be able to use any TaskRun/PipelineRun name as a name part for a result / record identifier.
The Result API only accepts lower case identifiers.
rpc error: code = InvalidArgument desc = rpc error: code = InvalidArgument desc = name must match (^[a-z0-9_-]{1,63})/results/([a-z0-9_-]{1,63}$)
We should loosen the regex to allow upper case letters.
Tracking issue for porting v1alpha1 UpdateResults to v1alpha2.
We have a lot of duplication in the watcher code, mostly because we don't have a generic type to use for TaskRuns and PipelineRuns.
Dynamic clients might be a good compromise here to use Unstructured types to support anything that is Object-like.
After a TaskRun/PipelineRun is completed and successfully uploaded, we should delete the etcd based data to free up resources on the user's cluster.
Requirements:
Task/PipelineRun clean up has been a much requested feature, but is at odds with results storage since once the CRD object is gone from etcd, the results are gone with it.
Making this a feature of the Results watcher means that we can ensure that this clean up happens after results have been stored durably.
See:
Tracking issue for integration with Tekton Triggers.
Our goal here is to start recording input events (e.g. Push/Pull Request payload) as part of results, and group this in the same Result as the Run executions.
Results watcher is already aware of trigger labels for result grouping, so the main thing here is having a hook point in trigger processing to upload the data to the Results API.
An initial idea for a non-invasive proof of concept is to use an interceptor that uploads the event just before the binding/template is processed. Eventually we'll likely need a deeper integration so that we can also capture events that did not result in executions.
Tracking issue for implementing TEP-0088 - Result Summaries
Tracking bug for enabling real e2e tests (already created in ./test/e2e).
Can safely upgrade between release versions.
Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden
This is being cause by kustomize being too aggressive with the commonLabels field - this is adding the version label to selectors and other fields that are intended to be immutable.
Hi there,
I found that the tekton result database only records pipelineruns that triggered from event listener, however if I re-run the pipeline directly from the tekton dashboard, then nothing changes in the database. No new records will be inserted, nor the update_time of the previous record will be updated.
Is this by design?
Thanks!
Watcher successfully comes up in dogfooding
$ k logs tekton-results-watcher-5b7c89678b-48b5w -n tekton-pipelines -f
2021/09/01 15:03:48 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
2021/09/01 15:03:48 dialing tekton-results-api-service.tekton-pipelines.svc.cluster.local:50051...
2021/09/01 15:04:18 did not connect: context deadline exceeded
I suspect this is an SSL problem - we are connecting to the local service address, but the SSL cert is for results.dogfooding.tekton.dev
.
Kubernetes version:
Output of kubectl version
:
Client Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.18-dispatcher", GitCommit:"de944d802c49735ad0f4bfe82ddfa19737ebe962", GitTreeState:"clean", BuildDate:"2021-05-13T17:51:55Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.20-gke.501", GitCommit:"de876a981b936ce528482c45794fb1dfc5005042", GitTreeState:"clean", BuildDate:"2021-06-25T02:54:31Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}
Tekton Results version:
Output of tkn version
or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'
v0.3.0
A common use case will be to order results by creation time (e.g. show me the most recent).
We should look into adding an ordering field to List methods to support this - https://google.aip.dev/132#ordering
Tracking issue for porting Delete Record (formerly known as Executions) behavior from v1alpha1 to v1alpha2.
In the schema, the bodies of Tekton's resources are stored as blob
types. Blobs are not suitable for indexing and do not allow easy querying by analysts or tools accessing the database instead of the gRPC service.
Switching to jsonb
would allow for more efficient storage, make it possible to index records and enable rich querying via JSON operators.
When I use kubectl port-forward -n tekton-pipelines service/tekton-results-api-service 50051:50051
and then run the simple ui, I should be able to see the results page
The go run .
command hangs and the simple ui page could not be opened.
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:48:33Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:03:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.22) and server (1.20) exceeds the supported minor version skew of +/-1
v0.29.0
The reason I've figured out is that grpc.withInsecure() will not work if there is tls configured in server side, however even if I add the cert file with grpc.WithTransportCredentials
, still I'll get error says that "Unable to find token", which comes from rbac.go.
I've little experience in Go so I guess I'm not able to fix it...
Tracking bug for migrating the watcher from the v1alpha1 Results API to v1alpha2.
"spec": {
"params": [
{
"name": "gitrevision",
"value": "9de276f1bb9078c967358ec7aa6fb1a8e8a56b29"
},
{
"name": "gitrepositoryurl",
"value": "[email protected]:tektoncd/results.git"
},
{
"name": "homepage",
"value": "https://github.com/tektoncd/results.git"
}
],
"pipelineRef": {
"name": "pipeline-XXX"
},
"serviceAccountName": "robot",
"timeout": "1h0m0s",
"workspaces": [
{
"name": "sonar-vault",
"secret": {
"secretName": "sonar-auth"
}
}
]
}
None
Kubernetes version:
Output of kubectl version
:
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:10:43Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.12", GitCommit:"7cd5e9086de8ae25d6a1514d0c87bac67ca4a481", GitTreeState:"clean", BuildDate:"2020-11-12T09:11:15Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
- Tekton Pipeline version:
**Output of `tkn version` or `kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'`**
Tekton version : v0.21.0
Once a result is deleted, its child records should also be deleted.
I managed to query our results deployment using tkn-results 🥳
Requesting json output however does fail for records list
./tkn-results records list $ID --output json --authtoken $TOKEN
should return json result
Error: unknown flag: --output
I´m using a57a9cc
Tracking bug for implementing TEP-0072 - JSON Serialized Results
results.tekton.dev/release
and version
labels match release version
Labels are stuck on devel
.
Need to update the release script (https://github.com/tektoncd/results/blob/main/release/release.sh) to update the version labels.
We should add support for reading records across namespaces, so that we can query for records across the database.
e.g. GET /default/results/-/records?filter=...
See https://google.aip.dev/159 for more details.
This allows users to make queries for a particular type of record. e.g. list all taskruns with a particular property. Currently, users must call ListRecords on each result individually and aggregate themselves.
DeleteResult and DeleteRecord RPCs are failing because gRPC wants us to return an Empty proto, not nil.
Primarily creating this for visibility of a known issue - this is fixed in #60
We should add gRPC health checking to the API server for interop with other gRPC tooling.
This would let us take advantage of other gRPC health check tooling (https://github.com/grpc-ecosystem/grpc-health-probe, grpc_cli, etc.)
Ability to use Requests with non-Postgres DB
I see in v0.3.0, the DB was moved from mysql to postgres and the reason -- from #121 -- is because of postgres support of Json types. Mysql does support json types.
Are there plans for folks who can't use postgres (due to say company policy) to be able to use Results?
Lines 52 to 55 in 8af4676
I was trying to read up docs to figure out what the name of the DB and the URL of the DB is expected. I see it hardcoded in the installation yaml above. Is it configurable, if not, should we make it configurable ? :)
status:
completionTime: "2022-01-27T07:07:48Z"
conditions:
- lastTransitionTime: "2022-01-27T07:07:48Z"
message: All Steps have completed executing
reason: Succeeded
status: "True"
type: Succeeded
podName: pipelinerun-with-params3-sum-params-cgrj4-pod-pnbkq
startTime: "2022-01-27T07:05:37Z"
records {
name: "tekton-pipelines/results/3137c0a0-e3e6-40c1-99f1-84e5c68d594d/records/18b63cf4-011d-4a97-bec4-2e528dd5ab74"
id: "2da6cf12-8466-4752-866e-de07de64e220"
data {
type: "tekton.dev/v1beta1.TaskRun"
value: "{\"kind\": \"TaskRun\", \"spec\": {\"params\": [{\"name\": \"a\", \"value\": \"100\"}, {\"name\": \"b\", \"value\": \"500\"}], \"taskRef\": {\"kind\": \"Task\", \"name\": \"sum-params\"}, \"timeout\": \"1h0m0s\", \"resources\": {}, \"serviceAccountName\": \"default\"}, \"status\": {\"steps\": [{\"name\": \"sum\", \"waiting\": {\"reason\": \"PodInitializing\"}, \"container\": \"step-sum\"}], \"podName\": \"pipelinerun-with-params3-sum-params-cgrj4-pod-pnbkq\", \"taskSpec\": {\"steps\": [{\"name\": \"sum\", \"image\": \"bash:latest\", \"script\": \"#!/usr/bin/env bash\\nsleep 120\\necho -n $(( \\\"$(inputs.params.a)\\\" + \\\"$(inputs.params.b)\\\" ))\\n\", \"resources\": {}}], \"params\": [{\"name\": \"a\", \"type\": \"string\", \"default\": \"1\", \"description\": \"The first integer\"}, {\"name\": \"b\", \"type\": \"string\", \"default\": \"1\", \"description\": \"The second integer\"}]}, \"startTime\": \"2022-01-27T07:05:37Z\", \"conditions\": [{\"type\": \"Succeeded\", \"reason\": \"Pending\", \"status\": \"Unknown\", \"message\": \"pod status \\\"Initialized\\\":\\\"False\\\"; message: \\\"containers with incomplete status: [place-tools place-scripts]\\\"\", \"lastTransitionTime\": \"2022-01-27T07:05:38Z\"}]}
...
Kubernetes version: v1.21.1
Tekton Pipeline version: v0.26.0
I noticed in pkg/watcher/results/results.go:167, reconciler only updates Record when Record's generation is not equal to object's generation, since the change of pipelineRun/taskRun status won't change generation number, watch can not update the Record even when the status is changed.
Tracking issue for integrating with tkn, the Tekton CLI.
We'd like to add support for querying the Results API via tkn. This can likely be done in a non-invasive way via tkn plugins.
Interesting things to figure out here include:
Tracking issue for porting Create Record (formerly known as Executions) behavior from v1alpha1 to v1alpha2.
Tracking bug for setting up fully automated releases for nightly and tagged releases.
It should be possible to specify parameters (for example filters) when querying via tkn-results.
❯ tkn results records list --help
List Records
Usage:
tkn-results records list [flags] <result parent>
<result parent>: Result parent name to query. This is typically "<namespace>/results/<result name>", but may vary depending on the API Server. "-" may be used as <result name> to query all Results for a given parent.
Flags:
-f, --filter string CEL Filter
-h, --help help for list
-l, --limit int32 number of items to return. Response may be truncated due to server limits.
-o, --output string output format. Valid values: tab|textproto|json (default "tab")
-p, --page string pagination token to use for next page
Global Flags:
-a, --addr string Result API server address
-t, --authtoken string authorization bearer token to use for authenticated requests
❯ tkn results records list -f "some-filter" "default/results/-"
unknown shorthand flag: 'f' in -f
Usage of /home/goldmann/.config/tkn/plugins/tkn-results:
-a, --addr string Result API server address
-t, --authtoken string authorization bearer token to use for authenticated requests
unknown shorthand flag: 'f' in -f
In the first run, it correctly printed correct flags, but when trying to set these -- it fails with a message that suggests it is considering only parent flags. In other words - subcomand flags are being ignored.
Kubernetes version:
Output of kubectl version
:
❯ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:34:54Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}
❯ tkn version
Client version: 0.21.0
Pipeline version: v0.32.1
Tracking issue for integrating docs with tekton.dev site.
This issue tracks adding authn/z to the Results API.
Admins should be able to control fine grain ACLs for reading or writing data to the Result API.
Allow the Results API to be exposed outside of the cluster so that off cluster clients may access data.
Tracking issue for integrating with the Tekton Dashboard.
We'd like to surface Results/Records in the Tekton UI so that users can visualize the logic groupings / extra data Results can store. A really rough version of this can be found here: https://github.com/tektoncd/results/tree/main/tools/simpleui
I am not sure how we can do this in a minimally invasive way. Open to ideas!
https://storage.googleapis.com/tekton-releases/results/latest/release.yaml
should return the latest release version
at present it is v0.3.1
returns an older version v0.1.1
Kubernetes version:
Output of kubectl version
:
(paste your output here)
Tekton Pipeline version:
Output of tkn version
or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'
Tracking issue for adding RPC metrics to the Result API.
https://github.com/grpc-ecosystem/go-grpc-prometheus looks like a promising solution for how to implement this.
Namespaced scoped SA should be able to auth records.
Permission denied. :(
Things work if the user is granted cluster-wide access - this smells like we're not parsing out the namespace correctly so we're making the wrong request.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: result-ro
subjects:
- kind: ServiceAccount
name: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: tekton-results-readonly
$ grpc_cli call --channel_creds_type=ssl --ssl_target=tekton-results-api-service.tekton-pipelines.svc.cluster.local --call_creds=access_token=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='result-reader')].data.token}"|base64 --decode) localhost:50051 tekton.results.v1alpha2.Results.ListRecords 'parent: "default/results/4ca70fb5-9cfd-4c0b-bdad-c683c02c7b59"'
Rpc failed with status code 16, error message: permission denied
$ k get clusterrole tekton-results-readonly -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: tekton-results-readonly
rules:
- apiGroups:
- results.tekton.dev
resources:
- results
- records
verbs:
- get
- list
Kubernetes version:
Output of kubectl version
:
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-21T23:01:33Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
Tekton Results version:
We've gotten feedback to make some small adjustments to API field names for clarity / bring us more inline with AIP docs.
Current name | Suggested name |
---|---|
id | uid |
created_time | create_time |
updated_time | update_time |
These are renames only, and have no impact on the meaning / usage of the fields.
Goal is to support both fields for 1 minor release, then remove the old fields in the following release.
We should be able to repurpose TaskRun names / IDs as Result name parts.
Certain names do not match the name regex because they are too large.
Example: tektonci/results/taskrun-check-pr-labels-rzjhs-check-kind-label-g6qxc-check-name-m-vxrhd
Full log:
{"level":"error","ts":"2021-01-13T10:44:08.825-0500","logger":"watcher.TaskRunResultsWatcher","caller":"taskrun/taskrun.go:62","msg":"error updating Record: rpc error: code = InvalidArgument desc = GetResult(tektonci/results/taskrun-check-pr-labels-rzjhs-check-kind-label-g6qxc-check-name-m-vxrhd): rpc error: code = InvalidArgument desc = rpc error: code = InvalidArgument desc = name must match (^[a-z0-9_-]{1,63})/results/([a-z0-9_-]{1,63}$)","knative.dev/traceid":"987d645f-0b1c-4dc3-9f1d-3c932334b4c2","knative.dev/key":"tektonci/check-pr-labels-rzjhs-check-kind-label-g6qxc-check-name-m-vxrhd","stacktrace":"github.com/tektoncd/results/pkg/watcher/reconciler/taskrun.(*Reconciler).Reconcile\n\t/usr/local/google/home/wlynch/src/github.com/tektoncd/results/pkg/watcher/reconciler/taskrun/taskrun.go:62\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\t/usr/local/google/home/wlynch/pkg/mod/knative.dev/[email protected]/controller/controller.go:511\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\t/usr/local/google/home/wlynch/pkg/mod/knative.dev/[email protected]/controller/controller.go:449"}
We need to set up release infrastructure to publish API Server and Watcher images to the Tekton repository.
We can likely use Pipelines/Triggers/Plumbing as a model to follow.
The db package used by the API server binary currently depends on github.com/mattn/go-sqlite3 which is a CGO wrapper around the sqlite library. CGO doesn't play nicely with ko
, and while there are probably workarounds we could employ to make ko
play more nicely, we should avoid having this package and binary be a snowflake.
https://github.com/go-sql-driver/mysql is a pure Go driver for MySQL which we can use instead, which would allow the API server to be built using ko
without workarounds.
See https://github.com/tektoncd/experimental/pull/539/files#diff-bb9ba4439e6c49da19cb1dd7b496d725e33fc36a9b1f472c7cbaec015a6ff38fR28 for a (reverted) example of this.
cc @afrittoli
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.