gridscale / gsclient-go Goto Github PK
View Code? Open in Web Editor NEWThe official gridscale API client written in Go
Home Page: https://gridscale.io
License: MIT License
The official gridscale API client written in Go
Home Page: https://gridscale.io
License: MIT License
We want to switch the default branch name from master to main for all gridscale Open Source projects including this repository.
This involves:
Would be nice to do this coordinated and in "one go" so that contributors are not surprised by inconsistent naming or repositories not having switched yet. If you want to help out here contact @bkircher
When printing debug output with gsclient-go, it prints out all headers including API token and user ID. Alas, this is not incorrect it is also not necessary.
Proposal: Obfuscate the output, so that only the first 6 characters are printed and the rest is omitted (indicated by ellipses or something).
This would allow users to just copy and paste their debug output into pastebin or whatever without accidentally publishing credentials. At the same time, it allows users to verify that the credentials are the expected ones when comparing.
Example: create a new client with debugMode set to true. Then do a simple API call. It will print something like
DEBU[2021-02-15T14:48:01+01:00] Request headers: map[Content-Type:[application/json] User-Agent:[gsclient-go/3.3.2 (darwin)] X-Auth-Token:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] X-Auth-Userid:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]]
where the Xs are real credentials.
This relate to terraform-provider-gridscale issue #149. The issue prevents the PaaS from upgrading its template UUID via gsclient-go. This is due to the incorrect information from the docs of PaaS Update operation.
There is no support in the API client yet that supports workflows with multiple projects in gridscale. I suppose, new objects are always added to the first project (named 'default').
Also, current gridscale API Specification (1.0.3) doesn't expose projects yet. So this would be needed first.
Ditch git flow, use GitHub regular work flow. Will make releases much easier and we can automate with tools like GoReleaser.
TODO:
origin/release/*
)GET /objects/servers//events as well as GET /objects/events and all other events end-points now include an additional key "initiator" whose value is a string.
From the docs, initiator is "The user that triggered the event. Usually the user's email if the event was triggered by request of a user, otherwise a short descriptive name of the system component responsible."
Docs here: https://gridscale.io/en/api-documentation/index.html#operation/getServerEvents
Please add this to EventProperties
struct.
To improve stability of our services, we need to identify issues and bottlenecks first.
In the past, when encountering issues in API reliability that were temporary in nature, we have implemented retry mechanisms into gsclient-go and the gridscale-terraform-plugin where applicable to improve the stability of the overall process. Also, underlying issues have been identified and, where possible, solved accordingly.
These retry mechanisms have the side-effect that, once in place, they mask underyling issues. To improve quality for everyone, we need to make them visible again.
This can be tackled from either the client or the server-side. For the latter, the APIs would provide metrics of request success. This does not make all issues visible though. In front of the actual APIs, there is an application server. In front of that, a web server. In front of that a reverse proxy. In front of that a firewall. In front of that a router. Etc. Issues in these layers, or the rolling deployment of the APIs itself, would not be visible in API-level metrics. Instead, we want our SDKs to report errors to us - with an opt-out functionality.
gsclient-go
has been selected as the first prototype - mainly, because it is used by terraform as well. Terraform, in return, makes use of higher concurrency than many other clients do and tends to trigger deficiencies in provisioning more efficiently.
The idea is to send these reports to a publically available sentry instance. Networking errors (connection reset f.e.) and provisioning errors (Storage creation error f.e.) alike. To investigate issues, I presume we need
X-Request-Id
, if available/objects/servers
) for aggregationGET
, POST
, PATCH
, DELETE
, etc) for aggregationThese reports shall also be sent when the requests is retried. They shall not block the SDK and, as such, shall send reports asynchronously. Users must be able to opt-out.
What are your thoughts?
Ref API docs. Network's pinned_server
API endpoints need to be added, so that we could set the static IP address to servers within a network.
Sshkey
to SSHKey
. Example: SshkeyCreateRequest
should be SSHKeyCreateRequest
gsclient-go doesn't have the field autoscaling
in the response of PaaS template getter. More info: https://gridscale.io/en/api-documentation/index.html#operation/getPaasServiceTemplates
Sometime resources are locked in the backend, and we have no control in the client-side. Some PATCH/DELETE requests cannot be applied to those resources, that causes 409. We should retry the requests if we get this error code.
Cool thing is that gsclient-go documentation coverage is quiet good. But there are some corners that could need some polishing.
Go through package and docs for public types and functions in gsclient-go and follow suggestions in this article. Also make sure every package has a dedicated doc string.
Don't use embedded struct for GSTime
. Right now it is
type GSTime struct {
time.Time
}
Make it
type GSTime time.Time
so that we can use type conversion.
Something that would be nice to have in gsclient-go is a way to easily trace how long an API call needs from request → to fulfillment.
With fulfillment I mean: I'm not interested in timings for each individual API call (e.g. HTTP request → HTTP response) but rather in "create storage" → "storage created"; how long did this take?
I thought about specifying a log-level in the client configuration, e.g., TRACE
(or whatever logrus is using). This log-level should only be used for tracing the individual requests and their timings. One line per API request, printed when the request has finished. Output format should be JSON so that it can be processed further easily.
Example: If log level is set to TRACE, you would see something like this on stderr:
{
"message": "Finished request CreateStorage",
"requestId": "d8260ce6-7439-4543-85c4-794592a34346",
"request": "CreateStorage",
"success": true,
"timeMs": 3042
}
timeMs is the time in milliseconds that has elapsed. success would be false when this errors, but I would still see timeMs.
What do you guys think? @fkr @nvthongswansea
👋 An IP object can be related to a storage? https://github.com/gridscale/gsclient-go/blob/master/ip.go#L113 PublicIPs
and Storages
seem to be always nil.
Optional field with json tag "omitempty" should be type of pointer, since the optional fields usually accepts 3 types of input: nil (omitted when the field is nil), empty (Should not be omitted. eg: "" for string, 0 for int), and not empty. E.g:
Labels []string json:"labels,omitempty"
should be
Labels *[]string json:"labels,omitempty"
Hey hey. README.md says in the Known Issues section:
* L3security isn't read in the network relation of a server.
Is this still relevant? Should we create a bug for this and actually tackle this? @nvthongswansea
Start building support for a meaningful subset of contract management API (https://api.gridscale.io/management/v1/).
Interesting sub-set would be to get out current billing information for a single contract.
A heads up. There will be some updates to the GET /objects/paas/service_templates
response.
A new release
parameter will be included, and the version
parameter will be modified from version="10.3"
to version="10.3-gs1"
.
For a postgres template example, it will look like:
"9008f1f1-190e-4887-99f7-8a3e72fdd1f8": {
"object_uuid": "9008f1f1-190e-4887-99f7-8a3e72fdd1f8",
"release": "10.3",
"version": "10.3-gs1",
"name": "PostgreSQL 10.3 16GB (Gen 1)",
Current docs here: https://gridscale.io/en/api-documentation/index.html#operation/getPaasServiceTemplates
So I think we want to include that?
When creating servers and ServerCreateRequest.ServerHardwareProfile
is omitted the resulting profile is pc-i440fx-2.4 which is a weird choice for a default.
When doing the same thing in the panel, I get pc-q35-2.10 ("q35"
in the API docs) which is a much saner choice. Please consider changing default profile and set it explicitly to "q35". Also update the docs for ServerCreateRequest.ServerHardwareProfile
, "HardwareProfile is not set => server hardware is normal type"
is really not saying anything at all.
Due to wrong information on official API docs https://gridscale.io/en//api-documentation/index.html#operation/rollbackStorageBackup, the method RollbackStorageBackup
won't work. The correct API endpoint is https://api.gridscale.io/objects/storages/{storage_uuid}/backups/{backup_uuid}/rollback
A rate-limited request should be retried until the rate-limit is reset and the request is processed successfully. If the request is delayed again (due to rate-limit is reached again), keep retrying.
Add DHPC options to private network's create/update requests, and add DHCP properties to network's properties.
This happens due to a wrong field's name (which is described in gridscale's API docs). storage_backups
is supposed to be storages_backups
.
We are currently in the process of developing a Prometheus exporter to make our Gridscale cloud costs transparent. In the context of this exporter, among others, we expose the running PaaS resources as metrics by periodically scraping the Gridscale API. To be able to determine the cost of a resource, as far as I am aware, we need the products it's using via the product number and in which location it is running. Usages can be retrieved via the usage API, but, in contrast to other resource types, such as servers, PaaS resources do not seem to expose a location attribute via the API.
Is it viable to indirectly determine the location of the PaaS resource via the connected network, or is there another mechanism to retrieve this information from the API?
CurrentPrice
(current_price
in the API) is deprecated in the public API. We should remove this field from all property structs when we bump the major version.
Searching for a way to conveniently overwrite existing or add new HTTP headers to requests.
Right now I can create a new config.Config
with config.NewConfiguration
or config.DefaultConfiguration
but I'm only allowed to specify URL, user id, and token. The returned config will have config.Config.userAgent
set for me automatically which is just one HTTP header.
Is there a convenient way for me to specify any custom header that is made during requests?
Hi, I was debugging an Issue with my code that turned out, I've missed the https protocol in the endpoint.
This
func main() {
token := "XXXXXXXXXXXXXXXXXXXXXX"
user := "ZZZZZZZZZZZZZZZZZZZZZZZZ"
endpoint := "http://api.gridscale.io"
emptyCtx := context.Background()
// config := gsclient.DefaultConfiguration(user, token)
config := gsclient.NewConfiguration(endpoint, user, token, true, true, 1000, 5)
client := gsclient.NewClient(config)
// err := apiAlive(token, user, endpoint, client)
serverCreateRequest := gsclient.ServerCreateRequest{
Name: "api-tests",
Memory: 1,
Cores: 2,
}
cServer, err := client.CreateServer(emptyCtx, serverCreateRequest)
if err != nil {
panic(err)
}
fmt.Printf("%v", cServer)
if err != nil {
panic(err)
}
}
results in
DEBU[2021-02-15T14:47:58+01:00] Preparing POST request sent to URL: http://api.gridscale.io/objects/servers
DEBU[2021-02-15T14:47:58+01:00] Request body: {{"name":"api-tests","memory":1,"cores":2}
}
DEBU[2021-02-15T14:47:58+01:00] Request headers: map[Content-Type:[application/json] User-Agent:[gsclient-go/3.3.2 (darwin)] X-Auth-Token:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] X-Auth-Userid:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]]
DEBU[2021-02-15T14:48:00+01:00] Status code: 200. Request UUID: 6e99e9a2-8c19-45d8-a416-c21e550f751d. Headers: map[Access-Control-Allow-Credentials:[true] Access-Control-Allow-Headers:[Origin, X-Requested-With, Content-Type, Accept, X-Auth-UserId, X-Auth-Token, X-Exec-Time, X-API-Version, X-Api-Client, X-API-Identity, X-Auth-ContractId] Access-Control-Allow-Methods:[GET, POST, PUT, PATCH, DELETE, OPTIONS] Access-Control-Allow-Origin:[*] Access-Control-Expose-Headers:[X-Exec-Time, X-API-Version, X-Request-Id, X-API-Identity] Cache-Control:[no-cache] Content-Length:[3708] Content-Type:[application/json] Date:[Mon, 15 Feb 2021 13:48:00 GMT] Etag:["W/e315f3cacb4e7a3a538f7a85dcb5d46f0f245264"] Ratelimit-Limit:[210] Ratelimit-Remaining:[208] Ratelimit-Reset:[1613396894217] X-Api-Identity:[2] X-Request-Id:[6e99e9a2-8c19-45d8-a416-c21e550f751d] X-Time-Provisioning:[0.547547]]
DEBU[2021-02-15T14:48:00+01:00] Response body: {"servers": {"28c1371a-cb47-498c-b852-cadbecbfda89": {"console_token": "e36965689121ac9a80aaaa8f1069edee5df7b5a9cfbaded9c5b9fe9714e9b90e1fe1d45b3923b2d7f217009e532e6bf2", "labels": [], "create_time": "2021-02-01T09:08:18Z", "change_time": "2021-02-01T09:15:14Z", "location_uuid": "45ed677b-3702-4b36-be2a-a2eab9827950", "name": "4dx-dashboard", "hardware_profile": "q35", "availability_zone": null, "memory": 2, "auto_recovery": true, "location_iata": "fra", "cores": 1, "power": true, "location_name": "de/fra", "object_uuid": "28c1371a-cb47-498c-b852-cadbecbfda89", "location_country": "de", "legacy": false, "status": "active", "relations": {"storages": [{"bus": 0, "controller": 0, "create_time": "2021-02-01T09:08:18Z", "target": 0, "bootdevice": true, "last_used_template": "fd65f8ce-e2c6-40af-8fc3-92efa0d4eecb", "lun": 0, "storage_type": "storage", "object_uuid": "79f10e8b-33b2-4973-99ce-40aa20747734", "object_name": "4dx-dashboard", "capacity": 10, "license_product_no": null}], "networks": [{"vxlan": null, "create_time": "2021-02-01T09:08:18Z", "ordering": 0, "public_net": true, "network_uuid": "23e0345d-4313-4307-b7b9-9eb666dfac43", "firewall_template_uuid": null, "bootdevice": false, "vlan": null, "l3security": [], "network_type": "network", "firewall": null, "mac": "16:40:87:d7:f8:01", "mcast": null, "object_uuid": "23e0345d-4313-4307-b7b9-9eb666dfac43", "l2security": true, "partner_uuid": "95a2980b-7012-43dd-81f2-07577cfcb9f0", "object_name": "Public Network"}], "public_ips": [{"create_time": "2021-02-01T09:08:18Z", "object_uuid": "cb47368e-723d-4b06-ac00-3967151a4fcd", "ip": "2a06:2380:0:1::4bb", "family": 6, "prefix": "2a06:2380:0:1::4bb/128"}, {"create_time": "2021-02-01T09:08:18Z", "object_uuid": "c10af4e6-4855-45cb-a56a-93ea7df51d84", "ip": "45.12.51.119", "family": 4, "prefix": "45.12.51.119/32"}], "isoimages": []}, "current_price": 99.99, "usage_in_minutes_cores": 20432, "usage_in_minutes_memory": 40864}, "e3e89eb2-963a-4f73-8090-ca5c97b74941": {"console_token": "5c8f74843e29917b3bf6d7bfe33d99f1d85360fe123062dca9c809ccfa33fe36f4ffc6bf28f714315234399babb74176", "labels": [], "create_time": "2021-01-08T23:24:30Z", "change_time": "2021-02-15T13:47:37Z", "location_uuid": "45ed677b-3702-4b36-be2a-a2eab9827950", "name": "gs-test", "hardware_profile": "q35", "availability_zone": null, "memory": 2, "auto_recovery": true, "location_iata": "fra", "cores": 1, "power": true, "location_name": "de/fra", "object_uuid": "e3e89eb2-963a-4f73-8090-ca5c97b74941", "location_country": "de", "legacy": false, "status": "active", "relations": {"storages": [{"bus": 0, "controller": 0, "create_time": "2021-01-08T23:24:30Z", "target": 0, "bootdevice": true, "last_used_template": "fd65f8ce-e2c6-40af-8fc3-92efa0d4eecb", "lun": 0, "storage_type": "storage_insane", "object_uuid": "6f826003-b32b-46ab-adc0-cccdf4698fa6", "object_name": "gs-test", "capacity": 10, "license_product_no": null}], "networks": [{"vxlan": null, "create_time": "2021-01-08T23:24:30Z", "ordering": 0, "public_net": true, "network_uuid": "23e0345d-4313-4307-b7b9-9eb666dfac43", "firewall_template_uuid": null, "bootdevice": false, "vlan": null, "l3security": [], "network_type": "network", "firewall": null, "mac": "22:cc:c9:3d:52:01", "mcast": null, "object_uuid": "23e0345d-4313-4307-b7b9-9eb666dfac43", "l2security": true, "partner_uuid": "95a2980b-7012-43dd-81f2-07577cfcb9f0", "object_name": "Public Network"}], "public_ips": [{"create_time": "2021-01-08T23:24:30Z", "object_uuid": "d0485231-935c-4ad6-9890-0010e4b35a2e", "ip": "185.201.144.29", "family": 4, "prefix": "185.201.144.29/32"}], "isoimages": []}, "current_price": 99.99, "usage_in_minutes_cores": 54140, "usage_in_minutes_memory": 108280}}}
DEBU[2021-02-15T14:48:01+01:00] Preparing GET request sent to URL: http://api.gridscale.io/requests/6e99e9a2-8c19-45d8-a416-c21e550f751d
DEBU[2021-02-15T14:48:01+01:00] Request body: {}
DEBU[2021-02-15T14:48:01+01:00] Request headers: map[Content-Type:[application/json] User-Agent:[gsclient-go/3.3.2 (darwin)] X-Auth-Token:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] X-Auth-Userid:[XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]]
DEBU[2021-02-15T14:48:02+01:00] Status code: 404. Request UUID: b9f05ddd-e0bf-4e86-9121-6a657bfb9f15. Headers: map[Access-Control-Allow-Credentials:[true] Access-Control-Allow-Headers:[Origin, X-Requested-With, Content-Type, Accept, X-Auth-UserId, X-Auth-Token, X-Exec-Time, X-API-Version, X-Api-Client, X-API-Identity, X-Auth-ContractId] Access-Control-Allow-Methods:[GET, POST, PUT, PATCH, DELETE, OPTIONS] Access-Control-Allow-Origin:[*] Access-Control-Expose-Headers:[X-Exec-Time, X-API-Version, X-Request-Id, X-API-Identity] Cache-Control:[no-cache] Content-Length:[154] Content-Type:[application/json] Date:[Mon, 15 Feb 2021 13:48:03 GMT] Etag:["W/502812e75a4eec35693a6554365702233177fd4f"] Ratelimit-Limit:[210] Ratelimit-Remaining:[207] Ratelimit-Reset:[1613396894217] Vary:[Accept] X-Api-Identity:[2] X-Request-Id:[b9f05ddd-e0bf-4e86-9121-6a657bfb9f15]]
ERRO[2021-02-15T14:48:02+01:00] Error message: {'type': 'HTTPError', 'status_code': 404, 'message': "(404, 'There is no Request for the given Request UUID.')"}. Title: 404 Code. Code: 404. Request UUID: b9f05ddd-e0bf-4e86-9121-6a657bfb9f15.
panic: Status code: 404. Error: {'type': 'HTTPError', 'status_code': 404, 'message': "(404, 'There is no Request for the given Request UUID.')"}. Request UUID: b9f05ddd-e0bf-4e86-9121-6a657bfb9f15.
API endpoint for creating a new storage from a specific backup: POST https://api.gridscale.io/objects/storages/import with {"backup":{"name":"new storage","backup_uuid":"XXXX-XXXX-XXXX-XXXX-XXXX"}}
as a body. This functionality is not described in the official API docs yet.
Using http.Request
when retrying may cause an error http: ContentLength=... with Body length 0
. Error happened here. Why did it happen.
https://github.com/gridscale/gsclient-go/blob/master/marketplace_application.go#L119
Authors string
json:"authors" should be changed to `Author string `json:"author"
Certificate
endpoint has been added to gridscale API more. We have to add this endpoint to the gsclient-go library, and update the loadbalancer
resource in the library as well loadbalancer
Hi there!
Is a CLI client planned, similar to how Azure CLi and so on work?
I ask since I would be interested in such a tool and would maybe like to contribute if you created this one open source.
Please excuse me asking this on the Go Client. I just figured the Go client would be a great starting point for a CLI tool 😉
Add simple support for batch API. Use simple builder pattern. Preliminary branch here: https://github.com/gridscale/gsclient-go/tree/batch
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.