influxdata / influxdb1-client Goto Github PK
View Code? Open in Web Editor NEWThe old clientv2 for InfluxDB 1.x
License: MIT License
The old clientv2 for InfluxDB 1.x
License: MIT License
Hi all,
I have an InfluxDB with a measurement called "connections" that keep all the info related to PostgreSQL connections.
The measurement (table) has the following fields:
and the following tags in the Grafana dashboard will be used to filter the data:
Now I use the influxdb1-client to query these info with the following code:
client, err = client.NewHTTPClient(
client.HTTPConfig{Addr: "http://localhost:8086",
Username: "myuser",
Password: "mypwd",
InsecureSkipVerify: "false"})
if err != nil {
return nil, err
}
q := client.Query{
Command: "select * from backends",
Database: "myinfluxdb",
}
if response, err := client.Query(q); err == nil {
if response.Error() != nil {
return res, response.Error()
}
res = response.Results
} else {
return res, err
}
This code works fine but my problem is that this code only retrieves fields. The field res[0].Series[0].Tags is empty.
How I should change the code to read also, for each row in the measurement, the relative tags?
Hi,
after some thorough testing with goroutines, I do not understand how the documentation can claim:
// client is safe for concurrent use as the fields are all read-only
// once the client is instantiated.
Indeed, trying this snippet of code in a for loop works properly:
q := client.NewQuery(query_string, "mydb", "s")
response, err := c.Query(q)
However trying the same in a for loop starting goroutines, most of my functions error out with:
2021/04/29 15:05:24 Error querying influx unable to decode json: received status code 200 err: unexpected EOF
Now with goroutines, the same snippet works with a sync.Mutex
:
l.Lock()
q := client.NewQuery(query_string, "schoonschip", "s")
response, err := c.Query(q)
l.Unlock()
What am I missing here?
the error message like this:
Post "http://1.2.3.4:8429/write?consistency=&db=&precision=ns&rp=": EOF
Hi there, really glad to see a separate repo for this as importing from influxdb
proper used to pull in a ton of other dependencies that we don't really need as a client consumer.
I noticed however that points.go
still imports from influxdata/influxdb
and influxdata/platform
which pulls in a bunch of other stuff:
influxdb1-client/models/points.go
Lines 18 to 19 in e237d47
To be honest part of my reasoning for wanting to avoid all these dependencies is that there are a couple dependencies on bzr
repositories (namely gocheck
). These tend to be much slower and flakier to fetch. Ideally Go modules would just ignore them (as we have no need to pull in test dependencies), but at least right now it really throws a wrench into our builds.
Not sure if it's a priority to keep this repo fairly self-contained, but removing that dependency would help a lot. If not I'm OK with this issue just being closed.
Thanks!
tks
Marshaling a field with type uint64
will trigger an "invalid number" error.
This seems to come from these lines: https://github.com/influxdata/influxdb1-client/blob/master/models/points.go#L2354-L2356
A solution would be to cast this to int64
similarly to the cases underneath it.
Shell log:
> INSERT convert,hostname=hanaharu wrote_bytes=2363546 1593054632437
> INSERT convert,hostname=hanaharu wrote_bytes=2363546u 1593054632437
ERR: {"error":"unable to parse 'convert,hostname=hanaharu wrote_bytes=2363546u 1593054632437': invalid number"}
InfluxDB v1.8 Data Types for the line protocol reference, which doesn't show unsigned types.
While working with the client, we noticed that NewClient creates an http.Client
with a custom transport.
Line 164 in f8cdb5d
There is no option to pass settings like MaxIdleConns
to the transport which means the client ends up using unlimited connections. I propose that the NewClient should be taking in a *http.Client
similar to how the GitHub client does.
https://0x46.net/thoughts/2018/12/29/go-libraries/ also talks about some issues with hardcoding an http.Client
. I am happy to send a PR for the change if the proposal sounds good.
Trying to build a go module (using go 1.11) with the InfluxDB client imported throws an error. Here is a snippet to reproduce this:
package main // import "github.com/user/influx"
import "github.com/influxdata/influxdb1-client/v2"
func main() {
// Do anything with the library
client.NewHTTPClient(client.HTTPConfig{})
}
Building without modules succeeds:
ak@zonda:/tmp/influx$ go build
ak@zonda:/tmp/influx$ ll
total 6404
drwxrwxr-x 2 ak ak 4096 Jan 24 19:59 ./
drwxrwxrwt 25 root root 16384 Jan 24 19:59 ../
-rwxrwxr-x 1 ak ak 6531840 Jan 24 19:59 influx*
-rw-rw-r-- 1 ak ak 191 Jan 24 19:56 main.go
Trying to build a module fails:
ak@zonda:/tmp/influx$ go mod init
go: creating new go.mod: module github.com/user/influx
ak@zonda:/tmp/influx$ go build
go: finding github.com/influxdata/influxdb1-client/v2 latest
go: downloading github.com/influxdata/influxdb1-client/v2 v2.0.0-20190118215656-f8cdb5d5f175
build github.com/user/influx: cannot find module for path github.com/influxdata/influxdb1-client/v2
Any tips? Thanks in advance!
I wrote some code with Influxdb client and it works fine on x64 but when I run the code on RaspberryPi 4 arm7 it fails to run with error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc pc=0x1de378]
goroutine 1 [running]:
github.com/influxdata/influxdb1-client/v2.(*Response).Error(0x0, 0x1, 0x1)
/home/pi/go/pkg/mod/github.com/influxdata/[email protected]/v2/client.go:477 +0x14
Tested on go version 1.11 and 1.14
I even made some simple code just getting number of points in measurement (code here is sanitized from some lines ):
queryString := fmt.Sprintf("SELECT Count(*) FROM %s", "location_dev")
q := client.NewQuery(queryString, "testdb", "")
if response, err := dbClient.Query(q); err == nil && response.Error() == nil {
if len(response.Results[0].Series) != 0 {
totalPoints, _ := response.Results[0].Series[0].Values[0][1].(json.Number).Int64()
} else
return
}
} else {
fmt.Println(err)
fmt.Println(response.Error())
}
}
And it still fails to run. What is strange is that there should be some points in db but it seems that the code go to last line fmt.Println(response.Error()) like there has been some error.
It always panics in 477 line of client.go
Has any one got the same error, do you have any solution?
As far as I can tell, there's no support at all for flux queries in the the currently stable go client. Since it's a feature of the 1.7 stable version of InfluxDB, it's a major disappointment when I first learned of this. The only implementation of a go client that supports flux seems to be the new go client which isn't close to stable.
Use case :
I have continues data flow so I am using
// Create a point and add to batch
pt, _ := client.NewPoint(mesurment, tags, fields, time.Now())
// additng point to batchpoints
influx.bp.AddPoint(pt)
// bunching all data of particular interval say 5s then writing it
influx.client.Write(influx.bp)
So is there is way to clear BatchPoints
?
since separation, i am missing:
// SetString sets the string value for a string key.
func (a *Tags) SetString(key, value string) {
Given that influxdb supports it I think it's a good idea to have the option to gzip data before sending it to influxdb for ingestion
Hello!
I am upgrading a codebase from go 1.13 to 1.16.
The following test is now failing (error message reproduced below):
influxdb_test.go:751: unexpected error. expected 'request canceled' error, got Post "http://127.0.0.1:61711/query?db=&q=": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Here are the lines where the error is occurring .
} else if !strings.Contains(err.Error(), "request canceled") &&
!strings.Contains(err.Error(), "use of closed network connection") {
t.Fatalf("unexpected error. expected 'request canceled' error, got %v", err)
}
}
In client.Point, there are two places where Precision can be entered. Why does the Precision I enter in it not take effect and does not affect the http request at all, so what is the use of it?
In Influx, if we want to write a timestamp of our own, we can write like this
insert measure1 timestamp1=15678267593i,name="Victoria"
and later fetch data based on the same, this works = "select * from measure1 where timestamp1=15678267593".
I do not understand how to enable the same with go influx client. I do not want to override time (using the Time field in Point structure).
Please let me know on how to enable the same or whether that functionality is supported in the existing influx go client or not.
According to example_test.go line 100, I received response {"error":"partial write: points beyond retention policy dropped=1"}
But I added Precision: "s",
in line 104 bps := client.BatchPoints{}
, this error disappears.
So, I want to ask what is the difference between them.
Thanks!
As map in Golang is unordered, so how to keep the order of tags in line ?
How to use this lib with a Influx Db Cloud like https://europe-west1-1.gcp.cloud2.influxdata.com/
the cloud dashboard gives me a token, but the go client asks for the user and password.
A simple example would not hurt either.
Right now the examples for querying show querying and immediately printing the result. It would be helpful to add an example showing a user querying and getting the structured data.
The struct returned is interesting https://godoc.org/github.com/influxdata/influxdb1-client/models#Row with a map, list, and list of lists of interface{}. A better example would be awesome.
a 10-min newbee with go on Fedora30.
Connected to http://localhost:8086 version 1.7.8
InfluxDB shell version: 1.7.8
go version go1.12.9 linux/amd64
influxdb.go:22:2: cannot find package "github.com/influxdata/influxdb1-client/models" in any of:
/usr/lib/golang/src/github.com/influxdata/influxdb1-client/models (from $GOROOT)
/root/go/src/github.com/influxdata/influxdb1-client/models (from $GOPATH)
go run: cannot run *_test.go files (example_test.go)
go get github.com/influxdata/influxdb1-client/v2
gives this error:
......\lib\src\github.com\influxdata\flux\values\dict.go:261:25: cannot use immutable.NewSortedMap(dictComparer(dictType)) (type *immutable.SortedMap) as type immutable.Comparer in argument to immutable.NewSortedMapBuilder:
*immutable.SortedMap does not implement immutable.Comparer (missing Compare method)
What could be the issue?
(r *ChunkedResponse) NextResponse()
is currently defined by the following code on client.go. The error checking forgets about the original error and then creates another one from the buffer. For timeout errors, this causes that the error message is empty ("").
// NextResponse reads the next line of the stream and returns a response.
func (r *ChunkedResponse) NextResponse() (*Response, error) {
var response Response
if err := r.dec.Decode(&response); err != nil {
if err == io.EOF {
return nil, err
}
// A decoding error happened. This probably means the server crashed
// and sent a last-ditch error message to us. Ensure we have read the
// entirety of the connection to get any remaining error text.
io.Copy(ioutil.Discard, r.duplex)
return nil, errors.New(strings.TrimSpace(r.buf.String()))
}
r.buf.Reset()
return &response, nil
}
I'd like to use this lib to create a new InfluxDB database like the SQL command create database xxx
.
Is it possible to do that?
Hi, I am using influxdb1.5.4, and want to know if there is a way to update password for user by code
Does the timeout in Timeout time.Duration
Line 47 in a9ab567
Does influxDB read is also limited by the timeout specified?
At least since InfluxDB server v1.8.0 precision in client cannot be set to micro seconds. Using "u"
, validation fails in NewBatchPoints()
, using "us"
, Write()
fails:
precision 'u' failed: NewBatchPoints failed: time: unknown unit u in duration 1u
precision 'us' failed: Write failed: {"error":"invalid precision \"us\" (use n, u, ms, s, m or h)"}
In influxdata/influxdb@275b02e a verification rejecting write requests with precision "us"
was added.
Hi,
I went through the API code to look for how to run the backup command. I couldn't find any. Is there any indirect way to execute the backup command using this API or modifications can be in the Query function to take backup as well?
Thank you.
`tkaagent@ubuntu1:~/jtimon$ sudo go build
./subscribe_cisco_iosxr.go:198:33: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to IOSXRExtensibleManagabilityService.NewGRPCConfigOperClient
./subscribe_juniper_junos.go:105:41: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to "github.com/nileshsimaria/jtimon/telemetry".NewOpenConfigTelemetryClient
./subscribe_juniper_junos.go:228:32: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to authentication.NewLoginClient
tkaagent@ubuntu1:~/jtimon$`
Hi all,
I have the following problem. I have an InfluxDB server where I have a go controller that fetches metrics remotely and store them into an InfluxDB. I need to create now a go forwarder program that reads these metrics from the N measurements (tables) and sends them to another server.
My problem is that I don't know which is the best approach.
I know this library support chunk responses but my problem is that during my query into a measurement:
select * from mymeasurement
Hope someone can help here.
name duration shardGroupDuration replicaN default
autogen 0s 168h0m0s 1 false
2_hours 168h0m0s 1h0m0s 1 false
onehours 4h0m0s 1h0m0s 1 true
when I use the autogen policy is ok But the other policy has the bellow error:
error: partial write: points beyond retention policy dropped=1
When I used UDP client to query, case panic: Querying via UDP is not supported
code summary:
c, err = client.NewUDPClient(client.UDPConfig{
Addr: alc.InfluxDBAddr,
})
...
client.NewQuery(fmt.Sprintf("CREATE DATABASE %s", "testdb"), "", "")
response, err := c.Query(q)
Just curious why this is done.
https://github.com/influxdata/influxdb1-client/blob/master/v2/client.go#L389
Thanks!
I have see the source code, can't find how to execute the "influxd backup" command.
As the code is written now, it doesn't seem that there is support for placing a key value field in the request header. For my use case I'm going through a SSOProxy that requires me to have a header 'Cookie': 'access_token=blahfooblah' access_token in the https header. Is there another way to add a header that I'm missing in the code?
Would it be 'correct' to add the arbitrary header to the config structure then, if that field has a value, add it to the req headers in all the different methods? Or is this considered an API change?
Incomplete reading of body causes the connection not reusable
Forwarded from email:
Customer found a bug in our original Golang client which caused HTTP connections to not be reused.
This means that every single query had to establish a new connection (so that’s time for TCP + HTTPS handshakes front-loaded onto every query).
Basically, the client hands the body of to the JSON decoder here: https://github.com/influxdata/influxdb1-client/blob/master/influxdb.go#L273
But, when the decoder reads the body it doesn’t read in the trailing newline (I guess because it’s found it’s final }). That leaves 1 byte unread from the body - because the body hasn’t been read in full, the connection can’t be re-used.
Suggested fix is to adjust the deferred close (https://github.com/influxdata/influxdb1-client/blob/master/influxdb.go#L273) to also perform a final read
defer func() {
io.Copy(ioutil.Discard, resp.Body)
resp.Body.Close()
}()
allowing the connection to be reused
dec := json.NewDecoder(resp.Body)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.