Hi all,

I have the following problem. I have an InfluxDB server where I have a go controller that fetches metrics remotely and store them into an InfluxDB. I need to create now a go forwarder program that reads these metrics from the N measurements (tables) and sends them to another server.

My problem is that I don't know which is the best approach.

I know this library support chunk responses but my problem is that during my query into a measurement:
select * from mymeasurement

1. other data can arrive
2. at each iteration I do not want to send the data already sent.

Hope someone can help here.

Hi all,

I have an InfluxDB with a measurement called "connections" that keep all the info related to PostgreSQL connections.
The measurement (table) has the following fields:

• active
• idle
• idle-transaction
• total
• waiting

and the following tags in the Grafana dashboard will be used to filter the data:

• env
• instance
• dbname
• hostname

Now I use the influxdb1-client to query these info with the following code:

    client, err = client.NewHTTPClient(
	client.HTTPConfig{Addr: "http://localhost:8086",
		Username:           "myuser",
		Password:           "mypwd",
		InsecureSkipVerify: "false"})
if err != nil {
	return nil, err
}
q := client.Query{
	Command:  "select * from backends",
	Database: "myinfluxdb",
}
if response, err := client.Query(q); err == nil {
	if response.Error() != nil {
		return res, response.Error()
	}
	res = response.Results
} else {
	return res, err
}

This code works fine but my problem is that this code only retrieves fields. The field res[0].Series[0].Tags is empty.
How I should change the code to read also, for each row in the measurement, the relative tags?

Right now the examples for querying show querying and immediately printing the result. It would be helpful to add an example showing a user querying and getting the structured data.

The struct returned is interesting https://godoc.org/github.com/influxdata/influxdb1-client/models#Row with a map, list, and list of lists of interface{}. A better example would be awesome.

In client.Point, there are two places where Precision can be entered. Why does the Precision I enter in it not take effect and does not affect the http request at all, so what is the use of it?

Given that influxdb supports it I think it's a good idea to have the option to gzip data before sending it to influxdb for ingestion

since separation, i am missing:

// SetString sets the string value for a string key.
func (a *Tags) SetString(key, value string) {

Hi,
I went through the API code to look for how to run the backup command. I couldn't find any. Is there any indirect way to execute the backup command using this API or modifications can be in the Query function to take backup as well?

Thank you.

According to example_test.go line 100， I received response {"error":"partial write: points beyond retention policy dropped=1"}

But I added Precision: "s", in line 104 bps := client.BatchPoints{}, this error disappears.

So, I want to ask what is the difference between them.

Thanks！

As the code is written now, it doesn't seem that there is support for placing a key value field in the request header. For my use case I'm going through a SSOProxy that requires me to have a header 'Cookie': 'access_token=blahfooblah' access_token in the https header. Is there another way to add a header that I'm missing in the code?

Would it be 'correct' to add the arbitrary header to the config structure then, if that field has a value, add it to the req headers in all the different methods? Or is this considered an API change?

a 10-min newbee with go on Fedora30.

docker exec -it 2256747aa0e2 influx

Connected to http://localhost:8086 version 1.7.8
InfluxDB shell version: 1.7.8

go version

go version go1.12.9 linux/amd64

go run influxdb.go

influxdb.go:22:2: cannot find package "github.com/influxdata/influxdb1-client/models" in any of:
/usr/lib/golang/src/github.com/influxdata/influxdb1-client/models (from $GOROOT)
/root/go/src/github.com/influxdata/influxdb1-client/models (from $GOPATH)

go run example_test.go

go run: cannot run *_test.go files (example_test.go)

Does the timeout in Timeout time.Duration

Does influxDB read is also limited by the timeout specified?

tks

I'd like to use this lib to create a new InfluxDB database like the SQL command create database xxx.
Is it possible to do that?

Hi there, really glad to see a separate repo for this as importing from influxdb proper used to pull in a ton of other dependencies that we don't really need as a client consumer.

I noticed however that points.go still imports from influxdata/influxdb and influxdata/platform which pulls in a bunch of other stuff:

To be honest part of my reasoning for wanting to avoid all these dependencies is that there are a couple dependencies on bzr repositories (namely gocheck). These tend to be much slower and flakier to fetch. Ideally Go modules would just ignore them (as we have no need to pull in test dependencies), but at least right now it really throws a wrench into our builds.

Not sure if it's a priority to keep this repo fairly self-contained, but removing that dependency would help a lot. If not I'm OK with this issue just being closed.

Thanks!

Hello!
I am upgrading a codebase from go 1.13 to 1.16.

The following test is now failing (error message reproduced below):

influxdb_test.go:751: unexpected error. expected 'request canceled' error, got Post "http://127.0.0.1:61711/query?db=&q=": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Here are the lines where the error is occurring .

	} else if !strings.Contains(err.Error(), "request canceled") &&
		!strings.Contains(err.Error(), "use of closed network connection") {
		t.Fatalf("unexpected error. expected 'request canceled' error, got %v", err)
	}
}

A simple example would not hurt either.

name duration shardGroupDuration replicaN default

autogen 0s 168h0m0s 1 false
2_hours 168h0m0s 1h0m0s 1 false
onehours 4h0m0s 1h0m0s 1 true
when I use the autogen policy is ok But the other policy has the bellow error:
error: partial write: points beyond retention policy dropped=1

I wrote some code with Influxdb client and it works fine on x64 but when I run the code on RaspberryPi 4 arm7 it fails to run with error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc pc=0x1de378]

goroutine 1 [running]:
github.com/influxdata/influxdb1-client/v2.(*Response).Error(0x0, 0x1, 0x1)
/home/pi/go/pkg/mod/github.com/influxdata/[email protected]/v2/client.go:477 +0x14

Tested on go version 1.11 and 1.14

I even made some simple code just getting number of points in measurement (code here is sanitized from some lines ):

queryString := fmt.Sprintf("SELECT Count(*) FROM %s", "location_dev")
      q := client.NewQuery(queryString, "testdb", "")
        if response, err := dbClient.Query(q); err == nil && response.Error() == nil {
                if len(response.Results[0].Series) != 0 {
                        totalPoints, _ := response.Results[0].Series[0].Values[0][1].(json.Number).Int64()
                } else 
                        return
                }
        } else {
                fmt.Println(err)
                fmt.Println(response.Error())

        }
}

And it still fails to run. What is strange is that there should be some points in db but it seems that the code go to last line fmt.Println(response.Error()) like there has been some error.
It always panics in 477 line of client.go

Has any one got the same error, do you have any solution?

When I used UDP client to query, case panic: Querying via UDP is not supported

code summary:

c, err = client.NewUDPClient(client.UDPConfig{
     Addr: alc.InfluxDBAddr,
})
...
client.NewQuery(fmt.Sprintf("CREATE DATABASE %s", "testdb"), "", "")
response, err := c.Query(q)

At least since InfluxDB server v1.8.0 precision in client cannot be set to micro seconds. Using "u", validation fails in NewBatchPoints(), using "us", Write() fails:

precision 'u' failed: NewBatchPoints failed: time: unknown unit u in duration 1u
precision 'us' failed: Write failed: {"error":"invalid precision \"us\" (use n, u, ms, s, m or h)"}

In influxdata/influxdb@275b02e a verification rejecting write requests with precision "us" was added.

In Influx, if we want to write a timestamp of our own, we can write like this
insert measure1 timestamp1=15678267593i,name="Victoria"
and later fetch data based on the same, this works = "select * from measure1 where timestamp1=15678267593".
I do not understand how to enable the same with go influx client. I do not want to override time (using the Time field in Point structure).

Please let me know on how to enable the same or whether that functionality is supported in the existing influx go client or not.

the error message like this:
Post "http://1.2.3.4:8429/write?consistency=&db=&precision=ns&rp=": EOF

While working with the client, we noticed that NewClient creates an http.Client with a custom transport.

There is no option to pass settings like MaxIdleConns to the transport which means the client ends up using unlimited connections. I propose that the NewClient should be taking in a *http.Client similar to how the GitHub client does.

https://0x46.net/thoughts/2018/12/29/go-libraries/ also talks about some issues with hardcoding an http.Client. I am happy to send a PR for the change if the proposal sounds good.

How to use this lib with a Influx Db Cloud like https://europe-west1-1.gcp.cloud2.influxdata.com/

the cloud dashboard gives me a token, but the go client asks for the user and password.

Use case :
I have continues data flow so I am using

 // Create a point and add to batch
 pt, _ := client.NewPoint(mesurment, tags, fields, time.Now())
 // additng point to batchpoints
influx.bp.AddPoint(pt) 
// bunching all data of particular interval say 5s then writing it 
 influx.client.Write(influx.bp)

So is there is way to clear BatchPoints ?

(r *ChunkedResponse) NextResponse() is currently defined by the following code on client.go. The error checking forgets about the original error and then creates another one from the buffer. For timeout errors, this causes that the error message is empty ("").

// NextResponse reads the next line of the stream and returns a response.
func (r *ChunkedResponse) NextResponse() (*Response, error) {
	var response Response
	if err := r.dec.Decode(&response); err != nil {
		if err == io.EOF {
			return nil, err
		}
		// A decoding error happened. This probably means the server crashed
		// and sent a last-ditch error message to us. Ensure we have read the
		// entirety of the connection to get any remaining error text.
		io.Copy(ioutil.Discard, r.duplex)
		return nil, errors.New(strings.TrimSpace(r.buf.String()))
	}

	r.buf.Reset()
	return &response, nil
}

Trying to build a go module (using go 1.11) with the InfluxDB client imported throws an error. Here is a snippet to reproduce this:

package main // import "github.com/user/influx"

import "github.com/influxdata/influxdb1-client/v2"

func main() {
	// Do anything with the library
	client.NewHTTPClient(client.HTTPConfig{})
}

Building without modules succeeds:

ak@zonda:/tmp/influx$ go build
ak@zonda:/tmp/influx$ ll
total 6404
drwxrwxr-x  2 ak   ak      4096 Jan 24 19:59 ./
drwxrwxrwt 25 root root   16384 Jan 24 19:59 ../
-rwxrwxr-x  1 ak   ak   6531840 Jan 24 19:59 influx*
-rw-rw-r--  1 ak   ak       191 Jan 24 19:56 main.go

Trying to build a module fails:

ak@zonda:/tmp/influx$ go mod init
go: creating new go.mod: module github.com/user/influx
ak@zonda:/tmp/influx$ go build
go: finding github.com/influxdata/influxdb1-client/v2 latest
go: downloading github.com/influxdata/influxdb1-client/v2 v2.0.0-20190118215656-f8cdb5d5f175
build github.com/user/influx: cannot find module for path github.com/influxdata/influxdb1-client/v2

Any tips? Thanks in advance!

I have see the source code, can't find how to execute the "influxd backup" command.

Hi, I am using influxdb1.5.4, and want to know if there is a way to update password for user by code

go get github.com/influxdata/influxdb1-client/v2
gives this error:

github.com/influxdata/flux/values

......\lib\src\github.com\influxdata\flux\values\dict.go:261:25: cannot use immutable.NewSortedMap(dictComparer(dictType)) (type *immutable.SortedMap) as type immutable.Comparer in argument to immutable.NewSortedMapBuilder:
*immutable.SortedMap does not implement immutable.Comparer (missing Compare method)

What could be the issue?

Hi,

after some thorough testing with goroutines, I do not understand how the documentation can claim:

// client is safe for concurrent use as the fields are all read-only
// once the client is instantiated.

Indeed, trying this snippet of code in a for loop works properly:

q := client.NewQuery(query_string, "mydb", "s")
response, err := c.Query(q)

However trying the same in a for loop starting goroutines, most of my functions error out with:
2021/04/29 15:05:24 Error querying influx unable to decode json: received status code 200 err: unexpected EOF

Now with goroutines, the same snippet works with a sync.Mutex:

l.Lock()
q := client.NewQuery(query_string, "schoonschip", "s")
response, err := c.Query(q)
l.Unlock()

What am I missing here?

As map in Golang is unordered, so how to keep the order of tags in line ?

Incomplete reading of body causes the connection not reusable

Forwarded from email:

Customer found a bug in our original Golang client which caused HTTP connections to not be reused.
This means that every single query had to establish a new connection (so that’s time for TCP + HTTPS handshakes front-loaded onto every query).

Basically, the client hands the body of to the JSON decoder here: https://github.com/influxdata/influxdb1-client/blob/master/influxdb.go#L273

But, when the decoder reads the body it doesn’t read in the trailing newline (I guess because it’s found it’s final }). That leaves 1 byte unread from the body - because the body hasn’t been read in full, the connection can’t be re-used.

Suggested fix is to adjust the deferred close (https://github.com/influxdata/influxdb1-client/blob/master/influxdb.go#L273) to also perform a final read

        defer func() {
                io.Copy(ioutil.Discard, resp.Body)
                resp.Body.Close()
        }()

allowing the connection to be reused

influxdb.go

        dec := json.NewDecoder(resp.Body)

Just curious why this is done.

https://github.com/influxdata/influxdb1-client/blob/master/v2/client.go#L389

Thanks!

`tkaagent@ubuntu1:~/jtimon$ sudo go build

_/home/tkaagent/jtimon

./subscribe_cisco_iosxr.go:198:33: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to IOSXRExtensibleManagabilityService.NewGRPCConfigOperClient
./subscribe_juniper_junos.go:105:41: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to "github.com/nileshsimaria/jtimon/telemetry".NewOpenConfigTelemetryClient
./subscribe_juniper_junos.go:228:32: cannot use conn (type *"google.golang.org/grpc".ClientConn) as type *"github.com/nileshsimaria/jtimon/vendor/google.golang.org/grpc".ClientConn in argument to authentication.NewLoginClient
tkaagent@ubuntu1:~/jtimon$`

As far as I can tell, there's no support at all for flux queries in the the currently stable go client. Since it's a feature of the 1.7 stable version of InfluxDB, it's a major disappointment when I first learned of this. The only implementation of a go client that supports flux seems to be the new go client which isn't close to stable.

Marshaling a field with type uint64 will trigger an "invalid number" error.

This seems to come from these lines: https://github.com/influxdata/influxdb1-client/blob/master/models/points.go#L2354-L2356

A solution would be to cast this to int64 similarly to the cases underneath it.

Additional information

Shell log:

> INSERT convert,hostname=hanaharu wrote_bytes=2363546 1593054632437
> INSERT convert,hostname=hanaharu wrote_bytes=2363546u 1593054632437
ERR: {"error":"unable to parse 'convert,hostname=hanaharu wrote_bytes=2363546u 1593054632437': invalid number"}

InfluxDB v1.8 Data Types for the line protocol reference, which doesn't show unsigned types.