blendle / zapdriver Goto Github PK
View Code? Open in Web Editor NEWBlazing fast, Zap-based Stackdriver logging.
License: ISC License
Blazing fast, Zap-based Stackdriver logging.
License: ISC License
For example https://github.com/kubernetes-sigs/controller-runtime/tree/master/pkg/log/zap returns a zap Log with a custom encoder. I can call log.Core()
on it, but WrapCore
uses the internal core
type rather than a zapcore.Core
What I'd like to do is have the controller-runtime log (which is part of kubebuilder among others) wrapped in zapdriver.
Maybe I'm going about this all wrong...
Thanks for the package!
For the httpRequest
log object: remoteIP
should be remoteIp
and serverIP
should be serverIp
(ref: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest).
logging.googleapis.com/labels
gets included twice, once with the intended content and again as an empty object.
"logging.googleapis.com/labels":{"hello":"world","hi":"universe"},"logging.googleapis.com/labels":{},
$ cat go.mod
module example.com/m
go 1.14
require github.com/blendle/zapdriver v1.3.1
$ cat main.go
package main
import (
"log"
"github.com/blendle/zapdriver"
)
func main() {
logger, err := zapdriver.NewProduction()
if err != nil {
log.Fatal(err)
}
logger.Info(
"Did something.",
zapdriver.Labels(
zapdriver.Label("hello", "world"),
zapdriver.Label("hi", "universe"),
),
)
}
$ go run .
{"severity":"INFO","timestamp":"2020-05-12T15:15:43.21090722-06:00","caller":"z/main.go:14","message":"Did something.","logging.googleapis.com/labels":{"hello":"world","hi":"universe"},"logging.googleapis.com/labels":{},"logging.googleapis.com/sourceLocation":{"file":"/home/tv/z/main.go","line":"14","function":"main.main"}}
Maybe I'm doing something wrong, this is my code:
logger, _ := zapdriver.NewProduction()
logger.Info("Test log message", zap.String("hello", "world"))
And this logs to console:
{
"severity": "INFO",
"timestamp": "2020-05-18T17:04:30.725484-03:00",
"caller": "redacted/main.go:32",
"message": "Test log message",
"hello": "world",
"logging.googleapis.com/labels": {},
"logging.googleapis.com/sourceLocation": {
"file": "redacted/main.go",
"line": "32",
"function": "main.init.0"
}
}
Shouldn't hello:world
be on the labels section? Thanks
Is there anyway to log trace and spanId as documented
Hi,
2 days ago jsonPayload mapping for ZAP to StackDriver stopped working - we now get JSON as textPayload.
It looks like that coincides with an auto-upgrade of master that updated the fluentd-gcp
DaemonSet to gcr.io/stackdriver-agents/stackdriver-logging-agent:0.7-1.6.1-1
Are you aware of changes that might break zapdriver ?
I have forked this repo and am going to be maintaining it moving forward. I just merged some of the useful PRs from this repository and made a new release. I'd really appreciate it if we can have new issues and PRs created on the new fork. Thanks!
Hey, thanks for zapdriver!
I am struggling to fit it in my toy project, and feel that a minimal working example as quickstart guide could be of benefit here.
For example, going with the very last section of the current README file, this does not yield the promised everything just works ™:
//name:above.go
package main
import (
"fmt"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
"github.com/blendle/zapdriver"
)
func main() {
var log *zap.Logger
{
config := zap.NewDevelopmentConfig()
config.EncoderConfig.EncodeLevel = zapcore.CapitalColorLevelEncoder
log, _ = config.Build(zapdriver.WrapCore()) // ←
// log = log.Named("hello-world")
defer log.Sync()
}
log.Info("stub start")
fmt.Println("Hello world!")
log.Info("stub end")
}
$ export GOOGLE_APPLICATION_CREDENTIALS="….json"
$ go run above.go
… nothing shows up in Stackdriver,
Stackdriver seems to be ignoring log entries with the time
field encoded as an RF3339 string. I was able to get it to recognize the compound formats (separate timestampSeconds
and timestampNanos
fields, or a timestamp
object with seconds
and nanos
fields) when appending to the log by hand, but I couldn't figure out how to get zap
to encode a time as anything but a native type.
I did discover that zapcore.EpochTimeEncoder
(floating point seconds since the epoch) works with the time
field. I'll file a bug with Google to clarify (or correct?) the docs.
Hi, is there a way to include traceId for every log? I'm using gin with otel and trying to figure out how to approach this issue
I am trying to get (some) error level logs reported to Error Reporting
service. It doesn't seem to be happening when I use this library. To get reported an error log entry must have a context object attached to it: https://cloud.google.com/error-reporting/docs/formatting-error-messages
Does this library support attaching such a context so that the error gets reported?
Is there a reason why default Encoding
mode for developing config is json
instead of console
, like zap core does?
When I log error using
config := zapdriver.NewProductionConfig()
logger, err := config.Build(zapdriver.WrapCore(
zapdriver.ReportAllErrors(true),
zapdriver.ServiceName("service"),
))
logger.Error("something happened", zap.Error(errors.New("my error"))
I can see error in "Error reporting', no stacktrace.
Stacktrace is in stacktrace
field, but it is not recognized by GCP error reporting?
According to: https://cloud.google.com/error-reporting/reference/rest/v1beta1/projects.events/report#ReportedErrorEvent
it should be included in message.
There is still a problem with concurrency in the Write function. Take the following snippet:
lbls.mutex.RLock()
c.tempLabels.mutex.Lock()
for k, v := range lbls.store {
c.tempLabels.store[k] = v
}
c.tempLabels.mutex.Unlock()
lbls.mutex.RUnlock()
// ...
c.tempLabels = newLabels()
The c.tempLabels
variable is re-initialized after it is unlocked, but if another go routine has already locked the mutex before this line executes then it will end up calling c.tempLabels.mutex.Unlock()
on the new instance of c.tempLabels
which will panic because that mutex is not locked since it is newly created.
I think a better solution is to add a reset function which re-creates the internal map (protected by a mutex) without destroying the actual mutex.
Tests are currently failing on master
panic: runtime error: index out of range [recovered]
panic: runtime error: index out of range
goroutine 11 [running]:
testing.tRunner.func1(0xc4200f4690)
/usr/local/go/src/testing/testing.go:711 +0x2d2
panic(0x6d5680, 0x8cee40)
/usr/local/go/src/runtime/panic.go:491 +0x283
github.com/blendle/zapdriver.TestWriteConcurrent(0xc4200f4690)
<go path>/src/github.com/blendle/zapdriver/core_test.go:138 +0xa40
testing.tRunner(0xc4200f4690, 0x740eb8)
/usr/local/go/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:789 +0x2de
exit status 2
FAIL github.com/blendle/zapdriver 0.008s
Is the purpose of this test running core.Write
concurrently? The test passed when I added a wait group so that logs.All()
does not return an empty array.
The following demo program prints two matching log entries:
package main
import (
"os"
"github.com/blendle/zapdriver"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
func main() {
consoleEncoder := zapcore.NewConsoleEncoder(zap.NewDevelopmentEncoderConfig())
core := zapcore.NewTee(
zapcore.NewCore(consoleEncoder, os.Stderr, zapcore.ErrorLevel),
zapcore.NewCore(consoleEncoder, os.Stdout, zapcore.InfoLevel),
)
logger := zap.New(core, zap.AddCaller())
logger = logger.WithOptions(zapdriver.WrapCore())
defer logger.Sync()
logger.Info("Starting...")
}
I am actually not sure this can be fixed or simply this is a limitation of Zap, but it seems to me that Check
simply cannot work properly in this case without calling Check
on the underlying core. Will investigate the issue further...
Hi, @JeanMertz.
I can't find license file.
Could you tell me this library license? (and please do commit LICENSE file!)
Best regards.
Line 71 in d2d292b
Hi, I found this line will print out fatal error: concurrent map writes
Maybe it should be wrap with a lock?
As the docs describe labels needs to be logged in a JSON log field called: "logging.googleapis.com/labels".
After deploying our application to CloudRun, we seeing that the labels are not part of the labels in the LogEntry, but still are part of jsonPayload.
jsonPayload: { caller: "logging.go:61" logging.googleapis.com/labels: { x-cloud-debug: "test" } logging.googleapis.com/trace_sampled: "true" message: "passing metadata [x-rid]=[1234354]" timestamp: "2019-11-07T13:42:33.524859762Z" } labels: { instanceId: "00bf4bf02d6b0da606a48c835435d7757866a1931b821b6f7bc6402039a3ebd98be580e0cff0273b976d46eddbdc9f5b8f740972efa2866598246ddf6b4c9d9682436f" }
Even though this library has support for defining log labels, this isn't actually properly parsed by Stackdriver at the moment. See this PR for more details.
If the outcome of that PR is that it won't be supported, I'll remove the code from this library, if it is, I'll change the key to the right one (probably logging.googleapis.com/labels
).
Until then, I'll leave it as is.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.