Giter Club home page Giter Club logo

go-proto-zap-marshaler's Introduction

go-proto-zap-marshaler

A protoc plugin whitch generates MarshalLogObject() functions for each generated go structs to implement zapcore.ObjectMarshaler interface for uber-go/zap. So you can use zap.Object("proto", someProtoMessage) to log contents of the proto message.

Install

There are 2 protoc plugins to generate marshaler functions for zap:

  • protoc-gen-zap-marshaler
    • which genearetes marshaler functions for all messages.
  • protoc-gen-zap-marshaler-secure
    • which genearetes marshaler functions for all messages, but only logs fields enabled by proto option.

You can install those plugins by the following command:

$ go get github.com/kazegusuri/go-proto-zap-marshaler/protoc-gen-zap-marshaler
$ go get github.com/kazegusuri/go-proto-zap-marshaler/protoc-gen-zap-marshaler-secure

Usage

protoc-gen-zap-marshaler

To generate marshaler functions from proto, use --zap-marshaler_out with protoc command. That runs protoc-gen-zap-marshaler internally and then generates *.zap.go files.

$ protoc --zap-marshaler_out=. path/to/example.proto

protoc-gen-zap-marshaler-secure

This plugin generates marshaler functions as well as protoc-gen-zap-marshaler, but the funtions does not log anything as default. The marshaler only marshal fields enabled by zap_marshaler option.

To enable field option for message fields, you need to define proto like this:

message SimpleMessage {
  string string_value = 1 [(kazegusuri.zap_mashaler.field) = {enabled: true}];
  bool bool_value = 2 [(kazegusuri.zap_mashaler.field) = {enabled: true}];
}

To generate marshaler functions from proto, use --zap-marshaler-secure_out with protoc command. That runs protoc-gen-zap-marshaler-secure internally and then generates *.zap.go files.

$ protoc --zap-marshaler-secure_out=. path/to/example.proto

Field option

message ZapMarshalerRule {
    bool enabled = 1;
}
  • enabled
    • The marshaler only logs a field whose value is true

Example

For this proto without field option, the secure plugin generates a go function:

syntax = "proto3";
package example;

message SimpleMessage {
  string string_value = 1;
  bool bool_value = 2;
}
func (m *SimpleMessage) MarshalLogObject(enc go_uber_org_zap_zapcore.ObjectEncoder) error {
	var keyName string
	_ = keyName

	return nil
}

The generated function does nothing.

By changing the proto with field option like this, the secure plugin generates a go function:

syntax = "proto3";
package example;

import "kazegusuri/go-prot-zap-marshaler/zap_marshaler.proto";

message SimpleMessage {
  string string_value = 1 [(kazegusuri.zap_mashaler.field) = {enabled: true}];
  bool bool_value = 2 [(kazegusuri.zap_mashaler.field) = {enabled: true}];
}
func (m *SimpleMessage) MarshalLogObject(enc go_uber_org_zap_zapcore.ObjectEncoder) error {
	var keyName string
	_ = keyName

	keyName = "string_value" // field string_value = 1
	enc.AddString(keyName, m.StringValue)

	keyName = "bool_value" // field bool_value = 2
	enc.AddBool(keyName, m.BoolValue)

	return nil
}

go-proto-zap-marshaler's People

Contributors

kazegusuri avatar stewartbzillow avatar upamune avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

go-proto-zap-marshaler's Issues

Simple message generates compile time error.

Hello @kazegusuri! I'm hoping to get some input on the following error which I've encountered while using this project.

I have the following proto message:

syntax = "proto3";

option go_package = "github.com/kashing/goofys/proto";

import "google/protobuf/timestamp.proto";

message NodeMetadata {
  map<string, NodeMetadata> children = 1;
  uint64 size = 2;
  google.protobuf.Timestamp last_modified_at = 3;
  string name = 4;
  bool directory = 5;
  bool root = 6;
}

After generating the ObjectMarshaler interface via the latest version of the plugin, I am left with the following file which generates a compile time error.

// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: github.com/kashing/goofys/proto/metadata.proto

package proto

import (
	fmt "fmt"
	math "math"
	proto "github.com/golang/protobuf/proto"
	_ "google.golang.org/protobuf/types/known/timestamppb"
	go_uber_org_zap_zapcore "go.uber.org/zap/zapcore"
	github_com_golang_protobuf_ptypes "github.com/golang/protobuf/ptypes"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

func (m *NodeMetadata) MarshalLogObject(enc go_uber_org_zap_zapcore.ObjectEncoder) error {
	var keyName string
	_ = keyName

	if m == nil {
		return nil
	}

	keyName = "children" // field children = 1
	enc.AddObject(keyName, go_uber_org_zap_zapcore.ObjectMarshalerFunc(func(enc go_uber_org_zap_zapcore.ObjectEncoder) error {
		for mk, mv := range m.Children {
			key := mk
			_ = key
			if mv != nil {
				var vv interface{} = mv
				if marshaler, ok := vv.(go_uber_org_zap_zapcore.ObjectMarshaler); ok {
					enc.AddObject(key, marshaler)
				}
			}
		}
		return nil
	}))

	keyName = "size" // field size = 2
	enc.AddUint64(keyName, m.Size_)

	keyName = "last_modified_at" // field last_modified_at = 3
	if t, err := github_com_golang_protobuf_ptypes.Timestamp(m.LastModifiedAt); err == nil {
		enc.AddTime(keyName, t)
	}

	keyName = "name" // field name = 4
	enc.AddString(keyName, m.Name)

	keyName = "directory" // field directory = 5
	enc.AddBool(keyName, m.Directory)

	keyName = "root" // field root = 6
	enc.AddBool(keyName, m.Root)

	return nil
}

The issue is at the following line when the size key is added to the encoder. (m.Size_ is not valid, this should be m.Size)

enc.AddUint64(keyName, m.Size_)

Looking for input on why this might be happening for this proto message alone (I've used to project with great success for many other proto files).

Happy to submit a PR to address this if you can point me in the right direction? I'm not seeing anything obvious after a cursory scan of the codebase (though i'm not terribly familiar working with the protoc compiler plugin interface, so there might be something simple)

Thanks for your work on this excellent project!

Inconsistent import order.

Hi, thanks for the project.

I noticed that when repeatedly running protoc with zap, the import order in the go code changes, despite nothing changing in the source proto files. It would be nice if it were always generating in the same order that gofmt sets.

For example with this small proto file.

main.proto

syntax = "proto3";

package test;

import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";

If I run it twice, I get a diff like this.

-	_ "google.golang.org/protobuf/types/known/structpb"
-	_ "google.golang.org/protobuf/types/known/timestamppb"
+	_ "google.golang.org/protobuf/types/known/timestamppb"
+	_ "google.golang.org/protobuf/types/known/structpb"

A full log is below.

$ protoc --zap-marshaler_out=. main.proto && cat main.zap.go 
main.proto:5:1: warning: Import google/protobuf/struct.proto is unused.
main.proto:6:1: warning: Import google/protobuf/timestamp.proto is unused.
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: main.proto

package test

import (
	fmt "fmt"
	math "math"
	proto "github.com/golang/protobuf/proto"
	_ "google.golang.org/protobuf/types/known/timestamppb"      <<<<---- timestamp first
	_ "google.golang.org/protobuf/types/known/structpb"              <<<<---- structpb second
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

$ protoc --zap-marshaler_out=. main.proto && cat main.zap.go
main.proto:5:1: warning: Import google/protobuf/struct.proto is unused.
main.proto:6:1: warning: Import google/protobuf/timestamp.proto is unused.
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: main.proto

package test

import (
	fmt "fmt"
	math "math"
	proto "github.com/golang/protobuf/proto"
	_ "google.golang.org/protobuf/types/known/structpb"                <<<<---- structpb first
	_ "google.golang.org/protobuf/types/known/timestamppb"        <<<<---- timestamp second
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

Proto3's optional keyword causes issues in zap code generation

Proto3 introduced version 3.12 changelog introduced the optional field label.

However, a message like this:

message Sender { optional string polaris_id = 1; }

generated code like this

`
keyName = "polaris_id" // field polaris_id = 1
if ov, ok := m.GetXPolarisId().(*Sender_PolarisId); ok {
_ = ov
enc.AddString(keyName, ov.PolarisId)
}

`

However, the GetXPolarisId() doesn't exist and creates a compiletime error.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.