capnproto / go-capnp Goto Github PK

View Code? Open in Web Editor NEW

1.2K 29.0 108.0 6.87 MB

Cap'n Proto library and code generator for Go

Home Page: https://capnproto.org

License: Other

Go 86.52% Cap'n Proto 13.02% Shell 0.30% Python 0.15%

capn-proto rpc golang serialization capabilities capability-based go capnproto capnp object-capabilities

go-capnp's People

Contributors

Stargazers

Watchers

Forkers

hodduc alekstorm libliflin soargon awesomeleo glycerine startupit69 samdolan lvdlvd omriharel zenhack yarwelp zackslash pavius abraithwaite valery-barysok justincormack w4ngyi levrado glendc tj devopsmi izagz godeep rnzsgh bmartynov forging2012 tsuna happy-ferret wedancedalot jayconrod thsnr theradioguy bekirberksenel cbranch andrewchambers aloknnikhil 0-admin oncilla urandom2 corerabbit zetogk kiwicom kormat backwardn lukedirtwalker lstio linuxerwang ismiyati jriest asido ccmlm rtxlab bobg shmsr rflorenc strogo pr0ger taylorjdawson clarkguan madbase frumioj flier zalf-rpm seanpm2001 seanwallawalla-forks liranbg isabella232 theapemachine thomas5lab jpolaris lotdeef ajunlonglive jared-bounti mayhemheroes alistairking kentik mhart hspaay anima-os-dev pikrzysztof grishy piotrkowalczuk alrs rtonini-r7 alex242131 iq-scm mikelsr rickygrassmuck davidhubbard pyjcode vforks lizc2003 neuroradiology alexforster fcelda rexposadas spikebike matheusd homier

go-capnp's Issues

Include base capnproto schemas

I'm working on wrappers for sandstorm's APIs, at:

https://github.com/zenhack/go.sandstorm

Currently it just includes some scripts (with their output) that call capnpc-go on the sandstorm APIs, as well as the base capnproto schemas (as those are dependencies). It might make sense to include the latter in this repository, since those aren't sandstorm specific. Thoughts?

allow returning []byte in place of string fields

On v1, I was seeing alot of allocation due to string copying. So I added 2 small upgrades:

a) BenchmarkUnmarshalCapnpZeroCopyNoAlloc() in mem.go

b) Bytes() and TextList.AtBytes() methods that return the []byte slice directly instead of copying it to a string.

The reduction in memory allocation means that I can then read a ton of mmap-ed data and analyze the string fields, all while doing zero-allocation and having zero GC pressure.

See glycerine/go-capnproto@db36ab2

comparisons:

BenchmarkUnmarshalCapnp-4                    2000000           605 ns/op     766.08 MB/s         256 B/op          5 allocs/op              
BenchmarkUnmarshalCapnpZeroCopy-4           10000000           182 ns/op    2539.81 MB/s          88 B/op          3 allocs/op              
BenchmarkUnmarshalCapnpZeroCopyNoAlloc-4    100000000           24.9 ns/op  18668.12 MB/s          0 B/op          0 allocs/op

and

BenchmarkWithoutStringBytes-4               20000000            78.6 ns/op        32 B/op          1 allocs/op   
BenchmarkStringBytes-4                      50000000            29.4 ns/op         0 B/op          0 allocs/op

It would be great to add similar API []byte access to v2.

JSON marshalling/unmarshalling

go-capnproto2 doesn't generate MarshalJSON/UnmarshalJSON methods, which is a regression from v1. v1 solved this by doing codegen guarded by a compile-time constant, which means you had to recompile capnpc-go to generate JSON. This is quite hard to test, and if it were to be always enabled, bloats codegen for those that don't need JSON marshalling.

I think the ideal way to handle it is to store the parsed schema as a byte slice in the generated code, then write a generic utility function (in a separate package) that creates JSON based on a capnp.Struct, the type ID, and the parsed schema. This would also be applicable to #20, since the String representation would work very similarly.

Making calls on same connection as receiving call before acking will deadlock

From @zombiezen on August 6, 2015 21:11

How to deadlock:

Receive call from remote vat that calls a local capability
Call something in the remote vat from the local capability method before calling capnp.Ack

Why this occurs:

*importClient.Call and friends need to synchronize with the connections's "coordinate" goroutine. This deadlocks, because the coordinate goroutine is already attempting to deliver the call to the local capability. The coordinate goroutine must block in order to provide the ordering guarantees discussed in this thread.

I'm alleviating this already by doing some reflection on the client. There's an extensive TODO in rpc/introspect.go for how this might be fixed.

Copied from original issue: zombiezen/go-capnproto#3

internal error with capnpc-go

I'm new to using Cap'n Proto with Go. I have a schema file and it compiles fine with C++, capnp compile -oc++ serialcom.capnp

But with go-capnproto2 I get the following error:

$ capnp compile -I$GOPATH/src/zombiezen.com/go/capnproto2/std -ogo serialcom.capnp

capnpc-go: generating serialcom.capnp: field SerialCom.databits: internal error (bad schema?): missing import declaration for serialcom.capnp:SerialCom.DataBits
go: plugin failed: exit code 1

My serialcom.capnp file:

using Go = import "/go.capnp";

@0xe3d3b49ab383156d;
$Go.package("schema");

struct SerialCom {
    enum Parity {
        none @0;
        odd @1;
        even @2;
        mark @3;
        space @4;
    }
    enum Flow {
        none @0;
        xonXoff @1;
        rtsCts @2;
        dsrDtr @3;
    }
    enum DataBits {
        five @0;
        six @1;
        seven @2;
        eight @3;
    }
    enum StopBits {
        one @0;
        ondAndOneHalf @1;
        two @2;
    }
    baud @0 :UInt32;
    databits @1 :DataBits;
    stopbits @2 :StopBits;
    parity @3 :Parity;
    flow @4 :Flow;
}

$ capnp compile -I$GOPATH/src/zombiezen.com/go/capnproto2/std -ocapnp serialcom.capnp 

# serialcom.capnp
@0xe3d3b49ab383156d;
$import "/go.capnp".package("schema");
struct SerialCom @0xcfcef7366f2a7f37 {  # 16 bytes, 0 ptrs
  baud @0 :UInt32;  # bits[0, 32)
  databits @1 :DataBits;  # bits[32, 48)
  stopbits @2 :StopBits;  # bits[48, 64)
  parity @3 :Parity;  # bits[64, 80)
  flow @4 :Flow;  # bits[80, 96)
  enum Parity @0x9141192533ac9d3c {
    none @0;
    odd @1;
    even @2;
    mark @3;
    space @4;
  }
  enum Flow @0xeb0d81c11a747fd5 {
    none @0;
    xonXoff @1;
    rtsCts @2;
    dsrDtr @3;
  }
  enum DataBits @0xe51c73a9ef981bfc {
    five @0;
    six @1;
    seven @2;
    eight @3;
  }
  enum StopBits @0x85c7d8fddd6bc8fb {
    one @0;
    ondAndOneHalf @1;
    two @2;
  }
}

I'm I doing something wrong when compiling from capnp-go?
I'm using go-capnproto2 from Git, built from 98dbe11 (June 19 10:27:45 2016 ).

Missing import declaration

I have the two following capnp files:

using Go = import "../../zombiezen.com/go/capnproto2/go.capnp";

@0xcde0a8c73d1ea02f;

$Go.package("capnptemplates");

struct Timezone {
  hours @0 :Int8;
  minutes @1 :UInt8;
}

using Go = import "../../zombiezen.com/go/capnproto2/go.capnp";
using import "timezone.capnp".Timezone;

@0xd95c6bcd88adc7f9;

$Go.package("capnptemplates");

struct Date {
  year @0 :Int16;
  month @1 :UInt8;
  day @2 :UInt8;
  tz @3 :Timezone;
}

When I run, I get the following error:

capnp compile -ogo timezone.capnp
capnp compile -ogo date.capnp
capnpc-go: Generating Go for date.capnp failed: missing import declaration for timezone.capnp:Timezone
go: plugin failed: exit code 1
make: *** [date] Error 1

I already tried to put $Go.import("timezone.capnp.go") , $Go.import("myproject/capnptemplates") and $Go.import("myproject/capnptemplates/timezone.capnp.go") right after the date's package statement and still didn't work.

Any ideas?

How to use a Generic Map type?

Hi,

Based on the official capnproto doc about generic types (https://capnproto.org/language.html#generic-types), I have copied that generic map and would like to use it, but so far no success...

@0xb73b820feb20d1a6;

using Go = import "/go.capnp";
$Go.package("x");
$Go.import("xl");

struct Map(Key, Value) {
    entries @0 :List(Entry);
    struct Entry {
        key @0 :Key;
        value @1 :Value;
    }
}

@0x8b18fa36c203198d;

using Go = import "/go.capnp";
$Go.package("xyz");
$Go.import("xyz");

using import "Map.capnp".Map;  #This is the Map schema defined above 

struct MyMsg {
    using MyMap = Map(Text, Data); # Setting the type of key to Text, and type of value to Data

    myMap @0 :MyMap;
}

The the following code is generated for the Map.capnp schema:

package x

// AUTO GENERATED - DO NOT EDIT

import (
    capnp "zombiezen.com/go/capnproto2"
)

type Map struct{ capnp.Struct }

func NewMap(s *capnp.Segment) (Map, error) {
    st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
    if err != nil {
        return Map{}, err
    }
    return Map{st}, nil
}

func NewRootMap(s *capnp.Segment) (Map, error) {
    st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
    if err != nil {
        return Map{}, err
    }
    return Map{st}, nil
}

func ReadRootMap(msg *capnp.Message) (Map, error) {
    root, err := msg.RootPtr()
    if err != nil {
        return Map{}, err
    }
    return Map{root.Struct()}, nil
}

func (s Map) Entries() (Map_Entry_List, error) {
    p, err := s.Struct.Ptr(0)
    if err != nil {
        return Map_Entry_List{}, err
    }

    return Map_Entry_List{List: p.List()}, nil

}

func (s Map) HasEntries() bool {
    p, err := s.Struct.Ptr(0)
    return p.IsValid() || err != nil
}

func (s Map) SetEntries(v Map_Entry_List) error {

    return s.Struct.SetPtr(0, v.List.ToPtr())
}

// Map_List is a list of Map.
type Map_List struct{ capnp.List }

// NewMap creates a new list of Map.
func NewMap_List(s *capnp.Segment, sz int32) (Map_List, error) {
    l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)
    if err != nil {
        return Map_List{}, err
    }
    return Map_List{l}, nil
}

func (s Map_List) At(i int) Map           { return Map{s.List.Struct(i)} }
func (s Map_List) Set(i int, v Map) error { return s.List.SetStruct(i, v.Struct) }

// Map_Promise is a wrapper for a Map promised by a client call.
type Map_Promise struct{ *capnp.Pipeline }

func (p Map_Promise) Struct() (Map, error) {
    s, err := p.Pipeline.Struct()
    return Map{s}, err
}

type Map_Entry struct{ capnp.Struct }

func NewMap_Entry(s *capnp.Segment) (Map_Entry, error) {
    st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
    if err != nil {
        return Map_Entry{}, err
    }
    return Map_Entry{st}, nil
}

func NewRootMap_Entry(s *capnp.Segment) (Map_Entry, error) {
    st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
    if err != nil {
        return Map_Entry{}, err
    }
    return Map_Entry{st}, nil
}

func ReadRootMap_Entry(msg *capnp.Message) (Map_Entry, error) {
    root, err := msg.RootPtr()
    if err != nil {
        return Map_Entry{}, err
    }
    return Map_Entry{root.Struct()}, nil
}

func (s Map_Entry) Key() (capnp.Pointer, error) {

    return s.Struct.Pointer(0)

}

func (s Map_Entry) HasKey() bool {
    p, err := s.Struct.Ptr(0)
    return p.IsValid() || err != nil
}

func (s Map_Entry) KeyPtr() (capnp.Ptr, error) {

    return s.Struct.Ptr(0)

}

func (s Map_Entry) SetKey(v capnp.Pointer) error {

    return s.Struct.SetPointer(0, v)
}

func (s Map_Entry) SetKeyPtr(v capnp.Ptr) error {

    return s.Struct.SetPtr(0, v)
}

func (s Map_Entry) Value() (capnp.Pointer, error) {

    return s.Struct.Pointer(1)

}

func (s Map_Entry) HasValue() bool {
    p, err := s.Struct.Ptr(1)
    return p.IsValid() || err != nil
}

func (s Map_Entry) ValuePtr() (capnp.Ptr, error) {

    return s.Struct.Ptr(1)

}

func (s Map_Entry) SetValue(v capnp.Pointer) error {

    return s.Struct.SetPointer(1, v)
}

func (s Map_Entry) SetValuePtr(v capnp.Ptr) error {

    return s.Struct.SetPtr(1, v)
}

// Map_Entry_List is a list of Map_Entry.
type Map_Entry_List struct{ capnp.List }

// NewMap_Entry creates a new list of Map_Entry.
func NewMap_Entry_List(s *capnp.Segment, sz int32) (Map_Entry_List, error) {
    l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2}, sz)
    if err != nil {
        return Map_Entry_List{}, err
    }
    return Map_Entry_List{l}, nil
}

func (s Map_Entry_List) At(i int) Map_Entry           { return Map_Entry{s.List.Struct(i)} }
func (s Map_Entry_List) Set(i int, v Map_Entry) error { return s.List.SetStruct(i, v.Struct) }

// Map_Entry_Promise is a wrapper for a Map_Entry promised by a client call.
type Map_Entry_Promise struct{ *capnp.Pipeline }

func (p Map_Entry_Promise) Struct() (Map_Entry, error) {
    s, err := p.Pipeline.Struct()
    return Map_Entry{s}, err
}

func (p Map_Entry_Promise) Key() *capnp.Pipeline {
    return p.Pipeline.GetPipeline(0)
}

func (p Map_Entry_Promise) Value() *capnp.Pipeline {
    return p.Pipeline.GetPipeline(1)
}

I am unable to figure out how I should create a map with 1 dummy data. This is where I am, it almost works, but I do not know how to set the key and the value of my entry:

testMap, _ := x.NewMap(segment)
mapEntryList, _ := x.NewMap_Entry_List(segment, 1)  # list with 1 element
mapEntry, _ := x.NewMap_Entry(segment)

mapEntry.SetKeyPtr("key1")    # How to add key1 string as Key?
mapEntry.SetValuePtr([]byte{1,2,3})   # How to add byte slice as Value?

mapEntryList.Set(0, mapEntry)
testMap.SetEntries(mapEntryList)

So the MapEntry has SetKey/Value and SetKeyPtr/ValuePtr functions, which accepts capnp.Pointer and capnp.Ptr respectively. How can I turn my string and byte slice to one of these types? And which function pair should I use?

how to return a list of interfaces

Hi,

I'm still struggling with capnp concepts, so I maybe what I'm trying to do does not even make sense, but here is my question. I have the following schema:

using Go = import "../../../../zombiezen.com/go/capnproto2/go.capnp";
$Go.package("capnp");
$Go.import("github.com/little-dude/tgen/capnp");
@0xef97cf4069588836;

interface Controller {
    getPorts @0 () -> (ports :List(Port));
}

interface Port {
    getName      @0  () -> (name :Text);
    # [ other methods ... ]
}

So I want to return a list of interfaces with getPorts(). From the quickstart and the doc, I came to this code:

package main

import (
    capnp "github.com/little-dude/tgen/capnp"
    "net"
    "zombiezen.com/go/capnproto2/rpc"
)

type port struct {
    name string
}

type controller struct {
    ports []port
}

func (c controller) GetPorts(call capnp.Controller_getPorts) error {
    // retrieve the list of the port on the host
    itfs, e := net.Interfaces()
    // store the ports
    ports := make([]port, 0)
    for _, itf := range itfs {
        ports = append(ports, port{name: itf.Name})
    }
    // return the ports
    portsList, e := call.Results.Ports()
    for i, p := range c.ports {
        portsList.SetPtr(i, capnp.Port_ServerToClient(p).ToPtr())
    }
    return nil
}

func (p port) GetName(call capnp.Port_getName) error {
    return call.Results.SetName(p.name)
}

func main() {
    l, _ := net.Listen("tcp", ":1234")
    main := capnp.Controller_ServerToClient(controller{})
    conn := rpc.NewConn(rpc.StreamTransport(l), rpc.MainInterface(main.Client))
    err := conn.Wait()
}

But it fails with this:

./main.go:28: "github.com/little-dude/tgen/capnp".Port_ServerToClient(p).ToPtr undefined (type "github.com/little-dude/tgen/capnp".Port has no field or method ToPtr)

I cannot figure out how to populate the call.Result.Ports() list, despite many different attempts. Could you give me some pointers, please?

Thinking: inference of import paths and package names

I don't have a good solution for this yet, but it has been suggested to me that capnpc-go can/should be inferring the import path and package name for the generated code. This can certainly be introduced in a backward-compatible way, since we currently require import paths and package names to be explicitly annotated, so this is just relaxing the rule. Note that solving this would make #41 obsolete.

Experience has shown me that finding the appropriate Go package boundaries is challenging. Finding a good solution should address the following three use-cases:

You have a bunch of schemas in one directory, but they're probably separate packages. For example, the schemas included in capnproto (e.g. schema.capnp, persistent.capnp, rpc.capnp). If foo.capnp is located at $root/foo.capnp, the generated Go file should be placed at $root/foo/foo.capnp.go.
You have a bunch of schemas in one directory, but they're tightly coupled and should be one package. If foo.capnp is located at $root/foo.capnp, the generated Go file should be placed at $root/foo.capnp.go.
You are writing a Go project from scratch and so you place one schema per directory. If foo.capnp is located at $root/foo.capnp, the generated Go file should be placed at $root/foo.capnp.go. Note that this is a strict subset of the second use-case.

Import failed: zombiezen.com/go/capnproto2/std/go.capnp

As mentioned in issue #36 . I changed import path to: zombiezen.com/go/capnproto2/std/go.capnp

capnp compile -ogo message.capnp

message.capnp:1:19-61: error: Import failed: zombiezen.com/go/capnproto2/std/go.capnp
message.capnp:3:2-4: error: Not defined: Go
message.capnp:4:2-4: error: Not defined: Go

and when i go to this dir:

zombiezen.com/go/capnproto2/internal/demo/books

and run: capnp compile -ogo books.capnp
I get this:

books.capnp:1:19-30: error: Import failed: /go.capnp
books.capnp:3:2-4: error: Not defined: Go
books.capnp:4:2-4: error: Not defined: Go

BTW: I'm using capnp v0.5.3

Invalid memory address or nil pointer: mem.go

My go-capnproto2 package is up-to-date. When I run a small example with capnproto it gets an error. All code can be found here.

The error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x84ab4]

goroutine 1 [running]:
panic(0x106138, 0x1040a010)
        /home/mjohn/software/go/src/runtime/panic.go:500 +0x33c
zombiezen.com/go/capnproto2.alloc(0x0, 0x18, 0x0, 0x0, 0x0, 0x0)
        /home/mjohn/workspaces/goprojects/src/zombiezen.com/go/capnproto2/mem.go:264 +0x7c
zombiezen.com/go/capnproto2.NewStruct(0x0, 0x0, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/mjohn/workspaces/goprojects/src/zombiezen.com/go/capnproto2/struct.go:18 +0x128
crown/fram/imm.NewVehicle(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/mjohn/workspaces/goprojects/src/crown/fram/imm/schema.capnp.go:581 +0x5c
crown/fram/imm.Fram.NewVehicle(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/mjohn/workspaces/goprojects/src/crown/fram/imm/schema.capnp.go:1050 +0x7c
main.main()
        /home/mjohn/workspaces/goprojects/src/crown/fram/imm/utils/wr.go:25 +0x29c

No "String" method generated for structs

There is no "String" method generated for a given struct so it cannot easily be printed. This would be useful for debugging/logging. The C++ version generates a toString().

Document which RPC features are implemented

Can you please document which level of the cap'n proto API this library implements? Thanks!

check for oid comparison underflow

fyi: glycerine/go-capnproto#34

Make RPC docs more discoverable

From @zombiezen on August 24, 2015 23:15

Based on this thread, the overall docs for the RPC package are not easy to find. It may be worth making a GitHub page that breaks out the "hello world" examples.

Copied from original issue: zombiezen/go-capnproto#8

Setting field type as another struct results in import error

When I have this simple schema

# schema/example.capnp
@0xb42bbb69f69a82f8;

using Go = import "../vendor/zombiezen.com/go/capnproto2/std/go.capnp";
$Go.package("main");


struct A {
    b @0 :B;
}
struct B {}

And then run capnp compile -ogo example.capnp I get error:

capnpc-go: generating example.capnp: field A.b: internal error (bad schema?): missing import declaration for example.capnp:B
go: plugin failed: exit code 1

Is something wrong with my schema or is it a bug?

error when go get

go get -u -t zombiezen.com/go/capnproto2/...
go build zombiezen.com/go/capnproto2/internal/demo: no buildable Go source files in /home/vtolstov/go/src/zombiezen.com/go/capnproto2/internal/demo

capnpc-go: deal with subtyping/inheritance in capnp interfaces

So, I just got a basic program running that responds to a GET / via the normal net/http package API, but talking to the sandstorm api server. woo! but it involved this hack:

zenhack/go.sandstorm@1d5b98e

In the capnp schema, WebSession inherits from UiSession, but the generated go packages don't reflect this in a useful way.

Don't import capnproto as C in generated Code

From @Merovius on August 15, 2015 21:36

It is very confusing, having the documentation in godoc read something like
type Foo C.Struct and trying to find out for several minutes, if this uses cgo and thus imports C. A different mnemonic ("c" by example, if you want one letter) would be better, but even better would be to just not alias it (and import it as capnproto accordingly). This would make the documentation of generated code clearer (and agree with the documentation which uses in every example the package name capnproto). It's generated code anyway, brevity is not required.

Copied from original issue: zombiezen/go-capnproto#6

Generated FooBytes method for string fields does not handle defaults

The new FooBytes method will always return nil in case of string fields with defaults.

rpc: add function to obtain local server from a capnp.Client

In order to enable some APIs, there should be a function that goes from capnp.Client to a local server. This is similar to CapabilityServerSet::getLocalServer in the C++ API. However, just returning the *server.server isn't sufficient because a) it is an unexported type and b) does not contain a direct reference to the value from the _ServerToClient function. I propose that the *server.server type should grow an interface{} field that can be anything, but is prepopulated by the generated _ServerToClient function to the passed-in server value.

Rough implementation plan:

Export server.server type and change server.New to return *server.Server. Slightly breaking, but since the primary caller is generated code and the return type still obeys the capnp.Client interface, I think it's okay.
Add a Value interface{} field to *server.Server.
Update capnpc-go to populate Server.Value inside _ServerToClient functions.
Add a new interface to the capnp package (and make the appropriate clients implement it):

// ClientPromiser is an optional interface that a Client can implement if it can later resolve to another client.
type ClientPromiser interface {
  // ClientPromise returns a channel that is closed when the client is resolved.
  // The returned channel will be nil if this client is a real client, not a promise.
  ClientPromise() <- chan struct{}

  // ResolvedClient returns the resolved client, or nil if the client has not been resolved or the client is not a promise.
  ResolvedClient() capnp.Client
}

// ResolveClient waits until a client is fully resolved and returns it.
func ResolveClient(c capnp.Client) capnp.Client

Add a new function to server package:

// Resolve waits until a client is fully resolved, determines whether it is a Server, and if so, returns its Value field.  Otherwise, it returns nil.
func Resolve(c capnp.Client) interface{}

There's some hand-wavy details here, since either Resolve may need to cut through refcounts, but I think this is what I'd like to see happen.

/cc @zenhack for thoughts, since you're using the rpc package.

Plain Ol' Go Structs

Similar to the support planned for C++: https://groups.google.com/d/topic/capnproto/Reg0wInHBdY/discussion

Design sketch:

package pogs // import "zombiezen.com/go/capnproto2/pogs"

// GeneratedStruct is the interface implemented by a struct generated by capnpc-go.
type GeneratedStruct interface {
  GeneratedStruct() (id uint64, schemaData []byte, s capnp.Struct)
}

// Unmarshal copies from a Cap'n Proto struct to a plain Go struct.
func Unmarshal(val interface{}, s GeneratedStruct) error {
  // ...
}

// Marshal copies from a plain Go struct into a Cap'n Proto struct.
func Marshal(s GeneratedStruct, val interface{}) error {
  // ...
}

Usage:

// Assuming you have a capnp schema foo.capnp:
// struct Foo {
//   bar @0 :UInt64;
// }

type Foo struct {
    Bar uint64
}

func main() {
    msg, err := capnp.Unmarshal(myByteSource)
    if err != nil {
        panic(err)
    }
    foo, err := fooschema.ReadRootFoo(msg)
    if err != nil {
        panic(err)
    }
    // New API:
    var fooStruct Foo
    if err := pogs.Unmarshal(&fooStruct, foo); err != nil {
        panic(err)
    }
    fmt.Println(fooStruct.Bar)

    // And new API for writing back out:
    if err := pogs.Marshal(foo, &fooStruct); err != nil {
        panic(err)
    }
        fmt.Print(msg.Marshal())
}

capnpc-go could be optionally extended to generate the structs automatically, but much like the json package, users could also define narrow Go structs if they only need certain fields.

cc @lvdlvd

Encode/Decode vs. Marshal/Unmarshal

Hello,

Why or when would I want to use one over the other?

Thanks.

rpc.Conn doesn't support promise capabilities

From @zombiezen on August 6, 2015 21:1

The CapDescriptor struct has a senderPromise field which rpc.Conn does not support, along with the corresponding Resolve message. The Go implementation does not emit this descriptor type, so it is only a concern if interoperating with another implementation that does.

Copied from original issue: zombiezen/go-capnproto#2

Vanity url HTML is mildly malformed

$ curl https://zombiezen.com/go/capnproto2
<meta name="go-import" content="zombiezen.com/go/capnproto2 git https://github.com/zombiezen/go-capnproto2">
<h1>zombiezen.com/go/capnproto2</h1>
<a href="https://github.com/zombiezen/go-capnproto2">Git Repository on GitHub</a>

The meta tag should self-close. Also, it'd be nice to have <!DOCTYPE html> and put the meta tags in a <head>, like the canonical example:

$ curl rsc.io/pdf
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<meta name="go-import" content="rsc.io/pdf git https://github.com/rsc/pdf">
<meta http-equiv="refresh" content="0; url=https://godoc.org/rsc.io/pdf">
</head>
<body>
Nothing to see here; <a href="https://godoc.org/rsc.io/pdf">move along</a>.
</body>
</html>

Obviously this is pretty minor, but for some reason it's breaking gvt.

Has* methods on messages ambiguous for union types.

Just noticed this. If you have a union type in a struct and you check msg.Has* for the different union types, they'll both return true if only one is set.

From the generated code:

func (s Message) HasLog() bool {
    p, err := s.Struct.Ptr(1)
    return p.IsValid() || err != nil
}

func (s Message) HasRequest() bool {
    p, err := s.Struct.Ptr(1)
    return p.IsValid() || err != nil
}

Vanity server broken

I'm unable to install the package:

go get -u -v zombiezen.com/go/capnproto2/...
Fetching https://zombiezen.com/go/capnproto2?go-get=1
ignoring https fetch with status code 500
Parsing meta tags from https://zombiezen.com/go/capnproto2?go-get=1 (status code 500)
import "zombiezen.com/go/capnproto2/...": parsing zombiezen.com/go/capnproto2: http: read on closed response body
package zombiezen.com/go/capnproto2/...: unrecognized import path "zombiezen.com/go/capnproto2/..."

Add benchmarks for string field as bytes access

Follow-on to #18: port over benchmarks from https://github.com/glycerine/go-capnproto/blob/master/stringbytes_test.go.

Allow nil to be used for param functions

From @zombiezen on August 26, 2015 0:19

Quite often, it's useful to have interface methods that have no parameters. It's somewhat unruly to write out:

promise := obj.NiladicFunc(ctx, func(p schema.Object_niladicFunc_Params) error {
  return nil
})

instead of the more concise:

promise := obj.NiladicFunc(ctx, nil)

Copied from original issue: zombiezen/go-capnproto#9

rpc: panic during addExport

Hey there, just came across this possible bug, any idea what its about?

panic: runtime error: index out of range
goroutine 358 [running]:
panic(0xa4efc0, 0xc82000a150)
        /nix/store/jvsx0g86is0ryjxdrsql5q7nv349jnf1-go-1.6.3/share/go/src/runtime/panic.go:481 +0x3e6
zombiezen.com/go/capnproto2/rpc.(*Conn).addExport(0xc8202a41a0, 0x7f4327c1fb00, 0xc82009a060, 0x0)
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/tables.go:175 +0x3df
zombiezen.com/go/capnproto2/rpc.(*Conn).descriptorForClient(0xc8202a41a0, 0xc8201d4060, 0x800000050, 0xffffffffffff0001, 0xfffffffffffffffe, 0x1, 0x7f4327c1f9e8, 0xc8200c0800, 0x0, 0x0)
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/introspect.go:202 +0x156
zombiezen.com/go/capnproto2/rpc.(*Conn).makeCapTable(0xc8202a41a0, 0xc8201d4060, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/rpc.go:576 +0x3e2
zombiezen.com/go/capnproto2/rpc.(*answer).fulfill(0xc820076a50, 0xc8202ac060, 0x8, 0x100000000, 0xffffffffffffffff, 0x0, 0x0, 0x0)
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/answer.go:76 +0x366
zombiezen.com/go/capnproto2/rpc.joinAnswer(0xc820076a50, 0x7f4327d8b9e8, 0xc820362778)
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/answer.go:225 +0x30b
created by zombiezen.com/go/capnproto2/rpc.(*Conn).routeCallMessage
        /home/fader/golang/src/zombiezen.com/go/capnproto2/rpc/rpc.go:698 +0x69f

Thx for the lib by the way awesome work.

Stop tests from writing to package directory

From @zombiezen on August 6, 2015 21:43

Temporary directories should be used instead.

Copied from original issue: zombiezen/go-capnproto#5

pogs: fixed unions are hard to understand

Not sure if this is a bug in the code or docs, but I found an issue with pogs deserialization.

I made up an example of all the passing / failing cases here (each commit has different cases):

https://github.com/abraithwaite/pogs_example/commits/master

Specifically, when including an embedded struct which is represented by a union in capnproto, the docs say to use the "which=" tag to specify the capnp object to use. This doesn't work for me. Looking at the code, it appears that codepath exists still on HEAD:

https://github.com/zombiezen/go-capnproto2/blob/e255c9f719103c5329bc65aac0adcf12137b7424/pogs/fields.go#L174-L223

What's the expected behavior? I'm happy to submit a patch.

I think the issue might actually be somewhere here:

https://github.com/zombiezen/go-capnproto2/blob/e255c9f719103c5329bc65aac0adcf12137b7424/pogs/fields.go#L30-L46

Add generated code for alloc-and-set list fields

From @zombiezen on August 24, 2015 22:44

Just like the NewField struct methods, there should be corresponding NewField list methods. Not as straightforward, since the list types might be part of the capnp package.

Copied from original issue: zombiezen/go-capnproto#7

Sending HTTP traffic to an RPC port panics

(h/t to @bcspragu for discovering)

If you send an HTTP GET onto an RPC port, the decoder panics when trying to allocate a large header. This is much more likely to panic on GOARCH=386 or other platforms where int is 32 bits wide.

Decode should have user-configurable limits on how much it reads from the stream, but we should also prevent a high number of segments, since even the header can overflow.

Make capnpc-go fail files earlier if they don't have an import annotation

A number of people are confused by the labeling of lacking an import annotation an "internal error". This error occurs lazily when the import disambiguation is required. Perhaps it should fail earlier in the process.

(Issues #37 and #40 were the user issues.)

Remove goconvey dependency

From @zombiezen on August 6, 2015 21:43

goconvey is noisy and depends on goroutine-local storage (ew). I'd rather have the tests using the vanilla testing package.

Copied from original issue: zombiezen/go-capnproto#4

zombiezen.com checkout path doesn't work

zombiezen.com has been broken for a few days, breaking checking out the go-capnproto2 project. E.g.:

Fetching https://zombiezen.com/go/capnproto2?go-get=1
https fetch failed: Get https://zombiezen.com/go/capnproto2?go-get=1: EOF
package zombiezen.com/go/capnproto2: unrecognized import path "zombiezen.com/go/capnproto2" (https fetch: Get https://zombiezen.com/go/capnproto2?go-get=1: EOF)

terrible performance of SetXXX methods

My capnp file is as follow:

struct Request {
    sourceID @0         :Text;
    reportID @1         :Text;
    reportVersion @2    :Int32;
    klen @3             :Int32;
    key @4              :Text;
    data @5             :Data;
}

struct Requests {
    id @0       :Text;
    count @1    :UInt32;
    reqs @2      :List(Request);
}

I have a Batcher struct that batches multiple Request into one Requests.

type Batcher struct {
    // ... some fields
    // capnp structs
    msg     *capnp.Message
    seg     *capnp.Segment
    reqList capnpmsg.Request_List

    // ... some other fields
}

Now we initialize the Batcher with this method

func (this *Batcher) prepare() (err error) {
    if this.msg, this.seg, err = capnp.NewMessage(capnp.SingleSegment(nil)); err != nil {
        return err
    }
    if r, err := capnpmsg.NewRootRequests(this.seg); err != nil {
        return err
    } else {
        if this.reqList, err = r.NewReqs(int32(this.capacity)); err != nil {
            return err
        }
        // some Ops...
    }
    return err
}

Now start batching Request

func (this *Batcher) Batch(sid string, rid string, rv int, klen int, key string, types []string, values []interface{}) error {
    // some code....
    req, err := capnpmsg.NewRequest(this.seg)
    if err != nil {
        return err
    }
    req.SetSourceID(sid)
    req.SetReportID(rid)
    req.SetReportVersion(int32(rv))
    req.SetKlen(int32(klen))
    req.SetKey(key)
    req.SetData(buff)
    if err := this.reqList.Set(int(this.curSize), req); err != nil {
        return err
    }
    // some code...
    return nil
}

the profiling result is as follow:

As you can see, [SetSourceID SetReportID SetKey SetData] take about 70% of the Batch's running time. Am i not using the api right? How to improve the performance? Thanks!!

Reduce code duplication in queued clients

From @zombiezen on August 6, 2015 20:59

The queued clients in fulfiller.go, rpc/answer.go, and rpc/question.go duplicate a lot of code.

The only way I can think to avoid this is extract the server type to a separate package, which changes generated code and is API-breaking.

Copied from original issue: zombiezen/go-capnproto#1

No "HasFoo" method generated for pointer fields

The C++ implementation of Cap'n Proto exposes null pointers with a hasFoo accessor. capnpc-go should provide a similar method.

Code size of generated packages is excessive.

For the sandstorm schema, on linux amd64 grain.a is 19MiB (!). Many of the others are rather large as well. As a point of comparision, the largest package in the go standard library is net/http, the .a for which is ~3.7MiB.

One thing I have noticed is that the struct wrappers and accessor functions add up. I did some mesurments a while back and never got around to filing the bug. IIRC, each of these cost us one or two hundred KiB:

https://github.com/zenhack/go.sandstorm/blob/master/capnp/grain/grain.capnp.go#L15

Those got noticably smaller when I took out the if statement (it's pretty easy to imagine a compiler inlining it into a noop in that case, though I don't think it actually does). Not enough to solve the problem on it's own though.

I started to play with killing the struct wrappers and making types direct aliases of the field they're now wrapping, which IIRC would allow more functions to be effectively noops, but ran into issues with copying non-exported fields, which I didn't spend much time fussing with.

I'd like to visualize the sizes of various functions to get a sense of where the space is being taken up; probably more efficient than the futzing with schema and modifications to templates I was doing. Would need to research tooling for that; haven't dug into it much yet.

dispatch goroutine leak

I'm following through the getting started for rpc, and I have managed to work out most of my questions, but I have one that I can't quite work out.

Say I have some schema like

interface Database { # the main interface
  dolargequery @0 () -> (cursor :Cursor);
} 
interface Cursor {
  next @0 () -> ( data :Data );
}

And say the client calls dolargequery, the server creates some go struct that, for arguments sake, occupies a bunch of memory and implements Cursor. The server sends the cursor capability back to the client.

Now my question is how do I idiomatically control the lifetime of the cursor struct? Presumably the server is holding on to a reference to it so that when it receives calls on the capability it can execute them, but how does the client "free" the capability so that the underlying struct I passed to Cursor_ServerToClient can be freed?

I did some experiments with printing in finalizers (which is always a bit shady) but it doesn't seem that the structure I pass to Cursor_ServerToClient becomes unreferenced even after the rpc connection's Wait() returns.

How does it all work?

Import path for go.capnp is wrong in godoc

The instructions for generating code give the following code samples:

# First, install capnpc-go to $PATH.
go install zombiezen.com/go/capnproto2/capnpc-go
# Then, generate Go files.
capnp compile -ogo *.capnp

using Go = import "zombiezen.com/go/capnproto2/go.capnp";
$Go.package("main");
$Go.import("zombiezen.com/go/capnproto2/example");

The former should have a -Istd flag, and the latter should be updated to import "/go.capnp". I should also check the wiki for similar issues.

/cc @bcspragu

Debug flaky release test

From @zombiezen on September 5, 2015 23:22

A recent Travis build uncovered a bug in release_test. Logs seem to indicate an issue with double-Closing a client.

Copied from original issue: zombiezen/go-capnproto#10

pogs subdir tests failing

pogs seems pretty awesome. Very similar to what I ended up doing in practice anyway: I always ended up defining a plain Go struct that mirrored the capnp struct (and thus motivated the github.com/glycerine/bambam mini-project for the reverse direction). So I'd love to see go-capnproto2's pogs in action.

Strangely the tests seem to fail for me, using go1.7 on osx 10.11.6 El Capitan.

Certainly could be user error on my part...I didn't try to troubleshoot much here... please advise.

jaten@jatens-MacBook-Pro ~/src/zombiezen.com/go/capnproto2/pogs (master) $ go test -v
go test -v
=== RUN   TestExtract_Embed
--- FAIL: TestExtract_Embed (0.00s)
    embed_test.go:40: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:43: Extract produced {}; want {VerVal:{Val:123}}
=== RUN   TestExtract_EmbedPtr
--- FAIL: TestExtract_EmbedPtr (0.00s)
    embed_test.go:60: Extract error: pogs: extract @0xf705dc45c94766fd: unexpected EOF
    embed_test.go:63: Extract produced {}; want {VerVal:{Val:123},Duo:456}
=== RUN   TestExtract_EmbedOmit
--- FAIL: TestExtract_EmbedOmit (0.00s)
    embed_test.go:80: Extract error: pogs: extract @0xf705dc45c94766fd: unexpected EOF
    embed_test.go:83: Extract produced {}; want {Duo:456}
=== RUN   TestExtract_EmbedName
--- FAIL: TestExtract_EmbedName (0.00s)
    embed_test.go:108: Extract error: pogs: extract @0xe1c9eac512335361: unexpected EOF
    embed_test.go:111: Extract produced {}; want {PlaneBase:{Name:"ALL YOUR BASE",Rating:5,CanFly:true}}
=== RUN   TestInsert_Embed
--- FAIL: TestInsert_Embed (0.00s)
    embed_test.go:127: Insert({VerVal:{Val:123}}) error: pogs: insert @0xfca3742893be4cde: unexpected EOF
    embed_test.go:130: Insert({VerVal:{Val:123}}) produced 
=== RUN   TestInsert_EmbedPtr
--- FAIL: TestInsert_EmbedPtr (0.00s)
    embed_test.go:146: Insert({VerVal:{Val:123},Duo:456}) error: pogs: insert @0xf705dc45c94766fd: unexpected EOF
    embed_test.go:149: Insert({VerVal:{Val:123},Duo:456}) produced 
=== RUN   TestInsert_EmbedNilPtr
--- FAIL: TestInsert_EmbedNilPtr (0.00s)
    embed_test.go:165: Insert({Duo:456}) error: pogs: insert @0xf705dc45c94766fd: unexpected EOF
    embed_test.go:168: Insert({Duo:456}) produced 
=== RUN   TestInsert_EmbedOmit
--- FAIL: TestInsert_EmbedOmit (0.00s)
    embed_test.go:184: Insert({VerVal:{Val:123},Duo:456}) error: pogs: insert @0xf705dc45c94766fd: unexpected EOF
    embed_test.go:187: Insert({VerVal:{Val:123},Duo:456}) produced 
=== RUN   TestInsert_EmbedNamed
--- FAIL: TestInsert_EmbedNamed (0.00s)
    embed_test.go:203: Insert({PlaneBase:{Name:"ALL YOUR BASE",Rating:5,CanFly:true}}) error: pogs: insert @0xe1c9eac512335361: unexpected EOF
    embed_test.go:214: Insert({PlaneBase:{Name:"ALL YOUR BASE",Rating:5,CanFly:true}}) produced 
=== RUN   TestExtract_EmbedCollide
--- FAIL: TestExtract_EmbedCollide (0.00s)
    embed_test.go:303: top: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:306: top: Extract produced {}; want {Val:123}
    embed_test.go:303: no tags: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:303: 1 tag: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:306: 1 tag: Extract produced {}; want {VerValTag1:{Val:123}}
    embed_test.go:303: 1 tag with untagged: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:306: 1 tag with untagged: Extract produced {}; want {VerValTag1:{Val:123}}
    embed_test.go:303: 2 tags: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:303: 3 tags: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:303: top with lower collision: Extract error: pogs: extract @0xfca3742893be4cde: unexpected EOF
    embed_test.go:306: top with lower collision: Extract produced {}; want {Val:123}
=== RUN   TestInsert_EmbedCollide
--- FAIL: TestInsert_EmbedCollide (0.00s)
    embed_test.go:354: top: Insert(..., {Val:123,VerVal:{Val:456}}): %!v(MISSING)
    embed_test.go:357: top: Insert(..., {Val:123,VerVal:{Val:456}}) produced ; want (val = 123)
    embed_test.go:354: no tags: Insert(..., {VerVal:{Val:123},VerValNoTag:{Val:456}}): %!v(MISSING)
    embed_test.go:354: 1 tag: Insert(..., {VerVal:{Val:123},VerValTag1:{Val:456}}): %!v(MISSING)
    embed_test.go:357: 1 tag: Insert(..., {VerVal:{Val:123},VerValTag1:{Val:456}}) produced ; want (val = 456)
    embed_test.go:354: 1 tag with untagged: Insert(..., {VerVal:{Val:123},VerValTag1:{Val:456},VerValNoTag:{Val:789}}): %!v(MISSING)
    embed_test.go:357: 1 tag with untagged: Insert(..., {VerVal:{Val:123},VerValTag1:{Val:456},VerValNoTag:{Val:789}}) produced ; want (val = 456)
    embed_test.go:354: 2 tags: Insert(..., {VerValTag1:{Val:123},VerValTag2:{Val:456}}): %!v(MISSING)
    embed_test.go:354: 3 tags: Insert(..., {VerValTag1:{Val:123},VerValTag2:{Val:456},VerValTag3:{Val:789}}): %!v(MISSING)
    embed_test.go:354: top with lower collision: Insert(..., {Val:123,VerVal:{Val:456},VerValNoTag:{Val:789}}): %!v(MISSING)
    embed_test.go:357: top with lower collision: Insert(..., {Val:123,VerVal:{Val:456},VerValNoTag:{Val:789}}) produced ; want (val = 123)
=== RUN   TestExtract
--- FAIL: TestExtract (0.00s)
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:2,F64:3.5}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:3,F32:3.5}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:4,I64:-123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:5,I32:-123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:6,I16:-123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:7,I8:-123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:8,U64:123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:9,U32:123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:10,U16:123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:11,U8:123}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:12,Bool:true}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:12}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:13,Text:"Hello, World!"}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:14,Blob:[72,101,108,108,111,44,32,87,111,114,108,100,33]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:15}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:15,F64vec:[-2,4.5]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:16}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:16,F32vec:[-2,4.5]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:17}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:17,I64vec:[-123,0,123]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:20}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:20,I8vec:[-123,0,123]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:21}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:21,U64vec:[0,123]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:24}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:24,U8vec:[0,123]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:39}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:39,Boolvec:[false,true,false]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:40}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:40,Datavec:[[104,105],[98,121,101]]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:41}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:41,Textvec:["John","Paul","George","Ringo"]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:25,Zvec:[{Which:4,I64:-123},{Which:13,Text:"Hi"}]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:26,Zvecvec:[[{Which:4,I64:1},{Which:4,I64:2}],[{Which:4,I64:3},{Which:4,I64:4}]]}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:32}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:32,Planebase:{Name:"Boeing",Homes:[2,5],Rating:123,CanFly:true,Capacity:100,MaxSpeed:9001}}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:33,Airport:2}
    pogs_test.go:148: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:151: Extract() produced {}; want {Which:42,Grp:{First:123,Second:456}}
=== RUN   TestInsert
--- FAIL: TestInsert (0.00s)
    pogs_test.go:170: Insert({Which:2,F64:3.5}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:2,F64:3.5}) produced 
    pogs_test.go:170: Insert({Which:3,F32:3.5}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:3,F32:3.5}) produced 
    pogs_test.go:170: Insert({Which:4,I64:-123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:4,I64:-123}) produced 
    pogs_test.go:170: Insert({Which:5,I32:-123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:5,I32:-123}) produced 
    pogs_test.go:170: Insert({Which:6,I16:-123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:6,I16:-123}) produced 
    pogs_test.go:170: Insert({Which:7,I8:-123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:7,I8:-123}) produced 
    pogs_test.go:170: Insert({Which:8,U64:123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:8,U64:123}) produced 
    pogs_test.go:170: Insert({Which:9,U32:123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:9,U32:123}) produced 
    pogs_test.go:170: Insert({Which:10,U16:123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:10,U16:123}) produced 
    pogs_test.go:170: Insert({Which:11,U8:123}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:11,U8:123}) produced 
    pogs_test.go:170: Insert({Which:12,Bool:true}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:12,Bool:true}) produced 
    pogs_test.go:170: Insert({Which:12}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:12}) produced 
    pogs_test.go:170: Insert({Which:13,Text:"Hello, World!"}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:13,Text:"Hello, World!"}) produced 
    pogs_test.go:170: Insert({Which:14,Blob:[72,101,108,108,111,44,32,87,111,114,108,100,33]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:14,Blob:[72,101,108,108,111,44,32,87,111,114,108,100,33]}) produced 
    pogs_test.go:170: Insert({Which:15}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:15}) produced 
    pogs_test.go:170: Insert({Which:15,F64vec:[-2,4.5]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:15,F64vec:[-2,4.5]}) produced 
    pogs_test.go:170: Insert({Which:16}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:16}) produced 
    pogs_test.go:170: Insert({Which:16,F32vec:[-2,4.5]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:16,F32vec:[-2,4.5]}) produced 
    pogs_test.go:170: Insert({Which:17}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:17}) produced 
    pogs_test.go:170: Insert({Which:17,I64vec:[-123,0,123]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:17,I64vec:[-123,0,123]}) produced 
    pogs_test.go:170: Insert({Which:20}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:20}) produced 
    pogs_test.go:170: Insert({Which:20,I8vec:[-123,0,123]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:20,I8vec:[-123,0,123]}) produced 
    pogs_test.go:170: Insert({Which:21}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:21}) produced 
    pogs_test.go:170: Insert({Which:21,U64vec:[0,123]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:21,U64vec:[0,123]}) produced 
    pogs_test.go:170: Insert({Which:24}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:24}) produced 
    pogs_test.go:170: Insert({Which:24,U8vec:[0,123]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:24,U8vec:[0,123]}) produced 
    pogs_test.go:170: Insert({Which:39}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:39}) produced 
    pogs_test.go:170: Insert({Which:39,Boolvec:[false,true,false]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:39,Boolvec:[false,true,false]}) produced 
    pogs_test.go:170: Insert({Which:40}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:40}) produced 
    pogs_test.go:170: Insert({Which:40,Datavec:[[104,105],[98,121,101]]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:40,Datavec:[[104,105],[98,121,101]]}) produced 
    pogs_test.go:170: Insert({Which:41}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:41}) produced 
    pogs_test.go:170: Insert({Which:41,Textvec:["John","Paul","George","Ringo"]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:41,Textvec:["John","Paul","George","Ringo"]}) produced 
    pogs_test.go:170: Insert({Which:25,Zvec:[{Which:4,I64:-123},{Which:13,Text:"Hi"}]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:25,Zvec:[{Which:4,I64:-123},{Which:13,Text:"Hi"}]}) produced 
    pogs_test.go:170: Insert({Which:26,Zvecvec:[[{Which:4,I64:1},{Which:4,I64:2}],[{Which:4,I64:3},{Which:4,I64:4}]]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:26,Zvecvec:[[{Which:4,I64:1},{Which:4,I64:2}],[{Which:4,I64:3},{Which:4,I64:4}]]}) produced 
    pogs_test.go:170: Insert({Which:32}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:32}) produced 
    pogs_test.go:170: Insert({Which:32,Planebase:{Name:"Boeing",Homes:[2,5],Rating:123,CanFly:true,Capacity:100,MaxSpeed:9001}}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:32,Planebase:{Name:"Boeing",Homes:[2,5],Rating:123,CanFly:true,Capacity:100,MaxSpeed:9001}}) produced 
    pogs_test.go:170: Insert({Which:33,Airport:2}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:33,Airport:2}) produced 
    pogs_test.go:170: Insert({Which:42,Grp:{First:123,Second:456}}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:175: Insert({Which:42,Grp:{First:123,Second:456}}) produced 
=== RUN   TestInsert_Size
--- FAIL: TestInsert_Size (0.00s)
    pogs_test.go:403: void into 0-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=8 ptrs=0}), {}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: void into 1-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=9 ptrs=0}), {}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: bool into 1 byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=9 ptrs=0}), {Which:12,Bool:true}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: int8 into 1-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=9 ptrs=0}), {Which:7,I8:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: uint8 into 1-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=9 ptrs=0}), {Which:11,U8:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: int16 into 2-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=10 ptrs=0}), {Which:6,I16:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: uint16 into 2-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=10 ptrs=0}), {Which:10,U16:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: enum into 2-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=10 ptrs=0}), {Which:33,Airport:1}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: int32 into 4-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=12 ptrs=0}), {Which:5,I32:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: uint32 into 4-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=12 ptrs=0}), {Which:9,U32:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: float32 into 4-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=12 ptrs=0}), {Which:3,F32:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: int64 into 8-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=16 ptrs=0}), {Which:4,I64:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: uint64 into 8-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=16 ptrs=0}), {Which:8,U64:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: float64 into 8-byte: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=16 ptrs=0}), {Which:2,F64:123}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: text into 1 pointer: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=8 ptrs=1}), {Which:13,Text:"hi"}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: data into 1 pointer: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=8 ptrs=1}), {Which:14,Blob:[104,105]}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: list into 1 pointer: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=8 ptrs=1}), {Which:15,F64vec:[123]}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
    pogs_test.go:403: struct into 1 pointer: Insert(0xea26e9973bd6a0d9, capnp.NewStruct(seg, {datasz=8 ptrs=1}), {Which:32,Planebase:{}}) = pogs: insert @0xea26e9973bd6a0d9: unexpected EOF; want nil
=== RUN   TestExtract_StringBytes
--- FAIL: TestExtract_StringBytes (0.00s)
    pogs_test.go:432: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:436: Extract() produced {}; want {Which:13,Text:[72,101,108,108,111,44,32,87,111,114,108,100,33]}
=== RUN   TestExtract_StringListBytes
--- FAIL: TestExtract_StringListBytes (0.00s)
    pogs_test.go:455: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:459: Extract() produced {}; want {Which:41,Textvec:[[72,111,108,109,101,115],[87,97,116,115,111,110]]}
=== RUN   TestInsert_StringBytes
--- FAIL: TestInsert_StringBytes (0.00s)
    pogs_test.go:475: Insert({Which:13,Text:[72,101,108,108,111,44,32,87,111,114,108,100,33]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:481: Insert({Which:13,Text:[72,101,108,108,111,44,32,87,111,114,108,100,33]}) produced 
=== RUN   TestInsert_StringListBytes
--- FAIL: TestInsert_StringListBytes (0.00s)
    pogs_test.go:497: Insert({Which:41,Textvec:[[72,111,108,109,101,115],[87,97,116,115,111,110]]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:503: Insert({Which:41,Textvec:[[72,111,108,109,101,115],[87,97,116,115,111,110]]}) produced 
=== RUN   TestExtract_StructNoPtr
--- FAIL: TestExtract_StructNoPtr (0.00s)
    pogs_test.go:530: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:534: Extract() produced {}; want {Which:32,Planebase:{Name:"foo"}}
=== RUN   TestExtract_StructListNoPtr
--- FAIL: TestExtract_StructListNoPtr (0.00s)
    pogs_test.go:555: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:561: Extract() produced {}; want {Which:25,Zvec:[{Which:4,I64:123}]}
=== RUN   TestExtract_GroupNoPtr
--- FAIL: TestExtract_GroupNoPtr (0.00s)
    pogs_test.go:580: Extract() error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:584: Extract() produced {}; want {Which:42,Grp:{First:123,Second:456}}
=== RUN   TestInsert_StructNoPtr
--- FAIL: TestInsert_StructNoPtr (0.00s)
    pogs_test.go:600: Insert({Which:32,Planebase:{Name:"foo"}}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:606: Insert({Which:32,Planebase:{Name:"foo"}}) produced 
=== RUN   TestInsert_StructListNoPtr
--- FAIL: TestInsert_StructListNoPtr (0.00s)
    pogs_test.go:624: Insert({Which:25,Zvec:[{Which:4,I64:123}]}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:632: Insert({Which:25,Zvec:[{Which:4,I64:123}]}) produced 
=== RUN   TestInsert_GroupNoPtr
--- FAIL: TestInsert_GroupNoPtr (0.00s)
    pogs_test.go:648: Insert({Which:42,Grp:{First:123,Second:456}}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:654: Insert({Which:42,Grp:{First:123,Second:456}}) produced 
=== RUN   TestExtract_Tags
--- FAIL: TestExtract_Tags (0.00s)
    pogs_test.go:705: renamed field: Extract error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:708: renamed field: Extract produced {}; want {Which:2,Float64:3.5}
    pogs_test.go:705: omitted field: Extract error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:708: omitted field: Extract produced {}; want {Which:4}
    pogs_test.go:705: field with overlapping name: Extract error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:708: field with overlapping name: Extract produced {}; want {Which:12,U8:true}
=== RUN   TestInsert_Tags
--- FAIL: TestInsert_Tags (0.00s)
    pogs_test.go:748: renamed field: Insert({Which:2,Float64:3.5}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:753: renamed field: Insert({Which:2,Float64:3.5}) produced 
    pogs_test.go:748: omitted field: Insert({Which:4,I64:42}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:753: omitted field: Insert({Which:4,I64:42}) produced 
    pogs_test.go:748: field with overlapping name: Insert({Which:12,U8:true}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:753: field with overlapping name: Insert({Which:12,U8:true}) produced 
=== RUN   TestExtract_WhichTag
--- FAIL: TestExtract_WhichTag (0.00s)
    pogs_test.go:777: Extract error: pogs: extract @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:780: Extract produced {}; want {Bool:true}
=== RUN   TestExtract_WhichTagMismatch
--- PASS: TestExtract_WhichTagMismatch (0.00s)
=== RUN   TestInsert_WhichTag
--- FAIL: TestInsert_WhichTag (0.00s)
    pogs_test.go:814: Insert({Bool:true}) error: pogs: insert @0xea26e9973bd6a0d9: unexpected EOF
    pogs_test.go:820: Insert({Bool:true}) produced 
=== RUN   TestExtraFields
--- PASS: TestExtraFields (0.00s)
=== RUN   ExampleExtract
--- FAIL: ExampleExtract (0.00s)
panic: pogs: extract @0x8100cc88d7d4d47c: unexpected EOF [recovered]
    panic: pogs: extract @0x8100cc88d7d4d47c: unexpected EOF

goroutine 1 [running]:
panic(0x172be0, 0xc42020e120)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
testing.runExample.func2(0xecf52be3b, 0xc40701daf1, 0x2797e0, 0xc42002a050, 0xc42002a010, 0xc4201f2c00, 0x1aa5af, 0xe, 0x1c5440, 0x1ae93c, ...)
    /usr/local/go/src/testing/example.go:109 +0x392
panic(0x172be0, 0xc42020e120)
    /usr/local/go/src/runtime/panic.go:458 +0x243
zombiezen.com/go/capnproto2/pogs_test.ExampleExtract()
    /Users/jaten/src/zombiezen.com/go/capnproto2/pogs/example_test.go:45 +0x2ef
testing.runExample(0x1aa5af, 0xe, 0x1c5440, 0x1ae93c, 0x1f, 0x0, 0x0)
    /usr/local/go/src/testing/example.go:114 +0x1a0
testing.RunExamples(0x1c4d08, 0x272e80, 0x2, 0x2, 0xe200)
    /usr/local/go/src/testing/example.go:38 +0x15e
testing.(*M).Run(0xc420037ee8, 0x1)
    /usr/local/go/src/testing/testing.go:744 +0xbd
main.main()
    zombiezen.com/go/capnproto2/pogs/_test/_testmain.go:122 +0xc6
exit status 2
FAIL    zombiezen.com/go/capnproto2/pogs    1.255s
jaten@jatens-MacBook-Pro ~/src/zombiezen.com/go/capnproto2/pogs (master) $

capnpc-go should not generate name collisions

One of the notable examples is in persistent.capnp that the annotation and the type collide. However, there are subtler cases, such as the struct field in schema.capnp should not collide with the Struct field of the wrapper struct.

Import failed: zombiezen.com/go/capnproto2/go.capnp

Greetings,
I was trying to generate go files by running:
$ capnp compile -ogo message/books.capnp
message/books.capnp is as follow:

using Go = import "zombiezen.com/go/capnproto2/go.capnp";
@0x85d3acc39d94e0f8;
$Go.package("message");
$Go.import("myProject/message");

struct Book {
    title @0 :Text;
    # Title of the book.

    pageCount @1 :Int32;
    # Number of pages in the book.
}

Then i get

message/books.capnp:1:19-57: error: Import failed: zombiezen.com/go/capnproto2/go.capnp
message/books.capnp:3:2-4: error: Not defined: Go
message/books.capnp:4:2-4: error: Not defined: Go

How do i fix this? Thanks.

Regression: error on sandstorm's util.capnp

Given the util.capnp from sandstorm with this patch applied (just adds the annotations needed for go):

diff --git a/home/isd/src/foreign/sandstorm/src/sandstorm/util.capnp b/capnp/capnp_gen/util.capnp
index d5bd809..45e7ded 100644
--- a/home/isd/src/foreign/sandstorm/src/sandstorm/util.capnp
+++ b/capnp/capnp_gen/util.capnp
@@ -166,3 +166,6 @@ interface Assignable(T) {
     set @0 (value :T) -> ();
   }
 }
+using Go = import "/zombiezen.com/go/capnproto2/go.capnp";
+$Go.package("util");
+$Go.import("zenhack.net/go/sandstorm/capnp/util");

I get this error:

$ capnp compile -I $GOPATH/src/ -ogo util.capnp 
capnpc-go: generating util.capnp: interface client util.capnp:ByteStream: template: :74:193: executing "interfaceClient" at <$.G.RemoteName>: RemoteName is not a field of struct type *main.generator
go: plugin failed: exit code 1

git bisect blames 4c8af43.

Add pointer security checks to runtime support

Now that all pointer reads/writes can report failure, we should implement the checks detailed in Security Considerations to prevent amplification attacks and stack overflow DoS attacks.

I'm imagining this will be implemented by adding the counters to Struct and List. However, this has the side-effect that pointers to the same object from different paths will not be equal based on Go's == operator. This is mitigated because in valid Cap'n Proto messages, "no more than one pointer can point at each object", according to the spec. Because of the relatively little utility, the lack of documentation in this library around depending on this behavior, this seems reasonable to me.

capnpc-go: panic if accessing a field that does not match discriminant

Currently, the generated code will blindly return a field that does not match the set discriminant, which is undefined behavior (bad). Panicing is a much saner behavior.