tracingplane / tracingplane-java Goto Github PK

A bag could predefine the point at which overflow is acceptable, so that the bag itself can overflow without causing all subsequent bags to overflow

To achieve this, baggage buffers would add a way of annotating the acceptable size limit

Then in the baggage protocol, we would need a per-bag overflow marker. I think the best way of doing this would be to add an indexed child with no index specified (which would therefore be lexicographically less than the bag's other children)

One-of fields are useful, especially for use cases like supporting 64/128 bit IDs

• Create a fork of brownsys/tracing or a new branch.
• Remove baggage implementation from there and add this one as a dependency
• Update usage of baggage in X-Trace, Retro, and Pivot Tracing.

Not a tutorial, but a list of the features available for use in Baggage Buffers.

Instructions to download, install pre-requisites, and build.

Similar to http://brownsys.github.io/tracing-framework/docs/tutorials.html

Javadocs at the package level so that stuff shows up on this page: https://jonathanmace.github.io/tracingplane/doc/javadoc/index.html

Structs support dynamic-size variables, so add lists and maps

• Package level javadoc (https://jonathanmace.github.io/tracingplane/doc/javadoc/index.html). Can just be a duplicate of the static class docs.
• Explanation of examples here: https://jonathanmace.github.io/tracingplane/doc/javadoc/edu/brown/cs/systems/baggage_buffers_examples/package-summary.html
• Good documentation of main apis in all four layers.
• Reasonable documentation of code elsewhere.

When you want to merge multiple fields as a single entity, want to use structs.

Structs also impose lower overhead.

Create aspects subproject (either in tracingplane or as a separate repository) and migrate aspects from brownsys/tracing to use the new APIs.

Enum of values that can just map to integers

Can then be used for flag sets

All variables should be optional in a struct

After migrating Hadoop instrumentation and brownsys/tracing, do some end-to-end tests to measure overheads and check correctness.

Write a getting started tutorial for instrumenting systems w/ the transit layer.

Add couscous-generated website (a la http://brownsys.github.io/tracing-framework/) for markdown / docs in this repository.

Write a tutorial for using Baggage Buffers to create a tracing application.

In the current implementation, users manually manipulate object fields, because that was quicker to implement.

Ideally, it should be more protobuf-style, with getters, setters, and existence checkers.

This will also make it easier to provide things such as optionals and one-ofs

Volatile fields are fields that we only want one side of a branch to ever see.

When we branch a context, only one side of the branch will get the field; the other side will get null.

Volatile fields enable the implementation of state-based CRDTs based on vector clocks that have random component IDs; you declare a component id field that is volatile, and a map<id, ...> for the values.

Volatile fields are always transient, because we can't guarantee an intermediary component will respect the volatile-ness

Migrate the instrumentation of brownsys/hadoop (pivottracing branches) to instead use the transit layer APIs of this repository.

For child data with index < 15, it is cheaper to inline the child as a data atom.

The following features are yet to be implemented in Baggage Buffers:

• Inline fields
• Max and Min
• Add/Subtract Counter
• Take comments in .bb file and put them as class/field comments in generated code.

Need a spec so that other people know what to implement if they wanted to integrate tracingplane into other languages.

README markdowns in root and project subdirs:

• Readmes for root project and all subprojects. Figure out appropriate way of presenting the information.
• Tracingplane organization documentation
• Directory of resources (wiki pages, javadocs, package docs)

Javadocs

• Package level documentation
• Docs on all public classes
• Fold some of the package doc README info into javadocs too

Wiki pages

• One of several pages outlining researchy parts of tracing plane, mostly pulled from paper
• High level overview of how to use tracingplane
• Tutorial: Downloading / cloning / building / installing
• Tutorial: set up a maven project or add jar to classpath
• Tutorial: instrumenting a system (threads, runnables, callables); instrumenting a system (custom queues); instrumenting a system (network / io); advanced: using aspectj
• FAQ page: what is the license? do you have systems instrumented already that i can plug in to? is it compatible with scala?

Other Project TODOs:

• Make download links for distribution JARs with dependencies
• Make license explicit somewhere
• For distribution JARs, enumerate all dependencies and their licenses
• Split out BDL compiler to separate project
• Fix scala versions

Bags should support transient fields. Transient fields are omitted during serialization; that is, they only live within a single process.

Transient fields should behave like other fields -- eg they should be bags, and support merge/join

Since transient fields are not serialized, transient fields do not need a field number. They could be declared like:

int64 myregularid = 0;
int64 mytransientid = transient;

That's a little ugly but i think it'll do for now

As an extension to tutorials, write an xtrace lite application that will aid transit layer instrumentation.

Implement true-wins and false-wins flags.

With inline data atoms, this can be a zero-length data atom whose presence indicates the value.

Split out compiler as a separate repo from the java library

Update Brown Tracing Framework website with pointers to tracingplane

http://brownsys.github.io/tracing-framework/

Bag options could be specified as a separate child bag rather than in the header. This would be better for supporting changes to the options (eg, if you want to specify a lower bag threshold)