From the trace file description:

If provided systemTraceEvents is a string of Linux ftrace data or Windows ETW trace data. This data must start with # tracer: and adhere to the Linux ftrace format or adhere to Windows ETW format.

So I think we can potentially teach traced to read ftrace data, and dump it out at the end of the trace.

Right now, chunks are only submitted when the chunk is full, or systrace_deinit is called. This is bad for a few reasons.

Firstly: it does not handle abrupt termination
Secondly: it does not handle quiet threads (that only report 1-2 events)
Thirdly: it does not handle threads that report a lot, but quit at some point

I think we need to change the way chunk allocation works to fix this, but changing that may fix a few other problems at the same time.

Suggested workflow:

• Client asks traced for a chunk name
• traced hands it one, records it as in use
• Client continues to notify traced of full chunks (but we now also use this to get a new chunk allocated at the same time

Every N (and at startup):

• traced looks over open chunks
  • If chunk PID is dead, or chunk TID is dead, process (and free) chunk

Note that if a chunk is kept open by a thread, but the thread is not reporting much, this is still a problem (until termination). I don't see this as being a problem, but if it turns out to be one, we can always look at a solution for that. I'm a little wary about trying to handle that case now, because it could likely get ugly fast.

Rationale

Having minimal dependencies is good for running on "strange" systems if needed. Having minimal dependencies also means minimal chances of performance issues in unexpected places.

Broad plan

• Blocking socket I/O
• Thread per connection
  • (and possibly subsequently per chunk to process? memcpy the buffer out of shm, and release the chunk back ASAP, then process? Stretch goal, maybe a separate issue)

systrace_init/systrace_deinit will still be required (although I am tempted to remove the ctor attributes from them, or at least make them optional) - too magical for my liking...

The rest should go into the Chromium-influenced TRACE_* macros (or rather "implementation" of them)

Picture:

• PID 55 starts, allocates a chunk, logs a duration begin event.
• PID 55 then forks a child of PID 66
• They both log a duration end event

Currently, this means that sort of bad things will happen. The SHM chunk will probably be shared between processes (I'm not entirely sure), and on top of that, screwy things happen to the events.

I'm not sure what the right fix for this is, as we can't easily know which "side" of the fork is actually supposed to win. I think we may need a way to flag a chunk as being "ghost events", which are copied from the parent's, and used in the child if necessary (e.g. if an end is seen with no matching begin, try find a ghost event that matches it, otherwise the ghosts are just ignored).

An alternative might be to just say "sorry, don't expect events to work across fork boundaries", and drop them, but that seems a bit harsh.

Shouldn't be too difficult, but there will definitely be some changes. Right now it's all Mac-specific.

To control traced without restarting it.

For instance:

• tracectl start -- I want to see what's going on, please start dumping to a file
• tracectl stop -- I don't care now. Stop dumping and clear buffers if any.
• tracectl restart -- obvious

Other possibilities, who knows... "show me only PIDs X, Y, Z, thread IDs A, B, C"... but that can come later.

There's some ifdef'd out code in traced that dumps memory info into the JSON. So we know roughly what it needs to look like. Next steps are to get that information.

Chromium has a whole API for this. I think we'll need one too, but maybe a bit simpler.

Suggested design at the moment:

• A memory dumper is a function pointer, registered with systrace as an API call (systrace_memdump_register(fptr))
• systrace_init spawns a thread which sleeps, and every N seconds, calls the registered memory dumpers.
• Memory dumpers call systrace_memdump_report_byte_total("Subsystem", byteCount) where byteCount is the number of bytes used
• At the end of the memdump thread's run, this is all passed in messages to traced for it to dump into JSON.

These of course will need thread-safety, then, but I think that there's not really any "nice" way to make this happen otherwise.