Giter Club home page Giter Club logo

parigot's Introduction

parigot

parigot is an adjective in French that mean parisien or of paris. In english, one might use "NewYork-ish" in the same way.

If you just have experience with microservices or RPC systems, and want to check out the basic info in the shortest possible time, check out I know how this works

Our docs are kept at parigot.info.

parigot is a new way to build, test, and deploy microservices. parigot is not ready for prime time, rational or irrational time, production usage, or fibonacci time. parigot is currently under heavy development (multiple commits per day) and as such might be at any degree of brokenness at any time.

If you are intersted and want to talk to the author, the best way is using our Zulip

parigot's People

Contributors

bradwich avatar cindyuc avatar iansmith avatar yorinasub17 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

jeff-rick

parigot's Issues

Shrink size of dev container

Currently, the dev container is 4.6GB because it includes an ubuntu server and a ton of added packages.

Shrink the size of the devcontainer using multi-stage builds

With alpine, we might have to ditch the machinery we currently have that creates a parigot user and then makes all the tools and such owned by the parigot user and group. It's ok if this goes away to get us to a much smaller container size.

There is are already some tags that provide access to an alpine-based container image with golang.

1.19.4-alpine3.17, 1.19-alpine3.17, 1-alpine3.17, alpine3.17, 1.19.4-alpine, 1.19-alpine, 1-alpine, alpine

Call id propagation

Currently, we are not passing the call id to every place that it needs to go. A call id should be generated the initial call site (in a client) and then pushed (via the pctx) to all downstream calls. This data needs to also be added to the LogRequest so that all log messages have the call id on them.

Further, the log viewer application should probably do bookkeeping to keep all the calls in a particular chain (sharing the same call id) are grouped together.

standard for package and service names, printed and used

Create a type for package+service names and a standard way of printing them, maybe (file.File)

Code that is handling these needs to be using a specific type internally. Calls like Locate() and Require() that pass around that information should do so with a protobuf defined type.

It's ok if the programming language binding uses two strings for now, it's more important to update the internal usages so that it can be changed by somebody who wants to use a different definition.

Distinguish semantic errors produced server implementations

The serverdecl.tmpl in protoc-gen-parigot has a problem with the error returns. When it calls a function like Open (at line 110 of FileServer.p.go at the moment) in the implementation it is expecting two return values, the returned value and a possible error. It should be expecting three return values, the two previous and a new one that is lib.Id.

  • The semantic errors of the go-side of the world are currently being converted into an “error” but should be returned separately so we can do better error reporting.

Log viewer

The current log viewer UI is quite primitive. Some improvements that are needed are:

  • Clear button to clear the display

  • Don't make so wide, maybe wrap or horizontal scrolling. The current behavior is that it gets wider to fit the longest log message on one line.

  • Add another pane for internal messages (probably below the log messages and visually distinct)

    • Internal here means messages from the log viewer itself
  • Visually Distinguish the kernel and internal service log messages

    • The terminal based logger already does this with ** and >> respectively.
  • Have a way to choose what log level to filter on... probably everything at the level and “above” it

    • Probably need a pull down menu for this
  • On #6 we are suggesting that we should be propagating the call id with all log messages.

    • The log viewer should track what call ids it has seen and put new log messages with that call id at the end of the sequence of messages on that call id. Probably want to cycle through some colors and make all the text for a given call id distinct from the predecessor and successor.

keybindings for dev in web version of VSCode

There is a plugin for the web version of vscode that changes the keybindings for chrome such that the "normal" Chrome key accelerators work properly and the bindings for VSCode are changed to not interfere.

https://marketplace.visualstudio.com/items?itemName=erikpukinskis.chrome-codespaces-keymap

If this can be confirmed to work on a desktop, it should then be confirmed on an iPad plus some type of android tablet to see if work could actually be done with just a tablet and GitHub code spaces.

For apple users, the plugin needs to be duplicated (?) or at least tested for Safari.

No Locate() without Require()

Add some bookkeeping so that user program cannot call Locate() on something they did not require.

This situation clearly results from a broken program and the Locate() should panic.

Fix primary branch

The primary branch of parigot is master not main. I don't know if GitHub supports using master as the
primary branch anymore. It would probably be best to simply delete the existing main branch and then rename
master->main.

Now() should be added in the kernel code

Now() needs to be under test code control, for easier testing of things that need to be done hourly, daily, every weekday, etc.

Need to add a Now() call in the kernel that works with a "SetNow()" that should be in the atlanta.test area.

Now() should be implemented using the normal sys call machinery, but should respect SetNow if anybody has set the current time. SetNow() probably needs a flag also that says "freeze time" or not. If freeze time is true, then the clock stops running until the next SetNow() time.

The Now() machinery needs to also make sure that the start time of a call is recorded as what Now returns and that this gets passed via the pctx mechanism to downstream calls. This will require that upon receiving a call, receivers need to probably store the Now() value as an offset inside the pctx and then use the pctx for generating a timestamp on any log messages.

The log problem: how to structure it and other future builtins

The Problem

We have a logging infrastructure that is called Pctx (and pctx and PCtx and this array of names probably indicate something is wrong). Pctx was originally envisioned as a replacement for context.Context but language neutral so it could be used in languages like python or rust. You can read more about the three primary features of Pctx here.

Although all these goals are all laudable, I think the logging part should be removed from the Pctx world and
converted to a system provided service. This would be more "regular" with the RPC-based model of parigot
and would allow users to formulate drop in replacements. This service would be the first truly parigot-
provided service and it has revealed some naming problems.

Currently, we have multiple "projects" that we coordinate via the go.work file at the repo root. The ones that
are important for this discussion are:

  • api
  • lib
  • sys
    as well as an example program that exercises the these projects such as example/vvv.

You will notice that each of these has a go.mod at the nominal root of the project like sys/atlanta.base/go.mod.

Note that something seems to be broken in api right now and that go.mod has gone missing but I'll likely put it back soon.

The api defines the services provided by paragot (such as the problematic logging service related to this ticket), lib/atlanta.base/go is the client side library that users of parigot should link into their go programs. sys/atlanta.base is the implementation of the code that runs inside wastime and provides the system call interface--which should be language independent.

The problem hinges on 3 key issues:

  1. When you write a .proto file you cannot import files from a different project (in the sense of those from the previous paragraph) with a path like ../../api/atlanta.base/... so the implementation of the api must be in the api project. This is poor if you are trying to have the api directory/project just be .proto files that user code can reference from any language and have various implementations in the lib project/directory.
  2. The entire point of the go.work feature of go is to work with multiple repositories at the same time. You can do go build github.com/iansmith/parigot/api/... to build the API project, for example. However, the fact that all these directories/projects are actually part of the same repo (github.com/iansmith/parigot) makes me feel like I should not be using this feature of go in this way. It seems like it is intended more for totally different repositories, like perhaps github.com/iansmith/parigot and github.com/iansmith/parigot-sample-programs.
  3. We need language neutrality in some of these cases (api, to some extent sys) but not all.

So the problem is that we to do some kind of a reorganization of how the repository is structured so that we can
provide sample implementations of services and not prevent users from creating their own implementations of
the services as well. The fact that logging is one that many users may want to replace with their own version
has brought this to a head now.

fix startup args passed to the wasm level

currently this code inside a warm module will fail:

flag.Parse()

because there are zero command line arguments.

A minimal fix would be to pass the name of the warm module as arg[0] which would allow flag.Parse() to trivially
work. It would be better if command line flags could be passed to the warm module, although this probably means
changes to runner and the docker-compose to allow the passing of args.

Compatibility with spaces and fleet

Jet brains has introduced spaces (a kind of competitor to GitHub) and an editor that works well with it called Fleet.

Work with the .devcontainer/Dockerfile to create the appropriate machinery for GitHub and Spaces and Fleet from JetBrains.

make bind ignore test methods

Bind() should ignore attempts to bind a method that is not actually in the set of methods. It should probably issue a warning when it detects this situation.

Consider a .proto which says that foo() is a test method. That method may be implemented in some service (say bar) but not intended for use in prod. A nefarious person could try to bind foo() and then invoke it, which would be bad if the author of foo() expected it to only exist in a test situation.

This problem is really nasty because the decision of "is foo a test method()" is made at the time the code is generated and (intentionally) does not distinguish between test and normal methods so the tests can "work normally" without a bunch of shenanigans. The problem is that we would need to have a way to know when a bind is issued if it is legal or not, which at some level is going to need the .proto definition.

docker container dot files

The current .devcontainer/Dockerfile seems to have a messed up configuration. Originally it was intended to install zsh for the user parigot but that doesn't appear to always work.

It would probably be best for everybody if we just supported some dotfiles program:

github's info on dot files

not fully typing the server definition in generated go code

Currently, the code generator (protoc-gen-parigot) is generating something that is not attractive or convenient
in the server side go code.

In template/go_/serverdecl.tmpl line 30 is always outputting the type proto.Message but it should be
generating the specific type for that declaration. As it is, the user is forced to use a cast to get the parameter
value into the proper type:

req := inProto.(*pb.LogRequest)

This defeats the purpose of generating code and the code generator has the type information.

Code complexity

I love the KISS principle and so I've written a linter that pinpoints to complex functionality.
(This can hide bugs, and is more difficult to maintain.)

Here the list, that could be candidates for refactoring:
parigot/api/plugin/httpconnector/httpconnectorhost.go:108:1 - callHostDispatch has complexity: 18
parigot/api/plugin/queue/queuehost.go:416:1 - receive has complexity: 19
parigot/command/protoc-gen-parigot/codegen/finder.go:37:1 - GoPackageOption has complexity: 16
parigot/command/protoc-gen-parigot/codegen/finder.go:108:1 - AddressingNameFromMessage has complexity: 32
parigot/command/protoc-gen-parigot/codegen/finder.go:161:1 - FindMessageByName has complexity: 23
parigot/command/protoc-gen-parigot/codegen/generate.go:21:1 - BasicGenerate has complexity: 57
parigot/command/protoc-gen-parigot/codegen/generate.go:138:1 - Collect has complexity: 16
parigot/command/protoc-gen-parigot/codegen/pass.go:81:1 - MethodsPass has complexity: 31
parigot/command/protoc-gen-parigot/main.go:128:1 - generateNeutral has complexity: 50
parigot/command/protoc-gen-parigot/test/main.go:15:1 - main has complexity: 18
parigot/command/protoc-gen-parigot/util/plugin.go:27:1 - ReadStdinIntoBuffer has complexity: 18
parigot/command/runner/runner/config.go:131:1 - Parse has complexity: 50
parigot/sys/kernel/block.go:20:1 - ReadOne has complexity: 30
parigot/sys/kernel/starter.go:157:1 - requirementsMet has complexity: 16
parigot/sys/kernel/starter.go:205:1 - detectCycle has complexity: 33
142 = files
818 = functions
57 = highest complexity
2.90 = overall average complexity

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.