I am struggling a bit with the organization. Currently LFC appears as both code-generator and backend and build system. The distinction between the two latter is also not clear. What is really the meaning of backend and build-system in the context of lingo?

I'm using the latest lfc from master.

Output:

- embedded: Error: Command exited with status exit status: 1: "/home/marten/lingua-franca-master/git/lingua-franca/bin/lfc" "--json={\"out\":\"/home/marten/lf-lang/lingo/sandbox/target\",\"properties\":{\"no-compile\":true},\"src\":\"/home/marten/lf-lang/lingo/sandbox/src/Main2.lf\"}"
- git-hook: Error: Command exited with status exit status: 1: "/home/marten/lingua-franca-master/git/lingua-franca/bin/lfc" "--json={\"out\":\"/home/marten/lf-lang/lingo/sandbox/target\",\"properties\":{\"logging\":\"info\",\"no-compile\":true},\"src\":\"/home/marten/lf-lang/lingo/sandbox/src/Main.lf\"}"

Also observed by @gundralaa who is blocked by this issue.

Running lfc --json={"out":"/home/marten/lf-lang/lingo/sandbox/target","properties":{"logging":"info","no-compile":true},"src":"/home/marten/lf-lang/lingo/sandbox/src/Main.lf"} results in:

Usage: lfc [-cfhlnqV] [--print-statistics] [--build-type=<buildType>]
           [--external-runtime-path=<externalRuntimePath>]
           [--logging=<logging>] [-o=<outputPath>] [-r=<rti>]
           [--runtime-version=<runtimeVersion>] [-s=<scheduler>]
           [-t=<true/false>] [--target-compiler=<targetCompiler>]
           [-w=<workers>] (FILES... | --json=<jsonString> |
           --json-file=<jsonFile> | --stdin)
      FILES...              Paths to one or more Lingua Franca programs.
      --build-type=<buildType>
                            The build type to use.
  -c, --clean               Clean before building.
      --external-runtime-path=<externalRuntimePath>
                            Specify an external runtime library to be used by
                              the compiled binary.
  -f, --federated           Treat main reactor as federated.
  -h, --help                Show this help message and exit.
      --json=<jsonString>   JSON object containing CLI arguments.
      --json-file=<jsonFile>
                            JSON file containing CLI arguments.
  -l, --lint                Enable linting of generated code.
      --logging=<logging>   The logging level to use by the generated binary
  -n, --no-compile          Do not invoke target compiler.
  -o, --output-path=<outputPath>
                            Specify the root output directory.
      --print-statistics    Instruct the runtime to collect and print
                              statistics.
  -q, --quiet               Suppress output of the target compiler and other
                              commands
  -r, --rti=<rti>           Specify the location of the RTI.
      --runtime-version=<runtimeVersion>
                            Specify the version of the runtime library used for
                              compiling LF programs.
  -s, --scheduler=<scheduler>
                            Specify the runtime scheduler (if supported).
      --stdin               Read paths to Lingua Franca programs from stdin.
  -t, --threading=<true/false>
                            Specify whether the runtime should use
                              multi-threading (true/false).
      --target-compiler=<targetCompiler>
                            Target compiler to invoke.
  -V, --version             Print version information and exit.
  -w, --workers=<workers>   Specify the default number of worker threads.

There seem to be two problems that need to be fixed:

• Improve the error reporting in lingo so that it is clear what went wrong
• Fix lfc so that it can actually handle the --json argument (lf-lang/lingua-franca#1909)

We have to decide on if we want to use nix as a backend or not.

How the Demo works

The demo reads the Barrel.toml then generates the nix code. Then the nix builder is called with evaluates the expression and builds the package. The actual target wrapper and lingua-franca packages are defined in lf-pkgs.

Barrel with Nix

If we decide to use nix as a backend installing libraries in your system won´t work anymore, because nix has a isolated build environment. So just make install in your system wont´t work. Libraries need to be packages or pulled into the environment, when nixpkgs didn´t already package it. Packaging something in nix is compared to other systems like debian trivial but Industry folks probably don´t have the nerves to do it. The easiest way to pull in other libraries is by pulling them in as a git-submodule (e.g. lf-alarm-clock)

Build Process

flowchart TB
    A[Start] -->B(Fetch Meta from packages)
    subgraph barrel
	B --> C(Generate Nix Code)
    end
    subgraph nix-env
    C --> D(Evaluate Nix)
    D --> E(Nix Deamon Builds Package)
    end

Nix Architecture

• the current lf-pkgs repo would stop holding package definitions and only define the different target wrappers.
• barrel would check which version is specified and then generate the correct fetchgitor fetchTarball

The upside of using the nix-env would that we get a package definition and we can profit of the eco-system e.g. Remote-Builders, Package Caches ...

Barrel without Nix*

The difference between the two approaches is mainly that we have build everything ourself but don't have the constraints that are introduced by nix for reproduce-ability. This would include fetching, unpacking, moving stuff, invoking lfc.

flowchart TB
    A[Start] -->B(Fetch Meta from packages)
    subgraph barrel
	B --> C(manually fetch Tarballs)
    C -->  D(Move Sources and Setup Build-Env)
    D --> E(Invoke correctly parameterization lfc)
    end

Rust or Kotlin

On main. If I run:
lingo init -l c or lingo init -l cpp we no longer get an example project, only a Lingo.toml is created.

JSON via Stdin

We currently pass JSON using the --json={} option, but it would be cleaner to use stdin.

Steps

• modify lingo to give lfc properties via stdin @revol-xut
• modify lfc to receive via stdin @lhstrh

Lingo 0.2.0 Release

Deadline: 30.9

• #32
• This needs to be a little reworked so target properties are merged on lfc site and are not just treated as command line parameters. SOMEBODY NEEDED @lhstrh
• proper testing @cmnrd @lhstrh @revol-xut
• crates.io release @revol-xut

It would be great to be able to call something like lingo format and lingo format --check to format all the files in a project.

Building ...
Running "/home/erling/tools/lingua-franca/build/install/lf-cli/bin/lfc" "--json={\"out\":\"/home/erling/dev/lf/lf-ex/my-synth/target\",\"properties\":{\"no-compile\":false},\"src\":\"/home/erling/dev/lf/lf-ex/my-synth/./src/Main.lf\"}"
Unmatched argument at index 4: 'false'

I think it might be because --no-compile=false is not a valid command line argument. If you want to invoke the target compiler, just dont pass any --no-compile arg

I'm running into issues running multiple LF program regression tests on hardware.
Doing this by hand and collecting the results is quite tedious.
It would be nice for lingo run to utilize a probe-rs backend to allow for this automation for binaries meant for embedded devices.

This is a continues issue about testing while the code base is relatively small.

Integration Tests

• CI with Lingua-Franca
  • Forwarding of the target properties
• Running Lingo on LFC Tests full integration style
  • Check a Lingo.toml into Lingua-Franca repo where all the test are defined.
  • Lingo run needs to take care of the result of the binary

Unit Tests

• Backends
  • CMake
• Source Gen
• Pass Clap Arg Structs

lingo install <nightly/stable>

to install latest lingua-franca toolchain.

Feature Request

 $ lingo clean 

should remove all build artifacts generated by lingo

target C;

main reactor {
    reaction (startup)  {=
        lf_print("Hello World!");
        lf_print_error_and_exit("ERROR");
    =}
} 

Yields:

Running "/home/erling/dev/lf/lingo/tmp/c/target/bin/Main"
FATAL ERROR: ERROR
---- Start execution at time Tue Oct 10 17:15:55 2023
---- plus 538501132 nanoseconds
Environment 0: ---- Spawning 1 workers.
Hello World!
---- Terminating environment 0
---- Elapsed logical time (in nsec): 0
---- Elapsed physical time (in nsec): 235,323
- Main: Success

See lf-lang/lingua-franca@75ebf6d#r1278621320 for more context.

Proposal of Barrel.toml

[package]
name = "example_project"
version = "0.1.0"
authors = ["[email protected]"]
homepage = "https://lf-lang.org"
license = "Weird Stallman License"
description = "A little Barrel.toml so people can discuss it"

# shared properties of all binaries
[properties]
fast = true

# first binary in the project
[[app]]
name = "git-hook" # <-- should this default to package.name ?
target = "cpp"
# main_reactor defaults to src/main.lf

# dependencies
[[app.dependencies]]
git = "0.3.2"
tarfetcher = "0.4.2"

# replacement for target properties
[[app.properties]]
cmake-include = "./my-cmake.cmake"
logging = true

# second binary
[[app]]
name = "embedded"
main_reactor = "src/EmbeddedMain.lf"
target = "zephyre"

# [CONTROVERSIAL] AND NOT PART OF THE PROPOSAL
# for projects that are way to complicated to build ourself
# build-script = "./build.sh"
# 
# build-script-inline = '''
# 	barrel generate --out ./src-gen
# 	mkdir build
# 	cd build
# 	cmake ../src-gen
# 	make
# '''

[[app.dependencies]]
blink = "0.1.2"

[[app.properties]]
no-compile = true

@petervdonovan did you say you could provide a Python backend for lingo. For the initial version of lingo only a very shallow shell around lfc is needed.

I dont think I have the access to do it. But lets disallow pushing directly to main?

We current are just wrapping lfc, but we want to provide native build support.

Desired behavior

	$ lingo init cpp
	$ lingo init zephyre

Creates Lingo.toml and ./src/main.lf with the correct target set and potentially other needed files.

Help

=> The explanation refers to #6

• Under ./src/packages/mod.rs is a ConfigFile this struct has a ConfigFile::new() function which generates the struct which is then written into Lingo.toml
• in the ./src/main.rs are the command line arguments handled and files are currently created there but this can be moved.
• in the ./src/args.rs is the expected command line argument schema.

As discussed in todays meeting we should implement proper error handling with our custom LingoError.

I think this should be merged in separate from the init --platform zephyr

In today's (July 25th) meeting, we discussed how lfc could/should be changed to output JSON on the stdout which can be consumed by lingo. The usecase motivating this is currently that for the C target, lfc will generate compile definitions based on the source code, i.e. not only based on the target properties. Currently lfc spits out a CompileDefinitions.txt in the src-gen folder with these compile defs as well. For lingo to generate the CMakeLists.txt by itself, it would need this information. Either by using this CompileDefinitions.txt file or, as discussed today, by changing the output of lfc to be e.g. a JSON which contains this info.

lfc also outputs all sorts of other useful information while compiling which we also need. A potential disadvantage of having it output a JSON is that we might have to wait until the compilation is completed before lfc can output this JSON, this would be very bad for the user experience.