Giter Club home page Giter Club logo

bdemeta's Introduction

CI Coverage

bdemeta

Build and test BDE-style code.

Synopsis

bdemeta walk [-t] CONFIG TARGET [TARGET ...]
bdemeta dot [-t] CONFIG TARGET [TARGET ...]
bdemeta cmake [-p] [-t] CONFIG TARGET [TARGET ...]
bdemeta runtests [-e EXECUTOR] [-m MAX_CASES] [TEST ...]

Description

bdemeta is a set of basic tools to assist building and testing BDE-style source trees. It can generate CMake files for package groups and test drivers within them. It can also invoke BDE-style test drivers.

bdemeta supports finding targets across disconnected directory structures.

Installation

Platforms running Python 3.7 or newer are supported. Install using pip:

$ pip install git+https://github.com/frutiger/bdemeta

Modes

bdemeta runs in one of four modes as given by the first positional argument:

  • walk [-t] CONFIG TARGET [TARGET ...]:
    Walk and topologically sort dependencies

  • dot [-t] CONFIG TARGET [TARGET ...]:
    Generate a directed graph in the DOT language

  • cmake [-p] [-t] CONFIG TARGET [TARGET ...]:
    Generate a CMake lists file

  • runtests [-e EXECUTOR] [-m MAX_CASES] [TEST ...]:
    Run specified or discovered unit tests

Configuration

bdemeta is configured by a JSON configuration file supplied as the first argument to the walk, dot and cmake modes. The configuration is as follows:

{
    "roots": [
        "<root>",
        ...
    ],
    "conan_roots": [
        "<conan_root>",
        ...
    ],
    "standalones": [
        "<standalone>",
        ...
    ],
    "providers": {
        "<target1>: ["<target2>", "<target3>", ...],
        ...
    },
    "runtime_libraries": ["<target4>", "<target5">, ...],
    "pkg_configs": {
        "<target6>": "<pkg1>",
        ...
    },
    "extra_dependencies": {
        "<target7>": ["<target8>", "<target9>", ...],
        ...
    }
}

The meaning of each block is explained below.

Roots

bdemeta will look for targets in directories specified by (possibly multiple) <root>s in the configuration. This makes it easy to build code across multiple BDE-style repositories, including your own. Relative <root>s are considered relative to the path of the configuration file, not relative to the current working directory.

In particular, bdemeta will search for targets by name within each <root> directory:

  • package groups in <root>/groups/<name>
  • standalone packages in <root>/standalones/<name>
  • applications in <root>/applications/<name> (these must contain at least one main file named <name>.m.cpp, but may contain additional components if listed in <root>/applications/<name>/package/<name>.mem)
  • third party CMake packages in:
    • <root>/thirdparty/CMakeLists.txt
    • <root>/CMakeLists.txt

The set of directories searched for standalone packages can be extended by specifying multiple <standalone> directories in the configuration.

Conan roots

Conan will be relied on to provide binaries for each <conan_root> specified. bdemeta will not attempt to build targets that belong to these roots.

Test-only dependencies

Supplying -t (or --test-deps) to the walk, dot or cmake modes will include <target>.t.dep when calculating dependencies of a BDE-style package group or package.

Target providers

A number of third party targets may be specified by a single CMakeLists.txt. However, the dependency from a target is on another target (i.e. library), not on a directory. A "target provider" may be used to specify that a directory containing a CMakeLists.txt will actually provide other targets.

The sample configuration above indicates that the CMakeLists.txt in <target1> will actually provide <target2> and <target3>. This allows bdemeta to consider the targets <target2> and <target3> found once it finds <target1>.

Note that the providers block is optional.

Runtime libraries

Some platforms require undefined symbols to be provided at link time. However, when building plug-in libraries, some symbols are expected to be supplied by the hosting executable at runtime. Enumerating the libraries that contain symbols that will be supplied at runtime allows bdemeta to ensure that any targets that depend those libraries are linked allowing undefined symbols.

The sample configuration indicates that any target depending (transitively or not) on <target4> or <target5> should be linked allowing undefined symbols.

Note that the runtime_libraries block is optional.

Package Config

A target may have its dependencies defined by the pkg-config, already available on the system. The pkg_configs block is consulted to map a target name <target6> to a pkg-config package named <pkg1>. This block is only consulted if the search through every root as described above has been exhausted.

Note that the pkg_configs block is optional.

Extra Dependencies

Often a CMake target or a PkgConfig target may require additional dependencies that bdemeta is expected to resolve. Since bdemeta does not parse CMake files, it needs to be informed about such dependencies. The extra_dependencies block introduces a dependency from <target7> onto <target8>, <target9>, etc.

CMake

For every target specified to the cmake subcommand, bdemeta walks all transitive dependencies based on the configuration described above.

For each BDE-type group or package dependency, bdemeta generates:

  • a CMake library target
  • a CMake executable target for each test driver
  • a CMake custom target comprising all the test drivers, named <name>.t
  • a 'development' install target for the library & headers
  • a 'runtime' install target for the library

For each BDE-type application dependency, bdemeta generates a CMake executable target, comprising the <name>.m.cpp main file in addition to any components listed in <name>.mem.

For each PkgConfig-type dependency, bdemeta generates a CMake interface target consisting of the discovered include directories, compile options and link libraries.

Plugin Tests

Code that is intended to be loaded as a shared library or plugin into another program will often need symbols to provided by the hosting program. bdemeta will generate test targets as shared libraries instead of executables if -p (or --plugin-tests) is supplied to the cmake subcommand.

Running Tests

The runtests subcommand is provided as a helper utility to iterate through multiple test drivers in parallel supporting BDE-style test case status codes. In particular, each driver takes the test case number to run as the first command-line argument and exits with 0 upon success, -1 if there is no such test case and greater than 0 upon failure.

By default, test drivers are executed by the system. If a custom executor is specified with the -e flag, that is invoked instead, supplying the test driver and test case as trailing arguments. The status codes from the custom executor must match the rules described in the previous paragraph.

By default, up to 100 cases will be run per test driver. This can be modified by specifiying the number of desired cases with the -m flag.

License

Copyright (C) 2013 Masud Rahman

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

bdemeta's People

Contributors

dharesign avatar frutiger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

bdemeta's Issues

Distinguish between internal and external cflags

Required to be able to handle plugins statically or structurally linked plugins (.a or a .so bound by the linker as opposed to dlopen/LoadLibrary). The difference between internal and external cflags is that the internal is needed when building the item (plugin or library) while the external are needed when compiling dependent items, such as an executable that includes a structural plugin. But this is not just for plugins, but for all dependencies that are "insulated" by the depended item, therefore their cflags are not needed when building the dependent item. We already have such a library that needs an extra flag option -- because it is built upon a C library that (ab)uses the C preprocessor on ways that look cute first, and when you realize what happens it is too late for you (to regain your sanity -- imagine .c files including .c files and using MACROS to rewrite them).

We may just introduce a local .bdemetarc for the above kind of C code that touches itself in ways no code should do in polite company, because those are "static". However such tricks does not save us from the headaches of plugins: where we have 2 sets of dependencies. One is for building the plugin and the other is for using the plugin. My plugin may need the mabi_mandarin implementation, but the whole point of the plugin (which should rather be called "companion"...) is to not makes the executable dependent on any of the plugin's specifics. Shared-library plugins will contain all their dependencies, while static ones will require us to link them into the final executable. However neither will require us to add any flag dependencies to the building of the executable.

Current bdemeta does not support the distinction between inner and outer dependencies. We may decide that for plugins we will treat every dependency as inner, and users must declare the dependencies explicitly that are necessary for using the plugin (such as dependencies for some protocols); but even in that case, internally, bdemeta will need to have two kinds of cflags: the this-is-what-I-need-to-build-the-thing and the this-is-what-is-passed-on-to-my-dependents.

I suggest that we call such dependencies _insulated dependencies_ (note that it is not insulted) as they are called in Lakosia; as it all so clearly describes what our plugins do: insulate our main program from the need to know about every subsystem.

Sorry for being so verbose.

external_cflags prefix all other flags

If I have a .bdemetarc file which looks like:

{
    "roots": [
        "/some/path/containing/A"
    ],
    "units": {
        "#universal": {
            "external_cflags": [
                "-I/some/other/path/containing/A_headers"
            ]
        }
    }
}

Then the resulting ninja flags looks like:

flags = -I/some/other/path/containing/A_headers -I/some/path/containing/A/package

Which means it picks up the out-of-date versions of the headers. Really the #universal stuff should be sorted to the back, as I think the README even suggests:

The optionally specified <univ_cflag> is appended when generating cflags for any component.

Provide a -D flag specifying plugin type

When plugins are being built, they may be built in one of three modes (see #12, #13, and #14). Provide a -D rule with well-known values for code being compiled under the plugin so that it can be aware of which mode it is being compiled in.

This issue was originally reported in #7.

Support plugins automatically loaded at runtime

A plugin automatically loaded at runtime is a shared library that is provided as a link-time dependency of some other code. This other code may then freely use symbols from the plugin as long the name of the exported symbols are known (e.g. through header files).

These shared libraries would have to be deployed together with the code using them in order to avoid mismatched symbol names and symbol signatures/behaviours. This stands in contrast to #14 where the resulting binary contains all the requisite dependencies.

This would implement option (2) from #7.

Support plugins available at link time

A plugin available at link time is a static library that is provided as a link-time dependency of some other code. This other code may then freely use functions from the library as long the name of the exported symbols are known (e.g. through header files).

The code may also then be deployed without any further requirements. This stands in contrast to #13, which requires the using code and the shared library to be deployed together.

This would implement option (3) from #7.

Solve the issue of multiple kinds of targets

When I am making a plugin I may have 3 kinds of different ways of building it.

  1. A shared library (that I am going to open with dlopen/LoadLibrary).
  2. A shared library that I link in structurally, just like the C runtime is linked in.
  3. A static library that I fully link in, as if linking in all of its objects.

Side note: These 3 might require differences in the source, which we may have to contain with #if ugliness (unless we can hide the differences in components -- not enough experiences to say which will be true). We assume the "worst case" for the build system: we build all 3 variant from the same source code. In which case we will need:

  1. A way to say (with the executable) not only what plugins we want to build with it, but which variant of the 3
  2. In the plugin sources we would like to know what are we building, so a adding some -Dtarget-type-thing sounds like a good idea
  3. We probably don't want to build all variants of a plugin, just what is asked for, when building for production.
  4. We don't want to build all variants of plugins even when testing, because not all variants may be supported; so we need some sort of capabilities info as well.

The problem is that, at the moment, as a programmer, I cannot have a static and a dynamic plugin, unless I edit the source code of bdemeta; as the tool has no concept of multiple different outputs. So plugins (all plugins) are either static or dynamic. So to step closer to a solution we need to be able to specify for each element (group, plugin) what kind of targets do they support and their users (the dependent executable) should be able to specify what flavor do they want. (This is somewhat similar to thread-aware/single-threaded variants. We don't need groups ending up being shared libraries right now, but BDE is already doing that for Windows.) Whatever way we do these in the metafiles, it has to be the same how BDE wants to do it -- assuming they have one.

Support plugins explicitly loaded at runtime

A plugin explicitly loaded at runtime is a dynamically loadable module that exports some set of symbols. Other code may then load and use those symbols using operating system provided facilities (e.g. dlopen or LoadLibrary) if it knows the names and types of the exported symbols.

These dynamically loadable modules would have to be deployed together with the code using them in order to avoid mismatched symbol names and symbol signatures/behaviours.

This would implement option (1) from #7.

Support for CXXFLAGS

The cflags variables are used for both C and C++ files, which prevents specifying options which are relevant to C++ only, e.g., -std=c++03. We may want to add equivalent cxxflags variables.

Support standalone packages

Add support for the various types of standalone packages:

adapters/
  a_foo/
applications/
  foo/
groups/
  s_foo/

Handle spaces in roots

Currently the path used in a root cannot contain a space, at least not if done via the .bdemetarc file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.