p4c is a reference compiler for the P4 programming language. It supports both P4-14 and P4-16; you can find more information about P4 here and the specifications for both versions of the language here. One fact attesting to the level of quality and completeness of p4c's code is that its front-end code, mid-end code, and p4c-graphs back end are used as the basis for at least one commercially supported P4 compiler.
p4c is modular; it provides a standard frontend and midend which can be combined with a target-specific backend to create a complete P4 compiler. The goal is to make adding new backends easy.
The code contains seven sample backends:
- p4c-bm2-ss: can be used to target the P4
simple_switch
written using the BMv2 behavioral model https://github.com/p4lang/behavioral-model, - p4c-dpdk: can be used to target the DPDK software switch (SWX) pipeline https://doc.dpdk.org/guides/rel_notes/release_20_11.html,
- p4c-ebpf: can be used to generate C code which can be compiled to eBPF and then loaded in the Linux kernel. The eBPF backend currently implements two architecture models: ebpf_model.p4 for packet filtering and the fully-featured PSA (Portable Switch Architecture) model.
- p4test: a source-to-source P4 translator which can be used for testing, learning compiler internals and debugging,
- p4c-graphs: can be used to generate visual representations of a P4 program; for now it only supports generating graphs of top-level control flows, and
- p4c-ubpf: can be used to generate eBPF code that runs in user-space.
- p4tools: a platform for P4 test utilities, including a test-case generator for P4 programs. Sample command lines:
Compile P4_16 or P4_14 source code. If your program successfully
compiles, the command will create files with the same base name as the
P4 program you supplied, and the following suffixes instead of the
.p4
:
- a file with suffix
.p4i
, which is the output from running the preprocessor on your P4 program. - a file with suffix
.json
that is the JSON file format expected by BMv2 behavioral modelsimple_switch
.
p4c --target bmv2 --arch v1model my-p4-16-prog.p4
p4c --target bmv2 --arch v1model --std p4-14 my-p4-14-prog.p4
By adding the option --p4runtime-files <filename>.txt
as shown in
the example commands below, p4c will also create a file
<filename>.txt
. This is a text format "P4Info" file, containing a
description of the tables and other objects in your P4 program that
have an auto-generated control plane API.
p4c --target bmv2 --arch v1model --p4runtime-files my-p4-16-prog.p4info.txt my-p4-16-prog.p4
p4c --target bmv2 --arch v1model --p4runtime-files my-p4-14-prog.p4info.txt --std p4-14 my-p4-14-prog.p4
All of these commands take the --help
argument to show documentation
of supported command line options. p4c --target-help
shows the
supported "target, arch" pairs.
p4c --help
p4c --target-help
Auto-translate P4_14 source to P4_16 source:
p4test --std p4-14 my-p4-14-prog.p4 --pp auto-translated-p4-16-prog.p4
Check syntax of P4_16 or P4_14 source code, without limitations that might be imposed by any particular compiler back end. There is no output for these commands other than error and/or warning messages.
p4test my-p4-16-prog.p4
p4test --std p4-14 my-p4-14-prog.p4
Generate GraphViz ".dot" files for parsers and controls of a P4_16 or P4_14 source program.
p4c-graphs my-p4-16-prog.p4
p4c-graphs --std p4-14 my-p4-14-prog.p4
Generate PDF of parser instance named "ParserImpl" generated by the
p4c-graphs
command above (search for graphviz below for its install
instructions):
dot -Tpdf ParserImpl.dot > ParserImpl.pdf
p4c has package support for several Ubuntu and Debian distributions.
A p4c package is available in the following repositories for Ubuntu 20.04 and newer.
. /etc/os-release
echo "deb https://download.opensuse.org/repositories/home:/p4lang/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/home:p4lang.list
curl -L "https://download.opensuse.org/repositories/home:/p4lang/xUbuntu_${VERSION_ID}/Release.key" | sudo apt-key add -
sudo apt-get update
sudo apt install p4lang-p4c
For Debian 11 (Bullseye) it can be installed as follows:
echo 'deb https://download.opensuse.org/repositories/home:/p4lang/Debian_11/ /' | sudo tee /etc/apt/sources.list.d/home:p4lang.list
curl -fsSL https://download.opensuse.org/repositories/home:p4lang/Debian_11/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/home_p4lang.gpg > /dev/null
sudo apt update
sudo apt install p4lang-p4c
If you cannot use a repository to install p4c, you can download the .deb
file
for your release and install it manually. You need to download a new file each
time you want to upgrade p4c.
-
Go to https://build.opensuse.org/package/show/home:p4lang/p4lang-p4c, click on "Download package" and choose your operating system version.
-
Install p4c, changing the path below to the path where you downloaded the package.
sudo dpkg -i /path/to/package.deb
-
Clone the repository. It includes submodules, so be sure to use
--recursive
to pull them in:git clone --recursive https://github.com/p4lang/p4c.git
If you forgot
--recursive
, you can update the submodules at any time using:git submodule update --init --recursive
-
Install dependencies. You can find specific instructions for Ubuntu 20.04 here and for macOS 11 here. You can also look at the CI installation script.
-
Build. Building should also take place in a subdirectory named
build
.mkdir build cd build cmake .. <optional arguments> make -j4 make -j4 check
The cmake command takes the following optional arguments to further customize the build:
-DCMAKE_BUILD_TYPE=RELEASE|DEBUG
-- set CMAKE_BUILD_TYPE to RELEASE or DEBUG to build with optimizations or with debug symbols to run in gdb. Default is RELEASE.-DCMAKE_INSTALL_PREFIX=<path>
-- set the directory wheremake install
installs the compiler. Defaults to /usr/local.-DENABLE_BMV2=ON|OFF
. Enable the bmv2 backend. Default ON.-DENABLE_EBPF=ON|OFF
. Enable the ebpf backend. Default ON.-DENABLE_UBPF=ON|OFF
. Enable the ubpf backend. Default ON.-DENABLE_DPDK=ON|OFF
. Enable the DPDK backend. Default ON.-DENABLE_P4C_GRAPHS=ON|OFF
. Enable the p4c-graphs backend. Default ON.-DENABLE_P4TEST=ON|OFF
. Enable the p4test backend. Default ON.-DENABLE_TEST_TOOLS=ON|OFF
. Enable the p4tools backend. Default OFF.-DENABLE_DOCS=ON|OFF
. Build documentation. Default is OFF.-DENABLE_GC=ON|OFF
. Enable the use of the garbage collection library. Default is ON.-DENABLE_GTESTS=ON|OFF
. Enable building and running GTest unit tests. Default is ON.-DENABLE_PROTOBUF_STATIC=ON|OFF
. Enable the use of static protobuf libraries. Default is ON.-DENABLE_MULTITHREAD=ON|OFF
. Use multithreading. Default is OFF.-DBUILD_LINK_WITH_GOLD=ON|OFF
. Use Gold linker for build if available.-DBUILD_LINK_WITH_LLD=ON|OFF
. Use LLD linker for build if available (overrides BUILD_LINK_WITH_GOLD).-DENABLE_LTO=ON|OFF
. Use Link Time Optimization (LTO). Default is OFF.-DENABLE_WERROR=ON|OFF
. Treat warnings as errors. Default is OFF.
If adding new targets to this build system, please see instructions.
-
(Optional) Install the compiler and the P4 shared headers globally.
sudo make install
The compiler driver
p4c
and binaries for each of the backends are installed in/usr/local/bin
by default; the P4 headers are placed in/usr/local/share/p4c
. -
You're ready to go! You should be able to compile a P4-16 program for BMV2 using:
p4c -b bmv2-ss-p4org program.p4 -o program.bmv2.json
If you plan to contribute to p4c, you'll find more useful information here.
Ubuntu 20.04 is the officially supported platform for p4c. There's also unofficial support for macOS 11. Other platforms are untested; you can try to use them, but YMMV.
-
A C++17 compiler. GCC 9.1 or later or Clang 6.0 or later is required.
-
git
for version control -
GNU autotools for the build process
-
CMake 3.10.2 or higher
-
Boehm-Weiser garbage-collector C++ library
-
GNU Bison and Flex for the parser and lexical analyzer generators.
-
Google Protocol Buffers 3.0 or higher for control plane API generation
-
C++ boost library (minimally used)
-
Python 3 for scripting and running tests
-
Optional: Documentation generation (enabled when configuring with --enable-doxygen-doc) requires Doxygen (1.8.10 or higher) and Graphviz (2.38.0 or higher).
Backends may have additional dependencies. The dependencies for the backends
included with p4c
are documented here:
Most dependencies can be installed using apt-get install
:
sudo apt-get install cmake g++ git automake libtool libgc-dev bison flex \
libfl-dev libboost-dev libboost-iostreams-dev \
libboost-graph-dev llvm pkg-config python3 python3-pip \
tcpdump
pip3 install ipaddr scapy ply "clang-format>=15.0.4"
For documentation building:
sudo apt-get install -y doxygen graphviz texlive-full
p4c
also depends on Google Protocol Buffers (Protobuf). p4c
requires version
3.0 or higher, so the packaged version provided in Ubuntu 20.04 should
work. However, all our CI testing is done with a more recent version of Protobuf
(at the moment, 3.18.1), which we install from source. If you are experiencing
issues with the Protobuf version shipped with your OS distribution, we recommend
that we install Protobuf 3.18.1 from source. You can find instructions
here.
After cloning Protobuf and before you build, check-out version 3.18.1:
git checkout v3.18.1
Please note that while all Protobuf versions newer than 3.0 should work for
p4c
itself, you may run into trouble with some extensions and other p4lang
projects unless you install version 3.18.1.
sudo dnf install -y cmake g++ git automake libtool gc-devel bison flex \
libfl-devel gmp-devel boost-devel boost-iostreams boost-graph llvm pkg-config \
python3 python3-pip tcpdump protobuf-devel protobuf-static
sudo pip3 install scapy ply
For documentation building:
sudo dnf install -y doxygen graphviz texlive-scheme-full
You can also look at the dependencies installation script for a fresh Fedora instance.
Installing on macOS:
-
Enable XCode's command-line tools:
xcode-select --install
-
Install Homebrew:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Be sure to add
/usr/local/bin/
to your$PATH
. -
Install dependencies using Homebrew:
brew install autoconf automake libtool bdw-gc boost bison pkg-config
By default, Homebrew doesn't link programs into
/usr/local/bin
if they would conflict with a version provided by the base system. This includes Bison, since an older version ships with macOS.make check
depends on the newer Bison we just installed from Homebrew (see #83), so you'll want to add it to your$PATH
one way or another. One simple way to do that is to request that Homebrew link it into/usr/local/bin
:brew link --force bison
Optional documentation building tools:
brew install doxygen graphviz
Homebrew offers a
protobuf
formula. It installs version 3.2, which should work for p4c itself but may cause problems with some extensions. It's preferable to install Protocol Buffers 3.0 from source using the instructions here. Check out the newest tag in the 3.0 series (v3.0.2
as of this writing) before you build.
P4c relies on BDW garbage collector
to manage its memory. By default, the p4c executables are linked with
the garbage collector library. When the GC causes problems, this can
be disabled by setting ENABLE_GC
cmake option to OFF
. However,
this will dramatically increase the memory usage by the compiler, and
may become impractical for compiling large programs. Do not disable
the GC, unless you really have to. We have noticed that this may be
a problem on MacOS.
There is a variety of design and development documentation here.
We recommend using clang++
with no optimizations for speeding up
compilation and simplifying debugging.
We recommend installing a new version of gdb., because older gdb versions do not always handle C++11 or newer correctly.
We recommend exuberant ctags for navigating source code in Emacs and vi. sudo apt-get install exuberant-ctags.
The Makefile targets make ctags
and make etags
generate tags for vi and Emacs respectively. (Make sure that you are
using the correct version of ctags; there are several competing programs with
the same name in existence.)
To enable building code documentation, please run cmake .. -DENABLE_DOCS=ON
. This enables the make docs
rule to generate
documentation. The HTML output is available in
build/doxygen-out/html/index.html
.
Occasionally formatting commits are applied to P4C. These pollute the git history. To ignore these commits in git blame, run this command
git config blame.ignoreRevsFile .git-blame-ignore-revs
A Dockerfile is included. You can generate an image which contains a copy of p4c
in /p4c/build
by running:
docker build -t p4c .
On some platforms Docker limits the memory usage of any container, even
containers used during the docker build
process. On macOS in particular the
default is 2GB, which is not enough to build p4c. Increase the memory limit to
at least 4GB via Docker preferences or you are likely to see "internal compiler
errors" from gcc which are caused by low memory.
The project can also be build using Bazel:
bazel build //...
We run continuous integration to ensure this works with the latest version of Bazel.
We also provide a p4_library
rule for invoking
p4c during the build process of 3rd party Bazel projects.
See bazel/example for an example of how to use or extend p4c in your own Bazel project. You may use it as a template to get you started.
The build system is based on cmake. This section describes how it can be customized.
When building a new backend target, add it into the development tree in the extensions subdirectory. The cmake-based build system will automatically include it if it contains a CMakeLists.txt file.
For a new backend, the cmake file should contain the following rules:
Backend specific IR definition files should be added to the global list of IR_DEF_FILES as they are processed together with the core IR files. Use the following rule:
set (IR_DEF_FILES ${IR_DEF_FILES} ${MY_IR_DEF_FILES} PARENT_SCOPE)
where MY_IR_DEF_FILES
is a list of file names with absolute path
(for example, use ${CMAKE_CURRENT_SOURCE_DIR}
).
If in addition you have additional supporting source files, they should be added to the frontend sources, as follows:
set(EXTENSION_FRONTEND_SOURCES ${EXTENSION_FRONTEND_SOURCES} ${MY_IR_SRCS} PARENT_SCOPE)
Again, MY_IR_SRCS
is a list of file names with absolute path.
Sources (.cpp and .h) should be added to the cpplint and clang_format target using the following rule:
add_cpplint_files (${CMAKE_CURRENT_SOURCE_DIR} "${MY_SOURCES_AND_HEADERS}")
add_clang_format_files (${CMAKE_CURRENT_SOURCE_DIR} "${MY_SOURCES_AND_HEADERS}")
where mybackend
is the name of the directory you added under extensions.
The p4c CMakeLists.txt will use that name to figure the full path of the files to lint.
Unlike cpplint, clang-format needs to be installed before the linter can be used. clang-format can be installed with the following command:
pip3 install --user "clang-format>=15.0.4"
clang-format can be checked using the make clang-format
command. Complaints can be fixed by running make clang-format-fix-errors
.
Both cpplint and clang-format run as checks as port of P4C's continuous integration process. To make sure that these tests pass, we recommend installing the appropriate git hooks. This can be done by running
./tools/install_git_hooks.sh
clang-format and cpplint checks will be enforced on every branch commit. In cases where checks are failing but the commit is sound, one can bypass the hook enforcement using git commit --no-verify
.
Define a target for your executable. The target should link against
the core P4C_LIBRARIES
and P4C_LIB_DEPS
. P4C_LIB_DEPS
are
package dependencies. If you need additional libraries for your
project, add them to P4C_LIB_DEPS
.
In addition, your target should depend on the genIR
target, since
you need all the IR generation to happen before you start compiling
your backend. If you chose to have your backend as a library (seem the
backends/bmv2 example), the library should depend on genIR
, and
there is no longer necessary for your executable to depend on it.
add_executable(p4c-mybackend ${MY_SOURCES})
target_link_libraries (p4c-mybackend ${P4C_LIBRARIES} ${P4C_LIB_DEPS})
add_dependencies(p4c-mybackend genIR)
We implemented support equivalent to the automake make check
rules.
All tests should be included in make check
and in addition, we support
make check-*
rules. To enable this support, add the following rules:
set(MY_DRIVER <driver or compiler executable>)
set (MY_TEST_SUITES
${P4C_SOURCE_DIR}/testdata/p4_16_samples/*.p4
${P4C_SOURCE_DIR}/testdata/p4_16_errors/*.p4
)
set (MY_XFAIL_TESTS
testdata/p4_16_errors/this_test_fails.p4
)
p4c_add_tests("mybackend" ${MY_DRIVER} "${MY_TEST_SUITES}" "${MY_XFAIL_TESTS}")
In addition, you can add individual tests to a suite using the following macro:
set(isXFail FALSE)
set(SWITCH_P4 testdata/p4_14_samples/switch_20160512/switch.p4)
p4c_add_test_with_args ("mybackend" ${MY_DRIVER} ${isXFail}
"switch_with_custom_profile" ${SWITCH_P4} "-DCUSTOM_PROFILE")
See the documentation for
p4c_add_test_with_args
and
p4c_add_tests
for more information on the
arguments to these macros.
To pass custom arguments to p4c, you can set the environment variable P4C_ARGS
:
make check P4C_ARGS="-Xp4c=MY_CUSTOM_FLAG"
When making changes to p4c, it is sometimes useful to be able to run
the tests while overwriting the expected output files that are saved
in this repository. One such situation is when your changes to p4c
cause the names of compiler-generated local variables to change. To
force the expected output files to be rewritten while running the
tests, assign a value to the shell environment variable
P4TEST_REPLACE
. Here is one example Bash command to do so:
P4TEST_REPLACE=1 make check
Define rules to install your backend. Typically you need to install the binary, the additional architecture headers, and the configuration file for the p4c driver.
install (TARGETS p4c-mybackend
RUNTIME DESTINATION ${P4C_RUNTIME_OUTPUT_DIRECTORY})
install (DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/p4include
DESTINATION ${P4C_ARTIFACTS_OUTPUT_DIRECTORY})
install (FILES ${CMAKE_CURRENT_SOURCE_DIR}/driver/p4c.mybackend.cfg
DESTINATION ${P4C_ARTIFACTS_OUTPUT_DIRECTORY}/p4c_src)
Issues with the compiler are tracked on GitHub. Before opening a new issue, please check whether a similar issue is already opened. Opening issues and submitting a pull request with fixes for those issues is much appreciated.
In addition to the list of issues on Github, there are a number of currently unsupported features listed below:
-
extern/blackbox attributes -- there is support for carrying them in the IR, but they are lost if P4_16 code is output. Backends can access them from the IR
-
Nonstandard extension primitives from P4_14
- Execute_meter extra arguments
- Recirculate/clone/resubmit variants
- Bypass_egress
- Sample_ primitives
- invalidate
-
No support for P4_14 parser exceptions.
- Tables with multiple apply calls
See also unsupported P4_16 language features.