ovn-org / ovn Goto Github PK

View Code? Open in Web Editor NEW

471.0 41.0 239.0 92.23 MB

Open Virtual Network

License: Apache License 2.0

Shell 3.65% Makefile 1.18% Ruby 0.04% M4 2.09% Python 1.78% C 91.03% Roff 0.06% Dockerfile 0.18%

ovn's Introduction

OVN

https://readthedocs.org/projects/ovn/badge/?version=latest

What is OVN?

OVN (Open Virtual Network) is a series of daemons that translates virtual network configuration into OpenFlow, and installs them into Open vSwitch. It is licensed under the open source Apache 2 license.

OVN provides a higher-layer abstraction than Open vSwitch, working with logical routers and logical switches, rather than flows. OVN is intended to be used by cloud management software (CMS). For details about the architecture of OVN, see the ovn-architecture manpage. Some high-level features offered by OVN include:

Distributed virtual routers
Distributed logical switches
Access Control Lists
DHCP
DNS server

Like Open vSwitch, OVN is written in platform-independent C. OVN runs entirely in userspace and therefore requires no kernel modules to be installed.

Until recently, OVN code lived within the Open vSwitch codebase. OVN has recently been split into its own repo. There is much to do to complete this split entirely. See the TODO_SPLIT.rst file for a list of known tasks that need to be completed.

What's here?

The main components of this distribution are:

ovn-northd, a centralized daemon that translates northbound configuration from a CMS into logical flows for the southbound database.
ovn-controller, a daemon that runs on every hypervisor in the cluster. It translates the logical flows in the southbound database into OpenFlow for Open vSwitch. It also handles certain traffic, such as DHCP and DNS.
ovn-nbctl, a tool for interfacing with the northbound database.
ovn-sbctl, a tool for interfacing with the southbound database.
ovn-trace, a debugging utility that allows for tracing of packets through the logical network.
ovn-debug, a tool to simplify debugging of OVN setup.
Scripts and specs for building RPMs.

What other documentation is available?

To install OVN on a regular Linux or FreeBSD host, please read the installation guide. For specifics around installation on a specific platform, refer to one of the other installation guides

For answers to common questions, refer to the FAQ.

To learn about some advanced features of the Open vSwitch software switch, read the tutorial.

Each OVN program is accompanied by a manpage. Many of the manpages are customized to your configuration as part of the build process, so we recommend building OVN before reading the manpages.

License

The following is a summary of the licensing of files in this distribution. As mentioned, OVN is licensed under the open source Apache 2 license. Some files may be marked specifically with a different license, in which case that license applies to the file in question.

File build-aux/cccl is licensed under the GNU General Public License, version 2.

Contact

[email protected]

ovn's People

Contributors

Stargazers

Watchers

Forkers

ovsrobot numansiddique wlm328cs fantastic2085 ryzhyk robbbit vadorovsky noah8713 dceara hzhou8 lorenzobianconi dvandra flavio-fernandes djlwilder russellb dcbw zwjhuhu peter-teslenko cubeek fnordahl venuiyer midu16 kot-begemot-uk dolphincc larsks hehuatang cathy-zhou abhat booxter javacruft vishnoianil kubeovn zhuoqiang08 rwmacleod venkataanil akraino-icn thomasnew xbzhang99 mccv1r0 igsilya mangelajo danalsan cailiyutest dave-tucker jijianwen yang123vc jacksunshine shenjiangc odivlad ryandgoulding oilbeater ateleshev girishmg th3architect mauroseb huangxiangxiangpmlabs mimoccc crackercat ckji 5gapp gabrielecerami cybertan ikukaku narahari92 tenxcloud gaozhengwei markdgray abembmyum luis5tb ruoniao robinbraemer hughnhan lk668 xiaoruiguo winsopc aojea slawqo fydrah gokube cygmris sleepless-poet shubhamoli laik timyinshi deadalnix vatsan62 yu2003w phvalguima gongmingchen 10255447 1900gm sunny821 vjayaramrh roytman otherwiseguy msherif1234 spiderdetective shh0613 ruicao93 hunchback

ovn's Issues

where can i get the document

I can get the document in http://www.openvswitch.org/support/dist-docs-2.5/，but the version is 2.5.0,i want the 2.14.x document,and where can i get it ?thank you

OVN performance needs to be optimized

Test Environment

OVN Version：20.03
OVN_NB：cluster mode with 3 nodes
OVN_SB：cluster mode with 3nodes
OVN-Northd：active and standby mode with 3 nodes
OVN-Controller：33 nodes
OS：CentOS Linux release 8.0.1905 (Core)
Kernel：4.18.0-80.el8.x86_64

1. About ovn-northd process
In order to find the changed data, each loop in the main function will load the full amount of data from ovnnb_db and ovnsb_db. When the amount of data is not large, such as created 2000 switches or 2000 ports, the CPU corresponding to the ovn-northd process is 100% busy, regardless of whether there are data changes or new operations. At this time, the CPUs of ovnnb_db and ovnsb_db have been kept at about 50%.
This mechanism of full load and full check has seriously affected the performance of ovn. It is recommended to change the mechanism of detecting incremental change data to reduce the amount of calculation and improve performance.

2. About ovn-nbctl client tools
Any command executed via ovn-nbctl will load the full amount of data from ovnnb_db to the client, regardless of whether the command requires these data. When the amount of data is large, this command will take a long time to execute.
When the number of ports under the switch reaches 9197, any addition command, such as ls-add, will report "database connection failed (Broken pipe)" error。

I want to know whether our OVN technical roadmap will make optimization in the above aspects。

Is it a typo in check the validity of nat->logical_ip?

https://github.com/ovn-org/ovn/blob/master/northd/ovn-northd.c#L8831

For ipv4 used nat->logical_ip, but for ipv6 nat->external_ip:

            /* Check the validity of nat->logical_ip. 'logical_ip' can
             * be a subnet when the type is "snat". */
            if (is_v6) {
                error = ipv6_parse_masked(nat->external_ip, &ipv6, &mask_v6);
            } else {
                error = ip_parse_masked(nat->logical_ip, &ip, &mask);
            }

Can some one explain, why do we use nat->external_ip here?

ICMP6 time exceeded in-transit response not sent from OVN router when source is external

We've noticed that ICMP6 time exceeded in-transit response packets are not sent from the intermediate OVN logical router hop when an MTR trace is started from an external source address targeted at an IP inside the OVN logical network (all NAT disabled).

If the source address is changed to an IP inside the same subnet as the OVN router's external port address then the OVN router responds with the expected ICMP6 time exceeded in-transit response.

Example setup:

Container connected to a bridge called "lxdbr1" with an IP of fd42:cfe0:8030:65c4:216:3eff:fe07:96a3/64.

An OVN network behind an OVN logical router, with a provider port connected to another bridge called "lxdbr0". The OVN logical router port's IP address on the "lxdbr0" subnet is fd42:6610:36a8:1234:216:3eff:fe72:513b (a different /64 than lxdbr1).

The OVN router also has a static route added to it to route fd0b:bc39:4820:d4f8::/64 to a container connected to a logical switch port. This container has the IP setup of fd0b:bc39:4820:d4f8::1/64.

On the Linux host running the two bridges, there is also a static route of fd0b:bc39:4820:d4f8::1/64 configured to route to the OVN's external IP of fd42:6610:36a8:1234:216:3eff:fe72:513b.

If I run an MTR trace from the container connected to lxdbr1, with a source address of fd42:cfe0:8030:65c4:216:3eff:fe07:96a3/64 to the OVN container's address of fd0b:bc39:4820:d4f8::1/64, then we see this result:

                            My traceroute  [v0.93]
c1 (fd42:cfe0:8030:65c4:216:3eff:fe07:96a3)           2020-12-10T12:32:21+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                      Packets               Pings
 Host                               Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. fd42:cfe0:8030:65c4::1           0.0%    11    0.2   0.2   0.1   0.2   0.0
 2. (waiting for reply)
 3. fd0b:bc39:4820:d4f8::1           0.0%    10    0.2   0.3   0.2   1.1   0.3

The OVN router in the intermediate hop is not responding with the ICMP6 time exceeded in-transit response. A tcpdump trace shows two ICMP6 echo requests, one for the OVN router and one of the target IP, but only 1 reply packet coming from the target IP, and no response from the OVN router.

sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:34:16.659989 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33020, length 24
12:34:16.994012 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33021, length 24
12:34:16.994779 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33021, length 24
12:34:17.662156 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33023, length 24
12:34:17.996195 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33024, length 24
12:34:17.996258 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33024, length 24
12:34:18.664205 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33026, length 24
12:34:18.998078 IP6 fd42:cfe0:8030:65c4:216:3eff:fe07:96a3 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33027, length 24
12:34:18.998118 IP6 fd0b:bc39:4820:d4f8::1 > fd42:cfe0:8030:65c4:216:3eff:fe07:96a3: ICMP6, echo reply, seq 33027, length 24

However if I then setup an IPv6 NAT rule on lxdbr1 to translate the source to the host's IP on lxdbr0 (fd42:6610:36a8:1234::1) then we start seeing the expected responses:

                                     My traceroute  [v0.93]
c1 (fd42:cfe0:8030:65c4:216:3eff:fe07:96a3)                               2020-12-10T12:35:34+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                          Packets               Pings
 Host                                                   Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. fd42:cfe0:8030:65c4::1                               0.0%    17    0.1   0.2   0.1   0.2   0.0
 2. fd42:6610:36a8:1234:216:3eff:fe72:513b               0.0%    17    1.1   1.0   0.4   1.5   0.3
 3. fd0b:bc39:4820:d4f8::1                               0.0%    16    0.2   0.2   0.1   1.1   0.2

And a tcpdump shows the expected replies:

sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:35:52.782198 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33104, length 24
12:35:52.783417 IP6 fd42:6610:36a8:1234:216:3eff:fe72:513b > fd42:6610:36a8:1234::1: ICMP6, time exceeded in-transit for fd0b:bc39:4820:d4f8::1, length 72
12:35:53.116093 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33105, length 24
12:35:53.116787 IP6 fd0b:bc39:4820:d4f8::1 > fd42:6610:36a8:1234::1: ICMP6, echo reply, seq 33105, length 24
12:35:53.784050 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33107, length 24
12:35:53.785148 IP6 fd42:6610:36a8:1234:216:3eff:fe72:513b > fd42:6610:36a8:1234::1: ICMP6, time exceeded in-transit for fd0b:bc39:4820:d4f8::1, length 72
12:35:54.117792 IP6 fd42:6610:36a8:1234::1 > fd0b:bc39:4820:d4f8::1: ICMP6, echo request, seq 33108, length 24
12:35:54.117833 IP6 fd0b:bc39:4820:d4f8::1 > fd42:6610:36a8:1234::1: ICMP6, echo reply, seq 33108, length 24

So we were wondering is this a bug, or is this expected behaviour?

100% CPU utilization of the ovn-northd process

Test Environment

OVN Version：20.03
OVN_NB：Standalone Mode
OVN_SB：Standalone Mode
OVN-Northd：Standalone Mode
OVN-Controller：33 nodes
OS：CentOS Linux release 8.0.1905 (Core)
Kernel：4.18.0-80.el8.x86_64

Recently, OVN performance pressure test is being carried out, this time based on standalone mode.

After the stress test, the CPU utilization of the ovn-northd process has been very high, close to 100%, but the CPU utilization of the OVN-NB/OVN-SB process is very low, close to 0.

1. The OVN-NB log during the stress test as follows

2020-06-29T02:01:09.667Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.12.0
2020-06-29T02:01:19.677Z|00003|memory|INFO|6656 kB peak resident set size after 10.0 seconds
2020-06-29T02:01:19.677Z|00004|memory|INFO|cells:31 monitors:2 sessions:1
2020-06-29T02:48:42.312Z|00005|memory|INFO|peak resident set size grew 56% in last 2842.6 seconds, from 6656 kB to 10352 kB
2020-06-29T02:48:42.312Z|00006|memory|INFO|cells:34572 monitors:2 sessions:2
2020-06-29T02:50:32.331Z|00007|memory|INFO|peak resident set size grew 54% in last 110.0 seconds, from 10352 kB to 15956 kB
2020-06-29T02:50:32.332Z|00008|memory|INFO|cells:139273 monitors:2 sessions:2
2020-06-29T02:53:52.381Z|00009|memory|INFO|peak resident set size grew 52% in last 200.0 seconds, from 15956 kB to 24316 kB
2020-06-29T02:53:52.381Z|00010|memory|INFO|cells:299166 monitors:2 sessions:2
2020-06-29T03:00:22.523Z|00011|memory|INFO|peak resident set size grew 52% in last 390.1 seconds, from 24316 kB to 36872 kB
2020-06-29T03:00:22.523Z|00012|memory|INFO|cells:537462 monitors:2 sessions:2
2020-06-29T03:00:32.532Z|00013|memory|INFO|peak resident set size grew 116% in last 10.0 seconds, from 36872 kB to 79768 kB
2020-06-29T03:00:32.532Z|00014|memory|INFO|cells:542471 monitors:2 sessions:2
2020-06-29T03:16:32.928Z|00015|memory|INFO|peak resident set size grew 79% in last 960.4 seconds, from 79768 kB to 142532 kB
2020-06-29T03:16:32.928Z|00016|memory|INFO|cells:918878 monitors:2 sessions:2
2020-06-29T04:03:20.635Z|00017|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.651Z|00018|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.651Z|00019|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.662Z|00020|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.707Z|00021|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.713Z|00022|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.727Z|00023|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.748Z|00024|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:20.748Z|00025|poll_loop|INFO|wakeup due to [POLLIN] on fd 22 (/var/run/ovn/ovnnb_db.sock<->) at lib/stream-fd.c:157 (52% CPU usage)
2020-06-29T04:03:20.763Z|00026|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (52% CPU usage)
2020-06-29T04:03:25.019Z|00027|memory|INFO|peak resident set size grew 54% in last 2812.1 seconds, from 142532 kB to 219124 kB
2020-06-29T04:03:25.019Z|00028|memory|INFO|cells:1599256 monitors:2 sessions:2

2. The OVN-SB log during the stress test as follows

2020-06-29T02:01:09.728Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.12.0
2020-06-29T02:01:19.738Z|00003|memory|INFO|6656 kB peak resident set size after 10.0 seconds
2020-06-29T02:01:19.738Z|00004|memory|INFO|cells:227 monitors:2 sessions:1
2020-06-29T02:37:25.624Z|00005|jsonrpc|WARN|unix#2: send error: Broken pipe
2020-06-29T02:37:25.625Z|00006|reconnect|WARN|unix#2: connection dropped (Broken pipe)
2020-06-29T02:37:27.489Z|00007|jsonrpc|WARN|unix#3: send error: Broken pipe
2020-06-29T02:37:27.490Z|00008|reconnect|WARN|unix#3: connection dropped (Broken pipe)
2020-06-29T02:39:56.123Z|00009|jsonrpc|WARN|unix#6: receive error: Connection reset by peer
2020-06-29T02:39:56.123Z|00010|reconnect|WARN|unix#6: connection dropped (Connection reset by peer)
2020-06-29T02:40:46.428Z|00011|jsonrpc|WARN|unix#8: receive error: Connection reset by peer
2020-06-29T02:40:46.428Z|00012|reconnect|WARN|unix#8: connection dropped (Connection reset by peer)
2020-06-29T02:48:12.367Z|00013|memory|INFO|peak resident set size grew 90% in last 2812.6 seconds, from 6656 kB to 12628 kB
2020-06-29T02:48:12.367Z|00014|memory|INFO|cells:46047 monitors:3 sessions:34
2020-06-29T02:48:22.370Z|00015|memory|INFO|peak resident set size grew 151% in last 10.0 seconds, from 12628 kB to 31644 kB
2020-06-29T02:48:22.370Z|00016|memory|INFO|cells:97656 monitors:3 sessions:34
2020-06-29T02:49:12.608Z|00017|memory|INFO|peak resident set size grew 67% in last 50.2 seconds, from 31644 kB to 52816 kB
2020-06-29T02:49:12.608Z|00018|memory|INFO|cells:419671 monitors:3 sessions:34
2020-06-29T02:50:03.187Z|00019|memory|INFO|peak resident set size grew 62% in last 50.6 seconds, from 52816 kB to 85820 kB
2020-06-29T02:50:03.187Z|00020|memory|INFO|cells:729595 monitors:3 sessions:34
2020-06-29T02:51:14.654Z|00021|memory|INFO|peak resident set size grew 52% in last 71.5 seconds, from 85820 kB to 130108 kB
2020-06-29T02:51:14.654Z|00022|memory|INFO|cells:1116719 monitors:3 sessions:34
2020-06-29T02:53:34.956Z|00023|memory|INFO|peak resident set size grew 55% in last 140.3 seconds, from 130108 kB to 201172 kB
2020-06-29T02:53:34.957Z|00024|memory|INFO|cells:1813179 monitors:3 sessions:34
2020-06-29T02:57:27.556Z|00025|memory|INFO|peak resident set size grew 52% in last 232.6 seconds, from 201172 kB to 305564 kB
2020-06-29T02:57:27.556Z|00026|memory|INFO|cells:2789603 monitors:3 sessions:34
2020-06-29T03:04:24.792Z|00027|timeval|WARN|Unreasonably long 7210ms poll interval (6019ms user, 674ms system)
2020-06-29T03:04:24.792Z|00028|timeval|WARN|faults: 315533 minor, 0 major
2020-06-29T03:04:24.792Z|00029|timeval|WARN|disk: 0 reads, 304840 writes
2020-06-29T03:04:24.792Z|00030|timeval|WARN|context switches: 1376 voluntary, 602 involuntary
2020-06-29T03:04:24.792Z|00031|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=db894c43:
2020-06-29T03:04:24.792Z|00032|coverage|INFO|hmap_pathological          7.4/sec    10.200/sec        4.8067/sec   total: 17306
2020-06-29T03:04:24.792Z|00033|coverage|INFO|hmap_expand              42731.0/sec  8333.533/sec      757.0369/sec   total: 2726445
2020-06-29T03:04:24.792Z|00034|coverage|INFO|lockfile_lock              0.0/sec     0.000/sec        0.0000/sec   total: 1
2020-06-29T03:04:24.792Z|00035|coverage|INFO|poll_create_node         129.2/sec   218.500/sec      145.9342/sec   total: 525996
2020-06-29T03:04:24.792Z|00036|coverage|INFO|poll_zero_timeout          0.0/sec     0.000/sec        0.0261/sec   total: 99
2020-06-29T03:04:24.792Z|00037|coverage|INFO|seq_change                 0.0/sec     0.000/sec        0.0000/sec   total: 3
2020-06-29T03:04:24.792Z|00038|coverage|INFO|pstream_open               0.0/sec     0.000/sec        0.0003/sec   total: 3
2020-06-29T03:04:24.792Z|00039|coverage|INFO|util_xalloc              2314482.0/sec 445723.267/sec    41307.0894/sec   total: 148731186
2020-06-29T03:04:24.792Z|00040|coverage|INFO|71 events never hit
2020-06-29T03:04:24.792Z|00041|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.792Z|00042|memory|INFO|peak resident set size grew 405% in last 417.2 seconds, from 305564 kB to 1541964 kB
2020-06-29T03:04:24.792Z|00043|memory|INFO|cells:4077200 monitors:3 sessions:34
2020-06-29T03:04:24.793Z|00044|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.794Z|00045|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.795Z|00046|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.795Z|00047|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.796Z|00048|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.797Z|00049|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.798Z|00050|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.798Z|00051|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:04:24.799Z|00052|poll_loop|INFO|wakeup due to [POLLIN] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (92% CPU usage)
2020-06-29T03:34:28.261Z|00053|timeval|WARN|Unreasonably long 13569ms poll interval (11639ms user, 1074ms system)
2020-06-29T03:34:28.262Z|00054|timeval|WARN|faults: 614892 minor, 0 major
2020-06-29T03:34:28.262Z|00055|timeval|WARN|disk: 0 reads, 573808 writes
2020-06-29T03:34:28.262Z|00056|timeval|WARN|context switches: 2355 voluntary, 1133 involuntary
2020-06-29T03:34:28.262Z|00057|coverage|INFO|Skipping details of duplicate event coverage for hash=db894c43
2020-06-29T03:34:28.262Z|00058|poll_loop|INFO|Dropped 63 log messages in last 1804 seconds (most recently, 1801 seconds ago) due to excessive rate
2020-06-29T03:34:28.262Z|00059|poll_loop|INFO|wakeup due to [POLLIN] on fd 18 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:274 (94% CPU usage)
2020-06-29T03:34:28.262Z|00060|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.262Z|00061|memory|INFO|peak resident set size grew 87% in last 1803.5 seconds, from 1541964 kB to 2888308 kB
2020-06-29T03:34:28.262Z|00062|memory|INFO|cells:7666606 monitors:3 sessions:34
2020-06-29T03:34:28.263Z|00063|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.264Z|00064|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.264Z|00065|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.265Z|00066|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:28.266Z|00067|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (94% CPU usage)
2020-06-29T03:34:37.858Z|00068|timeval|WARN|Unreasonably long 9593ms poll interval (9501ms user, 58ms system)
2020-06-29T03:34:37.858Z|00069|timeval|WARN|faults: 919 minor, 0 major
2020-06-29T03:34:37.858Z|00070|timeval|WARN|context switches: 0 voluntary, 10 involuntary
2020-06-29T03:34:37.858Z|00071|coverage|INFO|Skipping details of duplicate event coverage for hash=db894c43
2020-06-29T03:34:37.858Z|00072|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.859Z|00073|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.861Z|00074|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:37.862Z|00075|poll_loop|INFO|wakeup due to [POLLIN][POLLHUP] on fd 21 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T03:34:40.262Z|00076|poll_loop|INFO|Dropped 442 log messages in last 3 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-29T03:34:40.262Z|00077|poll_loop|INFO|wakeup due to [POLLOUT] on fd 17 (/var/run/ovn/ovnsb_db.sock<->) at lib/stream-fd.c:153 (100% CPU usage)

3. After the stress test, the ovn-northd log as follows

2020-06-29T04:32:02.261Z|07385|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:02.261Z|07386|timeval|WARN|context switches: 24 voluntary, 5 involuntary
2020-06-29T04:32:02.261Z|07387|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (99% CPU usage)
2020-06-29T04:32:06.519Z|07388|timeval|WARN|Unreasonably long 4259ms poll interval (4245ms user, 0ms system)
2020-06-29T04:32:06.519Z|07389|timeval|WARN|faults: 128 minor, 0 major
2020-06-29T04:32:06.519Z|07390|timeval|WARN|context switches: 16 voluntary, 1 involuntary
2020-06-29T04:32:06.520Z|07391|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (101% CPU usage)
2020-06-29T04:32:10.769Z|07392|timeval|WARN|Unreasonably long 4250ms poll interval (4236ms user, 0ms system)
2020-06-29T04:32:10.770Z|07393|timeval|WARN|faults: 128 minor, 0 major
2020-06-29T04:32:10.770Z|07394|timeval|WARN|context switches: 12 voluntary, 3 involuntary
2020-06-29T04:32:15.024Z|07395|timeval|WARN|Unreasonably long 4255ms poll interval (4238ms user, 0ms system)
2020-06-29T04:32:15.024Z|07396|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:15.024Z|07397|timeval|WARN|context switches: 14 voluntary, 1 involuntary
2020-06-29T04:32:15.024Z|07398|poll_loop|INFO|Dropped 1 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-29T04:32:15.024Z|07399|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T04:32:19.266Z|07400|timeval|WARN|Unreasonably long 4241ms poll interval (4227ms user, 0ms system)
2020-06-29T04:32:19.266Z|07401|timeval|WARN|faults: 126 minor, 0 major
2020-06-29T04:32:19.266Z|07402|timeval|WARN|context switches: 18 voluntary, 1 involuntary
2020-06-29T04:32:19.266Z|07403|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)
2020-06-29T04:32:23.501Z|07404|timeval|WARN|Unreasonably long 4236ms poll interval (4219ms user, 0ms system)
2020-06-29T04:32:23.501Z|07405|timeval|WARN|faults: 4 minor, 0 major
2020-06-29T04:32:23.501Z|07406|timeval|WARN|context switches: 16 voluntary, 7 involuntary
2020-06-29T04:32:23.501Z|07407|coverage|INFO|Dropped 13 log messages in last 55 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:23.501Z|07408|coverage|INFO|Skipping details of duplicate event coverage for hash=1a8dc4b6
2020-06-29T04:32:27.754Z|07409|timeval|WARN|Unreasonably long 4252ms poll interval (4238ms user, 0ms system)
2020-06-29T04:32:27.754Z|07410|timeval|WARN|faults: 78 minor, 0 major
2020-06-29T04:32:27.754Z|07411|timeval|WARN|context switches: 10 voluntary, 3 involuntary
2020-06-29T04:32:27.754Z|07412|poll_loop|INFO|Dropped 1 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:27.754Z|07413|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (<->/var/run/ovn/ovnsb_db.sock) at lib/stream-fd.c:157 (100% CPU usage)
2020-06-29T04:32:31.996Z|07414|timeval|WARN|Unreasonably long 4242ms poll interval (4227ms user, 1ms system)
2020-06-29T04:32:31.996Z|07415|timeval|WARN|context switches: 18 voluntary, 1 involuntary
2020-06-29T04:32:31.996Z|07416|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)
2020-06-29T04:32:36.253Z|07417|timeval|WARN|Unreasonably long 4258ms poll interval (4243ms user, 0ms system)
2020-06-29T04:32:36.253Z|07418|timeval|WARN|faults: 92 minor, 0 major
2020-06-29T04:32:36.253Z|07419|timeval|WARN|context switches: 12 voluntary, 3 involuntary
2020-06-29T04:32:36.254Z|07420|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (<->/var/run/ovn/ovnnb_db.sock) at lib/stream-fd.c:157 (101% CPU usage)
2020-06-29T04:32:40.547Z|07421|timeval|WARN|Unreasonably long 4294ms poll interval (4279ms user, 0ms system)
2020-06-29T04:32:40.547Z|07422|timeval|WARN|context switches: 18 voluntary, 2 involuntary
2020-06-29T04:32:44.777Z|07423|timeval|WARN|Unreasonably long 4229ms poll interval (4213ms user, 1ms system)
2020-06-29T04:32:44.777Z|07424|timeval|WARN|faults: 62 minor, 0 major
2020-06-29T04:32:44.777Z|07425|timeval|WARN|context switches: 11 voluntary, 3 involuntary
2020-06-29T04:32:44.777Z|07426|poll_loop|INFO|Dropped 1 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-29T04:32:44.777Z|07427|poll_loop|INFO|wakeup due to 0-ms timeout at lib/reconnect.c:643 (100% CPU usage)

4. After the pressure test, use perf strace to track ovn-northd, the output as follows

     0.046 ( 0.004 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
     0.068 ( 0.028 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
     0.101 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
     0.107 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 141) = 141
     0.129 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 108) = 108
     0.134 ( 0.006 ms): poll(ufds: 0x5560db42dbf0, nfds: 4                                    ) = 0 Timeout
     0.141 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
     0.147 ( 0.012 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 140) = 140
     0.165 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 113) = 113
     0.511 ( 0.014 ms): sendto(fd: 14<socket:[552516759]>, buff: 0x55611315ff60, len: 41      ) = 41
     0.528 ( 0.002 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d4e4, size: 1532 ) = -1 EAGAIN Resource temporarily unavailable
     0.533 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = -1 EAGAIN Resource temporarily unavailable
  4095.413 ( 0.021 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
  4095.458 ( 0.003 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
  4095.477 ( 0.025 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
  4095.508 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 70) = 70
  4095.514 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 68) = 68
  4095.519 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
  4095.525 ( 0.005 ms): poll(ufds: 0x5560e3a55ad0, nfds: 4                                    ) = 1
  4095.531 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
  4095.535 ( 0.003 ms): fstat(fd: 14<socket:[552516759]>, statbuf: 0x7ffef8a0da60             ) = 0
  4095.540 ( 0.002 ms): getsockname(fd: 14<socket:[552516759]>, usockaddr: 0x7ffef8a0d970, usockaddr_len: 0x7ffef8a0d8f8) = 0
  4095.545 ( 0.003 ms): getpeername(fd: 14<socket:[552516759]>, usockaddr: 0x7ffef8a0d970, usockaddr_len: 0x7ffef8a0d8f8) = 0
  4095.556 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 150) = 150
  4095.564 ( 0.006 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d4e4, size: 1532 ) = 38
  4095.586 ( 0.012 ms): sendto(fd: 15<socket:[554039164]>, buff: 0x5560efee8e80, len: 41      ) = 41
  4095.601 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = -1 EAGAIN Resource temporarily unavailable
  8211.797 ( 0.023 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
  8211.967 ( 0.005 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
  8211.992 ( 0.025 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
  8212.023 ( 0.004 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
  8212.030 ( 0.005 ms): poll(ufds: 0x5560df0002e0, nfds: 4, timeout_msecs: 883                ) = 1
  8212.037 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
  8212.044 ( 0.003 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d50a, size: 1494 ) = -1 EAGAIN Resource temporarily unavailable
  8212.050 ( 0.006 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e1fe, size: 2914 ) = 38
 12381.888 ( 0.021 ms): accept(fd: 12<socket:[552516757]>, upeer_sockaddr: 0x7ffef8a0de60, upeer_addrlen: 0x7ffef8a0de5c) = -1 EAGAIN Resource temporarily unavailable
 12381.931 ( 0.003 ms): getrusage(who: 1, ru: 0x7ffef8a0deb0                                  ) = 0
 12381.953 ( 0.028 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 109) = 109
 12381.987 ( 0.003 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 70) = 70
 12381.993 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 89) = 89
 12381.998 ( 0.004 ms): poll(ufds: 0x5560e3a526d0, nfds: 4                                    ) = 0 Timeout
 12382.004 ( 0.002 ms): getrusage(who: 1, ru: 0x55609741ca18                                  ) = 0
 12382.010 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 140) = 140
 12382.018 ( 0.002 ms): write(fd: 5</var/log/ovn/ovn-northd.log>, buf: 0x5560d3e1f310, count: 113) = 113
 12382.386 ( 0.015 ms): sendto(fd: 14<socket:[552516759]>, buff: 0x5560f94f4ef0, len: 41      ) = 41
 12382.406 ( 0.003 ms): recvfrom(fd: 14<socket:[552516759]>, ubuf: 0x55609741d50a, size: 1494 ) = -1 EAGAIN Resource temporarily unavailable
 12382.414 ( 0.002 ms): recvfrom(fd: 15<socket:[554039164]>, ubuf: 0x55609741e224, size: 2876 ) = -1 EAGAIN Resource temporarily unavailable

5. After the pressure test, use top to view the process utilization

Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.7 us,  0.7 sy,  0.0 ni, 96.5 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem : 257572.1 total, 184899.2 free,  40058.3 used,  32614.6 buff/cache
MiB Swap:  16384.0 total,  16384.0 free,      0.0 used. 211729.8 avail Mem
PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 75974 openvsw+  10 -10 2107672   2.0g   3116 R  99.3   0.8 202:51.65 ovn-northd
 75949 openvsw+  20   0  420572 237748   4924 S   0.0   0.1  30:20.48 ovsdb-server
 75965 openvsw+  20   0 2776408   2.5g   4860 S   0.0   1.0   2:11.96 ovsdb-server

I would like to know the possible reasons. Does everyone have any ideas?
Does ovn-northd need parameter tuning？
@putnopvut @numansiddique

ovn with docker failed

docker daemon --cluster-store=consul://127.0.0.1:8500 --cluster-advertise=$HOST_IP:0
unknown flag: --cluster-store

DNS TCP responder

Is it possible to have OVN respond to internal DNS requests over TCP as well as UDP?

That way if a client is using TCP DNS, they can still resolve internal DNS names.

Does OVN support DPDK?

In case of br-int has dpdk interfaces below:

    Bridge br-int
        fail_mode: secure
        datapath_type: netdev
        Port tunbond
            Interface tun0
                type: dpdk
                options: {dpdk-devargs="0000:1a:00.1", n_rxq="2"}
            Interface tun1
                type: dpdk
                options: {dpdk-devargs="0000:d8:00.1", n_rxq="2"}
        Port br-int
            Interface br-int
                type: internal

We need to configure to ovn-encap-ip in the external-ids option

ovs-vsctl set open . external-ids:ovn-encap-ip={IP ADDRESS}

However, the dpdk interface does not have an ip adress.

Is there any other command for configure overlay?

Can OVN DHCP used in PXE？

I want to use ovn DHCP to set the PXE server by using “ovn-nbctl dhcp-options-set-options [dhcp-option-id] tftp_server=X.X.X.X", but it dose't work.
It seems that ovn dosen't set DHCP “next server” as tftp_server. How to resove it？

ovn qos only half of setting

environment：
# uname -a
Linux node101 4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14 14:37:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
# ovn-nbctl --version
ovn-nbctl 20.03.0
Open vSwitch Library 2.13.90
DB Schema 5.22.0

I set the qos of two port:
table=8 (ls_in_qos_meter ), priority=2002 , match=(inport == "23145b86-f915-48aa-952c-e757bae327da"), action=(set_meter(10000); next;)
table=8 (ls_in_qos_meter ), priority=2002 , match=(inport == "31a72696-47fd-4e6a-91ad-1d0a91d7449e"), action=(set_meter(10000); next;)
table=8 (ls_in_qos_meter ), priority=0 , match=(1), action=(next;)
table=5 (ls_out_qos_mark ), priority=0 , match=(1), action=(next;)
table=6 (ls_out_qos_meter ), priority=2002 , match=(outport == "23145b86-f915-48aa-952c-e757bae327da"), action=(set_meter(10000); next;)
table=6 (ls_out_qos_meter ), priority=2002 , match=(outport == "31a72696-47fd-4e6a-91ad-1d0a91d7449e"), action=(set_meter(10000); next;)

And i test the network by iperf.
it show only half of setting limit. i set 10000,iperf show only 5M/s

if the qos only set to one port, the result is correct.

has anyone encountered this problem？

packet loss between the network of iron servers and ovn networks

Good afternoon.
I am using ovn networks and am faced with the problem of packet loss between the network of iron servers and ovn networks.
Loss is not observed between hosts in virtual networks. How to localize the problem, maybe someone has already encountered a similar problem? I assume there are some customization features.
There is also a problem with ping packets to the external Internet.

[root@testvm ~]# ping -c 1000 -s 69 -i 0.01 -q 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 69(97) bytes of data.--- 8.8.8.8 ping statistics ---
1000 packets transmitted, 0 received, 100% packet loss, time 932ms

[root@testvm ~]# ping -c 1000 -s 68 -i 0.01 -q 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 68(96) bytes of data.--- 8.8.8.8 ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 122ms
rtt min/avg/max/mdev = 25.090/25.450/31.949/0.343 ms, pipe 3

DDlog: OVN_Southbound.dl and OVN_Northbound.dl are not regenerated if northd/automake.mk is updated

For example, when adding a new output table to the ovsdb2ddlog command in the makefile, the target is not automatically re-executed so OVN_Northbound.dl and OVN_Southbound.dl are not updated. The current workaround is to touch ovn-sb.vsschema and touch ovn-nb.vsschema.

"create" commands should output id of created object

I have a script that includes the following code:

dmzDhcp="$(ovn-nbctl create DHCP_Options cidr=172.16.255.128/26 \
options="\"server_id\"=\"172.16.255.129\" \"server_mac\"=\"02:ac:10:ff:01:29\" \
\"lease_time\"=\"3600\" \"router\"=\"172.16.255.129\"")"

I was curious if I could replace the raw database command with dhcp-options-create, which has slightly more convenient syntax. Unfortunately, while this command successfully populates the database, it doesn't output anything, so there's no way to get the UUID of the new row.

It would be nice if all create commands would output an appropriate key for referencing the new object.

Dose vxlan support multicast ?

when i use vxlan as the encapsulation,dose it support the multicast?and if it support multicast,what the engress port will be set to ?

ddlog build fail since ddlog 0.37.0

This error since v0.37.0. Build fine with v0.36.0.

ovsdb2ddlog -f /vagrant/ovn/ovn-nb.ovsschema --output-file northd/OVN_Northbound.dl $(cat /vagrant/ovn/northd/ovn-nb.dlopts)
ovsdb2ddlog -f /vagrant/ovn/ovn-sb.ovsschema --output-file northd/OVN_Southbound.dl $(cat /vagrant/ovn/northd/ovn-sb.dlopts)
ddlog -i /vagrant/ovn/northd/ovn_northd.dl -o ./northd -L /root/ddlog/lib -L ./northd 

error: /vagrant/ovn/northd/lrouter.dl:395.10-395.16: Type mismatch:
expected type: ddlog_std::Set<'V>
actual type: signed<64>
in
expected type: ('K2,ddlog_std::Set<'V>)
actual type: (('K2,ddlog_std::Set<'V>),ddlog_std::DDWeight<>)
in
expression 'var entry'
    for (entry in g) {
         ^^^^^^

OVN Private VLAN support

Hi all,
My question is does OVN supports PVLAN (Private VLAN) like the one implemented in Cisco to use same IP address ranges but isolating the layer 2 ports?

ovn-nbctl --help has wrong example for ha group chassis remove

[root@localhost ~]# ovn-nbctl --help|grep ha-chassis-group
ha-chassis-group-add GRP Create an HA chassis group GRP
ha-chassis-group-del GRP Delete the HA chassis group GRP
ha-chassis-group-list List the HA chassis groups
ha-chassis-group-add-chassis GRP CHASSIS [PRIORITY] Adds an HAchassis with optional PRIORITY to the HA chassis group GRP
ha-chassis-group-del-chassis GRP CHASSIS Deletes the HA chassisCHASSIS from the HA chassis group GRP
[root@localhost ~]# ovn-nbctl ha-chassis-group-del-chassis
ovn-nbctl: unknown command 'ha-chassis-group-del-chassis'; use --help for help

Actually the correct command is "ovn-nbctl ha-chassis-group-remove-chassis"

why do not use the reserverd bit？

In the commit b07f1bc,the commit log like below :Because of limited space in VXLAN VNI to pass over all three of -datapath id, ingress port, egress port - the implementation ignores ingress; and splits the remaining 24 bits of VNI into two chunks, 12 bits each - one for datapath and one for egress port
but why do not use the reserved bit in vxlan header ?thank you .

ovn-docker-overlay-driver --detach failed

 ovn-docker-overlay-driver --detach
Traceback (most recent call last):
  File "/usr/bin/ovn-docker-overlay-driver", line 441, in <module>
    prepare()
  File "/usr/bin/ovn-docker-overlay-driver", line 103, in prepare
    ovn_init_overlay()
  File "/usr/bin/ovn-docker-overlay-driver", line 86, in ovn_init_overlay
    OVN_NB = ovs_vsctl("get", "Open_vSwitch", ".",
TypeError: a bytes-like object is required, not 'str'

Does OVN have an LTS version

openvswitch has a clear LTS version.
openvswitch releases

ACL for pg doesn't take effect when update port for it

I used acl for port group.
when i update port for port like this:

ovn-nbctl pg-set-ports $port_group_name $port1 $port2 ...

the acl seems doesn't take effect.
but when i recreate the port group and then set ports with the same command above, it take effect.
does any one meet this problem?

btw, the ovn version is 20.03.
when i do test in version 2.12, it take effect when i update ports for pg and don't need to recreate the port group.

Build failure in `ddlog-dev-v2`

The following build failure comes up non-deterministically on make check -j6. I am guessing it has something to do with ovn-northd-ddlog not using IDL and therefore not requiring IDL bindings to be built early enough in the build process.

I was hoping someone with better understanding of the OVN build system than mine could look into this.

PYTHONPATH=./python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python2 /root/ovs/ovsdb/ovsdb-idlc.in annotate ./ovn-sb.ovsschema ./lib/ovn-sb-idl.ann > lib/ovn-sb-idl.ovsidl.tmp && \
mv lib/ovn-sb-idl.ovsidl.tmp lib/ovn-sb-idl.ovsidl
PYTHONPATH=./python":"$PYTHONPATH PYTHONDONTWRITEBYTECODE=yes /usr/bin/python2 /root/ovs/ovsdb/ovsdb-idlc.in annotate ./ovn-nb.ovsschema ./lib/ovn-nb-idl.ann > lib/ovn-nb-idl.ovsidl.tmp && \
mv lib/ovn-nb-idl.ovsidl.tmp lib/ovn-nb-idl.ovsidl
ovsdb2ddlog -f ovn-nb.ovsschema         \
			-o Logical_Switch_Port          \
			-k Logical_Switch_Port.name     \
			-o NB_Global                    \
			--ro NB_Global.nb_cfg           \
			--ro NB_Global.external_ids     \
			--ro NB_Global.connections      \
			--ro NB_Global.ssl              \
			> northd/OVN_Northbound.dl
ovsdb2ddlog -f ovn-sb.ovsschema \
			-o SB_Global        	\
			-o Logical_Flow     	\
			-o Multicast_Group  	\
			-o Meter            	\
			-o Meter_Band       	\
			-o Datapath_Binding 	\
			-o Port_Binding     	\
			-o Gateway_Chassis  	\
			-o Port_Group       	\
			-o MAC_Binding      	\
			-o DHCP_Options     	\
			-o DHCPv6_Options   	\
			-o Address_Set      	\
			-o DNS              	\
			-o RBAC_Role        	\
			-o RBAC_Permission  	\
			-o IP_Multicast         \
			-p Datapath_Binding 	\
			-p Port_Binding     	\
			-p Multicast_Group      \
			--ro Port_Binding.chassis       \
			--ro Port_Binding.encap         \
			--ro IP_Multicast.seq_no        \
			--ro SB_Global.ssl              \
			--ro SB_Global.connections      \
			--ro SB_Global.external_ids     \
			-k Multicast_Group.datapath     \
			-k Multicast_Group.name         \
			-k Multicast_Group.tunnel_key   \
			-k Port_Binding.logical_port    \
			-k DNS.external_ids             \
			-k Datapath_Binding.external_ids\
			-k RBAC_Role.name               \
			-k Address_Set.name             \
			-k Port_Group.name              \
			-k Meter.name                   \
			-k Logical_Flow.logical_datapath\
			-k Logical_Flow.pipeline		\
			-k Logical_Flow.table_id		\
			-k Logical_Flow.priority		\
			-k Logical_Flow.match			\
			-k Logical_Flow.actions			\
			-k IP_Multicast.datapath		\
			> northd/OVN_Southbound.dl
(echo '/* -*- mode: c; buffer-read-only: t -*- */' && sed < ./lib/ovn-dirs.c.in \
	-e 's,[@]srcdir[@],.,g' \
	-e 's,[@]LOGDIR[@],"/usr/local/var/log/ovn",g' \
	-e 's,[@]OVN_RUNDIR[@],"/usr/local/var/run/ovn",g' \
	-e 's,[@]DBDIR[@],"/usr/local/etc/ovn",g' \
	-e 's,[@]bindir[@],"/usr/local/bin",g' \
	-e 's,[@]sysconfdir[@],"/usr/local/etc",g' \
	-e 's,[@]pkgdatadir[@],"/usr/local/share/ovn",g') \
     > lib/ovn-dirs.c.tmp && \
mv lib/ovn-dirs.c.tmp lib/ovn-dirs.c
depbase=`echo lib/acl-log.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/acl-log.lo -MD -MP -MF $depbase.Tpo -c -o lib/acl-log.lo lib/acl-log.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lib/actions.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/actions.lo -MD -MP -MF $depbase.Tpo -c -o lib/actions.lo lib/actions.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/acl-log.lo -MD -MP -MF lib/.deps/acl-log.Tpo -c lib/acl-log.c -o lib/acl-log.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/actions.lo -MD -MP -MF lib/.deps/actions.Tpo -c lib/actions.c -o lib/actions.o
depbase=`echo lib/chassis-index.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/chassis-index.lo -MD -MP -MF $depbase.Tpo -c -o lib/chassis-index.lo lib/chassis-index.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lib/expr.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/expr.lo -MD -MP -MF $depbase.Tpo -c -o lib/expr.lo lib/expr.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lib/extend-table.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/extend-table.lo -MD -MP -MF $depbase.Tpo -c -o lib/extend-table.lo lib/extend-table.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lib/ip-mcast-index.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/ip-mcast-index.lo -MD -MP -MF $depbase.Tpo -c -o lib/ip-mcast-index.lo lib/ip-mcast-index.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lib/mcast-group-index.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.   -I ./include  -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib    -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow    -g -O2 -MT lib/mcast-group-index.lo -MD -MP -MF $depbase.Tpo -c -o lib/mcast-group-index.lo lib/mcast-group-index.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/chassis-index.lo -MD -MP -MF lib/.deps/chassis-index.Tpo -c lib/chassis-index.c -o lib/chassis-index.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/expr.lo -MD -MP -MF lib/.deps/expr.Tpo -c lib/expr.c -o lib/expr.o
lib/chassis-index.c:17:10: fatal error: lib/ovn-sb-idl.h: No such file or directory
 #include "lib/ovn-sb-idl.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/extend-table.lo -MD -MP -MF lib/.deps/extend-table.Tpo -c lib/extend-table.c -o lib/extend-table.o
make: *** [lib/chassis-index.lo] Error 1
make: *** Waiting for unfinished jobs....
Makefile:2509: recipe for target 'lib/chassis-index.lo' failed
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/ip-mcast-index.lo -MD -MP -MF lib/.deps/ip-mcast-index.Tpo -c lib/ip-mcast-index.c -o lib/ip-mcast-index.o
lib/ip-mcast-index.c:19:10: fatal error: lib/ovn-sb-idl.h: No such file or directory
 #include "lib/ovn-sb-idl.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [lib/ip-mcast-index.lo] Error 1
Makefile:2509: recipe for target 'lib/ip-mcast-index.lo' failed
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I /root/ovs/include -I /root/ovs/include -I /root/ovs/lib -I /root/ovs/lib -I /root/ovs -I /root/ovs -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -g -O2 -MT lib/mcast-group-index.lo -MD -MP -MF lib/.deps/mcast-group-index.Tpo -c lib/mcast-group-index.c -o lib/mcast-group-index.o
lib/mcast-group-index.c:19:10: fatal error: ovn-sb-idl.h: No such file or directory
 #include "ovn-sb-idl.h"
          ^~~~~~~~~~~~~~
compilation terminated.
make: *** [lib/mcast-group-index.lo] Error 1
Makefile:2509: recipe for target 'lib/mcast-group-index.lo' failed
ERROR: Job failed: exit code 1

Newest version of docker doesn't support for docker daemon

docker daemon --cluster-store=consul://127.0.0.1:8500 \
    --cluster-advertise=$HOST_IP:0

docker daemon are removed from

docker --version
Docker version 18.09.8-ce, build 0dd43dd87fd530113bf44c9bba9ad8b20ce4637f

Does OVN support DPDK?

Enhancement Request: Push debian official images for each of the OVN release to quay and/or docker repositories as official container images

Pushing the official images for the different releases would be a convenience as compared to having to build it manually. Projects at repos such as github.com/eBay/go-ovn can conveniently use the published images in their CI and for integration testing.

What OVN version does each Open vSwitch release work with?

My test results are as follows.

Open vSwitch	OVN	Work
2.13.1	20.06.2	✓
2.13.1	20.09.0	×
2.13.1	20.12.0	×

Make ddlog-dev branch read-only

I would like to keep this branch around for now, as it passes all tests (unlike ddlog-dev-v2) but mark it read-only, to make sure that all future PR's are submitted against ddlog-dev-v2.

If this sounds like a good plan, could someone with admin permissions apply this restriction?

why northd ipam can't use the first ip?

when use ovstest
#./ovstest test-ipam ipam_init_ipv4 192.168.1.0/24
start_ipv4: 192.168.1.1
total_ipv4s: 255
allocated_ipv4s: 192.168.1.1

the log show that 192.168.1.1 has been used and review the code find that the first ip will be mark taken ,why doing this?As usual ,we use 192.168.1.0 as the network number rather than 192.168.1.1

/* Mark first IP as taken */
bitmap_set1(info->allocated_ipv4s, 0);

OVN ovsdb servers do not create TCP sockets when secure remote is set

ovn-ctl CLI has parameters --db-nb-create-insecure-remote and --db-sb-create-insecure-remote which can take values yes or no (default). The name of these parameters strongly suggest that when set to yes, communication to the databases will be secured via SSL/TLS over TCP. However, when the same parameters are set to yes, no TCP socket is created.

Let's start with an insecure remote with key and certificates set:

/usr/share/ovn/scripts/ovn-ctl \
    --db-nb-create-insecure-remote=yes \
    --db-nb-addr=192.168.14.32 \
    --db-nb-port=6641 \
    --ovn-nb-db-ssl-key=/root/testca/testcert.key \
    --ovn-nb-db-ssl-cert=/root/testca/testcert.pem \
    --ovn-nb-db-ssl-ca-cert=/root/testca/ca.cert.pem \
    run_nb_ovsdb

ovn-ctl spawned an ovsdb-server process with the following parameters:

+ exec ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --remote=db:OVN_Northbound,NB_Global,connections --private-key=/root/testca/testcert.key --certificate=/root/testca/testcert.pem --ca-cert=/root/testca/ca.cert.pem --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers --remote=ptcp:6641:192.168.14.32 /etc/ovn/ovnnb_db.db

The TCP socket is created and listening on 192.168.14.32:6641 as expected:

[root@ovn1 ~]# ss -natulpe '( sport = 6641 )'
Netid      State       Recv-Q      Send-Q             Local Address:Port             Peer Address:Port
tcp        LISTEN      0           10                 192.168.14.32:6641                  0.0.0.0:*          users:(("ovsdb-server",pid=84022,fd=15)) ino:76722480 sk:99 <->

Even though certificates and key were passed in, ovsdb-server does not load them up (as expected):

[root@ovn1 ~]# openssl s_client -connect 192.168.14.32:6641
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 289 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Now, let's start ovsdb-server in secure remote mode with the same certificates and key set:

/usr/share/ovn/scripts/ovn-ctl \
    --db-nb-create-insecure-remote=no \
    --db-nb-addr=192.168.14.32 \
    --db-nb-port=6641 \
    --ovn-nb-db-ssl-key=/root/testca/testcert.key \
    --ovn-nb-db-ssl-cert=/root/testca/testcert.pem \
    --ovn-nb-db-ssl-ca-cert=/root/testca/ca.cert.pem \
    run_nb_ovsdb

This time there is no TCP socket:

[root@ovn1 ~]# ss -natulpe '( sport = 6641 )'
Netid      State      Recv-Q       Send-Q             Local Address:Port             Peer Address:Port
[root@ovn1 ~]#

The problem appears to be that ovn-ctl spawned an ovsdb-server process without a proper --remote= value set:

+ exec ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --remote=db:OVN_Northbound,NB_Global,connections --private-key=/root/testca/testcert.key --certificate=/root/testca/testcert.pem --ca-cert=/root/testca/ca.cert.pem --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /etc/ovn/ovnnb_db.db

A potential fix could look like this:

--- /usr/share/ovn/scripts/ovn-ctl        2021-03-12 12:55:31.963155494 +0000
+++ /usr/share/ovn/scripts/ovn-ctl.new      2021-03-12 12:55:34.276155494 +0000
@@ -283,6 +283,8 @@
 
     if test X"$create_insecure_remote" = Xyes; then
         set "$@" --remote=ptcp:$port:$addr
+    elif test X"$create_insecure_remote" = Xno && test X"$ovn_db_ssl_key" != X; then
+        set "$@" --remote=pssl:$port:$addr
     fi
 
     if test $mode = active_passive; then

ovsdb-server would be invoked like so:

+ exec ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --remote=db:OVN_Northbound,NB_Global,connections --private-key=/root/testca/testcert.key --certificate=/root/testca/testcert.pem --ca-cert=/root/testca/ca.cert.pem --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers --remote=pssl:6641:192.168.14.32 /etc/ovn/ovnnb_db.db

missing ovn-sim or something to allow helper commands in ovn sandbox

Prior to split, I could do commands from ovs-sim as shown below 1.

Trouble is that now ovs-sim no longer has ovn related functions such as ovn-nbctl
and there is no ovs-sim (nor ovn-sim) in ovg-org/ovn. I wonder if we could re-instate
ovs-sim inside ovn repo, or have a way of importing helper functions in ovn sandbox,
such as net_add and as.

Ref commit openvswitch/ovs@f3e2461 that removes ovn related
helpers from utilities/ovs-sim (for the right reasons!)

Also, I see that there are some left over ovs-sim "stuff" in ovn-org/ovn, so it was not
completely expunged:

$ grep -ilr 'ovs-sim' *
Documentation/automake.mk
Documentation/ref/index.rst
Documentation/ref/ovs-sim.1.rst
Documentation/conf.py
NEWS

[1]

cd ./utilities && rm -rf ./sandbox && ./ovs-sim

ovn_start

# Create a bridge do simulate connection between hypervisors
net_add n1
for i in 1 2; do
    sim_add hv$i
    as hv$i
    ovs-vsctl add-br br-phys
    ovn_attach n1 br-phys 192.168.0.$i
done

# Add port inside hv1's context
as hv1
ovs-vsctl -- add-port br-int hv1-vif1 -- \
    set interface hv1-vif1 mac="00\:11\:22\:33\:44\:55" \
    external-ids:iface-id=foo ofport-request=1
fooMac=$(ovs-ofctl show br-int | grep -E hv.-vif1 | grep -oP "(?<=addr:).*")

# Add port inside hv2's context
as hv2
ovs-vsctl -- add-port br-int hv2-vif1 -- \
    set interface hv2-vif1 external-ids:iface-id=bar ofport-request=1
barMac=$(ovs-ofctl show br-int | grep -E hv.-vif1 | grep -oP "(?<=addr:).*")

ovn-nbctl ls-add sw0
ovn-nbctl lsp-add sw0 foo
ovn-nbctl lsp-add sw0 bar
ovn-nbctl lsp-set-addresses foo $fooMac
ovn-nbctl lsp-set-addresses bar $barMac

Does OVN support networking_sfc?

Does ovn have any driver for networking_sfc of openstack neutron?

IGMP snooping fails when running OVN with RBAC

I am running OpenStack Ussuri on top of Ubuntu Bionic with OVN 20.03.
Also verified this issue with 20.06.

When enabling IGMP snooping as described on OpenStack documentation:
https://docs.openstack.org/neutron/latest/admin/ovn/igmp.html

As VMs submit join requests, IGMP_Group table is not getting populated.

On ovn-controller logs, I can see the following error:
2021-02-06T17:17:40.916Z|00028|ovsdb_idl|WARN|transaction error: {"details":"RBAC rules for client "REDACTED" role "ovn-controller" prohibit row insertion into table "IGMP_Group".","error":"permission error"}

Indeed, checking ovn-architecture document:
https://github.com/ovn-org/ovn/blob/master/ovn-architecture.7.xml#L2530

And it says RBAC_Permission is hard-coded to the values above. There is no entry for IGMP_Group.
We should extend RBAC_Permission to IGMP_Group table with rights to insert and delete for ovn-controller units.

creating ACL matching on address set name causes syntax error in lflow

When creating an ACL for an address group like:
create acl priority=1001 direction=to-lport "match="ip4.src == {$a11661077897478047653} && outport == @a15416135329930652905"" action=allow-related

ovn-nbctl accepts the command, but ovn-controller complains:
2020-02-13T20:00:22.083Z|00059|lflow|WARN|error parsing match "!ct.new && ct.est && !ct.rpl && ct_label.blocked == 0 && (ip4.src == {$a11661077897478047653} && outport == @a15416135329930652905)": Syntax error at `$a11661077897478047653' expecting address set name.

More logs here:
https://gist.githubusercontent.com/trozet/a9fb6c52be80a43aec9cb8aa1f36129f/raw/ae239271655ce030d45da69d8086f0b5ddb35d49/log.txt

How to implement port traffic statistics of OVN logical switch/router?

How to implement port traffic statistics of OVN logical switch/router? This function is very important in cloud network system. Is the current version of OVN's logical port traffic statistics not yet supported?

add support for '--if-exists' and '--may-exist' for lr-policy-del and lr-policy-add subcommands respectiviely

The ovn-kubernetes project makes use of Policy Based Routing and it configures the PBR using the lr-policy-add subcommand. Since this command doesn't support '-may-exist' construct, we are parsing the output of the lr-policy-add to check for 'already existed' in the output to determine if the policy already exists or not.

While fixing this it would be also nice to add '--if-exists' semantics to lr-policy-del subcommand.

Multiple Assigned dynamic IPv6 address and Duplicate IP set on switch messages when using dynamic addressing

Hi,

We are experiencing an issue with a flood of log messages when using OVN with dynamic addressing.

Example, every time we add a port, it then seems to flood the log messages for the new port plus all existing ports, e.g.

ovn-nbctl ls-add s1
ovn-nbctl set Logical_Switch s1 other_config:subnet=172.16.102.0/24 other_config:ipv6_prefix="fd42:a4ec:de94:b4a6::/64"
ovn-nbctl lsp-add s1 s1_r1
ovn-nbctl lsp-set-addresses s1_r1 "00:de:ad:ff:01:02 dynamic"

That last command will then generate this in /var/log/ovn/ovn-northd.log:

2020-12-11T11:15:56.401Z|00059|ovn_northd|INFO|Assigned dynamic IPv4 address '172.16.102.2' to port 's2_r1'
2020-12-11T11:15:56.401Z|00060|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'
2020-12-11T11:15:56.402Z|00061|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'
2020-12-11T11:15:56.402Z|00062|ovn_northd|WARN|Dropped 17 log messages in last 201 seconds (most recently, 169 seconds ago) due to excessive rate
2020-12-11T11:15:56.402Z|00063|ovn_northd|WARN|Duplicate IP set on switch s2: 172.16.102.2

Subsequently adding another port by running:

ovn-nbctl lsp-add s2 s2_r2

Will then generate:

2020-12-11T11:19:19.941Z|00064|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'
2020-12-11T11:19:19.941Z|00065|ovn_northd|WARN|Duplicate IP set on switch s2: 172.16.102.2
2020-12-11T11:19:19.942Z|00066|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'

And then running:

ovn-nbctl lsp-set-addresses s2_r2 "00:de:ad:ff:01:02 dynamic"

Generates:

2020-12-11T11:20:00.150Z|00067|ovn_northd|INFO|Assigned dynamic IPv4 address '172.16.102.3' to port 's2_r2'
2020-12-11T11:20:00.150Z|00068|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r2'
2020-12-11T11:20:00.150Z|00069|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'
2020-12-11T11:20:00.150Z|00070|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r2'
2020-12-11T11:20:00.150Z|00071|ovn_northd|INFO|Assigned dynamic IPv6 address 'fd42:a4ec:de94:b4a6:2de:adff:feff:102' to port 's2_r1'

This gets worse and worse as you add more ports, and becomes a problem for log space.

It seems we are not the first to experience similar issues, as this bug report shows https://bugzilla.redhat.com/show_bug.cgi?id=1894362

We experience this on the latest master and 20.03.1.

Can OVN be installed/used in LXD containers (ie System containers not Process containers)?

I see the documentation referring to using OVN in Kubernetes or Docker containers (which are process type containers).

But do you happen to know if it can/could be used in LXD containers (https://linuxcontainers.org).

LXD (not LXC) containers are "system" containers in that they run a complete Linux OS (CentOS, Debian, Alpine, Ubuntu, Oracle, Gentoo, etc). Like Docker they share the Host's Kernel though.

I don't have any problem using VxLAN or OVS with LXD containers.

But I'd like to know if anyone has any knowledge about using OVN with LXD?

Thanks
Brian

routers gateway addresses cann't reachable to the others on different ovn nodes

ovn-node1(192.168.200.30)
ovn-node2(192.168.200.50)

ovs-vsctl set open . external-ids:ovn-bridge-mappings=provider:vswith0 ( ovn-node1 and ovn-node2 )

ovn-nbctl ls-add sw1
ovn-nbctl lsp-add sw1 sw0-port1
ovn-nbctl lsp-set-addresses sw0-port1 "00:00:01:00:00:03 192.168.1.10"
ovn-nbctl ls-add sw2
ovn-nbctl lsp-add sw2 sw2-port1
ovn-nbctl lsp-set-addresses sw1-port1 "00:00:02:00:00:03 192.168.2.10"

ovn-nbctl lr-add lr1

Connect sw1 to lr1

ovn-nbctl lrp-add lr1 lr1-sw1 00:00:00:00:ff:01 192.168.1.0/24
ovn-nbctl lsp-add sw1 sw1-lr1
ovn-nbctl lsp-set-type sw1-lr1 router
ovn-nbctl lsp-set-addresses sw1-lr1 router
ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1

ovn-nbctl lr-add lr2

Connect sw2 to lr2

ovn-nbctl lrp-add lr2 lr1-sw2 00:00:00:00:ff:02 192.168.2.0/24
ovn-nbctl lsp-add sw2 sw2-lr2
ovn-nbctl lsp-set-type sw2-lr2 router
ovn-nbctl lsp-set-addresses sw2-lr2 router
ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2

ovn-nbctl ls-add public

Create a localnet port

ovn-nbctl lsp-add public ln-public
ovn-nbctl lsp-set-type ln-public localnet
ovn-nbctl lsp-set-addresses ln-public unknown
ovn-nbctl lsp-set-options ln-public network_name=provider

ovn-nbctl lrp-add lr1 lr1-public 00:00:20:20:12:13 192.166.200.72/24
ovn-nbctl lsp-add public public-lr1
ovn-nbctl lsp-set-type public-lr1 router
ovn-nbctl lsp-set-addresses public-lr1 router
ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public

ovn-nbctl lrp-add lr2 lr2-public 00:00:20:20:12:14 192.166.200.78/24
ovn-nbctl lsp-add public public-lr2
ovn-nbctl lsp-set-type public-lr2 router
ovn-nbctl lsp-set-addresses public-lr2 router
ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public

ovn-nbctl lrp-set-gateway-chassis lr1-public ovn-node1
ovn-nbctl lrp-set-gateway-chassis lr2-public ovn-node2

vm1(192.168.1.10)->sw1（192.168.1.0/24）->lr1(192.166.200.72)--
| cr-lrp（vswitch0 192.168.200.30）---> ovn-node1
public
| cr-lrp（vswitch0 192.168.200.50）---> ovn-node2
vm2(192.168.2.10)->sw2（192.168.2.0/24）->lr2(192.166.200.78)--

Actual results:
vm1 can ping 192.168.200.72 192.168.200.30 92.168.200.50, but cann't ping 192.166.200.78
vm2 can ping 192.168.200.78 192.168.200.30 92.168.200.50, but cann't ping 192.166.200.72

Expected results:
vm1 can ping 192.166.200.78
vm2 can ping 192.166.200.72

OVN 20.12.0 version compilation error

When compiling OVN 20.12.0 in a personal virtual machine, the following error occurs:

generic -c -o controller/ovn-controller.o controller/ovn-controller.c
controller/ovn-controller.c: In function 'update_sb_monitors':
controller/ovn-controller.c:266:9: error: void value not ignored as it ought to be
		sbrec_port_binding_set_condition(ovnsb_idl, &pb),
		^
controller/ovn-controller.c:267:9: error: void value not ignored as it ought to be
		sbrec_logical_flow_set_condition(ovnsb_idl, &lf),
		^
controller/ovn-controller.c:268:9: error: void value not ignored as it ought to be
		sbrec_logical_dp_group_set_condition(ovnsb_idl, &ldpg),
		^
controller/ovn-controller.c:269:9: error: void value not ignored as it ought to be
		sbrec_mac_binding_set_condition(ovnsb_idl, &mb),
		^
controller/ovn-controller.c:270:9: error: void value not ignored as it ought to be
		sbrec_multicast_group_set_condition(ovnsb_idl, &mg),
		^
controller/ovn-controller.c:271:9: error: void value not ignored as it ought to be
		sbrec_dns_set_condition(ovnsb_idl, &dns),
		^
		
controller/ovn-controller.c:272:9: error: void value not ignored as it ought to be
		sbrec_controller_event_set_condition(ovnsb_idl, &ce),
		^
		
controller/ovn-controller.c:273:9: error: void value not ignored as it ought to be
		sbrec_ip_multicast_set_condition(ovnsb_idl, &ip_mcast),
		^
		
controller/ovn-controller.c:274:9: error: void value not ignored as it ought to be
		sbrec_igmp_group_set_condition(ovnsb_idl, &igmp),
		^
		
controller/ovn-controller.c:275:9: error: void value not ignored as it ought to be
		sbrec_chassis_private_set_condition(ovnsb_idl, &chprv),
		^
		
make[2]: *** [controller/ovn-controller.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/root/ovn-20.12.0/rpm/rpmbuild/BUILD/ovn-20.12.0'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/ovn-20.12.0/rpm/rpmbuild/BUILD/ovn-20.12.0'
error: Bad exit status from /var/tmp/rpm-tmp.se9wqo (%build)

what does this mean? How to solve this problem？ @rnurgaliyev

In the ovn21.03 version, bfd does not work properly

Verified on version:

OVN：21.03
OVS：2.15.90
OS：Red Hat Enterprise Linux Server release 7.6 (Maipo)
Kernel：4.19.8-1.el7.elrepo.x86_64

Today, I want to verify the effectiveness of monitoring static routing based on the BFD protocol, but bfd does not work properly.
The problem reproduces as follows：

# ovn-nbctl lr-add r1
# ovn-nbctl lrp-add r1 r1-sw1 00:00:c0:a8:01:fe 192.168.1.254/24

# ovn-nbctl ls-add sw1
# ovn-nbctl lsp-add sw1 port1
# ovn-nbctl lsp-set-addresses port1 "00:00:c0:a8:01:01 192.168.1.1"
# ovn-nbctl lsp-add sw1 port2
# ovn-nbctl lsp-set-addresses port2 "00:00:c0:a8:01:02 192.168.1.2"
# ovn-nbctl lsp-add sw1 sw1-r1
# ovn-nbctl lsp-set-type sw1-r1 router
# ovn-nbctl lsp-set-addresses sw1-r1 00:00:c0:a8:01:fe
# ovn-nbctl lsp-set-options sw1-r1 router-port=r1-sw1

# ip netns add c01
# ip link add veth0 type veth peer name veth1
# ip link set veth1 netns c01
# ip netns exec c01 ip link set lo up
# ip netns exec c01 ip link set veth1 up
# ip netns exec c01 ip link set veth1 address 00:00:c0:a8:01:01
# ip netns exec c01 ip addr add 192.168.1.1/24 dev veth1
# ip netns exec c01 ip route add default via 192.168.1.254
# ip link set veth0 up
# ovs-vsctl add-port br-int veth0
# ovs-vsctl set Interface veth0 external_ids:iface-id=port1

# ovn-nbctl create bfd logical_port=r1-sw1 dst_ip=192.168.1.1 min_tx=1000 min_rx=1000 detect_mult=3
2d1cc34e-9de4-4765-a121-b45b3f3deb31
# ovn-nbctl --may-exist --policy=dst-ip  --bfd=2d1cc34e-9de4-4765-a121-b45b3f3deb31 lr-route-add r1 172.245.0.0/16 192.168.1.1 r1-sw1

# ip netns exec c01 bash
# bfdd-beacon --listen=0.0.0.0
# bfdd-control session new set mintx 1 s
# bfdd-control session new set minrx 1 s
# bfdd-control session new set multi 3
# bfdd-control connect local 192.168.1.1 remote 192.168.1.254
# exit

# ovn-nbctl list bfd
_uuid               : 2d1cc34e-9de4-4765-a121-b45b3f3deb31
detect_mult         : []
dst_ip              : "192.168.1.1"
external_ids        : {}
logical_port        : r1-sw1
min_rx              : []
min_tx              : []
options             : {}
status              : down

# ovn-nbctl list logical_router_static_route
_uuid               : b0c29507-dc56-4ad6-b60f-8f93823b9903
bfd                 : 2d1cc34e-9de4-4765-a121-b45b3f3deb31
external_ids        : {}
ip_prefix           : "172.245.0.0/16"
nexthop             : "192.168.1.1"
options             : {}
output_port         : r1-sw1
policy              : dst-ip

It can be found that the status in the ovn bfd table is always down。

In addition, I caught the packet in c01 and found that there was no bdf packet that was returned from ovn：

# ip netns exec c01 bash
# tcpdump -eni veth1 -vvv
tcpdump: listening on veth1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:08:12.007600 00:00:c0:a8:01:01 > 00:00:c0:a8:01:fe, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 255, id 43241, offset 0, flags [DF], proto UDP (17), length 52)
	192.168.1.1.51838 > 192.168.1.254.bfd-control: [bad udp cksum 0x8481 -> 0xdd9c!] BFDv1, length: 24
	Control, State Down, Flags: [none], Diagnostic: No Diagnostic (0x00)
	Detection Timer Multiplier: 3 (3000 ms Detection time), BFD Length: 24
	My Discriminator: 0x63a1b8e2, Your Discriminator: 0x00000000
	Desired min Tx Interval:    1000 ms
	Required min Rx Interval:   1000 ms
	Required min Echo Interval:    0 ms
23:08:12.811107 00:00:c0:a8:01:01 > 00:00:c0:a8:01:fe, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 255, id 43452, offset 0, flags [DF], proto UDP (17), length 52)
	192.168.1.1.51838 > 192.168.1.254.bfd-control: [bad udp cksum 0x8481 -> 0xdd9c!] BFDv1, length: 24
	Control, State Down, Flags: [none], Diagnostic: No Diagnostic (0x00)
	Detection Timer Multiplier: 3 (3000 ms Detection time), BFD Length: 24
	My Discriminator: 0x63a1b8e2, Your Discriminator: 0x00000000
	Desired min Tx Interval:    1000 ms
	Required min Rx Interval:   1000 ms
	Required min Echo Interval:    0 ms
23:08:13.694942 00:00:c0:a8:01:01 > 00:00:c0:a8:01:fe, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 255, id 43942, offset 0, flags [DF], proto UDP (17), length 52)
	192.168.1.1.51838 > 192.168.1.254.bfd-control: [bad udp cksum 0x8481 -> 0xdd9c!] BFDv1, length: 24
	Control, State Down, Flags: [none], Diagnostic: No Diagnostic (0x00)
	Detection Timer Multiplier: 3 (3000 ms Detection time), BFD Length: 24
	My Discriminator: 0x63a1b8e2, Your Discriminator: 0x00000000
	Desired min Tx Interval:    1000 ms
	Required min Rx Interval:   1000 ms
	Required min Echo Interval:    0 ms

The following is a logical flow table that simulates c01 sending bdf packages to ovn

# ovn-trace --detailed sw1 'inport == "port1" && eth.src == 00:00:c0:a8:01:01 && ip4.src == 192.168.1.1 && eth.dst == 00:00:c0:a8:01:fe && ip4.dst == 192.168.1.254 && udp.dst == 3784'
# udp,reg14=0x1,vlan_tci=0x0000,dl_src=00:00:c0:a8:01:01,dl_dst=00:00:c0:a8:01:fe,nw_src=192.168.1.1,nw_dst=192.168.1.254,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=0,tp_dst=3784

ingress(dp="sw1", inport="port1")
---------------------------------
0. ls_in_port_sec_l2 (ovn-northd.c:4793): inport == "port1", priority 50, uuid c40caf3c
	next;
23. ls_in_l2_lkup (ovn-northd.c:7385): eth.dst == 00:00:c0:a8:01:fe, priority 50, uuid 2ccc3979
	outport = "sw1-r1";
	output;

egress(dp="sw1", inport="port1", outport="sw1-r1")
--------------------------------------------------
0. ls_out_pre_lb (ovn-northd.c:4939): ip && outport == "sw1-r1", priority 110, uuid f4ea2f47
	next;
10. ls_out_port_sec_l2 (ovn-northd.c:4888): outport == "sw1-r1", priority 50, uuid 954503a0
	output;
	/* output to "sw1-r1", type "patch" */

ingress(dp="r1", inport="r1-sw1")
---------------------------------
0. lr_in_admission (ovn-northd.c:9326): eth.dst == 00:00:c0:a8:01:fe && inport == "r1-sw1", priority 50, uuid 387d8d1d
	xreg0[0..47] = 00:00:c0:a8:01:fe;
	next;
1. lr_in_lookup_neighbor (ovn-northd.c:9405): 1, priority 0, uuid dd08c88e
	reg9[2] = 1;
	next;
2. lr_in_learn_neighbor (ovn-northd.c:9414): reg9[2] == 1, priority 100, uuid de366c4f
	next;
3. lr_in_ip_input (ovn-northd.c:9239): ip4.dst == 192.168.1.254 && udp.dst == 3784, priority 110, uuid 0e297771
	handle_bfd_msg();

Can anyone tell me where the problem is, is my usage incorrect? @blp @dceara @numansiddique

Missing features in ovn-northd-ddlog

After rebasing ovn-northd-ddlog on top of the latest master, the following tests are failing:

In addition, @numansiddique is reporting:

153: ovn -- ACL rate-limited logging FAILED (ovn-e2e.at:5938)

It's not failing for me, so must be another instance of a missing wait.

Add configuration option to build DDLog with debug symbols

make distcheck is busted

make distcheck is failing in ovn

cd ovn
./boot.sh && ./configure --with-ovs-source=${PWD}/../ovs && time make distcheck
...
checking for OVS source directory... configure: error: OVS source dir path needs to be specified (use --with-ovs-source)
Makefile:2928: recipe for target 'distcheck' failed
make: *** [distcheck] Error 1

full log:
https://gist.github.com/flavio-fernandes/17d55d6663f9b77fa9a0189d04afc532

[ovn 2.12.90] testsuite: 77 failed

make check fails this test once in a while:

centos-8:  77: ovn -- 1 LR with distributed router gateway port
centos-8:  FAILED (ovn.at:9211)


centos-8: ERROR: All 203 tests were run,
centos-8: 1 failed unexpectedly.

centos-8: Please send `tests/testsuite.log' and all information you think might help:
centos-8:
centos-8:    To: <[email protected]>
centos-8:    Subject: [ovn 2.12.90] testsuite: 77 failed
centos-8:
centos-8: You may investigate any problem if you feel able to do so, in which
centos-8: case the test suite provides a good starting point.  Its output may
centos-8: be found below `tests/testsuite.dir'.
centos-8: make[2]: Leaving directory '/root/build/ovn'

using vagrant file is an easy way of making it happen

ovn-controller with ovs-dpdk ovn-controller.log has errors “Invalid or incomplete DHCP packet received”

ovn-controller（ovn-branch-20.06） Keep reporting errors in the function "pinctrl_handle_put_dhcp_opts"(pinctrl.c) .The error message is “Invalid or incomplete DHCP packet received”。I use OVN on the basis of ovs-dpdk, so the struct dp_packet member of the structure should use mbuf, but ovn does not have the macro “DPDK_NETDEV”, which makes it impossible to parse the message normally.
I found a temporary solution, define “DPDK_NETDEV” when ovn uses ovs lib struct dp_packet, and then add the dpdk header file to ovn, so that packetin dhcp packets can be parsed normally.

Dose ovn 21.03.0 support centralized dnat_and_snat for vlan backed DVR

I have done some work for vlan backed DVR.
1)I make new version (21.03.0) for ovn rpm pkgs , and install those rpm pkgs at my environment
2)Set local OVSDBs with 'external_ids:ovn-chassis-mac-mappings' on every chassis.
3)Set option redirect-type value 'bridged' for gateway of external traffic.
4)Let vm2 ping floating-ip of vm1, traffic failed.
5)Modify redirect-type value with 'overlay', traffic pass

I think ovn not completely support centralized floating-ip now at the scenarios of Vlan backed DVR. This issue also impact sriov.

Can't build Debian package due to changing tags and date format in the changelog

OSV: master
OVN: master
Debian: bullseye/sid

I met a few problems:

The first one:
sphinx builder can't parse new date format in the changelog:
https://github.com/ovn-org/ovn/blob/master/debian/changelog#L11

Exception occurred:                                                                                                                              
  File "/usr/lib/python3/dist-packages/sphinx/builders/gettext.py", line 204, in <module>                                                        
    timestamp = float(source_date_epoch)                                                                                                         
ValueError: could not convert string to float: ''                                                                                                
The full traceback has been saved in /tmp/sphinx-err-grd4xmzi.log, if you want to report the issue to the developers.                            
Please also report this if it was a user error, so that a better error message can be provided next time.                                        
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!                                               
Cannot find 'ovs-sphinx-theme' package. Falling back to default theme.
make[2]: *** [Makefile:3182: docs-check] Error 2
make[2]: Leaving directory '/usr/src/ovn'
make[1]: *** [Makefile:2586: install] Error 2
make[1]: Leaving directory '/usr/src/ovn'
dh_auto_install: error: make -j1 install DESTDIR=/usr/src/ovn/debian/tmp AM_UPDATE_INFO_DIR=no returned exit code 2
make: *** [debian/rules:24: binary] Error 25

The second one:
Changelog reader doesn't support tags in uppercase

dpkg-gencontrol: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500
dpkg-gencontrol: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500
dpkg-gencontrol: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500
dpkg-gencontrol: error: source package name 'OVN' is illegal: character 'O' not allowed
dpkg-gencontrol: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500
dh_gencontrol: error: dpkg-gencontrol -povn-host -ldebian/changelog -Tdebian/ovn-host.substvars -Pdebian/ovn-host returned exit code 255
dpkg-gencontrol: error: source package name 'OVN' is illegal: character 'O' not allowed
dh_gencontrol: error: dpkg-gencontrol -povn-common -ldebian/changelog -Tdebian/ovn-common.substvars -Pdebian/ovn-common returned exit code 255
dpkg-gencontrol: error: source package name 'OVN' is illegal: character 'O' not allowed
dh_gencontrol: error: dpkg-gencontrol -povn-controller-vtep -ldebian/changelog -Tdebian/ovn-controller-vtep.substvars -Pdebian/ovn-controller-vtep returned exit code 255
dpkg-gencontrol: error: source package name 'OVN' is illegal: character 'O' not allowed
dpkg-gencontrol: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500
dh_gencontrol: error: dpkg-gencontrol -povn-docker -ldebian/changelog -Tdebian/ovn-docker.substvars -Pdebian/ovn-docker returned exit code 255
dpkg-gencontrol: error: source package name 'OVN' is illegal: character 'O' not allowed
dh_gencontrol: error: dpkg-gencontrol -povn-central -ldebian/changelog -Tdebian/ovn-central.substvars -Pdebian/ovn-central returned exit code 255
dh_gencontrol: error: Aborting due to earlier error
make: *** [debian/rules:24: binary] Error 25

The last one:

dpkg-parsechangelog: warning:     debian/changelog(l5): badly formatted trailer line
LINE:  -- OVN team <[email protected]> Tue, 26 May 2020 13:10:35 -0500

Patch:

diff --git a/debian/changelog b/debian/changelog
index 96d4d2b46..a17bada69 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,16 +1,16 @@
-OVN (20.06.90-1) unstable; urgency=low
+ovn (20.06.90-1) unstable; urgency=low
 
    * New upstream version
 
- -- OVN team <[email protected]> Tue 26 May 2020 01:10:35 PM EDT
+ -- OVN team <[email protected]>  Tue, 26 May 2020 13:10:35 -0500
 
-OVN (20.06.0-1) unstable; urgency=low
+ovn (20.06.0-1) unstable; urgency=low
 
     * New upstream version
 
- -- OVN team <[email protected]> Tue 26 May 2020 01:10:35 PM E
+ -- OVN team <[email protected]>  Tue, 26 May 2020 13:10:35 -0500
 
-OVN (20.03.0-1) unstable; urgency=low
+ovn (20.03.0-1) unstable; urgency=low
 
    * New upstream version

Can this change be added to the master branch or do you have any remarks?

FYI - The link to the Open vSwitch software tutorial is broken in your README.md.

To learn about some advanced features of the Open vSwitch software switch, read the tutorial.

the following Link is no longer valid:
https://github.com/ovn-org/ovn/blob/master/Documentation/tutorials/ovs-advanced.rst

VMs cannot connect its own FIPs when tenant network type is Geneve

In some scenarios, the VM needs to connect its own Floating IP when tenant network type is Geneve. In my environment, VMs cannot connect their own FIPs, and they also cannot connect other VMs on the same compute node through FIPs. Using ovn-trace to track lfows ,it shows that no lfow matched after SNAT. On different compute nodes network works fine through FIP.
I add an OVS flow on compute node like this:

ovs-ofctl add-flow br-int "table=8,reg14=0x1,metadata=0x12,dl_src=fa:16:3e:85:0e:cb actions=resubmit(,9)"
then the FIP(10.0.41.228) works fine.

Verions:
openstack: victoria
ovn-northd: 20.03.2
ovn-sbctl: 20.03.2
ovn-controller: 20.03.2
Open vSwitch Library: 2.13.3
OpenFlow versions: 0x4:0x4
neutron-server: 17.1.3.dev4

neutron_ovn_distributed_fip: "yes"

=================================================================================
Network information:
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|type | IP | MAC | neutron port ID | neutron network ID | OVN Port_Binding UUID |tunnel_key | OVN Datapath_Binding UUID |tunnel_key |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|VM NIC | 172.20.25.162/24 | fa:16:3e:30:4b:85 | 103e7777-a376-419d-9629-31bc6c7fedb1 | fb1dc9cb-df50-433a-9388-89dc31e47d99 | 18c93367-0944-4572-a601-67ae9cf13198 | 0x1e | 3eb22556-9dea-46bd-972d-9097252927c9 | 0x12 |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|VM GW | 172.20.25.1/24 | fa:16:3e:9c:8b:f3 | 039ba2bc-997c-448a-adb7-54bd3af697c1 | fb1dc9cb-df50-433a-9388-89dc31e47d99 | fded7818-5a70-46a2-84cd-b88b9b82ea56 | 0x11 | 3eb22556-9dea-46bd-972d-9097252927c9 | 0x12 |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|VM FIP | 10.0.41.228/24 | fa:16:3e:85:0e:cb | 7266cfce-f20d-4281-872d-b89f79394467 | d32cbd3e-15ba-487e-82fc-23646d7b3d91 | 18c93367-0944-4572-a601-67ae9cf13198 | 0x1e | c01f5fca-0b89-42a3-8d15-f9da96509496 | 0x3 |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|FIP GW | 10.0.40.1/24 | b4:09:31:47:30:ef | N/A | d32cbd3e-15ba-487e-82fc-23646d7b3d91 | N/A | N/A | c01f5fca-0b89-42a3-8d15-f9da96509496 | 0x3 |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+
|router | 10.0.40.233/24 | fa:16:3e:4a:e3:b5 | e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9 | d32cbd3e-15ba-487e-82fc-23646d7b3d91 | 883bca93-6db8-44df-9ff6-3f66b1798d95 | 0x1 | c01f5fca-0b89-42a3-8d15-f9da96509496 | 0x3 |
|ext_port| | | | | | | | |
+--------+-------------------+--------------------+---------------------------------------+---------------------------------------+--------------------------------------+------------+--------------------------------------+------------+

==================================================================================
ovn-trace output:
(ovn-controller)# ovn-trace --db=tcp:172.16.xx.xx:6642,tcp:172.16.xx.yy:6642,tcp:172.16.xx.zz:6642 --no-friendly-names --ovs neutron-fb1dc9cb-df50-433a-9388-89dc31e47d99 'inport == "18c93367-0944-4572-a601-67ae9cf13198" && eth.src == fa:16:3e:30:4b:85 && eth.dst == fa:16:3e:9c:8b:f3 && ip4.src == 172.20.25.162 && ip4.dst == 10.0.41.228 && ip.ttl == 64 && icmp4.type == 8'

icmp,reg14=0x1e,vlan_tci=0x0000,dl_src=fa:16:3e:30:4b:85,dl_dst=fa:16:3e:9c:8b:f3,nw_src=172.20.25.162,nw_dst=10.0.41.228,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0

ingress(dp="af17a9c0-b600-407f-880f-535dfab2877d", inport="103e7777-a376-419d-9629-31bc6c7fedb1")

ls_in_port_sec_l2 (ovn-northd.c:4514): inport == "103e7777-a376-419d-9629-31bc6c7fedb1" && eth.src == {fa:16:3e:30:4b:85}, priority 50, uuid 36c79b1c
cookie=0x36c79b1c, duration=74244.455s, table=8, n_packets=288859, n_bytes=936110898, priority=50,reg14=0x1e,metadata=0xc,dl_src=fa:16:3e:30:4b:85 actions=resubmit(,9)
next;
ls_in_port_sec_ip (ovn-northd.c:4186): inport == "103e7777-a376-419d-9629-31bc6c7fedb1" && eth.src == fa:16:3e:30:4b:85 && ip4.src == {172.20.25.162}, priority 90, uuid c58a60a0
cookie=0xc58a60a0, duration=74244.323s, table=9, n_packets=287956, n_bytes=936072972, priority=90,ip,reg14=0x1e,metadata=0xc,dl_src=fa:16:3e:30:4b:85,nw_src=172.20.25.162 actions=resubmit(,10)
next;
ls_in_pre_acl (ovn-northd.c:4703): ip, priority 100, uuid 38b64b35
cookie=0x38b64b35, duration=74244.434s, table=11, n_packets=456863, n_bytes=954013905, priority=100,ip,metadata=0xc actions=load:0x1->NXM_NX_XXREG0[96],resubmit(,12)
cookie=0x38b64b35, duration=74244.380s, table=11, n_packets=0, n_bytes=0, priority=100,ipv6,metadata=0xc actions=load:0x1->NXM_NX_XXREG0[96],resubmit(,12)
reg0[0] = 1;
next;
ls_in_pre_stateful (ovn-northd.c:4890): reg0[0] == 1, priority 100, uuid dcd05000
cookie=0xdcd05000, duration=74244.366s, table=13, n_packets=0, n_bytes=0, priority=100,ipv6,reg0=0x1/0x1,metadata=0xc actions=ct(table=14,zone=NXM_NX_REG13[0..15])
cookie=0xdcd05000, duration=74244.334s, table=13, n_packets=456863, n_bytes=954013905, priority=100,ip,reg0=0x1/0x1,metadata=0xc actions=ct(table=14,zone=NXM_NX_REG13[0..15])
ct_next;

ct_next(ct_state=est|trk /* default (use --ct to customize) */)

ls_in_acl (ovn-northd.c:5083): (!ct.trk || (!ct.new && ct.est && !ct.rpl && ct_label.blocked == 0)) && (inport == @pg_79068b7d_a57d_4f24_b0ed_5d689d6a6859 && ip4), priority 2002, uuid e2c75b95
cookie=0xe2c75b95, duration=74244.497s, table=14, n_packets=0, n_bytes=0, priority=2002,ct_state=-trk,ip,reg14=0x22,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.270s, table=14, n_packets=166679, n_bytes=16622180, priority=2002,ct_state=-new+est-rpl+trk,ct_label=0/0x1,ip,reg14=0x1e,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.283s, table=14, n_packets=4456, n_bytes=361337, priority=2002,ct_state=-new+est-rpl+trk,ct_label=0/0x1,ip,reg14=0x22,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.304s, table=14, n_packets=0, n_bytes=0, priority=2002,ct_state=-trk,ip,reg14=0x13,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.371s, table=14, n_packets=0, n_bytes=0, priority=2002,ct_state=-trk,ip,reg14=0x1e,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.380s, table=14, n_packets=3672, n_bytes=555588, priority=2002,ct_state=-new+est-rpl+trk,ct_label=0/0x1,ip,reg14=0x28,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.420s, table=14, n_packets=3350, n_bytes=307439, priority=2002,ct_state=-new+est-rpl+trk,ct_label=0/0x1,ip,reg14=0x13,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.438s, table=14, n_packets=114262, n_bytes=11563996, priority=2002,ct_state=-new+est-rpl+trk,ct_label=0/0x1,ip,reg14=0x23,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.451s, table=14, n_packets=0, n_bytes=0, priority=2002,ct_state=-trk,ip,reg14=0x28,metadata=0xc actions=resubmit(,15)
cookie=0xe2c75b95, duration=74244.488s, table=14, n_packets=0, n_bytes=0, priority=2002,ct_state=-trk,ip,reg14=0x23,metadata=0xc actions=resubmit(,15)
next;
ls_in_l2_lkup (ovn-northd.c:6841): eth.dst == fa:16:3e:9c:8b:f3, priority 50, uuid 325f29ce
cookie=0x325f29ce, duration=74244.407s, table=27, n_packets=456618, n_bytes=953953284, priority=50,metadata=0xc,dl_dst=fa:16:3e:9c:8b:f3 actions=set_field:0xb->reg15,resubmit(,32)
outport = "039ba2bc-997c-448a-adb7-54bd3af697c1";
output;

egress(dp="af17a9c0-b600-407f-880f-535dfab2877d", inport="103e7777-a376-419d-9629-31bc6c7fedb1", outport="039ba2bc-997c-448a-adb7-54bd3af697c1")

ls_out_pre_acl (ovn-northd.c:4658): ip && outport == "039ba2bc-997c-448a-adb7-54bd3af697c1", priority 110, uuid 4dba6a66
cookie=0x4dba6a66, duration=74244.407s, table=41, n_packets=456824, n_bytes=954006123, priority=110,ip,reg15=0xb,metadata=0xc actions=resubmit(,42)
cookie=0x4dba6a66, duration=74244.346s, table=41, n_packets=0, n_bytes=0, priority=110,ipv6,reg15=0xb,metadata=0xc actions=resubmit(,42)
next;
ls_out_port_sec_l2 (ovn-northd.c:4580): outport == "039ba2bc-997c-448a-adb7-54bd3af697c1", priority 50, uuid ea1250eb
cookie=0xea1250eb, duration=74244.492s, table=49, n_packets=456618, n_bytes=953953284, priority=50,reg15=0xb,metadata=0xc actions=resubmit(,64)
output;
/* output to "039ba2bc-997c-448a-adb7-54bd3af697c1", type "patch" */

ingress(dp="32d08999-d00a-49d2-bf8d-9169894b2a45", inport="lrp-039ba2bc-997c-448a-adb7-54bd3af697c1")

lr_in_admission (ovn-northd.c:7836): eth.dst == fa:16:3e:9c:8b:f3 && inport == "lrp-039ba2bc-997c-448a-adb7-54bd3af697c1", priority 50, uuid 9a133c69
cookie=0x9a133c69, duration=74244.305s, table=8, n_packets=456618, n_bytes=953953284, priority=50,reg14=0x3,metadata=0x12,dl_dst=fa:16:3e:9c:8b:f3 actions=resubmit(,9)
next;
lr_in_lookup_neighbor (ovn-northd.c:7885): 1, priority 0, uuid 2d7be36f
cookie=0x2d7be36f, duration=74244.478s, table=9, n_packets=666756, n_bytes=1140691558, priority=0,metadata=0x12 actions=load:0x1->OXM_OF_PKT_REG4[3],resubmit(,10)
reg9[3] = 1;
next;
lr_in_learn_neighbor (ovn-northd.c:7890): reg9[3] == 1 || reg9[2] == 1, priority 100, uuid 664e0889
cookie=0x664e0889, duration=74244.321s, table=10, n_packets=0, n_bytes=0, priority=100,reg9=0x4/0x4,metadata=0x12 actions=resubmit(,11)
cookie=0x664e0889, duration=74244.318s, table=10, n_packets=666756, n_bytes=1140691558, priority=100,reg9=0x8/0x8,metadata=0x12 actions=resubmit(,11)
next;
lr_in_ip_routing (ovn-northd.c:7463): ip4.dst == 0.0.0.0/0, priority 1, uuid 60838bf6
cookie=0x60838bf6, duration=74244.364s, table=17, n_packets=456618, n_bytes=953953284, priority=1,ip,metadata=0x12 actions=dec_ttl(),load:0->OXM_OF_PKT_REG4[32..47],load:0xa002801->NXM_NX_XXREG0[96..127],load:0xa0028e9->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:4a:e3:b5->eth_src,set_field:0x1->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,18)
ip.ttl--;
reg8[0..15] = 0;
reg0 = 10.0.40.1;
reg1 = 10.0.40.233;
eth.src = fa:16:3e:4a:e3:b5;
outport = "lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9";
flags.loopback = 1;
next;
lr_in_ip_routing_ecmp (ovn-northd.c:9374): reg8[0..15] == 0, priority 150, uuid a0d19c8d
cookie=0xa0d19c8d, duration=74244.353s, table=18, n_packets=649421, n_bytes=1138767765, priority=150,reg8=0/0xffff,metadata=0x12 actions=resubmit(,19)
next;
lr_in_arp_resolve (ovn-northd.c:9854): ip4, priority 0, uuid 7d603a2d
cookie=0x7d603a2d, duration=74244.302s, table=20, n_packets=456618, n_bytes=953953284, priority=0,ip,metadata=0x12 actions=push:NXM_NX_REG0[],push:NXM_NX_XXREG0[96..127],pop:NXM_NX_REG0[],set_field:00:00:00:00:00:00->eth_dst,resubmit(,66),pop:NXM_NX_REG0[],resubmit(,21)
get_arp(outport, reg0);
/* MAC binding to b4:09:31:47:30:ef. */
next;
lr_in_gw_redirect (ovn-northd.c:9085): ip4.src == 172.20.25.162 && outport == "lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9", priority 100, uuid 2c2a44fc
cookie=0x2c2a44fc, duration=10206.559s, table=23, n_packets=42796, n_bytes=156952532, priority=100,ip,reg15=0x1,metadata=0x12,nw_src=172.20.25.162 actions=resubmit(,24)
next;
lr_in_arp_request (ovn-northd.c:10055): 1, priority 0, uuid f0c7e08b
cookie=0xf0c7e08b, duration=74244.285s, table=24, n_packets=649421, n_bytes=1138767765, priority=0,metadata=0x12 actions=resubmit(,32)
output;

egress(dp="32d08999-d00a-49d2-bf8d-9169894b2a45", inport="lrp-039ba2bc-997c-448a-adb7-54bd3af697c1", outport="lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9")

lr_out_undnat (ovn-northd.c:8983): ip && ip4.src == 172.20.25.162 && outport == "lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9", priority 100, uuid ca864081
cookie=0xca864081, duration=10206.560s, table=40, n_packets=42796, n_bytes=156952532, priority=100,ip,reg15=0x1,metadata=0x12,nw_src=172.20.25.162 actions=set_field:fa:16:3e:85:0e:cb->eth_src,ct(table=41,zone=NXM_NX_REG11[0..15],nat)
eth.src = fa:16:3e:85:0e:cb;
ct_dnat;

ct_dnat /* assuming no un-dnat entry, so no change */

lr_out_snat (ovn-northd.c:9015): ip && ip4.src == 172.20.25.0/24 && outport == "lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9" && is_chassis_resident("cr-lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9"), priority 153, uuid e041811b
*** no OpenFlow flows
ct_snat(10.0.40.233);

ct_snat(ip4.src=10.0.40.233)

lr_out_egr_loop (ovn-northd.c:9120): ip4.dst == 10.0.41.228 && outport == "lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9" && is_chassis_resident("103e7777-a376-419d-9629-31bc6c7fedb1"), priority 100, uuid 5a133ec9
cookie=0x5a133ec9, duration=10206.560s, table=42, n_packets=47, n_bytes=4606, priority=100,ip,reg15=0x1,metadata=0x12,nw_dst=10.0.41.228 actions=clone(ct_clear,move:NXM_NX_REG15[]->NXM_NX_REG14[],set_field:0->reg15,set_field:0->reg10,load:0x1->NXM_NX_REG10[0],load:0->NXM_NX_XXREG0[96..127],load:0->NXM_NX_XXREG0[64..95],load:0->NXM_NX_XXREG0[32..63],load:0->NXM_NX_XXREG0[0..31],load:0->NXM_NX_XXREG1[96..127],load:0->NXM_NX_XXREG1[64..95],load:0->NXM_NX_XXREG1[32..63],load:0->NXM_NX_XXREG1[0..31],load:0->OXM_OF_PKT_REG4[32..63],load:0->OXM_OF_PKT_REG4[0..31],load:0x1->OXM_OF_PKT_REG4[0],resubmit(,8))
clone { ct_clear; inport = outport; outport = ""; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[0] = 1; next(pipeline=ingress, table=0); };

clone

ct_clear;
inport = outport;
outport = "";
flags = 0;
flags.loopback = 1;
reg0 = 0;
reg1 = 0;
reg2 = 0;
reg3 = 0;
reg4 = 0;
reg5 = 0;
reg6 = 0;
reg7 = 0;
reg8 = 0;
reg9 = 0;
reg9[0] = 1;
next(pipeline=ingress, table=0);

ingress(dp="32d08999-d00a-49d2-bf8d-9169894b2a45", inport="lrp-e9e8c0cb-2e81-4d20-9c28-44962ad1cbb9")

lr_in_admission: no match (implicit drop)

ovn-nbctl compiled for database schema 5.30.0 fails when talking to 5.27.0 schema (lr-policy-add)

ovn-nbctl --db=tcp:172.17.0.10:6641 lr-policy-add ovn_cluster_router 20000 ip4.dst == 100.2.0.0/16 reroute 169.254.254.1
2021-01-04T16:45:58Z|00002|ovsdb_idl|WARN|Load_Balancer table in OVN_Northbound database lacks options column (database needs upgrade?)
2021-01-04T16:45:58Z|00003|ovsdb_idl|WARN|Logical_Router_Policy table in OVN_Northbound database lacks nexthops column (database needs upgrade?)
2021-01-04T16:45:58Z|00004|ovsdb_idl|WARN|transaction error: {"details":"No column nexthops in table Logical_Router_Policy.","error":"unknown column","syntax":"{\"action\":\"reroute\",\"match\":\"ip4.dst == 100.2.0.0/16\",\"nexthops\":\"169.254.254.1\",\"priority\":20000}"}
ovn-nbctl: transaction error: {"details":"No column nexthops in table Logical_Router_Policy.","error":"unknown column","syntax":"{\"action\":\"reroute\",\"match\":\"ip4.dst == 100.2.0.0/16\",\"nexthops\":\"169.254.254.1\",\"priority\":20000}"}

Would it be possible for ovn-nbctl to detect the DB schema in runtime and use nexthop instead?

@numansiddique ^ ;-)