nephio-project / sig-release Goto Github PK

See nephio-project/test-infra#56 (comment)

/assign @electrocucaracha

We need to setup CI/CD pipeline to set up the following tasks for the code repositories.

TODO

• #3
• #4

Create an E2E test bed per requirements defined in nephio-project/nephio#48
At a high level we need test bed that can a) Run in a local sandboxed environment b) can integrate with Nephio CI/CD (PROW) pipeline.

The name right now is us-central1-docker.pkg.dev/pure-faculty-367518/nephio-poc, which is not exactly memorable. We should look into:

• is it possible to create an alias like artifacts.nephio.dev/nephio-poc or artifacts-poc.nephio.dev
• alternatively can we create a project nephio-artifacts with Billing (requires LF approval) so that it can become us-central1-docker.pkg.dev/nephio-artifacts/poc

Moved from https://github.com/nephio-project/test-infra/issues/2

We need to establish clear roles and responsibilities in the community for helping with the Nephio release process. We can start by having a Release Lead (RL) role which will be a role that will be rotated across SIG release members for every minor release.
Release lead will be responsible for the following,
Coordinate with other SIG(s) on finalizing the issues that are being targeted for the release.
Lead all the phases of the release process defined earlier.
Lead the release postmortem for the release.
Identifying the RL for the next release by working with SIG release.

We can define additional roles as required for the release process as we learn from our experience with doing a few releases.

Expected Behavior:

fossology tests passing without errors

Current Behavior:

cp: cannot stat 'results': No such file or directory

Possible Solution:

Steps to Reproduce

Set up the PROW CI/CD infrastructure to build code and push the images to Nephio dockerHub

Define the components and design of the test bed such as simulators, injectors and other systems involved. Document the test bed details such as components, topology , flows etc.

• Management cluster creation and deployment of all Nephio component and dependencies.
• Creation of repos, creation of 2-3 edge clusters and deployment of Nephio components and dependencies on edge cluster. Configuration of edge clusters (such as configsync repo set up), GRPC connection set up with management cluster and inter edge cluster communication set up.

https://github.com/cncf/devstats-example

Low priority.

Moved from https://github.com/nephio-project/test-infra/issues/3

Project is open for commits from all contributors, with OWNERs files and other review processes in place

Configure Prow infrastructure to be sensitive to following labels

1. lgtm
2. donotmerge
3. ...

We need to define and document contributor guidelines for the project. We can take a lot of inspiration from the k8s project for this such as signing CLAs, contributor etiquettes, PR process, getting started etc. Here are the relevant links from the k8s project

https://github.com/kubernetes/community/tree/master/contributors/guide
https://github.com/kubernetes/community/blob/master/contributors/guide/contributing.md
https://www.kubernetes.dev/docs/contributor-cheatsheet/

Based on the above we need to define the guidelines as per our needs and document them in GitHub and/or wiki.

Setup TestGrid for easily accessing/debugging the test results

For example here is a k8s testgrid for reference

Define the scope and requirements of the integration and system tests. Document them in GitHub and/or wiki.

We need to configure Prow infrastructure to be aware of and take actions based on labels such as lgtm, donotmerge etc.

Todo

• #6
• #7

I built it from https://github.com/nokia/k8s-ipam and pushed it as nephio/resource-backend:latest, which is what the package expects. But we probably need to automate this in some way. Perhaps if the repo makes its way to nephio-project at some point.

cc @henderiw @radoslawc

See initial discussion in issue #10.

We need to make sure we have the docker registry available to push and pull images.

We need to define the GitHub project management process for the Nephio project. As a part of this we will define and document

• Issue types
• Issue Type templates
• labels
• Milestones
• Status
• Custom fields
• Project boards
• PR life cycle
• Branching /Tagging

These are discussed more in the following documents.
https://docs.google.com/document/d/17x5HNCAlsjNzFHhAlCkC-IlhX2I26kqghYunFTSW_Ls/edit#heading=h.amd9l4e7ztbx
https://docs.google.com/document/d/1y-m4VQk67qlwk0JpcbfGroIinOm78S8itFWrTZDIFw4/edit#

Configure Prow infrastructure to make sure CLA ( either ICLA or CCLA) is signed prior to contributions.

We need to setup the test bed for integration and system tests and integrate with Prow so that they can be automatically run as part of the pipeline. This involves multiple tasks. Following are some of the identified tasks and will be triaged further.

Todo

• #12
• #13
• #14
• #15

Setup PROW CI/CD pipeline to run unit tests on the different nephio repos.

Before I start, thanks for Nephio and all the great stuff! I am quite not 100% sure if this is the right place for issues with Nephio demo as well, however was not able to find another repo that seems to fit in better. If there is another place that I shall be reporting this issue, please let me know.

Expected Behavior:

According to the Free5GC user guide in R2, the UE pod is expected to perform following commands successfully.

UE_POD=$(kubectl --kubeconfig edge01-kubeconfig get pods -n ueransim -l app=ueransim -l component=ue -o jsonpath='{.items[0].metadata.name}')
kubectl --kubeconfig edge01-kubeconfig -n ueransim exec -it $UE_POD -- /bin/bash -c "ping -I uesimtun0 172.0.0.1"

Current Behavior:

However, unfortunately, the UE does not seem to have the uesimtun0 interface. Instead, it only has following interfaces:

root@ueransimue-edge01-56fccbc4b6-k647k:/ueransim/build# ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if29: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 26:5d:c2:c7:c3:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.23/24 brd 192.168.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::245d:c2ff:fec7:c392/64 scope link
       valid_lft forever preferred_lft forever

Thus resulting:

root@ueransimue-edge01-56fccbc4b6-k647k:/ueransim/build# ping -I uesimtun0 172.0.0.1
ping: SO_BINDTODEVICE: Invalid argument

I have been reinstalling Nephio and Free5GC in multiple GCP instances as well, however this does not get solved. So I am trying to create a new issue.

Possible Solution:

Honestly speaking, I have little background in UERANSIM as well as Free5gc-cp. However, I have been looking for solutions and found an issue with uesimtun0 not existing in a VM (yes, in Nephio it shall be a container). I have attached the link to the issue just in case this might help troubleshoot this issue in the future.

aligungr/UERANSIM#369

Steps to Reproduce

I have tried with R1 before as well, unfortunately, it gave me the same results as R2.

1. Follow the exact same steps from the official Free5GC user guide provided by Nephio till step 8.
2. Execute the mentioned command which sends a ping request to 172.0.0.1 using uesimtun0
3. The ping: SO_BINDTODEVICE: Invalid argument error appears

Further Info

Just in case this might provide extra information on what is going on with my cluster, the following text is the log from amf-regional pod from regional-admin@regional cluster:

2024-03-24T19:38:07Z [INFO][AMF][NGAP][172.2.0.254:38068][AMF_UE_NGAP_ID:1] Uplink NAS Transport (RAN UE NGAP ID: 1)
2024-03-24T19:38:07Z [INFO][LIB][FSM] Handle event[Gmm Message], transition from [Registered] to [Registered]
2024-03-24T19:38:07Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:1][SUPI:imsi-208930000000003] Handle UL NAS Transport
2024-03-24T19:38:07Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:1][SUPI:imsi-208930000000003] Transport 5GSM Message to SMF
2024-03-24T19:38:07Z [WARN][AMF][GMM][AMF_UE_NGAP_ID:1][SUPI:imsi-208930000000003] Duplicated PDU session ID[1]
2024-03-24T19:38:07Z [ERRO][AMF][GMM][AMF_UE_NGAP_ID:1][SUPI:imsi-208930000000003] PDU Session ID[1] can't be released in DUPLICATE_SESSION_ID case
2024-03-24T19:38:07Z [INFO][AMF][GMM][AMF_UE_NGAP_ID:1][SUPI:imsi-208930000000003] Send DL NAS Transport
2024-03-24T19:38:07Z [INFO][AMF][NGAP][172.2.0.254:38068][AMF_UE_NGAP_ID:1] Send Downlink Nas Transport

This is thrown when I try to execute the wrapper.sh inside the ueransimue like below:

root@ueransimue-edge01-56fccbc4b6-b9ttw:/ueransim/build# ./wrapper.sh 
mkdir: cannot create directory '/dev/net': File exists
mknod: /dev/net/tun: File exists
UERANSIM v3.2.6
[2024-03-24 19:38:41.617] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
[2024-03-24 19:38:41.618] [rrc] [debug] New signal detected for cell[1], total [1] cells in coverage
[2024-03-24 19:38:41.618] [nas] [info] Selected plmn[208/93]
[2024-03-24 19:38:41.618] [rrc] [info] Selected cell plmn[208/93] tac[1] category[SUITABLE]
[2024-03-24 19:38:41.618] [nas] [info] UE switches to state [MM-DEREGISTERED/PS]
[2024-03-24 19:38:41.618] [nas] [info] UE switches to state [MM-DEREGISTERED/NORMAL-SERVICE]
[2024-03-24 19:38:41.618] [nas] [debug] Initial registration required due to [MM-DEREG-NORMAL-SERVICE]
[2024-03-24 19:38:41.619] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2024-03-24 19:38:41.619] [nas] [debug] Sending Initial Registration
[2024-03-24 19:38:41.619] [nas] [info] UE switches to state [MM-REGISTER-INITIATED]
[2024-03-24 19:38:41.619] [rrc] [debug] Sending RRC Setup Request
[2024-03-24 19:38:41.620] [rrc] [info] RRC connection established
[2024-03-24 19:38:41.620] [rrc] [info] UE switches to state [RRC-CONNECTED]
[2024-03-24 19:38:41.620] [nas] [info] UE switches to state [CM-CONNECTED]
[2024-03-24 19:38:41.669] [nas] [debug] Authentication Request received
[2024-03-24 19:38:41.681] [nas] [debug] Security Mode Command received
[2024-03-24 19:38:41.681] [nas] [debug] Selected integrity[2] ciphering[0]
[2024-03-24 19:38:41.899] [nas] [debug] Registration accept received
[2024-03-24 19:38:41.899] [nas] [info] UE switches to state [MM-REGISTERED/NORMAL-SERVICE]
[2024-03-24 19:38:41.899] [nas] [debug] Sending Registration Complete
[2024-03-24 19:38:41.899] [nas] [info] Initial Registration is successful
[2024-03-24 19:38:41.899] [nas] [debug] Sending PDU Session Establishment Request
[2024-03-24 19:38:41.899] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2024-03-24 19:38:51.285] [nas] [error] PDU Session Establishment Reject received [OUT_OF_LADN_SERVICE_AREA]
[2024-03-24 19:38:51.285] [nas] [error] Received PTI [0] value is invalid

I have checked this issue, however, I am not sure how to fix this.

If any further logs are required, please let me know.
Thank you everyone in advance for any assistance.

Set up test infrastructure for integration and system tests based on scope and design.
Refer to #14 for details.

• #30
• #31
• [ ]

https://github.com/kubernetes/test-infra/tree/master/label_sync

Moved from https://github.com/nephio-project/test-infra/issues/4

Integrate test bed infrastructure with Prow for automatically invoking integration and system tests. This will include allocating/configuring/deallocating test bed infrastructure for running integration and system tests.

I have tried this in a couple PRs but had to manually close them. Example: nephio-project/nephio#222

I cannot recall if the other one was in the nephio repo or not; I think it might have been in test-infra or nephio-example-packages but I am not sure.

/assign @radoslawc

Integrated PROW with dockerHub to enable the delivery pipeline.

The docker hub used for Nephio images name right now is us-central1-docker.pkg.dev/pure-faculty-367518/nephio-poc, which is not exactly memorable. We should look into:

Is it possible to create an alias like artifacts.nephio.dev/nephio-poc or artifacts-poc.nephio.dev
alternatively can we create a project nephio-artifacts with Billing (requires LF approval) so that it can become us-central1-docker.pkg.dev/nephio-artifacts/poc
Moved from https://github.com/nephio-project/test-infra/issues/2

While testing out the e2e scripts, I was seeing a problem where some packages were not getting applied. It turns out I had a typo in my "version":

    - {pkg: configsync, version: config-sync/v1, namespaces: [config-management-monitoring, config-management-system, resource-group-system]}

The tag config-sync/v1 does not exist; the correct tag is configsync/v1. When running kpt pkg get manually, I get this:

jbelamaric@nephio-r1-e2e:~$ /usr/local/bin/kpt pkg get https://github.com/nephio-project/nephio-example-packages.git/configsync@config-sync/v1 /tmp/jj
Package "jj":
Fetching https://github.com/nephio-project/nephio-example-packages@config-sync/v1
Error: Unknown ref "config-sync/v1". Please verify that the reference exists in upstream repo "https://github.com/nephio-project/nephio-example-packages".

Details:
fatal: ambiguous argument 'config-sync/v1': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
 
jbelamaric@nephio-r1-e2e:~$ echo $?
1

With a correct tag, the return value is 0.

I think the kpt.py should fail on a non-zero return value from kpt.

Document the procedure to :

1. Create development and build environment
2. Build and test code
3. PR process and guidelines

Configure PROW to schedule build and test jobs in build cluster.

For Context please refer to https://docs.prow.k8s.io/docs/build-clusters/#let-prow-schedule-your-jobs-into-your-build-cluster

While testing out the sandbox provisioning, it was failing during the "install nephio-operator". Turns out there was no image pushed :)

I manually build and pushed an image, but we should have it automated.

This one: https://github.com/nephio-project/nephio/tree/main/operators/nephio-controller-manager

Nephio Cloud as target test environment

• cluster and network topology
• set up of IP address pools for each clusters
• VLAN for secondary interface segregation
• continuous running test environment (for two weeks before release) on Google cloud (Nephio account)
• IP address assignment to externally accessible service (such as EdgeWatcher)

SIG2 needs to provide requirements on what is expected from the integration i.e component test bed.

Define the scheme for release versioning and cycles and document them. We need to define

• Release Cadence ( How often we need to make a release? )
• What is the k8s version to test and support
• Length of support for a release
• Versioning scheme (x.y.z)
• Hot fixes/back porting mechanism