Giter Club home page Giter Club logo

cbl-marinerdemo's Introduction

Introduction

The CBL-Mariner repository provides detailed instructions for building CBL-Mariner from end-to-end. While it is possible to clone CBL-Mariner and build packages or images from that environment, for most users, it is not the recommended approach. Usually it is best to work in a smaller, problem focused environment where you can quickly build just what you need, and rely on the fact that the curated CBL-Mariner packages are already available in the cloud. In this way, you can customize an image with your preferred disk layout or adding supplemental packages that CBL-Mariner may not provide. If you are building a product based on CBL-Mariner, you may want your own repository with just the minimal set of packages for your business needs. This repo, the CBL-MarinerTutorials repo, provides a basic template for getting started. From here you can create a CBL-Mariner based product (aka a Derivative Image) or you may generate quick experimental or debug builds to try out new ideas.

When you build an ISO, VHD or VHDX image from this repository, the resulting image will contain additional content unavailable in the CBL-Mariner repo. The CBL-MarinerTutorials repository demonstrates how you can augment CBL-Mariner without forking the CBL-Mariner repository. This repository contains the SPEC file and source for building a simple "Hello World" application. This repository also includes a simple "os-subrelease" package that allows you to add identifying information about your derivative to an /etc/os-subrelease file.

Follow this decision tree to ensure you are using the correct repository for your use case:

---
title: Repo decision tree
---
flowchart TD
    id1{{Do you want to experiment with Mariner or contribute to Mariner?}}

    id2A[Do you want to build locally with Mariner?]
    id2B[Do you want to add an unsupported package? \n either a package Mariner has never supported \nor an updated major/minor version of a package Mariner supports]
    id2C[Do you want to use Mariner for your project? \n ex. bare metal, IoT, embedded devices, etc.]
    id2D[Use the CBL-MarinerTutorials repo]
    id1 -->|experiment| id2A
    id2A -.-|or| id2B
    id2B -.-|or| id2C
    id2C -->|yes to any of the above| id2D
    

    id3[Do you want to fix an issue in Mariner?]
    id3B[Do you want to add a common package? \n either a package supported by another major distro \nor a package used widely across popular open-source projects]
    id3C[Do you want to modify a supported Mariner package?]
    id3D[Do you want to rebuild Mariner from end-to-end?]
    id3E[Use the CBL-Mariner repo]
    id1 --> |contribute|id3
    id3 -.-|or| id3B
    id3B -.-|or| id3C
    id3C -.-|or| id3D
    id3D-->|yes to any of the above|id3E

Tutorial Table of Contents

cbl-marinerdemo's People

Contributors

anphel31 avatar christopherco avatar dmcilvaney avatar gjswalling avatar htaubenfeld avatar jslobodzian avatar mbykhovtsev-ms avatar microsoft-github-operations[bot] avatar microsoftopensource avatar ne-msft avatar neha170 avatar nicogbg avatar pawelwms avatar rikenm1 avatar rlmenge avatar samuel-lee-msft avatar sprt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cbl-marinerdemo's Issues

Issue at line 345 of CBL-Mariner/toolkit/tools/internal/safechroot/safechroot.go

I am doing a little testing - albeit outside the tested "x86_64 based machine using Ubuntu 18.04." I've made progress, but hit an issue with one failing test while building the toolkit.

Environment

I am testing in a 20.04 Gitpod environment - mimicking the 18.04 pre-requisites.

FROM gitpod/workspace-full:latest
RUN sudo apt update && \
    sudo apt upgrade --assume-yes && \
    sudo apt install rpm --assume-yes && \
    sudo apt install qemu-utils --assume-yes && \
    sudo apt install genisoimage --assume-yes && \
    sudo apt install pigz --assume-yes && \
    sudo ln -vsf /home/gitpod/go/bin/go /usr/bin/go

The above gives me the following as a starting point.

GNU Make 4.2.1
tar (GNU tar) 1.30
GNU Wget 1.20.3 built on linux-gnu.
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
RPM version 4.14.2.1
bash: qemu-utils: command not found
go version go1.16.5 linux/amd64
genisoimage 1.1.11 (Linux)
Python 3.8.11
bison (GNU Bison) 3.5.1
GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.2.0)
pigz 2.4
Docker version 19.03.15, build 99e3ed8919

Build Context

I am building the toolkit, with:

git clone https://github.com/microsoft/CBL-Mariner.git
pushd CBL-Mariner/toolkit
git checkout 1.0-stable
sudo make package-toolkit REBUILD_TOOLS=y
popd

At line 345 of CBL-Mariner/toolkit/tools/internal/safechroot/safechroot.go

	343:	// Remove this chroot from the list of active ones since it has now been cleaned up.
	344:	// Create a new slice that is -1 capacity of the current activeChroots.
	345:	newActiveChroots := make([]*Chroot, emptyLen, len(activeChroots)-1)

there is an unexpected error:

ERRO[0000] Mount failed on (/workspace/yarely/CBL-Mariner/toolkit/tools/internal/safechroot/_tmp/TestInitializeShouldCreateRoot/dev). Error: operation not permitted 
WARN[0000] Error creating mountpoints for chroot        
--- FAIL: TestInitializeShouldCreateRoot (0.00s)
    safechroot_test.go:62: 
                Error Trace:    safechroot_test.go:62
                Error:          Received unexpected error:
                                operation not permitted
                Test:           TestInitializeShouldCreateRoot
    safechroot_test.go:71: 
                Error Trace:    safechroot_test.go:71
                Error:          Should be true
                Test:           TestInitializeShouldCreateRoot
panic: runtime error: makeslice: cap out of range [recovered]
        panic: runtime error: makeslice: cap out of range
goroutine 21 [running]:
testing.tRunner.func1.2(0x5ea340, 0x665190)
        /home/gitpod/go/src/testing/testing.go:1143 +0x332
testing.tRunner.func1(0xc00008b200)
        /home/gitpod/go/src/testing/testing.go:1146 +0x4b6
panic(0x5ea340, 0x665190)
        /home/gitpod/go/src/runtime/panic.go:965 +0x1b9
microsoft.com/pkggen/internal/safechroot.(*Chroot).Close(0xc0000a3410, 0xc00008b200, 0x0, 0x0)
        /workspace/yarely/CBL-Mariner/toolkit/tools/internal/safechroot/safechroot.go:345 +0x1c8
microsoft.com/pkggen/internal/safechroot.TestInitializeShouldCreateRoot(0xc00008b200)
        /workspace/yarely/CBL-Mariner/toolkit/tools/internal/safechroot/safechroot_test.go:75 +0x251
testing.tRunner(0xc00008b200, 0x6371f8)
        /home/gitpod/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
        /home/gitpod/go/src/testing/testing.go:1238 +0x2b3

If I am reading this correctly, the issue is with releasing the chrooted enviroment (vs creating it). I don't know if this is specific to my Gitpod container, 20.04 or other. Happy to test a further if you have pointers.

dial tcp 216.58.200.49:443: i/o timeout,recipe for target ...internal.test_coverage' failed

after executed "sudo make package-toolkit REBUILD_TOOLS=y", I got Makefile:13: CONFIG_FILE is empty, defaulting to toolkit's core-efi.json unless CONFIG_FILE="" was set explicitly.
cd /home/my/CBL-Mariner/toolkit/tools/ &&
go test -covermode=atomic -coverprofile=/home/my/CBL-Mariner/build/tools/internal.test_coverage ./...
go: github.com/alecthomas/[email protected]: Get "https://proxy.golang.org/github.com/alecthomas/template/@v/v0.0.0-20190718012654-fb15b899a751.mod": dial tcp 216.58.200.49:443: i/o timeout
/home/my/CBL-Mariner/toolkit/scripts/tools.mk:89: recipe for target '/home/my/CBL-Mariner/build/tools/internal.test_coverage' failed

No matter what I setted proxy_all, it doesn't work. I tested curl www.google.com successfully. I don't know what to do

Go version 'go1.17.13' is less than minimum required version 'go1.19' while building toolkit

  • Ubuntu 22.04.03 LTS
  • x86_64
  • Clean OS Install

user@machine:~/local/repos/CBL-Mariner/toolkit$ sudo make package-toolkit REBUILD_TOOLS=y
Makefile:17: CONFIG_FILE is undefined, defaulting to toolkit's core-efi.json.
make: setfacl: No such file or directory

/home/user/local/repos/CBL-Mariner/toolkit/scripts/tools.mk:27: *** Go version 'go1.17.13' is less than minimum required version 'go1.19'. Stop

Cannot establish SSH connection to CBL-Mariner OS

I have installed the CBL-Mariner on my laptop and trying to SSH to the same from the laptop console. The host got the ECDSA key fingerprint but after that, it asks me to enter the password. It looks like the received public key is not matching with the stored version at the host. Any assistance you can provide would be greatly appreciated. Please find the log below.

ssh -v [email protected]

OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
debug1: Connecting to 172.31.88.105 [172.31.88.105] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_ed25519-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_xmss type -1
debug1: key_load_public: No such file or directory
debug1: identity file C:\Users\v-praraju/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.5
debug1: match: OpenSSH_8.5 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 172.31.88.105:22 as 'root'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: compression: none
debug1: kex: client->server cipher: [email protected] MAC: compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:bxV9a9vziCDEPZjonFKT3WByUxPc+HIql3hrvceQvA8
debug1: read_passphrase: can't open /dev/tty: No such file or directory
The authenticity of host '172.31.88.105 (172.31.88.105)' can't be established.
ECDSA key fingerprint is SHA256:bxV9a9vziCDEPZjonFKT3WByUxPc+HIql3hrvceQvA8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.31.88.105' (ECDSA) to the list of known hosts.
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected]>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey

debug1: Next authentication method: keyboard-interactive
debug1: read_passphrase: can't open /dev/tty: No such file or directory
Password:

404s when downloading toolchain

When downloading the toolchain via make build-packages

[azaugg@azaugg-ld3] [08:48:17] ~/CBL-Mariner/toolkit [2.0] $ sudo make build-packages CONFIG_FILE= Downloading toolchain RPM: filesystem-1.1-12.cm2.x86_64.rpm Downloading toolchain RPM: glibc-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-devel-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-i18n-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-iconv-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-lang-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-nscd-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: glibc-tools-2.35-3.cm2.x86_64.rpm Downloading toolchain RPM: zlib-1.2.12-2.cm2.x86_64.rpm Downloading toolchain RPM: zlib-devel-1.2.12-2.cm2.x86_64.rpm Downloading toolchain RPM: file-5.40-2.cm2.x86_64.rpm Downloading toolchain RPM: file-devel-5.40-2.cm2.x86_64.rpm Downloading toolchain RPM: file-libs-5.40-2.cm2.x86_64.rpm Downloading toolchain RPM: binutils-2.37-5.cm2.x86_64.rpm \nERROR: Failed to download toolchain package: binutils-2.37-5.cm2.x86_64.rpm. ERROR: Last 20 lines from log '/home/azaugg/CBL-Mariner/build/logs/toolchain/downloads/binutils-2.37-5.cm2.x86_64.rpm.log':\n Downloading toolchain RPM: binutils-2.37-5.cm2.x86_64.rpm https://packages.microsoft.com/cbl-mariner/2.0/prod/base/x86_64/binutils-2.37-5.cm2.x86_64.rpm: 2023-02-22 20:48:26 ERROR 404: Not Found. https://packages.microsoft.com/cbl-mariner/2.0/prod/base/debuginfo/x86_64/binutils-2.37-5.cm2.x86_64.rpm: 2023-02-22 20:48:27 ERROR 404: Not Found. https://packages.microsoft.com/cbl-mariner/2.0/prod/Microsoft/x86_64/binutils-2.37-5.cm2.x86_64.rpm: 2023-02-22 20:48:27 ERROR 404: Not Found. \nToolchain download failed. See above errors for more details. make: *** [/home/azaugg/CBL-Mariner/build/rpm_cache/cache/x86_64/binutils-2.37-5.cm2.x86_64.rpm] Error 1

Build-in-container instructions missing mariner-repos directory (and other notes)

https://github.com/microsoft/CBL-MarinerTutorials/tree/main/build-in-container

I tried running ./build-in-container/mariner-docker-builder.sh -c then ./build-in-container/mariner-docker-builder.sh -b from the root of the repo. It wasn't clear to me where you're meant to run it from (somewhere outside the repo?) based on this:

# Setup the container for 1st use
./CBL-MarinerTutorials/build-in-container/mariner-docker-builder.sh -t

After that, I got an error in the middle of the build, but it was a bit hard to understand what was going on from the end of the output because there's a lot of Go output to scroll through:

------------ Building Mariner toolkit ------------
++ echo '------------ Building Mariner toolkit ------------'
+++ nproc
++ make -j8 -C CBL-Mariner/toolkit package-toolkit BUILD_DIR=/mariner/build CHROOT_DIR=/temp/DockerStage/ CONCURRENT_PACKAGE_BUILDS=12 CONFIG_FILE= LOG_LEVEL=info OUT_DIR=/mariner/out REBUILD_TOOLS=y
make: Entering directory '/mariner/CBL-Mariner/toolkit'
SUDO_USER is not set, running 'go get' as user ''
echo "mariner-official-base.repo mariner-microsoft.repo" | sed -E 's:([^ ]+[.]repo):/mariner/SPECS/mariner-repos/\1:g' | xargs cat > /mariner/CBL-Mariner/toolkit/scripts/sodiff/sodiff.repo
cat: /mariner/SPECS/mariner-repos/mariner-official-base.repo: No such file or directory
cat: /mariner/SPECS/mariner-repos/mariner-microsoft.repo: No such file or directory
make: *** [/mariner/CBL-Mariner/toolkit/scripts/analysis.mk:73: /mariner/CBL-Mariner/toolkit/scripts/sodiff/sodiff.repo] Error 123
...
go: downloading github.com/google/uuid v1.3.0
go: downloading github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8
go: downloading golang.org/x/net v0.18.0
go: downloading github.com/golang-jwt/jwt/v5 v5.0.0
make: Leaving directory '/mariner/CBL-Mariner/toolkit'

I randomly guessed at adding this line to build-in-container/scripts/setup.sh to get a bit further:

         echo "------------ Building Mariner toolkit ------------"
+        cp -r CBL-Mariner/SPECS/mariner-repos SPECS/ && \
         make -j$(nproc) \

Then I got through Adding RPM to worker chroot and eventually it seemed not to find any specs. I left the hello_world_demo in place and thought it would build:

INFO[0003][rpmssnapshot] Generating RPMs snapshot from specs inside (/mariner/SPECS). 
INFO[0003][rpmssnapshot] Found 0 compatible specs.                    
INFO[0003][rpmssnapshot] The specs build 0 packages in total.         

(I was getting this working to have a local dev loop for microsoft/azurelinux#7446, but for the time being the buddy build seems fine.)

One of the pkgs does not exist and causing the build to fail

Hi.

After running the command sudo make iso CONFIG_FILE=../imageconfigs/demo_iso.json
I'm getting a 404 not found error for one of the pkgs.

Trying to get the pkg via browser confirms that it does not exist.

Full error from terminal:

ERROR: Failed to download toolchain package: gdbm-debuginfo-1.18-3.cm1.x86_64.rpm.
ERROR: Last 20 lines from log '/home/ohadm/CBL-MarinerDemo/build/logs/toolchain/downloads/gdbm-debuginfo-1.18-3.cm1.x86_64.rpm.log':

Downloading toolchain RPM: gdbm-debuginfo-1.18-3.cm1.x86_64.rpm
--2021-07-21 03:36:35--  https://packages.microsoft.com/cbl-mariner/1.0/prod/base/x86_64/rpms/gdbm-debuginfo-1.18-3.cm1.x86_64.rpm
Resolving packages.microsoft.com (packages.microsoft.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘packages.microsoft.com’
--2021-07-21 03:36:45--  https://packages.microsoft.com/cbl-mariner/1.0/prod/update/x86_64/rpms/gdbm-debuginfo-1.18-3.cm1.x86_64.rpm
Resolving packages.microsoft.com (packages.microsoft.com)... 104.214.230.139
Connecting to packages.microsoft.com (packages.microsoft.com)|104.214.230.139|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-07-21 03:36:46 ERROR 404: Not Found.

Toolchain download failed. See above errors for more details.

Also pic included.

Regards,

cbl-mariner-404-err

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.