The trick is to create a data partition at an appropriate offset using Ignition start_mib, such that the root FS gets resized to the desired size.

The current version of the Provisioning Fedora CoreOS on DigitalOcean documentation contains the following commands:

image_id=$(doctl compute image list-user | grep my-fcos-image | cut -f1 -d\ )
key_id=$(doctl compute ssh-key list | grep my-key | cut -f1 -d\ )

cut -f1 -d\, however, gives a syntax error. I believe cut -f1 -d \t is intended, as that command returns the droplet id and not an error. I will be submitting a PR for this shortly.

Thanks for all your hard work and consideration!

NM by default will try to DHCP on every interface that's not explicitly configured. Let's document how to disable that behavior because sometimes bringing up interfaces that aren't desired to be brought up has unintended consequences. Example:

storage:                                                                                                                                                                                                                                     
  files:                                                                                                                                                                                                                                                                                                                                                                                                                                             
    - path: /etc/NetworkManager/conf.d/10-disable-unconfigured-nics.conf                                                                                                                                                                     
      mode: 0644                                                                                                                                                                                                                             
      contents:                                                                                                                                                                                                                              
        inline: |                                                                                                                                                                                                                            
          [main]                                                                                                                                                                                                                             
          no-auto-default=*

As part of this we should link to the NM documentation that explains what this does in more detail.

We want to fix this problem eventually, but for now let's add a FAQ entry to address it that we can link people to. The FAQ entry should describe the issue briefly, link to the issue and display the workaround.

It might be beneficial to add a tutorial like what we did for devconf earlier this year to our documentation here. Should we consider trying to update the steps there, formalize the writing style, and add it to our docs somewhere?

I have noticed on the AWS CoreOS user guide[0], there is no mention of what user to login as if the user does not spin up the machine with an ignition file. This is fairly confusing. Can this be changed/modified to show that the user can log into the machine using the “core” account?

[0] https://docs.fedoraproject.org/en-US/fedora-coreos/getting-started/

Provide a recommendation and examples for how to run additional administration tools in a privileged container.

Hi,

During my migration from CL to FCOS, the biggest difficulty I ran into was configuring a static ip address using a pre-seeded configuration. For CL, this worked by inserting an interface configuration under /etc/systemd/network/, but since FCOS has standardized on NetworkManager (see coreos/fedora-coreos-tracker#24), I needed to migrate this. The following is a writeup of how I accomplished this, together with a description of my usecase and implementation details. If the project is interested, I can rework this into a pull request.

First off, my usecase. I run a bunch of centos8/kvm hypervisors, running FCOS virtuals, which host (sets of) docker/podman containers. The virtuals are using LVM logical volumes as block devices. Because of coreos/coreos-installer#91, I have a set of scripts which fetches the latest FCOS using a dockerized coreos-installer, and I use dd to write this to the LV. Next, I partprobe this device, mount the first partition, and upload my ignition config to ignition/config.ign, umount everything, generate a KVM configuration and boot up the VM. All this is wrapped in a Makefile, so 'make ' and 5 minutes of time is enough to fully bootstrap a FCOS vm.

To get this setup to work using NetworkManager, I had to change a few bits in my configuration:

• I set a fixed mac address for the VM
• I altered the kernel boot parameters to prevent dracut from requesting a DHCP lease:

sed -i -e 's,rd.neednet=1 ip=dhcp,rd.neednet=0,g' /mnt/grub2/grub.cfg

• I inserted a NM system connection using ignition:

storage:
  files:
    - path: /etc/NetworkManager/system-connections/eth0.nmconnection
      mode: 0600
      contents:
        inline: |
          [connection]
          id=Uplink
          type=802-3-ethernet
          autoconnect=true
          interface-name=eth0
          match-device=interface-name:eth0
          uuid=<randomly generated>
          [ethernet]
          mac-address=<generated mac address for vm>
          [ipv4]
          method=manual
          addresses=192.0.2.1/24
          gateway=192.0.2.255
          dns=192.0.2.53
          dns-search=example.com

After these changes, I was able to boot the VM using a static ip address, without it doing a DHCP request @ boot.

This is splitting off my comment from coreos/fedora-coreos-tracker#362 (comment):

Specifically, I'm interested in reaching a final UX flow where Ignition writes NM wireguard profiles (keyfiles) and secret material, and tunnel provisioning works without further custom scripting.

We should have a doc page showing how to perform this with fcct snippets.

(JL: Put under System Configuration in new "Configuring Network" subheading)

Since I am a new FCOS user I might have a nice uninformed perspective. Via some tickets I will share some problems I ran into, in the hope that others might find them and be helped by it.

When I follow the instructions on 2020-12-05 on this page :
https://web.archive.org/web/20201206115935/https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/
then the resulting ISO is fedora-coreos-32.20201104.3.0-live.x86_64.iso.

Later I found that instead of the podman approach (which was quite a hassle for me) I was also able to just download the Live ISO from here :
https://web.archive.org/web/20201205115430if_/https://getfedora.org/coreos/download?tab=metal_virtualized&stream=stable
but the resulting ISO is fedora-coreos-32.20201018.3.0-live.x86_64.iso.

Point of this ticket is to suggest 3 small fixes:

1. make sure that the ISO from the getfedora.org page is up to date (seems to be older)
2. add a link on the docs.fedoraproject.org to the getfedora.org page
3. clarify on the docs.fedoraproject.org page whether we get stable, testing or next

We need a highlevel document that explains the options users have to configure networking. Because Networking may be required in the initramfs to fetch the ignition configuration it can't be the only source of truth for networking information, thus we have kargs, coreos-installer --copy-network, and side channels on some platforms (VMWare). It would be good if we have a highlevel document that explains the relationship between these options and when the user should choose one over the other.

We should also explain the "conditional" nature of initramfs networking since we recently implemented a feature that would only bring up networking in the initramfs if it's determined to be needed to fetch the ignition config, or if the user explicitly asked for it.

Some good material for this could come from openshift/enhancements#394

Or at least add some links to documentation that explains the network configuration options and format. For example I need to configure bonds and vlans on bare metal and I can't find any information about how to do it.

I stumbled over some undocumented breaking changes. Fedora CoreOS uses --default-ulimit nofile=1024:1024 in /etc/sysconfig/docker. This breaks some applications. Affected containers can be started with docker run --ulimit nofile=[LIMIT] [CONTAINER].

Since I am a new FCOS user I might have a nice uninformed perspective. Via some tickets I will share some problems I ran into, in the hope that others might find them and be helped by it.

Please be aware that various ways of burning to USB stick may lead to different results. The purpose of this ticket is to see if we can improve the documentation to include a 'known working approach'.

Background :

When burning to a USB stick I get different response based on how I burn it:

Attempt 1:

• when burning "fedora-coreos-32.20201104.3.0-live.x86_64.iso" with tuxboot-0.8.3.exe
• and booting in BIOS mode the booting halts with the message "no configuration file"
• when I provide the IGN filename it results in the messsage "booting kernel failed: invalid argument"
• it seems that the configuration file it is missing is an SYSLINUX CFG file
• trying all CFG files on the stick, like "EFI/fedora/grub.cfg", result in "failed to load COM32 file EFI/fedora/grub.cfg.c32"

Attempt 2:

• when burning "fedora-coreos-32.20201018.3.0-live.x86_64.iso" with tuxboot-0.8.3.exe
• it fails while burning at the action "Running special E:\utils\win32\syslinux.exe on E:" which prompts Windows dialog "This file does not have an app associated with it..."

Attempt 3:

• when burning "fedora-coreos-32.20201018.3.0-live.x86_64.iso" with rufus-3.13p.exe in "ISO Image mode"
• and booting in either BIOS or UEFI mode the CoreOS splash-screen comes by, but booting fails with message "failed to mount /run/media/iso"

Since this at least starts the CoreOS, so this suggests that rufus is a better alternative for tuxboot here!
Earlier problems with Ubuntu Live USB in the past made me switch from unetbootin to tuxboot based on the Ubuntu documentation.
It might be worth to document a 'known working setup' for FCOS in the documentation as apparently YMMV.

Attempt 4:

• when burning "fedora-coreos-32.20201018.3.0-live.x86_64.iso" with rufus-3.13p.exe in "DD Image mode"
• and booting in the only available mode (unclear which it is) the CoreOS bash prompt comes up as expected

As we can see this took a lot of trial and error and the error messages above were not very helpful.

It might be worth to suggest in the documentation this known working approach for Windows: "use rufus-3.13p.exe in DD Image mode"

This might exist but I couldn't find it easily so I'll just put this here. We should add some examples of the local files feature of fcct:

variant: fcos
version: 1.1.0
storage:
  # call with --files-dir=/path/to/files/zram-generator
  trees:
    - local: fakeroot

$ tree /path/to/files/zram-generator/fakeroot/
/path/to/files/zram-generator/fakeroot
└── etc
    └── systemd
        ├── system
        │   └── [email protected]
        ├── system-generators
        │   └── zram-generator
        └── zram-generator.conf
$ cat fcct_zram_generator.yaml | fcct --files-dir=/path/to/files/zram-generator

Another example this time merging configs:

variant: fcos
version: 1.1.0
ignition:
  config:
    merge:
      - local: ./generated-files/master.ign

cat resources/fcct-control-plane.yaml | fcct -d ./ -o ./generated-files/control-plane-processed.ign

See https://docs.docker.com/engine/reference/commandline/dockerd.
E.g. to pick a few common ones: changing the log driver, debug mode, and insecure registries.

Would be nice to use configuration files, but it doesn't support drop-ins, so isn't very friendly to FCOS.

The moby-engine package we ship uses /etc/sysconfig/docker as an environment file for the daemon systemd unit. It's clunky but probably the cleanest approach for now is to do similarly to what's described in coreos/fedora-coreos-tracker#588 and add a systemd unit override to just redefine ExecStart so we can append arbitrary flags.

(Though we should also look into what are the upstream plans for supporting a config overlay paradigm.)

Acceptance Criteria:

• document how to change the log driver
• document how to add insecure registries
• (optional) document how to enable debug mode

https://docs.fedoraproject.org/en-US/fedora-coreos/ign-storage/

For example, the GCE doc tells you that you don't need to customize the instance and you can SSH in but never how to do so. The only place the 'core' user is mentioned seems to be in the container tutorial. The ask would be to add the basic here's how you use the 'core' user to SSH into a Fedora CoreOS VM in different environments in the getting started guides.

I've been trying to use FreeIPA Container to manage some new infrastructure using FCOS as the primary platform. I found this comment on the fedora-coreos-tracker issues that provided some of the info, and I added some bits that seem to be required from other reading I've done.

The result, is the fcct snippet that is attached.

fcct-sssd_freeipa-snippet.yml.txt

This configuration results in a somewhat working sssd. If I'm logged in as the core user, I can successfully get some information, but not everything.

[core@example-0 ~]$ id dcode
uid=1024800001(dcode) gid=1024800001(dcode) groups=1024800001(dcode)
[core@example-0 ~]$ getent passwd dcode
dcode:*:1024800001:1024800001:Derek Ditch:/home/dcode:/bin/sh
[core@example-0 ~]$ getent group dcode
dcode:*:1024800001:
[core@example-0 ~]$ getent group admins
admins:*:1024800000:
[core@example-0 ~]$ sss_ssh_authorizedkeys dcode
Error looking up public keys

Since sssd is clearly in the platform for this explicit purpose, can we get a working example against FreeIPA? and maybe some people might be interested in AD or something.

Thanks!

At https://docs.fedoraproject.org/en-US/fedora-coreos/fcct-config/#_disks there is:

wipe_partition_entry (boolean, optional)
    true: Ignition erases any existing partition that does not match this entry.
    false: Ignition leaves the partition unchanged, which will fail.

However, it is not defined what the default value is.

The official spec docs are on the FCCT docs site. Meanwhile, the FCOS docs site has a divergent, outdated, buggy, unmaintained spec writeup which also claims to be the official spec. We've concluded internally that maintaining two separate copies of the spec is unworkable, and that the path forward is to

1. replace that portion of the FCOS docs with a link to the FCCT spec docs, but
2. continue to maintain example FCCs on the FCOS site.

I don't see any LICENSE or COPYING file in the repository.

• Live PXE/ISO is awkwardly documented at the bottom of a page that bills itself as "Installing on Bare Metal". Maybe it should be a separate page, or maybe the "installing" part should be renamed.

• We don't document coreos-installer iso embed anywhere.

• We should highlight coreos-installer download.

• We don't document initrd-embedded Ignition configs anywhere except bullet 4 here. See coreos/coreos-installer#322.

• We should document hooking or replacing coreos-installer.service. In the latter case, we should figure out what to recommend. Units should at least pull in coreos-installer-pre.target.

• The live ISO now includes the live PXE artifacts. We should probably mention that.

ref: https://discussion.fedoraproject.org/t/zincati-update-strategy-time-based/21929/2

zincati 0.0.11 seems to be in all streams now.

Add a non-required CI job that checks PRs for:

• Removal of existing pages
• Removal of existing anchors (~= headings) within a page

The current bare metal documentation says:

This guide provides instructions to install Fedora CoreOS to bare metal. Two options are available:

• Installing from live ISO
• Installing from PXE

However, when you look at the download page, you'll not only see ISO and PXE, but also Raw and Raw (4k Native). Those two options are not documented.

Especially for "Raw (4k Native)", it's very unclear what it is. "4k" is most commonly known as a display resolution. So not only this would benefit from having documentation, but even renaming the download option to make it more clear would help a lot as well. We don't need to save on characters, so naming them e.g. Raw disk image and Raw disk image (native 4KB sectors) could go a long way.

I believe below is how you can 1) run any shell command during installation and 2) set the timezone of the machine. I think it would be helpful if this kind of information was spelled out in the documentation over at https://docs.fedoraproject.org/en-US/fedora-coreos/

This is technically a systemd service, but it only runs once. Seems an odd way to set the timezone to me, but I am not sure how else to do it in the fcc file.

systemd:
  units:
    - name: settimezone.service
      enabled: true
      contents: |
        [Unit]
        Description=Set the time zone

        [Service]
        ExecStart=/usr/bin/timedatectl set-timezone Europe/Paris
        RemainAfterExit=yes
        Type=oneshot
        
        [Install]
        WantedBy=multi-user.target

The "Provisioning Machines" section is currently missing a "Booting on OpenStack" page.

We do publish qcow2.xz images for the openstack platform, but we do need instructions on how to import the image to an openstack platform and launch a VM based on that.

A user may want a user unit to run on boot without them having to log in. sudo loginctl enable-linger $USER exists for this but running commands doesn't fit the model very well.

You can achieve the same goal as sudo loginctl enable-linger $USER by creating an empty file in /var/lib/systemd/linger/$USER. So for the core user it would be:

variant: fcos
version: 1.1.0
storage:
  files:
    - path: /var/lib/systemd/linger/core
      mode: 0644
      contents: 
        inline: ''

There are few dedicated options in FCCT/Ignition to manipulate GECOS user, but their documentation is a bit spotty. We can start from https://coreos.com/os/docs/latest/adding-users.html, polish that a bit, and add more FCCT examples.

Things I have in mind:

• mention the default (privileged) administration user core
• basic customizations to core: SSH keys, groups
• how to add new users/groups
• password generation and customization
• sudo setup

Something like this should work:

variant: fcos
version: 1.1.0
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - $sshpubkey
storage:
  files:
    - path: /etc/hostname
      mode: 0644
      contents:
        inline: |
          ignitionhostname

This issue is part of "Getting started" section - https://docs.fedoraproject.org/en-US/fedora-coreos/getting-started/

There is a mention to use coreos-installer to download the image.
But, from where to download coreos-installer binary itself is not mentioned.

Note: The path where it should be saved also needs be mentioned.

This is somewhat described in coreos/fedora-coreos-tracker#553. For now, because of what we decided, you'd need to do:

In all cases, apply the name to the real root:

storage:
  files:
    - path: /etc/systemd/network/25-infra.link
      mode: 0644
      contents:
        inline: |
          [Match]
          MACAddress=12:34:56:78:9a:bc
          [Link]
          Name=infra

In the initrd via kernel argument only if you need this customization in the initrd:

ifname=infra:12:34:56:78:9a:bc

See coreos/fedora-coreos-tracker#553 for why you might want this.

Acceptance Criteria:

• document how to set network interface via link file
• (optional) document how to set network interface using kernel arguments

Build on top of #121 and document live PXE with /var on disk. (See also https://fedorapeople.org/~walters/devconfcz2020/fcos-live-pxe.html.)

People get the mount units wrong a lot. Let's document our helper that makes things much easier. Example:

variant: fcos
version: 1.1.0
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAA...
storage:
  disks:
    - device: /dev/sda
      partitions:
        - number: 0
          label: var-log
          size_mib: 0
          start_mib: 0
  filesystems:
    - device: /dev/disk/by-partlabel/var-log
      path: /var/log
      format: xfs
      wipe_filesystem: true
      with_mount_unit: true

The "Provisioning Machines" section is currently missing a "Booting on Aliyun" page.

We do publish qcow2.xz for the aliyun platform, but we do need instructions on how to import the image to the cloud and launch a VM based on that.

This was originally reported in Red Hat Bugzilla, I'm copying it here for visibility. https://bugzilla.redhat.com/show_bug.cgi?id=1802159

Description of problem:

I was just giving Fedora Core OS a first try and installed it from the ISO image to the bare metal following the instructions at

https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/

Since I didn't have an ignition file yet, I just ran

sudo coreos-installer install /dev/sda

with out the ignition file, and this worked.

After rebooting, the system came up, but I could not login on the console. The Docs do not contain a hint about what username and what password to use. When i mounted the file system on a different linux machine I saw that there is a core use with no password set. Since I'm not familiar with ostree, I wasn't sure whether I was breaking anything if I just changed the password in the /etc/shadow on that disk, and this is probably not the intended way to use the system.

You end up with having a system that's of no use since you can't login.

Either the docs should tell how to login after plain install, or coreos-installer should prevent from installing a system with no use and no login.

Write an example FCCT config to setup a custom NTP server for Chrony.

Chrony documentation: https://chrony.tuxfamily.org/documentation.html

Right now, we have some storage examples nested in the "FCCT Specification" page. We should split that out into a dedicated "Configuring Storage" page. Examples of what this page could have:

• configuring /var (or submountpoints) on a separate partition on the primary disk
• configuring /var (or submountpoints) on a separate partition on a separate disk
• configuring /var on RAID

This will be prep as well for reconfiguring the root filesystem itself once we support that (coreos/fedora-coreos-tracker#94).

When I run the steps in https://docs.fedoraproject.org/en-US/fedora-coreos/tutorial-autologin/ :

Start a Fedora CoreOS virtual machine

virt-install --name=fcos --vcpus=2 --ram=2048 --os-variant=fedora32
--import --network=bridge=virbr0 --graphics=none
--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${PWD}/autologin.ign"
--disk=size=20,backing_store=${PWD}/fedora-coreos.qcow2

I received an error :
[root@centos7 coreos]# virt-install --name=fcos --vcpus=2 --cpu host-passthrough --ram=2048 --os-variant=fedora30 \

--import --network=bridge=virbr0 --graphics=none \
--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${PWD}/autologin.ign" \
--disk=size=20,backing_store=${PWD}/fedora-coreos.qcow2

Starting install...
Allocating 'fcos-1.qcow2' | 20 GB 00:00:00
ERROR internal error: process exited while connecting to monitor: qemu-kvm: -fw_cfg: invalid option
Removing disk 'fcos-1.qcow2' | 0 B 00:00:00
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh --connect qemu:///system start fcos
otherwise, please restart your installation.

I am trying to fix this issue and could not find. Is there anyone fix this issue?

As $title says, a-la-rustdoc --test or what Ignition does.

This would also ensure that we're not merging things which haven't actually hit stable yet.

When following the sections Booting with QEMU and Generating Ignition configs, using the example config provided adding my SSH key to the core user, and SSHing in with ssh core@<ip> (where <ip> is the address assigned by the QEMU built-in DHCP server) does not appear to work.

After editing /etc/ssh/sshd config in the FCOS VM to comment out PasswordAuthentication no, set PasswordAuthentication yes, and PubkeyAuthentication yes, and then systemctl restart sshd, SSHing into the VM gives the following (full log under Details below):

$ ssh -p 22 -vvv [email protected]
...
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/rfairley/.ssh/id_rsa RSA SHA256:dwilTJGh02bpSp0U7BLPrmC4wn2wJ40lxOpmkf0fsbY agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
...
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
[email protected]'s password: 
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
Permission denied, please try again.

(Entering the same password I set for core through an Ignition config, and used to log in through the serial console).

From looking at what cosa run does for SSH, pubkey authentication worked using hostfwd:

qemu-system-x86_64 -machine accel=kvm -m 2048 -cpu host -nographic -drive if=virtio,file=fedora-coreos-30.20190801.0-qemu.qcow2 --netdev user,id=eth0,hostname=coreos,hostfwd=tcp::2229-:22 -device virtio-net-pci,netdev=eth0 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci -fw_cfg name=opt/com.coreos/config,file=cfg.ign

Then from my host (10.15.17.111 is the output of hostname -i):

$ ssh -p 2229 [email protected]
Fedora 30.20190801.0 (CoreOS preview)
Tracker: https://github.com/coreos/fedora-coreos-tracker
Preview release: breaking changes may occur

Last login: Wed Aug 14 21:42:45 2019

If hostfwd (or another way of forwarding a port from the host to the VM) is strictly necessary to SSH into FCOS running on QEMU, it'd be helpful to note this in the getting started docs.

As to why authenticating when SSHing into [email protected] on QEMU didn't work, I'm still investigating - it could be something else missing in my setup (or I'm missing a reason why this wouldn't work).

This was with FCOS 30.20190801.0 with qemu-system-x86-core-3.1.0-9.fc30.x86_64 on my host.

$ ssh -p 22 -vvv [email protected]
OpenSSH_8.0p1, OpenSSL 1.1.1c FIPS  28 May 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 51: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.2.15 originally 10.0.2.15
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-gex-sha1-,gss-group14-sha1-,gss-group1-sha1-]
debug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 10.0.2.15 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 51: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug2: checking match for 'final all' host 10.0.2.15 originally 10.0.2.15
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-gex-sha1-,gss-group14-sha1-,gss-group1-sha1-]
debug3: kex names ok: [curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1]
debug2: ssh_connect_direct
debug1: Connecting to 10.0.2.15 [10.0.2.15] port 22.
debug1: Connection established.
debug1: identity file /home/rfairley/.ssh/id_rsa type 0
debug1: identity file /home/rfairley/.ssh/id_rsa-cert type -1
debug1: identity file /home/rfairley/.ssh/id_dsa type -1
debug1: identity file /home/rfairley/.ssh/id_dsa-cert type -1
debug1: identity file /home/rfairley/.ssh/id_ecdsa type -1
debug1: identity file /home/rfairley/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/rfairley/.ssh/id_ed25519 type -1
debug1: identity file /home/rfairley/.ssh/id_ed25519-cert type -1
debug1: identity file /home/rfairley/.ssh/id_xmss type -1
debug1: identity file /home/rfairley/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to 10.0.2.15:22 as 'core'
debug3: hostkeys_foreach: reading file "/home/rfairley/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/rfairley/.ssh/known_hosts:28
debug3: load_hostkeys: loaded 1 keys from 10.0.2.15
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c
debug2: host key algorithms: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],[email protected],[email protected],ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc
debug2: ciphers stoc: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:8HK9Qz9AdiCmhFDCmp16yna/LsETNRARwSxCj1/JAjA
debug3: hostkeys_foreach: reading file "/home/rfairley/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/rfairley/.ssh/known_hosts:28
debug3: load_hostkeys: loaded 1 keys from 10.0.2.15
debug1: Host '10.0.2.15' is known and matches the ECDSA host key.
debug1: Found key in /home/rfairley/.ssh/known_hosts:28
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /home/rfairley/.ssh/id_rsa RSA SHA256:dwilTJGh02bpSp0U7BLPrmC4wn2wJ40lxOpmkf0fsbY agent
debug1: Will attempt key: /home/rfairley/.ssh/id_dsa 
debug1: Will attempt key: /home/rfairley/.ssh/id_ecdsa 
debug1: Will attempt key: /home/rfairley/.ssh/id_ed25519 
debug1: Will attempt key: /home/rfairley/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Server host/[email protected] not found in Kerberos database


debug3: send packet: type 50
debug2: we sent a gssapi-with-mic packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/rfairley/.ssh/id_rsa RSA SHA256:dwilTJGh02bpSp0U7BLPrmC4wn2wJ40lxOpmkf0fsbY agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/rfairley/.ssh/id_dsa
debug3: no such identity: /home/rfairley/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/rfairley/.ssh/id_ecdsa
debug3: no such identity: /home/rfairley/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/rfairley/.ssh/id_ed25519
debug3: no such identity: /home/rfairley/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /home/rfairley/.ssh/id_xmss
debug3: no such identity: /home/rfairley/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
[email protected]'s password: 
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
Permission denied, please try again.

Let's add some information to the docs addressing https://discussion.fedoraproject.org/t/are-fedora-coreos-disk-images-hybrid-bios-uefi-bootable/21911/2

• add a FAQ entry for this
• add some helpful text in https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/
• not sure if there is anything about this we can add to https://docs.fedoraproject.org/en-US/fedora-coreos/platforms/

Would be neat to document how FCOS boots, and the disk layout.

Mostly filing this so we don't forget what Andrew started on in #9. Should be able to just take that to start, update anything that's outdated, and fill in the few missing spots.

The following projects are currently using Fedora CoreOS:

• OKD: https://www.okd.io/
• typhoon: https://github.com/poseidon/typhoon
• OpenStack Magnum: https://wiki.openstack.org/wiki/Magnum

Let's create a page to showcase them with a short description and corresponding links.

For now, we can put this under the "Reference pages" section until we re-organize things.

We have some items in the works that will make settting/editing kargs easier but for now let's just get up a page that is a starting point for people to use to configure kernel arguments persistently.

I'm thinking it should be at the same place in the navigation as https://docs.fedoraproject.org/en-US/fedora-coreos/sysctl/ but for kargs.

I could not find documentation about persistence when doing updates and reboots.

This could be documented in the FAQ.

So far the information I got from IRC is that updates will preserve changes by default and that /etc persist across reboots.

EDIT:
Further details from machine-config-operator which links to the ostree adapting exists page.

Stated more generally, on an OSTree managed system, all content in /etc and /var is preserved by default across upgrades.

Confusingly, the second link only state that /var persists.

Comparing:

• https://docs.fedoraproject.org/en-US/fedora-coreos/ (English, updated)
• https://docs.fedoraproject.org/fr/fedora-coreos/ (French, not updated)

I have also filled this at pagure.io/fedora-docs/docs-fp-o but adding it here for visibility.

For example, see the in-progress French translation that is not updated from Weblate: https://translate.fedoraproject.org/languages/fr/fedora-docs-l10n-fedora-coreos/

There should be documentation on how the automatic system updates work. I can't find anything about it.
Is it a timer that is run occasionally? Does it restart the system? Does it cause any downtime?

We currently don't have any documentation on how to prepare, customize and first-boot FCOS on the vmware platform.

We can start by porting over the docs from Container Linux at https://coreos.com/os/docs/latest/booting-on-vmware.html#vmware-guestinfo-interface.

More interestingly, now there seems to be an official PowerShell based container which should even support setting guestinfo.