Build FCOS based on Fedora 43 using `podman build`

[jlebon]

Follow-up to #1861. Let's aim towards having FCOS built using the new container-native flow as part of the f43 rebase. Which means sprinting towards turning this on in the rawhide stream first.

Rawhide requirements:

• Add cosa import coreos-assembler#4164
• Make cosa build support building "ostree" artifact using podman/buildah coreos-assembler#4242
• config-bot: don't clobber build-args.conf fedora-coreos-releng-automation#214
• build-rootfs: inject our repo definitions into build stage fedora-coreos-config#3612
• [rawhide] build-args.conf: update for rawhide fedora-coreos-config#3611
• tests: bump kernel-replace to 20G fedora-coreos-config#3666
• cmd-build-with-buildah: retain overrides/ API functionality coreos-assembler#4256
• build-rootfs: add overrides/{rpm,rootfs} API fedora-coreos-config#3664
• testiso: stop checking the ostree commit in the aleph file coreos-assembler#4268
• coreos-bootc-minimal-plus: stop patching prepare-root.conf fedora-coreos-config#3501
• cmd-build-with-buildah: add fedora-coreos.stream as annotation too coreos-assembler#4271
• kola: Add stream from oci image labels to meta.json and use in kola. coreos-assembler#4274
• cmd-build-with-buildah: strip setgid bit from source coreos-assembler#4277
• config.yaml: turn on COSA_BUILD_WITH_BUILDAH for rawhide fedora-coreos-pipeline#1194

Production requirements:

• Pin on digests and autobump them

[jlebon]

Pin on digests, matching fedora-bootc git submodule

Was chatting with @jbtrystram and @travier about this. We don't have https://gitlab.com/fedora/bootc/tracker/-/issues/34 currently for the production fedora-bootc repo (since it's still hooked to Pungi), but we do have them in the Konflux output. See e.g.:

• https://quay.io/repository/bootc-devel/fedora-bootc-42-standard?tab=tags
• https://quay.io/repository/bootc-devel/fedora-bootc-rawhide-standard?tab=tags

So we could use those in the meantime.

---

One tricky thing though is that not the same git commits are built for e.g. fedora-bootc 42 and 43 because their Konflux pipelines are triggered on different files. Notably, the f42 one will rebuild when the f42 lockfiles are updated, and similarly for f43.

So there isn't one base-images git commit that exactly corresponds to digests we could pin to for f42 and f43. But that's OK I think. As mentioned in #1861, we don't want "no-op" bumps that don't actually change manifests anyway. So we should do a git diff and filter out the lockfiles when determining if something actually changed.

So then, one way to do this is (this would be in the current git submodule bump GHA):

• For the current $releasever of testing-devel, inspect the :latest tag for the fedora-bootc-$releasever-standard image.
• Get the git commit for that image from metadata.
• Look over the fedora-bootc git diff and omit the lockfiles. If nothing changed, we're done. Otherwise, bump the fedora-bootc git submodule to that git commit.
• Inspect the :latest tag for fedora-bootc-rawhide-standard (and also e.g. fedora-bootc-$releasever+1-standard if needed).
• Get the git commit for that image from metadata.
• Look over the fedora-bootc git diff and omit the lockfiles. Verify that the diff is the same. If not, print a message and stop here.
• Open PR against testing-devel to bump the git submodule and pinned digests.

This would require keeping the digests for e.g. rawhide in testing-devel so they can be updated in sync. One way to do this is to have build-args.conf actually be a symlink to e.g. build-args-f42.conf in testing-devel, but point to build-args-f43.conf in the rawhide branch.

[jlebon]

One thing that came up is signing. We are moving to non-encapsulated container images here so the current cosa sign flow doesn't make sense. Once we build the container image with Konflux, we should be getting signing for free.

See e.g.:

$ cosign tree quay.io/bootc-devel/fedora-bootc-42-standard
📦 Supply Chain Security Related artifacts for an image: quay.io/bootc-devel/fedora-bootc-42-standard
└── 💾 Attestations for an image tag: quay.io/bootc-devel/fedora-bootc-42-standard:sha256-588ad22845b6e45ccf68ba8290f0f51c7a01cf9bfdbbedcaf74d8e89be8664a6.att
   └── 🍒 sha256:1e89af88e91d741ff183588295e5b03377b80e9393954dc7db8c50bc683fe8ec
└── 🔐 Signatures for an image tag: quay.io/bootc-devel/fedora-bootc-42-standard:sha256-588ad22845b6e45ccf68ba8290f0f51c7a01cf9bfdbbedcaf74d8e89be8664a6.sig
   └── 🍒 sha256:0094daf968b2b2b7e32e3b0d005573c4288f517fed52fb6d0b193f832cf38892
└── 📦 SBOMs for an image tag: quay.io/bootc-devel/fedora-bootc-42-standard:sha256-588ad22845b6e45ccf68ba8290f0f51c7a01cf9bfdbbedcaf74d8e89be8664a6.sbom
   └── 🍒 sha256:44a5a4468dc72512d861fa3218c996696a18255b8bc2f3cf2dd554a000fdd7fd

So then we'll want to have a policy on the host to do signature checking (see also https://gitlab.com/fedora/bootc/tracker/-/issues/73).

I don't think we should block moving rawhide to the container-native path over this, but certainly we should have all this in place before it hits production streams.

[dustymabe]

I don't think we should block moving rawhide to the container-native path over this

I agree.

, but certainly we should have all this in place before it hits production streams.

Agree here too, but that also means the deadline is coming up quick because we move next over to F43 at F43 beta time (mid september).

[jbtrystram]

I think we can add #1986 in the checklist here

[jbtrystram]

Filed #1990 as a tracking issue to make jenkins build boot image deriving an OCI image built by konflux