Skip to content

[rawhide][aarch64] coreos.unique.boot.failure times out intermittently #2019

@aaradhak

Description

@aaradhak

The kola test coreos.unique.boot.failure is intermittently failing on the aarch64 arch in rawhide builds.

07:08:33  === RUN   coreos.unique.boot.failure
07:08:33  mke2fs 1.47.2 (1-Jan-2025)
07:12:39  --- FAIL: coreos.unique.boot.failure (244.60s)
07:12:39          qemufailure.go:69: timed out waiting for initramfs error: context deadline exceeded
07:12:39  FAIL, output in /home/jenkins/agent/workspace/build-arch/tmp/kola-tIcaN/kola/rerun
07:12:39  Error: harness: test suite failed
07:12:39  2025-08-29T14:12:37Z cli: harness: test suite failed
07:12:39  failed to execute cmd-kola: exit status 1

The test is designed to verify that the system correctly enters an emergency shell when two filesystems are labeled "boot". While the system does enter the emergency state as expected, the test framework times out because it fails to detect the corresponding error message.

From the logs, it seems to be like the test intentionally creates a disk setup with two filesystems labeled "boot".

[ 3.279074] rdcore[593]: Error: System has 2 devices with a filesystem labeled 'boot': ["/dev/vdb3", "/dev/vda"]

[    3.328633] systemd[1]: ignition-fetch-offline.service: Main process exited, code=killed, status=15/TERM^M
[    3.329975] systemd[1]: ignition-fetch-offline.service: Failed with result 'signal'.^M
[^[[0;32m  OK  ^[[0m] Stopped ^[[0;1;39mignition-fetch-offline.service^[[0m - Ignition (fetch-offline).^M[    3.333006] systemd[1]: Stopped ignition-fetch-offline.service - Ignition (fetch-offline).
^M  
[    3.335002] systemd[1]: ignition-fetch-offline.service: Triggering OnFailure= dependencies.^M

The console log from the failed run shows that the journal dump service could not find the necessary virtio device.

[^[[0;32m OK ^[[0m] Stopped ^[[0;1;39mdracut-cmdline.service^[[0m - dracut cmdline hook.^M[ 3.435360] ignition-virtio-dump-journal[616]: Didn't find virtio port /dev/virtio-ports/com.coreos.ignition.journal

Coming from this

### Actual Behavior

The system correctly detects the duplicate boot devices and enters the emergency shell. However, the ignition-virtio-dump-journal.service fails to find the virtio port.

This results in the Kola test harness never receiving the journal contents so it cannot find the expected error message. The test ultimately fails with a timeout.

### Findings so far:
The failure appears to be a race condition between udev creating the device symlink and the ignition-virtio-dump-journal.service running.

The system correctly identifies the duplicate boot labels and enters an emergency shell. This part of the test is working.

The test framework times out waiting for the expected error message: timed out waiting for initramfs error: context deadline exceeded.

Metadata

Metadata

Assignees

No one assigned

    Labels

    pipeline failureThis issue or pull request is derived from CI failures

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions