2

Loss of network in Proxmox VE 7 upon reboot

 2 years ago
source link: https://dustri.org/b/loss-of-network-in-proxmox-ve-7-upon-reboot.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Loss of network in Proxmox VE 7 upon reboot

This morning I woke up to my hypervisor being down. Because it's automatically rebooting upon kernel upgrades, maybe it was crashing at boot time, the unblinking LED on the network card was also pointing in this direction. So I plugged a screen in, and investigated: the machine was up, and I got a shell.

root@pve:~# ip a | grep DOWN
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state DOWN group default qlen 1000
3: iw0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

# journalctl -e
Feb 23 13:30:25 systemd[1]: systemd-fsckd.service: Succeeded.
Feb 23 13:31:55 systemd[1]: ifupdown2-pre.service: Main process exited, code=exited, status=1/FAILURE
Feb 23 13:31:55 systemd[1]: ifupdown2-pre.service: Failed with result 'exit-code'.
Feb 23 13:31:55 systemd[1]: Failed to start Helper to synchronize boot up for ifupdown.
Feb 23 13:31:55 systemd[1]: Dependency failed for Network initialization.
Feb 23 13:31:55 systemd[1]: networking.service: Job networking.service/start failed with result 'dependency'.
Feb 23 13:31:55 systemd[1]: Reached target Network.
Feb 23 13:31:55 systemd[1]: Reached target Network is Online.

Looks like something is wrong in the ifupdown2-pre service, which is a dependency of networking, timeout'ing after ~2 minutes. Let's take a look to see what ifupdown2-pre is supposed to do:

# cat /usr/lib/systemd/system/ifupdown2-pre.service 
[Unit]
Description=Helper to synchronize boot up for ifupdown
DefaultDependencies=no
Wants=systemd-udevd.service
After=systemd-udev-trigger.service
Before=network.target

[Service]
Type=oneshot
TimeoutSec=180
RemainAfterExit=yes
EnvironmentFile=-/etc/default/networking
ExecStart=/bin/udevadm settle

What is udevadm settle?

# apropos udevadm 
udevadm (8)          - udev management tool
# udevadm --help | grep settle                                                                                                  
  settle        Wait for pending udev events

So I guess it's waiting on some events in a blocking manner, reaching the timeout, and preventing the networking service from correctly being started. Since my hypervisor doesn't have any fancy hardware relying on udev events shenanigans, I simply disabled masked the service and rebooted:

# systemctl mask ifupdown2-pre.service 
# reboot

And things went back to normal. This is apparently a more-or-less known bug since 2020, sigh.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK