Reinstalling the operating system on a VM fails

Reinstalling the operating system may fail due to a failure in the nftables subsystem. If the cluster node hosts a large number of virtual machines (VMs), nftables will limit the maximum number of open files. To fix the problem, you need to increase this limit.

Diagnostics

1. Connect to the cluster node via SSH.

2. Run the command:

   systemctl status nftables -l

   If the problem is related to nftables, the output of the command will contain errors like Could not open file "/etc/nftables/vm/<id_name of VM>.nft": Too many open files.

Solution

1. Connect to the cluster node via SSH.

2. Add the line LimitNOFILE=10240 to the Service section of the nftables config file:

   systemctl edit nftables.service

   Example of a config file

   [Unit]
   Description=Netfilter Tables
   Documentation=man:nft(8)
   Wants=network-pre.target
   Before=network-pre.target
   
   [Service]
   Type=oneshot
   ProtectSystem=full
   ProtectHome=true
   ExecStart=/sbin/nft -f /etc/sysconfig/nftables.conf
   ExecReload=/sbin/nft 'flush ruleset; include "/etc/sysconfig/nftables.conf";'
   ExecStop=/sbin/nft flush ruleset
   RemainAfterExit=yes
   LimitNOFILE=10240
   
   [Install]
   WantedBy=multi-user.target

3. Restart nftables:

   systemctl daemon-reload

   systemctl restart nftables