Proxmox/iomemory-vsl

dkms

dkms status

dkms autoinstall -k 6.5.11-7-pve

Ceph

OSD fails to start on reboot.

journalctl -u ceph-osd@2.service

auth: unable to find a keyring on /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory
AuthRegistry(0x55821f874a40) no keyring found at /var/lib/ceph/osd/ceph-2/keyring, disabling cephx

Solution:

# Export key
ceph auth export osd.2 -o keyring.export
# Adapt user and permissions
chown ceph:ceph keyring.export
chmod 0600 keyring.export
# Remove caps lines
sed -i '/^\s*caps/d' keyring.export
# Put keyring into expected location
mv keyring.export /var/lib/ceph/osd/ceph-2/keyring
# Reset failed counter for service
systemctl reset-failed ceph-osd@2.service
# Start osd service
systemctl start ceph-osd@2.service

echo "bluestore" > /var/lib/ceph/osd/ceph-2/type

udev auto mount LVM rule

Also create this udev rule to auto mount your LVM after reboots, or your OSDs won't be happy after reboots.

/etc/udev/rules.d/60-persistent-storage-fio.rules

cat > /etc/udev/rules.d/60-persistent-storage-fio.rules << "EOF"
# persistent storage links: /dev/disk/{by-id,by-uuid,by-label,by-path}
# scheme based on "Linux persistent device names", 2004, Hannes Reinecke <hare@suse.de>

ACTION=="remove", GOTO="persistent_storage_end"
ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}=="1", GOTO="persistent_storage_end"

SUBSYSTEM!="block", GOTO="persistent_storage_end"
KERNEL!="fio*", GOTO="persistent_storage_end"

# ignore partitions that span the entire disk
TEST=="whole_disk", GOTO="persistent_storage_end"

# For partitions import parent disk ID_* information, except ID_FS_*.
#
# This is particularly important on media where a filesystem superblock and
# partition table are found on the same level, e.g. common Linux distro ISO
# installation media.
#
# In the case where a partition device points to the same filesystem that
# was detected on the parent disk, the ID_FS_* information is already
# present on the partition devices as well as the parent, so no need to
# propagate it. In the case where the partition device points to a different
# filesystem, merging the parent ID_FS_ properties would lead to
# inconsistencies, so we avoid doing so.
ENV{DEVTYPE}=="partition", \
  IMPORT{parent}="ID_[!F]*", IMPORT{parent}="ID_", \
  IMPORT{parent}="ID_F[!S]*", IMPORT{parent}="ID_F", \
  IMPORT{parent}="ID_FS[!_]*", IMPORT{parent}="ID_FS"

# probe filesystem metadata of disks
IMPORT{builtin}="blkid"

# by-label/by-uuid links (filesystem metadata)
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_LABEL_ENC}=="?*", SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"

# by-partlabel/by-partuuid links (partition metadata)
ENV{ID_PART_ENTRY_UUID}=="?*", SYMLINK+="disk/by-partuuid/$env{ID_PART_ENTRY_UUID}"
ENV{ID_PART_ENTRY_SCHEME}=="gpt", ENV{ID_PART_ENTRY_NAME}=="?*", SYMLINK+="disk/by-partlabel/$env{ID_PART_ENTRY_NAME}"

LABEL="persistent_storage_end"
EOF

The content is gone after the reboot · Issue #55 · RemixVSL/iomemory-vsl · GitHub
https://github.com/RemixVSL/iomemory-vsl/issues/55