Ceph/MicroCeph

MicroCeph

Ubuntu 22.04 LTS MicroCeph with Snap

These installation instructions use MicroCeph - Ceph in a snap. MicroCeph is a pure upstream Ceph distribution designed for small scale and edge deployments, which can be installed and maintained with minimal knowledge and effort.

Install Ceph on Ubuntu | Ubuntu
https://ubuntu.com/ceph/install

Note: If snap is not installed:

sudo apt install snapd

Install microceph:

sudo snap install microceph

Bootstrap cluster from first node:

sudo microceph cluster bootstrap

From first node add other nodes:

sudo microceph cluster add node[x]

Copy join output onto node[x]:

sudo microceph cluster join [pasted-output-from-node1]

Check status:

sudo microceph.ceph status

Add some disks to each node (as OSDs):

sudo microceph disk add /dev/sd[x] --wipe

Config files are found at:

/var/snap/microceph/current/conf/
  /var/snap/microceph/current/conf/ceph.conf
  /var/snap/microceph/current/conf/metadata.yaml
  /var/snap/microceph/current/conf/ceph.keyring

Create links:

mkdir -p /etc/ceph  # not needed if you 'apt install ceph-common'
ln -s /var/snap/microceph/current/conf/ceph.conf /etc/ceph/ceph.conf
ln -s /var/snap/microceph/current/conf/ceph.keyring /etc/ceph/ceph.keyring

Verify cluster:

sudo microceph.ceph status
sudo microceph.ceph osd status

microceph disk list

Create File System

Create data and metadata pool:

ceph osd pool create cephfs_data
ceph osd pool create cephfs_metadata

Create filesystem:

# ceph fs new <fs_name> <metadata_ppool> <data_pool>
ceph fs new cephfs cephfs_metadata cephfs_data

View filesystems:

ceph fs ls

Stat the MDs:

ceph mds stat

Reference:

Create a Ceph file system — Ceph Documentation
https://docs.ceph.com/en/quincy/cephfs/createfs/

Local Mount

If you are on one of the cluster nodes, and if you look at the ceph.keyring, you will notice the default user was likely 'admin' (client.admin).

Install mount.ceph wrapper:

apt install ceph-common

Assuming /etc/ceph/ceph.keyring exits:

mount -t ceph :/ /mnt -o name=admin

New style, with the mount.ceph helper, assuming the fs was created as 'mycfs':

mount.ceph admin@.mycfs=/ /mnt

# Or:
mount -t ceph admin@.cfs=/ /mnt

/etc/fstab:

admin@.mycfs=/    /mycfs    ceph    defaults,noauto    0 0

Specify it all:

mount -t ceph {device-string}={path-to-mounted} {mount-point} -o {key-value-args} {other-args}

mount -t ceph <name>@<fsid>.<fs_name>=/ /mnt

mount -t ceph cephuser@.cephfs=/ /mnt -o secretfile=/etc/ceph/cephuser.secret

mount -t ceph cephuser@.cephfs=/ /mnt -o secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==

Reference:

Mount CephFS using Kernel Driver — Ceph Documentation
https://docs.ceph.com/en/latest/cephfs/mount-using-kernel-driver/

Documentation

Charmed Ceph Documentation | Ubuntu
https://ubuntu.com/ceph/docs

Remove MicroCeph

If you decide you want to remove everything cleanly:

sudo snap remove microceph