Ceph cluster

From wiki.techunit.org
Jump to: navigation, search


  • Aim is to setup a simple ceph cluster
  • Realized on CentOS 7
  • Ceph version : 9.2.1 (AKA Infernalis)

Test environment

  • 5 virtual machines
    • ceph-adm - 1 virtual HDD :
    • ceph01 - 2 virtual HDD :
    • ceph02 - 2 virtual HDD :
    • ceph03 - 2 virtual HDD :
    • ceph-client

Cluster Configuration

add hosts to /etc/hosts (optional)

  • Edit /etc/hosts on servers ceph-adm ceph01 ceph02 ceph03

Avoid errors

  • On all servers
  • Disable selinux or set to permissive
  • Modify default sudoers config, edit /etc/sudoers and comment this line
Defaults    requiretty

Create a ceph user

  • On all servers (except the client)
# Here user is ceph
useradd -d /home/ceph -m ceph
passwd ceph
echo "ceph ALL = (root) NOPASSWD:ALL" | tee /etc/sudoers.d/ceph
chmod 0440 /etc/sudoers.d/ceph

Create a ssh key

  • On ceph-adm
su - ceph
ssh-copy-id -i ~/.ssh/id_rsa.pub ceph@ceph01
ssh-copy-id -i ~/.ssh/id_rsa.pub ceph@ceph02
ssh-copy-id -i ~/.ssh/id_rsa.pub ceph@ceph03

Install repository

  • On ceph-adm
  • Edit /etc/yum.repo.d/ceph-deploy.repo
name=Ceph noarch packages
  • Then install ceph-deploy
# Import the key
rpm --import 'https://download.ceph.com/keys/release.asc'
yum -y install ceph-deploy

Create the cluster

  • On ceph-adm
su - ceph
ceph-deploy install ceph-adm ceph01 ceph02 ceph03
ceph-deploy new ceph01 ceph02 ceph03
ceph-deploy mon create-initial
ceph-deploy mon create ceph01
ceph-deploy calamari connect ceph01 ceph02 ceph03
ceph-deploy admin ceph-adm
ceph-deploy disk zap ceph01:/dev/sdb ceph02:/dev/sdb ceph03:/dev/sdb
ceph-deploy osd prepare ceph01:sdb:ssdb ceph02:sdb:ssdb ceph03:sdb:ssdb
ceph-deploy osd activate ceph01:sdb1:ssdb ceph02:sdb1:ssdb ceph03:sdb1:ssdb
(as root) chown ceph: /etc/ceph/ceph.client.admin.keyring
ceph-deploy mds create ceph01
ceph health (-> HEALTH_OK)

Useful commands

  • On ceph-adm
  • If you need to restart your configuration
su - ceph
# remove packages
ceph-deploy purge ceph01 ceph02 ceph03
# Remove configuration
ceph-deploy purgedata ceph01 ceph02 ceph03
# Delete keys
ceph-deploy forgetkeys