ceph集群報 Monitor clock skew detected 錯誤問題排查,解決
????????? ?
告警信息如下:
[root@ceph-100-80 ceph]# ceph -w
??? cluster ddc1b10b-6d1a-4ef9-8a01-d561512f3c1d
???? health HEALTH_WARN
??????????? clock skew detected on mon.ceph-100-81, mon.ceph-100-82
??????????? Monitor clock skew detected
???? monmap e1: 3 mons at {ceph-100-80=172.16.100.80:6789/0,ceph-100-81=172.16.100.81:6789/0,ceph-100-82=172.16.100.82:6789/0}
??????????? election epoch 22, quorum 0,1,2 ceph-100-80,ceph-100-81,ceph-100-82
???? mdsmap e21: 1/1/1 up {0=ceph-100-80=up:active}, 2 up:standby
???? osdmap e116: 6 osds: 6 up, 6 in
????? pgmap v205: 576 pgs, 3 pools, 1962 bytes data, 20 objects
??????????? 234 MB used, 269 GB / 269 GB avail
???????????????? 576 active+clean
? ?
1:添加配置參數:???????????? ?
vim /etc/ceph/ceph.conf
?
[mon.ceph-100-80]
host = ceph-100-80
mon_data = /var/lib/ceph/mon/ceph-ceph-100-80/
mon_addr = 172.16.100.80:6789
# 添加內容如下:
mon clock drift allowed = 2
mon clock drift warn backoff = 30?? ?
2:同步配置文件
ceph-deploy --overwrite-conf admin ceph-100-{80..82}
3:重啟mon 服務
/etc/init.d/ceph restart mon
4:驗證:
[root@ceph-100-80 ceph]# ceph -w?????? ?
??? cluster ddc1b10b-6d1a-4ef9-8a01-d561512f3c1d
???? health HEALTH_OK
???? monmap e1: 3 mons at {ceph-100-80=172.16.100.80:6789/0,ceph-100-81=172.16.100.81:6789/0,ceph-100-82=172.16.100.82:6789/0}
??????????? election epoch 24, quorum 0,1,2 ceph-100-80,ceph-100-81,ceph-100-82
???? mdsmap e21: 1/1/1 up {0=ceph-100-80=up:active}, 2 up:standby
???? osdmap e116: 6 osds: 6 up, 6 in
????? pgmap v205: 576 pgs, 3 pools, 1962 bytes data, 20 objects
??????????? 234 MB used, 269 GB / 269 GB avail
???????????????? 576 active+clean
??????????????? ?
??????????????? ?
再次查看,告警內容消失。
問題總結:
本問題主要是mon節點服務器,時間偏差比較大導致,本次遇到問題為測試環境,通過修改ceph對時間偏差閥值,規避的告警信息,線上業務環境,注意排查服務器時間同步問題。 ???????????
轉載于:https://blog.51cto.com/michaelkang/1729186