Ubuntu-22.04安裝MegaCLI并查看MegaRAID的配置

提示：文章寫完后，目錄可以自動生成，如何生成可參考右邊的幫助文檔

文章目錄

前言
一、MegaRAID
二、安裝MegaCLI
- 1.Ubuntu-18.04
- 2.Ubuntu-22.04
- 3.Ubuntu-20.04
- 4.CentOS-7
- 5.其它發行版
三、使用MegaCLI
- 1.查看所有適配器信息
- 2.查看所有物理磁盤信息
四、方便的腳本
五、配置方法
總結

前言

經常使用服務器的都知道RAID的重要性，現代的服務器基本都是基于RAID部署的。舉個例子，基于RAID 1部署的磁盤陣列互為鏡像，只要不是兩塊盤同時壞，數據一定不會丟，只需要替換掉故障的盤，什么都不用做，RAID會自動重建的。還有RAID 0、RAID 5、RAID 10等這里就不贅述了，不是本篇討論的重點，今天我們只討論怎么查看服務器的RAID配置。

本篇基于MegaRAID，不涉及其它RAID。

一、MegaRAID

MegaRAID是一種常用于服務器和高端存儲設備的RAID（冗余陣列）控制器技術。通過MegaRAID，用戶可以配置和管理硬盤驅動器（HDD）或固態硬盤（SSD）的RAID陣列，以提供數據冗余、提高性能和增加存儲容量。

MegaRAID是LSI公司的產品，LSI后來被Avago公司收購。
LSI公司

我們面臨的問題是在Linux系統里面沒有辦法直接看到硬RAID信息（通過BIOS創建），只能看到軟RAID信息（通過軟件創建）。所以各家廠商提供了專門的工具幫助開發者或運維者查看和管理RAID。我們講的MegaRAID就是這樣的，我所使用的RAID就是在BIOS里面設置的。

二、安裝MegaCLI

MegaRAID提供了一個工具管理RAID叫MegaCLI，這個工具在博通網站就能下載到，支持多種Linux發行版（Ubuntu、CentOS、RedHat…），今天主要講Ubuntu上的MegaCLI安裝。

注意：Ubuntu-18.04和以上版本安裝方式不一樣，后面會細講！

1.Ubuntu-18.04

這個版本比較容易，使用APT就可以安裝了。以下方法按照順序執行，我經過多方認證確定不會出現問題，如果發生沖突大概率是你的環境有問題。

#/etc/apt/sources.list末尾添加源
sudo sed -i '$a\deb http://hwraid.le-vert.net/ubuntu precise main' /etc/apt/sources.list
#添加驗證密鑰
wget -O - http://hwraid.le-vert.net/debian/hwraid.le-vert.net.gpg.key | sudo apt-key add -
#更新源
sudo apt update
#安裝
sudo apt-get install megacli megactl megaraid-status

2.Ubuntu-22.04

這個版本不能用上面的方式安裝，因為megacli依賴Python2，而22.04已經移除了Python2，強行安裝或許可以，但不是最好的方法。

這里我自己上傳了個MegaRAID.tar，這是我從CentOS7中提取的可執行文件+庫，因為官方只提供了.rpm的包，沒有提供.deb的包，所以我用這種方法移植到Ubuntu了。
資源鏈接

下載完成之后是MegaRAID.tar，執行下面的命令：

解壓
sudo tar -xmf MegaRAID.tar移動
sudo mv MegaRAID /opt創建軟鏈
sudo ln -s /opt/MegaRAID/MegaCli/MegaCli64 /usr/sbin/megacli缺少依賴庫就安裝，不缺少就跳過
sudo apt install libncurses5

這個時候就可以開始使用了。

3.Ubuntu-20.04

放在22.04后面是因為我手上沒有20.04的版本，APT方式安裝的megacli依賴Python2，我也忘了20.04是不是閹割了Python2，使用which python2來測試是不是預裝了python2.7，如果有按照18.04的方式安裝，沒有的話就按照22.04的方式安裝。

4.CentOS-7

實測可以按照22.04的方式安裝。

5.其它發行版

我常用的就這些發行版，需要其它版本請下載下面的多版本試試。
MegaCLI

三、使用MegaCLI

確保已經安裝完成了。

注意：需要sudo權限或者切換root用戶！

查看機器型號    
dmidecode | grep "Product" 
查看廠商     
dmidecode| grep  "Manufacturer" 
查看序列號     
dmidecode | grep  "Serial Number" 
查看CPU信息     
dmidecode | grep  "CPU" 
查看CPU個數     
dmidecode | grep  "Socket Designation: CPU" |wc –l 
查看出廠日期     
dmidecode | grep "Date" 
查看充電狀態     
megacli -AdpBbuCmd -GetBbuStatus -aALL |grep "Charger Status" 
顯示BBU狀態信息     
megacli -AdpBbuCmd -GetBbuStatus –aALL 
顯示BBU容量信息     
megacli -AdpBbuCmd -GetBbuCapacityInfo –aALL 
顯示BBU設計參數     
megacli -AdpBbuCmd -GetBbuDesignInfo –aALL 
顯示當前BBU屬性     
megacli -AdpBbuCmd -GetBbuProperties –aALL 
查看充電進度百分比     
megacli -AdpBbuCmd -GetBbuStatus -aALL |grep "Relative State of Charge" 
查詢Raid陣列數     
megacli -cfgdsply -aALL |grep "Number of DISK GROUPS:" 
顯示Raid卡型號，Raid設置，Disk相關信息       
megacli -cfgdsply –aALL 
顯示所有物理信息     
megacli -PDList -aALL 
顯示所有邏輯磁盤組信息     
megacli -LDInfo -LALL –aAll 
查看物理磁盤重建進度(重要)     
megacli -PDRbld -ShowProg -PhysDrv [1:5] -a0 
查看適配器個數    
megacli –adpCount 
查看適配器時間    
megacli -AdpGetTime –aALL 
顯示所有適配器信息    
megacli -AdpAllInfo –aAll 
查看Cache 策略設置     
megacli -cfgdsply -aALL |grep Polic

簡單說下常用的兩個命令，第一個是查看RAID配置，第二個是查看物理磁盤。

1.查看所有適配器信息

沒錯，MegaRAID稱所有的RAID組合為適配器（Adapter），每個適配器都有一個唯一ID。

sudo megacli -LDInfo -LALL –aAllAdapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 893.75 GB
Sector Size         : 512
Is VD emulated      : Yes
Mirror Data         : 893.75 GB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 2
Span Depth          : 1
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Default Power Savings Policy: Controller Defined
Current Power Savings Policy: None
Can spin up in 1 minute: No
LD has drives that support T10 power conditions: No
LD's IO profile supports MAX power savings with cached writes: No
Bad Blocks Exist: No
Is VD Cached: No

RAID Level：就是配置的RAID方式，我這里是RAID 1

RAID Level對應關系：
RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 #RAID 1
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0 #RAID 0
RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3 #RAID 5

Size：邏輯磁盤容量，就是組成RAID之后你在系統里看到的可用容量

Sector Size：鏃大小，不細講了，你可以自行研究下。

Number Of Drives：多少個物理磁盤參與了這個RAID，這里顯示是2個。

注：關于RAID 1支持的硬盤數，我查了好久，可以確認的是Intel的RAID是僅限2個磁盤，Wiki上說至少2塊，我見過的都是2塊，大于2塊的我沒見過。因為2塊其實已經很安全了，2塊磁盤同時壞的可能性很低。所以似乎也沒多少必要用3塊乃至更多的磁盤，畢竟成本在那里，服務器磁盤可不便宜！

2.查看所有物理磁盤信息

這才是我真正想看到的，從Linux系統層面無法看到插了幾塊硬盤，每塊硬盤都插在了哪個槽位，每塊硬盤屬于哪個RAID，這都是很重要的信息。雖然從BIOS可以看到，但是我總沒有必要去重啟服務器吧，有些服務器可是24小時運行的呢！

sudo megacli -PDList -aALL Adapter #0Enclosure Device ID: 32
Slot Number: 0
Drive's position: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: 1
Device Id: 0
WWN: **************
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATARaw Size: 894.252 GB [0x6fc81ab0 Sectors]
Non Coerced Size: 893.752 GB [0x6fb81ab0 Sectors]
Coerced Size: 893.75 GB [0x6fb80000 Sectors]
Sector Size:  512
Logical Sector Size:  512
Physical Sector Size:  4096
Firmware state: Online, Spun Up
Device Firmware Level: 0100
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221104000000
Connected Port Number: 1(path0)
Inquiry Data: PHYF8454006A960CGN  INTEL SSDSC2KB960G8                     XCV10100
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Solid State Device
Drive:  Not Certified
Drive Temperature :18C (64.40 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Drive's NCQ setting : N/A
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : NoEnclosure Device ID: 32
Slot Number: 1
Drive's position: DiskGroup: 0, Span: 0, Arm: 1
Enclosure position: 1
Device Id: 1
WWN: *************
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATARaw Size: 894.252 GB [0x6fc81ab0 Sectors]
Non Coerced Size: 893.752 GB [0x6fb81ab0 Sectors]
Coerced Size: 893.75 GB [0x6fb80000 Sectors]
Sector Size:  512
Logical Sector Size:  512
Physical Sector Size:  4096
Firmware state: Online, Spun Up
Device Firmware Level: 0100
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x4433221100000000
Connected Port Number: 0(path0)
Inquiry Data: PHYF845406QH960CGN  INTEL SSDSC2KB960G8                     XCV10100
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Solid State Device
Drive:  Not Certified
Drive Temperature :19C (66.20 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Drive's NCQ setting : N/A
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : No

Slot Number：槽位號

PD Type：物理接口，SATA就是SATA控制器

Device Speed：設備速度規格

Link Speed：連接速度規格，這個應該是如果SATA3的設備接SATA2的接口它倆可能不一樣。

Media Type：磁盤物理材質，我只見過固態和機械，其它的請查詢官方文檔

Drive Temperature：硬盤溫度，攝氏度/華氏度，一般50℃算偏高了，正常40℃左右差不多，太高了就檢查下機器散熱。

四、方便的腳本

我從網上找了一個方便的腳本，不用每次都輸入那么多指令了。使用方式也很簡單，復制保存為lsi.sh，你輸入./lsi.sh會給出指示。

注意：這個腳本更適合運維場景，方便快速排查，畢竟這些命令記起來還挺頭疼的，復雜用法還得參考官方文檔。

#!/bin/bash
#
# Calomel.org
#     https://calomel.org/megacli_lsi_commands.html
#     LSI MegaRaid CLI
#     lsi.sh @ Version 0.05
#
# description: MegaCLI script to configure and monitor LSI raid cards.# Full path to the MegaRaid CLI binary
#MegaCli="/usr/local/sbin/MegaCli64"
MegaCli=`which megacli`# The identifying number of the enclosure. Default for our systems is "8". Use
# "MegaCli64 -PDlist -a0 | grep "Enclosure Device"" to see what your number
# is and set this variable.
ENCLOSURE="8"if [ $# -eq 0 ]thenecho ""echo "            OBPG  .:.  lsi.sh $arg1 $arg2"echo "-----------------------------------------------------"echo "status        = Status of Virtual drives (volumes)"echo "drives        = Status of hard drives"echo "ident \$slot   = Blink light on drive (need slot number)"echo "good \$slot    = Simply makes the slot \"Unconfigured(good)\" (need slot number)"echo "replace \$slot = Replace \"Unconfigured(bad)\" drive (need slot number)"echo "progress      = Status of drive rebuild"echo "errors        = Show drive errors which are non-zero"echo "bat           = Battery health and capacity"echo "batrelearn    = Force BBU re-learn cycle"echo "logs          = Print card logs"echo "checkNemail   = Check volume(s) and send email on raid errors"echo "allinfo       = Print out all settings and information about the card"echo "settime       = Set the raid card's time to the current system time"echo "setdefaults   = Set preferred default settings for new raid setup"echo ""exitfi# General status of all RAID virtual disks or volumes and if PATROL disk check
# is running.
if [ $1 = "status" ]then$MegaCli -LDInfo -Lall -aALL -NoLogecho "###############################################"$MegaCli -AdpPR -Info -aALL -NoLogecho "###############################################"$MegaCli -LDCC -ShowProg -LALL -aALL -NoLogexit
fi# Shows the state of all drives and if they are online, unconfigured or missing.
if [ $1 = "drives" ]then$MegaCli -PDlist -aALL -NoLog | egrep 'Slot|state' | awk '/Slot/{if (x)print x;x="";}{x=(!x)?$0:x" -"$0;}END{print x;}' | sed 's/Firmware state://g'exit
fi# Use to blink the light on the slot in question. Hit enter again to turn the blinking light off.
if [ $1 = "ident" ]then$MegaCli  -PdLocate -start -physdrv[$ENCLOSURE:$2] -a0 -NoLoglogger "`hostname` - identifying enclosure $ENCLOSURE, drive $2 "read -p "Press [Enter] key to turn off light..."$MegaCli  -PdLocate -stop -physdrv[$ENCLOSURE:$2] -a0 -NoLogexit
fi# When a new drive is inserted it might have old RAID headers on it. This
# method simply removes old RAID configs from the drive in the slot and make
# the drive "good." Basically, Unconfigured(bad) to Unconfigured(good). We use
# this method on our FreeBSD ZFS machines before the drive is added back into
# the zfs pool.
if [ $1 = "good" ]then# set Unconfigured(bad) to Unconfigured(good)$MegaCli -PDMakeGood -PhysDrv[$ENCLOSURE:$2] -a0 -NoLog# clear 'Foreign' flag or invalid raid header on replacement drive$MegaCli -CfgForeign -Clear -aALL -NoLogexit
fi# Use to diagnose bad drives. When no errors are shown only the slot numbers
# will print out. If a drive(s) has an error you will see the number of errors
# under the slot number. At this point you can decided to replace the flaky
# drive. Bad drives might not fail right away and will slow down your raid with
# read/write retries or corrupt data.
if [ $1 = "errors" ]thenecho "Slot Number: 0"; $MegaCli -PDlist -aALL -NoLog | egrep -i 'error|fail|slot' | egrep -v ' 0'exit
fi# status of the battery and the amount of charge. Without a working Battery
# Backup Unit (BBU) most of the LSI read/write caching will be disabled
# automatically. You want caching for speed so make sure the battery is ok.
if [ $1 = "bat" ]then$MegaCli -AdpBbuCmd -aAll -NoLogexit
fi# Force a Battery Backup Unit (BBU) re-learn cycle. This will discharge the
# lithium BBU unit and recharge it. This check might take a few hours and you
# will want to always run this in off hours. LSI suggests a battery relearn
# monthly or so. We actually run it every three(3) months by way of a cron job.
# Understand if your "Current Cache Policy" is set to "No Write Cache if Bad
# BBU" then write-cache will be disabled during this check. This means writes
# to the raid will be VERY slow at about 1/10th normal speed. NOTE: if the
# battery is new (new bats should charge for a few hours before they register)
# or if the BBU comes up and says it has no charge try powering off the machine
# and restart it. This will force the LSI card to re-evaluate the BBU. Silly
# but it works.
if [ $1 = "batrelearn" ]then$MegaCli -AdpBbuCmd -BbuLearn -aALL -NoLogexit
fi# Use to replace a drive. You need the slot number and may want to use the
# "drives" method to show which drive in a slot is "Unconfigured(bad)". Once
# the new drive is in the slot and spun up this method will bring the drive
# online, clear any foreign raid headers from the replacement drive and set the
# drive as a hot spare. We will also tell the card to start rebuilding if it
# does not start automatically. The raid should start rebuilding right away
# either way. NOTE: if you pass a slot number which is already part of the raid
# by mistake the LSI raid card is smart enough to just error out and _NOT_
# destroy the raid drive, thankfully.
if [ $1 = "replace" ]thenlogger "`hostname` - REPLACE enclosure $ENCLOSURE, drive $2 "# set Unconfigured(bad) to Unconfigured(good)$MegaCli -PDMakeGood -PhysDrv[$ENCLOSURE:$2] -a0 -NoLog# clear 'Foreign' flag or invalid raid header on replacement drive$MegaCli -CfgForeign -Clear -aALL -NoLog# set drive as hot spare$MegaCli -PDHSP -Set -PhysDrv [$ENCLOSURE:$2] -a0 -NoLog# show rebuild progress on replacement drive just to make sure it starts$MegaCli -PDRbld -ShowProg -PhysDrv [$ENCLOSURE:$2] -a0 -NoLogexit
fi# Print all the logs from the LSI raid card. You can grep on the output.
if [ $1 = "logs" ]then$MegaCli -FwTermLog -Dsply -aALL -NoLogexit
fi# Use to query the RAID card and find the drive which is rebuilding. The script
# will then query the rebuilding drive to see what percentage it is rebuilt and
# how much time it has taken so far. You can then guess-ti-mate the
# completion time.
if [ $1 = "progress" ]thenDRIVE=`$MegaCli -PDlist -aALL -NoLog | egrep 'Slot|state' | awk '/Slot/{if (x)print x;x="";}{x=(!x)?$0:x" -"$0;}END{print x;}' | sed 's/Firmware state://g' | egrep build | awk '{print $3}'`$MegaCli -PDRbld -ShowProg -PhysDrv [$ENCLOSURE:$DRIVE] -a0 -NoLogexit
fi# Use to check the status of the raid. If the raid is degraded or faulty the
# script will send email to the address in the $EMAIL variable. We normally add
# this method to a cron job to be run every few hours so we are notified of any
# issues.
if [ $1 = "checkNemail" ]thenEMAIL="raidadmin@localhost"# Check if raid is in good conditionSTATUS=`$MegaCli -LDInfo -Lall -aALL -NoLog | egrep -i 'fail|degrad|error'`# On bad raid status send email with basic drive informationif [ "$STATUS" ]; then$MegaCli -PDlist -aALL -NoLog | egrep 'Slot|state' | awk '/Slot/{if (x)print x;x="";}{x=(!x)?$0:x" -"$0;}END{print x;}' | sed 's/Firmware state://g' | mail -s `hostname`' - RAID Notification' $EMAILfi
fi# Use to print all information about the LSI raid card. Check default options,
# firmware version (FW Package Build), battery back-up unit presence, installed
# cache memory and the capabilities of the adapter. Pipe to grep to find the
# term you need.
if [ $1 = "allinfo" ]then$MegaCli -AdpAllInfo -aAll -NoLogexit
fi# Update the LSI card's time with the current operating system time. You may
# want to setup a cron job to call this method once a day or whenever you
# think the raid card's time might drift too much.
if [ $1 = "settime" ]then$MegaCli -AdpGetTime -aALL -NoLog$MegaCli -AdpSetTime `date +%Y%m%d` `date +%H:%M:%S` -aALL -NoLog$MegaCli -AdpGetTime -aALL -NoLogexit
fi# These are the defaults we like to use on the hundreds of raids we manage. You
# will want to go through each option here and make sure you want to use them
# too. These options are for speed optimization, build rate tweaks and PATROL
# options. When setting up a new machine we simply execute the "setdefaults"
# method and the raid is configured. You can use this on live raids too.
if [ $1 = "setdefaults" ]then# Read Cache enabled specifies that all reads are buffered in cache memory.$MegaCli -LDSetProp -Cached -LAll -aAll -NoLog# Adaptive Read-Ahead if the controller receives several requests to sequential sectors$MegaCli -LDSetProp ADRA -LALL -aALL -NoLog# Hard Disk cache policy enabled allowing the drive to use internal caching too$MegaCli -LDSetProp EnDskCache -LAll -aAll -NoLog# Write-Back cache enabled$MegaCli -LDSetProp WB -LALL -aALL -NoLog# Continue booting with data stuck in cache. Set Boot with Pinned Cache Enabled.$MegaCli -AdpSetProp -BootWithPinnedCache -1 -aALL -NoLog# PATROL run every 672 hours or monthly (RAID6 77TB @60% rebuild takes 21 hours)$MegaCli -AdpPR -SetDelay 672 -aALL -NoLog# Check Consistency every 672 hours or monthly$MegaCli -AdpCcSched -SetDelay 672 -aALL -NoLog# Enable autobuild when a new Unconfigured(good) drive is inserted or set to hot spare$MegaCli -AdpAutoRbld -Enbl -a0 -NoLog# RAID rebuild rate to 60% (build quick before another failure)$MegaCli -AdpSetProp \{RebuildRate -60\} -aALL -NoLog# RAID check consistency rate to 60% (fast parity checks)$MegaCli -AdpSetProp \{CCRate -60\} -aALL -NoLog# Enable Native Command Queue (NCQ) on all drives$MegaCli -AdpSetProp NCQEnbl -aAll -NoLog# Sound alarm disabled (server room is too loud anyways)$MegaCli -AdpSetProp AlarmDsbl -aALL -NoLog# Use write-back cache mode even if BBU is bad. Make sure your machine is on UPS too.$MegaCli -LDSetProp CachedBadBBU -LAll -aAll -NoLog# Disable auto learn BBU check which can severely affect raid speedsOUTBBU=$(mktemp /tmp/output.XXXXXXXXXX)echo "autoLearnMode=1" > $OUTBBU$MegaCli -AdpBbuCmd -SetBbuProperties -f $OUTBBU -a0 -NoLogrm -rf $OUTBBUexit
fi### EOF ###