原文:http://blog.csdn.net/shushugood/article/details/9000628
--------------------------------------------------------
該項目是中國聯通xxxx話務系統,我的架構設計+需求設計,+運維保障+數據庫開發,全套服務。
在今天開發完畢后,突然有個模塊的需求,用戶號碼為必須選擇,感覺有點郁悶,因為1小時有1000w數據,把所有用戶號碼顯示出來,是不是有點畫蛇添足呢。
我的開始設想是查詢詳單,像中移營業廳,需要輸入號碼,或者省份證查詢模糊查詢,沒有謂詞不能查詢。(感覺設計合情合理)
1.但是想了解整個系統用戶分布情況,必須輸入條件,是不是有點不可用。
2.并且沒有謂詞過濾,查詢會慢,非常慢(1-2分鐘出結果),目標是3-5秒內出數據。
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??? 注:優化難點是把2秒變成1秒, ?反之,把2小時變成2分鐘非常簡單。
第1步:
下面看看語句和執行計劃:
- SQL>??explain?plan?for??SELECT?/*+?parallel(8)???*/??
- ??2???starttime?starttime,??
- ??3???cv.groupid,??
- ??4???cs.custmangerid,??
- ??5???callercarrier?callercarrier,??
- ??6???callernum?callernum,??
- ??7???calledcarrier?calledcarrier,??
- ??8???callednum?callednum,??
- ??9???calleenum?calleenum,??
- ?10???round(duration?/?60,?2)?CallTimeLen,??
- ?11???count(*)?over(ORDER?BY?NULL?ROWS?BETWEEN?UNBOUNDED?PRECEDING?AND?UNBOUNDED?FOLLOWING)?"@totalrows"??
- ?12????FROM?CS_xxxx?dt,?cfg_vipphones?cv,?cfg_vipusers?cs??
- ?13???WHERE?dt.StartTime?>=?'2013-05-31?13:00:00'??
- ?14?????and?dt.StartTime?<?'2013-05-31?14:00:00'??
- ?15?????AND?dt.Callercarrier?=?2??
- ?16?????AND?dt.callernum?>=?cv.beginphone??
- ?17?????and?dt.callernum?<=?cv.endphone??
- ?18?????and?cv.groupid?=?cs.groupid;??
- ???
- Explained??
- ???
- SQL>?select?*?from?table(dbms_xplan.display);??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- Plan?hash?value:?2172492340??
- --------------------------------------------------------------------------------??
- |?Id??|?Operation??????????????????????????????|?Name????????????|?Rows??|?Bytes??
- --------------------------------------------------------------------------------??
- |???0?|?SELECT?STATEMENT???????????????????????|?????????????????|???478K|????34??
- |*??1?|??PX?COORDINATOR????????????????????????|?????????????????|???????|??
- |???2?|???PX?SEND?QC?(RANDOM)??????????????????|?:TQ10001????????|???478K|????34??
- |???3?|????WINDOW?BUFFER???????????????????????|?????????????????|???478K|????34??
- |*??4?|?????FILTER?????????????????????????????|?????????????????|???????|??
- |???5?|??????MERGE?JOIN????????????????????????|?????????????????|???478K|????34??
- |???6?|???????SORT?JOIN????????????????????????|?????????????????|????11?|???363??
- |???7?|????????BUFFER?SORT?????????????????????|?????????????????|???????|??
- |???8?|?????????PX?RECEIVE?????????????????????|?????????????????|???????|??
- |???9?|??????????PX?SEND?BROADCAST?????????????|?:TQ10000????????|???????|??
- |??10?|???????????NESTED?LOOPS?????????????????|?????????????????|???????|??
- |??11?|????????????NESTED?LOOPS????????????????|?????????????????|????11?|???363??
- |??12?|?????????????TABLE?ACCESS?BY?INDEX?ROWID|?CFG_VIPUSERS????|?????3?|????18??
- |??13?|??????????????INDEX?FULL?SCAN???????????|?PK_CFG_VIPUSERS?|?????3?|??
- |*?14?|?????????????INDEX?RANGE?SCAN???????????|?VIPUSERS_FK?????|?????4?|??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- |??15?|????????????TABLE?ACCESS?BY?INDEX?ROWID?|?CFG_VIPPHONES???|?????4?|???108??
- |*?16?|???????FILTER???????????????????????????|?????????????????|???????|??
- |*?17?|????????SORT?JOIN???????????????????????|?????????????????|???516K|????21??
- |??18?|?????????PX?BLOCK?ITERATOR??????????????|?????????????????|???516K|????21??
- |*?19?|??????????TABLE?ACCESS?FULL?????????????|?CS_xxxx??????????|???516K|????21??
- --------------------------------------------------------------------------------??
- Predicate?Information?(identified?by?operation?id):??
- ---------------------------------------------------??
- ???1?-?filter(TO_DATE('2013-05-31?13:00:00')<TO_DATE('2013-05-31?14:00:00'))??
- ???4?-?filter(TO_DATE('2013-05-31?13:00:00')<TO_DATE('2013-05-31?14:00:00'))??
- ??14?-?access("CV"."GROUPID"="CS"."GROUPID")??
- ??16?-?filter("DT"."CALLERNUM"<="CV"."ENDPHONE")??
- ??17?-?access("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ???????filter("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ??19?-?filter("DT"."CALLERCARRIER"=2?AND?"DT"."STARTTIME">='2013-05-31?13:00:00'??
- Note??
- -----??
- ???-?Degree?of?Parallelism?is?8?because?of?hint??
- ???
- 41?rows?selected??
- ???
- SQL>???
OLAP和OLTP 又有很大區別了,包含數據庫參數設定,sql寫法,hint是否啟用等
第2步:
我懷疑是3張表關聯,謂詞出了問題。
注意看filter,看看是否是分區表搞的鬼。 查看后一切正常,因為是我寫的,我最清楚。哈哈。。。
在多表關聯時,如果有視圖,可以考慮視圖的合并,關聯的優先選擇,再hash。 都試過了,不行。
第3 步:
懷疑是并行出錯了,看看表的并且度,索引并行,
或者我不要并行試試。果然,8-10秒出結果
- SQL>??explain?plan?for??SELECT??
- ??2???starttime?starttime,??
- ??3???cv.groupid,??
- ??4???cs.custmangerid,??
- ??5???callercarrier?callercarrier,??
- ??6???callernum?callernum,??
- ??7???calledcarrier?calledcarrier,??
- ??8???callednum?callednum,??
- ??9???calleenum?calleenum,??
- ?10???round(duration?/?60,?2)?CallTimeLen,??
- ?11???count(*)?over(ORDER?BY?NULL?ROWS?BETWEEN?UNBOUNDED?PRECEDING?AND?UNBOUNDED?FOLLOWING)?"@totalrows"??
- ?12????FROM?CS——xx?dt,?cxg_vippxxx?cv,?cxg_vipxxx?cs??
- ?13???WHERE?dt.StartTime?>=?'2013-05-31?13:00:00'??
- ?14?????and?dt.StartTime?<?'2013-05-31?14:00:00'??
- ?15?????AND?dt.Callercarrier?=?2??
- ?16?????AND?dt.callernum?>=?cv.beginphone??
- ?17?????and?dt.callernum?<=?cv.endphone??
- ?18?????and?cv.groupid?=?cs.groupid;??
- ???
- Explained??
- ???
- SQL>?select?*?from?table(dbms_xplan.display);??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- Plan?hash?value:?1705527799??
- --------------------------------------------------------------------------------??
- |?Id??|?Operation?????????????????????????|?Name????????????|?Rows??|?Bytes?|Tem??
- --------------------------------------------------------------------------------??
- |???0?|?SELECT?STATEMENT??????????????????|?????????????????|???478K|????34M|??
- |???1?|??WINDOW?BUFFER????????????????????|?????????????????|???478K|????34M|??
- |*??2?|???FILTER??????????????????????????|?????????????????|???????|???????|??
- |???3?|????MERGE?JOIN?????????????????????|?????????????????|???478K|????34M|??
- |???4?|?????SORT?JOIN?????????????????????|?????????????????|????11?|???363?|??
- |???5?|??????NESTED?LOOPS?????????????????|?????????????????|???????|???????|??
- |???6?|???????NESTED?LOOPS????????????????|?????????????????|????11?|???363?|??
- |???7?|????????TABLE?ACCESS?BY?INDEX?ROWID|?CFGxxUSERS????|?????3?|????18?|??
- |???8?|?????????INDEX?FULL?SCAN???????????|?PK_CFG_VIPUSERS?|?????3?|???????|??
- |*??9?|????????INDEX?RANGE?SCAN???????????|?VIPUSERS_FK?????|?????4?|???????|??
- |??10?|???????TABLE?ACCESS?BY?INDEX?ROWID?|?CFG_xxNES???|?????4?|???108?|??
- |*?11?|?????FILTER????????????????????????|?????????????????|???????|???????|??
- |*?12?|??????SORT?JOIN????????????????????|?????????????????|???516K|????21M|??
- |??13?|???????PARTITION?RANGE?ITERATOR????|?????????????????|???516K|????21M|??
- |*?14?|????????TABLE?ACCESS?FULL??????????|?CS_xxx?????????|???516K|????21M|??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- --------------------------------------------------------------------------------??
- Predicate?Information?(identified?by?operation?id):??
- ---------------------------------------------------??
- ???2?-?filter(TO_DATE('2013-05-31?13:00:00')<TO_DATE('2013-05-31?14:00:00'))??
- ???9?-?access("CV"."GROUPID"="CS"."GROUPID")??
- ??11?-?filter("DT"."CALLERNUM"<="CV"."ENDPHONE")??
- ??12?-?access("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ???????filter("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ??14?-?filter("DT"."CALLERCARRIER"=2?AND?"DT"."STARTTIME">='2013-05-31?13:00:00'??
- ??????????????14:00:00')??
- ???
- 32?rows?selected??
- ???
- SQL>???
第4步:
看看并行設置,這個也有很大關系,因為并行的模塊太多,造成排隊擁塞的情況
<1>如果有并行度低于系統最大并行數的查詢在跑,那接下來的并行查詢會怎么跑呢?
When you specify parallel degree 4 oracle tries to allocate 4 producer slaves and 4 consumer slaves. The producers can feed any of the consumers.?
If there are only 2 slaves available then we use these.?
If there is only 1 slave available then we go serial?
If there are none available then we use serial.?
If parallel_min_percent is set then we error ora 12827 instead of using a lower number of slaves or going serial
<2>設定parallel_max_servers 多大為好?
在多CPU的環境中,一般把CPU-1或CPU的數量做個最大并行數,因為并行查詢運行時還需要一個進程協調各并行進程.對于單CPU沒什么好說的.
<3>并行查詢能提高系統的性能嗎?
并行查詢運行時,很容易會使機器運行在高負荷下,令系統對其它事務的處理時間大大加長.并行查詢一般適合在非業務高峰值人工執行,并不適合在程序中指定運行并行查詢.
PINNER:
并行不等于快速,僅僅是適合在數據倉庫環境,低業務請求與低并發操作的時候
典型的OLTP系統,如果我們的系統,是絕對不允許并行查詢出現的。?
(引薦哈)
第5步:
問題解決,注意看看問題,paralle的寫法,當一個表時,用parallel(8) , 表示當前表并行8個進程
?當有多個表是,請指定某一個表,否則會默認3個表,當然執行計劃上看不出來,可以trace一把 看看
- SQL>??explain?plan?for??SELECT?/*+?parallel(dt,8)???*/??
- ??2???starttime?starttime,??
- ??3???cv.groupid,??
- ??4???cs.custmangerid,??
- ??5???callercarrier?callercarrier,??
- ??6???callernum?callernum,??
- ??7???calledcarrier?calledcarrier,??
- ??8???callednum?callednum,??
- ??9???calleenum?calleenum,??
- ?10???round(duration?/?60,?2)?CallTimeLen,??
- ?11???count(*)?over(ORDER?BY?NULL?ROWS?BETWEEN?UNBOUNDED?PRECEDING?AND?UNBOUNDED?FOLLOWING)?"@totalrows"??
- ?12????FROM?CS_CDR?dt,?cfg_vipphones?cv,?cfg_vipusers?cs??
- ?13???WHERE?dt.StartTime?>=?'2013-05-31?13:00:00'??
- ?14?????and?dt.StartTime?<?'2013-05-31?14:00:00'??
- ?15?????AND?dt.Callercarrier?=?2??
- ?16?????AND?dt.callernum?>=?cv.beginphone??
- ?17?????and?dt.callernum?<=?cv.endphone??
- ?18?????and?cv.groupid?=?cs.groupid;??
- ???
- Explained??
- ???
- SQL>?select?*?from?table(dbms_xplan.display);??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- Plan?hash?value:?2172492340??
- --------------------------------------------------------------------------------??
- |?Id??|?Operation??????????????????????????????|?Name????????????|?Rows??|?Bytes??
- --------------------------------------------------------------------------------??
- |???0?|?SELECT?STATEMENT???????????????????????|?????????????????|???478K|????34??
- |*??1?|??PX?COORDINATOR????????????????????????|?????????????????|???????|??
- |???2?|???PX?SEND?QC?(RANDOM)??????????????????|?:TQ10001????????|???478K|????34??
- |???3?|????WINDOW?BUFFER???????????????????????|?????????????????|???478K|????34??
- |*??4?|?????FILTER?????????????????????????????|?????????????????|???????|??
- |???5?|??????MERGE?JOIN????????????????????????|?????????????????|???478K|????34??
- |???6?|???????SORT?JOIN????????????????????????|?????????????????|????11?|???363??
- |???7?|????????BUFFER?SORT?????????????????????|?????????????????|???????|??
- |???8?|?????????PX?RECEIVE?????????????????????|?????????????????|???????|??
- |???9?|??????????PX?SEND?BROADCAST?????????????|?:TQ10000????????|???????|??
- |??10?|???????????NESTED?LOOPS?????????????????|?????????????????|???????|??
- |??11?|????????????NESTED?LOOPS????????????????|?????????????????|????11?|???363??
- |??12?|?????????????TABLE?ACCESS?BY?INDEX?ROWID|?CFG_VIPUSERS????|?????3?|????18??
- |??13?|??????????????INDEX?FULL?SCAN???????????|?PK_CFG_VIPUSERS?|?????3?|??
- |*?14?|?????????????INDEX?RANGE?SCAN???????????|?VIPUSERS_FK?????|?????4?|??
- ???
- PLAN_TABLE_OUTPUT??
- --------------------------------------------------------------------------------??
- |??15?|????????????TABLE?ACCESS?BY?INDEX?ROWID?|?CFG_VIPPHONES???|?????4?|???108??
- |*?16?|???????FILTER???????????????????????????|?????????????????|???????|??
- |*?17?|????????SORT?JOIN???????????????????????|?????????????????|???516K|????21??
- |??18?|?????????PX?BLOCK?ITERATOR??????????????|?????????????????|???516K|????21??
- |*?19?|??????????TABLE?ACCESS?FULL?????????????|?CS_xxxx?????????|???516K|????21??
- --------------------------------------------------------------------------------??
- Predicate?Information?(identified?by?operation?id):??
- ---------------------------------------------------??
- ???1?-?filter(TO_DATE('2013-05-31?13:00:00')<TO_DATE('2013-05-31?14:00:00'))??
- ???4?-?filter(TO_DATE('2013-05-31?13:00:00')<TO_DATE('2013-05-31?14:00:00'))??
- ??14?-?access("CV"."GROUPID"="CS"."GROUPID")??
- ??16?-?filter("DT"."CALLERNUM"<="CV"."ENDPHONE")??
- ??17?-?access("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ???????filter("DT"."CALLERNUM">="CV"."BEGINPHONE")??
- ??19?-?filter("DT"."CALLERCARRIER"=2?AND?"DT"."STARTTIME">='2013-05-31?13:00:00'??
- ???
- 37?rows?selected??
- ???
- SQL>???
目前是3秒出結果,已經達到預期,當然謂詞為1小時,或者有號碼過濾絕對是1秒內響應速度。