Paimon版本1.17
Hive版本3.1.3
1、Paimon集成Hive
將paimon-hive-connector.jar復制到auxlib中，下載鏈接Index of /groups/snapshots/org/apache/
https://repository.apache.org/snapshots/org/apache/paimon/
通過flink進入查看paimon
/opt/softwares/flink-1.17.0/bin/sql-client.sh -s yarn-session -i /opt/softwares/flink-1.17.0/conf/sql-client-init.sql?sql-client-init.sql
CREATE CATALOG fs_catalog WITH ('type' = 'paimon','warehouse' = 'hdfs://node154:8020/paimon/fs'
);CREATE CATALOG hive_catalog WITH ('type' = 'paimon','metastore' = 'hive','uri' = 'thrift://node154:9083','hive-conf-dir' = '/opt/softwares/hive/conf','warehouse' = 'hdfs://node154:8020/paimon/hive'
);USE CATALOG hive_catalog;SET 'sql-client.execution.result-mode' = 'tableau';
注意，加載配置文件進入flink之后，雖然說使用的是hive_catalog,但是使用的database是default的，需要使用test，否則找不到表歐。
?表ws_t;和名為test的database都是之前是在flink中操作paimon在hive_catalog 創建出來的，步驟看
paimon中批和流查看過去的快照的數據及變動的數據-CSDN博客文章瀏覽閱讀258次，點贊10次，收藏4次。paimon中批和流查看過去的快照的數據及變動的數據https://blog.csdn.net/yyf960126/article/details/147930584?spm=1001.2014.3001.5502
進入hive
hive中
use test;
SELECT * FROM ws_t;
補充知識點，hive中使用【test】database來創建hive表和paimon中使用使用hive_catalog中【test】的database創建出的paimon表存儲位置不同，建表語句也能看出來。但是都能在hive中【test】的database查到。
hive查看test庫中的表為
orders ? ? ?paimon表
ws1? ? ? ? ? paimon表
ws_t? ? ? ? ?paimon表
test_hive ? hive表
yyf? ? ? ? ? hive表文件存儲為如圖：
---------------paimon表---------------------
CREATE TABLE `ws_t`(`id` int COMMENT 'from deserializer', `ts` bigint COMMENT 'from deserializer', `vc` int COMMENT 'from deserializer')
ROW FORMAT SERDE 'org.apache.paimon.hive.PaimonSerDe'
STORED BY 'org.apache.paimon.hive.PaimonStorageHandler' LOCATION'hdfs://node154:8020/paimon/hive/test.db/ws_t'
TBLPROPERTIES ('transient_lastDdlTime'='1747128118')
-----------------hive表------------------
CREATE TABLE `yyf`(`a` int)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION'hdfs://node154:8020/user/hive/warehouse/test.db/yyf'
TBLPROPERTIES ('bucketing_version'='2', 'transient_lastDdlTime'='1747066788')?在hive中創建paimon表
--使用hive_catalog的存儲路徑
SET hive.metastore.warehouse.dir=hdfs://node154:8020/paimon/hive;
--數據處理按照paimon來
CREATE TABLE test_h(a INT COMMENT 'The a field',b STRING COMMENT 'The b field'
)
STORED BY 'org.apache.paimon.hive.PaimonStorageHandler'
?
?通過創建hive外部表來使用現有的paimon表
字段隨著paimon源表的修改而自動變動，paimon表的特性
CREATE EXTERNAL TABLE test.paimon_ex_ws_t
STORED BY 'org.apache.paimon.hive.PaimonStorageHandler'
LOCATION 'hdfs://node154:8020/paimon/hive/test.db/ws_t';--或將路徑寫在表屬性中：
CREATE EXTERNAL TABLE paimon_ex_ws_t
STORED BY 'org.apache.paimon.hive.PaimonStorageHandler'
TBLPROPERTIES ('paimon_location' ='hdfs://node154:8020/paimon/hive/test.db/ws_t'
);
)
