1.數據源管理

1.1 添加Gravitino數據源

添加成功之后，會在Gravitino中創建一個名為配置的中的meatalake

1.2. 添加Paimon數據源

屬性gravitinoId可以關聯前面創建的Gravitino數據源，關聯后，會在gravitino下創建一個該數據源的catalog。

2. 集成演示

2.1 創建任務

• 入口：通過頂部菜單欄選擇 任務開發，或通過快捷入口 快速創建任務。

• 任務類型：選擇 FlinkPipeline。

  

  

2.2 配置任務

點擊任務名稱，進入任務詳情頁。任務節點如下

功能說明：

使用DataGen節點生成100條測試數據，使用JDBCWrite將數據寫入mysql的user表，再通過JDBCRead讀取mysql的user表，使用SQLExecute寫入到Paimon。通過SQLQuery讀取Paimon,最后通過ShowData節點輸入讀取的數據。

{"flow": {"engineType": "flink","name": "flink_gravition_paimon","paths": [{"from": "Gravitino_tuky5","inport": "","outport": "","to": "DataGen_r6QRQ"},{"from": "DataGen_r6QRQ","inport": "","outport": "","to": "JDBCWrite__yYGc"},{"from": "JDBCWrite__yYGc","inport": "","outport": "","to": "JDBCRead_WREvh"},{"from": "JDBCRead_WREvh","inport": "","outport": "","to": "SQLExecute_bMeVm"},{"from": "SQLExecute_bMeVm","inport": "","outport": "","to": "SQLQuery_8jwT7"},{"from": "SQLQuery_8jwT7","inport": "","outport": "","to": "ShowData_aMIL3"}],"runMode": "DEBUG","stops": [{"bundle": "cn.piflow.bundle.flink.catalog.Gravitino","customizedProperties": {},"name": "Gravitino_tuky5","properties": {"gravitinoUri": "http://127.0.0.1:8092/","metalake": "metalake"},"uuid": "Gravitino_tuky5"},{"bundle": "cn.piflow.bundle.flink.common.DataGen","customizedProperties": {},"name": "DataGen_r6QRQ","properties": {"count": "100","ratio": "10","registerTableName": "datagen_source","schema": "[{\"id\":\"323868\",\"filedName\":\"id\",\"filedType\":\"INT\",\"kind\":\"sequence\",\"start\":\"1\",\"end\":\"1000\"},{\"id\":\"479324\",\"filedName\":\"name\",\"filedType\":\"STRING\",\"kind\":\"random\",\"length\":\"5\",\"index\":1}]"},"uuid": "DataGen_r6QRQ"},{"bundle": "cn.piflow.bundle.flink.jdbc.JDBCWrite","customizedProperties": {},"name": "JDBCWrite__yYGc","properties": {"driver": "com.mysql.jdbc.Driver","password": "123456","properties": "{}","tableDefinition": "{\"tableBaseInfo\":{},\"physicalColumnDefinition\":[{\"columnName\":\"id\",\"columnType\":\"INT\",\"length\":null,\"precision\":null,\"scale\":null,\"nullable\":false,\"primaryKey\":false,\"partitionKey\":false,\"comment\":\"\"},{\"columnName\":\"name\",\"columnType\":\"STRING\",\"length\":null,\"precision\":null,\"scale\":null,\"nullable\":false,\"primaryKey\":false,\"partitionKey\":false,\"comment\":\"\"}],\"metadataColumnDefinition\":[],\"computedColumnDefinition\":[],\"watermarkDefinition\":{}}","tableName": "user","url": "jdbc:mysql://127.0.0.1:3306/test2?characterEncoding=utf8&autoReconnect=true&tinyInt1isBit=false&serverTimezone=Asia/Shanghai","username": "root"},"uuid": "JDBCWrite__yYGc"},{"bundle": "cn.piflow.bundle.flink.jdbc.JDBCRead","customizedProperties": {},"name": "JDBCRead_WREvh","properties": {"driver": "com.mysql.jdbc.Driver","fetchSize": "10","password": "123456","properties": "{}","tableDefinition": "{\"tableBaseInfo\":{\"catalogName\":\"\",\"dbname\":\"\",\"schema\":\"\",\"registerTableName\":\"t_user_source\",\"registerTableComment\":\"\",\"ifNotExists\":true,\"selectStatement\":\"\",\"likeStatement\":\"\"},\"physicalColumnDefinition\":[{\"columnName\":\"id\",\"columnType\":\"INT\",\"length\":null,\"precision\":null,\"scale\":null,\"nullable\":false,\"primaryKey\":false,\"partitionKey\":false,\"comment\":\"\"},{\"columnName\":\"name\",\"columnType\":\"STRING\",\"length\":null,\"precision\":null,\"scale\":null,\"nullable\":false,\"primaryKey\":false,\"partitionKey\":false,\"comment\":\"\"}],\"metadataColumnDefinition\":[],\"computedColumnDefinition\":[],\"watermarkDefinition\":{}}","tableName": "user","url": "jdbc:mysql://127.0.0.1:3306/test2?characterEncoding=utf8&autoReconnect=true&tinyInt1isBit=false&serverTimezone=Asia/Shanghai","username": "root","useTableEnv": "true"},"uuid": "JDBCRead_WREvh"},{"bundle": "cn.piflow.bundle.flink.common.SQLQuery","customizedProperties": {},"name": "SQLQuery_8jwT7","properties": {"registerResultViewName": "","registerSourceViewName": "","sql": "select id, name from paimon.test.t_user","useTableEnv": "true"},"uuid": "SQLQuery_8jwT7"},{"bundle": "cn.piflow.bundle.flink.common.SQLExecute","customizedProperties": {},"name": "SQLExecute_bMeVm","properties": {"sql": "create table if not exists paimon.test.t_user (\r\n id int,\r\n name string\r\n);\r\n\r\ninsert into paimon.test.t_user  select * from t_user_source;","useTableEnv": "true"},"uuid": "SQLExecute_bMeVm"},{"bundle": "cn.piflow.bundle.flink.common.ShowData","customizedProperties": {},"name": "ShowData_aMIL3","properties": {"changeLog": "false","showNumber": "100"},"uuid": "ShowData_aMIL3"}],"uuid": "111"}
}

2.3 運行任務

• 點擊 運行 按鈕啟動任務。
  

🔗 平臺體驗地址：DataStudio （http://1.94.182.15:8090）