打包命令
打包整個項目
mvn clean package -DskipTests -Drat.skip=true
打包單個模塊
mvn clean package -DskipTests -Drat.skip=true -pl flink-dist
如果該模塊依賴其他模塊,可能需要先將其他模塊 install 到本地,如果依賴的模塊的源代碼有修改,則需要重新install,比如 flink-dist 依賴的 flink-clients 模塊源代碼有改動,則需要先:
mvn clean install -DskipTests -Drat.skip=true -pl flink-clients
修改代碼
如果要打的源代碼修改過,那么需要先執行:
mvn spotless:apply
# 或只對單個模塊
mvn spotless:apply -pl flink-clients
否則會報:
The following files had format violations:
……
Run 'mvn spotless:apply' to fix these violations.
flink-runtime-web
Windows
windows 打包會報如下錯誤
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26:06 min
[INFO] Finished at: 2025-04-24T10:03:43+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.0:npm (npm install) on project flink-runtime-web: Failed to run task: 'npm ci --cache-max=0 --no-save ${npm.proxy}' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :flink-runtime-web
該異常沒有解決,需要在Linux上打包。但是如果在Windows上開發,我們可以選擇跳過 flink-runtime-web ,在Windows打包成功后驗證一下我們修改代碼的邏輯沒問題后再將代碼傳到Linux服務器進行完整打包。跳過 flink-runtime-web 的方法只需要修改 pom 即可:
Windows打包截圖:
也就是打包時沒有包含flink-runtime-web,這樣打出來的flink-dist 包運行的flink 應用對應的 web ui 是打不開的。另外對于我們要修改的代碼對應的模塊如果打包時如果不涉及 flink-runtime-web ,這樣我們只需要打包對應的模塊,只上傳該模塊對應的jar包就行,因為不涉及 flink-runtime-web,所以無需在 Linux 打完整的包。
Linux
如果我們修改的代碼打包時涉及 flink-runtime-web 模塊,或者我們想完整的打包,那么就需要在 Linux上打包,打包 flink-runtime-web 需要安裝 node、npm,也就是需要下載node包,但是因為網絡等原因,打包時下載可能會失敗,這樣我們可以提前將對應版本的包下載到本地,然后放到對應的maven 倉庫路徑下即可,以 flink 1.15.3為例:
wget https://nodejs.org/dist/v16.13.2/node-v16.13.2-linux-x64.tar.gz
mkdir /opt/workspace/m2/repository/com/github/eirslett/node/16.13.2
mv node-v16.13.2-linux-x64.tar.gz /opt/workspace/m2/repository/com/github/eirslett/node/16.13.2/node-16.13.2-linux-x64.tar.gz
另外打包時因為涉及連接外網,可能會遇到很多因網絡問題導致的失敗,需要重試很多次,這樣我們如果從頭打包因為很多重復的沒必要的模塊浪費時間,我們可以將之前打包成功的模塊install本地,然后單獨打 flink-runtime-web,打包命令:
mvn clean package -DskipTests -Drat.skip=true -pl flink-runtime-web
這樣比較節省時間,當flink-runtime-web打包成功后,再全部打一遍即可。當然有條件的最好通過配置VPN連接外網,這樣打包會順利很多。
JDK 版本
在Linux打包時遇到了JDK版本的問題,記錄如下
jdk1.8.0_45
:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /root/workspace/flink/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/operations/utils/OperationTreeBuilder.java:[663,57] unreported exception X; must be caught or declared to be thrown
[INFO] 1 error
openjdk-1.8.0.181
:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /opt/workspace/flink/flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/internal/converter/OracleRowConverter.java:[164,43] cannot find symbolsymbol: method getTimeZone()location: variable ts of type oracle.sql.TIMESTAMPTZ
[INFO] 1 error
最后升級到 jdk1.8.0_421
成功,對應的安裝包下載地址:
- https://www.oracle.com/java/technologies/downloads/?er=221886#java8-linux
- https://www.oracle.com/java/technologies/downloads/?er=221886#license-lightbox
- https://download.oracle.com/otn/java/jdk/8u421-b09/d8aa705069af427f9b83e66b34f5e380/jdk-8u421-linux-x64.tar.gz
Jar 包路徑
每個模塊的target目錄下是有對應的jar包的,但是有的模塊對應的jar包名帶有scala版本,比如 flink-dist_2.12-1.15.3.jar,但默認的安裝包是不帶 scala版本的,真正的安裝包對應的目錄為 flink-dist/target/flink-${project.version}
-bin/flink-${project.version}
以flink 1.15.3 為例:flink-dist/target/flink-1.15.3-bin/flink-1.15.3,所有相關的jar包,腳本都在這個目錄下。另外在Linux上打包時會創建一個該目錄的軟連接:build-target,所以我們在Linux上只需要在build-target找自己需要的包即可,但對于Windows則沒有對應的build-target文件夾。