Postgresql源碼（140）理解PG的編譯流程（make、Makefile、Makefile.global.in）

PG16

PG中使用的makefile看起來代碼比較多，但是實際邏輯比較簡單，這里做一些抽象總結。

總結

Makefile.global.in的$(recurse)宏自動生成了target，可以方便的進入內存目錄進行編譯。

all: all-common-recurse
all-common-recurse:  submake-generated-headersMAKE -C common allall: all-backend-recurse
all-backend-recurse:  submake-generated-headersMAKE -C backend allinstall: install-common-recurse
install-common-recurse:  submake-generated-headersMAKE -C common installinstall: install-backend-recurse
install-backend-recurse:  submake-generated-headersMAKE -C backend installcheck: check-common-recurse
check-common-recurse:  submake-generated-headers  temp-installMAKE -C common checkcheck: check-backend-recurse
check-backend-recurse:  submake-generated-headers  temp-installMAKE -C backend check

Makefile.global.in同時定義了對.o合.bc文件的生成規則：

%.o : %.c@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi$(COMPILE.c) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po%.o : %.cpp@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi$(COMPILE.cc) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po%.bc : %.c$(COMPILE.c.bc) -o $@ $<%.bc : %.cpp$(COMPILE.cxx.bc) -o $@ $<

在最內層目錄的Makefile定義了OBJS例如：OBJS=aset.o mcxt.o ...，然后調用common.mk文件，對OBJS中的每一個元素進行隱式編譯，common.mk：

簡化版內層Makefile：

OBJS = aset.o mcxt.o# 沒有OBJS生成規則，make隱式生成.o文件，調用上面定義的%.o : %.c規則
objfiles.txt: $(OBJS)# echo or touch objfiles

如果打開了llvm，簡化版內層Makefile：

OBJS = aset.o mcxt.o# 沒有aset.bc mcxt.bc生成規則，make隱式生成.bc文件，調用上面定義的%.bc : %.c規則。
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))     ## objfiles.txt: aset.bc mcxt.bc
# 沒有OBJS生成規則，make隱式生成.o文件，調用上面定義的%.o : %.c規則
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)          ## aset.bc mcxt.bc: aset.o mcxt.o

1 執行make后發生了什么？

簡單工程中的makefile中可以看到很多target，依賴關系一般也比較直接，例如下面例子中：

OrcV2CBindingsAddObjectFile 依賴 OrcV2CBindingsAddObjectFile.o 依賴 OrcV2CBindingsAddObjectFile.c

all: __BUILD_DIR \$(BUILD_DIR)/OrcV2CBindingsAddObjectFile \$(BUILD_DIR)/OrcV2CBindingsBasicUsage.PHONY: __BUILD_DIR$(BUILD_DIR)/OrcV2CBindingsAddObjectFile: $(BUILD_DIR)/OrcV2CBindingsAddObjectFile.o${CXX} $< ${CXXFLAGS} ${LLVM_LD_FLAGS} -o $@ 
$(BUILD_DIR)/OrcV2CBindingsAddObjectFile.o: OrcV2CBindingsAddObjectFile.c${CC} -c $< ${CFLAGS} ${LLVM_CC_FLAGS} -o $@

但打開PG根目錄下的Makefile：

subdir = src
top_builddir = ..
include Makefile.globalSUBDIRS = \common \port \timezone \backend \backend/utils/mb/conversion_procs \backend/snowball \include \interfaces \backend/replication/libpqwalreceiver \backend/replication/pgoutput \fe_utils \bin \pl \makefiles \test/regress \test/isolation \test/perlifeq ($(with_llvm), yes)
SUBDIRS += backend/jit/llvm
endif# There are too many interdependencies between the subdirectories, so
# don't attempt parallel make here.
.NOTPARALLEL:$(recurse)...
...

看不到target信息，代碼的可讀性比較差，因為所有target都是生成的。優點是target動態生成，代碼量少，比較靈活。
所有的細節都封裝在$(recurse)中，具體在Makefile.global.in中定義。

2 Makefile.global.in

截取關鍵代碼：


standard_targets = all install installdirs uninstall distprep clean distclean maintainer-clean coverage check checkprep installcheck init-po update-po
# these targets should recurse even into subdirectories not being built:
standard_always_targets = distprep clean distclean maintainer-clean.PHONY: $(standard_targets) install-strip html man installcheck-parallel update-unicode# make `all' the default target
all:##########################################################################
#
# Programs and flags# CompilersCPP = @CPP@
CPPFLAGS = @CPPFLAGS@
PG_SYSROOT = @PG_SYSROOT@override CPPFLAGS := $(ICU_CFLAGS) $(CPPFLAGS)ifdef PGXS
override CPPFLAGS := -I$(includedir_server) -I$(includedir_internal) $(CPPFLAGS)
else # not PGXS
override CPPFLAGS := -I$(top_srcdir)/src/include $(CPPFLAGS)
ifdef VPATH
override CPPFLAGS := -I$(top_builddir)/src/include $(CPPFLAGS)
endif
endif # not PGXSCC = @CC@
GCC = @GCC@
SUN_STUDIO_CC = @SUN_STUDIO_CC@
CXX = @CXX@
CFLAGS = @CFLAGS@
CFLAGS_SL = @CFLAGS_SL@
# *_MODULE are for flags applied to extension libraries
CFLAGS_SL_MODULE = @CFLAGS_SL_MODULE@
CXXFLAGS_SL_MODULE = @CXXFLAGS_SL_MODULE@
CFLAGS_UNROLL_LOOPS = @CFLAGS_UNROLL_LOOPS@
CFLAGS_VECTORIZE = @CFLAGS_VECTORIZE@
CFLAGS_CRC = @CFLAGS_CRC@
PERMIT_DECLARATION_AFTER_STATEMENT = @PERMIT_DECLARATION_AFTER_STATEMENT@
CXXFLAGS = @CXXFLAGS@LLVM_CPPFLAGS = @LLVM_CPPFLAGS@
LLVM_CFLAGS = @LLVM_CFLAGS@
LLVM_CXXFLAGS = @LLVM_CXXFLAGS@# Kind-of compilersBISON = @BISON@
BISONFLAGS = @BISONFLAGS@ $(YFLAGS)
FLEX = @FLEX@
FLEXFLAGS = @FLEXFLAGS@ $(LFLAGS)
DTRACE = @DTRACE@
DTRACEFLAGS = @DTRACEFLAGS@
ZIC = @ZIC@# LinkingAR = @AR@
AROPT = crs
LIBS = @LIBS@
LDAP_LIBS_FE = @LDAP_LIBS_FE@
LDAP_LIBS_BE = @LDAP_LIBS_BE@
UUID_LIBS = @UUID_LIBS@
LLVM_LIBS=@LLVM_LIBS@# Tree-wide build supportall install check installcheck: submake-generated-headers.PHONY: submake-generated-headerssubmake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)$(MAKE) -C $(top_builddir)/src/backend generated-headers
endif
endif##########################################################################
#
# Recursive make supportdefine _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)$$(MAKE) -C $(2) $(3)
endefrecurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call

在最后面我們看到了recurse的定義：
recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))
這里調用了_create_recursive_target函數來生成所有目錄的target。

PG的例子太復雜，這里抽象出來一個最小用例來繼續分析。

3 $(recurse)最小DEMO

Makefile


subdir = src
top_builddir = ..
include Makefile.globalSUBDIRS = \common \port \timezone \backend$(recurse)

Makefile.global


standard_targets = all install check
.PHONY: $(standard_targets)all:###############################
all install check installcheck: submake-generated-headers.PHONY: submake-generated-headerssubmake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)$(info ------------GENERATEHEADERS------------)
endif
endif###############################
define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)$(info ------------GENERATE TATGETS------------)$(info $$1: $1, $$2: $2, $$3: $3)$(info $(1): $(1)-$(2)-recurse)$(info $(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install))$(info MAKE -C $(2) $(3))$$(MAKE) -C $(2) $(3)
endefrecurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))# recurse = \
#     $(foreach target,$(if $1,$1,$(standard_targets)),\
#         $(foreach subdir,$(if $2,$2,$(SUBDIRS)),\
#             $(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\
#         )\
#     )

執行結果：
在這里插入圖片描述

4 $(recurse)最小DEMO結果分析

recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))

展開后

recurse = \$(foreach target,$(if $1,$1,$(standard_targets)),\$(foreach subdir,$(if $2,$2,$(SUBDIRS)),\$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\)\)

外層循環遍歷standard_targets = all install check，也就是PG定義的所有make指令，需要為每個指令定義target。
內層循環遍歷SUBDIRS = common backend，這個是Makefile中定義的，也就是誰include Makefile.global，誰負責定義好SUBDIRS。表示當前目錄下，哪些子目錄需要進行編譯。

調用_create_recursive_target函數，為makefile動態定義target：

define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)$$(MAKE) -C $(2) $(3)
endef

函數執行后，生成一系列target，兩個小細節：

如果target是all install check installcheck集中情況，需要增加submake-generated-headers依賴，這個是生成頭文件的，PG的一些頭文件是編譯時生成出來的。
如果target是check，需要增加temp-install依賴，temp-install是負責regress測試的target。

all: all-common-recurse
all-common-recurse:  submake-generated-headersMAKE -C common allall: all-backend-recurse
all-backend-recurse:  submake-generated-headersMAKE -C backend allinstall: install-common-recurse
install-common-recurse:  submake-generated-headersMAKE -C common installinstall: install-backend-recurse
install-backend-recurse:  submake-generated-headersMAKE -C backend installcheck: check-common-recurse
check-common-recurse:  submake-generated-headers  temp-installMAKE -C common checkcheck: check-backend-recurse
check-backend-recurse:  submake-generated-headers  temp-installMAKE -C backend check

5 繼續完善demo

前面提到的demo只構造了target，并沒有真正編譯.c文件。
繼續增加了.c文件編譯到.o的部分。

$ tree
.
├── common
│   ├── aset.c
│   └── Makefile
├── common.mk
├── Makefile
└── Makefile.global1 directory, 5 files

common/Makefile

subdir = src/backend/utils/mmgr
top_builddir = ../../../..
include ../Makefile.globalOBJS = \aset.oinclude ../common.mk

common.mk

subsysfilename = objfiles.txtifneq ($(subdir), src/backend)
all: $(subsysfilename)
endifobjfiles.txt: $(OBJS)$(if $(filter-out $(OBJS),$?),( $(if $(SUBDIROBJS),cat $(SUBDIROBJS); )echo $(addprefix $(subdir)/,$(OBJS)) ) >$@,touch $@)

Makefile


subdir = src
top_builddir = ..
include Makefile.globalSUBDIRS = \common \backend$(recurse)

Makefile.global


standard_targets = all install check
.PHONY: $(standard_targets)all:###############################
all install check installcheck: submake-generated-headers.PHONY: submake-generated-headerssubmake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)$(info ------------GENERATEHEADERS------------)
endif
endif###############################
define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)$(info ------------GENERATE TATGETS------------)$(info $$1: $1, $$2: $2, $$3: $3)$(info $(1): $(1)-$(2)-recurse)$(info $(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install))$(info MAKE -C $(2) $(3))$$(MAKE) -C $(2) $(3)
endefrecurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))# recurse = \
#     $(foreach target,$(if $1,$1,$(standard_targets)),\
#         $(foreach subdir,$(if $2,$2,$(SUBDIRS)),\
#             $(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\
#         )\
#     )

這里主要需要關注的就是common.mk。每個源碼目錄中都有一些文件需要編譯，所以都需要配一個Makefile，這些Makefile中相同的邏輯抽取到了common.mk中。

objfiles.txt: $(OBJS)......

注意這里OBJS = aset.o，這里makefile構造objfiles.txt依賴aset.o，而aset.o觸發了Makefile的隱式規則，會自動從aset.c編譯為aset.o。

除了規則外，具體的編譯方法也可以在Makefile中定義，例如PG的makefile.global.in中定義了對.o文件的處理方法。

%.o : %.c@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi$(COMPILE.c) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po%.o : %.cpp@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi$(COMPILE.cc) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po

llvm支持：common.mk中還有一段，如果打開了llvm后，objfiles.txt會同時依賴.o和.bc文件。

ifeq ($(with_llvm), yes)
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)
endif

在makefile.global.in中也定義了對.bc文件的處理方法：

%.bc : %.c$(COMPILE.c.bc) -o $@ $<%.bc : %.cpp$(COMPILE.cxx.bc) -o $@ $<