前面兩節學習到了各種Source的創建和extractor service的啟動，本節將以本地播放為例記錄下GenericSource是如何創建一個extractor的。extractor是在PrepareAsync()方法中被創建出來的，為了不過多贅述，我們直接從GenericSource的onPrepareAsync()開始看。

onPrepareAsync()

Android系統自帶了很多源生的extractor，我們這里主要基于MP4 extractor來進行以下內容的分析。

//frameworks/av/media/libmediaplayerservice/nuplayer/GenericSource.cpp
void NuPlayer::GenericSource::onPrepareAsync() {mDisconnectLock.lock();// delayed data source creationif (mDataSource == NULL) {// set to false first, if the extractor// comes back as secure, set it to true then.mIsSecure = false;if (!mUri.empty()) {//省略} else {//第一部分if (property_get_bool("media.stagefright.extractremote", true) &&!PlayerServiceFileSource::requiresDrm(mFd.get(), mOffset, mLength, nullptr /* mime */)) {sp<IBinder> binder =defaultServiceManager()->getService(String16("media.extractor"));if (binder != nullptr) {ALOGD("FileSource remote");sp<IMediaExtractorService> mediaExService(interface_cast<IMediaExtractorService>(binder));sp<IDataSource> source;mediaExService->makeIDataSource(base::unique_fd(dup(mFd.get())), mOffset, mLength, &source);ALOGV("IDataSource(FileSource): %p %d %lld %lld",source.get(), mFd.get(), (long long)mOffset, (long long)mLength);if (source.get() != nullptr) {mDataSource = CreateDataSourceFromIDataSource(source);}//省略}//省略}//省略}//省略mDisconnectLock.unlock();//第二部分// init extractor from data sourcestatus_t err = initFromDataSource();if (err != OK) {ALOGE("Failed to init from data source!");notifyPreparedAndCleanup(err);return;}if (mVideoTrack.mSource != NULL) {sp<MetaData> meta = getFormatMeta_l(false /* audio */);sp<AMessage> msg = new AMessage;err = convertMetaDataToMessage(meta, &msg);if(err != OK) {notifyPreparedAndCleanup(err);return;}notifyVideoSizeChanged(msg);}notifyFlagsChanged(// FLAG_SECURE will be known if/when prepareDrm is called by the app// FLAG_PROTECTED will be known if/when prepareDrm is called by the appFLAG_CAN_PAUSE |FLAG_CAN_SEEK_BACKWARD |FLAG_CAN_SEEK_FORWARD |FLAG_CAN_SEEK);//第三部分finishPrepareAsync();ALOGV("onPrepareAsync: Done");
}

上述代碼中省略了mp4文件播放時不會走到的流程，只抓主干做了解。我將onPrepareAsync()分成了三個部分，下面逐個進行分析。

DataSource的創建

初始階段GenericSource的mDataSource是沒有值的，因此需要基于setDataSource()傳遞下來的文件fd/offset/length變量來創建一個。先將步驟總結如下：

• 獲取"media.extractor" service的本地代理，為調用其接口做準備。
• 基于被打開MP4文件的fd/offset/length創建一個RemoteDataSource，并返回其Bp端（BpDataSource）。
• 將BpDataSource轉化為TinyCacheSource，保存到mDataSource中。

第一步沒啥好講的，直接開始講第二步：

//frameworks/av/services/mediaextractor/MediaExtractorService.cpp
::android::binder::Status MediaExtractorService::makeIDataSource(base::unique_fd fd,int64_t offset,int64_t length,::android::sp<::android::IDataSource>* _aidl_return) {sp<DataSource> source = DataSourceFactory::getInstance()->CreateFromFd(fd.release(), offset, length);*_aidl_return = CreateIDataSourceFromDataSource(source);return binder::Status::ok();
}//frameworks/av/media/libdatasource/DataSourceFactory.cpp
sp<DataSourceFactory> DataSourceFactory::getInstance() {Mutex::Autolock l(sInstanceLock);if (!sInstance) {sInstance = new DataSourceFactory();}return sInstance;
}sp<DataSource> DataSourceFactory::CreateFromFd(int fd, int64_t offset, int64_t length) {sp<FileSource> source = new FileSource(fd, offset, length);return source->initCheck() != OK ? nullptr : source;
}//frameworks/av/media/libstagefright/InterfaceUtils.cpp
sp<IDataSource> CreateIDataSourceFromDataSource(const sp<DataSource> &source) {if (source == nullptr) {return nullptr;}return RemoteDataSource::wrap(source);
}//frameworks/av/media/libstagefright/include/media/stagefright/RemoteDataSource.h
static sp<IDataSource> wrap(const sp<DataSource> &source) {if (source.get() == nullptr) {return nullptr;}if (source->getIDataSource().get() != nullptr) {return source->getIDataSource();}return new RemoteDataSource(source);
}

這里直接調用extractor service的makeIDataSource()方法，在該方法中會先構建一個FileSource實例，通過這個實例可以讀取文件內容。基于FileSource再封裝成一個RemoteDataSource實例，通過binder回傳到GenericSource那的已經是Bp端了。

接下來是第三步：

//frameworks/av/media/libstagefright/InterfaceUtils.cpp
sp<DataSource> CreateDataSourceFromIDataSource(const sp<IDataSource> &source) {if (source == nullptr) {return nullptr;}return new TinyCacheSource(new CallbackDataSource(source));
}

可以很清楚的看到，BpDataSource被先后封裝了兩層最終返回的則是TinyCacheSource實例。

到這里，第一部分結束了。

initFromDataSource()

第二部分則是重點了，這里是創建extractor的位子所在。

//frameworks/av/media/libmediaplayerservice/nuplayer/GenericSource.cpp
status_t NuPlayer::GenericSource::initFromDataSource() {sp<IMediaExtractor> extractor;sp<DataSource> dataSource;{Mutex::Autolock _l_d(mDisconnectLock);dataSource = mDataSource;}CHECK(dataSource != NULL);// This might take long time if data source is not reliable.extractor = MediaExtractorFactory::Create(dataSource, NULL);//省略sp<MetaData> fileMeta = extractor->getMetaData();size_t numtracks = extractor->countTracks();//省略mFileMeta = fileMeta;//省略for (size_t i = 0; i < numtracks; ++i) {sp<IMediaSource> track = extractor->getTrack(i);if (track == NULL) {continue;}sp<MetaData> meta = extractor->getTrackMetaData(i);//省略// Do the string compare immediately with "mime",// we can't assume "mime" would stay valid after another// extractor operation, some extractors might modify meta// during getTrack() and make it invalid.if (!strncasecmp(mime, "audio/", 6)) {if (mAudioTrack.mSource == NULL) {mAudioTrack.mIndex = i;mAudioTrack.mSource = track;mAudioTrack.mPackets =new AnotherPacketSource(mAudioTrack.mSource->getFormat());if (!strcasecmp(mime, MEDIA_MIMETYPE_AUDIO_VORBIS)) {mAudioIsVorbis = true;} else {mAudioIsVorbis = false;}mMimes.add(String8(mime));}} else if (!strncasecmp(mime, "video/", 6)) {if (mVideoTrack.mSource == NULL) {mVideoTrack.mIndex = i;mVideoTrack.mSource = track;mVideoTrack.mPackets =new AnotherPacketSource(mVideoTrack.mSource->getFormat());// video always at the beginningmMimes.insertAt(String8(mime), 0);}}//省略}//省略return OK;
}

上述代碼只保留了主干，這段代碼的主要做了這些事情：

• 創建RemoteMediaExtractor，并返回其Bp端（BpMediaExtractor）。這里比較復雜，稍后詳細展開。
• 通過BpMediaExtractor調用getMetaData()讀取并解析MP4文件的metadata，保存到mFileMeta中。
• 調用countTracks()獲取MP4文件中包含的track數量。
• 依次遍歷這些track，根據其內的MIME type將對應的track區分為video還是audio track，保存在mVideoTrack/mAudioTrack中。mVideoTrack/mAudioTrack每個都會創建一個AnotherPacketSource保存起來，這個AnotherPacketSource應該就是為后面解碼提供數據了。

MediaExtractorFactory::Create()

下面來解析下MediaExtractorFactory::Create()。

//frameworks/av/media/libstagefright/MediaExtractorFactory.cpp
sp<IMediaExtractor> MediaExtractorFactory::Create(const sp<DataSource> &source, const char *mime) {ALOGV("MediaExtractorFactory::Create %s", mime);// remote extractorALOGV("get service manager");sp<IBinder> binder = defaultServiceManager()->getService(String16("media.extractor"));if (binder != 0) {sp<IMediaExtractorService> mediaExService(interface_cast<IMediaExtractorService>(binder));sp<IMediaExtractor> ex;mediaExService->makeExtractor(CreateIDataSourceFromDataSource(source),mime ? std::optional<std::string>(mime) : std::nullopt,&ex);return ex;}
}

調用extractor的makeExtractor()方法直接創建extractor。在此之前，需要先從TinyCacheSource對象中剝離出BpDataSource，因為需要跨binder傳輸。

//frameworks/av/media/libstagefright/InterfaceUtils.cpp
sp<IDataSource> CreateIDataSourceFromDataSource(const sp<DataSource> &source) {if (source == nullptr) {return nullptr;}return RemoteDataSource::wrap(source);
}//frameworks/av/media/libstagefright/include/media/stagefright/RemoteDataSource.h
static sp<IDataSource> wrap(const sp<DataSource> &source) {if (source.get() == nullptr) {return nullptr;}if (source->getIDataSource().get() != nullptr) {return source->getIDataSource();}return new RemoteDataSource(source);
}

來看看makeExtractor()方法：

//frameworks/av/services/mediaextractor/MediaExtractorService.cpp
::android::binder::Status MediaExtractorService::makeExtractor(const ::android::sp<::android::IDataSource>& remoteSource,const ::std::optional< ::std::string> &mime,::android::sp<::android::IMediaExtractor>* _aidl_return) {ALOGV("@@@ MediaExtractorService::makeExtractor for %s", mime ? mime->c_str() : nullptr);sp<DataSource> localSource = CreateDataSourceFromIDataSource(remoteSource);MediaBuffer::useSharedMemory();sp<IMediaExtractor> extractor = MediaExtractorFactory::CreateFromService(localSource,mime ? mime->c_str() : nullptr);ALOGV("extractor service created %p (%s)",extractor.get(),extractor == nullptr ? "" : extractor->name());if (extractor != nullptr) {registerMediaExtractor(extractor, localSource, mime ? mime->c_str() : nullptr);}*_aidl_return = extractor;return binder::Status::ok();
}

這里remoteSource經過binder已經處于extractor service端了，那已經是RemoteDataSource的本體了。在service端會通過CreateDataSourceFromIDataSource()將RemoteDataSource重新封裝成另一個TinyCacheSource對象。雖然這里和GenericSource端的TinyCacheSource是不同的東西，但其核心都是指向extractor service端的RemoteDataSource。

接下來就要開始真正創建extractor了。

//frameworks/av/media/libstagefright/MediaExtractorFactory.cpp
sp<IMediaExtractor> MediaExtractorFactory::CreateFromService(const sp<DataSource> &source, const char *mime) {ALOGV("MediaExtractorFactory::CreateFromService %s", mime);void *meta = nullptr;void *creator = NULL;FreeMetaFunc freeMeta = nullptr;float confidence;sp<ExtractorPlugin> plugin;uint32_t creatorVersion = 0;creator = sniff(source, &confidence, &meta, &freeMeta, plugin, &creatorVersion);if (!creator) {ALOGV("FAILED to autodetect media content.");return NULL;}MediaExtractor *ex = nullptr;if (creatorVersion == EXTRACTORDEF_VERSION_NDK_V1 ||creatorVersion == EXTRACTORDEF_VERSION_NDK_V2) {CMediaExtractor *ret = ((CreatorFunc)creator)(source->wrap(), meta);if (meta != nullptr && freeMeta != nullptr) {freeMeta(meta);}ex = ret != nullptr ? new MediaExtractorCUnwrapper(ret) : nullptr;}ALOGV("Created an extractor '%s' with confidence %.2f",ex != nullptr ? ex->name() : "<null>", confidence);return CreateIMediaExtractorFromMediaExtractor(ex, source, plugin);
}void *MediaExtractorFactory::sniff(const sp<DataSource> &source, float *confidence, void **meta,FreeMetaFunc *freeMeta, sp<ExtractorPlugin> &plugin, uint32_t *creatorVersion) {*confidence = 0.0f;*meta = nullptr;std::shared_ptr<std::list<sp<ExtractorPlugin>>> plugins;{Mutex::Autolock autoLock(gPluginMutex);if (!gPluginsRegistered) {return NULL;}plugins = gPlugins;}void *bestCreator = NULL;for (auto it = plugins->begin(); it != plugins->end(); ++it) {ALOGV("sniffing %s", (*it)->def.extractor_name);float newConfidence;void *newMeta = nullptr;FreeMetaFunc newFreeMeta = nullptr;void *curCreator = NULL;if ((*it)->def.def_version == EXTRACTORDEF_VERSION_NDK_V1) {curCreator = (void*) (*it)->def.u.v2.sniff(source->wrap(), &newConfidence, &newMeta, &newFreeMeta);} else if ((*it)->def.def_version == EXTRACTORDEF_VERSION_NDK_V2) {curCreator = (void*) (*it)->def.u.v3.sniff(source->wrap(), &newConfidence, &newMeta, &newFreeMeta);}if (curCreator) {if (newConfidence > *confidence) {*confidence = newConfidence;if (*meta != nullptr && *freeMeta != nullptr) {(*freeMeta)(*meta);}*meta = newMeta;*freeMeta = newFreeMeta;plugin = *it;bestCreator = curCreator;*creatorVersion = (*it)->def.def_version;} else {if (newMeta != nullptr && newFreeMeta != nullptr) {newFreeMeta(newMeta);}}}}return bestCreator;
}//frameworks/av/media/libstagefright/InterfaceUtils.cpp
sp<IMediaExtractor> CreateIMediaExtractorFromMediaExtractor(MediaExtractor *extractor,const sp<DataSource> &source,const sp<RefBase> &plugin) {if (extractor == nullptr) {return nullptr;}return RemoteMediaExtractor::wrap(extractor, source, plugin);
}

羅列下CreateFromService()做的事情：

• 調用自身的sniff()方法來依次遍歷注冊在系統內的gPlugins（ExtractorPlugin list），逐個調用每個extractor實現的sniff()來解析文件，成功解析則會返回一個confidence。然后再根據這個confidence來選取一個得分最高的extractor，本文則選取的是libmp4extractor。
• sniff()執行完，返回的是libmp4extractor的CreateExtractor函數指針。直接執行CreateExtractor()，這里會創建一個MPEG4Extractor并wrap成CMediaExtractor返回。
• CMediaExtractor進一步被wrap成MediaExtractorCUnwrapper對象。
• 為了能夠跨binder操作，又通過CreateIMediaExtractorFromMediaExtractor()將MediaExtractorCUnwrapper封裝成RemoteMediaExtractor對象。

看到這里，可以看出這個RemoteMediaExtractor已經和libmp4extractor中創建的MPEG4Extractor掛鉤了。

MPEG4Extractor關于sniff()和CreateExtractor()代碼這里就不貼了，代碼位置在frameworks/av/media/extractors/mp4/，大家自行查看。

extractor相關操作

上面的分析完，extractor已經創建了，接下來就是執行initFromDataSource()中的四個操作了：

• getMetaData()
• countTracks()
• getTrack()
• getTrackMetaData()

上述四個接口看名字都能大概知道是在做什么。四個接口都會調用到readMetaData()方法。

//frameworks/av/media/extractors/mp4/MPEG4Extractor.cpp
status_t MPEG4Extractor::readMetaData() {if (mInitCheck != NO_INIT) {return mInitCheck;}off64_t offset = 0;status_t err;bool sawMoovOrSidx = false;while (!((mHasMoovBox && sawMoovOrSidx && (mMdatFound || mMoofFound)) ||(mIsHeif && (mPreferHeif || !mHasMoovBox) &&(mItemTable != NULL) && mItemTable->isValid()))) {off64_t orig_offset = offset;err = parseChunk(&offset, 0);if (err != OK && err != UNKNOWN_ERROR) {break;} else if (offset <= orig_offset) {// only continue parsing if the offset was advanced,// otherwise we might end up in an infinite loopALOGE("did not advance: %lld->%lld", (long long)orig_offset, (long long)offset);err = ERROR_MALFORMED;break;} else if (err == UNKNOWN_ERROR) {sawMoovOrSidx = true;}}if ((mIsAvif || mIsHeif) && (mItemTable != NULL) && (mItemTable->countImages() > 0)) {//avif/heif圖片相關處理，省略}if (mInitCheck == OK) {if (findTrackByMimePrefix("video/") != NULL) {AMediaFormat_setString(mFileMetaData,AMEDIAFORMAT_KEY_MIME, MEDIA_MIMETYPE_CONTAINER_MPEG4);} else if (findTrackByMimePrefix("audio/") != NULL) {AMediaFormat_setString(mFileMetaData,AMEDIAFORMAT_KEY_MIME, "audio/mp4");} else if (findTrackByMimePrefix(MEDIA_MIMETYPE_IMAGE_ANDROID_HEIC) != NULL) {AMediaFormat_setString(mFileMetaData,AMEDIAFORMAT_KEY_MIME, MEDIA_MIMETYPE_CONTAINER_HEIF);} else if (findTrackByMimePrefix(MEDIA_MIMETYPE_IMAGE_AVIF) != NULL) {AMediaFormat_setString(mFileMetaData,AMEDIAFORMAT_KEY_MIME, MEDIA_MIMETYPE_IMAGE_AVIF);} else {AMediaFormat_setString(mFileMetaData,AMEDIAFORMAT_KEY_MIME, "application/octet-stream");}} else {mInitCheck = err;}CHECK_NE(err, (status_t)NO_INIT);// copy pssh data into file metadata//pssh DRM解密相關處理，省略return mInitCheck;
}

這里的主體內容就是那個while循環以及循環內的parseChunk()函數。這個parseChunk()的命名感覺不太合適，個人覺得改成parseBox()更好，不容易引起初學者的誤解（我剛學的時候乍一看以為是media data中的chunk概念）。

parseChunk()方法很長，這里就不貼了，簡單解釋以下它的功能：

它是一個遞歸函數，在外層while循環里會從MP4文件的開頭開始啟動parseChunk()函數去依次解析文件中的每個box，如果這個box是一個container box，那么它就會去遞歸的解析下一級的box直到沒有更下一級的box為止。解析出來的信息會保存到MPEG4Extractor的變量中。

說一句題外話，大家學習的時候如果能下載到對應視頻格式解析軟件，最好還是下載一個。我這里用的是“MP4 Inspector”軟件。實際做extractor開發和維護工作還是需要諸多的spec來支撐的。

用這個軟件打開我用的MP4文件的信息可以很清晰的看到如下內容：

返回正文，parseChunk()方法讀取文件的功能則是通過mDataSource->readAt()來做到的，實際就是調用上文中創建的FileSource去讀取。

第二部分到這里就分析結束了。AnotherPacketSource的內容在本節暫不展開了，等后續學習完了在其他章節解讀。

在開始講解第三部分之前，簡單提一下notifyVideoSizeChanged()和notifyFlagsChanged()這兩個方法。

• notifyVideoSizeChanged()是將從視頻文件中讀取到的video的width和height通知到NuPlayer中去。
• notifyFlagsChanged()是將FLAG_CAN_PAUSE/FLAG_CAN_SEEK_BACKWARD/FLAG_CAN_SEEK_FORWARD/FLAG_CAN_SEEK這四個flags通知到NuPlayer中去并保存到mPlayerFlags中。在java層會調用getMetadata()接口時在NuPlayer中會根據mPlayerFlags構造成一個Metadata返回。

finishPrepareAsync()

//frameworks/av/media/libmediaplayerservice/nuplayer/GenericSource.cpp
void NuPlayer::GenericSource::finishPrepareAsync() {ALOGV("finishPrepareAsync");status_t err = startSources();if (err != OK) {ALOGE("Failed to init start data source!");notifyPreparedAndCleanup(err);return;}if (mIsStreaming) {mCachedSource->resumeFetchingIfNecessary();mPreparing = true;schedulePollBuffering();} else {notifyPrepared();}if (mAudioTrack.mSource != NULL) {postReadBuffer(MEDIA_TRACK_TYPE_AUDIO);}if (mVideoTrack.mSource != NULL) {postReadBuffer(MEDIA_TRACK_TYPE_VIDEO);}
}status_t NuPlayer::GenericSource::startSources() {// Start the selected A/V tracks now before we start buffering.// Widevine sources might re-initialize crypto when starting, if we delay// this to start(), all data buffered during prepare would be wasted.// (We don't actually start reading until start().)//// TODO: this logic may no longer be relevant after the removal of widevine// supportif (mAudioTrack.mSource != NULL && mAudioTrack.mSource->start() != OK) {ALOGE("failed to start audio track!");return UNKNOWN_ERROR;}if (mVideoTrack.mSource != NULL && mVideoTrack.mSource->start() != OK) {ALOGE("failed to start video track!");return UNKNOWN_ERROR;}return OK;
}

這里主要關注兩個函數：startSources()和postReadBuffer()。由于篇幅原因，不再展開code。直接文字簡要描述他倆的功能：

• startSources()：看名字是start，其實還沒有start起來。這里主要是在分配、創建MediaBuffer并加入管理。
• postReadBuffer()：這個才是真正開始從視頻文件中讀取media data的地方。

總結

onPrepareAsync() 函數到這里結束，主要內容基本都過了一遍，暫時還缺少了MediaBuffer的部分沒有涉及到。下面還是老規矩，以圖的方式總結下本節的內容：

圖一?onPrepareAsync()執行流程

圖二 MP4 extractor關系架構圖

看代碼感覺還沒那么強烈，但是從圖二的架構圖來看，就可以看出設計NuPlayer這個架構的架構師太牛了。圖中綠色方框框起來的是MP4 extractor自己實現的內容，其他extractor也是按照這種方式去替換方框中的實現即可。這種plugin的設計模式太溜了。

圖三 mp4常見組成box示意圖

圖三是我簡單查看spec稍微畫的一個示意圖，只畫了常見的一些內容，并不專業和正確，只是方便我自己回顧。