【UE5醫學影像可視化】讀取dicom數據生成2D紋理并顯示

文章目錄

- 1.實現目標
- 2.實現過程
- - 2.1 數據準備
  - 2.2 創建項目
  - 2.3 dcmtk庫集成
  - 2.4 流程&原理
  - 2.5 材質
  - 2.6 應用實現
- 3.參考資料

1.實現目標

本文在UE5中讀取本地的dicom文件，解析像素值、窗寬窗位等信息，生成2D紋理，在UE場景中實現簡單的2D醫學影像可視化

在這里插入圖片描述

2.實現過程

包括數據準備，dicom三方庫在UE工程中的集成，dicom文件中像素值，窗寬窗位Tag的解析，紋理的生成，顯示處理的材質等，以實現最終在UE場景中顯示2D醫學影像

2.1 數據準備

（1）基于開源的dicom數據，鏈接在文章第三部分參考資料中

在這里插入圖片描述

（2）在Radiant Viewer中選擇一張dicom查看，如下圖所示，也是此次本地使用的單張單幀dicom測試數據

在這里插入圖片描述

2.2 創建項目

創建引擎自帶的 C++項目，這里不再贅述

在這里插入圖片描述

2.3 dcmtk庫集成

由于gdcm庫在集成過程中，有較多的沖突需要解決，為了集成方便起見，所以本文這里直接對dcmtk庫進行集成。（直接使用github上編譯好的庫，也可以下載源碼自己本地編譯）

（1）以插件的形式集成三方庫，這也是目前UE官方所推薦的方式。這里使用的是空白的插件模板。

在這里插入圖片描述

（2）插件目錄，其中后續使用的usf等shader相關文件，都放在Shaders文件夾中。
第三方庫dcmtk的相關內容，都放在插件source中的ThirdParty文件夾下

在這里插入圖片描述

（3）下載dcmtk庫，直接從github上下載dcmtk的release版本（https://github.com/DCMTK/dcmtk/releases）
其中包含了需要的頭文件，以及靜態庫lib和動態庫dll等。（也可以下載源碼，自己本地編譯）

在這里插入圖片描述

（4）在插件的Build.cs中配置對三方庫的引用，防止找不到頭文件或者庫等報錯

在這里插入圖片描述

插件的build.cs完整代碼如下所示：

// Copyright Epic Games, Inc. All Rights Reserved.using System.IO;
using UnrealBuildTool;public class DicomVIS : ModuleRules
{public DicomVIS(ReadOnlyTargetRules Target) : base(Target){PCHUsage = ModuleRules.PCHUsageMode.UseExplicitOrSharedPCHs;PublicIncludePaths.AddRange(new string[] {// ... add public include paths required here ...});PrivateIncludePaths.AddRange(new string[] {// ... add other private include paths required here ...});PublicDependencyModuleNames.AddRange(new string[]{"Core","ImageWrapper"// ... add other public dependencies that you statically link with here ...});PrivateDependencyModuleNames.AddRange(new string[]{"CoreUObject","Engine","Slate","SlateCore",// for custom shaders"RenderCore"});DynamicallyLoadedModuleNames.AddRange(new string[]{// ... add any modules that your module loads dynamically here ...});PublicIncludePaths.AddRange(new string[] { Path.Combine(ModuleDirectory, "ThirdParty/dcmtk/include")	});string libPath = Path.Combine(ModuleDirectory, "ThirdParty/dcmtk/lib");string[] allLibs = Directory.Exists(libPath) ? Directory.GetFiles(libPath, "*.lib") : new string[0];PublicAdditionalLibraries.AddRange(allLibs);string dllPath = Path.Combine(ModuleDirectory, "ThirdParty/dcmtk/bin");string[] allDlls = Directory.Exists(dllPath) ? Directory.GetFiles(dllPath, "*.dll") : new string[0];foreach (string currentDll in allDlls){string dllName = Path.GetFileName(currentDll);PublicDelayLoadDLLs.Add(dllName);// copy dll to project directory binaries dirstring targetDllPath = Path.Combine(PluginDirectory, "../../Binaries/Win64/" + dllName);RuntimeDependencies.Add(targetDllPath, currentDll);}}
}

（5）集成過程中可能會遇到的問題

① 引入dcmtk庫相關頭文件時，verify、check等宏沖突

報錯如下所示：

在這里插入圖片描述

解決方法：

// DCMTK uses their own verify and check macros.
// Also, they include some effed up windows headers which for example include min and max macros for that
// extra bit of _screw you_
#pragma push_macro("verify")
#pragma push_macro("check")
#undef verify
#undef check#include "dcmtk/dcmdata/dcdatset.h"
#include "dcmtk/dcmdata/dcdeftag.h"
#include "dcmtk/dcmdata/dcfilefo.h"
#include "dcmtk/dcmdata/dcpixel.h"
#include "dcmtk/dcmimgle/dcmimage.h"#pragma pop_macro("verify")
#pragma pop_macro("check")

②"UpdateResourceW": 不是 "UTexture2D" 的成員，原因是dcmtk中引用了windows的相關頭文件WinBase.h中已經定義了相關的宏

在這里插入圖片描述

報錯如下所示：

在這里插入圖片描述

解決方法：前置聲明

在這里插入圖片描述

2.4 流程&原理

使用dcmtk庫解析dicom數據中的像素數據，以及窗寬窗位等數據，再按照dicom標準的顯示處理流程處理即可。
（完整源碼在本文2.6部分，此部分僅作為原理說明）

（1）使用dcmtk庫解析dicom數據中的像素值，和窗寬窗位tag信息，這三個信息是顯示的必要參數

直接根據相關tag解析即可，本文這里使用的無符號的16bit測試數據，其他類型暫無考慮（原理相同，參數修改下即可）

在這里插入圖片描述

（2）窗寬窗位特殊處理

在上一篇文章中介紹了常規的dicom數據顯示流程，即先根據slope和intercept進行modality變換，在根據windowCenter和windowWidth等進行VOI變換。但本文這里是直接將沒有經過modality變換的原始像素值直接生成了2D紋理，后續的顯示處理都在shader中處理，所以這里直接對窗寬窗位應用modality反變換，這樣就在shader中節約了一次modality變換的消耗

在這里插入圖片描述

（3）創建紋理并更新像素數據

在這里插入圖片描述

（4）最終顯示：使用材質處理完成后，在Plane等StaticMesh的載體上顯示即可

2.5 材質

關于dicom顯示過程中的一些特殊處理，如VOI變換等，可以使用Material Function等方式，為了后續使用的方便，本文這里使用usf的shader方式實現

（1）在插件中添加對shader文件夾的映射，以便可以在材質編輯器中使用。

// find shaders
FString shaderDir = FPaths::Combine(FPaths::ProjectPluginsDir(), "DicomVIS/Shaders/");
AddShaderSourceDirectoryMapping("/DicomVIS", shaderDir);

在這里插入圖片描述

（2）在UE編輯器中創建材質，并設置為Unlit無光照模式，以避免UE中光照對dicom醫學影像造成的顏色偏差等

在這里插入圖片描述

（3）該材質的主要節點如下所示。其中SliceTex參數為dicom像素值生成的2D紋理，WindowWidth和WindowCenter為窗寬窗位信息，DataRange為數據的范圍，如本文使用的G16格式紋理，則該值為65535；若材質為G8格式的紋理，則該值為255

在這里插入圖片描述

（4）新建usf，用于dicom顯示過程中的VOI變換

float GetVOITransformation(float inputValue, float windowCenter, float windowWidth, float range)
{return saturate((inputValue - windowCenter + 0.5 / range) / (windowWidth - 1.0 / range) + 0.5);
}

在這里插入圖片描述

（5）新建自定義材質節點，并設置輸入和輸出的參數類型

在這里插入圖片描述

（6）設置自定義材質節點的Code和包含頭文件路徑，以便可以在材質中使用usf中的函數

在這里插入圖片描述

2.6 應用實現

包括具體的實現步驟，可以在UE中Editor環境下加載Dicom數據，生成2D紋理和最終顯示。
本文這里只演示了16位無符號的dicom數據處理流程，其他類型的類似，需要修改生成材質的格式等參數即可

（1）dicom解析，以及生成紋理的相關功能，本文這里基于ActorCompoent實現，具體C++代碼如下所示：

SliceDataLoader.h:

// Fill out your copyright notice in the Description page of Project Settings.#pragma once#include "CoreMinimal.h"
#include "Components/ActorComponent.h"
#include "Engine/StaticMeshActor.h"
// DCMTK uses their own verify and check macros.
// Also, they include some effed up windows headers which for example include min and max macros for that
// extra bit of _screw you_
#pragma push_macro("verify")
#pragma push_macro("check")
#undef verify
#undef check#include "dcmtk/dcmdata/dcdatset.h"
#include "dcmtk/dcmdata/dcdeftag.h"
#include "dcmtk/dcmdata/dcfilefo.h"
#include "dcmtk/dcmdata/dcpixel.h"
#include "dcmtk/dcmimgle/dcmimage.h"#pragma pop_macro("verify")
#pragma pop_macro("check")#include "SliceDataLoader.generated.h"#define UpdateResource UpdateResourceUCLASS( ClassGroup=(Custom), meta=(BlueprintSpawnableComponent) )
class DICOMVIS_API USliceDataLoader : public UActorComponent
{GENERATED_BODY()public:	// Sets default values for this component's propertiesUSliceDataLoader();// The dicom data path, that is the relative path of ue game project directory.UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")FString TargetFilePath = "Data/test.dcm";// Plane static mesh componentUPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")AStaticMeshActor* pPlaneMesh = nullptr;// 2d slice materialUPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")UMaterial* pMaterial = nullptr;// The dicom pixel data texture2DUPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")UTexture2D* pTexture2D;// Window Center of dicomUPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")float WindowCenter;// Window width of dicomUPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Dicom")float WindowWidth;// The min value of dicom dataUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")float Min = 0;// The range of dicom data, range = max - minUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")float Range = 1;// slope value of dicomUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")float Slope = 0;// intercept value of dicomUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")float Intercept = 0;// dicom image widthUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")int Width = 0;// dicom image heightUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")int Height = 0;// dicom image depthUPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Dicom")int Depth = 0;// Load dicom data from load file path, parse pixel data to texture2DUFUNCTION(CallInEditor, BlueprintCallable, Category = "Dicom")void LoadDicom();protected:// Called when the game startsvirtual void BeginPlay() override;// use dicom pixel data to generate texture (uint16)void GenerateTexture(UTexture2D*& pTexture, uint32 width, uint32 height, uint16* pixelData);public:	// Called every framevirtual void TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction) override;
};

SliceDataLoader.cpp：

// Fill out your copyright notice in the Description page of Project Settings.#include "SliceDataLoader.h"
#include <Kismet/KismetSystemLibrary.h>
#include <ImageUtils.h>// Sets default values for this component's properties
USliceDataLoader::USliceDataLoader()
{// Set this component to be initialized when the game starts, and to be ticked every frame.  You can turn these features// off to improve performance if you don't need them.PrimaryComponentTick.bCanEverTick = true;// ...
}void USliceDataLoader::LoadDicom()
{FString dicomFilePath = UKismetSystemLibrary::GetProjectDirectory() + TargetFilePath;if (!FPaths::FileExists(dicomFilePath)){UE_LOG(LogTemp, Error, TEXT("dicom file is not exist, please check!"));return;}// ToDo: use other thread to processAsyncTask(ENamedThreads::GameThread, [=, this]() {DcmFileFormat fileFormat;if (fileFormat.loadFile(TCHAR_TO_ANSI(*dicomFilePath)).good()){UE_LOG(LogTemp, Log, TEXT("dicom file loaded successfully!"));DcmDataset* dataset = fileFormat.getDataset();dataset->chooseRepresentation(EXS_LittleEndianImplicit, nullptr);Float64 windowCenter;dataset->findAndGetFloat64(DCM_WindowCenter, windowCenter);this->WindowCenter = windowCenter;Float64 windowWidth;dataset->findAndGetFloat64(DCM_WindowWidth, windowWidth);this->WindowWidth = windowWidth;Float64 slope;dataset->findAndGetFloat64(DCM_RescaleSlope, slope);this->Slope = slope;Float64 intercept;dataset->findAndGetFloat64(DCM_RescaleIntercept, intercept);this->Intercept = intercept;Uint16 bitsAllocated;dataset->findAndGetUint16(DCM_BitsAllocated, bitsAllocated);this->Depth = bitsAllocated;Uint8 pixelRepresentation = 0;dataset->findAndGetUint8(DCM_PixelRepresentation, pixelRepresentation);bool isSigned = pixelRepresentation == 1;DicomImage* dcmImage = new DicomImage(TCHAR_TO_ANSI(*dicomFilePath));if (dcmImage->getStatus() != EIS_Normal){UE_LOG(LogTemp, Error, TEXT("dicom file image loaded failed!"));return;}const int width = dcmImage->getWidth();this->Width = width;const int height = dcmImage->getHeight();this->Height = height;DcmElement* pixelDataElement;dataset->findAndGetElement(DCM_PixelData, pixelDataElement);// Tips：current just support r16 formatif (bitsAllocated == 16){TArray<int> resArray;if (!isSigned){uint16* pixelData = nullptr;pixelDataElement->getUint16Array(pixelData);this->GenerateTexture(this->pTexture2D, this->Width, this->Height, pixelData);}}}});
}// Called when the game starts
void USliceDataLoader::BeginPlay()
{Super::BeginPlay();// ...}void USliceDataLoader::GenerateTexture(UTexture2D*& pTexture, uint32 width, uint32 height, uint16* pixelData)
{//FImageUtils::CreateTexture2DEPixelFormat pixelFormat = EPixelFormat::PF_G16;if (pTexture == nullptr){pTexture = UTexture2D::CreateTransient(width, height, pixelFormat);pTexture->AddToRoot();pTexture->MipGenSettings = TMGS_NoMipmaps;pTexture->CompressionSettings = TC_Grayscale;// srgb may not effect of 16 bit.pTexture->SRGB = true;pTexture->NeverStream = true;pTexture->Filter = TextureFilter::TF_Nearest;pTexture->AddressX = TextureAddress::TA_Clamp;pTexture->AddressY = TextureAddress::TA_Clamp;}FTexture2DMipMap& mipMap = pTexture->GetPlatformData()->Mips[0];uint16* byteArray = static_cast<uint16*>(mipMap.BulkData.Lock(LOCK_READ_WRITE));long size = width * height;FMemory::Memcpy(byteArray, pixelData, size * sizeof(uint16));mipMap.BulkData.Unlock();pTexture->UpdateResource();// 更新材質if (pMaterial){UMaterialInstanceDynamic* pMaterialInsDy = UMaterialInstanceDynamic::Create(pMaterial, this);pMaterialInsDy->SetTextureParameterValue(FName("SliceTex"), pTexture);// inverset tranform window center and width by slope and intercept;FFloat16 transWL = (this->WindowCenter - this->Intercept) / this->Slope * 1 / 65535.0;FFloat16 transWW = (this->WindowWidth) / this->Slope * 1 / 65535.0;pMaterialInsDy->SetScalarParameterValue(FName("WindowCenter"), transWL);pMaterialInsDy->SetScalarParameterValue(FName("WindowWidth"), transWW);pMaterialInsDy->SetScalarParameterValue(FName("DataRange"), 65535.0);auto pStaticMeshComponent = pPlaneMesh->GetStaticMeshComponent();pStaticMeshComponent->SetMaterial(0, pMaterialInsDy);}
}// Called every frame
void USliceDataLoader::TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction)
{Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
}