Double/Debiased Machine Learning

獨立同步分布的觀測數據 $\{W_i=(Y_i,D_i,X_i)| i\in \{1,...,n\}\}$ ，其中 $Y_i$ 表示結果變量， $D_i$ 表示因變量， $X_i$ 表示控制變量。

目標參數 $\theta_0$ 的一般定義形式為：

$E[m(W;\theta_0,\eta_0)] = 0$

$W$ 為觀測到的變量， $\theta_0\in \Theta$ 為目標參數， $\eta_0\in \mathcal{T}$ 為輔助參數

例如，ATE 的定義為：

$\theta_0^{ATE}\equiv E[E[Y_i|D_i=1,X_i] - E[Y_i|D_i=0,X_i]]$

ATE的IPW估計定義為：

$m_{IPW}(W_i;\theta,\alpha)\equiv \alpha(D_i,X_i)Y_i - \theta \equiv [\frac{D_i}{E[D_i|X_i]} - \frac{1-D_i}{1-E[D_i|X_i]}]Y_i - \theta$

ATE的Doubly Robust估計的定義為：

$m_{DR}(W_i;\theta,\eta)\equiv \alpha(D_i,X_i)(Y_i - E[Y_i|D_i,X_i])Y_i + E[Y_i|D_i=1,X_i]- E[Y_i|D_i=0,X_i]-\theta$

$\equiv [\frac{D_i}{E[D_i|X_i]} - \frac{1-D_i}{1-E[D_i|X_i]}] Y_i + E[Y_i|D_i=1,X_i]- E[Y_i|D_i=0,X_i]-\theta$

一般情況下，目標參數 $\theta_0$ 的估計值定義為：

$\hat{\theta}:\frac{1}{n}\sum_{i=1}^nm(W_i;\hat{\theta},\hat{\eta}) = 0$

一階泰勒展得出：

$\frac{1}{n}\sum_{i=1}^nm(W_i;\hat{\theta},\hat{\eta}) \approx \frac{1}{n}\sum_{i=1}^nm(W_i;\theta_0,\eta_0) + \frac{1}{n}\sum_{i=1}^n\frac{\partial}{\partial\theta}m(W_i;\theta_0,\eta_0)(\hat{\theta} - \theta_0) + \frac{1}{n}\sum_{i=1}^n\frac{\partial}{\partial\eta}m(W_i;\theta_0,\eta_0)(\hat{\eta} - \eta_0) \approx 0$

$(\theta_0 - \hat{\theta})\approx [\frac{1}{n}\sum_{i=1}^n\frac{\partial}{\partial\theta}m(W_i;\theta_0,\eta_0)]^{-1}\frac{1}{n}\sum_{i=1}^nm(W_i;\theta_0,\eta_0) + [\frac{1}{n}\sum_{i=1}^n\frac{\partial}{\partial\theta}m(W_i;\theta_0,\eta_0)]^{-1}(\hat{\eta} - \eta_0)\frac{1}{n}\sum_{i=1}^n\frac{\partial}{\partial\eta}m(W_i;\theta_0,\eta_0)$

目標參數的估計偏差 $(\theta_0 - \hat{\theta})$ 將受到輔助參數估計偏差 $(\hat{\eta} - \eta_0)$ 的影響，說明目標參數的估計偏差的兩種來源分別是：

輔助參數的估計偏差 $(\hat{\eta} - \eta_0)$ 本身，稱之為正則化偏差
輔助參數的估計偏差 $(\hat{\eta} - \eta_0)$ 與 $W_i$ 的強相關性，稱之為過擬合偏差

Neyman Orthogonality

$\frac{\partial}{\partial\lambda}\{E[\psi(W_i;\theta_0,\eta_0 + \lambda(\eta-\eta_0))]\}|_{\lambda=0}= 0,\forall\eta\in \mathcal{T}$

$m_{IPW}$ is not Neyman orthogonal, $m_{DR}$ is Neyman orthogonal.

Cross Fitting

$\hat{\theta}:\frac{1}{n}\sum_{k=1}^K\sum_{i\in I_k}m(W_i;\hat{\theta},\hat{\eta}_{-k}) = 0$

DML

$\hat{\theta}:\frac{1}{n}\sum_{k=1}^K\sum_{i\in I_k}\psi(W_i;\hat{\theta},\hat{\eta}_{-k}) = 0$

直接回歸不滿足 Neyman 正交性

$\theta T + g(X) + \epsilon$

$m(W;\theta,g) = Y - \theta T - g(X) + \epsilon$

$\frac{\partial }{\partial \lambda}E[m(w;\theta,g + \lambda\Delta g)]|_{\lambda=0} = E[-\Delta g(x)] \ne 0$

DML 滿足Neyman正交性

$\theta (T - m(x)) + \epsilon',l(x) = E[Y|X=x],m(x)=E[T|X=x]$

$m(W;\theta,\eta) = Y-l(x) - \theta (T - m(x)) - \epsilon',\eta = (l, m)$

$\frac{\partial}{\partial\lambda}E[W;\theta,\eta + \lambda\Delta\eta]|_{\lambda=0} = E[-\Delta l(x) + \theta\Delta m(x)] = 0$

Example

模擬數據

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import math
import dowhy.datasets, dowhy.plotter
rvar = 1 if np.random.uniform() > 0.2 else 0
is_linear = False # A non-linear dataset. Change to True to see results for a linear dataset.
data_dict = dowhy.datasets.xy_dataset(10000, effect=rvar,num_common_causes=2,is_linear=is_linear,sd_error=0.2)
df = data_dict['df']
print(df.head())
dowhy.plotter.plot_treatment_outcome(df[data_dict["treatment_name"]], df[data_dict["outcome_name"]],df[data_dict["time_val"]])

請添加圖片描述

因果關系假設：

基于領域知識提出因果關系的假設，定義模型結構

from dowhy import CausalModel
model= CausalModel(data=df,treatment=data_dict["treatment_name"],outcome=data_dict["outcome_name"],common_causes=data_dict["common_causes_names"],instruments=data_dict["instrument_names"])
model.view_model(layout="dot")

請添加圖片描述

因果關系識別：

identified_estimand = model.identify_effect(proceed_when_unidentifiable=True)
print(identified_estimand)

因果關系估計：

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LassoCV
from sklearn.ensemble import GradientBoostingRegressor
dml_estimate = model.estimate_effect(identified_estimand, method_name="backdoor.econml.dml.DML",control_value = 0,treatment_value = 1,confidence_intervals=False,method_params={"init_params":{'model_y':GradientBoostingRegressor(),'model_t': GradientBoostingRegressor(),"model_final":LassoCV(fit_intercept=False),'featurizer':PolynomialFeatures(degree=2, include_bias=True)},"fit_params":{}})
print(dml_estimate)

因果關系反駁測試：

res_placebo=model.refute_estimate(identified_estimand, dml_estimate,method_name="placebo_treatment_refuter", placebo_type="permute",num_simulations=20)
print(res_placebo)

本文來自互聯網用戶投稿，該文觀點僅代表作者本人，不代表本站立場。本站僅提供信息存儲空間服務，不擁有所有權，不承擔相關法律責任。
如若轉載，請注明出處：http://www.pswp.cn/bicheng/83913.shtml
繁體地址，請注明出處：http://hk.pswp.cn/bicheng/83913.shtml
英文地址，請注明出處：http://en.pswp.cn/bicheng/83913.shtml

如若內容造成侵權/違法違規/事實不符，請聯系多彩編程網進行投訴反饋email:809451989@qq.com，一經查實，立即刪除！