对抗验证——训练测试集特征分布是否一致

一、目的：
探查训练集和测试集的数据(特征)分布是否一致，

二、原理：
将训练数据对应的label设为1，将测试数据的label设为0，如果模型分类效果很好，说明训练集和测试集的特征分布差异较大，说明本次比赛难度会有点大，如果模型效果差，说明训练集和测试集分布差异不大；

## 对抗验证
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
# from tqdm.notebook import tqdm

train = df[df['是否流失'].notnull()].reset_index(drop = True)
test = df[~df['是否流失'].notnull()].reset_index(drop = True)
features = [i  for i in train.columns.tolist() if i not in ['客户ID', '是否流失',]]

X_train = train.copy()
X_test = test.copy()

X_train['是否流失'] = 1
X_test['是否流失'] = 0
train_test = pd.concat([X_train, X_test]).reset_index(drop = True)
train1, test1 = train_test_split(train_test, test_size=0.3, random_state=2022, shuffle=True)
train_y = train1['是否流失'].values
test_y = test1['是否流失'].values
del train1['是否流失'], test1['是否流失']

adversarial_result = pd.DataFrame(index=train1.columns,columns=['roc'])
for i in tqdm(features):
    clf = lgb.LGBMClassifier(
        random_state=2021,
        max_depth=4,
        metric='auc',
        n_estimators=1000,
    )
    clf.fit(
        np.array(train1[i]).reshape(-1,1),
        train_y,
        eval_set=[(np.array(test1[i]).reshape(-1,1),test_y)],
        early_stopping_rounds=200,
        verbose=0)
    temp_pred = clf.predict_proba(np.array(test1[i]).reshape(-1,1))[:,1]
    roc1 = roc_auc_score(test_y,temp_pred)
    adversarial_result.loc[i,'roc'] = roc1

a = adversarial_result.sort_values('roc')

这个就是一个分布不一致的例子
roc
15_22 0.536254
sceneref 0.541208
sceneswitch 0.551622
subgroup 0.565108
backnet 0.57079
datasync 0.572246
webdevice 0.573115
23_6 0.574551
scene 0.580132
delfail 0.580294
offxiaoda 0.583549
normal 0.595378
oauth2 0.596591
phonespeech 0.597359
timing 0.602798
7_14 0.605467
colour_dataJ 0.799984
get_song 0.801399
switch_ledBfalse 0.801763
is_key_lock0 0.802107
power32 0.802835
power21 0.803138
start_music0 0.80334
switch_ledBtrue 0.804109
start_music1 0.808921
refresh_list3 0.809204
power22 0.809629
power11 0.810882
power31 0.811934
sendcontrol1 0.81248
work12 0.81345
power12 0.813471
work11 0.814219
uid NaN