数据来源：Kaggle数据集 → 共有1309名乘客数据，其中891是已知存活情况（train.csv），剩下418则是需要进行分析预测的（test.csv）
字段意义：
PassengerId: 乘客编号
Survived :存活情况（存活：1 ; 死亡：0）
Pclass : 客舱等级
Name : 乘客姓名
Sex : 性别
Age : 年龄
SibSp : 同乘的兄弟姐妹/配偶数
Parch : 同乘的父母/小孩数
Ticket : 船票编号
Fare : 船票价格
Cabin :客舱号
Embarked : 登船港口
目的：通过已知获救数据，预测乘客生存情况
import pandas as pd import numpy as np import matplotlib.pyplot as plt
titanic_train = pd.read_csv("titanic_train.csv")
titanic_train.head(5)
| PassengerId | Survived | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 3 | Braund, Mr. Owen Harris | male | 22.0 | 1 | 0 | A/5 21171 | 7.2500 | NaN | S |
| 1 | 2 | 1 | 1 | Cumings, Mrs. John Bradley (Florence Briggs Th... | female | 38.0 | 1 | 0 | PC 17599 | 71.2833 | C85 | C |
| 2 | 3 | 1 | 3 | Heikkinen, Miss. Laina | female | 26.0 | 0 | 0 | STON/O2. 3101282 | 7.9250 | NaN | S |
| 3 | 4 | 1 | 1 | Futrelle, Mrs. Jacques Heath (Lily May Peel) | female | 35.0 | 1 | 0 | 113803 | 53.1000 | C123 | S |
| 4 | 5 | 0 | 3 | Allen, Mr. William Henry | male | 35.0 | 0 | 0 | 373450 | 8.0500 | NaN | S |
titanic_test = pd.read_csv("test.csv")
titanic_test.head(5)
| PassengerId | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 892 | 3 | Kelly, Mr. James | male | 34.5 | 0 | 0 | 330911 | 7.8292 | NaN | Q |
| 1 | 893 | 3 | Wilkes, Mrs. James (Ellen Needs) | female | 47.0 | 1 | 0 | 363272 | 7.0000 | NaN | S |
| 2 | 894 | 2 | Myles, Mr. Thomas Francis | male | 62.0 | 0 | 0 | 240276 | 9.6875 | NaN | Q |
| 3 | 895 | 3 | Wirz, Mr. Albert | male | 27.0 | 0 | 0 | 315154 | 8.6625 | NaN | S |
| 4 | 896 | 3 | Hirvonen, Mrs. Alexander (Helga E Lindqvist) | female | 22.0 | 1 | 1 | 3101298 | 12.2875 | NaN | S |
print(titanic_train.describe()) print(titanic_train.info())
PassengerId Survived Pclass Age SibSp
count 891.000000 891.000000 891.000000 714.000000 891.000000
mean 446.000000 0.383838 2.308642 29.699118 0.523008
std 257.353842 0.486592 0.836071 14.526497 1.102743
min 1.000000 0.000000 1.000000 0.420000 0.000000
25% 223.500000 0.000000 2.000000 20.125000 0.000000
50% 446.000000 0.000000 3.000000 28.000000 0.000000
75% 668.500000 1.000000 3.000000 38.000000 1.000000
max 891.000000 1.000000 3.000000 80.000000 8.000000
Parch Fare
count 891.000000 891.000000
mean 0.381594 32.204208
std 0.806057 49.693429
min 0.000000 0.000000
25% 0.000000 7.910400
50% 0.000000 14.454200
75% 0.000000 31.000000
max 6.000000 512.329200
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 PassengerId 891 non-null int64
1 Survived 891 non-null int64
2 Pclass 891 non-null int64
3 Name 891 non-null object
4 Sex 891 non-null object
5 Age 714 non-null float64
6 SibSp 891 non-null int64
7 Parch 891 non-null int64
8 Ticket 891 non-null object
9 Fare 891 non-null float64
10 Cabin 204 non-null object
11 Embarked 889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.7+ KB
None
print(titanic_test.describe()) print(titanic_test.info())
PassengerId Pclass Age SibSp Parch Fare count 418.000000 418.000000 332.000000 418.000000 418.000000 417.000000 mean 1100.500000 2.265550 30.272590 0.447368 0.392344 35.627188 std 120.810458 0.841838 14.181209 0.896760 0.981429 55.907576 min 892.000000 1.000000 0.170000 0.000000 0.000000 0.000000 25% 996.250000 1.000000 21.000000 0.000000 0.000000 7.895800 50% 1100.500000 3.000000 27.000000 0.000000 0.000000 14.454200 75% 1204.750000 3.000000 39.000000 1.000000 0.000000 31.500000 max 1309.000000 3.000000 76.000000 8.000000 9.000000 512.329200RangeIndex: 418 entries, 0 to 417 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 PassengerId 418 non-null int64 1 Pclass 418 non-null int64 2 Name 418 non-null object 3 Sex 418 non-null object 4 Age 332 non-null float64 5 SibSp 418 non-null int64 6 Parch 418 non-null int64 7 Ticket 418 non-null object 8 Fare 417 non-null float64 9 Cabin 91 non-null object 10 Embarked 418 non-null object dtypes: float64(2), int64(4), object(5) memory usage: 36.0+ KB None
train数据集：
- Age只有714，而不是891，说明存在缺失值
- cabin存在缺失值
- embarked存在缺失值，889<891
test数据集：
- age、cabin、fare缺失
#绘制单个属性分布图来进行探测
#coding:utf-8
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif']=['SimHei'] #用来正常显示中文标签
plt.rcParams['axes.unicode_minus']=False #用来正常显示负号
#存活人数
print(titanic_train['Survived'].value_counts())
bar_sur=titanic_train['Survived'].value_counts()
bar_sur.plot.bar(color='r')
plt.xlabel('存活类别（生/死）')
plt.ylabel('人数')
plt.show()
#客舱等级
print(titanic_train['Pclass'].value_counts())
bar_pclass=titanic_train['Pclass'].value_counts()
bar_pclass.plot.bar(color='y')
plt.xlabel('客舱等级')
plt.ylabel('人数')
plt.show()
#性别
print(titanic_train['Sex'].value_counts())
bar_sex=titanic_train['Sex'].value_counts()
bar_sex.plot.bar(color='b')
plt.xlabel('性别')
plt.ylabel('人数')
plt.show()
#同乘的兄弟姐妹/配偶数
print(titanic_train['SibSp'].value_counts())
bar_SibSp=titanic_train['SibSp'].value_counts()
bar_SibSp.plot.bar(color='black')
plt.xlabel('同乘的兄弟姐妹/配偶数')
plt.ylabel('人数')
plt.show()
#同乘的父母/小孩数
print(titanic_train['Parch'].value_counts())
bar_parch=titanic_train['Parch'].value_counts()
bar_parch.plot.bar(color='g')
plt.xlabel('同乘的父母/小孩数')
plt.ylabel('人数')
plt.show()
#登船港口
print(titanic_train['Embarked'].value_counts())
bar_Emb=titanic_train['Embarked'].value_counts()
bar_Emb.plot.bar(color='orange')
plt.xlabel('登船港口')
plt.ylabel('人数')
plt.show()
0 549 1 342 Name: Survived, dtype: int64
3 491 1 216 2 184 Name: Pclass, dtype: int64
male 577 female 314 Name: Sex, dtype: int64
0 608 1 209 2 28 4 18 3 16 8 7 5 5 Name: SibSp, dtype: int64
0 678 1 118 2 80 5 5 3 5 4 4 6 1 Name: Parch, dtype: int64
S 644 C 168 Q 77 Name: Embarked, dtype: int64
- 预测存活类别分布较均衡
- 客舱等级人数，等级3>1>2
- 性别：男人比女人多
- 同乘的兄弟姐妹/配偶数以及乘的父母/小孩数大多为0
- 登船港口人数：S>C>Q
#绘制票价分布图 plt.hist(titanic_train['Fare'])
(array([732., 106., 31., 2., 11., 6., 0., 0., 0., 3.]),
array([ 0. , 51.23292, 102.46584, 153.69876, 204.93168, 256.1646 ,
307.39752, 358.63044, 409.86336, 461.09628, 512.3292 ]),
)
可以看出大部人买的便宜的票
#绘制年龄分布图 plt.hist(titanic_train['Age'],color='r')
(array([ 54., 46., 177., 169., 118., 70., 45., 24., 9., 2.]),
array([ 0.42 , 8.378, 16.336, 24.294, 32.252, 40.21 , 48.168, 56.126,
64.084, 72.042, 80. ]),
)
大部分人在20-40年龄段
数据预处理sex不是数字量，变成数值量，male为0，female为1
#对训练集处理 print(titanic_train['Sex'].unique()) titanic_train.loc[titanic_train['Sex']=='male','Sex']=0 titanic_train.loc[titanic_train['Sex']=="female",'Sex']=1
['male' 'female']
titanic_train.head()
| PassengerId | Survived | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 3 | Braund, Mr. Owen Harris | 0 | 22.0 | 1 | 0 | A/5 21171 | 7.2500 | NaN | S |
| 1 | 2 | 1 | 1 | Cumings, Mrs. John Bradley (Florence Briggs Th... | 1 | 38.0 | 1 | 0 | PC 17599 | 71.2833 | C85 | C |
| 2 | 3 | 1 | 3 | Heikkinen, Miss. Laina | 1 | 26.0 | 0 | 0 | STON/O2. 3101282 | 7.9250 | NaN | S |
| 3 | 4 | 1 | 1 | Futrelle, Mrs. Jacques Heath (Lily May Peel) | 1 | 35.0 | 1 | 0 | 113803 | 53.1000 | C123 | S |
| 4 | 5 | 0 | 3 | Allen, Mr. William Henry | 0 | 35.0 | 0 | 0 | 373450 | 8.0500 | NaN | S |
登船港口:S取0，C取1，Q取2
print(titanic_train['Embarked'].unique())
['S' 'C' 'Q' nan]
pd.value_counts(titanic_train['Embarked'])
S 644 C 168 Q 77 Name: Embarked, dtype: int64
#缺失值填充:用多的填充，S多S填充
titanic_train['Embarked']=titanic_train['Embarked'].fillna('S')
titanic_train.loc[titanic_train['Embarked']=='S','Embarked']=0 titanic_train.loc[titanic_train['Embarked']=='C','Embarked']=1 titanic_train.loc[titanic_train['Embarked']=='Q','Embarked']=2
print(titanic_train['Embarked'].unique())
[0 1 2]
将训练集embarked填充完成
titanic_train.info()
RangeIndex: 891 entries, 0 to 890 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 PassengerId 891 non-null int64 1 Survived 891 non-null int64 2 Pclass 891 non-null int64 3 Name 891 non-null object 4 Sex 891 non-null object 5 Age 714 non-null float64 6 SibSp 891 non-null int64 7 Parch 891 non-null int64 8 Ticket 891 non-null object 9 Fare 891 non-null float64 10 Cabin 204 non-null object 11 Embarked 891 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB
age数目只有714，而不是891，样本值缺失，用随机森林填充
from sklearn.ensemble import RandomForestRegressor from sklearn.metrics import mean_squared_error,r2_score #输入特征x x_age=titanic_train.loc[:,['Age','Pclass','Sex','SibSp','Parch','Fare','Embarked']] x_known_age=x_age[pd.notnull(x_age['Age'])] x_unknown_age=x_age[np.logical_not(pd.notnull(x_age['Age']))] x_train_age=x_known_age.iloc[:,1:] y_train_age=x_known_age.iloc[:,0] x_test_age=x_unknown_age.iloc[:,1:]
rfr=RandomForestRegressor(n_estimators=1500)
rfr.fit(x_train_age,y_train_age) y_train_age_pre=rfr.predict(x_train_age) y_test_age_pre=rfr.predict(x_test_age)
print('MSE train:%.3f '
%(mean_squared_error(y_train_age,y_train_age_pre)))
print('R^2:%.3f'%r2_score(y_train_age,y_train_age_pre))
MSE train:64.002 R^2:0.696
titanic_train.loc[titanic_train['Age'].isnull(),'Age']=y_test_age_pre
titanic_train.info()
RangeIndex: 891 entries, 0 to 890 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 PassengerId 891 non-null int64 1 Survived 891 non-null int64 2 Pclass 891 non-null int64 3 Name 891 non-null object 4 Sex 891 non-null object 5 Age 891 non-null float64 6 SibSp 891 non-null int64 7 Parch 891 non-null int64 8 Ticket 891 non-null object 9 Fare 891 non-null float64 10 Cabin 204 non-null object 11 Embarked 891 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB
titanic_train.head()
| PassengerId | Survived | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 3 | Braund, Mr. Owen Harris | 0 | 22.0 | 1 | 0 | A/5 21171 | 7.2500 | NaN | 0 |
| 1 | 2 | 1 | 1 | Cumings, Mrs. John Bradley (Florence Briggs Th... | 1 | 38.0 | 1 | 0 | PC 17599 | 71.2833 | C85 | 1 |
| 2 | 3 | 1 | 3 | Heikkinen, Miss. Laina | 1 | 26.0 | 0 | 0 | STON/O2. 3101282 | 7.9250 | NaN | 0 |
| 3 | 4 | 1 | 1 | Futrelle, Mrs. Jacques Heath (Lily May Peel) | 1 | 35.0 | 1 | 0 | 113803 | 53.1000 | C123 | 0 |
| 4 | 5 | 0 | 3 | Allen, Mr. William Henry | 0 | 35.0 | 0 | 0 | 373450 | 8.0500 | NaN | 0 |
对test数据集处理
#对测试集处理 print(titanic_test['Sex'].unique()) titanic_test.loc[titanic_test['Sex']=='male','Sex']=0 titanic_test.loc[titanic_test['Sex']=="female",'Sex']=1
['male' 'female']
pd.value_counts(titanic_test['Embarked'])
S 270 C 102 Q 46 Name: Embarked, dtype: int64
#缺失值填充:用多的填充，S多S填充
titanic_test['Embarked']=titanic_test['Embarked'].fillna('S')
titanic_test.loc[titanic_test['Embarked']=='S','Embarked']=0 titanic_test.loc[titanic_test['Embarked']=='C','Embarked']=1 titanic_test.loc[titanic_test['Embarked']=='Q','Embarked']=2
#对fare填充 titanic_test.loc[pd.isnull(titanic_test['Fare']),'Fare']=titanic_test['Fare'].mean()
titanic_test.info()
RangeIndex: 418 entries, 0 to 417 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 PassengerId 418 non-null int64 1 Pclass 418 non-null int64 2 Name 418 non-null object 3 Sex 418 non-null object 4 Age 332 non-null float64 5 SibSp 418 non-null int64 6 Parch 418 non-null int64 7 Ticket 418 non-null object 8 Fare 418 non-null float64 9 Cabin 91 non-null object 10 Embarked 418 non-null object dtypes: float64(2), int64(4), object(5) memory usage: 36.0+ KB
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error,r2_score
#输入特征x
x_age1=titanic_test.loc[:,['Age','Pclass','Sex','SibSp','Parch','Fare','Embarked']]
x_known_age1=x_age1[pd.notnull(x_age1['Age'])]
x_unknown_age1=x_age1[np.logical_not(pd.notnull(x_age1['Age']))]
#提取训练集和预测y的x
x_train_age1=x_known_age1.iloc[:,1:]
y_train_age1=x_known_age1.iloc[:,0]
x_test_age1=x_unknown_age1.iloc[:,1:]
#构建随机森林模型
rfr=RandomForestRegressor(n_estimators=1500)
#拟合和预测缺失age
rfr.fit(x_train_age1,y_train_age1)
y_train_age_pre1=rfr.predict(x_train_age1)
y_test_age_pre1=rfr.predict(x_test_age1)
#性能评价
print('MSE train:%.3f '
%(mean_squared_error(y_train_age1,y_train_age_pre1)))
print('R^2:%.3f'%r2_score(y_train_age1,y_train_age_pre1))
MSE train:41.033 R^2:0.795
#预测的age替代空的age titanic_test.loc[titanic_test['Age'].isnull(),'Age']=y_test_age_pre1
titanic_test.info()
特征工程 对train数据集处理RangeIndex: 418 entries, 0 to 417 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 PassengerId 418 non-null int64 1 Pclass 418 non-null int64 2 Name 418 non-null object 3 Sex 418 non-null object 4 Age 418 non-null float64 5 SibSp 418 non-null int64 6 Parch 418 non-null int64 7 Ticket 418 non-null object 8 Fare 418 non-null float64 9 Cabin 91 non-null object 10 Embarked 418 non-null object dtypes: float64(2), int64(4), object(5) memory usage: 36.0+ KB
#对票价标准化 from sklearn.preprocessing import StandardScaler sc=StandardScaler() titanic_train['Age']=sc.fit_transform(titanic_train['Age'].values.reshape(-1,1)) titanic_train['Fare']=sc.fit_transform(titanic_train['Fare'].values.reshape(-1,1))
titanic_train.head()
| PassengerId | Survived | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 3 | Braund, Mr. Owen Harris | 0 | -0.550556 | 1 | 0 | A/5 21171 | -0.502445 | NaN | 0 |
| 1 | 2 | 1 | 1 | Cumings, Mrs. John Bradley (Florence Briggs Th... | 1 | 0.614812 | 1 | 0 | PC 17599 | 0.786845 | C85 | 1 |
| 2 | 3 | 1 | 3 | Heikkinen, Miss. Laina | 1 | -0.259214 | 0 | 0 | STON/O2. 3101282 | -0.488854 | NaN | 0 |
| 3 | 4 | 1 | 1 | Futrelle, Mrs. Jacques Heath (Lily May Peel) | 1 | 0.396306 | 1 | 0 | 113803 | 0.420730 | C123 | 0 |
| 4 | 5 | 0 | 3 | Allen, Mr. William Henry | 0 | 0.396306 | 0 | 0 | 373450 | -0.486337 | NaN | 0 |
#删去cabin特征和passebger、name、ticket titanic_train.drop(labels=['PassengerId','Name','Ticket','Cabin'],axis=1,inplace=True)对test的处理
#对票价标准化 from sklearn.preprocessing import StandardScaler sc=StandardScaler() titanic_test['Age']=sc.fit_transform(titanic_test['Age'].values.reshape(-1,1)) titanic_test['Fare']=sc.fit_transform(titanic_test['Fare'].values.reshape(-1,1))
#删去cabin特征和passebger、name、ticket titanic_test.drop(labels=['PassengerId','Name','Ticket','Cabin'],axis=1,inplace=True)
titanic_test
| Pclass | Sex | Age | SibSp | Parch | Fare | Embarked | |
|---|---|---|---|---|---|---|---|
| 0 | 3 | 0 | 0.357906 | 0 | 0 | -0.498407 | 2 |
| 1 | 3 | 1 | 1.316026 | 1 | 0 | -0.513274 | 0 |
| 2 | 2 | 0 | 2.465770 | 0 | 0 | -0.465088 | 2 |
| 3 | 3 | 0 | -0.216966 | 0 | 0 | -0.483466 | 0 |
| 4 | 3 | 1 | -0.600214 | 1 | 1 | -0.418471 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 413 | 3 | 0 | -0.245439 | 0 | 0 | -0.494448 | 0 |
| 414 | 1 | 1 | 0.702829 | 0 | 0 | 1.313753 | 1 |
| 415 | 3 | 0 | 0.664505 | 0 | 0 | -0.508792 | 0 |
| 416 | 3 | 0 | -0.245439 | 0 | 0 | -0.494448 | 0 |
| 417 | 3 | 0 | -0.727514 | 1 | 1 | -0.237906 | 1 |
418 rows × 7 columns
模型构建#X,y获取
X=titanic_train.iloc[:,1:]
y=titanic_train.iloc[:,0]
print(X.shape,y.shape)
#数据划分
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,stratify=y,test_size=0.2,
random_state=90)
#构建基本模型
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
foreast=RandomForestClassifier(n_estimators=500)
foreast.fit(X_train,y_train)
print('foreast train score:%.3f,foreast test score:%.3f'
%(foreast.score(X_train,y_train),foreast.score(X_test,y_test)))
score = cross_val_score(foreast,X_train,y_train,cv=10).mean()
print(score)
(891, 7) (891,) foreast train score:0.980,foreast test score:0.793 0.8160798122065728
从score来看，训练集和测试集较大，存在过拟合
特征选择feature_names = titanic_train.columns[1:]
feature_names
Index(['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked'], dtype='object')
#基于随机森林的特征选择 forest = RandomForestClassifier(random_state=0) forest.fit(X_train, y_train) importances = forest.feature_importances_ std = np.std([tree.feature_importances_ for tree in forest.estimators_], axis=0)
#描述杂质的重要性
forest_importances = pd.Series(importances, index=feature_names)
fig, ax = plt.subplots()
forest_importances.plot.bar(yerr=std, ax=ax)
ax.set_title("Feature importances using MDI")
ax.set_ylabel("Mean decrease in impurity")
fig.tight_layout()
X_train_sel=X_train.loc[:,['Sex','Age','Fare']] X_test_sel=X_test.loc[:,['Sex','Age','Fare']]
foreast1=RandomForestClassifier()
foreast1.fit(X_train_sel,y_train)
print('foreast train score:%.3f,foreast test score:%.3f'
%(foreast1.score(X_train_sel,y_train),foreast1.score(X_test_sel,y_test)))
score = cross_val_score(foreast,X_train,y_train,cv=10).mean()
print(score)
foreast train score:0.980,foreast test score:0.827 0.8146713615023475
过拟合
超参数最优#分层交叉验证
import time
from sklearn.model_selection import cross_val_score
#n_estimators调参
start=time.time()
scorel = []
for i in range(0,1000,10): # 迭代建立包含0-1000棵决策树的RF模型进行对比
foreast= RandomForestClassifier(n_estimators=i+1,n_jobs=-1,random_state=90)
score = cross_val_score(foreast,X_train_sel,y_train,cv=10).mean()
scorel.append(score)
print(max(scorel),(scorel.index(max(scorel))*10)+1)
best_n_estimators=scorel.index(max(scorel))*10+1
end=time.time()
print('Running time: %s Seconds'%(end-start))
plt.figure(figsize=[20,5])
plt.plot(range(1,1001,10),scorel)
plt.show()
0.7711658841940532 161 Running time: 448.6951653957367 Seconds
#max_depth调参：网格搜索
from sklearn.model_selection import GridSearchCV
start=time.time()
param_grid={'max_depth':np.arange(1,20,2)}
foreast=RandomForestClassifier(n_estimators=best_n_estimators
,random_state=90)
GS=GridSearchCV(foreast,param_grid,cv=10)
GS.fit(X_train_sel,y_train)
print(GS.best_params_)
print(GS.best_score_)
best_max_depth=GS.best_params_['max_depth']
end=time.time()
print('Running time: %s Seconds'%(end-start))
{'max_depth': 5}
0.793622848200313
Running time: 18.917694807052612 Seconds
#max_features调参
from sklearn.model_selection import GridSearchCV
start=time.time()
param_grid={'max_features':np.arange(1,3,1)}
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
random_state=90)
GS=GridSearchCV(foreast,param_grid,cv=10)
GS.fit(X_train_sel,y_train)
print(GS.best_params_)
print(GS.best_score_)
best_max_features=GS.best_params_['max_features']
end=time.time()
print('Running time: %s Seconds'%(end-start))
{'max_features': 1}
0.793622848200313
Running time: 3.8559486865997314 Seconds
#min_samples_leaf
from sklearn.model_selection import GridSearchCV
start=time.time()
param_grid={'min_samples_leaf':np.arange(1,20,2)}
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
max_features=best_max_features,
random_state=90)
GS=GridSearchCV(foreast,param_grid,cv=10)
GS.fit(X_train_sel,y_train)
print(GS.best_params_)
print(GS.best_score_)
best_min_samples_leaf=GS.best_params_['min_samples_leaf']
end=time.time()
print('Running time: %s Seconds'%(end-start))
{'min_samples_leaf': 1}
0.793622848200313
Running time: 17.986785173416138 Seconds
#min_samples_split
from sklearn.model_selection import GridSearchCV
start=time.time()
param_grid={'min_samples_split':np.arange(2,20,2)}
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
max_features=best_max_features,
min_samples_leaf=best_min_samples_leaf,
random_state=90)
GS=GridSearchCV(foreast,param_grid,cv=10)
GS.fit(X_train_sel,y_train)
print(GS.best_params_)
print(GS.best_score_)
best_min_samples_split=GS.best_params_['min_samples_split']
end=time.time()
print('Running time: %s Seconds'%(end-start))
{'min_samples_split': 2}
0.793622848200313
Running time: 17.482104063034058 Seconds
#criterion
from sklearn.model_selection import GridSearchCV
start=time.time()
param_grid={'criterion':['gini','entropy']}
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
max_features=best_max_features,
min_samples_leaf=best_min_samples_leaf,
min_samples_split=best_min_samples_split,
random_state=90)
GS=GridSearchCV(foreast,param_grid,cv=10)
GS.fit(X_train_sel,y_train)
print(GS.best_params_)
print(GS.best_score_)
best_criterion=GS.best_params_['criterion']
end=time.time()
print('Running time: %s Seconds'%(end-start))
{'criterion': 'gini'}
0.793622848200313
Running time: 3.8784661293029785 Seconds
#最优参数整合
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
max_features=best_max_features,
min_samples_leaf=best_min_samples_leaf,
min_samples_split=best_min_samples_split,
criterion=best_criterion,
random_state=90)
foreast.fit(X_train_sel,y_train)
print('foreast train score:%.3f,foreast test score:%.3f'
%(foreast.score(X_train_sel,y_train),foreast.score(X_test_sel,y_test)))
score = cross_val_score(foreast,X_train_sel,y_train,cv=10).mean()
print(score)
foreast train score:0.831,foreast test score:0.855 0.793622848200313
性能提升了0.3
评估模型#测试集准确率
from sklearn.metrics import accuracy_score
y_pred=foreast.predict(X_test_sel)
acc_score_test=accuracy_score(y_test,y_pred)
print('accuracy score test:%.3f'%acc_score_test)
accuracy score test:0.855
#测试集精确率
from sklearn.metrics import precision_score
#微平均精确率
micro_pre_score=precision_score(y_test,y_pred,average='micro')
print('micro precision score:%.3f'%micro_pre_score)
#宏平均精确率
macro_pre_score=precision_score(y_test,y_pred,average='macro')
print('macro precision score:%.3f'%macro_pre_score)
micro precision score:0.855 macro precision score:0.861
#测试集召回率
from sklearn.metrics import recall_score
test_recall=recall_score(y_test,y_pred)
print('Recall score test:%.3f'%test_recall)
Recall score test:0.725
#测试集F1
from sklearn.metrics import f1_score
test_f1_score=f1_score(y_test,y_pred)
print('F1 score test:%.3f'%test_f1_score)
F1 score test:0.794
#分类综合报告 from sklearn.metrics import classification_report target_names=['class0','class1'] print(classification_report(y_test,y_pred,target_names=target_names))
precision recall f1-score support
class0 0.84 0.94 0.89 110
class1 0.88 0.72 0.79 69
accuracy 0.85 179
macro avg 0.86 0.83 0.84 179
weighted avg 0.86 0.85 0.85 179
#混淆矩阵可视化 from sklearn.metrics import confusion_matrix conf_matrix=confusion_matrix(y_test,y_pred,labels=[0,1]) conf_matrix_df=pd.DataFrame(conf_matrix,index=target_names,columns=target_names)
conf_matrix_df
| class0 | class1 | |
|---|---|---|
| class0 | 103 | 7 |
| class1 | 19 | 50 |
plt.matshow(conf_matrix, cmap=plt.cm.Blues)
plt.colorbar()
for i in range(len(conf_matrix)):
for j in range(len(conf_matrix)):
plt.annotate(conf_matrix[i,j], xy=(i, j), horizontalalignment='center', verticalalignment='center')
plt.ylabel('True label')
plt.xlabel('Predicted label')
plt.show()
#获得训练子集和测试子集的精确度
from sklearn.model_selection import learning_curve
train_sizes,train_scores,test_scores=learning_curve(estimator=foreast,
X=X_train_sel,
y=y_train,
train_sizes=np.linspace(0.1,1,10),
cv=10,
n_jobs=1)
train_sizes
array([ 64, 128, 192, 256, 320, 384, 448, 512, 576, 640])
train_mean=np.mean(train_scores,axis=1) train_mean
array([0.984375 , 0.921875 , 0.90052083, 0.89960938, 0.8746875 ,
0.85572917, 0.84933036, 0.8453125 , 0.83697917, 0.836875 ])
train_std=np.std(train_scores,axis=1) train_std
array([0. , 0.01048157, 0.01686889, 0.01472506, 0.01540153,
0.00896076, 0.010763 , 0.00724501, 0.00886777, 0.00568544])
test_mean=np.mean(test_scores,axis=1) test_mean
array([0.69536385, 0.74029734, 0.74874804, 0.76416275, 0.75577074,
0.76140454, 0.76420188, 0.78382238, 0.78519171, 0.79643975])
test_std=np.std(test_scores,axis=1) test_std
array([0.03854721, 0.05002497, 0.06001087, 0.0683186 , 0.04782528,
0.05277893, 0.05322023, 0.04111021, 0.05066113, 0.04403105])
plt.plot(train_sizes,train_mean,color='b',
marker='o',markersize=5,label='training accuracy')
plt.fill_between(train_sizes,train_mean+train_std,
train_mean-train_std,alpha=0.15,
color='b')
plt.plot(train_sizes,test_mean,color='g',
marker='s',markersize=5,label='validation accuracy',
linestyle='--')
plt.fill_between(train_sizes,test_mean+test_std,
test_mean-test_std,alpha=0.15,
color='g')
plt.grid()#生成网格
plt.xlabel('number of trainnig samples')
plt.ylabel('accuracy')
plt.legend(loc='best')
plt.ylim([0.60,1.01])
plt.show()
将titantic_test应用到模型
titanic_test
| Pclass | Sex | Age | SibSp | Parch | Fare | Embarked | |
|---|---|---|---|---|---|---|---|
| 0 | 3 | 0 | 0.357906 | 0 | 0 | -0.498407 | 2 |
| 1 | 3 | 1 | 1.316026 | 1 | 0 | -0.513274 | 0 |
| 2 | 2 | 0 | 2.465770 | 0 | 0 | -0.465088 | 2 |
| 3 | 3 | 0 | -0.216966 | 0 | 0 | -0.483466 | 0 |
| 4 | 3 | 1 | -0.600214 | 1 | 1 | -0.418471 | 0 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 413 | 3 | 0 | -0.245439 | 0 | 0 | -0.494448 | 0 |
| 414 | 1 | 1 | 0.702829 | 0 | 0 | 1.313753 | 1 |
| 415 | 3 | 0 | 0.664505 | 0 | 0 | -0.508792 | 0 |
| 416 | 3 | 0 | -0.245439 | 0 | 0 | -0.494448 | 0 |
| 417 | 3 | 0 | -0.727514 | 1 | 1 | -0.237906 | 1 |
418 rows × 7 columns
X=titanic_test.loc[:,['Sex','Age','Fare']]
foreast=RandomForestClassifier(n_estimators=best_n_estimators,
max_depth=best_max_depth,
max_features=best_max_features,
min_samples_leaf=best_min_samples_leaf,
min_samples_split=best_min_samples_split,
criterion=best_criterion,
random_state=90)
foreast.fit(X_train_sel,y_train)
y=foreast.predict(X)
y
array([0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0,
1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1,
1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1,
1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1,
1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0,
0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0,
1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1,
0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1,
0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0,
1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1,
0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1,
0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1,
0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0,
0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0,
1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1,
0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0],
dtype=int64)
test = pd.read_csv("test.csv")
output = pd.DataFrame({'PassengerId': test.PassengerId, 'Survived': y})
output.to_csv('my_submission.csv', index=False)
