🚢 泰坦尼克实战

📖 泰坦尼克：EDA实战项目

泰坦尼克号数据集是数据分析的"Hello World"——每个数据科学家的入门项目。1912年泰坦尼克号沉没，2224名乘客中1502人遇难。通过分析乘客数据，我们试图回答：什么因素影响了生存？

1. 数据概览

import seaborn as sns
import pandas as pd

df = sns.load_dataset('titanic')
print(f"形状: {df.shape}")     # (891, 15)
print(f"缺失值:\n{df.isnull().sum()}")
# age: 177, deck: 688, embarked: 2

🔍 生存率分析

总体与分组生存率

# 总生存率
print(f"总生存率: {df['survived'].mean():.2%}")  # 38.4%

# 按性别
gender_survival = df.groupby('sex')['survived'].mean()
# female: 74.2%, male: 18.9%  → 女性生存率是男性的4倍！

# 按舱位
class_survival = df.groupby('class')['survived'].mean()
# First: 63.0%, Second: 47.3%, Third: 24.2%

# 按年龄组
df['age_group'] = pd.cut(df['age'], bins=[0,12,18,35,60,100],
                         labels=['儿童','青少年','青年','中年','老年'])
age_survival = df.groupby('age_group', observed=True)['survived'].mean()
# 儿童: 58.0% → 最高！

🎨 可视化分析

import matplotlib.pyplot as plt

fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# 生存分布(按性别)
sns.countplot(data=df, x='survived', hue='sex', ax=axes[0,0], palette='coolwarm')

# 舱位与生存率
sns.barplot(data=df, x='class', y='survived', hue='sex', ax=axes[0,1])

# 年龄分布
sns.histplot(data=df, x='age', hue='survived', bins=30, ax=axes[1,0])

# 票价箱线图
sns.boxplot(data=df, x='class', y='fare', ax=axes[1,1])

家庭大小分析

df['family_size'] = df['sibsp'] + df['parch'] + 1
family_survival = df.groupby('family_size')['survived'].agg(['mean', 'count'])
# family=4时生存率最高(72.4%)
# 独行(family=1)和大家庭(>6)生存率低

📝 EDA结论

📐 2024-2025 EDA前沿

• AutoEDA：ydata-profiling、sweetviz自动生成报告
• 交互式探索：PyGWalker拖拽分析，无需写代码
• 大数据EDA：Dask/Vaex处理GB级数据
• 因果EDA：DoWhy/do-operators，从相关走向因果
• ML辅助EDA：用聚类/异常检测发现隐藏模式

📊 高级可视化分析

import seaborn as sns, matplotlib.pyplot as plt

df = sns.load_dataset('titanic')
fig, axes = plt.subplots(2, 2, figsize=(14, 10))

# 年龄-票价联合分布
sns.kdeplot(data=df, x='age', y='fare', hue='survived',
            ax=axes[0,0], fill=True, alpha=0.5, palette='coolwarm')

# 生存率热力图
df['age_bin'] = pd.cut(df['age'], bins=[0,12,18,35,60,100])
heat = df.pivot_table('survived', index='age_bin',
                      columns='class', observed=True)
sns.heatmap(heat, annot=True, fmt='.2f', cmap='coolwarm', ax=axes[0,1])

# 家庭大小与生存率
df['family_size'] = df['sibsp'] + df['parch'] + 1
sns.barplot(data=df, x='family_size', y='survived',
            ax=axes[1,1], color='#3b82f6')

🧮 统计检验

from scipy import stats

# 卡方检验: 性别与生存
contingency = pd.crosstab(df['sex'], df['survived'])
chi2, p, dof, expected = stats.chi2_contingency(contingency)
print(f"chi2={chi2:.2f}, p={p:.2e}")

# t检验: 年龄差异
surv_age = df[df['survived']==1]['age'].dropna()
died_age = df[df['survived']==0]['age'].dropna()
t, p = stats.ttest_ind(surv_age, died_age)

# ANOVA: 票价差异
f, p = stats.f_oneway(
    df[df['class']=='First']['fare'],
    df[df['class']=='Second']['fare'],
    df[df['class']=='Third']['fare'])

💻 完整实战代码

#!/usr/bin/env python3
# 泰坦尼克实战 — 完整实战

import pandas as pd
import numpy as np
import seaborn as sns

# ============ 加载数据 ============
df = sns.load_dataset('titanic')
print(f"数据形状: {df.shape}")
print(f"列名: {list(df.columns)}")
print(f"\n缺失值:\n{df.isnull().sum()}")

# ============ 生存率分析 ============
survival_rate = df['survived'].mean()
print(f"\n总生存率: {survival_rate:.2%}")

gender_survival = df.groupby('sex')['survived'].mean()
print(f"按性别:\n{gender_survival.apply(lambda x: f'{x:.2%}')}")

class_survival = df.groupby('class')['survived'].mean()
print(f"按舱位:\n{class_survival.apply(lambda x: f'{x:.2%}')}")

# ============ 年龄组分析 ============
df['age_group'] = pd.cut(df['age'], bins=[0,12,18,35,60,100],
                         labels=['儿童','青少年','青年','中年','老年'])
age_survival = df.groupby('age_group', observed=True)['survived'].mean()
print(f"按年龄组:\n{age_survival.apply(lambda x: f'{x:.2%}')}")

# ============ 数据清洗 ============
df_clean = df.copy()
df_clean['age'] = df_clean['age'].fillna(df_clean['age'].median())
df_clean['embarked'] = df_clean['embarked'].fillna(df_clean['embarked'].mode()[0])
df_clean.drop(columns=['deck'], inplace=True, errors='ignore')
print(f"\n清洗后缺失值:\n{df_clean.isnull().sum()[df_clean.isnull().sum()>0]}")

# ============ 交叉分析 ============
cross = pd.crosstab([df['class'], df['sex']], df['survived'], normalize='index')
print(f"\n交叉分析(生存率):\n{(cross[1]*100).round(1).astype(str)+'%'}")

# ============ 家庭大小分析 ============
df['family_size'] = df['sibsp'] + df['parch'] + 1
family_survival = df.groupby('family_size')['survived'].agg(['mean', 'count'])
print(f"\n家庭大小与生存:\n{family_survival}")

print("\n✅ Python验证通过 — EDA完整分析报告")

📝 课后练习

1. 对泰坦尼克数据做完整的统计检验（卡方检验/t检验）
2. 创建新特征（称谓/船票前缀）并分析其预测力
3. 用决策树可视化生存决策规则
4. 对比不同缺失值处理方法的结果差异
5. 在Kaggle上提交你的第一个预测模型

🗺️ EDA分析框架

EDA自动化工具