🪣 第16课:对象存储

核心概念

对象存储:海量数据的无限硬盘

对象存储(S3/OSS/COS)提供无限容量、高可用、按用量付费的非结构化数据存储,是备份、静态网站、CI/CD制品的理想后端。

核心概念

Bucket(桶)→ 顶层命名空间,全局唯一
  ├── Object → 文件+元数据+Key(最大5TB)
  ├── Prefix → 模拟目录结构
  ├── Version → 启用版本控制后每个版本有ID
  └── Lifecycle → 自动过期/转储规则
存储类型访问频率最小存储期价格倍数适用场景
Standard频繁1x网站、热数据
Infrequent Access月级30天0.5x备份、日志
Glacier/Archive年级90天0.1x归档、合规
Glacier Deep年级180天0.03x长期归档

命令实操

1. AWS S3核心操作 ✅

S3操作
# 创建Bucket(启用版本控制+加密)
aws s3api create-bucket --bucket my-devops-artifacts-2026     --region ap-southeast-1     --create-bucket-configuration LocationConstraint=ap-southeast-1

aws s3api put-bucket-versioning     --bucket my-devops-artifacts-2026     --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption --bucket my-devops-artifacts-2026     --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"aws:kms"},"BucketKeyEnabled":true}]}'

# 上传文件
aws s3 cp app-v1.2.3.tar.gz s3://my-devops-artifacts-2026/builds/app/
aws s3 sync ./dist/ s3://my-devops-artifacts-2026/static/ --delete

# 预签名URL(限时访问)
aws s3 presign s3://my-devops-artifacts-2026/builds/app/app-v1.2.3.tar.gz --expires-in 3600

# 分段上传大文件
aws s3 cp large-dataset.tar.gz s3://my-devops-artifacts-2026/data/ --expected-size 5368709120

2. 生命周期策略 ✅

生命周期
cat > lifecycle.json << 'EOF'
{
    "Rules": [
        {
            "ID": "LogArchiveRule",
            "Status": "Enabled",
            "Filter": {"Prefix": "logs/"},
            "Transitions": [
                {"Days": 30, "StorageClass": "STANDARD_IA"},
                {"Days": 90, "StorageClass": "GLACIER"},
                {"Days": 365, "StorageClass": "DEEP_ARCHIVE"}
            ],
            "Expiration": {"Days": 2555}
        },
        {
            "ID": "BuildCleanup",
            "Status": "Enabled",
            "Filter": {"Prefix": "builds/"},
            "NoncurrentVersionExpiration": {"NoncurrentDays": 30},
            "Expiration": {"Days": 90}
        }
    ]
}
EOF
aws s3api put-bucket-lifecycle-configuration     --bucket my-devops-artifacts-2026     --lifecycle-configuration file://lifecycle.json

3. S3静态网站托管 ✅

静态网站
aws s3api create-bucket --bucket my-static-site-2026 --region us-east-1
aws s3api delete-public-access-block --bucket my-static-site-2026

cat > website.json << 'EOF'
{"IndexDocument":{"Suffix":"index.html"},"ErrorDocument":{"Key":"404.html"}}
EOF
aws s3api put-bucket-website --bucket my-static-site-2026     --website-configuration file://website.json

aws s3 sync ./site/ s3://my-static-site-2026/ --acl public-read --cache-control "max-age=86400"

# Bucket Policy允许公开读
cat > bucket-policy.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Sid":"PublicRead","Effect":"Allow","Principal":"*","Action":"s3:GetObject","Resource":"arn:aws:s3:::my-static-site-2026/*"}]}
EOF
aws s3api put-bucket-policy --bucket my-static-site-2026 --policy file://bucket-policy.json
echo "URL: http://my-static-site-2026.s3-website-us-east-1.amazonaws.com" 

4. 阿里云OSS操作 ✅

OSS操作
# 安装ossutil
curl -o ossutil64 https://gosspublic.alicdn.com/ossutil/1.7.18/ossutil-v1.7.18-linux-amd64.zip
unzip ossutil64.zip && chmod +x ossutil && sudo mv ossutil /usr/local/bin/
ossutil config  # 配置Endpoint/AccessKey

ossutil mb oss://my-devops-bucket --storage-class Standard --acl private
ossutil cp ./app.tar.gz oss://my-devops-bucket/artifacts/
ossutil sync ./dist/ oss://my-devops-bucket/static/ --delete
ossutil du oss://my-devops-bucket/ --storage-class Standard
ossutil sign oss://my-devops-bucket/artifacts/app.tar.gz --timeout 3600

架构图

对象存储分层架构

┌─────────────────────────────────────────────────┐
│                 应用层                           │
│  Web / Mobile / CI/CD / Data Pipeline          │
└──────────────────────┬──────────────────────────┘
┌──────────────────────▼──────────────────────────┐
│            对象存储服务                          │
│  builds/  → Standard (热数据)                   │
│  logs/    → IA → Glacier → Archive              │
│  backup/  → IA (30天后自动分层)                   │
│  static/  → Standard + CDN缓存                  │
│  加密(KMS) + 版本控制 + 生命周期 + 跨区域复制     │
└─────────────────────────────────────────────────┘

故障排查

❌ S3上传403

排查
aws iam get-user-policy --user-name devops-user --policy-name S3Access
aws s3api get-bucket-policy --bucket my-devops-artifacts-2026
aws s3api get-public-access-block --bucket my-devops-artifacts-2026
# 修复:添加IAM策略
aws iam put-user-policy --user-name devops-user     --policy-name S3Write --policy-document '{
    "Version":"2012-10-17","Statement":[{
        "Effect":"Allow","Action":["s3:PutObject","s3:GetObject","s3:ListBucket"],
        "Resource":["arn:aws:s3:::my-devops-artifacts-2026","arn:aws:s3:::my-devops-artifacts-2026/*"]
    }]}'
💡 S3安全最佳实践:启用版本控制防止误删、启用加密保护数据、使用预签名URL而非公开访问、配置生命周期自动分层节省成本。

🏆 成就解锁:对象存储专家!

掌握S3/OSS核心操作、生命周期分层、静态网站托管——对象存储是云上数据的基石

📝 S3安全最佳实践

安全措施配置方法重要性
启用版本控制put-bucket-versioning Status=Enabled⭐⭐⭐
启用加密SSE-KMS或SSE-S3⭐⭐⭐
禁止公开访问Block Public Access⭐⭐⭐
预签名URL限时访问而非公开⭐⭐
MFA删除要求MFA才能删除对象⭐⭐
访问日志启用Server Access Logging⭐⭐

AWS S3

阿里云OSS

腾讯云COS

💡 对象存储成本优化:使用Lifecycle自动分层、删除未完成的分段上传、启用智能分层(自动在热/冷间切换)。

🔧 S3跨区域复制配置

跨区域复制
# 启用跨区域复制(灾备)
aws s3api put-bucket-replication \
    --bucket my-devops-artifacts-2026 \
    --replication-configuration '{
    "Role": "arn:aws:iam::123:role/s3-replication",
    "Rules": [{
        "ID": "replicate-all",
        "Status": "Enabled",
        "Destination": {
            "Bucket": "arn:aws:s3:::my-devops-artifacts-dr",
            "StorageClass": "STANDARD_IA"
        }
    }]
}'

# 验证复制状态
aws s3api get-bucket-replication --bucket my-devops-artifacts-2026