271 lines
6.4 KiB
Markdown
271 lines
6.4 KiB
Markdown
|
|
---
|
|||
|
|
name: minio
|
|||
|
|
description: MinIO 对象存储操作技能。当用户提到 MinIO、S3 存储、对象存储、文件上传下载、公网地址、外链等关键词时必须使用此技能。
|
|||
|
|
vm0_secrets:
|
|||
|
|
- MINIO_ACCESS_KEY
|
|||
|
|
- MINIO_SECRET_KEY
|
|||
|
|
vm0_vars:
|
|||
|
|
- MINIO_ENDPOINT
|
|||
|
|
- MINIO_PUBLIC_URL
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# MinIO 对象存储技能
|
|||
|
|
|
|||
|
|
统一的 MinIO 对象存储操作技能,支持文件上传下载、桶管理、公网地址生成等功能。
|
|||
|
|
|
|||
|
|
> 官方文档: `https://min.io/docs/minio/linux/reference/minio-mc.html`
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 触发条件
|
|||
|
|
|
|||
|
|
**当用户提到以下关键词时必须使用此技能:**
|
|||
|
|
- MinIO、minio
|
|||
|
|
- S3 存储、对象存储
|
|||
|
|
- 文件上传、文件下载
|
|||
|
|
- 公网地址、外链、访问链接
|
|||
|
|
- 桶、bucket
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 环境配置
|
|||
|
|
|
|||
|
|
| 变量名 | 说明 | 示例 |
|
|||
|
|
|--------|------|------|
|
|||
|
|
| `MINIO_ENDPOINT` | MinIO 服务地址 | `http://192.168.11.13:10061` |
|
|||
|
|
| `MINIO_ACCESS_KEY` | 访问密钥 | `iUTez6g5DYVg5vhXhMzc` |
|
|||
|
|
| `MINIO_SECRET_KEY` | 密钥 | `xx...` |
|
|||
|
|
| `MINIO_PUBLIC_URL` | 公网访问域名(可选) | `https://oss.example.com` |
|
|||
|
|
|
|||
|
|
### 配置 mc 客户端
|
|||
|
|
|
|||
|
|
首次使用需配置别名:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
mc alias set myminio $MINIO_ENDPOINT $MINIO_ACCESS_KEY $MINIO_SECRET_KEY
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 功能列表
|
|||
|
|
|
|||
|
|
### 1. 桶管理
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 列出所有桶
|
|||
|
|
mc ls myminio/
|
|||
|
|
|
|||
|
|
# 创建新桶
|
|||
|
|
mc mb myminio/new-bucket
|
|||
|
|
|
|||
|
|
# 删除空桶
|
|||
|
|
mc rb myminio/empty-bucket
|
|||
|
|
|
|||
|
|
# 删除非空桶(危险操作)
|
|||
|
|
mc rb --force myminio/bucket-to-delete
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. 文件上传
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 上传单个文件
|
|||
|
|
mc cp /path/to/local/file.txt myminio/bucket/path/to/remote/file.txt
|
|||
|
|
|
|||
|
|
# 上传并指定按日期组织的路径
|
|||
|
|
mc cp myfile.tar.gz myminio/files/$(date +%Y/%m/%d)/myfile.tar.gz
|
|||
|
|
|
|||
|
|
# 上传整个目录
|
|||
|
|
mc mirror ./local_dir myminio/bucket/remote_dir/
|
|||
|
|
|
|||
|
|
# 上传目录并显示进度
|
|||
|
|
mc mirror --watch ./local_dir myminio/bucket/remote_dir/
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3. 文件下载
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 下载单个文件
|
|||
|
|
mc cp myminio/bucket/path/to/file.txt ./local_file.txt
|
|||
|
|
|
|||
|
|
# 下载整个目录
|
|||
|
|
mc mirror myminio/bucket/remote_dir/ ./local_dir/
|
|||
|
|
|
|||
|
|
# 下载指定前缀的所有文件
|
|||
|
|
mc mirror myminio/bucket/prefix/ ./local_dir/
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 4. 文件管理
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 列出桶内文件
|
|||
|
|
mc ls myminio/bucket/
|
|||
|
|
mc ls myminio/bucket/path/
|
|||
|
|
|
|||
|
|
# 递归列出
|
|||
|
|
mc ls --recursive myminio/bucket/
|
|||
|
|
|
|||
|
|
# 查看文件详情
|
|||
|
|
mc stat myminio/bucket/path/to/file.txt
|
|||
|
|
|
|||
|
|
# 删除文件
|
|||
|
|
mc rm myminio/bucket/path/to/file.txt
|
|||
|
|
|
|||
|
|
# 批量删除
|
|||
|
|
mc rm --recursive --force myminio/bucket/path/
|
|||
|
|
|
|||
|
|
# 复制文件(桶内或跨桶)
|
|||
|
|
mc cp myminio/source-bucket/file.txt myminio/dest-bucket/file.txt
|
|||
|
|
|
|||
|
|
# 移动/重命名文件
|
|||
|
|
mc mv myminio/bucket/old-name.txt myminio/bucket/new-name.txt
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 5. 公网访问地址
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 列出所有桶内对象的公网地址
|
|||
|
|
# 使用端点地址作为公网地址
|
|||
|
|
mc ls --recursive myminio/bucket/ | while read -r line; do
|
|||
|
|
file=$(echo "$line" | awk '{print $NF}')
|
|||
|
|
echo "http://192.168.11.13:10061/bucket/$file"
|
|||
|
|
done
|
|||
|
|
|
|||
|
|
# 生成临时分享链接(默认7天有效)
|
|||
|
|
mc share download myminio/bucket/path/to/file.txt
|
|||
|
|
|
|||
|
|
# 生成指定有效期的分享链接
|
|||
|
|
mc share download --expire 24h myminio/bucket/path/to/file.txt
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 6. Python 脚本操作(适用于复杂场景)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
import boto3
|
|||
|
|
from botocore.client import Config
|
|||
|
|
import os
|
|||
|
|
|
|||
|
|
# MinIO 配置
|
|||
|
|
endpoint = os.environ.get('MINIO_ENDPOINT', 'http://192.168.11.13:10061')
|
|||
|
|
access_key = os.environ.get('MINIO_ACCESS_KEY')
|
|||
|
|
secret_key = os.environ.get('MINIO_SECRET_KEY')
|
|||
|
|
|
|||
|
|
# 创建 S3 客户端
|
|||
|
|
s3 = boto3.client(
|
|||
|
|
's3',
|
|||
|
|
endpoint_url=endpoint,
|
|||
|
|
aws_access_key_id=access_key,
|
|||
|
|
aws_secret_access_key=secret_key,
|
|||
|
|
config=Config(signature_version='s3v4')
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
# 列出所有桶
|
|||
|
|
buckets = s3.list_buckets()
|
|||
|
|
for bucket in buckets['Buckets']:
|
|||
|
|
print(bucket['Name'])
|
|||
|
|
|
|||
|
|
# 上传文件
|
|||
|
|
s3.upload_file('local_file.txt', 'bucket-name', 'remote/path/file.txt')
|
|||
|
|
|
|||
|
|
# 下载文件
|
|||
|
|
s3.download_file('bucket-name', 'remote/path/file.txt', 'local_file.txt')
|
|||
|
|
|
|||
|
|
# 列出桶内对象
|
|||
|
|
objects = s3.list_objects_v2(Bucket='bucket-name', Prefix='path/')
|
|||
|
|
for obj in objects.get('Contents', []):
|
|||
|
|
print(f"{obj['Key']} - {obj['Size']} bytes")
|
|||
|
|
|
|||
|
|
# 生成预签名URL(临时访问链接)
|
|||
|
|
url = s3.generate_presigned_url(
|
|||
|
|
'get_object',
|
|||
|
|
Params={'Bucket': 'bucket-name', 'Key': 'path/to/file.txt'},
|
|||
|
|
ExpiresIn=3600 # 1小时有效
|
|||
|
|
)
|
|||
|
|
print(url)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 常用场景示例
|
|||
|
|
|
|||
|
|
### 场景1:上传文件并返回公网地址
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 上传文件到按日期组织的路径
|
|||
|
|
DATE_PATH=$(date +%Y/%m/%d)
|
|||
|
|
FILENAME="myfile.tar.gz"
|
|||
|
|
mc cp $FILENAME myminio/files/$DATE_PATH/$FILENAME
|
|||
|
|
|
|||
|
|
# 返回公网地址
|
|||
|
|
echo "公网地址: http://192.168.11.13:10061/files/$DATE_PATH/$FILENAME"
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 场景2:列出所有公网地址
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 快速列出桶内所有文件的公网地址
|
|||
|
|
python3 << 'EOF'
|
|||
|
|
import boto3
|
|||
|
|
from botocore.client import Config
|
|||
|
|
import os
|
|||
|
|
from urllib.parse import quote
|
|||
|
|
|
|||
|
|
endpoint = os.environ.get('MINIO_ENDPOINT')
|
|||
|
|
access_key = os.environ.get('MINIO_ACCESS_KEY')
|
|||
|
|
secret_key = os.environ.get('MINIO_SECRET_KEY')
|
|||
|
|
|
|||
|
|
s3 = boto3.client(
|
|||
|
|
's3',
|
|||
|
|
endpoint_url=endpoint,
|
|||
|
|
aws_access_key_id=access_key,
|
|||
|
|
aws_secret_access_key=secret_key,
|
|||
|
|
config=Config(signature_version='s3v4')
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
# 列出所有桶
|
|||
|
|
for bucket in s3.list_buckets()['Buckets']:
|
|||
|
|
bucket_name = bucket['Name']
|
|||
|
|
print(f"\n=== 桶: {bucket_name} ===")
|
|||
|
|
|
|||
|
|
# 列出桶内所有对象
|
|||
|
|
paginator = s3.get_paginator('list_objects_v2')
|
|||
|
|
for page in paginator.paginate(Bucket=bucket_name):
|
|||
|
|
for obj in page.get('Contents', []):
|
|||
|
|
key = obj['Key']
|
|||
|
|
# URL 编码文件路径
|
|||
|
|
encoded_key = quote(key, safe='/')
|
|||
|
|
print(f"{endpoint}/{bucket_name}/{encoded_key}")
|
|||
|
|
EOF
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 场景3:打包目录并上传
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 打包目录
|
|||
|
|
tar -czf output.tar.gz ./my_directory/
|
|||
|
|
|
|||
|
|
# 上传
|
|||
|
|
mc cp output.tar.gz myminio/files/$(date +%Y/%m/%d)/output.tar.gz
|
|||
|
|
|
|||
|
|
# 清理本地文件
|
|||
|
|
rm output.tar.gz
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 工具脚本
|
|||
|
|
|
|||
|
|
列出所有公网地址的脚本:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
/app/working/workspaces/default/active_skills/minio/scripts/list_public_urls.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 注意事项
|
|||
|
|
|
|||
|
|
1. **不要上传到根目录**:始终将文件上传到有意义的路径,如按日期组织 `files/YYYY/MM/DD/`
|
|||
|
|
2. **大文件上传**:对于大文件,使用 `mc` 命令行工具比 Python 脚本更稳定
|
|||
|
|
3. **临时链接**:分享敏感文件时,使用 `mc share download` 生成临时链接而非公网地址
|
|||
|
|
4. **权限控制**:确保 bucket 策略配置正确,公开访问的 bucket 才能通过公网地址直接访问
|