|
|
@@ -0,0 +1,936 @@
|
|
|
+# Wenshu Platform 部署文档
|
|
|
+
|
|
|
+> 服务器要求:Ubuntu 22.04,8vCPU,32GB RAM,100GB SSD(不含业务数据)
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 架构总览
|
|
|
+
|
|
|
+```
|
|
|
+┌─────────────────────────────────────────────────────────┐
|
|
|
+│ Ubuntu 22.04 服务器 │
|
|
|
+│ │
|
|
|
+│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
|
|
|
+│ │ Nginx │ │ Backend │ │ Python-Iceberg │ │
|
|
|
+│ │ :80 │───▶│ :8080 │───▶│ :8090 │ │
|
|
|
+│ └──────────┘ └────┬─────┘ └────────┬─────────┘ │
|
|
|
+│ │ │ │
|
|
|
+│ ┌─────────────┼───────────────────┘ │
|
|
|
+│ │ │ │
|
|
|
+│ ┌──────┴───┐ ┌──────┴───┐ ┌──────────┐ ┌────────┐ │
|
|
|
+│ │ MySQL │ │ Milvus │ │ Polaris │ │ TEI │ │
|
|
|
+│ │ :3306 │ │ :19530 │ │ :8181 │ │ :7997 │ │
|
|
|
+│ └──────────┘ └──────────┘ └────┬─────┘ └────────┘ │
|
|
|
+│ │ │
|
|
|
+│ ┌──────┴──────┐ │
|
|
|
+│ │ Postgres │ │
|
|
|
+│ │ (internal) │ │
|
|
|
+│ └─────────────┘ │
|
|
|
+└─────────────────────────────────────────────────────────┘
|
|
|
+```
|
|
|
+
|
|
|
+### 端口一览
|
|
|
+
|
|
|
+| 服务 | 端口 | 说明 |
|
|
|
+|------|------|------|
|
|
|
+| Nginx(前端入口) | 80 | 前端静态文件 + 反向代理 `/api` `/ws` |
|
|
|
+| Spring Boot 后端 | 8080 | REST API |
|
|
|
+| Python Iceberg 服务 | 8090 | Iceberg 数据读写 |
|
|
|
+| MySQL | 3306 | 主业务数据库 |
|
|
|
+| Milvus | 19530 | 向量数据库 |
|
|
|
+| TEI Embedding | 7997 | 文本向量化(OpenAI 兼容) |
|
|
|
+| Apache Polaris | 8181 | Iceberg REST Catalog |
|
|
|
+
|
|
|
+> **k8s-orchestrator**(Spark/StarRocks 编排器)需要独立 Kubernetes 集群,见[第七章](#7-k8s-orchestrator-可选)。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第一章:系统初始化
|
|
|
+
|
|
|
+### 1.1 更新系统
|
|
|
+
|
|
|
+```bash
|
|
|
+sudo apt update && sudo apt upgrade -y
|
|
|
+sudo apt install -y ca-certificates curl gnupg lsb-release jq unzip tar git wget
|
|
|
+```
|
|
|
+
|
|
|
+### 1.2 配置服务器变量(后续步骤都会用到)
|
|
|
+
|
|
|
+```bash
|
|
|
+# 替换为你的服务器公网 IP(本机部署用 127.0.0.1)
|
|
|
+export SERVER_IP=<你的服务器公网IP>
|
|
|
+echo "export SERVER_IP=${SERVER_IP}" >> ~/.bashrc
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第二章:安装 Docker + Docker Compose
|
|
|
+
|
|
|
+```bash
|
|
|
+# 添加 Docker 官方 GPG 密钥
|
|
|
+sudo install -m 0755 -d /etc/apt/keyrings
|
|
|
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
|
|
|
+ | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
|
|
+sudo chmod a+r /etc/apt/keyrings/docker.gpg
|
|
|
+
|
|
|
+# 添加 apt 源
|
|
|
+echo \
|
|
|
+ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
|
|
|
+ https://download.docker.com/linux/ubuntu \
|
|
|
+ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
|
|
|
+ | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
|
|
+
|
|
|
+# 安装 Docker Engine
|
|
|
+sudo apt update
|
|
|
+sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
|
|
+sudo systemctl enable --now docker
|
|
|
+
|
|
|
+# 当前用户免 sudo 使用 Docker(需重新登录或执行 newgrp)
|
|
|
+sudo usermod -aG docker $USER
|
|
|
+newgrp docker
|
|
|
+
|
|
|
+# 验证
|
|
|
+docker version
|
|
|
+docker compose version
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第三章:获取代码
|
|
|
+
|
|
|
+```bash
|
|
|
+# 假设代码已在此路径,如需从 git 克隆请替换为实际仓库地址
|
|
|
+cd ~
|
|
|
+git clone <你的仓库地址> wenshu-platform
|
|
|
+cd wenshu-platform
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第四章:编写 Dockerfile
|
|
|
+
|
|
|
+在项目根目录下创建各服务的 Dockerfile。
|
|
|
+
|
|
|
+### 4.1 后端 Dockerfile
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > backend/Dockerfile <<'EOF'
|
|
|
+# ---- 构建阶段 ----
|
|
|
+FROM maven:3.9-eclipse-temurin-17 AS build
|
|
|
+WORKDIR /app
|
|
|
+COPY pom.xml .
|
|
|
+# 先下载依赖,利用 Docker 缓存层
|
|
|
+RUN mvn dependency:go-offline -q
|
|
|
+COPY src ./src
|
|
|
+RUN mvn package -DskipTests -q
|
|
|
+
|
|
|
+# ---- 运行阶段 ----
|
|
|
+FROM eclipse-temurin:17-jre-jammy
|
|
|
+WORKDIR /app
|
|
|
+COPY --from=build /app/target/*.jar app.jar
|
|
|
+EXPOSE 8080
|
|
|
+ENTRYPOINT ["java", "-jar", "app.jar"]
|
|
|
+EOF
|
|
|
+```
|
|
|
+
|
|
|
+### 4.2 Python Iceberg Dockerfile
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > python-iceberg/Dockerfile <<'EOF'
|
|
|
+FROM python:3.11-slim
|
|
|
+WORKDIR /app
|
|
|
+COPY requirements.txt .
|
|
|
+# openpyxl 在 requirements.txt 中缺失,手动补充
|
|
|
+RUN pip install --no-cache-dir -r requirements.txt openpyxl
|
|
|
+COPY app.py .
|
|
|
+EXPOSE 8090
|
|
|
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8090"]
|
|
|
+EOF
|
|
|
+```
|
|
|
+
|
|
|
+### 4.3 前端 Dockerfile(多阶段:Node 构建 → Nginx 服务)
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > frontend/Dockerfile <<'EOF'
|
|
|
+# ---- 构建阶段 ----
|
|
|
+FROM node:20-alpine AS build
|
|
|
+WORKDIR /app
|
|
|
+COPY package*.json ./
|
|
|
+RUN npm ci
|
|
|
+COPY . .
|
|
|
+RUN npm run build
|
|
|
+
|
|
|
+# ---- Nginx 服务阶段 ----
|
|
|
+FROM nginx:1.27-alpine
|
|
|
+COPY --from=build /app/dist /usr/share/nginx/html
|
|
|
+COPY nginx.conf /etc/nginx/conf.d/default.conf
|
|
|
+EXPOSE 80
|
|
|
+EOF
|
|
|
+```
|
|
|
+
|
|
|
+### 4.4 前端 Nginx 配置
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > frontend/nginx.conf <<'EOF'
|
|
|
+server {
|
|
|
+ listen 80;
|
|
|
+ server_name _;
|
|
|
+ client_max_body_size 500M;
|
|
|
+
|
|
|
+ root /usr/share/nginx/html;
|
|
|
+ index index.html;
|
|
|
+
|
|
|
+ # 前端 SPA 路由(刷新不 404)
|
|
|
+ location / {
|
|
|
+ try_files $uri $uri/ /index.html;
|
|
|
+ }
|
|
|
+
|
|
|
+ # 代理 REST API 到后端
|
|
|
+ location /api/ {
|
|
|
+ proxy_pass http://backend:8080;
|
|
|
+ proxy_set_header Host $host;
|
|
|
+ proxy_set_header X-Real-IP $remote_addr;
|
|
|
+ proxy_read_timeout 300s;
|
|
|
+ }
|
|
|
+
|
|
|
+ # 代理 WebSocket(SSH 终端)
|
|
|
+ location /ws {
|
|
|
+ proxy_pass http://backend:8080;
|
|
|
+ proxy_http_version 1.1;
|
|
|
+ proxy_set_header Upgrade $http_upgrade;
|
|
|
+ proxy_set_header Connection "upgrade";
|
|
|
+ proxy_set_header Host $host;
|
|
|
+ proxy_read_timeout 3600s;
|
|
|
+ }
|
|
|
+}
|
|
|
+EOF
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第五章:修改应用配置
|
|
|
+
|
|
|
+### 5.1 修改后端 application.yml
|
|
|
+
|
|
|
+编辑 `backend/src/main/resources/application.yml`,将所有外部服务地址改为 Docker 容器名(Docker Compose 内部通信):
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > backend/src/main/resources/application.yml <<'EOF'
|
|
|
+server:
|
|
|
+ port: 8080
|
|
|
+
|
|
|
+spring:
|
|
|
+ datasource:
|
|
|
+ url: jdbc:mysql://mysql:3306/wenshu_platform?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
|
|
|
+ username: wenshu
|
|
|
+ password: wenshu_db_pass
|
|
|
+ driver-class-name: com.mysql.cj.jdbc.Driver
|
|
|
+ sql:
|
|
|
+ init:
|
|
|
+ mode: never
|
|
|
+ jackson:
|
|
|
+ time-zone: Asia/Shanghai
|
|
|
+ servlet:
|
|
|
+ multipart:
|
|
|
+ max-file-size: 500MB
|
|
|
+ max-request-size: 500MB
|
|
|
+
|
|
|
+mybatis:
|
|
|
+ mapper-locations: classpath*:mapper/**/*.xml
|
|
|
+ configuration:
|
|
|
+ map-underscore-to-camel-case: true
|
|
|
+
|
|
|
+audit:
|
|
|
+ log:
|
|
|
+ query-max-page-size: 200
|
|
|
+ export-max-rows: 5000
|
|
|
+ retry-delay-ms: 10000
|
|
|
+
|
|
|
+alert:
|
|
|
+ rule:
|
|
|
+ query-max-page-size: 200
|
|
|
+ event:
|
|
|
+ query-max-page-size: 200
|
|
|
+ notify:
|
|
|
+ retry-times: 2
|
|
|
+ retry-interval-ms: 1000
|
|
|
+ prometheus:
|
|
|
+ rules-file: tmp/prometheus/alert-rules.yml
|
|
|
+
|
|
|
+logging:
|
|
|
+ level:
|
|
|
+ com.wenshu.platform: INFO
|
|
|
+
|
|
|
+stock:
|
|
|
+ ssh:
|
|
|
+ crypto-key: change-me-stock-ssh-key-32chars
|
|
|
+
|
|
|
+# ---- Polaris(Iceberg REST Catalog)----
|
|
|
+polaris:
|
|
|
+ host: http://polaris:8181
|
|
|
+ realm: POLARIS
|
|
|
+ client-id: root
|
|
|
+ client-secret: s3cr3t
|
|
|
+ default-catalog: demo_catalog
|
|
|
+ oss:
|
|
|
+ endpoint: https://oss-cn-hangzhou.aliyuncs.com # 替换为你的 OSS endpoint
|
|
|
+ access-key-id: <你的OSS AK> # 替换
|
|
|
+ access-key-secret: <你的OSS SK> # 替换
|
|
|
+ region: cn-hangzhou # 替换
|
|
|
+ bucket: <你的OSS Bucket> # 替换
|
|
|
+ cache-ttl-minutes: 30
|
|
|
+
|
|
|
+iceberg-service:
|
|
|
+ url: http://python-iceberg:8090
|
|
|
+
|
|
|
+# ---- RAG:Embedding + 向量检索 ----
|
|
|
+rag:
|
|
|
+ embedding:
|
|
|
+ dimension: 768
|
|
|
+ local-url: http://tei:80/v1
|
|
|
+ local-model: intfloat/multilingual-e5-base
|
|
|
+ milvus:
|
|
|
+ uri: http://milvus-standalone:19530
|
|
|
+ collection: knowledge_vectors
|
|
|
+ retrieval:
|
|
|
+ top-k: 8
|
|
|
+ score-threshold: 0.3
|
|
|
+EOF
|
|
|
+```
|
|
|
+
|
|
|
+> **注意**:`polaris.oss.*` 配置是真实 OSS 凭据,请替换为你自己的值,不要提交到 git。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第六章:一键 Docker Compose 部署
|
|
|
+
|
|
|
+在项目根目录创建主 `docker-compose.yml`:
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > docker-compose.yml <<'YAML'
|
|
|
+services:
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────hf-
|
|
|
+ # 1. MySQL 主数据库
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ mysql:
|
|
|
+ image: mysql:8.0
|
|
|
+ container_name: ws-mysql
|
|
|
+ restart: unless-stopped
|
|
|
+ environment:
|
|
|
+ TZ: Asia/Shanghai
|
|
|
+ MYSQL_ROOT_PASSWORD: root_pass_change_me
|
|
|
+ MYSQL_DATABASE: wenshu_platform
|
|
|
+ MYSQL_USER: wenshu
|
|
|
+ MYSQL_PASSWORD: wenshu_db_pass
|
|
|
+ volumes:
|
|
|
+ - ./data/mysql:/var/lib/mysql
|
|
|
+ - ./backend/src/main/resources/db/schema.sql:/docker-entrypoint-initdb.d/01-schema.sql:ro
|
|
|
+ ports:
|
|
|
+ - "3306:3306"
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "wenshu", "-pwenshu_db_pass"]
|
|
|
+ interval: 10s
|
|
|
+ timeout: 5s
|
|
|
+ retries: 10
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 2. Milvus 向量数据库(依赖 etcd + minio)
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ milvus-etcd:
|
|
|
+ image: quay.io/coreos/etcd:v3.5.18
|
|
|
+ container_name: ws-milvus-etcd
|
|
|
+ restart: unless-stopped
|
|
|
+ environment:
|
|
|
+ ETCD_AUTO_COMPACTION_MODE: revision
|
|
|
+ ETCD_AUTO_COMPACTION_RETENTION: "1000"
|
|
|
+ ETCD_QUOTA_BACKEND_BYTES: "4294967296"
|
|
|
+ ETCD_SNAPSHOT_COUNT: "50000"
|
|
|
+ volumes:
|
|
|
+ - ./data/milvus/etcd:/etcd
|
|
|
+ command: >
|
|
|
+ etcd
|
|
|
+ -advertise-client-urls=http://127.0.0.1:2379
|
|
|
+ -listen-client-urls=http://0.0.0.0:2379
|
|
|
+ --data-dir=/etcd
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD", "etcdctl", "endpoint", "health"]
|
|
|
+ interval: 30s
|
|
|
+ timeout: 20s
|
|
|
+ retries: 3
|
|
|
+
|
|
|
+ milvus-minio:
|
|
|
+ image: minio/minio:RELEASE.2023-03-20T20-16-18Z
|
|
|
+ container_name: ws-milvus-minio
|
|
|
+ restart: unless-stopped
|
|
|
+ environment:
|
|
|
+ MINIO_ACCESS_KEY: minioadmin
|
|
|
+ MINIO_SECRET_KEY: minioadmin
|
|
|
+ volumes:
|
|
|
+ - ./data/milvus/minio:/minio_data
|
|
|
+ command: minio server /minio_data --console-address ":9001"
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
|
|
+ interval: 30s
|
|
|
+ timeout: 20s
|
|
|
+ retries: 3
|
|
|
+
|
|
|
+ milvus-standalone:
|
|
|
+ image: milvusdb/milvus:v2.4.15
|
|
|
+ container_name: ws-milvus-standalone
|
|
|
+ restart: unless-stopped
|
|
|
+ depends_on:
|
|
|
+ milvus-etcd:
|
|
|
+ condition: service_healthy
|
|
|
+ milvus-minio:
|
|
|
+ condition: service_healthy
|
|
|
+ environment:
|
|
|
+ ETCD_ENDPOINTS: milvus-etcd:2379
|
|
|
+ MINIO_ADDRESS: milvus-minio:9000
|
|
|
+ volumes:
|
|
|
+ - ./data/milvus/milvus:/var/lib/milvus
|
|
|
+ ports:
|
|
|
+ - "19530:19530"
|
|
|
+ - "9091:9091"
|
|
|
+ command: ["milvus", "run", "standalone"]
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
|
|
|
+ interval: 30s
|
|
|
+ timeout: 20s
|
|
|
+ retries: 5
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 3. Apache Polaris(Iceberg REST Catalog)
|
|
|
+ # 依赖专属 Postgres
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ polaris-postgres:
|
|
|
+ image: postgres:16
|
|
|
+ container_name: ws-polaris-postgres
|
|
|
+ restart: unless-stopped
|
|
|
+ environment:
|
|
|
+ POSTGRES_DB: polaris
|
|
|
+ POSTGRES_USER: polaris
|
|
|
+ POSTGRES_PASSWORD: polaris_pg_pass
|
|
|
+ volumes:
|
|
|
+ - ./data/polaris/postgres:/var/lib/postgresql/data
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD-SHELL", "pg_isready -U polaris -d polaris"]
|
|
|
+ interval: 10s
|
|
|
+ timeout: 5s
|
|
|
+ retries: 10
|
|
|
+
|
|
|
+ polaris:
|
|
|
+ image: apache/polaris:1.3.0-incubating
|
|
|
+ container_name: ws-polaris
|
|
|
+ restart: unless-stopped
|
|
|
+ depends_on:
|
|
|
+ polaris-postgres:
|
|
|
+ condition: service_healthy
|
|
|
+ ports:
|
|
|
+ - "8181:8181"
|
|
|
+ environment:
|
|
|
+ POLARIS_PERSISTENCE_TYPE: relational-jdbc
|
|
|
+ QUARKUS_DATASOURCE_USERNAME: polaris
|
|
|
+ QUARKUS_DATASOURCE_PASSWORD: polaris_pg_pass
|
|
|
+ QUARKUS_DATASOURCE_JDBC_URL: jdbc:postgresql://polaris-postgres:5432/polaris
|
|
|
+ QUARKUS_HTTP_PORT: 8181
|
|
|
+ AWS_ACCESS_KEY_ID: ${OSS_AK:-}
|
|
|
+ AWS_SECRET_ACCESS_KEY: ${OSS_SK:-}
|
|
|
+ AWS_REQUEST_CHECKSUM_CALCULATION: WHEN_REQUIRED
|
|
|
+ AWS_RESPONSE_CHECKSUM_VALIDATION: WHEN_REQUIRED
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD-SHELL", "bash -c '</dev/tcp/localhost/8181' 2>/dev/null && exit 0 || exit 1"]
|
|
|
+ interval: 15s
|
|
|
+ timeout: 5s
|
|
|
+ retries: 20
|
|
|
+ start_period: 60s
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 4. TEI 向量化服务(CPU 模式)
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ tei:
|
|
|
+ image: ghcr.io/huggingface/text-embeddings-inference:cpu-latest
|
|
|
+ container_name: ws-tei
|
|
|
+ restart: unless-stopped
|
|
|
+ volumes:
|
|
|
+ - ./data/huggingface:/data
|
|
|
+ ports:
|
|
|
+ - "7997:80"
|
|
|
+ command: ["--model-id", "intfloat/multilingual-e5-base"]
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD", "curl", "-sf", "http://localhost:80/health"]
|
|
|
+ interval: 30s
|
|
|
+ timeout: 10s
|
|
|
+ retries: 10
|
|
|
+ start_period: 120s # 首次下载模型需要时间
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 5. Spring Boot 后端
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ backend:
|
|
|
+ build:
|
|
|
+ context: ./backend
|
|
|
+ dockerfile: Dockerfile
|
|
|
+ container_name: ws-backend
|
|
|
+ restart: unless-stopped
|
|
|
+ environment:
|
|
|
+ TZ: Asia/Shanghai
|
|
|
+ depends_on:
|
|
|
+ mysql:
|
|
|
+ condition: service_healthy
|
|
|
+ milvus-standalone:
|
|
|
+ condition: service_healthy
|
|
|
+ ports:
|
|
|
+ - "8080:8080"
|
|
|
+ healthcheck:
|
|
|
+ test: ["CMD-SHELL", "bash -c '</dev/tcp/localhost/8080' 2>/dev/null && exit 0 || exit 1"]
|
|
|
+ interval: 15s
|
|
|
+ timeout: 5s
|
|
|
+ retries: 20
|
|
|
+ start_period: 90s
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 6. Python Iceberg 服务
|
|
|
+ # 启动时会调用 backend 获取配置,必须在 backend 之后启动
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ python-iceberg:
|
|
|
+ build:
|
|
|
+ context: ./python-iceberg
|
|
|
+ dockerfile: Dockerfile
|
|
|
+ container_name: ws-python-iceberg
|
|
|
+ restart: unless-stopped
|
|
|
+ depends_on:
|
|
|
+ backend:
|
|
|
+ condition: service_healthy
|
|
|
+ polaris:
|
|
|
+ condition: service_healthy
|
|
|
+ ports:
|
|
|
+ - "8090:8090"
|
|
|
+ environment:
|
|
|
+ BACKEND_URL: http://backend:8080
|
|
|
+
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ # 7. 前端(Nginx)
|
|
|
+ # ──────────────────────────────────────────
|
|
|
+ frontend:
|
|
|
+ build:
|
|
|
+ context: ./frontend
|
|
|
+ dockerfile: Dockerfile
|
|
|
+ container_name: ws-frontend
|
|
|
+ restart: unless-stopped
|
|
|
+ depends_on:
|
|
|
+ - backend
|
|
|
+ ports:
|
|
|
+ - "80:80"
|
|
|
+
|
|
|
+YAML
|
|
|
+```
|
|
|
+
|
|
|
+### 6.1 创建数据目录
|
|
|
+
|
|
|
+```bash
|
|
|
+mkdir -p data/mysql data/milvus/etcd data/milvus/minio data/milvus/milvus \
|
|
|
+ data/polaris/postgres data/huggingface
|
|
|
+```
|
|
|
+
|
|
|
+### 6.2 创建 .env 文件(OSS 凭据)
|
|
|
+
|
|
|
+```bash
|
|
|
+cat > .env <<'ENV'
|
|
|
+# OSS 凭据(替换为你的真实值)
|
|
|
+OSS_AK=<你的阿里云OSS AccessKeyId>
|
|
|
+OSS_SK=<你的阿里云OSS AccessKeySecret>
|
|
|
+ENV
|
|
|
+```
|
|
|
+
|
|
|
+### 6.3 一键启动所有服务
|
|
|
+
|
|
|
+```bash
|
|
|
+docker compose up -d --build
|
|
|
+
|
|
|
+# 查看启动状态
|
|
|
+docker compose ps
|
|
|
+
|
|
|
+# 查看日志(如遇问题)
|
|
|
+docker compose logs -f backend
|
|
|
+docker compose logs -f python-iceberg
|
|
|
+```
|
|
|
+
|
|
|
+> **首次启动说明**:
|
|
|
+> - TEI 会自动下载约 278MB 的模型(`intfloat/multilingual-e5-base`),需等待 1-3 分钟
|
|
|
+> - Backend 镜像构建(Maven 编译)约需 3-5 分钟
|
|
|
+> - Frontend 镜像构建(npm build)约需 1-2 分钟
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第七章:初始化 Polaris
|
|
|
+
|
|
|
+Polaris 首次启动后需要初始化 realm 和 root 管理凭据(**只需执行一次**)。
|
|
|
+
|
|
|
+```bash
|
|
|
+# 等待 Polaris 完全启动
|
|
|
+docker compose logs -f polaris | grep -m1 "started"
|
|
|
+
|
|
|
+# 加载 OSS 凭据
|
|
|
+source .env
|
|
|
+
|
|
|
+# 获取 Docker 网络名(格式:目录名_default)
|
|
|
+NET="$(basename "$PWD")_default"
|
|
|
+
|
|
|
+# 初始化 POLARIS realm 并创建 root 客户端
|
|
|
+docker run --rm --network "$NET" \
|
|
|
+ --env="polaris.persistence.type=relational-jdbc" \
|
|
|
+ --env="quarkus.datasource.username=polaris" \
|
|
|
+ --env="quarkus.datasource.password=polaris_pg_pass" \
|
|
|
+ --env="quarkus.datasource.jdbc.url=jdbc:postgresql://polaris-postgres:5432/polaris" \
|
|
|
+ apache/polaris-admin-tool:1.3.0-incubating \
|
|
|
+ bootstrap -r "POLARIS" -c "POLARIS,root,s3cr3t"
|
|
|
+
|
|
|
+echo "Polaris 初始化完成"
|
|
|
+```
|
|
|
+
|
|
|
+### 7.1 验证 Polaris
|
|
|
+
|
|
|
+```bash
|
|
|
+# 获取 token
|
|
|
+curl -s -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
|
|
|
+ -H "Polaris-Realm: POLARIS" \
|
|
|
+ -H "Content-Type: application/x-www-form-urlencoded" \
|
|
|
+ --data "grant_type=client_credentials&client_id=root&client_secret=s3cr3t&scope=PRINCIPAL_ROLE:ALL" \
|
|
|
+ | jq .access_token
|
|
|
+```
|
|
|
+
|
|
|
+返回非空 token 字符串即表示 Polaris 初始化成功。
|
|
|
+
|
|
|
+### 7.2 创建默认 Catalog
|
|
|
+
|
|
|
+> 必须创建 catalog 后,平台才能正常创建 database/table。
|
|
|
+
|
|
|
+```bash
|
|
|
+# 获取 token
|
|
|
+TOKEN=$(curl -s -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \
|
|
|
+ -H "Polaris-Realm: POLARIS" \
|
|
|
+ -H "Content-Type: application/x-www-form-urlencoded" \
|
|
|
+ --data "grant_type=client_credentials&client_id=root&client_secret=s3cr3t&scope=PRINCIPAL_ROLE:ALL" \
|
|
|
+ | jq -r '.access_token')
|
|
|
+
|
|
|
+# 创建 demo_catalog(替换 <你的OSS Bucket> 为实际值,需与 application.yml 中 polaris.oss.bucket 一致)
|
|
|
+curl -s -X POST http://localhost:8181/api/management/v1/catalogs \
|
|
|
+ -H "Authorization: Bearer $TOKEN" \
|
|
|
+ -H "Polaris-Realm: POLARIS" \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ -d '{
|
|
|
+ "catalog": {
|
|
|
+ "type": "INTERNAL",
|
|
|
+ "name": "demo_catalog",
|
|
|
+ "properties": {
|
|
|
+ "default-base-location": "s3://<你的OSS Bucket>/demo_catalog"
|
|
|
+ },
|
|
|
+ "storageConfigInfo": {
|
|
|
+ "storageType": "S3",
|
|
|
+ "allowedLocations": ["s3://<你的OSS Bucket>/demo_catalog"],
|
|
|
+ "region": "cn-hangzhou",
|
|
|
+ "endpoint": "https://oss-cn-hangzhou.aliyuncs.com",
|
|
|
+ "pathStyleAccess": false,
|
|
|
+ "stsUnavailable": true
|
|
|
+ }
|
|
|
+ }
|
|
|
+ }' | jq .
|
|
|
+
|
|
|
+# 验证
|
|
|
+curl -s http://localhost:8181/api/management/v1/catalogs \
|
|
|
+ -H "Authorization: Bearer $TOKEN" \
|
|
|
+ -H "Polaris-Realm: POLARIS" | jq '.catalogs[].name'
|
|
|
+# 应输出 "demo_catalog"
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第八章:验证所有服务
|
|
|
+
|
|
|
+```bash
|
|
|
+echo "=== MySQL ===" && \
|
|
|
+ docker exec ws-mysql mysqladmin ping -h localhost -u wenshu -pwenshu_db_pass --silent && echo "OK"
|
|
|
+
|
|
|
+echo "=== Milvus ===" && \
|
|
|
+ curl -sf http://localhost:9091/healthz && echo "OK"
|
|
|
+
|
|
|
+echo "=== TEI Embedding ===" && \
|
|
|
+ curl -s http://localhost:7997/v1/embeddings \
|
|
|
+ -H "Content-Type: application/json" \
|
|
|
+ -d '{"model":"intfloat/multilingual-e5-base","input":"passage: 测试文本"}' \
|
|
|
+ | jq '.data[0].embedding | length'
|
|
|
+ # 应输出 768
|
|
|
+
|
|
|
+echo "=== Polaris ===" && \
|
|
|
+ curl -sf http://localhost:8181/api/catalog/v1/config && echo "OK"
|
|
|
+
|
|
|
+echo "=== Backend ===" && \
|
|
|
+ curl -sf http://localhost:8080/api/auth/login -X POST -H "Content-Type: application/json" \
|
|
|
+ -d '{}' | jq . # 返回 400/422 即表示后端已启动(无需登录成功)
|
|
|
+
|
|
|
+echo "=== Python Iceberg ===" && \
|
|
|
+ curl -sf http://localhost:8090/health && echo "OK"
|
|
|
+
|
|
|
+echo "=== Frontend ===" && \
|
|
|
+ curl -sf http://localhost/ | grep -q "wenshu" && echo "OK"
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第九章:防火墙开放端口
|
|
|
+
|
|
|
+如果使用云服务器,还需在**云安全组**放行以下端口(根据实际需求选择):
|
|
|
+
|
|
|
+```bash
|
|
|
+# Ubuntu ufw 防火墙
|
|
|
+sudo ufw allow 80/tcp # 前端 HTTP
|
|
|
+sudo ufw allow 8080/tcp # 后端 API(如需外部直接访问)
|
|
|
+sudo ufw allow 8181/tcp # Polaris(如需外部访问)
|
|
|
+sudo ufw allow 19530/tcp # Milvus(如需外部访问)
|
|
|
+sudo ufw allow 7997/tcp # TEI(如需外部访问)
|
|
|
+sudo ufw status
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 第十章:k8s-orchestrator(可选)
|
|
|
+
|
|
|
+> k8s-orchestrator 管理 Spark/StarRocks 集群的编排,需要独立 Kubernetes 集群。如不使用集群编排功能,可跳过本章。
|
|
|
+
|
|
|
+### 10.1 安装 Kubernetes(单控制平面节点)
|
|
|
+
|
|
|
+所有节点执行:
|
|
|
+
|
|
|
+```bash
|
|
|
+# 关闭 swap
|
|
|
+sudo swapoff -a
|
|
|
+sudo sed -ri '/\sswap\s/s/^#?/#/' /etc/fstab
|
|
|
+
|
|
|
+# 内核参数
|
|
|
+cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
|
|
|
+overlay
|
|
|
+br_netfilter
|
|
|
+EOF
|
|
|
+sudo modprobe overlay && sudo modprobe br_netfilter
|
|
|
+
|
|
|
+cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
|
|
|
+net.bridge.bridge-nf-call-iptables = 1
|
|
|
+net.bridge.bridge-nf-call-ip6tables = 1
|
|
|
+net.ipv4.ip_forward = 1
|
|
|
+EOF
|
|
|
+sudo sysctl --system
|
|
|
+
|
|
|
+# 安装 containerd
|
|
|
+sudo apt-get install -y containerd
|
|
|
+sudo mkdir -p /etc/containerd
|
|
|
+containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
|
|
|
+sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
|
|
|
+sudo systemctl restart containerd && sudo systemctl enable containerd
|
|
|
+
|
|
|
+# 安装 kubeadm / kubelet / kubectl
|
|
|
+export K8S_MINOR=v1.30
|
|
|
+sudo mkdir -p /etc/apt/keyrings
|
|
|
+curl -fsSL "https://pkgs.k8s.io/core:/stable:/${K8S_MINOR}/deb/Release.key" \
|
|
|
+ | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
|
|
|
+echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
|
|
|
+ https://pkgs.k8s.io/core:/stable:/${K8S_MINOR}/deb/ /" \
|
|
|
+ | sudo tee /etc/apt/sources.list.d/kubernetes.list
|
|
|
+sudo apt-get update
|
|
|
+sudo apt-get install -y kubelet kubeadm kubectl
|
|
|
+sudo apt-mark hold kubelet kubeadm kubectl
|
|
|
+sudo systemctl enable --now kubelet
|
|
|
+```
|
|
|
+
|
|
|
+控制平面节点执行:
|
|
|
+
|
|
|
+```bash
|
|
|
+sudo kubeadm init --pod-network-cidr=192.168.0.0/16
|
|
|
+
|
|
|
+mkdir -p $HOME/.kube
|
|
|
+sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
|
|
+sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
|
|
+
|
|
|
+# 安装 CNI(Calico)
|
|
|
+kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml
|
|
|
+
|
|
|
+kubectl get nodes
|
|
|
+```
|
|
|
+
|
|
|
+### 10.2 安装 Spark + StarRocks Operator
|
|
|
+
|
|
|
+```bash
|
|
|
+# Spark Operator
|
|
|
+helm repo add spark-operator https://kubeflow.github.io/spark-operator
|
|
|
+helm repo update
|
|
|
+helm upgrade --install spark-operator spark-operator/spark-operator \
|
|
|
+ --namespace spark-operator --create-namespace
|
|
|
+
|
|
|
+# StarRocks Operator
|
|
|
+kubectl apply -f https://raw.githubusercontent.com/StarRocks/starrocks-kubernetes-operator/main/deploy/starrocks.com_starrocksclusters.yaml
|
|
|
+kubectl apply -f https://raw.githubusercontent.com/StarRocks/starrocks-kubernetes-operator/main/deploy/operator.yaml
|
|
|
+```
|
|
|
+
|
|
|
+### 10.3 安装 Go 1.22
|
|
|
+
|
|
|
+```bash
|
|
|
+wget https://go.dev/dl/go1.22.12.linux-amd64.tar.gz
|
|
|
+sudo tar -C /usr/local -xzf go1.22.12.linux-amd64.tar.gz
|
|
|
+echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
|
|
|
+source ~/.bashrc
|
|
|
+go version
|
|
|
+```
|
|
|
+
|
|
|
+### 10.4 构建并部署 k8s-orchestrator
|
|
|
+
|
|
|
+```bash
|
|
|
+cd ~/wenshu-platform/k8s-orchestrator
|
|
|
+
|
|
|
+# 写 Dockerfile(API 服务)
|
|
|
+cat > Dockerfile.api <<'EOF'
|
|
|
+FROM golang:1.22 AS build
|
|
|
+WORKDIR /src
|
|
|
+COPY go.mod go.sum ./
|
|
|
+RUN go mod download
|
|
|
+COPY . .
|
|
|
+RUN CGO_ENABLED=0 GOOS=linux go build -o /out/orchestrator-api ./cmd/orchestrator-api
|
|
|
+
|
|
|
+FROM gcr.io/distroless/static-debian12:nonroot
|
|
|
+COPY --from=build /out/orchestrator-api /orchestrator-api
|
|
|
+ENTRYPOINT ["/orchestrator-api"]
|
|
|
+EOF
|
|
|
+
|
|
|
+# 写 Dockerfile(Controller)
|
|
|
+cat > Dockerfile.controller <<'EOF'
|
|
|
+FROM golang:1.22 AS build
|
|
|
+WORKDIR /src
|
|
|
+COPY go.mod go.sum ./
|
|
|
+RUN go mod download
|
|
|
+COPY . .
|
|
|
+RUN CGO_ENABLED=0 GOOS=linux go build -o /out/orchestrator-controller ./cmd/orchestrator-controller
|
|
|
+
|
|
|
+FROM gcr.io/distroless/static-debian12:nonroot
|
|
|
+COPY --from=build /out/orchestrator-controller /orchestrator-controller
|
|
|
+ENTRYPOINT ["/orchestrator-controller"]
|
|
|
+EOF
|
|
|
+
|
|
|
+# 替换为你的镜像仓库
|
|
|
+export REGISTRY=<your-registry>
|
|
|
+export TAG=v1.0.0
|
|
|
+
|
|
|
+docker build -f Dockerfile.api -t ${REGISTRY}/ws-orchestrator-api:${TAG} .
|
|
|
+docker build -f Dockerfile.controller -t ${REGISTRY}/ws-orchestrator-controller:${TAG} .
|
|
|
+docker push ${REGISTRY}/ws-orchestrator-api:${TAG}
|
|
|
+docker push ${REGISTRY}/ws-orchestrator-controller:${TAG}
|
|
|
+
|
|
|
+# 部署到 K8s
|
|
|
+kubectl apply -f config/samples/namespace.yaml
|
|
|
+kubectl apply -k config/crd
|
|
|
+kubectl apply -k config/rbac
|
|
|
+
|
|
|
+JOIN_CMD="$(kubeadm token create --print-join-command)"
|
|
|
+kubectl -n orchestrator-system create secret generic kubeadm-join \
|
|
|
+ --from-literal=joinCommand="${JOIN_CMD}" \
|
|
|
+ --dry-run=client -o yaml | kubectl apply -f -
|
|
|
+
|
|
|
+kubectl apply -f config/samples/deployment.yaml
|
|
|
+kubectl -n orchestrator-system set image deploy/spark-orchestrator-api \
|
|
|
+ api=${REGISTRY}/ws-orchestrator-api:${TAG}
|
|
|
+kubectl -n orchestrator-system set image deploy/spark-orchestrator-controller \
|
|
|
+ controller=${REGISTRY}/ws-orchestrator-controller:${TAG}
|
|
|
+
|
|
|
+# 验证
|
|
|
+kubectl -n orchestrator-system get pods
|
|
|
+kubectl -n orchestrator-system rollout status deploy/spark-orchestrator-api
|
|
|
+```
|
|
|
+
|
|
|
+然后将 orchestrator 的 API 地址配置到 Spring Boot 的 `application.yml`(如有相关配置项)。
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 附录 A:常用运维命令
|
|
|
+
|
|
|
+```bash
|
|
|
+# 查看所有服务状态
|
|
|
+docker compose ps
|
|
|
+
|
|
|
+# 重启单个服务
|
|
|
+docker compose restart backend
|
|
|
+
|
|
|
+# 查看实时日志
|
|
|
+docker compose logs -f <service-name>
|
|
|
+
|
|
|
+# 停止所有服务(保留数据)
|
|
|
+docker compose down
|
|
|
+
|
|
|
+# 停止并删除所有数据(谨慎!)
|
|
|
+docker compose down -v
|
|
|
+
|
|
|
+# 源代码更新后重新部署
|
|
|
+git pull # 拉取最新代码
|
|
|
+docker compose up -d --build backend # 后端代码改了
|
|
|
+docker compose up -d --build frontend # 前端代码改了
|
|
|
+docker compose up -d --build python-iceberg # Python 代码改了
|
|
|
+docker compose up -d --build backend frontend # 多个一起重建
|
|
|
+docker compose up -d --build # 全部重建
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 附录 B:配置修改速查
|
|
|
+
|
|
|
+| 需要修改的值 | 文件 | 字段 |
|
|
|
+|---|---|---|
|
|
|
+| OSS AK/SK | `.env` | `OSS_AK`, `OSS_SK` |
|
|
|
+| OSS AK/SK(后端) | `backend/src/main/resources/application.yml` | `polaris.oss.access-key-id/secret` |
|
|
|
+| OSS Bucket | `application.yml` | `polaris.oss.bucket` |
|
|
|
+| OSS Region/Endpoint | `application.yml` | `polaris.oss.region/endpoint` |
|
|
|
+| MySQL 密码 | `docker-compose.yml` + `application.yml` | 两处保持一致 |
|
|
|
+| Polaris root 密码 | `application.yml` + 第七章初始化命令 | `polaris.client-secret` |
|
|
|
+| SSH 加密密钥 | `application.yml` | `stock.ssh.crypto-key` |
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 附录 C:TEI GPU 加速(可选)
|
|
|
+
|
|
|
+如果服务器有 NVIDIA GPU(驱动 ≥ 525),将 `docker-compose.yml` 中的 `tei` 服务替换为:
|
|
|
+
|
|
|
+```yaml
|
|
|
+ tei:
|
|
|
+ image: ghcr.io/huggingface/text-embeddings-inference:latest # GPU 版
|
|
|
+ container_name: ws-tei
|
|
|
+ restart: unless-stopped
|
|
|
+ deploy:
|
|
|
+ resources:
|
|
|
+ reservations:
|
|
|
+ devices:
|
|
|
+ - driver: nvidia
|
|
|
+ count: 1
|
|
|
+ capabilities: [gpu]
|
|
|
+ volumes:
|
|
|
+ - ./data/huggingface:/data
|
|
|
+ ports:
|
|
|
+ - "7997:80"
|
|
|
+ command: ["--model-id", "intfloat/multilingual-e5-base"]
|
|
|
+```
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## 附录 D:完整启动顺序参考
|
|
|
+
|
|
|
+当 `docker compose up -d` 执行后,服务按如下顺序健康就绪:
|
|
|
+
|
|
|
+```
|
|
|
+mysql (healthy)
|
|
|
+ └── backend (healthy)
|
|
|
+ └── python-iceberg (started)
|
|
|
+
|
|
|
+milvus-etcd (healthy) ──┐
|
|
|
+milvus-minio (healthy) ─┴── milvus-standalone (healthy)
|
|
|
+
|
|
|
+polaris-postgres (healthy) ── polaris (healthy)
|
|
|
+
|
|
|
+tei (下载模型后 healthy)
|
|
|
+
|
|
|
+frontend (no dependency, starts immediately)
|
|
|
+```
|