docs: add reviewed product and architecture specifications
This commit is contained in:
commit
069c1964e5
15
.editorconfig
Normal file
15
.editorconfig
Normal file
@ -0,0 +1,15 @@
|
||||
root = true
|
||||
|
||||
[*]
|
||||
charset = utf-8
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
indent_style = space
|
||||
indent_size = 2
|
||||
trim_trailing_whitespace = true
|
||||
|
||||
[*.py]
|
||||
indent_size = 4
|
||||
|
||||
[Makefile]
|
||||
indent_style = tab
|
||||
45
.env.example
Normal file
45
.env.example
Normal file
@ -0,0 +1,45 @@
|
||||
APP_ENV=development
|
||||
APP_NAME=enterprise-knowledge-platform
|
||||
LOG_LEVEL=INFO
|
||||
API_HOST=0.0.0.0
|
||||
API_PORT=8000
|
||||
|
||||
DATABASE_URL=postgresql+psycopg://ekp:ekp@localhost:5432/ekp
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
RABBITMQ_URL=amqp://guest:guest@localhost:5672//
|
||||
|
||||
OBJECT_STORAGE_PROVIDER=s3
|
||||
OBJECT_STORAGE_ENDPOINT=http://localhost:9000
|
||||
OBJECT_STORAGE_ACCESS_KEY=
|
||||
OBJECT_STORAGE_SECRET_KEY=
|
||||
OBJECT_STORAGE_BUCKET=ekp
|
||||
|
||||
VECTOR_STORE_PROVIDER=mochow
|
||||
MOCHOW_ENDPOINT=
|
||||
MOCHOW_API_KEY=
|
||||
MOCHOW_DATABASE=
|
||||
MOCHOW_TABLE=
|
||||
|
||||
EMBEDDING_PROVIDER=qianfan
|
||||
EMBEDDING_MODEL=bge-large-zh
|
||||
EMBEDDING_ENDPOINT=
|
||||
EMBEDDING_API_KEY=
|
||||
|
||||
RERANKER_PROVIDER=qianfan
|
||||
RERANKER_MODEL=
|
||||
RERANKER_ENDPOINT=
|
||||
RERANKER_API_KEY=
|
||||
|
||||
LLM_PROVIDER=qianfan
|
||||
LLM_MODEL=
|
||||
LLM_ENDPOINT=
|
||||
LLM_API_KEY=
|
||||
|
||||
PARSER_PROVIDER=mineru
|
||||
OCR_PROVIDER=mineru
|
||||
|
||||
JWT_SECRET=
|
||||
API_KEY_PEPPER=
|
||||
|
||||
ENABLE_KNOWLEDGE_COMPILATION=true
|
||||
ENABLE_MCP=true
|
||||
17
.gitattributes
vendored
Normal file
17
.gitattributes
vendored
Normal file
@ -0,0 +1,17 @@
|
||||
* text=auto eol=lf
|
||||
*.sh text eol=lf
|
||||
*.py text eol=lf
|
||||
*.md text eol=lf
|
||||
*.yaml text eol=lf
|
||||
*.yml text eol=lf
|
||||
*.json text eol=lf
|
||||
*.toml text eol=lf
|
||||
*.ts text eol=lf
|
||||
*.tsx text eol=lf
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
*.jpeg binary
|
||||
*.pdf binary
|
||||
*.docx binary
|
||||
*.pptx binary
|
||||
*.xlsx binary
|
||||
60
.gitignore
vendored
Normal file
60
.gitignore
vendored
Normal file
@ -0,0 +1,60 @@
|
||||
# Environment / secrets
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.pyright/
|
||||
.ruff_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# Node / frontend
|
||||
node_modules/
|
||||
dist/
|
||||
build/
|
||||
.next/
|
||||
coverage/
|
||||
*.tsbuildinfo
|
||||
|
||||
# IDE / OS
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Runtime / local data
|
||||
logs/
|
||||
tmp/
|
||||
data/
|
||||
storage/
|
||||
uploads/
|
||||
models/
|
||||
artifacts/
|
||||
reports/
|
||||
|
||||
# Secrets / credentials
|
||||
*.pem
|
||||
*.key
|
||||
*.p12
|
||||
credentials.json
|
||||
secrets.json
|
||||
|
||||
# Model artifacts
|
||||
*.bin
|
||||
*.safetensors
|
||||
*.onnx
|
||||
*.pt
|
||||
*.pth
|
||||
|
||||
# Local DB
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
46
AGENTS.md
Normal file
46
AGENTS.md
Normal file
@ -0,0 +1,46 @@
|
||||
# Enterprise Knowledge Platform — Codex Rules
|
||||
|
||||
## Read first
|
||||
|
||||
Before modifying code, read the documents relevant to the task:
|
||||
|
||||
- `docs/product/PRD.md`
|
||||
- `docs/architecture/ARCHITECTURE.md`
|
||||
- `docs/architecture/ADR.md`
|
||||
- `docs/api/API_CONTRACT.md`
|
||||
- `docs/api/MCP_CONTRACT.md`
|
||||
- `docs/plans/V1_EXECUTION_PLAN.md`
|
||||
- `docs/plans/TASKS.md`
|
||||
- `docs/development/DEFINITION_OF_DONE.md`
|
||||
|
||||
Do not redesign accepted architecture unless the task explicitly asks for an architecture change.
|
||||
|
||||
## Architecture invariants
|
||||
|
||||
1. API First.
|
||||
2. V1 uses a modular monolith plus independent async/GPU workers and an MCP gateway.
|
||||
3. PostgreSQL + object storage hold canonical data.
|
||||
4. Vector databases are rebuildable indexes, not sources of truth.
|
||||
5. Business services depend on `VectorStore` abstractions, never directly on Mochow.
|
||||
6. Embedding, reranking, parsing, OCR, object storage, and LLMs use provider interfaces.
|
||||
7. Original source documents are immutable.
|
||||
8. Compiled Knowledge is derived and traceable to source document versions/chunks.
|
||||
9. ACL is resolved before or during retrieval. Never retrieve global Top-K and filter afterward.
|
||||
10. MCP exposes business capabilities, not infrastructure/provider details.
|
||||
11. Index profiles are versioned; model/chunk changes use shadow indexes before activation.
|
||||
12. Do not split V1 into unnecessary microservices.
|
||||
13. Never hardcode credentials or production endpoints.
|
||||
14. Core retrieval changes require tests and RAG regression evaluation.
|
||||
|
||||
## Task discipline
|
||||
|
||||
- Work on one Task ID at a time.
|
||||
- Keep scope limited to the task.
|
||||
- Do not opportunistically implement future tasks.
|
||||
- Update relevant contracts/docs when behavior changes.
|
||||
|
||||
## Before finishing
|
||||
|
||||
Run relevant checks: format, lint, typecheck, unit tests, relevant integration tests, migration checks, API/MCP contract checks, and RAG evaluation for retrieval changes.
|
||||
|
||||
Report changed files, tests/results, unresolved risks, and follow-up tasks.
|
||||
19
CODEX_START_HERE.md
Normal file
19
CODEX_START_HERE.md
Normal file
@ -0,0 +1,19 @@
|
||||
# Codex Start Here
|
||||
|
||||
Do not ask Codex to build all of V1 in one task.
|
||||
|
||||
Recommended first prompt:
|
||||
|
||||
> You are taking over the Enterprise Knowledge Platform repository. Do not modify code yet. Read AGENTS.md and the product, architecture, API/MCP, execution-plan, task and Definition-of-Done documents it references. Inspect the repository. Then output: (1) current implementation status, (2) gap analysis against V1, (3) top five technical risks, (4) next 1–3 recommended Task IDs, and (5) blockers. Do not redesign accepted ADRs.
|
||||
|
||||
Preferred cadence:
|
||||
|
||||
1. Plan one task.
|
||||
2. Review the plan.
|
||||
3. Implement one task.
|
||||
4. Run validation.
|
||||
5. Self-review as a PR.
|
||||
6. Commit / open PR.
|
||||
7. Move to the next task.
|
||||
|
||||
One Task = one branch = one PR whenever possible.
|
||||
35
Makefile
Normal file
35
Makefile
Normal file
@ -0,0 +1,35 @@
|
||||
.PHONY: help install format lint typecheck test test-unit test-integration eval up down
|
||||
|
||||
help:
|
||||
@echo "Enterprise Knowledge Platform"
|
||||
@echo " install Install development dependencies"
|
||||
@echo " format Format code"
|
||||
@echo " lint Run linters"
|
||||
@echo " typecheck Run static type checks"
|
||||
@echo " test Run default tests"
|
||||
@echo " test-unit Run unit tests"
|
||||
@echo " test-integration Run integration tests"
|
||||
@echo " eval Run RAG evaluation"
|
||||
@echo " up Start local dependencies"
|
||||
@echo " down Stop local dependencies"
|
||||
|
||||
install:
|
||||
@echo "TODO: implement after toolchain selection"
|
||||
format:
|
||||
@echo "TODO: wire formatter"
|
||||
lint:
|
||||
@echo "TODO: wire linter"
|
||||
typecheck:
|
||||
@echo "TODO: wire type checker"
|
||||
test:
|
||||
@echo "TODO: wire test suite"
|
||||
test-unit:
|
||||
@echo "TODO: wire unit tests"
|
||||
test-integration:
|
||||
@echo "TODO: wire integration tests"
|
||||
eval:
|
||||
@echo "TODO: wire RAG benchmark"
|
||||
up:
|
||||
docker compose -f docker-compose.dev.yml up -d
|
||||
down:
|
||||
docker compose -f docker-compose.dev.yml down
|
||||
36
README.md
Normal file
36
README.md
Normal file
@ -0,0 +1,36 @@
|
||||
# Enterprise Knowledge Platform
|
||||
|
||||
Enterprise internal knowledge infrastructure.
|
||||
|
||||
## Core capabilities
|
||||
|
||||
- Multi-format document ingestion
|
||||
- OCR and document parsing
|
||||
- Structure-aware chunking
|
||||
- Multiple embedding providers
|
||||
- Hybrid retrieval and reranking
|
||||
- ACL-aware search
|
||||
- Knowledge Compilation
|
||||
- REST API
|
||||
- MCP Server
|
||||
- Admin Console
|
||||
|
||||
## Architecture baseline
|
||||
|
||||
- API First + Admin Console
|
||||
- Modular Monolith + async workers + MCP gateway
|
||||
- PostgreSQL + object storage as canonical source of truth
|
||||
- Mochow as the initial retrieval index, behind a replaceable adapter
|
||||
- Embedding / reranker / parser / LLM providers are replaceable
|
||||
- Raw source is immutable
|
||||
- Compiled Knowledge is derived and traceable
|
||||
- ACL is enforced before or during retrieval, never after Top-K
|
||||
|
||||
## Start here
|
||||
|
||||
1. Read `AGENTS.md`
|
||||
2. Read `CODEX_START_HERE.md`
|
||||
3. Read `docs/product/PRD.md`
|
||||
4. Read `docs/architecture/ARCHITECTURE.md`
|
||||
5. Read `docs/plans/V1_EXECUTION_PLAN.md`
|
||||
6. Pick exactly one task from `docs/plans/TASKS.md`
|
||||
0
apps/admin-web/.gitkeep
Normal file
0
apps/admin-web/.gitkeep
Normal file
0
apps/api/.gitkeep
Normal file
0
apps/api/.gitkeep
Normal file
0
apps/mcp-server/.gitkeep
Normal file
0
apps/mcp-server/.gitkeep
Normal file
0
deploy/docker/.gitkeep
Normal file
0
deploy/docker/.gitkeep
Normal file
30
docker-compose.dev.yml
Normal file
30
docker-compose.dev.yml
Normal file
@ -0,0 +1,30 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17
|
||||
environment:
|
||||
POSTGRES_DB: ekp
|
||||
POSTGRES_USER: ekp
|
||||
POSTGRES_PASSWORD: ekp
|
||||
ports: ["5432:5432"]
|
||||
volumes: ["ekp_postgres:/var/lib/postgresql/data"]
|
||||
|
||||
redis:
|
||||
image: redis:7
|
||||
ports: ["6379:6379"]
|
||||
|
||||
rabbitmq:
|
||||
image: rabbitmq:4-management
|
||||
ports: ["5672:5672", "15672:15672"]
|
||||
|
||||
minio:
|
||||
image: minio/minio
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: minioadmin
|
||||
MINIO_ROOT_PASSWORD: minioadmin
|
||||
ports: ["9000:9000", "9001:9001"]
|
||||
volumes: ["ekp_minio:/data"]
|
||||
|
||||
volumes:
|
||||
ekp_postgres:
|
||||
ekp_minio:
|
||||
7
docs/api/API_CONTRACT.md
Normal file
7
docs/api/API_CONTRACT.md
Normal file
@ -0,0 +1,7 @@
|
||||
# REST API Contract
|
||||
|
||||
Version all public APIs under `/api/v1`.
|
||||
|
||||
Initial domains: auth, users/groups, knowledge-bases, documents, compiled-knowledge, search, evaluations, jobs, audit-logs, providers and MCP administration.
|
||||
|
||||
Use OpenAPI as executable source of truth once implementation starts. Provider-specific concepts must not leak into public contracts.
|
||||
11
docs/api/MCP_CONTRACT.md
Normal file
11
docs/api/MCP_CONTRACT.md
Normal file
@ -0,0 +1,11 @@
|
||||
# MCP Contract
|
||||
|
||||
P0 business tools:
|
||||
|
||||
- `search_knowledge`
|
||||
- `get_document`
|
||||
- `get_document_section`
|
||||
- `get_source`
|
||||
- `list_knowledge_bases`
|
||||
|
||||
Every request maps to a user/service identity and uses the same authorization/application services as REST. Do not expose Mochow, Qwen, BGE, MinerU or other infrastructure choices.
|
||||
10
docs/architecture/ADR.md
Normal file
10
docs/architecture/ADR.md
Normal file
@ -0,0 +1,10 @@
|
||||
# Architecture Decision Records
|
||||
|
||||
- ADR-001: API First + Admin Console — Accepted
|
||||
- ADR-002: Modular monolith + workers + MCP gateway for V1 — Accepted
|
||||
- ADR-003: PostgreSQL + object storage are canonical — Accepted
|
||||
- ADR-004: Mochow is initial retrieval backend but replaceable — Accepted
|
||||
- ADR-005: Provider abstraction for embedding/reranker/parser/LLM — Accepted
|
||||
- ADR-006: RAG + Knowledge Compilation — Accepted
|
||||
- ADR-007: ACL before/during retrieval — Accepted
|
||||
- ADR-008: MCP hides infrastructure details — Accepted
|
||||
749
docs/architecture/ARCHITECTURE.md
Normal file
749
docs/architecture/ARCHITECTURE.md
Normal file
@ -0,0 +1,749 @@
|
||||
# 企业内部智能知识库 V1 技术方案(简版)
|
||||
|
||||
> 版本:v0.1
|
||||
> 目标:完成企业内部知识库第一版本,支持多类型文件上传、自动解析清洗、混合检索、Reranker、权限控制、Knowledge Compilation、REST API、MCP 接入及 Admin Console。
|
||||
|
||||
---
|
||||
|
||||
## 1. V1 建设目标
|
||||
|
||||
V1 聚焦形成完整可用闭环:
|
||||
|
||||
```text
|
||||
创建知识库
|
||||
↓
|
||||
上传文件
|
||||
↓
|
||||
解析 / OCR / 清洗
|
||||
↓
|
||||
结构化 Chunk
|
||||
↓
|
||||
Embedding + BM25 索引
|
||||
↓
|
||||
Hybrid Retrieval
|
||||
↓
|
||||
Reranker
|
||||
↓
|
||||
ACL 权限过滤
|
||||
↓
|
||||
REST API / MCP
|
||||
↓
|
||||
WorkBuddy 等客户端调用
|
||||
```
|
||||
|
||||
同时支持第二条知识沉淀链路:
|
||||
|
||||
```text
|
||||
原始知识 Source
|
||||
↓
|
||||
结构化知识 Structured Knowledge
|
||||
↓
|
||||
Knowledge Compilation
|
||||
↓
|
||||
Wiki / FAQ / SOP / Summary 等派生知识
|
||||
↓
|
||||
审核、发布、持续更新
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. V1 总体架构
|
||||
|
||||
```text
|
||||
┌─────────────────────┐
|
||||
│ Admin Console │
|
||||
└──────────┬──────────┘
|
||||
│ REST
|
||||
▼
|
||||
┌────────────┐ ┌─────────────────────┐
|
||||
│ WorkBuddy │──── MCP ────▶│ Knowledge API │◀──── REST ─── Other Systems
|
||||
└────────────┘ │ FastAPI / Python │
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
┌────────────────────────┼────────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ PostgreSQL │ │ Object Store │ │ Mochow │
|
||||
│ Canonical DB │ │ S3/MinIO │ │ Search Index │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
▲ ▲
|
||||
│ │
|
||||
┌───────┴────────────────────────┘
|
||||
│
|
||||
┌──────────────┐
|
||||
│ Async Worker │
|
||||
│ Celery │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌──────▼───────┐
|
||||
│ GPU Worker │
|
||||
│ Parser/OCR │
|
||||
│ Embedding │
|
||||
│ Reranker │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
### 架构原则
|
||||
|
||||
1. **API First**:后台、MCP、其他业务系统均调用统一 API/Service。
|
||||
2. **模块化单体优先**:V1 不拆大量微服务,API 服务 + 异步 Worker + MCP Gateway 足够。
|
||||
3. **Canonical Data 独立**:PostgreSQL + 对象存储保存事实数据,Vector DB 仅保存可重建索引。
|
||||
4. **Provider 可替换**:Embedding、Reranker、Vector DB、Parser、LLM 均通过 Adapter 抽象。
|
||||
5. **MCP 不暴露底层实现**:客户端只看到 `search_knowledge` 等业务工具。
|
||||
|
||||
---
|
||||
|
||||
## 3. 当前技术选型
|
||||
|
||||
| 模块 | V1 当前选型 | 说明 |
|
||||
|---|---|---|
|
||||
| Backend | Python + FastAPI | AI / 数据处理生态成熟,适合 MCP、异步任务和模型调用 |
|
||||
| Metadata DB | PostgreSQL | 保存用户、权限、文档、版本、任务、Compiled Knowledge 等 Canonical Data |
|
||||
| Object Storage | S3 Compatible / MinIO | 保存原始文件、解析产物、中间文件 |
|
||||
| Task Queue | Celery + RabbitMQ | 文件解析、Embedding、索引、Compilation 等异步任务 |
|
||||
| PDF / OCR Parser | MinerU 优先 | 中文复杂 PDF、扫描件、表格、版面解析能力较强 |
|
||||
| Office Parser | Docling 补充 | DOCX/PPTX/XLSX/HTML 等格式统一解析 |
|
||||
| Vector/Search DB | Mochow | 延续公司现有经验,支持向量、BM25、Hybrid、Filter |
|
||||
| Embedding | 多模型 Benchmark 决定 | 基线 bge-large-zh,对比 Qwen3 Embedding 等 |
|
||||
| Reranker | Qwen3 Reranker 等候选 | V1 正式检索链必选能力 |
|
||||
| LLM | Provider 抽象,云端优先 | 用于 Knowledge Compilation、分类、摘要等 |
|
||||
| MCP | 独立 MCP Gateway | 面向 WorkBuddy 等客户端,隐藏内部技术栈 |
|
||||
| Frontend | Admin Console | 知识库、文档、权限、检索调试、Benchmark、任务管理 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 核心模块
|
||||
|
||||
### 4.1 Knowledge Base 管理
|
||||
|
||||
支持:
|
||||
|
||||
- 创建 / 编辑 / 归档知识库;
|
||||
- Folder 分类;
|
||||
- Owner / Member;
|
||||
- 知识库级和文档级权限;
|
||||
- Retrieval Profile;
|
||||
- Embedding / Reranker 配置;
|
||||
- Knowledge Compilation 配置。
|
||||
|
||||
---
|
||||
|
||||
### 4.2 文件上传与自动入库
|
||||
|
||||
V1 支持:
|
||||
|
||||
- PDF
|
||||
- DOCX
|
||||
- PPTX
|
||||
- XLSX
|
||||
- CSV
|
||||
- Markdown
|
||||
- TXT
|
||||
- HTML
|
||||
- JPG / PNG
|
||||
|
||||
处理流程:
|
||||
|
||||
```text
|
||||
Upload
|
||||
→ Hash 去重
|
||||
→ Parser Router
|
||||
→ OCR(必要时)
|
||||
→ Cleaning
|
||||
→ Structure Normalize
|
||||
→ Chunk
|
||||
→ Embedding
|
||||
→ BM25 / Vector Index
|
||||
→ Completed
|
||||
```
|
||||
|
||||
任务状态:
|
||||
|
||||
```text
|
||||
Queued / Parsing / Cleaning / Chunking / Embedding / Indexing / Completed / Failed
|
||||
```
|
||||
|
||||
文件必须支持:
|
||||
|
||||
- Hash 去重;
|
||||
- Document Version;
|
||||
- 失败重试;
|
||||
- Re-parse / Re-embed / Re-index;
|
||||
- 软删除;
|
||||
- 来源页码和章节追踪。
|
||||
|
||||
---
|
||||
|
||||
### 4.3 文档结构与 Chunk
|
||||
|
||||
解析后形成统一结构:
|
||||
|
||||
```text
|
||||
Document
|
||||
├─ Heading
|
||||
├─ Paragraph
|
||||
├─ List
|
||||
├─ Table
|
||||
└─ Page
|
||||
```
|
||||
|
||||
Chunk 采用 **Structure-aware + Token-aware** 策略,禁止仅按固定字符数粗暴截断。
|
||||
|
||||
Chunk 至少保存:
|
||||
|
||||
```text
|
||||
chunk_id
|
||||
knowledge_base_id
|
||||
document_id
|
||||
document_version
|
||||
content
|
||||
title
|
||||
section_path
|
||||
page_start
|
||||
page_end
|
||||
content_type
|
||||
metadata
|
||||
acl
|
||||
index_profile_id
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 检索方案
|
||||
|
||||
V1 正式检索链:
|
||||
|
||||
```text
|
||||
Query
|
||||
↓
|
||||
Query Normalize
|
||||
↓
|
||||
ACL Resolve
|
||||
↓
|
||||
Dense Search + BM25
|
||||
↓
|
||||
Fusion
|
||||
↓
|
||||
Candidate Top N
|
||||
↓
|
||||
Reranker
|
||||
↓
|
||||
Top K
|
||||
↓
|
||||
Source Enrichment
|
||||
```
|
||||
|
||||
### 必须支持
|
||||
|
||||
- Dense Semantic Search;
|
||||
- BM25 / Keyword Search;
|
||||
- Hybrid Search;
|
||||
- Metadata Filter;
|
||||
- ACL Filter;
|
||||
- Reranker;
|
||||
- Knowledge Base / Folder / Document Scope;
|
||||
- 来源追踪。
|
||||
|
||||
禁止使用“全局检索后再删除无权限结果”的方式,权限过滤必须在召回阶段生效。
|
||||
|
||||
---
|
||||
|
||||
## 6. Embedding Benchmark
|
||||
|
||||
V1 不锁死单一 Embedding 模型。
|
||||
|
||||
### 首批候选
|
||||
|
||||
- bge-large-zh(现有基线);
|
||||
- Qwen3-Embedding-0.6B;
|
||||
- Qwen3-Embedding-4B;
|
||||
- 视 GPU 与效果补充 8B 或其他中文模型。
|
||||
|
||||
### Benchmark 指标
|
||||
|
||||
```text
|
||||
Recall@1 / 3 / 5 / 10
|
||||
MRR
|
||||
NDCG@10
|
||||
P50 / P95 Latency
|
||||
Index Throughput
|
||||
GPU Memory
|
||||
API Cost
|
||||
Storage Cost
|
||||
```
|
||||
|
||||
### 测试集
|
||||
|
||||
建立企业内部 Golden Dataset,V1 建议 200~500 条真实查询,覆盖:
|
||||
|
||||
- 中文自然语言;
|
||||
- 精确术语 / 编号;
|
||||
- 产品型号;
|
||||
- 同义表达;
|
||||
- 中英文混写;
|
||||
- 长问题;
|
||||
- 多条件问题;
|
||||
- 跨文档问题。
|
||||
|
||||
最终比较的是完整 Pipeline:
|
||||
|
||||
```text
|
||||
Embedding × Hybrid Retrieval × Reranker
|
||||
```
|
||||
|
||||
而不是只比较 Embedding 排行榜。
|
||||
|
||||
---
|
||||
|
||||
## 7. Index Profile 与模型切换
|
||||
|
||||
Embedding、Chunk、Sparse、Reranker 配置统一形成 `IndexProfile`。
|
||||
|
||||
示例:
|
||||
|
||||
```text
|
||||
profile_v1
|
||||
- chunk_strategy: structure_v1
|
||||
- embedding: bge-large-zh
|
||||
- sparse: bm25
|
||||
|
||||
profile_v2
|
||||
- chunk_strategy: structure_v2
|
||||
- embedding: qwen3-embedding-4b
|
||||
- sparse: bm25
|
||||
```
|
||||
|
||||
新模型上线时:
|
||||
|
||||
```text
|
||||
创建新 Profile
|
||||
→ 构建 Shadow Index
|
||||
→ Benchmark
|
||||
→ 切换 Active Index
|
||||
→ 保留旧索引回滚窗口
|
||||
```
|
||||
|
||||
避免直接原地覆盖现有生产索引。
|
||||
|
||||
---
|
||||
|
||||
## 8. Vector Store 设计
|
||||
|
||||
### 当前默认:Mochow
|
||||
|
||||
选择理由:
|
||||
|
||||
- 公司已有使用经验;
|
||||
- 已有百度云体系;
|
||||
- 支持 Dense Vector;
|
||||
- 支持 BM25 / Full Text;
|
||||
- 支持 Hybrid Search;
|
||||
- 支持 Metadata / Scalar Filter;
|
||||
- 满足 V1 ACL 下推需求。
|
||||
|
||||
### 保留替换能力
|
||||
|
||||
统一:
|
||||
|
||||
```text
|
||||
VectorStoreProvider
|
||||
├─ Mochow
|
||||
├─ Qdrant
|
||||
├─ Milvus
|
||||
└─ pgvector
|
||||
```
|
||||
|
||||
业务 Domain Model 不依赖 Mochow SDK 数据结构。
|
||||
|
||||
Mochow 中的数据全部视为 **可重建索引**,真实 Canonical Data 保存在 PostgreSQL / Object Storage。
|
||||
|
||||
---
|
||||
|
||||
## 9. Knowledge Compilation
|
||||
|
||||
V1 支持 **RAG + Knowledge Compilation**。
|
||||
|
||||
三层知识模型:
|
||||
|
||||
```text
|
||||
L0 Source
|
||||
原始文件,不允许 AI 修改
|
||||
|
||||
L1 Structured
|
||||
Parsed / Chunk / Index
|
||||
|
||||
L2 Compiled
|
||||
Wiki / FAQ / SOP / Summary / Comparison
|
||||
```
|
||||
|
||||
### Compilation Pipeline
|
||||
|
||||
```text
|
||||
Source 新增或更新
|
||||
→ Impact Analysis
|
||||
→ 查找相关 Knowledge Page
|
||||
→ LLM 生成 Draft
|
||||
→ Source Verification
|
||||
→ Conflict Detection
|
||||
→ Review Required
|
||||
→ Publish
|
||||
```
|
||||
|
||||
### Compiled Knowledge 类型
|
||||
|
||||
- Wiki Page;
|
||||
- FAQ;
|
||||
- SOP;
|
||||
- Summary;
|
||||
- Concept;
|
||||
- Entity;
|
||||
- Comparison。
|
||||
|
||||
所有 L2 Knowledge 必须保存 underlying source,并可追溯至 L0。
|
||||
|
||||
Source 删除、更新或权限变化时,相关 Compiled Knowledge 自动标记 `Stale`。
|
||||
|
||||
V1 默认 AI 只生成 Draft,重要知识需人工审核后发布。
|
||||
|
||||
---
|
||||
|
||||
## 10. 权限模型
|
||||
|
||||
权限主体:
|
||||
|
||||
```text
|
||||
User
|
||||
Group
|
||||
Role
|
||||
```
|
||||
|
||||
资源层级:
|
||||
|
||||
```text
|
||||
Organization
|
||||
Knowledge Base
|
||||
Folder
|
||||
Document
|
||||
```
|
||||
|
||||
权限类型:
|
||||
|
||||
```text
|
||||
Read
|
||||
Write
|
||||
Manage
|
||||
```
|
||||
|
||||
检索流程:
|
||||
|
||||
```text
|
||||
User Identity
|
||||
→ Permission Resolver
|
||||
→ Allowed Scope / ACL Filter
|
||||
→ Mochow Hybrid Retrieval
|
||||
```
|
||||
|
||||
MCP / REST / Admin Console 必须复用同一权限服务。
|
||||
|
||||
---
|
||||
|
||||
## 11. MCP
|
||||
|
||||
对外提供统一:
|
||||
|
||||
```text
|
||||
Enterprise Knowledge MCP Server
|
||||
```
|
||||
|
||||
V1 建议工具:
|
||||
|
||||
### `search_knowledge`
|
||||
|
||||
搜索当前用户有权限访问的企业知识。
|
||||
|
||||
### `get_document`
|
||||
|
||||
读取文档 metadata。
|
||||
|
||||
### `get_document_section`
|
||||
|
||||
获取指定页 / Section / Chunk 的连续上下文。
|
||||
|
||||
### `get_source`
|
||||
|
||||
获取检索结果的来源信息。
|
||||
|
||||
### `list_knowledge_bases`
|
||||
|
||||
列出当前身份可访问的知识库。
|
||||
|
||||
MCP Response 不出现:
|
||||
|
||||
```text
|
||||
Mochow
|
||||
Qdrant
|
||||
bge-large-zh
|
||||
Qwen3
|
||||
MinerU
|
||||
```
|
||||
|
||||
等底层实现信息。
|
||||
|
||||
首要验证客户端:**WorkBuddy Enterprise**。
|
||||
|
||||
---
|
||||
|
||||
## 12. REST API
|
||||
|
||||
统一前缀:
|
||||
|
||||
```text
|
||||
/api/v1/
|
||||
```
|
||||
|
||||
主要 Domain:
|
||||
|
||||
```text
|
||||
/auth
|
||||
/users
|
||||
/groups
|
||||
/knowledge-bases
|
||||
/documents
|
||||
/compiled-knowledge
|
||||
/search
|
||||
/evaluations
|
||||
/jobs
|
||||
/audit-logs
|
||||
/providers
|
||||
/mcp
|
||||
```
|
||||
|
||||
核心搜索接口:
|
||||
|
||||
```http
|
||||
POST /api/v1/search
|
||||
```
|
||||
|
||||
返回至少包含:
|
||||
|
||||
```text
|
||||
request_id
|
||||
results
|
||||
source
|
||||
score
|
||||
rerank_score
|
||||
document
|
||||
page
|
||||
section
|
||||
chunk_id
|
||||
source_type
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Admin Console
|
||||
|
||||
V1 一级功能建议:
|
||||
|
||||
```text
|
||||
Overview
|
||||
Knowledge Bases
|
||||
Documents
|
||||
Compiled Knowledge
|
||||
Search Lab
|
||||
Evaluation
|
||||
Models
|
||||
MCP & API
|
||||
Users & Permissions
|
||||
Jobs
|
||||
Audit Logs
|
||||
System Settings
|
||||
```
|
||||
|
||||
### Search Lab
|
||||
|
||||
Search Lab 是 V1 P0 工具,用于查看:
|
||||
|
||||
```text
|
||||
Dense Recall
|
||||
Sparse Recall
|
||||
Fusion
|
||||
Reranker
|
||||
Final Result
|
||||
```
|
||||
|
||||
帮助定位 Parser、Chunk、Embedding、BM25、Reranker 哪一层出现问题。
|
||||
|
||||
---
|
||||
|
||||
## 14. 异步任务与 GPU
|
||||
|
||||
异步 Job:
|
||||
|
||||
```text
|
||||
ParseJob
|
||||
OCRJob
|
||||
ChunkJob
|
||||
EmbeddingJob
|
||||
IndexJob
|
||||
CompilationJob
|
||||
ReindexJob
|
||||
EvaluationJob
|
||||
```
|
||||
|
||||
GPU 节点优先承担:
|
||||
|
||||
- MinerU / OCR;
|
||||
- Local Embedding;
|
||||
- Local Reranker。
|
||||
|
||||
LLM V1 可继续使用云端 Provider,不要求部署本地大模型。
|
||||
|
||||
---
|
||||
|
||||
## 15. 数据存储原则
|
||||
|
||||
### PostgreSQL
|
||||
|
||||
保存:
|
||||
|
||||
- User / Group / Role;
|
||||
- Knowledge Base;
|
||||
- Document / Version;
|
||||
- Chunk metadata;
|
||||
- ACL;
|
||||
- Job;
|
||||
- Index Profile;
|
||||
- Compiled Knowledge;
|
||||
- Audit Log;
|
||||
- Provider Config。
|
||||
|
||||
### Object Storage
|
||||
|
||||
保存:
|
||||
|
||||
- 原始文件;
|
||||
- Parser 输出;
|
||||
- OCR 结果;
|
||||
- Markdown / JSON 中间产物;
|
||||
- 需要持久化的大对象。
|
||||
|
||||
### Mochow
|
||||
|
||||
仅保存:
|
||||
|
||||
- Dense Vector;
|
||||
- Sparse/BM25 Index;
|
||||
- Retrieval metadata;
|
||||
- ACL 所需过滤字段。
|
||||
|
||||
数据库损坏或更换时,可由 Canonical Data 全量重建。
|
||||
|
||||
---
|
||||
|
||||
## 16. 部署建议
|
||||
|
||||
V1 推荐按以下逻辑角色部署:
|
||||
|
||||
```text
|
||||
Reverse Proxy / Ingress
|
||||
│
|
||||
├─ Admin Console
|
||||
├─ Knowledge API
|
||||
└─ MCP Gateway
|
||||
|
||||
PostgreSQL
|
||||
RabbitMQ
|
||||
Object Storage
|
||||
|
||||
CPU Worker
|
||||
GPU Worker
|
||||
|
||||
Mochow
|
||||
|
||||
Observability
|
||||
```
|
||||
|
||||
当前几十~百人内部用户规模,不建议一开始拆大量微服务。
|
||||
|
||||
优先采用:
|
||||
|
||||
```text
|
||||
模块化单体 API
|
||||
+ 独立 Async Worker
|
||||
+ 独立 GPU Worker
|
||||
+ 独立 MCP Gateway
|
||||
```
|
||||
|
||||
未来根据负载再拆服务。
|
||||
|
||||
---
|
||||
|
||||
## 17. V1 验收重点
|
||||
|
||||
### 文件入库
|
||||
|
||||
P0 文件能够完成:
|
||||
|
||||
```text
|
||||
Upload → Parse → Clean → Chunk → Embed → Index
|
||||
```
|
||||
|
||||
### 检索
|
||||
|
||||
正式链路支持:
|
||||
|
||||
```text
|
||||
Dense + BM25 + Hybrid + Reranker + ACL
|
||||
```
|
||||
|
||||
### 来源
|
||||
|
||||
每条结果能够追踪:
|
||||
|
||||
```text
|
||||
Source File → Version → Page → Section → Chunk
|
||||
```
|
||||
|
||||
### 权限
|
||||
|
||||
Admin Console、REST、MCP 均不能访问无权限知识。
|
||||
|
||||
### Benchmark
|
||||
|
||||
能够使用同一个 Golden Dataset 对多套 Retrieval Pipeline 进行对比。
|
||||
|
||||
### Knowledge Compilation
|
||||
|
||||
能够完成:
|
||||
|
||||
```text
|
||||
Source → Draft → Source Linking → Review → Publish → Stale Detection
|
||||
```
|
||||
|
||||
### MCP
|
||||
|
||||
WorkBuddy 可以通过 MCP 完成知识搜索和 Source 获取。
|
||||
|
||||
---
|
||||
|
||||
## 18. V1 当前最终选型结论
|
||||
|
||||
```text
|
||||
Backend Python + FastAPI
|
||||
Metadata PostgreSQL
|
||||
Object Storage S3 Compatible / MinIO
|
||||
Queue RabbitMQ + Celery
|
||||
Parser MinerU 主力 + Docling 补充
|
||||
Vector Store Mochow Primary,Adapter 可替换
|
||||
Embedding bge-large-zh 基线,多模型 Benchmark 后决定生产模型
|
||||
Reranker Qwen3 Reranker 等候选,V1 必选
|
||||
LLM Provider 抽象,允许云端
|
||||
Frontend Admin Console
|
||||
Protocol REST API + MCP
|
||||
Architecture Modular Monolith + Workers + MCP Gateway
|
||||
Knowledge RAG + Knowledge Compilation
|
||||
```
|
||||
|
||||
核心原则:**先保证知识可正确解析、可靠检索、严格授权、完整追溯,再逐步增加更高级的知识编译和智能化能力。**
|
||||
8
docs/development/CODING_RULES.md
Normal file
8
docs/development/CODING_RULES.md
Normal file
@ -0,0 +1,8 @@
|
||||
# Coding Rules
|
||||
|
||||
- Keep infrastructure SDKs behind provider interfaces.
|
||||
- Do not hardcode credentials, tenant IDs, endpoints, model names or collection names.
|
||||
- Prefer typed contracts and cohesive modules.
|
||||
- Make DB migrations forward-only.
|
||||
- Core tests must work without external SaaS, internet or GPU.
|
||||
- Avoid premature microservices.
|
||||
3
docs/development/DEFINITION_OF_DONE.md
Normal file
3
docs/development/DEFINITION_OF_DONE.md
Normal file
@ -0,0 +1,3 @@
|
||||
# Definition of Done
|
||||
|
||||
A task is done only when scope is implemented, architecture invariants hold, format/lint/typecheck pass, relevant tests pass, migrations/contracts/docs are updated as needed, no secrets are committed, retrieval changes include RAG regression results, and known limitations are documented.
|
||||
9
docs/development/SECURITY.md
Normal file
9
docs/development/SECURITY.md
Normal file
@ -0,0 +1,9 @@
|
||||
# Security Baseline
|
||||
|
||||
- Enforce tenant + ACL scope before/during retrieval.
|
||||
- Never log credentials or raw auth tokens.
|
||||
- Protect source-document downloads.
|
||||
- Audit sensitive reads and administrative changes.
|
||||
- Support `cloud_allowed` and `local_only` data policies per Knowledge Base.
|
||||
- Never send `local_only` content to cloud AI providers.
|
||||
- Validate MCP identity before protected operations.
|
||||
16
docs/development/TEST_STRATEGY.md
Normal file
16
docs/development/TEST_STRATEGY.md
Normal file
@ -0,0 +1,16 @@
|
||||
# Test Strategy
|
||||
|
||||
Unit tests must run without Mochow, Qianfan, internet, GPU or production credentials.
|
||||
|
||||
Provide deterministic fakes:
|
||||
- FakeEmbeddingProvider
|
||||
- FakeRerankerProvider
|
||||
- FakeVectorStore
|
||||
- FakeLLMProvider
|
||||
- FakeObjectStorage
|
||||
|
||||
External integrations use separate integration suites/markers.
|
||||
|
||||
E2E covers upload -> parse -> chunk -> embed -> index -> retrieve -> source trace, and source -> compilation draft -> review -> publish -> retrieval.
|
||||
|
||||
Any parsing/chunking/embedding/fusion/retrieval/reranking/query-routing change must run the Golden Dataset evaluation.
|
||||
9
docs/evaluation/RAG_EVALUATION.md
Normal file
9
docs/evaluation/RAG_EVALUATION.md
Normal file
@ -0,0 +1,9 @@
|
||||
# RAG Evaluation
|
||||
|
||||
Maintain a versioned Golden Dataset and report Recall@1/3/5/10, MRR, NDCG@10, latency P50/P95.
|
||||
|
||||
Benchmark the full pipeline: Embedding × Retrieval Strategy × Fusion × Reranker.
|
||||
|
||||
Initial embedding candidates: bge-large-zh, Qwen3-Embedding-0.6B, Qwen3-Embedding-4B, optional 8B.
|
||||
|
||||
Use synthetic/public/anonymized fixtures only.
|
||||
7
docs/plans/PROGRESS.md
Normal file
7
docs/plans/PROGRESS.md
Normal file
@ -0,0 +1,7 @@
|
||||
# Progress
|
||||
|
||||
Current phase: Phase 0 — Repository / developer harness
|
||||
|
||||
Completed: none
|
||||
In progress: none
|
||||
Blocked: none
|
||||
61
docs/plans/TASKS.md
Normal file
61
docs/plans/TASKS.md
Normal file
@ -0,0 +1,61 @@
|
||||
# V1 Tasks
|
||||
|
||||
## Phase 0
|
||||
- [ ] EKP-001 Repository bootstrap
|
||||
- [ ] EKP-002 Backend Python toolchain
|
||||
- [ ] EKP-003 Admin Web toolchain
|
||||
- [ ] EKP-004 Application configuration
|
||||
- [ ] EKP-005 Structured logging
|
||||
- [ ] EKP-006 CI baseline
|
||||
- [ ] EKP-007 Fake provider framework
|
||||
|
||||
## Canonical domain / ingestion
|
||||
- [ ] EKP-010 Core domain package
|
||||
- [ ] EKP-011 PostgreSQL schema and migrations
|
||||
- [ ] EKP-012 User/group/role model
|
||||
- [ ] EKP-013 Knowledge Base model
|
||||
- [ ] EKP-014 Folder model
|
||||
- [ ] EKP-015 Document + DocumentVersion model
|
||||
- [ ] EKP-016 ACL model
|
||||
- [ ] EKP-017 Job + audit models
|
||||
- [ ] EKP-020 ObjectStorage abstraction + S3 adapter
|
||||
- [ ] EKP-021 Document upload API
|
||||
- [ ] EKP-022 Source hash/dedup/version activation
|
||||
- [ ] EKP-023 Async job framework
|
||||
- [ ] EKP-024 Parser abstraction
|
||||
- [ ] EKP-025 MinerU adapter
|
||||
- [ ] EKP-026 Docling adapter
|
||||
- [ ] EKP-027 OCR + cleaning pipeline
|
||||
- [ ] EKP-028 Canonical parsed document model
|
||||
- [ ] EKP-029 Structure-aware / spreadsheet chunking
|
||||
|
||||
## Indexing / retrieval
|
||||
- [ ] EKP-040 EmbeddingProvider + fake
|
||||
- [ ] EKP-041 Qianfan BGE adapter
|
||||
- [ ] EKP-042 Qwen embedding adapter
|
||||
- [ ] EKP-043 IndexProfile model
|
||||
- [ ] EKP-044 VectorStore + fake
|
||||
- [ ] EKP-045 Mochow adapter
|
||||
- [ ] EKP-046 Index build + shadow index lifecycle
|
||||
- [ ] EKP-050 Permission resolver + ACL retrieval filters
|
||||
- [ ] EKP-051 Dense retrieval
|
||||
- [ ] EKP-052 BM25/sparse retrieval
|
||||
- [ ] EKP-053 Hybrid fusion
|
||||
- [ ] EKP-054 RerankerProvider + fake + Qwen adapter
|
||||
- [ ] EKP-055 Source enrichment + Search service/API
|
||||
- [ ] EKP-056 Retrieval diagnostics / Search Lab backend
|
||||
|
||||
## MCP / Admin / Compilation / Evaluation
|
||||
- [ ] EKP-070 MCP server + identity
|
||||
- [ ] EKP-071 MCP P0 tools
|
||||
- [ ] EKP-072 WorkBuddy compatibility test
|
||||
- [ ] EKP-080 Admin Console core pages
|
||||
- [ ] EKP-081 Permissions/jobs/chunk inspector/Search Lab UI
|
||||
- [ ] EKP-090 CompiledKnowledge model/workflow
|
||||
- [ ] EKP-091 Source linking/conflict/stale/review/publish
|
||||
- [ ] EKP-092 Compiled retrieval integration
|
||||
- [ ] EKP-100 Golden Dataset + metrics
|
||||
- [ ] EKP-101 Multi-embedding benchmark
|
||||
- [ ] EKP-102 Reranker benchmark
|
||||
- [ ] EKP-103 Security/performance/rebuild tests
|
||||
- [ ] EKP-104 V1 release hardening
|
||||
11
docs/plans/V1_EXECUTION_PLAN.md
Normal file
11
docs/plans/V1_EXECUTION_PLAN.md
Normal file
@ -0,0 +1,11 @@
|
||||
# V1 Execution Plan
|
||||
|
||||
1. Repository / developer harness
|
||||
2. Canonical domain and ACL
|
||||
3. Document ingestion and parsing
|
||||
4. Embedding / Index Profile / Mochow adapter
|
||||
5. ACL-aware hybrid retrieval + reranker
|
||||
6. MCP gateway and WorkBuddy verification
|
||||
7. Admin Console
|
||||
8. Knowledge Compilation
|
||||
9. Evaluation, security and release hardening
|
||||
2537
docs/product/PRD.md
Normal file
2537
docs/product/PRD.md
Normal file
File diff suppressed because it is too large
Load Diff
3
fixtures/documents/README.md
Normal file
3
fixtures/documents/README.md
Normal file
@ -0,0 +1,3 @@
|
||||
# Test Documents
|
||||
|
||||
Only commit synthetic, public, or properly anonymized fixtures. Never commit real internal enterprise documents, HR data, customer data, contracts, credentials, or confidential material.
|
||||
2
fixtures/evaluation/golden.jsonl
Normal file
2
fixtures/evaluation/golden.jsonl
Normal file
@ -0,0 +1,2 @@
|
||||
{"id":"sample-001","query":"员工出差住宿标准是多少?","expected_documents":["sample-travel-policy.md"],"expected_sections":["住宿标准"],"tags":["制度","自然语言"]}
|
||||
{"id":"sample-002","query":"产品A退款周期","expected_documents":["sample-product-a-policy.md"],"expected_sections":["退款"],"tags":["产品","短查询"]}
|
||||
0
migrations/.gitkeep
Normal file
0
migrations/.gitkeep
Normal file
0
packages/core/.gitkeep
Normal file
0
packages/core/.gitkeep
Normal file
0
packages/domain/.gitkeep
Normal file
0
packages/domain/.gitkeep
Normal file
0
packages/providers/embedding/.gitkeep
Normal file
0
packages/providers/embedding/.gitkeep
Normal file
0
packages/providers/llm/.gitkeep
Normal file
0
packages/providers/llm/.gitkeep
Normal file
0
packages/providers/object_storage/.gitkeep
Normal file
0
packages/providers/object_storage/.gitkeep
Normal file
0
packages/providers/parser/.gitkeep
Normal file
0
packages/providers/parser/.gitkeep
Normal file
0
packages/providers/reranker/.gitkeep
Normal file
0
packages/providers/reranker/.gitkeep
Normal file
0
packages/providers/vector_store/.gitkeep
Normal file
0
packages/providers/vector_store/.gitkeep
Normal file
0
packages/retrieval/.gitkeep
Normal file
0
packages/retrieval/.gitkeep
Normal file
0
packages/shared/.gitkeep
Normal file
0
packages/shared/.gitkeep
Normal file
37
prompts/CODEX_PROMPTS.md
Normal file
37
prompts/CODEX_PROMPTS.md
Normal file
@ -0,0 +1,37 @@
|
||||
# Codex Prompt Library
|
||||
|
||||
## P00 — Repository takeover
|
||||
Read AGENTS.md and all referenced product/architecture/API/MCP/plan/DoD docs. Do not modify code. Inspect the repository and report current status, V1 gaps, top 5 risks, next 1–3 Task IDs, and blockers. Do not redesign accepted ADRs.
|
||||
|
||||
## P01 — Plan one task
|
||||
Plan `<TASK_ID>` only. Read relevant docs/code. Output goal, touched modules, steps, tests, migration/API impact, risks and out-of-scope. Do not modify files.
|
||||
|
||||
## P02 — Implement one task
|
||||
Implement `<TASK_ID>` only. Preserve AGENTS.md invariants. Do not implement future tasks. Run format/lint/typecheck/tests and RAG eval if retrieval behavior changes. Report summary, changed files, validation, risks and follow-ups.
|
||||
|
||||
## P03 — Self-review as PR
|
||||
Review the current branch for correctness, architecture violations, ACL leakage, secrets, provider coupling, migration safety, idempotency, test coverage, API/MCP compatibility and source traceability. Fix Critical/High findings within scope and rerun validation.
|
||||
|
||||
## P04 — Bug fix
|
||||
Reproduce `<BUG>`, identify root cause, add failing regression test, implement smallest safe fix, run relevant checks, and report root cause + validation.
|
||||
|
||||
## P05 — Retrieval change
|
||||
For `<TASK_ID>`, identify changed retrieval stage. Preserve ACL-before/during-retrieval and source metadata. Run Golden Dataset before/after and report quality + latency deltas.
|
||||
|
||||
## P06 — Embedding benchmark
|
||||
Compare `<MODELS>` using the same Golden Dataset, corpus, chunk profile and retrieval config. Report Recall@K, MRR, NDCG@10, P50/P95 latency, throughput, dimensions, GPU/API needs, storage/cost and recommendation.
|
||||
|
||||
## P07 — Parser benchmark
|
||||
Compare MinerU and Docling on anonymized Chinese enterprise docs: native/scanned PDF, tables, multi-column, DOCX, PPTX, XLSX where applicable. Evaluate completeness, order, structure, tables, traceability, OCR, latency and GPU. Recommend routing rules.
|
||||
|
||||
## P08 — MCP task
|
||||
Implement/review `<TASK_ID>`. Expose business semantics only; bind identity; reuse REST authorization/application services; preserve traceability; do not expose underlying provider choices.
|
||||
|
||||
## P09 — ACL audit
|
||||
Verify Knowledge Base, folder inheritance, document overrides, REST, MCP, source download, retrieval, compiled knowledge and jobs. Fail review if unauthorized content can enter candidate Top-K before filtering.
|
||||
|
||||
## P10 — Knowledge Compilation
|
||||
Implement `<TASK_ID>` while preserving immutable source, source links, permission inheritance/intersection, stale detection and conflict safety. Never silently merge conflicting claims.
|
||||
|
||||
## P11 — Release readiness
|
||||
Assess V1 against PRD/DoD/tasks/ADRs: functionality, migrations, ACL/security, secrets, observability, backup/rebuild, performance, RAG quality, MCP, Admin Console and Compilation. Return blockers, high risks, known limitations and go/no-go.
|
||||
0
scripts/.gitkeep
Normal file
0
scripts/.gitkeep
Normal file
0
tests/e2e/.gitkeep
Normal file
0
tests/e2e/.gitkeep
Normal file
0
tests/evaluation/.gitkeep
Normal file
0
tests/evaluation/.gitkeep
Normal file
0
tests/integration/.gitkeep
Normal file
0
tests/integration/.gitkeep
Normal file
0
tests/unit/.gitkeep
Normal file
0
tests/unit/.gitkeep
Normal file
0
workers/compilation/.gitkeep
Normal file
0
workers/compilation/.gitkeep
Normal file
0
workers/embedding/.gitkeep
Normal file
0
workers/embedding/.gitkeep
Normal file
0
workers/ingestion/.gitkeep
Normal file
0
workers/ingestion/.gitkeep
Normal file
0
workers/parser/.gitkeep
Normal file
0
workers/parser/.gitkeep
Normal file
Loading…
x
Reference in New Issue
Block a user