Enterprise Knowledge Platform
Enterprise internal knowledge infrastructure.
Core capabilities
- Multi-format document ingestion
- OCR and document parsing
- Structure-aware chunking
- Multiple embedding providers
- Hybrid retrieval and reranking
- ACL-aware search
- Knowledge Compilation
- REST API
- MCP Server
- Admin Console
Architecture baseline
- API First + Admin Console
- Modular Monolith + async workers + MCP gateway
- PostgreSQL + object storage as canonical source of truth
- Mochow as the initial retrieval index, behind a replaceable adapter
- Embedding / reranker / parser / LLM providers are replaceable
- Raw source is immutable
- Compiled Knowledge is derived and traceable
- ACL is enforced before or during retrieval, never after Top-K
Start here
- Read
AGENTS.md - Read
CODEX_START_HERE.md - Read
docs/product/PRD.md - Read
docs/architecture/ARCHITECTURE.md - Read
docs/plans/V1_EXECUTION_PLAN.md - For the current milestone, read
docs/demo/DEMO_SCOPE.mdand pick exactly oneDEMO-*task. The V1 tasks indocs/plans/TASKS.mdremain the deferred product backlog.
Demo development
The current implementation target is the retrieval demo defined in
docs/demo/DEMO_SCOPE.md.
Prerequisites:
uv- Docker (for PostgreSQL and later integration dependencies)
Install dependencies and run the API:
cp .env.example .env
make install
make run
The health endpoint is available at http://localhost:8000/healthz, and the
versioned OpenAPI document is available at
http://localhost:8000/api/v1/openapi.json.
Run the development checks with:
make format-check lint typecheck test migration-check eval
Document metadata is canonical in PostgreSQL. Demo source files use the local
storage/ directory through the provider-neutral ObjectStorage interface;
existing objects cannot be overwritten. The upload service accepts PDF, DOCX,
TXT, and MD sources, streams at most MAX_UPLOAD_BYTES, and compensates stored
bytes if metadata persistence fails. The ingestion service then routes PDF to
PyMuPDF, DOCX to python-docx, and UTF-8 TXT/Markdown to lightweight parsers. It
normalizes whitespace, retains available page/section context, and stores a
versioned parsed JSON artifact under parsed/{document_id}/document.json.
The structure_v1 chunk strategy keeps heading/section boundaries, combines
paragraphs, and creates configurable 450-token windows with a 75-token overlap.
Chunk metadata, including deterministic IDs, filename, page range, section,
index, and strategy version, is stored atomically in PostgreSQL. The lightweight
token counter is replaceable so DEMO-006 can inject model-specific behavior.
Documents remain processing until embedding/indexing completes; pipeline
failures become failed without deleting the immutable original. Public upload
endpoints are introduced by DEMO-009.
DEMO-006 adds a provider-neutral EmbeddingProvider and VectorStore, a
validated Qianfan bge-large-zh adapter, and an isolated pymochow adapter.
Chunk vectors are upserted with source metadata under a versioned index profile;
PostgreSQL records canonical index state while Mochow remains rebuildable. A
document becomes ready only after every chunk is indexed. See
docs/development/MOCHOW_SETUP.md for the required environment variables and
pre-provisioned table schema.
DEMO-007 adds query embedding and profile-scoped Dense Top-N retrieval. The
default is Top 20 with a configurable Mochow HNSW ef; the in-memory fake uses
exact cosine search for deterministic offline regression. Returned objects are
raw dense candidates only. Reranking and final source-result assembly remain in
DEMO-008, while public search REST endpoints remain in DEMO-009.