EmboFlow/design/02-architecture/deployment-architecture.md

EmboFlow Deployment Architecture

V1 Deployment Target

The first deployment target is a single public server. The platform should be deployed in a way that is operationally simple now and migration-friendly later.

Recommended V1 Deployment Topology

• Reverse proxy
• Web frontend service
• API service
• Worker service
• MongoDB
• Optional MinIO
• Host Docker runtime for execution containers

Deployment Principles

• Single-host deployment first
• All major services containerized
• Persistent state mounted on host volumes
• Object storage can be external or self-hosted
• Execution workers separated from API service
• Future scheduler migration should not require domain model changes

Recommended Runtime Layout

Edge

• Nginx or equivalent reverse proxy
• HTTPS termination
• Static web delivery or web upstream routing

Application

• web
• api
• worker

Data

• mongo
• minio optional

Object Storage Strategy

The product should support both:

• Cloud object storage such as BOS or S3-compatible services
• Self-hosted MinIO for development, demos, or private deployment

The application should expose a unified storage abstraction instead of embedding provider-specific logic across modules.

Local Scheduler In V1

V1 should use a local scheduler. Worker processes execute tasks on the same deployment host.

Design constraints:

• RuntimeSpec must already exist
• Scheduler abstraction must already exist
• Docker executor must already be scheduler-compatible

This keeps future migration to Kubernetes or Volcano feasible.

Host-Level Persistent Directories

Recommended host directories:

• application config
• mongodb data
• minio data
• uploaded file staging
• execution temp workspace
• logs
• backup data

Execution Isolation

The host Docker runtime serves two different purposes:

• Running the platform deployment stack
• Running task execution containers

These must be treated as separate concerns in configuration and security design.

Future Migration Path

Stage 1

• Single-host deployment
• Local scheduler
• Docker executor

Stage 2

• Kubernetes-based service deployment
• Kubernetes scheduler adapter for workflow tasks

Stage 3

• Volcano scheduler adapter
• Better support for large batch jobs and training-adjacent workloads

Operational Baseline

V1 should provide basic operational support for:

• health checks
• service restart
• execution failure visibility
• disk space monitoring
• object storage connectivity checks
• MongoDB backup and restore procedures
• worker online status