Architecture
This document describes the structure of the Ark Seed Multimodal MCP Server as shipped today. For the original design rationale, see ../plans/PLAN_ARK_SEED_MULTIMODAL_MCP.md.
Design goals
Section titled “Design goals”- Small, typed tool surface — a handful of Pydantic-validated tools rather than a wide REST API.
- Dedicated provider gateways, one domain layer — Seedance and Seedream
share the ModelArk host and Bearer auth; Seed Audio uses a separate host and
X-Api-Key; VOD AI MediaKit uses its own Bearer-authenticated convenience endpoint for enhancement, transcoding, and audio separation. The differences are hidden behind normalized adapters. - Durable artifacts — known provider media URLs expire (2h Seed Audio; 24h
ModelArk image/video and observed MediaKit outputs), so outputs are persisted
to a local store and re-exposed as stable
seed-media://artifacts/{id}MCP resources. MediaKit persistence is best-effort. - Safe by default — local
stdiorequires no auth; remote HTTP requires shared-key hash verification, Host/Origin protection, and body limits. - Observable and budget-aware — structured logs, Prometheus metrics, and a per-principal daily budget ledger.
- Timeout-safe long operations — generation, transcription, upload, and
provider-submission calls run through the ordinary
ark_job_*tools by default, or native MCP task augmentation where a client supports it; completed-media retrieval supports optional background persistence.
Layered structure
Section titled “Layered structure”src/ark_mcp/├── __main__.py # entry point; truststore injection; transport wiring├── server.py # FastMCP factory; tool/resource/route registration├── config/ # settings (env), model capability registry│ ├── env.py│ └── model_capabilities.py├── domain/ # pure models: ArtifactRef, MediaSource, errors│ ├── artifacts.py│ ├── media.py│ ├── models.py│ └── errors.py├── tools/ # MCP tool implementations (+ _cost, _parallel, _errors)├── providers/ # dedicated HTTP gateways + retry policy│ ├── base.py # BaseHttpGateway: spans, metrics, error normalization│ ├── retry.py│ ├── modelark.py # Seedream + Seedance│ ├── seed_speech.py # Seed Audio│ ├── vod_mediakit/ # VOD AI MediaKit (Bearer; enhancement, transcode, subtitles, separation)│ └── vision_cv/ # Vision AI (Bearer; OmniHuman 2.0 submit/query/cancel)├── runtime.py # lifespan-owned services (limiter, budget, ownership)├── artifacts/ # durable artifact store (filesystem backend)│ ├── store.py # ArtifactStore protocol│ └── filesystem_store.py├── security/ # auth, SSRF-safe downloads, URL/media policy, body limit├── observability/ # structured logging + Prometheus metrics└── transports (via FastMCP) # stdio + Streamable HTTPProvider gateway domain layer
Section titled “Provider gateway domain layer”flowchart LR
Client["MCP Client\n(stdio / HTTP)"] --> Server["FastMCP server\n(server.py)"]
Server --> Domain["Domain layer\n(tools/ + domain/)"]
Domain --> Gateway["Provider gateways\n(providers/)"]
Gateway -->|"Bearer auth"| ModelArk["ModelArk\nSeedream + Seedance"]
Gateway -->|"X-Api-Key"| SeedSpeech["Seed Speech\nSeed Audio"]
Gateway -->|"Bearer auth"| MediaKit["VOD AI MediaKit\nenhancement + transcode + subtitles + separation"]
Gateway -->|"Bearer auth"| VisionCv["Vision AI\nOmniHuman 2.0"]
Gateway -->|"AK/SK V4 signing"| OpenApi["ModelArk OpenAPI\nassets + moderation lookup"]
Server -.durable.-> Store["Artifact store\n(filesystem)"]
Server -.state.-> Runtime["Runtime services\n(runtime.py)"]
- ModelArk gateway (
providers/modelark.py) — serves Seedream (image) and Seedance (video). UsesAuthorization: Bearerand base URLhttps://ark.ap-southeast.bytepluses.com/api/v3. - Seed Speech gateway (
providers/seed_speech.py) — serves Seed Audio. UsesX-Api-Keyand base URLhttps://voice.ap-southeast-1.bytepluses.com. - Vision AI gateway (
providers/vision_cv/) — serves theomnihuman_*tools through the common v3 task API (submit_task_v2,get_result_v2,cancel_task). It uses Bearer auth (BYTEPLUS_CV_API_KEY), defaults tohttps://openapi.cv.bytepluses.com, and treatscode == 10000in the response body as the only success signal. - VOD AI MediaKit gateway (
providers/vod_mediakit/) — serves thevod_enhance_video,vod_transcode_video,vod_add_subtitles,vod_remove_subtitles, andvod_separate_audiosubmit tools (for examplePOST /tools/separate-voice) and the singlevod_get_taskpoller (GET /tasks/{task_id}), which dispatches on theoperationeach submit tool returns. It uses Bearer auth and defaults tohttps://mediakit.ap-southeast-1.bytepluses.com/api/v1. Its success schemas are isolated in the adapters; the gateway accepts only known result shapes and rejects unknown shapes. - ModelArk OpenAPI (
providers/modelark/openapi.py) — signed AK/SK control plane for assets and opt-inark_moderation_get. Moderation is account-wide trusted stdio/local-mode loopback only, foreground-only, with one request, no automatic retries or persistence. Shared-key HTTP excludesmoderation:read. Bounds and live-unverified status: moderation.md. - These gateways extend
BaseHttpGateway(providers/base.py), which wraps every outbound request in an OpenTelemetry span, records Prometheus provider metrics, and normalizes transport/HTTP errors into a singleProviderErrorcarrying aNormalizedProviderError.
MediaKit enhancement, transcode, subtitle, and separation submissions are non-idempotent
mutations and bypass the automatic retry helper: a timeout can be ambiguous after
the provider has begun work. All operations use the verified task-status
endpoint (GET /tasks/{task_id}), so vod_get_task polls it for every
operation and reuses the shared ownership store and task-artifact cache under the vod-mediakit provider
key. For all surfaces, a completed provider URL is preserved and
persistence is attempted separately as a best-effort operation (under the
200 MiB video limit for video, 10 MiB audio limit for separated tracks).
Long-running task flow
Section titled “Long-running task flow”Generation and variation calls, transcription, uploads, multimodal
understanding, and Seedance, Seed 3D, and MediaKit submissions are registered
with execution.taskSupport="required" and a two-second recommended poll
interval. FastMCP’s Docket worker runs the existing tool handler after returning
the MCP task ID, while the handler retains the normal provider timeout, budget,
concurrency, error, and usage accounting paths.
Because most clients cannot negotiate task augmentation, the default path
submits the same registered handlers through the ordinary ark_job_submit tool
and polls ark_job_get. The adapter is intentionally thin: it resolves an
allowlisted task-enabled tool, enforces the target’s scope and ownership, and
delegates creation, status, and cancellation to the pinned FastMCP task
backend. It does not introduce a second queue or execute provider work inside
the foreground request.
Seedance, Seed 3D, and MediaKit get calls use optional task support: foreground execution remains available for short processing-status checks, while task augmentation protects the completed-output download and persistence path. List, presign, artifact-read, and cancel/delete operations stay foreground. Moderation lookup is also foreground-only and excluded from both task paths.
sequenceDiagram
participant C as MCP client
participant S as FastMCP server
participant W as Docket worker
participant A as BytePlus API or object storage
alt Ordinary-tool client (default)
C->>S: ark_job_submit(target, arguments)
S-->>C: Ark job ID (working)
else Client supports task augmentation
C->>S: original tools/call + task metadata
S-->>C: MCP task ID (working)
end
S->>W: enqueue long-running tool
W->>A: generation, transcription, upload, submission, or persistence
loop recommended every 2 seconds until terminal
alt Ordinary-tool path
C->>S: ark_job_get(job ID)
else Native task path
C->>S: tasks/get
end
S-->>C: working status or terminal result
end
A-->>W: completion
W->>S: store final tool result
Note over C,S: terminal response preserves the typed tool output
opt Provider submission result
Note over S,C: typed output includes provider task ID
C->>S: product get tool(provider task ID)
S->>A: foreground status request
A-->>S: provider task status
S-->>C: typed provider task output
end
Server lifecycle and runtime services
Section titled “Server lifecycle and runtime services”server.py::create_server builds the FastMCP instance. The server lifespan
is owned by runtime.py::build_lifespan, which constructs a single
RuntimeServices object and yields it as the FastMCP lifespan context:
sequenceDiagram
participant M as __main__ / FastMCP
participant L as build_lifespan
participant R as RuntimeServices
participant T as Tool call
M->>L: start lifespan
L->>R: create_runtime_services(settings)
L->>M: yield {"runtime": RuntimeServices}
M->>T: invoke tool
T->>R: get_runtime(ctx)
R-->>T: services
T->>T: billed_provider_slot(...)
M->>L: shutdown
L->>R: close_runtime_services (artifact, ownership, object-key, budget, cache stores)
RuntimeServices holds ten components (see runtime.md for
full detail):
| Field | Purpose |
|---|---|
settings |
resolved Settings |
artifact_store |
FilesystemArtifactStore — durable media |
safe_downloader |
SSRF-safe HTTP downloader |
ownership_store |
SQLiteTaskOwnershipStore — provider task ownership |
object_key_ownership_store |
SQLiteObjectKeyOwnershipStore — uploaded-object ownership |
budget_ledger |
BudgetLedger — per-principal UTC daily budget |
provider_limiters |
ProviderLimiters — provider + principal concurrency |
moderation_query_limiter |
ModerationQueryLimiter — ten lookups per rolling minute per runtime, shared across callers and ID types |
task_artifact_cache |
SQLiteTaskArtifactCache — provider task → artifact ref cache |
task_artifact_locks |
TaskArtifactPersistenceLocks — process-local per-task single-flight |
close_runtime_services closes the five resources that own I/O state:
artifact_store, ownership_store, object_key_ownership_store,
budget_ledger, and task_artifact_cache. The lock registry is process-local,
removes entries after the last holder or waiter exits, and needs no close step.
Request flow for a billable tool
Section titled “Request flow for a billable tool”The identity path is implemented by security/http_auth.py and
runtime.py::get_principal. HTTP authentication rejects an invalid key before
tool dispatch; generic internal bearer claims remain available for ownership
tests, not as a configurable remote identity mechanism.
flowchart TD
A["MCP tool invoked"] --> B{"local mode?"}
B -->|yes| C["PrincipalContext()\nlocal/local"]
B -->|no| D["Read verified shared-key context: sub and fixed tenant_id"]
D --> C2["PrincipalContext: shared-team/shared-team, fixed ordinary scopes"]
C --> E["reserve budget\n(BudgetLedger.reserve)"]
C2 --> E
E -->|"over limit"| F["BudgetExceededError\n(pre-dispatch)"]
E -->|"reserved"| G["acquire provider + principal\nsemaphores (billed_provider_slot)"]
G --> H["call_with_retry\n(provider gateway)"]
H --> I{"outcome"}
I -->|success| J["commit reservation"]
I -->|"ambiguous timeout"| J
I -->|"retryable error"| K["release reservation"]
J --> L["persist artifact + return ArtifactRef"]
K --> L
Three layers of control compose on a single billable call: the provider
bucket semaphore (per provider, global, default 5), the principal semaphore
(per (tenant, principal), default 3; skipped in local mode), and — for parallel
variation tools — a per-batch local semaphore (max_concurrent=5). All
shared-key members share that principal semaphore and budget. See runtime.md.
Moderation uses the modelark-openapi concurrency bucket followed by its
dedicated rolling-minute limiter, not the billable-generation budget flow.
Its quota is in-memory, not shared across replicas or other account clients;
lookup pricing is unspecified.
Transports
Section titled “Transports”| Transport | When | Auth |
|---|---|---|
stdio |
local default | none (single trusted principal local) |
| Streamable HTTP | remote / shared | api_key required off loopback; one fixed shared-team identity with ordinary scopes |
local is the Python default and is restricted to stdio/loopback HTTP.
Docker, Compose and Terraform default to api_key. Shared HTTP excludes
account-wide asset and moderation administration; it does not isolate members
of a team.
truststore.inject_into_ssl() runs at module import in both server.py and
__main__.py, so every run path loads the macOS system Keychain for TLS.
Transport wiring, Host/Origin protection, and body-limit middleware live in
__main__.py. See transports.md and security.md.
Where to read more
Section titled “Where to read more”| Topic | Document |
|---|---|
| Runtime services (limiter, budget, ownership, retry) | runtime.md |
| Logging, metrics, tracing | observability.md |
| Consolidated security model | security.md |
| Model capability registry | models.md |
| Durable artifact lifecycle | artifacts.md |
| Tool contracts | api-reference.md |
| Configuration | configuration.md |