Model Capability Registry
The capability registry (config/model_capabilities.py) describes, per
configured model, what inputs the server will accept before it ever calls
the provider. It validates image sizes, output formats, video resolutions,
and durations against family-specific rules, so unsupported requests fail fast
with an actionable ValueError.
Families
Section titled “Families”ModelFamily (StrEnum) includes:
| Member | Value |
|---|---|
SEEDREAM_5_5 |
seedream_5_5 |
SEEDREAM_PRO |
seedream_pro |
SEEDREAM_LITE |
seedream_lite |
SEEDREAM_4X |
seedream_4x |
SEEDANCE_2 |
seedance_2 |
SEEDANCE_2_FAST |
seedance_2_fast |
SEEDANCE_2_MINI |
seedance_2_mini |
SEEDANCE_2_5 |
seedance_2_5 |
SEEDANCE_2_5_FLASH |
seedance_2_5_flash |
SEED_2_1_PRO |
seed_2_1_pro |
SEED_2_1_TURBO |
seed_2_1_turbo |
SEED3D_HYPER3D |
seed3d_hyper3d |
SEED3D_HITEM3D |
seed3d_hitem3d |
The binding enums (config/env.py):
SeedreamFamily:SEEDREAM_5_5 = "seedream_5_5",PRO = "pro",LITE = "lite",V4X = "4x".SeedanceFamily:STANDARD = "standard",FAST = "fast",MINI = "mini",SEEDANCE_2_5 = "seedance_2_5",SEEDANCE_2_5_FLASH = "seedance_2_5_flash".SeedUnderstandingFamily:PRO = "pro",TURBO = "turbo".Seed3DFamily:HYPER3D = "hyper3d",HITEM3D = "hitem3d".
Image capabilities (ImageCapabilities)
Section titled “Image capabilities (ImageCapabilities)”| Field | SEEDREAM_5_5 |
PRO |
LITE |
4X |
|---|---|---|---|---|
max_references |
10 | 10 | 14 | 14 |
supports_batch |
False |
False |
True |
True |
supports_streaming |
False |
False |
True |
True |
supported_output_formats |
("png", "jpeg") |
("png", "jpeg") |
("png", "jpeg") |
("jpeg",) |
default_output_format |
"png" |
None |
None |
None |
supported_sizes |
("1K", "1.5K", "2K") |
None |
None |
None |
custom_size_pixels_range |
(921600, 4624220) |
None |
None |
None |
custom_size_aspect_ratio_range |
(1/16, 16) |
None |
None |
None |
supports_watermark |
True |
True |
True |
True |
supports_prompt_optimization |
True |
True |
True |
True |
prompt_optimization_modes |
("standard",) |
("standard", "fast") |
("standard", "fast") |
("standard", "fast") |
default_prompt_optimization_mode |
None |
SEEDREAM_PROMPT_OPTIMIZATION_MODE ("standard") |
None |
None |
supports_transparent_background |
True |
True |
False |
False |
supports_transparent_generation |
True |
False |
False |
False |
supports_auto_background |
True |
False |
False |
False |
supports_safety_identifier |
True |
True |
False |
False |
Seedream 5.5 is the built-in image default. Start with the
Seedream tool reference
for transparent images and request previews.
This integration covers generation and editing,
not layer decomposition or OCR; those input fields are rejected. background="transparent" supports a text-only
prompt or up to ten references without requiring existing alpha. 5.0 Pro retains
its requirement for exactly one already-transparent PNG. Transparent output
uses PNG when output_format is omitted; JPEG is rejected locally.
5.5 accepts named tiers 1K, 1.5K, 2K (provider default), or positive
WIDTHxHEIGHT dimensions with inclusive pixel count 921,600-4,624,220 and
width/height ratio 1/16-16. auto, 3K, and 4K sizes are rejected for generation.
Older-family size validation is unchanged and remains provider-side.
background defaults to auto on 5.5: the prompt determines transparency,
except JPEG is always opaque. Explicit prompt_optimization="standard" is
accepted; fast is rejected. Omitted optimization uses the provider’s standard
default, not the Pro-only operator setting.
Discovery exposes named tiers, custom-size bounds, background_modes,
prompt_optimization_modes, and supports_safety_identifier, alongside the
transparency flags. Optional safety_identifier accepts 1-64 non-space printable
ASCII characters on 5.5 and Pro. Use a stable hashed user identifier, not raw
personal data. Dry-run previews and structured logs redact it; generation
metadata preserves the provider’s echo.
The generation controls and size/reference bounds match the GA
image API
(accessed 2026-11-16). Seedream 5.5 does not document seed in either the
parameter reference or the official OpenAPI schema. The MCP retains it as an
explicit compatibility field; omit it unless provider acceptance has been
verified. The schema also lags GA fields such as background="auto", so schema
omissions alone are not proof of rejection. These contract checks do not
establish live editing quality, output alpha or account access. Use the opt-in
scripts/smoke_seedream55.py checks and review the results before release;
the script is offline by default.
Per-call provenance records the provider request ID, submitted seed, final-prompt SHA-256 and allowlisted options. Advisory byte inspection measures format, dimensions and actual alpha without replacing provider reports. It does not prove prompt adherence or visual quality; see artifacts and the review/edit workflow.
default_prompt_optimization_modecontrols what 5.0 Pro sends asoptimize_prompt_options.modewhen a caller omitsprompt_optimization. It is operator-configurable throughSEEDREAM_PROMPT_OPTIMIZATION_MODEand defaults tostandardfor prompt-following quality; setfastto favor latency.Both modes are accepted by the 5.0 Pro API — verified against the live endpoint, where
fastandstandardeach returned HTTP 200 with an image. An earlier field report thatfastis rejected did not reproduce. The choice is a quality-versus-latency tradeoff, not a compatibility constraint.An explicit per-call
prompt_optimizationalways wins over the configured default; families without a default send no mode and the provider applies its own.
Images have no separate
ratiotool input. 5.5 customWIDTHxHEIGHTdimensions are checked againstcustom_size_aspect_ratio_rangeandcustom_size_pixels_range; older families leave those bounds unspecified.
Video capabilities (VideoCapabilities)
Section titled “Video capabilities (VideoCapabilities)”The three Seedance 2.0 families share: max_reference_images=9,
max_reference_videos=3, max_reference_audios=3, supports_seed=False,
supports_camera_fixed=False, supports_frames=False,
supports_service_tier_flex=False, duration_range=(4, 15),
priority_range=(0, 9), execution_expires_after_range=(3600, 259200).
Only supported_resolutions differs:
| Family | supported_resolutions |
|---|---|
seedance_2_5 |
("480p", "720p", "1080p", "4k") |
seedance_2_5_flash |
("480p", "720p") |
MINI |
("480p", "720p") |
FAST |
("480p", "720p") |
STANDARD |
("480p", "720p", "1080p", "4k") |
Seedance 2.5 differs from the 2.0 families in duration (
duration_range=(4, 30)versus(4, 15);-1for auto is accepted separately) and reference counts (max_reference_images=30,max_reference_videos=10,max_reference_audios=10). All four resolutions, including4k, use the same regular 2.5 model. Flash shares the 2.5 duration range but has reference limits of 10 images, 5 videos and 5 audios (20 total), and supports only 480p/720p and no Draft mode. Both 2.5 families setsupports_audio_only_input=True,supported_task_types=("auto", "reference", "edit", "extend"), andsupported_output_formats=("mp4", "mov").No additional model or feature flag is required for 4K. Custom endpoints bound to
seedance_2_5share these same resolution limits.Draft mode is described by
supports_draftanddraft_final_resolution: Only regular 2.5 setssupports_draft=True; Flash sets it toFalsewithdraft_final_resolution=None. Drafts are always480p, and a final rendered from a Draft is1080p. 4K Draft promotion is not advertised or accepted.
The GA create-task API (accessed 2026-11-16) confirms Flash’s 10/5/5 reference limits. Drafts created at or after 2026-10-28 10:00 UTC+8 (02:00 UTC) are valid for 14 days; earlier Drafts retain seven days. The local check selects the window using the provider’s creation timestamp, not the current date. Missing or unparsable timestamps leave the expiry decision to the provider.
Regular 2.5 4K contract caveat: the server retains its existing 4k
resolution acceptance on the same model, but the current model-specific API
docs list regular 2.5 only through 1080p and assign 4K to 2.0. The generic
OpenAPI resolution enum includes 4k without establishing support for a
specific model. A successful dry run proves local validation only, not
provider acceptance; confirm support before a paid regular 2.5 4K request.
Every Seedance model is called through one tool, seedance_create_task.
Its input schema advertises the widest limits; at runtime the tool checks the
request against the selected model’s VideoCapabilities (including
supported_ratios) and reports every violation in one error that points to
ark_list_models(kind='video'), which exposes these limits to clients.
Seed Speech ASR (Speech-to-Text)
Section titled “Seed Speech ASR (Speech-to-Text)”Speech-to-text uses the Seed Speech ASR HTTP API. It reuses
BYTEPLUS_SEED_SPEECH_API_KEY — the same key that powers Seed Audio. Audio
is submitted via HTTP and polled until transcription is complete; the full
TranscriptionResult is returned in the background result after the client
submits through the ordinary ark_job_* tools, or native MCP task augmentation
where supported.
ASR does not expose a separate provider task tool: its HTTP submit-and-poll lifecycle is contained within the Ark background job.
Supported audio formats: wav, mp3, ogg, raw, flac.
The HTTP base URL is configured via SEED_SPEECH_ASR_BASE_URL (default
https://voice.ap-southeast-1.bytepluses.com). Poll interval and max wait
are tunable via SEED_SPEECH_ASR_POLL_INTERVAL_SECONDS and
SEED_SPEECH_ASR_POLL_MAX_SECONDS.
Default model IDs
Section titled “Default model IDs”| Field | Env var | Default | Implied family |
|---|---|---|---|
seedream_default_model |
SEEDREAM_DEFAULT_MODEL |
dola-seedream-5-5-261028 |
SEEDREAM_5_5 |
seedance_default_model |
SEEDANCE_DEFAULT_MODEL |
dreamina-seedance-2-5-260628 |
SEEDANCE_2_5 |
seed_understanding_default_model |
SEED_UNDERSTANDING_DEFAULT_MODEL |
dola-seed-2-1-turbo-260628 |
TURBO |
seed_audio_understanding_model |
SEED_AUDIO_UNDERSTANDING_MODEL |
seed-2-0-lite-260428 |
— (single model, no family registry) |
The “implied family” defaults are hard-coded in Settings.validate_model_bindings
and apply only when the default model ID equals one of the recognized built-in
IDs. dola-seedream-5-0-pro-260628 remains recognized as PRO when explicitly
configured. See the default upgrade note.
dola-seed-evolving is also a recognized built-in ID that auto-resolves to
family PRO when explicitly set as the default (no need to set
SEED_UNDERSTANDING_MODEL_FAMILY for it).
SEED_AUDIO_UNDERSTANDING_MODEL is a plain model ID for seed_audio_understand
with no family binding. It must accept Chat Completions input_audio parts
with deep thinking; seed-2-0-lite-260428 is an interim choice and is expected
to be replaced by a newer audio-capable model.
The default model limits direct URL audio to 25 MiB. seed_audio_understand
can upload URL audio/video through the Files API (up to 512 MB per file);
video frames sampled at 0.2 fps also consume input tokens. delivery selects
automatic fallback, direct-only input, or explicit Files API upload.
See audio understanding.
ark_list_models also returns cached account-specific pricing when AK/SK
enables the background pricing refresher. Unavailable or stale rates are null.
Image and understanding estimates use recognized fresh catalog units;
custom endpoint IDs without a catalog match retain built-in estimates.
Environment variables
Section titled “Environment variables”| Env var | Format | Default |
|---|---|---|
SEEDREAM_MODEL_BINDINGS |
JSON array of {"model_id": str, "family": "seedream_5_5"|"pro"|"lite"|"4x"} |
[] |
SEEDANCE_MODEL_BINDINGS |
JSON array of {"model_id": str, "family": "standard"|"fast"|"mini"|"seedance_2_5"|"seedance_2_5_flash"} |
[] |
SEED_UNDERSTANDING_MODEL_BINDINGS |
JSON array of {"model_id": str, "family": "pro"|"turbo"} |
[] |
SEEDREAM_MODEL_FAMILY |
single family string | "" |
SEEDANCE_MODEL_FAMILY |
single family string | "" |
SEED_UNDERSTANDING_MODEL_FAMILY |
single family string | "" |
Examples (from .env.example):
SEEDREAM_MODEL_BINDINGS=[{"model_id":"my-image-endpoint","family":"pro"}]SEEDANCE_MODEL_BINDINGS=[{"model_id":"my-video-endpoint","family":"standard"}]How a model_id resolves to a family
Section titled “How a model_id resolves to a family”Resolution is binding-table based, not string-pattern based (no inference from the model ID string):
- If
SEEDREAM_MODEL_BINDINGS(resp.SEEDANCE_MODEL_BINDINGS) is set, it is the source of truth — each binding explicitly pairsmodel_id↔family. - If the bindings list is empty and
SEEDREAM_MODEL_FAMILY(resp.SEEDANCE_MODEL_FAMILY) is non-empty, a single binding is synthesized from the default model ID + that family. - If both are empty and the default model ID equals a recognized built-in
ID, its known family is used (Seedream
SEEDREAM_5_5or legacyPRO, SeedanceSTANDARD,SEEDANCE_2_5, orSEEDANCE_2_5_FLASH, Seed understandingTURBOfor the defaultdola-seed-2-1-turbo-260628, andPROfordola-seed-evolvingwhen explicitly opted in). - Otherwise (custom default model, no family, no bindings) → startup fails
with
ValueError.
When synthesizing Seedance bindings from an empty list, the server also adds
dreamina-seedance-2-5-flash-261115 as seedance_2_5_flash, unless it is already
the default. Explicit non-empty binding lists are never augmented. Regular
2.5 remains the default when SEEDANCE_DEFAULT_MODEL is omitted.
Validation rules
Section titled “Validation rules”CapabilityRegistry methods (get_image_capabilities(model_id),
get_video_capabilities(model_id), list_image_models(),
list_video_models(), plus validate_image_size,
validate_output_format, validate_resolution, validate_duration):
| Check | Rejected when |
|---|---|
Unknown model_id |
not in the configured registry (ValueError lists allowed IDs) |
Image output_format |
not in supported_output_formats (e.g. png on a 4X model, which only supports jpeg) |
Image size |
Not a supported named tier or valid custom dimensions. 5.5 enforces pixel-count and aspect-ratio bounds; older families leave sizes to the provider |
Video resolution |
not in supported_resolutions (e.g. 1080p/4k on MINI/FAST) |
Video duration |
outside duration_range unless -1 (auto) |
Settings startup additionally rejects: duplicate model IDs within a
family’s bindings, a default model ID missing from bindings, a custom
default model with no family/binding, unknown family strings, non-HTTPS
provider URLs, non-positive ARTIFACT_TTL_SECONDS /
MCP_INLINE_MEDIA_MAX_BYTES / timeouts, invalid allowed origins, shared-key
auth with a missing or malformed MCP_API_KEY_SHA256, and non-loopback HTTP
without MCP_AUTH_MODE=api_key.
Module-level singleton
Section titled “Module-level singleton”get_capability_registry()— returns a process-cachedCapabilityRegistry(lazily built).refresh_capability_registry()— force-rebuilds after config changes (callrefresh_settings()first).
What the registry does not do
Section titled “What the registry does not do”- It is not a cost table. Cost estimation (see runtime.md) is per product and per variation; video families select a static per-task fallback, not a resolution-, duration-, or token-based provider quote.
- It does not infer size or ratio limits for families without configured bounds.
- It does not call the provider — it is a local, pre-flight validation layer built entirely from configuration.
Checking capability drift
Section titled “Checking capability drift”The server validates requests against the static registry in
config/model_capabilities.py. ark_list_models reports those limits to
clients, restricting understanding modalities and media counts to the named
tool’s input contract (audio-only, up to eight clips for
seed_audio_understand). To compare the registry with BytePlus’s ArkModels catalog:
make audit-capabilities CATALOG=path/to/arkmodels.jsonThe dump may be an ArkModels list or arkcli models get <id> --format json
output. Each row reports OK, DRIFT or UNKNOWN for one parameter of one
configured binding; --strict exits non-zero on drift. The report is
advisory. The ArkModels catalog has known stale entries (for example,
Seedance 2.5 Draft mode marked unsupported, which the official tutorial shows
working), so review drift against the official docs before changing the
registry.