Skip to content

Model Capability Registry

The capability registry (config/model_capabilities.py) describes, per configured model, what inputs the server will accept before it ever calls the provider. It validates image sizes, output formats, video resolutions, and durations against family-specific rules, so unsupported requests fail fast with an actionable ValueError.

ModelFamily (StrEnum) includes:

Member Value
SEEDREAM_5_5 seedream_5_5
SEEDREAM_PRO seedream_pro
SEEDREAM_LITE seedream_lite
SEEDREAM_4X seedream_4x
SEEDANCE_2 seedance_2
SEEDANCE_2_FAST seedance_2_fast
SEEDANCE_2_MINI seedance_2_mini
SEEDANCE_2_5 seedance_2_5
SEEDANCE_2_5_FLASH seedance_2_5_flash
SEED_2_1_PRO seed_2_1_pro
SEED_2_1_TURBO seed_2_1_turbo
SEED3D_HYPER3D seed3d_hyper3d
SEED3D_HITEM3D seed3d_hitem3d

The binding enums (config/env.py):

  • SeedreamFamily: SEEDREAM_5_5 = "seedream_5_5", PRO = "pro", LITE = "lite", V4X = "4x".
  • SeedanceFamily: STANDARD = "standard", FAST = "fast", MINI = "mini", SEEDANCE_2_5 = "seedance_2_5", SEEDANCE_2_5_FLASH = "seedance_2_5_flash".
  • SeedUnderstandingFamily: PRO = "pro", TURBO = "turbo".
  • Seed3DFamily: HYPER3D = "hyper3d", HITEM3D = "hitem3d".
Field SEEDREAM_5_5 PRO LITE 4X
max_references 10 10 14 14
supports_batch False False True True
supports_streaming False False True True
supported_output_formats ("png", "jpeg") ("png", "jpeg") ("png", "jpeg") ("jpeg",)
default_output_format "png" None None None
supported_sizes ("1K", "1.5K", "2K") None None None
custom_size_pixels_range (921600, 4624220) None None None
custom_size_aspect_ratio_range (1/16, 16) None None None
supports_watermark True True True True
supports_prompt_optimization True True True True
prompt_optimization_modes ("standard",) ("standard", "fast") ("standard", "fast") ("standard", "fast")
default_prompt_optimization_mode None SEEDREAM_PROMPT_OPTIMIZATION_MODE ("standard") None None
supports_transparent_background True True False False
supports_transparent_generation True False False False
supports_auto_background True False False False
supports_safety_identifier True True False False

Seedream 5.5 is the built-in image default. Start with the Seedream tool reference for transparent images and request previews. This integration covers generation and editing, not layer decomposition or OCR; those input fields are rejected. background="transparent" supports a text-only prompt or up to ten references without requiring existing alpha. 5.0 Pro retains its requirement for exactly one already-transparent PNG. Transparent output uses PNG when output_format is omitted; JPEG is rejected locally.

5.5 accepts named tiers 1K, 1.5K, 2K (provider default), or positive WIDTHxHEIGHT dimensions with inclusive pixel count 921,600-4,624,220 and width/height ratio 1/16-16. auto, 3K, and 4K sizes are rejected for generation. Older-family size validation is unchanged and remains provider-side.

background defaults to auto on 5.5: the prompt determines transparency, except JPEG is always opaque. Explicit prompt_optimization="standard" is accepted; fast is rejected. Omitted optimization uses the provider’s standard default, not the Pro-only operator setting.

Discovery exposes named tiers, custom-size bounds, background_modes, prompt_optimization_modes, and supports_safety_identifier, alongside the transparency flags. Optional safety_identifier accepts 1-64 non-space printable ASCII characters on 5.5 and Pro. Use a stable hashed user identifier, not raw personal data. Dry-run previews and structured logs redact it; generation metadata preserves the provider’s echo.

The generation controls and size/reference bounds match the GA image API (accessed 2026-11-16). Seedream 5.5 does not document seed in either the parameter reference or the official OpenAPI schema. The MCP retains it as an explicit compatibility field; omit it unless provider acceptance has been verified. The schema also lags GA fields such as background="auto", so schema omissions alone are not proof of rejection. These contract checks do not establish live editing quality, output alpha or account access. Use the opt-in scripts/smoke_seedream55.py checks and review the results before release; the script is offline by default.

Per-call provenance records the provider request ID, submitted seed, final-prompt SHA-256 and allowlisted options. Advisory byte inspection measures format, dimensions and actual alpha without replacing provider reports. It does not prove prompt adherence or visual quality; see artifacts and the review/edit workflow.

default_prompt_optimization_mode controls what 5.0 Pro sends as optimize_prompt_options.mode when a caller omits prompt_optimization. It is operator-configurable through SEEDREAM_PROMPT_OPTIMIZATION_MODE and defaults to standard for prompt-following quality; set fast to favor latency.

Both modes are accepted by the 5.0 Pro API — verified against the live endpoint, where fast and standard each returned HTTP 200 with an image. An earlier field report that fast is rejected did not reproduce. The choice is a quality-versus-latency tradeoff, not a compatibility constraint.

An explicit per-call prompt_optimization always wins over the configured default; families without a default send no mode and the provider applies its own.

Images have no separate ratio tool input. 5.5 custom WIDTHxHEIGHT dimensions are checked against custom_size_aspect_ratio_range and custom_size_pixels_range; older families leave those bounds unspecified.

The three Seedance 2.0 families share: max_reference_images=9, max_reference_videos=3, max_reference_audios=3, supports_seed=False, supports_camera_fixed=False, supports_frames=False, supports_service_tier_flex=False, duration_range=(4, 15), priority_range=(0, 9), execution_expires_after_range=(3600, 259200).

Only supported_resolutions differs:

Family supported_resolutions
seedance_2_5 ("480p", "720p", "1080p", "4k")
seedance_2_5_flash ("480p", "720p")
MINI ("480p", "720p")
FAST ("480p", "720p")
STANDARD ("480p", "720p", "1080p", "4k")

Seedance 2.5 differs from the 2.0 families in duration (duration_range=(4, 30) versus (4, 15); -1 for auto is accepted separately) and reference counts (max_reference_images=30, max_reference_videos=10, max_reference_audios=10). All four resolutions, including 4k, use the same regular 2.5 model. Flash shares the 2.5 duration range but has reference limits of 10 images, 5 videos and 5 audios (20 total), and supports only 480p/720p and no Draft mode. Both 2.5 families set supports_audio_only_input=True, supported_task_types=("auto", "reference", "edit", "extend"), and supported_output_formats=("mp4", "mov").

No additional model or feature flag is required for 4K. Custom endpoints bound to seedance_2_5 share these same resolution limits.

Draft mode is described by supports_draft and draft_final_resolution: Only regular 2.5 sets supports_draft=True; Flash sets it to False with draft_final_resolution=None. Drafts are always 480p, and a final rendered from a Draft is 1080p. 4K Draft promotion is not advertised or accepted.

The GA create-task API (accessed 2026-11-16) confirms Flash’s 10/5/5 reference limits. Drafts created at or after 2026-10-28 10:00 UTC+8 (02:00 UTC) are valid for 14 days; earlier Drafts retain seven days. The local check selects the window using the provider’s creation timestamp, not the current date. Missing or unparsable timestamps leave the expiry decision to the provider.

Regular 2.5 4K contract caveat: the server retains its existing 4k resolution acceptance on the same model, but the current model-specific API docs list regular 2.5 only through 1080p and assign 4K to 2.0. The generic OpenAPI resolution enum includes 4k without establishing support for a specific model. A successful dry run proves local validation only, not provider acceptance; confirm support before a paid regular 2.5 4K request.

Every Seedance model is called through one tool, seedance_create_task. Its input schema advertises the widest limits; at runtime the tool checks the request against the selected model’s VideoCapabilities (including supported_ratios) and reports every violation in one error that points to ark_list_models(kind='video'), which exposes these limits to clients.

Speech-to-text uses the Seed Speech ASR HTTP API. It reuses BYTEPLUS_SEED_SPEECH_API_KEY — the same key that powers Seed Audio. Audio is submitted via HTTP and polled until transcription is complete; the full TranscriptionResult is returned in the background result after the client submits through the ordinary ark_job_* tools, or native MCP task augmentation where supported.

ASR does not expose a separate provider task tool: its HTTP submit-and-poll lifecycle is contained within the Ark background job.

Supported audio formats: wav, mp3, ogg, raw, flac.

The HTTP base URL is configured via SEED_SPEECH_ASR_BASE_URL (default https://voice.ap-southeast-1.bytepluses.com). Poll interval and max wait are tunable via SEED_SPEECH_ASR_POLL_INTERVAL_SECONDS and SEED_SPEECH_ASR_POLL_MAX_SECONDS.

Field Env var Default Implied family
seedream_default_model SEEDREAM_DEFAULT_MODEL dola-seedream-5-5-261028 SEEDREAM_5_5
seedance_default_model SEEDANCE_DEFAULT_MODEL dreamina-seedance-2-5-260628 SEEDANCE_2_5
seed_understanding_default_model SEED_UNDERSTANDING_DEFAULT_MODEL dola-seed-2-1-turbo-260628 TURBO
seed_audio_understanding_model SEED_AUDIO_UNDERSTANDING_MODEL seed-2-0-lite-260428 — (single model, no family registry)

The “implied family” defaults are hard-coded in Settings.validate_model_bindings and apply only when the default model ID equals one of the recognized built-in IDs. dola-seedream-5-0-pro-260628 remains recognized as PRO when explicitly configured. See the default upgrade note. dola-seed-evolving is also a recognized built-in ID that auto-resolves to family PRO when explicitly set as the default (no need to set SEED_UNDERSTANDING_MODEL_FAMILY for it).

SEED_AUDIO_UNDERSTANDING_MODEL is a plain model ID for seed_audio_understand with no family binding. It must accept Chat Completions input_audio parts with deep thinking; seed-2-0-lite-260428 is an interim choice and is expected to be replaced by a newer audio-capable model.

The default model limits direct URL audio to 25 MiB. seed_audio_understand can upload URL audio/video through the Files API (up to 512 MB per file); video frames sampled at 0.2 fps also consume input tokens. delivery selects automatic fallback, direct-only input, or explicit Files API upload. See audio understanding.

ark_list_models also returns cached account-specific pricing when AK/SK enables the background pricing refresher. Unavailable or stale rates are null. Image and understanding estimates use recognized fresh catalog units; custom endpoint IDs without a catalog match retain built-in estimates.

Env var Format Default
SEEDREAM_MODEL_BINDINGS JSON array of {"model_id": str, "family": "seedream_5_5"|"pro"|"lite"|"4x"} []
SEEDANCE_MODEL_BINDINGS JSON array of {"model_id": str, "family": "standard"|"fast"|"mini"|"seedance_2_5"|"seedance_2_5_flash"} []
SEED_UNDERSTANDING_MODEL_BINDINGS JSON array of {"model_id": str, "family": "pro"|"turbo"} []
SEEDREAM_MODEL_FAMILY single family string ""
SEEDANCE_MODEL_FAMILY single family string ""
SEED_UNDERSTANDING_MODEL_FAMILY single family string ""

Examples (from .env.example):

SEEDREAM_MODEL_BINDINGS=[{"model_id":"my-image-endpoint","family":"pro"}]
SEEDANCE_MODEL_BINDINGS=[{"model_id":"my-video-endpoint","family":"standard"}]

Resolution is binding-table based, not string-pattern based (no inference from the model ID string):

  1. If SEEDREAM_MODEL_BINDINGS (resp. SEEDANCE_MODEL_BINDINGS) is set, it is the source of truth — each binding explicitly pairs model_id ↔ family.
  2. If the bindings list is empty and SEEDREAM_MODEL_FAMILY (resp. SEEDANCE_MODEL_FAMILY) is non-empty, a single binding is synthesized from the default model ID + that family.
  3. If both are empty and the default model ID equals a recognized built-in ID, its known family is used (Seedream SEEDREAM_5_5 or legacy PRO, Seedance STANDARD, SEEDANCE_2_5, or SEEDANCE_2_5_FLASH, Seed understanding TURBO for the default dola-seed-2-1-turbo-260628, and PRO for dola-seed-evolving when explicitly opted in).
  4. Otherwise (custom default model, no family, no bindings) → startup fails with ValueError.

When synthesizing Seedance bindings from an empty list, the server also adds dreamina-seedance-2-5-flash-261115 as seedance_2_5_flash, unless it is already the default. Explicit non-empty binding lists are never augmented. Regular 2.5 remains the default when SEEDANCE_DEFAULT_MODEL is omitted.

CapabilityRegistry methods (get_image_capabilities(model_id), get_video_capabilities(model_id), list_image_models(), list_video_models(), plus validate_image_size, validate_output_format, validate_resolution, validate_duration):

Check Rejected when
Unknown model_id not in the configured registry (ValueError lists allowed IDs)
Image output_format not in supported_output_formats (e.g. png on a 4X model, which only supports jpeg)
Image size Not a supported named tier or valid custom dimensions. 5.5 enforces pixel-count and aspect-ratio bounds; older families leave sizes to the provider
Video resolution not in supported_resolutions (e.g. 1080p/4k on MINI/FAST)
Video duration outside duration_range unless -1 (auto)

Settings startup additionally rejects: duplicate model IDs within a family’s bindings, a default model ID missing from bindings, a custom default model with no family/binding, unknown family strings, non-HTTPS provider URLs, non-positive ARTIFACT_TTL_SECONDS / MCP_INLINE_MEDIA_MAX_BYTES / timeouts, invalid allowed origins, shared-key auth with a missing or malformed MCP_API_KEY_SHA256, and non-loopback HTTP without MCP_AUTH_MODE=api_key.

  • get_capability_registry() — returns a process-cached CapabilityRegistry (lazily built).
  • refresh_capability_registry() — force-rebuilds after config changes (call refresh_settings() first).
  • It is not a cost table. Cost estimation (see runtime.md) is per product and per variation; video families select a static per-task fallback, not a resolution-, duration-, or token-based provider quote.
  • It does not infer size or ratio limits for families without configured bounds.
  • It does not call the provider — it is a local, pre-flight validation layer built entirely from configuration.

The server validates requests against the static registry in config/model_capabilities.py. ark_list_models reports those limits to clients, restricting understanding modalities and media counts to the named tool’s input contract (audio-only, up to eight clips for seed_audio_understand). To compare the registry with BytePlus’s ArkModels catalog:

Terminal window
make audit-capabilities CATALOG=path/to/arkmodels.json

The dump may be an ArkModels list or arkcli models get <id> --format json output. Each row reports OK, DRIFT or UNKNOWN for one parameter of one configured binding; --strict exits non-zero on drift. The report is advisory. The ArkModels catalog has known stale entries (for example, Seedance 2.5 Draft mode marked unsupported, which the official tutorial shows working), so review drift against the official docs before changing the registry.