> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modelslab.com/llms.txt
> Use this file to discover all available pages before exploring further.
# ControlNet Video
> Generate videos from a text prompt guided by a control video using the ControlNet Video API. The supported model_id is `h3-minimax-controlnet`.
Generate a video from a prompt while keeping the structure of a control video: edges, depth, pose and similar signals. Send the ControlNet video as `init_video`, and set `controlnet_type` to the kind of signal it carries.
`model_id` is required and must be `h3-minimax-controlnet`. `controlnet_type` must be one of the ControlNet models listed on ModelsLab, such as `canny`, `depth`, `openpose`, `lineart` or `scribble`.
`init_video` should be a ControlNet video, meaning a video that already shows the control signal, such as a canny edge map, a depth map or an OpenPose skeleton. It is not the source footage. Its signal must match `controlnet_type`; for example, send a canny edge video with `controlnet_type: canny`.
The maximum resolution is **1440** pixels: `width` and `height` must each be between 320 and 1440.
## Request
Make a `POST` request to below endpoint and pass the required parameters in the request body.
```curl curl theme={"theme":{"light":"github-light","dark":"github-dark"}}
--request POST 'https://modelslab.com/api/v6/video/controlnet' \
```
## Body
```json json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"key": "your_api_key",
"model_id": "h3-minimax-controlnet",
"prompt": "A cinematic astronaut walking gently on the moon, preserve the helmet silhouette and arm positions from the control video",
"controlnet_type": "canny",
"init_video": "https://huggingface.co/alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0/resolve/7d2c95de2e351ed6a0b360af45f62daa04c26f88/asset/canny.mp4",
"convert_to_controlnet_input": false,
"duration": 5,
"width": 1280,
"height": 704,
"num_inference_steps": 40,
"seed": 43,
"webhook": null,
"track_id": null
}
```
Keep `convert_to_controlnet_input` set to `false` when `init_video` is a ControlNet video, which is the expected input. Setting it to `true` makes the model extract the `controlnet_type` signal from an ordinary video first.
## Response
The request is queued and answers right away with `processing`. The `future_links` field holds the URLs where the video will appear. Poll [Fetch Video](/video-api/fetch-video) with the returned `id`, or pass a `webhook` to be notified when it's ready.
```json json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"status": "processing",
"tip": "Get 20x faster image generation using enterprise plan. Click here : https://modelslab.com/enterprise",
"eta": 80,
"message": "Try to fetch request after seconds estimated",
"fetch_result": "https://modelslab.com/api/v6/video/fetch/123456789",
"id": 123456789,
"output": [],
"future_links": [
"https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/generations/7f3c2a1e-1b2c-4d5e-8f90-1234567890ab-0.mp4"
],
"meta": {
"prompt": "A cinematic astronaut walking gently on the moon, preserve the helmet silhouette and arm positions from the control video",
"controlnet_type": "canny",
"control_video": "https://huggingface.co/alibaba-pai/MiniMax-H3-Fun-Controlnet-Union-2.0/resolve/7d2c95de2e351ed6a0b360af45f62daa04c26f88/asset/canny.mp4",
"convert_to_controlnet_input": false,
"duration": 5,
"width": 1280,
"height": 704,
"num_inference_steps": 40,
"num_samples": 1,
"seed": 43
}
}
```
## OpenAPI
````yaml POST /video/controlnet
openapi: 3.1.0
info:
title: ModelsLab Video API
description: >-
A comprehensive API for AI-driven video generation including text-to-video,
image-to-video, scene transitions, and video management capabilities
license:
name: MIT
version: 6.0.0
servers:
- url: https://modelslab.com/api/v6
security: []
paths:
/video/controlnet:
post:
summary: Generate video with ControlNet
description: >-
Generates a video from a text prompt, keeping the structure (edges,
depth, pose…) of a control video. Use model_id `h3-minimax-controlnet`.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/ControlnetVideoRequest'
responses:
'200':
description: ControlNet video generation response
content:
application/json:
schema:
$ref: '#/components/schemas/VideoResponse'
'400':
description: Bad request - invalid parameters
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
'401':
description: Unauthorized - invalid API key
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
'500':
description: Internal server error
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
ControlnetVideoRequest:
type: object
required:
- key
- model_id
- prompt
- controlnet_type
- init_video
properties:
key:
type: string
description: Your API Key used for request authorization
model_id:
type: string
enum:
- h3-minimax-controlnet
description: The model to use. This endpoint only serves `h3-minimax-controlnet`.
prompt:
type: string
description: Text prompt describing the video to generate.
controlnet_type:
type: string
description: >-
The ControlNet model the control video represents, e.g. `canny`,
`depth`, `openpose`, `lineart`, `scribble`. Must be one of the
ControlNet models listed on ModelsLab.
init_video:
type: string
format: uri
description: >-
URL of the ControlNet video: a video that already shows the control
signal (e.g. a canny edge map, depth map or OpenPose skeleton)
matching `controlnet_type`, not the source footage. The output
follows its structure.
convert_to_controlnet_input:
type: boolean
default: false
description: >-
Set to true to send an ordinary video and let the model extract the
`controlnet_type` signal from it. Keep false when `init_video` is a
ControlNet video, which is the expected input.
duration:
type: integer
default: 5
minimum: 5
maximum: 15
description: Output length in seconds. Values outside 5–15 are clamped.
width:
type: integer
default: 1280
minimum: 320
maximum: 1440
description: Output width in pixels, from 320 to 1440.
height:
type: integer
default: 704
minimum: 320
maximum: 1440
description: Output height in pixels, from 320 to 1440.
num_inference_steps:
type: integer
default: 40
description: >-
Number of denoising steps. Higher values may improve quality but
increase processing time.
seed:
oneOf:
- type: integer
- type: 'null'
description: Seed for reproducible results. Random if omitted.
instant_response:
type: boolean
default: true
description: Return future links immediately instead of waiting for the result.
temp:
type: boolean
default: false
description: Store the output in temporary storage.
webhook:
type: string
format: uri
description: URL to receive a POST call once generation completes.
track_id:
type: string
description: ID returned in the webhook payload to identify this request.
VideoResponse:
type: object
properties:
status:
type: string
enum:
- success
- processing
- error
description: Status of the video generation
generationTime:
type: number
description: Time taken to generate the video in seconds
id:
type: integer
description: Unique identifier for the video generation
output:
type: array
items:
type: string
format: uri
description: Array of generated video URLs
proxy_links:
type: array
items:
type: string
format: uri
description: Array of proxy video URLs
future_links:
type: array
items:
type: string
format: uri
description: Array of future video URLs for queued requests
meta:
type: object
description: Metadata about the video generation including all parameters used
eta:
type: integer
description: Estimated time for completion in seconds (processing status)
message:
type: string
description: Status message or additional information
tip:
type: string
description: Additional information or tips for the user
fetch_result:
type: string
format: uri
description: URL to fetch the result when processing
Error:
type: object
required:
- status
- message
properties:
status:
type: string
enum:
- error
message:
type: string
description: Error message description
````
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.