POST
/
controlnet
Generate images using ControlNet
curl --request POST \
  --url https://modelslab.com/api/v5/controlnet \
  --header 'Content-Type: application/json' \
  --data '{
  "key": "<string>",
  "model_id": "<string>",
  "controlnet_model": "canny",
  "controlnet_type": "canny",
  "auto_hint": "yes",
  "guess_mode": "no",
  "prompt": "<string>",
  "negative_prompt": "<string>",
  "init_image": "<string>",
  "control_image": "<string>",
  "mask_image": "<string>",
  "width": 512,
  "height": 512,
  "samples": 1,
  "scheduler": "UniPCMultistepScheduler",
  "tomesd": "yes",
  "use_karras_sigmas": "yes",
  "algorithm_type": "none",
  "vae": "<string>",
  "lora_strength": "<string>",
  "lora_model": "<string>",
  "num_inference_steps": 30,
  "safety_checker": "no",
  "ip_adapter_id": "ip-adapter_sdxl",
  "ip_adapter_scale": 0.5,
  "ip_adapter_image": "<string>",
  "enhance_prompt": "yes",
  "controlnet_conditioning_scale": 2.55,
  "strength": 0.55,
  "guidance_scale": 7.5,
  "seed": 123,
  "webhook": "<string>",
  "track_id": 123,
  "upscale": "yes",
  "clip_skip": 4,
  "base64": "no",
  "temp": "no"
}'
{
  "status": "success",
  "id": 123,
  "output": [
    "<string>"
  ],
  "future_links": [
    "<string>"
  ],
  "meta": {
    "prompt": "<string>",
    "model_id": "<string>",
    "controlnet_model": "<string>",
    "controlnet_type": "<string>",
    "negative_prompt": "<string>",
    "scheduler": "<string>",
    "safety_checker": "<string>",
    "auto_hint": "<string>",
    "guess_mode": "<string>",
    "strength": 123,
    "W": 123,
    "H": 123,
    "guidance_scale": 123,
    "controlnet_conditioning_scale": "<string>",
    "seed": 123,
    "steps": 123,
    "n_samples": 123,
    "clip_skip": 123
  },
  "eta": 123,
  "message": "<string>",
  "messege": "<string>",
  "tip": "<string>",
  "tip_1": "<string>",
  "fetch_result": "<string>"
}
The ControlNet models are available in this API.
You can also use this endpoint to inpaint images with ControlNet. Just make sure to pass the link to the mask_image in the request body and use the controlnet_model parameter with “inpaint” value.
Contonet image

​
Request

Send a POST request to below endpoint.
curl
--request POST 'https://modelslab.com/api/v5/controlnet' \

​
Models

ControlNet API using ControlNet 1.1 as the default: Supported ControlNet Models:
For Flux ControlNet, use
`model_id`: `fluxdev`
which supports following controlnet types:canny, tile, depth , blur, pose, gray and low_quality.
ControlNet Model/TypeSDXLSD1.5FluxDev
CannyYesYesYes
TileYesYesYes
DepthYesYesYes
BlurYesYesYes
PoseYesYesYes
MLSDYesYesNo
LineartYesYesNo
HEDYesYesNo
NormalYesYesNo
OpenPoseYesYesNo
SegmentationYesYesNo
InpaintYesYesNo
SoftEdgeYesYesNo
ShuffleYesYesNo
QRCodeYesYesNo
Low QualityNoNoYes
GrayNoNoYes

​
ControlNet Types Overview

TypeDescriptionBest For
CannyEdge detectionLine art, outlines, structural control
DepthDepth map control3D structure, perspective control
HEDAdvanced edge detectionDetailed edge preservation
MLSDLine segment detectionArchitecture, geometric structures
NormalSurface normal maps3D lighting, surface details
ScribbleSketch to imageConverting drawings to realistic images
In-PaintFill masked areasRemoving or replacing objects
Soft EdgeSmooth edge detectionNatural, organic shapes
Line ArtClean line artAnime, cartoon, illustration styles

​
Key Parameters

  • controlnet_type & controlnet_model: Must match the desired control method
  • auto_hint: Set to “yes” to automatically process input images
  • controlnet_conditioning_scale: Controls strength (0.1-2.0, default 0.5-1.0)
  • init_image: Source image for control guidance
  • mask_image: Required only for inpainting

​
Body

You can also use multi ControlNet. Just make sure to pass comma separated controlnet models to the controlnet_model as canny,depth and init_image in the request body.

​
ControlNet API Examples

{
  "key": "your_api_key",
  "controlnet_model": "canny",
  "controlnet_type": "canny",
  "model_id": "realistic-vision-51",
  "auto_hint": "yes",
  "guess_mode": "no",
  "prompt": "a girl, wearing red bikini, looking at camera, ocean in background",
  "negative_prompt": "human, unstructure, (black object, white object), colorful background, nsfw",
  "init_image": "https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/livewire-tmp/qyA2waUVja192VOgQ3WwIRuKxJzRpA-metaZDc3NTFmOGE3NjgxYzA4MGQ1Yjg3NjA2ZjFlYzU1YTMuanBn-.jpg",
  "mask_image": null,
  "width": "512",
  "height": "512",
  "samples": "1",
  "scheduler": "UniPCMultistepScheduler",
  "num_inference_steps": "30",
  "safety_checker": "no",
  "enhance_prompt": "yes",
  "guidance_scale": 7.5,
  "strength": 0.55,
  "seed": null,
  "webhook": null,
  "track_id": null
}

Body

application/json
​
key
string
required

Your API Key used for request authorization

​
model_id
string

The ID of the model to be used. It can be public or your trained model. Note: Multi ControlNet does not apply when using model with 'flux'

​
controlnet_model
enum<string>

ControlNet model ID. Can be single model or comma-separated for multi-ControlNet (e.g., 'canny,depth,openpose')

Available options:
canny,
depth,
hed,
mlsd,
normal,
openpose,
scribble,
segmentation,
inpaint,
softedge,
lineart,
shuffle,
tile,
face_detector,
qrcode,
blur,
pose,
gray,
low_quality
​
controlnet_type
enum<string>

ControlNet model type. Should match one of the controlnet_model values

Available options:
canny,
depth,
hed,
mlsd,
normal,
openpose,
scribble,
segmentation,
inpaint,
softedge,
lineart,
shuffle,
tile,
face_detector,
qrcode,
blur,
pose,
gray,
low_quality
​
auto_hint
enum<string>
default:yes

Auto hint image generation

Available options:
yes,
no
​
guess_mode
enum<string>
default:no

Set to 'yes' if you don't pass any prompt. The model will try to guess what's in the init_image and create best variations

Available options:
yes,
no
​
prompt
string

Text prompt with description of required image modifications. Make it as detailed as possible for best results

​
negative_prompt
string

Items you don't want in the image

​
init_image
string<uri>

Link to the initial image to be used as a reference

​
control_image
string<uri>

Link to the ControlNet image

​
mask_image
string<uri>

Link to the mask image for inpainting

​
width
integer
default:512

Width of the generated image. Maximum 1024x1024

Required range: 64 <= x <= 1024
​
height
integer
default:512

Height of the generated image. Maximum 1024x1024

Required range: 64 <= x <= 1024
​
samples
integer
default:1

Number of images to be returned in response. Maximum value is 4

Required range: 1 <= x <= 4
​
scheduler
enum<string>
default:UniPCMultistepScheduler

Scheduler to use for image generation

Available options:
DDPMScheduler,
DDIMScheduler,
PNDMScheduler,
LMSDiscreteScheduler,
EulerDiscreteScheduler,
EulerAncestralDiscreteScheduler,
DPMSolverMultistepScheduler,
HeunDiscreteScheduler,
KDPM2DiscreteScheduler,
DPMSolverSinglestepScheduler,
KDPM2AncestralDiscreteScheduler,
UniPCMultistepScheduler,
DDIMInverseScheduler,
DEISMultistepScheduler,
IPNDMScheduler,
KarrasVeScheduler,
ScoreSdeVeScheduler,
LCMScheduler
​
tomesd
enum<string>
default:yes

Enable tomesd to generate images with fast results

Available options:
yes,
no
​
use_karras_sigmas
enum<string>
default:yes

Use Karras sigmas to generate images with nice results

Available options:
yes,
no
​
algorithm_type
enum<string>
default:none

Algorithm type used in DPMSolverMultistepScheduler

Available options:
dpmsolver+++,
none
​
vae
string

Use custom VAE in generating images

​
lora_strength
string

Specify the strength of the LoRa model. If using multiple LoRa, provide comma-separated values. Range: 0.1 to 1

​
lora_model
string

Multi LoRa supported, pass comma-separated values. Example: 'contrast-fix,yae-miko-genshin'

​
num_inference_steps
enum<integer>
default:30

Number of denoising steps

Available options:
21,
31
​
safety_checker
enum<string>
default:no

A checker for NSFW images. If detected, replaces with blank image

Available options:
yes,
no
​
ip_adapter_id
enum<string>

IP adapter ID for additional image conditioning

Available options:
ip-adapter_sdxl,
ip-adapter_sd15,
ip-adapter-plus-face_sd15
​
ip_adapter_scale
number

Scale for IP adapter, should be between 0 and 1

Required range: 0 <= x <= 1
​
ip_adapter_image
string<uri>

Valid image URL for IP adapter

​
enhance_prompt
enum<string>
default:yes

Enhance prompts for better results

Available options:
yes,
no
​
controlnet_conditioning_scale
number

Scale for ControlNet guidance. Accepts floating values from 0.1 to 5

Required range: 0.1 <= x <= 5
​
strength
number
default:0.55

Prompt strength when using init_image. 1.0 corresponds to full destruction of information in the init image

Required range: 0 <= x <= 1
​
guidance_scale
number
default:7.5

Guidance scale for generation

Required range: 1 <= x <= 20
​
seed
integer

Seed for reproducible results. Pass null for random number

​
webhook
string<uri>

URL to receive POST notification when image generation is complete

​
track_id
integer

ID returned in webhook response for request identification

​
upscale
enum<string>

Upscale image resolution 2x (e.g., 512x512 becomes 1024x1024)

Available options:
yes,
no
​
clip_skip
integer

Clip skip value

Required range: 1 <= x <= 8
​
base64
enum<string>
default:no

Get response as base64 string, pass init_image, mask_image, control_image as base64

Available options:
yes,
no
​
temp
enum<string>
default:no

Create temporary image link valid for 24 hours

Available options:
yes,
no

Response

ControlNet generation response

​
status
enum<string>

Status of the image generation

Available options:
success,
processing,
error
​
id
integer

Unique identifier for the image generation

​
output
string<uri>[]

Array of generated image URLs

​
future_links
string<uri>[]

Array of future image URLs for queued requests

​
meta
object

Metadata about the image generation including all parameters used

​
eta
number

Estimated time for completion in seconds (for processing status)

​
message
string

Status message or additional information

​
messege
string

Status message (alternative spelling used in API)

​
tip
string

Additional tips or information

​
tip_1
string

Additional tips or information

​
fetch_result
string<uri>

URL to fetch the result when processing