Function Calling

Function calling (also called “tool use”) lets models invoke structured functions you define. The model decides when to call a function, generates the arguments, and you execute the function and return the result.

OpenAI-Compatible Function Calling

Use the tools parameter with the Chat Completions endpoint:

Define Tools

{
  "model": "Qwen/Qwen2.5-VL-72B-Instruct-together",
  "messages": [
    {"role": "user", "content": "What's the weather in San Francisco?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City and state, e.g. 'San Francisco, CA'"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"],
              "description": "Temperature unit"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

Model Response with Tool Call

When the model decides to use a tool, the response includes a tool_calls array:

{
  "id": "chat-abc123",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"fahrenheit\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

Return Tool Results

Send the tool result back to continue the conversation:

{
  "model": "Qwen/Qwen2.5-VL-72B-Instruct-together",
  "messages": [
    {"role": "user", "content": "What's the weather in San Francisco?"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"fahrenheit\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "{\"temperature\": 62, \"unit\": \"fahrenheit\", \"condition\": \"foggy\"}"
    }
  ]
}

Full Python Example

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_MODELSLAB_API_KEY",
    base_url="https://modelslab.com/api/v7/llm",
)

# Define your tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City name"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

# Your actual function implementations
def get_weather(location: str) -> str:
    # Replace with real weather API call
    return json.dumps({"temperature": 62, "condition": "foggy"})

# Step 1: Send message with tools
response = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-72B-Instruct-together",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    tool_choice="auto",
)

message = response.choices[0].message

# Step 2: Check if the model wants to call a function
if message.tool_calls:
    # Execute the function
    tool_call = message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    result = get_weather(**args)

    # Step 3: Send the result back
    final_response = client.chat.completions.create(
        model="Qwen/Qwen2.5-VL-72B-Instruct-together",
        messages=[
            {"role": "user", "content": "What's the weather in Paris?"},
            message,
            {
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result,
            },
        ],
        tools=tools,
    )
    print(final_response.choices[0].message.content)
else:
    print(message.content)

Anthropic-Compatible Tool Use

Use the tools parameter with the Messages endpoint:

{
  "model": "Qwen/Qwen2.5-VL-72B-Instruct-together",
  "max_tokens": 1024,
  "tools": [
    {
      "name": "get_weather",
      "description": "Get the current weather for a location",
      "input_schema": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "City name"
          }
        },
        "required": ["location"]
      }
    }
  ],
  "messages": [
    {"role": "user", "content": "What's the weather in San Francisco?"}
  ]
}

Response with Tool Use

{
  "id": "msg_abc123",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "tool_use",
      "id": "toolu_abc123",
      "name": "get_weather",
      "input": {"location": "San Francisco"}
    }
  ],
  "stop_reason": "tool_use"
}

Return Tool Results

{
  "model": "Qwen/Qwen2.5-VL-72B-Instruct-together",
  "max_tokens": 1024,
  "tools": [...],
  "messages": [
    {"role": "user", "content": "What's the weather in San Francisco?"},
    {
      "role": "assistant",
      "content": [
        {
          "type": "tool_use",
          "id": "toolu_abc123",
          "name": "get_weather",
          "input": {"location": "San Francisco"}
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "tool_result",
          "tool_use_id": "toolu_abc123",
          "content": "{\"temperature\": 62, \"condition\": \"foggy\"}"
        }
      ]
    }
  ]
}

Tool Choice Options

Control when the model uses tools:

Value	Behavior
`"auto"`	Model decides whether to call a tool (default)
`"none"`	Model will not call any tools
`"required"`	Model must call at least one tool
`{"type": "function", "function": {"name": "get_weather"}}`	Force a specific tool

Tips

Write clear tool descriptions

The model uses the description field to decide when to call a tool. Be specific about what the tool does and when it should be used.

Use JSON Schema for parameters

Define parameters with JSON Schema including types, descriptions, enums, and required fields. The more precise your schema, the better the model’s arguments will be.

Handle multiple tool calls

Models may want to call multiple tools in a single response. Always check for and handle all tool calls in the tool_calls array.

Parallel tool calls

Some models support calling multiple tools in parallel within a single response. Process all tool calls and return all results before sending the next message.

Using the APIs

Our AI APIs

OpenAI-Compatible Function Calling

Define Tools

Model Response with Tool Call

Return Tool Results

Full Python Example

Anthropic-Compatible Tool Use

Response with Tool Use

Return Tool Results

Tool Choice Options

Tips

​OpenAI-Compatible Function Calling

​Define Tools

​Model Response with Tool Call

​Return Tool Results

​Full Python Example

​Anthropic-Compatible Tool Use

​Response with Tool Use

​Return Tool Results

​Tool Choice Options

​Tips

OpenAI-Compatible Function Calling

Define Tools

Model Response with Tool Call

Return Tool Results

Full Python Example

Anthropic-Compatible Tool Use

Response with Tool Use

Return Tool Results

Tool Choice Options

Tips