Skip to main content

Chat Endpoint

Overview

Create chat endpoint allows you to talk about anything you want, with no restrictions, no rules and censorship, by passing the model_id of the preferred model you want to use.

Request

--request POST 'https://modelslab.com/api/v6/llm/chat' \

Make a POST request to https://modelslab.com/api/v6/llm/chat endpoint and pass the required parameters in the request body.

Body Attributes

ParameterDescriptionValues
keyYour unique API key used for authorization.string
model_idThe ID of the LLM model you want to use.string
chat_idThe ID of the chat you want to associate with your conversation.ID
system_promptOptional. The persona the model should follow. Defaults to a helpful prompt.string
promptThe topic or content you would like to discuss in the multi-turn conversation.string
max_new_tokensThe maximum number of tokens to generate, ignoring those in the prompt.integer (Max: 1024, Default: 128)
do_sampleWhether or not to use sampling. If not, uses greedy decoding.true or false
temperatureThe value used to modulate the next token probabilities. Should be between 0.1 and 0.3.float (Default: 0.3)
top_kThe number of highest probability vocabulary tokens to keep for top-k-filtering.integer (Max: 128, Default: 50)
top_pThe smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation.float (Default: 0.95)
no_repeat_ngram_sizeIf set to an integer greater than 0, all ngrams of that size can only occur once.integer (Default: 5)
seedThe seed used to reproduce results. Pass null for a random number.integral value
tempWhether or not to send proxy links.true or false (Default: false)
generator_typeThe expected data type of the output.json, choices, format, text
generator_choicesDepends on generator_type. If choices, provide categorical choices; if json, provide JSON schema; if format, provide data type.string
resetIf true, resets the conversation with a new system prompt, forgetting all previous context.true or false
uncensored_system_promptInitialize conversation with an uncensored system prompt.true or false
webhookProvide a URL to receive a POST API call once the operation is complete.url
track_idThis ID identifies the webhook request in the response.integral value

Start Conversation

When starting a chat conversation, it is important to pass the system_prompt so that the system knows what kind of persona the model should follow. chat_id is generated on the first request, so you do not have to pass it when starting a chat. Below is a sample on how to start a chat

Body
{
"key": "",
"model_id" : "zephyr-7b-beta",
"system_prompt": "Gold Mining around the world",
"prompt" : "Write a step by step guide on how to mine a gold",
"max_new_tokens": 64,
"do_sample": true,
"temperature": 1,
"top_k": 50,
"top_p": 10,
"no_repeat_ngram_size": 5,
"seed": 1235,
"temp": false,
"webhook": null,
"track_id" : null
}

Once the endpoint is called, the sample response looks like so;

Body Raw
{
"status": "success",
"output": [
"https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
],
"message": "Please note that gold mining is a complex and dangerous process that requires specialized equipment, expertise, and regulatory permits. This guide provides a general overview of the steps involved in small-scale gold mining and shouldnot be interpreted as a substitute for professional guidance and safety precautions.\n\nStep 1: Research the",
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"meta": {
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"created_at": "2023-11-27T02:48:47.911559",
"do_sample": "yes",
"max_new_tokens": 64,
"model_id": "zephyr-7b-beta",
"no_repeat_ngram_size": 5,
"num_return_sequences": 1,
"pipeline_tag": "text-generation",
"prompt": "Write a step by step guide on how to mine a gold",
"seed": 1235,
"temp": "no",
"temperature": 1,
"top_k": 50,
"top_p": 1,
"updated_at": "2023-11-27T02:48:52.942389"
}
}

Continue Chat Conversation

From the response above, chat_id is being returned. Continue using the chat by passing the chat_id and other parameter to the endpoint. Here the system_prompt is not compulsory. Below is how the sample json body looks like;

Body
{
"key": "",
"chat_id":"49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"model_id" : "zephyr-7b-beta",
"prompt" : "Tell me the dangers involved in this process",
"max_new_tokens": 64,
"do_sample": true,
"temperature": 1,
"top_k": 50,
"top_p": 10,
"no_repeat_ngram_size": 5,
"seed": 1235,
"temp": false,
"webhook": null,
"track_id" : null
}

The response of the above request looks like so,

Body
{
"status": "success",
"output": [
"https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
],
"message": "Certainly, here are some of the dangers involved in gold mining:\n\n1. Collapsed Mines: Gold mining can take place in underground tunnels, which can sometimes collapse, trapping and injuring miners.\n\n2. Poisonous Gases: Mines can sometimes contain poison",
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"meta": {
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"created_at": "2023-11-27T02:48:47.911559",
"do_sample": "yes",
"max_new_tokens": 64,
"model_id": "zephyr-7b-beta",
"no_repeat_ngram_size": 5,
"num_return_sequences": 1,
"pipeline_tag": "text-generation",
"prompt": "Tell me the dangers involved in this process",
"seed": 1235,
"temp": "no",
"temperature": 1,
"top_k": 50,
"top_p": 1,
"updated_at": "2023-11-27T02:51:44.603069"
}
}

Request

var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/json");

var raw = JSON.stringify({
"key": "",
"model_id" : "zephyr-7b-beta",
"system_prompt": "Gold Mining around the world",
"prompt" : "Write a step by step guide on how to mine a gold",
"max_new_tokens": 64,
"do_sample": true,
"temperature": 1,
"top_k": 50,
"top_p": 10,
"no_repeat_ngram_size": 5,
"seed": 1235,
"temp": false,
"webhook": null,
"track_id" : null
});

var requestOptions = {
method: 'POST',
headers: myHeaders,
body: raw,
redirect: 'follow'
};

fetch("https://modelslab.com/api/v6/llm/chat", requestOptions)
.then(response => response.text())
.then(result => console.log(result))
.catch(error => console.log('error', error));

Response

Example Response
{
"status": "success",
"output": [
"https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
],
"message": "Please note that gold mining is a complex and dangerous process that requires specialized equipment, expertise, and regulatory permits. This guide provides a general overview of the steps involved in small-scale gold mining and shouldnot be interpreted as a substitute for professional guidance and safety precautions.\n\nStep 1: Research the",
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"meta": {
"chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
"created_at": "2023-11-27T02:48:47.911559",
"do_sample": "yes",
"max_new_tokens": 64,
"model_id": "zephyr-7b-beta",
"no_repeat_ngram_size": 5,
"num_return_sequences": 1,
"pipeline_tag": "text-generation",
"prompt": "Write a step by step guide on how to mine a gold",
"seed": 1235,
"temp": "no",
"temperature": 1,
"top_k": 50,
"top_p": 1,
"updated_at": "2023-11-27T02:48:52.942389"
}
}