Skip to main content

Chat Endpoint

Overview​

Create chat endpoint allows you to talk about anything you want, with no restrictions, no rules and censorship, by passing the model_id of the preferred model you want to use.

Request​

--request POST 'https://modelslab.com/api/v1/enterprise/llm/chat' \

Make a POST request to https://modelslab.com/api/v1/enterprise/llm/chat endpoint and pass the required parameters in the request body.

Body Attributes​

ParameterDescription
keyYour API Key used for request authorization
model_idID of the model you want to use. Here are LLM Models
chat_idID of the chat you want to associate in your conversation. Your prompt and the chat response will be saved based on this
system_promptWhat kind of persona the model should follow. This is an optional parameter and, by default, is set to a helpful prompt. Note that, once context length gets too long, the model will forget about it's persona and might need to be reminded.
promptWhat you would like to discuss about in the multi-turn conversation.
max_new_tokensThe maximum numbers of tokens to generate, ignoring the number of tokens in the prompt. Max: 1024. Defaults to 128.
do_sampleWhether or not to use sampling. If not, uses greedy decoding. Should be true or false. Defaults to true.
temperatureThe value used to modulate the next token probabilities. Should be between 0.1 and 0.3. Defaults to 0.3
top_kThe number of highest probability vocabulary tokens to keep for top-k-filtering. Max: 128. Defaults to 50.
top_pThe smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation. Default is 0.95
no_repeat_ngram_sizeIf set to int > 0, all ngrams of that size can only occur once. Defaults to 5.
seedSeed is used to reproduce results, same seed will give you same response. Pass null for a random number.
tempWhether or not to send proxy links. Should be true or false. Defaults to false.
resetIf you want to reset any given conversation with a new system prompt and start afresh by forgetting all previous context, you can specify true.
uncensored_system_promptShould be one of true or false based on if you want to initialize your conversation with an uncensored system prompt or not. If provided along with your own system_prompt, the system_prompt is taken by preference and this parameter is ignored.
webhookSet an URL to get a POST API call once the image generation is complete.
track_idThis ID is returned in the response to the webhook API call. This will be used to identify the webhook request.

Start Conversation​

When starting a chat conversation, it is important to pass the system_prompt so that the system knows what kind of persona the model should follow. chat_id is generated on the first request, so you do not have to pass it when starting a chat. Below is a sample on how to start a chat

Body
{
  "key": "",
  "model_id" : "zephyr-7b-beta",
  "system_prompt": "Gold Mining around the world",
  "prompt" : "Write a step by step guide on how to mine a gold",
  "max_new_tokens": 64,
  "do_sample": true,
  "temperature": 1,
  "top_k": 50,
  "top_p": 10,
  "no_repeat_ngram_size": 5,
  "seed": 1235,
  "temp": false,
  "webhook": null,
  "track_id" : null
}

Once the endpoint is called, the sample response looks like so;

Body Raw
{
    "status": "success",
    "output": [
        "https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
    ],
    "message": "Please note that gold mining is a complex and dangerous process that requires specialized equipment, expertise, and regulatory permits. This guide provides a general overview of the steps involved in small-scale gold mining and shouldnot be interpreted as a substitute for professional guidance and safety precautions.\n\nStep 1: Research the",
    "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
    "meta": {
        "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
        "created_at": "2023-11-27T02:48:47.911559",
        "do_sample": "yes",
        "max_new_tokens": 64,
        "model_id": "zephyr-7b-beta",
        "no_repeat_ngram_size": 5,
        "num_return_sequences": 1,
        "pipeline_tag": "text-generation",
        "prompt": "Write a step by step guide on how to mine a gold",
        "seed": 1235,
        "temp": "no",
        "temperature": 1,
        "top_k": 50,
        "top_p": 1,
        "updated_at": "2023-11-27T02:48:52.942389"
    }
}

Continue Chat Conversation​

From the response above, chat_id is being returned. Continue using the chat by passing the chat_id and other parameter to the endpoint. Here the system_prompt is not compulsory. Below is how the sample json body looks like;

Body
{
 "key": "",
  "chat_id":"49afc793-803b-4b4a-bac9-d2f9318c2b8f",
  "model_id" : "zephyr-7b-beta",
  "prompt" : "Tell me the dangers involved in this process",
  "max_new_tokens": 64,
  "do_sample": true,
  "temperature": 1,
  "top_k": 50,
  "top_p": 10,
  "no_repeat_ngram_size": 5,
  "seed": 1235,
  "temp": false,
  "webhook": null,
  "track_id" : null
}

The response of the above request looks like so,

Body
{
    "status": "success",
    "output": [
        "https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
    ],
    "message": "Certainly, here are some of the dangers involved in gold mining:\n\n1. Collapsed Mines: Gold mining can take place in underground tunnels, which can sometimes collapse, trapping and injuring miners.\n\n2. Poisonous Gases: Mines can sometimes contain poison",
    "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
    "meta": {
        "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
        "created_at": "2023-11-27T02:48:47.911559",
        "do_sample": "yes",
        "max_new_tokens": 64,
        "model_id": "zephyr-7b-beta",
        "no_repeat_ngram_size": 5,
        "num_return_sequences": 1,
        "pipeline_tag": "text-generation",
        "prompt": "Tell me the dangers involved in this process",
        "seed": 1235,
        "temp": "no",
        "temperature": 1,
        "top_k": 50,
        "top_p": 1,
        "updated_at": "2023-11-27T02:51:44.603069"
    }
}

Request​

var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/json");

var raw = JSON.stringify({
  "key": "",
  "model_id" : "zephyr-7b-beta",
  "system_prompt": "Gold Mining around the world",
  "prompt" : "Write a step by step guide on how to mine a gold",
  "max_new_tokens": 64,
  "do_sample": true,
  "temperature": 1,
  "top_k": 50,
  "top_p": 10,
  "no_repeat_ngram_size": 5,
  "seed": 1235,
  "temp": false,
  "webhook": null,
  "track_id" : null
});

var requestOptions = {
  method: 'POST',
  headers: myHeaders,
  body: raw,
  redirect: 'follow'
};

fetch("https://modelslab.com/api/v1/enterprise/llm/chat", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log('error', error));

Response​

Example Response
{
    "status": "success",
    "output": [
        "https://pub-3626123a908346a7a8be8d9295f44e26.r2.dev/chats/49afc793-803b-4b4a-bac9-d2f9318c2b8f.json"
    ],
    "message": "Please note that gold mining is a complex and dangerous process that requires specialized equipment, expertise, and regulatory permits. This guide provides a general overview of the steps involved in small-scale gold mining and shouldnot be interpreted as a substitute for professional guidance and safety precautions.\n\nStep 1: Research the",
    "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
    "meta": {
        "chat_id": "49afc793-803b-4b4a-bac9-d2f9318c2b8f",
        "created_at": "2023-11-27T02:48:47.911559",
        "do_sample": "yes",
        "max_new_tokens": 64,
        "model_id": "zephyr-7b-beta",
        "no_repeat_ngram_size": 5,
        "num_return_sequences": 1,
        "pipeline_tag": "text-generation",
        "prompt": "Write a step by step guide on how to mine a gold",
        "seed": 1235,
        "temp": "no",
        "temperature": 1,
        "top_k": 50,
        "top_p": 1,
        "updated_at": "2023-11-27T02:48:52.942389"
    }
}