Proxy Agent

Most of our agents allow you to specify a set of parameters like temperature, model, max_tokens, vendor, etc. However, there are some cases where you might need to define those parameters dynamically.

For example, you might want to use a different temperature or model based on some external factors. This is where the Proxy Agent comes in.

The Proxy Agent allows you to define a set of parameters dynamically for each request using the request body.

Use cases

• Chat application where you want to allow users to choose their preferred model.
• A/B testing different parameters to see which one works best.

Should I use this agent?

✅ You need to switch between different parameters for each request.

⛔ You prefer to define the parameters in the agent configuration and make sure they are consistent across all requests.

Creating a Proxy Agent

To create a Proxy Agent, follow this guide.

Request Body

The Proxy Agent expects the following parameters in the request body:

{
    "model": "<NAME_OF_THE_MODEL>", // Required
    "vendor": "<NAME_OF_THE_VENDOR>", // Required
    "messages": [
        {
        "role": "system",
        "content": "<SYSTEM_DEFINITION_GOES_HERE>"
        },
        {
        "role": "user",
        "content": "<USER_INPUT_GOES_HERE>"
        }
    ], // Required
    "frequency_penalty": 0.0, // Optional. Between -2.0 and 2.0. 
    "max_tokens": 4096, // Optional.
    "presence_penalty": 0.0, // Optional. Between -2.0 and 2.0. 
    "top_p": 1.0, // Optional. Between 0.0 and 1.0.
    "temperature": 0.5, // Optional. Between 0.0 and 1.0.
    "stream": true, // Optional
    "top_k": 10, // Optional. Between 0 and 100. Default value depends on the model.
    "userIdentifier": "<USER_IDENTIFIER>", // Optional. Only available in certain agents
    "groupIdentifier": "<GROUP_IDENTIFIER>" // Optional. Only available in certain agents
}

Usage Examples

cURL

curl -X POST https://api.serenitystar.ai/api/agent/{AGENT_ENDPOINT}/execute \
  -H "Content-Type: application/json" \
  -H "X-API-KEY: {API_KEY}" \
  -d '{
    "model": "mistral-medium-latest",
    "vendor": "Mistral",
    "messages": [
      {
        "role": "system",
        "content": "You are a chatbot specializing in cooking recipes."
      },
      {
        "role": "user",
        "content": "Can you give me a recipe for a chocolate cake?"
      }
    ],
    "temperature": 0.5,
    "max_tokens": 4096,
    "stream": true,
    "userIdentifier": "xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx",
    "groupIdentifier": "xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx"
  }'

C#

using System;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

class Program
{
    static async Task Main()
    {
        var client = new HttpClient();
        var request = new HttpRequestMessage
        {
            Method = HttpMethod.Post,
            RequestUri = new Uri("https://api.serenitystar.ai/api/agent/{AGENT_ENDPOINT}/execute"),
            Headers =
            {
                { "X-API-KEY", "{API_KEY}" },
                { "Content-Type", "application/json" },
            },
            Content = new StringContent(@"{
                ""model"": ""mistral-medium-latest"",
                ""vendor"": ""Mistral"",
                ""messages"": [
                    {
                        ""role"": ""system"",
                        ""content"": ""You are a chatbot specializing in cooking recipes.""
                    },
                    {
                        ""role"": ""user"",
                        ""content"": ""Can you give me a recipe for a chocolate cake?""
                    }
                ],
                ""temperature"": 0.5,
                ""max_tokens"": 4096,
                ""stream"": true,
                ""userIdentifier"": ""xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx"",
                ""groupIdentifier"": ""xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx""
            }", Encoding.UTF8, "application/json"),
        };

        var response = await client.SendAsync(request);
        var responseContent = await response.Content.ReadAsStringAsync();
        Console.WriteLine(responseContent);
    }
}

Python

import requests
import json

url = 'https://api.serenitystar.ai/api/agent/{AGENT_ENDPOINT}/execute'

headers = {
    'Content-Type': 'application/json',
    'X-API-KEY': '{API_KEY}'
}

payload = json.dumps({
    "model": "mistral-medium-latest",
    "vendor": "Mistral",
    "messages": [
        {
            "role": "system",
            "content": "You are a chatbot specializing in cooking recipes."
        },
        {
            "role": "user",
            "content": "Can you give me a recipe for a chocolate cake?"
        }
    ],
    "temperature": 0.5,
    "max_tokens": 4096,
    "stream": True,
    "userIdentifier": "xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx",
    "groupIdentifier": "xxxxxxxx-xxxxxx-xxxxxx-xxxxxxxx"
})

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)