Cloud Persona Custom Endpoint
Inside of proto cloud when creating a new persona you are now given the option to choose between our default LLM or connect to your own custom LLM API endpoint. We will be adding more custom configurations over time.
When you select Default, it will use AWS Bedrocks Haiku 4.5 with a temperature of 0.3 . More models options are coming soon.
When selecting Custom it will give you the option to add in a Base URL and optional Bearer token or API Key.
Why would you want to use a custom endpoint?
You are able to manage your data, system prompt, etc on your own infra.
You have the ability to add things like logging and analysis to the conversation.
You can implement RAG db, Tool calling, Web search or anything else you wish to customize.
Any LLM of your choosing.
Your API must follow the request and response format stated below.
Request:
For authorization you can leave it as
none
Bearer token (Request example “Authorization: Bearer XXXXXXXXXXXXXXXXX")
API Key (Request example "X-API-Key: XXXXXXXXXXXXXX")
The body will be an array of messages from the current conversation:
```json
{
"conversation": [
{
"role": "user",
"content": "What's your name?"
},
{
"role": "assistant",
"content": "I'm Nolan. Who are you?"
},
{
"role": "user",
"content": "I am your AI persona. Great to meet you."
}
]
}
```
Response:
Your API should simply return the text, not a JSON, of what the persona should say out loud.
Ideally your API returns the text as a stream to keep latency as low as possible.
Example:
I am your AI persona. Great to meet you.