Prompt Prefilling
Use Prompt Prefilling to control the initial output of the completion.
Prompt prefilling is a powerful feature that allows you to control the initial output of your models. This can be particularly useful for maintaining context, structuring outputs, or continuing previous dialogues.
How It Works
To use prompt prefilling, include an assistant message at the end of your input with the following characteristics:
- Set the
role
to “assistant” - Set the
name
to “prefill” - Include your desired prefill content in the
content
field
Our completions will start their response taking the prefilled content into account, effectively “continuing” from where you left off.
Example Usage
Basic Prefilling
const input = {
messages: [
{ role: "user", content: "Write a story about a brave knight." },
{
role: "assistant",
name: "prefill",
content: "Once upon a time, in a kingdom far away, there lived a brave knight named",
},
],
};
// Model output:
// " Lancelot. He rode through the countryside seeking adventures, and wherever he went he..."
The response will continue the story from there.
Structured Output
Prefilling can be used to enforce specific output structures:
const input = {
messages: [
{ role: "user", content: "List three benefits of exercise." },
{
role: "assistant",
name: "prefill",
content: "Here are three key benefits of regular exercise:\n\n1.",
},
],
};
// Model output:
// "Improved cardiovascular health\n\n2. Increased muscle strength and endurance\n\n3. Improved mental health and mood"
This ensures the response starts with the desired format.
Maintaining Character in Roleplays
For roleplay scenarios, prefilling can help maintain character consistency:
const input = {
messages: [
{ role: "system", content: "You are a pirate captain from the 18th century." },
{ role: "user", content: "What's our next destination?" },
{
role: "assistant",
name: "prefill",
content: "Arr, me hearty! Our next destination be",
},
],
};
// Model output:
// " the Caribbean Sea, me hearty! Let's set sail!"
Notes
- Prefilling only works when interacting with our OpenPipe finetuned models.
- You can use this feature while finetuning as well, maintaining the same characteristics for the assistant message.
By leveraging prompt prefilling, you can create more controlled, consistent, and context-aware interactions in your applications.