Prompt Prefilling

Prompt prefilling is a powerful feature that allows you to control the initial output of your models. This can be particularly useful for maintaining context, structuring outputs, or continuing previous dialogues.

​

How It Works

To use prompt prefilling, include an assistant message at the end of your input with the following characteristics:

• Set the role to “assistant”
• Set the name to “prefill”
• Include your desired prefill content in the content field

Our completions will start their response taking the prefilled content into account, effectively “continuing” from where you left off.

​

Example Usage

​

Basic Prefilling

const input = {
  messages: [
    { role: "user", content: "Write a story about a brave knight." },
    {
      role: "assistant",
      name: "prefill",
      content: "Once upon a time, in a kingdom far away, there lived a brave knight named",
    },
  ],
};
// Model output:
// " Lancelot. He rode through the countryside seeking adventures, and wherever he went he..."

The response will continue the story from there.

​

Structured Output

Prefilling can be used to enforce specific output structures:

const input = {
  messages: [
    { role: "user", content: "List three benefits of exercise." },
    {
      role: "assistant",
      name: "prefill",
      content: "Here are three key benefits of regular exercise:\n\n1.",
    },
  ],
};
// Model output:
// "Improved cardiovascular health\n\n2. Increased muscle strength and endurance\n\n3. Improved mental health and mood"

This ensures the response starts with the desired format.

​

Maintaining Character in Roleplays

For roleplay scenarios, prefilling can help maintain character consistency:

const input = {
  messages: [
    { role: "system", content: "You are a pirate captain from the 18th century." },
    { role: "user", content: "What's our next destination?" },
    {
      role: "assistant",
      name: "prefill",
      content: "Arr, me hearty! Our next destination be",
    },
  ],
};
// Model output:
// " the Caribbean Sea, me hearty! Let's set sail!"

​

Notes

• Prefilling only works when interacting with our OpenPipe finetuned models.
• You can use this feature while finetuning as well, maintaining the same characteristics for the assistant message.

By leveraging prompt prefilling, you can create more controlled, consistent, and context-aware interactions in your applications.