Request logs are a great way to get to know your data. More importantly, you can import recorded logs directly into your training datasets. That means it’s really easy to train on data you’ve collected in production.

We recommend collecting request logs for both base and fine-tuned models. We provide several options for recording your requests.

​
SDK

The simplest way to start ingesting request logs into OpenPipe is by installing our Python or TypeScript SDK. Requests to both OpenAI and OpenPipe models will automatically be recorded. Logging doesn’t add any latency to your requests, because our SDK calls the OpenAI server directly and returns your completion before kicking off the request to record it in your project.

We provide a drop-in replacement for the OpenAI SDK, so the only code you need to update is your import statement:

# from openai import OpenAI
from openpipe import OpenAI

# Nothing else changes

client = OpenAI()

completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "system", "content": "count to 10"}],
    # searchable tags are highly recommended
    openpipe={"tags": {"prompt_id": "counting", "any_key": "any_value"}},
)

See Installing the SDK for a quick guide on how to get started.

​
Proxy

If you’re developing in a language other than Python or TypeScript, the best way to ingest data into OpenPipe is through our proxy. We provide a /chat/completions endpoint that is fully compatible with OpenAI, so you can continue using the latest features like tool calls and streaming without a hitch.

Integrating the Proxy and logging requests requires a couple steps.

  1. Add an OpenAI key to your project in the project settings page.
  2. Set the authorization token of your request to be your OpenPipe API key.
  3. Set the destination url of your request to be https://app.openpipe.ai/api/v1/chat/completions.
  4. When making any request that you’d like to record, add the op-log-request http header. We also recommend that you add the op-tags header to distinguish data collected from different prompts.
{
    ...other headers,
    "op-log-request": "true",
    # Adding searchable tags is highly recommended
    "op-tags": '{"prompt_id": "first_prompt"}',
}

Here’s an example of steps 2-4 put together in both a raw cURL request and with the Python SDK:

curl --request POST \
  --url https://app.openpipe.ai/api/v1/chat/completions \
  --header 'Authorization: Bearer YOUR_OPENPIPE_API_KEY' \
  --header 'Content-Type: application/json' \
  --header 'op-log-request: true' \
  --header 'op-tags: {"prompt_id": "first_prompt"}' \
  --data '{
  "model": "gpt-4-0613",
  "messages": [
    {
      "role": "system",
      "content": "count to 5"
    }
  ],
  "max_tokens": 100,
  "temperature": 0
}'

​
Reporting

If you need more flexibility in how you log requests, you can use the report endpoint. This gives you full control over when and how to create request logs.

import time
from openai import OpenAI
from openpipe.client import OpenPipe

client = OpenAI()
op_client = OpenPipe()

payload = {
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Count to 10"}],
}

completion = client.chat.completions.create(**payload)

op_client.report(
    requested_at=int(time.time() * 1000),
    received_at=int(time.time() * 1000),
    req_payload=payload,
    resp_payload=completion,
    status_code=200,
    tags={
        "prompt_id": "My prompt id",
    },
)

If you’re developing in a language other than Python or TypeScript, you can also make a raw HTTP request to the report endpoint.

Once you’ve set up logging, you will see the data on the Request Logs page. From there, you’ll be able to search through your requests and train your models. See Training on Logs to learn more.

Installing the SDKLogging Anthropic Requests