OpenAI Python

The Martin Python library is a drop in r

V1

Using OpenAI's SDK, the following function

from openai import OpenAI

client = OpenAI(
    api_key="My OPENAI Key",  # defaults to os.environ.get("OPENAI_API_KEY")
)

chat_completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
)

becomes

from openai import OpenAI

client = OpenAI(
    api_key="My MARTIAN Key",  # defaults to os.environ.get("OPENAI_API_KEY")
    base_url="https://withmartian.com/api/openai/v1",
)

chat_completion = client.chat.completions.create(
    model="router",
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
)

Setting Parameters

To route across a subset of models, specifiy the array of whitelisted origins:

chat_completion = client.chat.completions.create(
    model="router",
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    extra_body={
        "models": ["gpt-3.5-turbo", "claude-2.1"]
    }
)

To specify a max cost and willingness to pay for a 10% improvement on model quality on each request, set the parameters in the following way:

client.chat.completions.create(
    model="router",
    messages=[
        {"role": "user", "content": "Hello world!"}
    ],
    extra_body={
        "models": ["gpt-3.5-turbo", "claude-2.1"],
        "max_cost": 0.02, 
        "willingness_to_pay": 0.01 
    },
)

Finally, to associate metadata (for instance about a user or region) with each request, set the extra body parameter like so:

client.chat.completions.create(
    model="router",
    messages=[
        {"role": "user", "content": "Write small poem"},
    ],
    extra_body={
        "models": ["gpt-3.5-turbo", "claude-2.1"],
        "max_cost": 0.02,
        "willingness_to_pay": 0.01,
        "extra": {
            "ip": "123.123.123.123",
            "Timezone": "UTC+0",
            "Country": "US",
            "City": "New York",
        }
    },
)

V0

Installation

pip install martian-python==0.0.2

Using legacy OpenAI's SDK, the following function

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

response = openai.ChatCompletion.create(
  messages=[
    {
      "role": "user",
      "content": "Hello world!"
    }
  ],
  model="gpt-3.5-turbo",
)

becomes

import os
from martian import openai

openai.martian_api_key = os.getenv("MARTIAN_API_KEY")

response = openai.ChatCompletion.create(
  messages=[
    {
      "role": "user",
      "content": "Hello world!"
    }
  ],
  model="router",
)