Introduction

The Martian Model Router dynamically routes requests to the best LLM for your prompt in real-time.

This lets you:

  • Get higher performance and lower costs

  • Future-proof your application as new models are added

  • Make your application more resilient by routing away from models with degraded performance

  • Focus on building a better product instead of finding a better model

The Model Router is the first tool we've released on our mission to make better AI tooling by understanding how AI models work.

The Model Router is a revolutionary AI tool developed by Martian that optimizes the use of various language learning models (LLMs) based on their performance and cost. It operates by routing each individual query to the most suitable LLM in real-time, ensuring optimal performance at a lower cost than any individual provider.

The Model Router's effectiveness is rooted in a deep understanding of how AI models work, specifically what makes them succeed or fail. This understanding is achieved through a process called model mapping, which involves converting models into a new format while preserving the properties we care about in the model.

In the case of the Model Router, this mapping process allows us to predict how well a given model will perform on a request without having to run that model. As a result, we can route the request to the model that will produce the best result, leading to higher performance and lower costs.

By using a sophisticated method for learning how models operate, we can achieve significant performance gains and cost reductions when using the Model Router

In testing, the Model Router has proven capable of outperforming GPT-4 on OpenAI’s own evaluations while reducing costs by an average of 20%. This tool is a tangible demonstration of the capabilities enabled by model mapping and represents a significant advancement in AI infrastructure.

The simplest way to get started is to go through our Hello World! example.

Hello World

Once you complete that example to understand how the library works, you can use the router by following the steps on Integrating Martian Into Your Codebase.

Integrating Router

When properly integrated, the model router can outperform any individual models (including latest o1-models) on both cost and performance.

Last updated