How The Router Works

The Model Router is a revolutionary AI tool developed by Martian that optimizes the use of various language learning models (LLMs) based on their performance and cost. It operates by routing each individual query to the most suitable LLM in real-time, ensuring optimal performance at a lower cost than any individual provider.

The Model Router's effectiveness is rooted in a deep understanding of how AI models work, specifically what makes them succeed or fail. This understanding is achieved through a process called model mapping, which involves converting models into a new format while preserving the properties we care about in the model.

In the case of the Model Router, this mapping process allows us to predict how well a given model will perform on a request without having to run that model. As a result, we can route the request to the model that will produce the best result, leading to higher performance and lower costs.

By using a sophisticated method for learning how models operate, we can achieve significant performance gains and cost reductions when using the Model Router

In testing, the Model Router has proven capable of outperforming GPT-4 on OpenAI’s own evaluations while reducing costs by an average of 20%. This tool is a tangible demonstration of the capabilities enabled by model mapping and represents a significant advancement in AI infrastructure.

You can read more about model mapping and martian's mission at https://blog.withmartian.com or replicate our results on openai/evals here.