Release Notes

To see changes in Martian's releases, see the following release notes.

Introducing Martian's LLM Router (Feb 19 2024)

We are pleased to announce the release of Martian's LLM Router Version 1.0. Martian specializes in routing for large language models, focusing on developing tools that improve the handling and distribution of requests across different services. This initial version introduces features that aim to streamline the process of integrating and managing large language models within various systems.

New Features

• Dynamic Routing Algorithm: Automatically routes requests to the most suitable LLM based on current load, response time, and accuracy metrics.

• Multi-Model Support: Seamlessly integrates with a wide range of LLMs found here.

• LLM Streaming Support: Enables real-time, continuous data streaming to and from Large Language Models, reducing latency and enhancing interactive applications like live chatbots and streaming analytics.

• API Gateway Integration: Easy integration with existing API gateways for streamlined request handling and response delivery from a wide range of LLMs found here.

• Model Router Playground: An interactive playground enabling users to interactively explore and test the routing capabilities of Martian LLM Router through a web-based interface.

Known Issues and Workarounds

• Dashboard Latency: Some users may experience minor latency in the analytics dashboard. We are actively working on scaling our infrastructure to address this issue.