Feedback

Submit structured feedback on individual inferences or full conversation episodes.

The feedback endpoint accepts flexible value types (booleans, numbers, strings, or structured scores) and supports nested keys that are automatically flattened with dot notation.

Submit Feedback

The /v1/feedback endpoint accepts feedback tied to either an episode (conversation-level) or a single inference (request-level).

Endpoint

POST https://api.withmartian.com/v1/feedback

Required Parameters

  • Name
    feedback
    Type
    object
    Description

    A dictionary mapping user-defined keys to feedback values. Keys are arbitrary strings you define. Values can be booleans, numbers, strings, structured objects, or nested dictionaries. An empty object {} is valid and returns an empty feedback_ids map.

  • Name
    episode_id
    Type
    string (UUID)
    Description

    Attach feedback to an episode (conversation-level). See Obtaining IDs.

  • Name
    inference_id
    Type
    string (UUID)
    Description

    Attach feedback to a single inference (request-level). See Obtaining IDs.

Provide either episode_id or inference_id, not both. Omitting both or providing both returns a 400 error.

Optional Parameters

  • Name
    tags
    Type
    object
    Description

    Metadata key-value pairs to attach to all feedback entries. Both keys and values must be strings.

  • Name
    optimize
    Type
    string
    Description

    Optimization direction for numeric and boolean metrics. "max" (default) means higher is better, "min" means lower is better.

Response

Returns a JSON object with a feedback_ids dictionary mapping each feedback key to its unique ID (UUID).

{
  "feedback_ids": {
    "helpful": "019503a1-b2c3-7d4e-8f01-234567890abc",
    "quality.accuracy": "019503a1-b2c3-7d4e-8f01-234567890def"
  }
}

For nested feedback, the response keys use the flattened dot-notation form.

Example Request

curl https://api.withmartian.com/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "episode_id": "019503a1-b2c3-7d4e-8f01-234567890abc",
    "feedback": {
      "helpful": true,
      "rating": 4.5
    },
    "optimize": "max"
  }'

Obtaining IDs

The episode_id and inference_id are returned as top-level fields in inference responses. For streaming, the IDs are included in the first event.

{
  "id": "chatcmpl-...",
  "episode_id": "019503a1-...",
  "inference_id": "019503a1-...",
  "choices": [...],
  ...
}

Feedback Values

Each value in the feedback object can be one of the following types:

  • Name
    boolean
    Type
    bool
    Description

    Thumbs up/down style feedback. Mapped to a boolean metric.

    Example: "helpful": true

  • Name
    number
    Type
    int | float
    Description

    Numeric score or rating. Integers are converted to floats. Mapped to a float metric.

    Example: "rating": 4.5

  • Name
    string
    Type
    string
    Description

    Free-text comment or note.

    Example: "note": "Great response, very detailed"

  • Name
    structured
    Type
    object
    Description

    A score with an optional reason. The metric type is determined by the type of score (boolean, number, or string). The reason is stored as metadata.

    Example: "accuracy": {"score": 0.95, "reason": "Correct but missed one edge case"}

    • Name
      score
      Type
      bool | int | float | string
      Description

      The feedback value. Determines the metric type.

    • Name
      reason
      Type
      string
      Description

      Optional explanation for the score, stored as tag metadata on the feedback entry.

  • Name
    nested object
    Type
    object
    Description

    A dictionary of feedback values. Keys at each level are joined with dots. See Nested Feedback.

    Example: "quality": {"accuracy": 0.95, "clarity": 4.5}

Unsupported value types (such as arrays or null) return a 400 Bad Request error.

Example with Mixed Types

curl https://api.withmartian.com/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "inference_id": "019503a2-d4e5-7f60-a1b2-c3d4e5f60718",
    "feedback": {
      "thumbs_up": true,
      "relevance": 0.92,
      "comment": "Answered the question clearly",
      "accuracy": {
        "score": 4,
        "reason": "Correct but could include more detail"
      }
    },
    "tags": {
      "evaluator": "human",
      "task": "question-answering"
    }
  }'

Nested Feedback

Feedback values can be nested to arbitrary depth. Nested keys are automatically flattened with dot notation before submission.

{
  "episode_id": "019503a1-b2c3-7d4e-8f01-234567890abc",
  "feedback": {
    "evaluation": {
      "quality": {
        "precision": 0.95,
        "recall": 0.87
      }
    },
    "helpful": true
  }
}

This produces feedback entries with keys evaluation.quality.precision, evaluation.quality.recall, and helpful, each with its own feedback ID in the response:

{
  "feedback_ids": {
    "evaluation.quality.precision": "019503a3-1111-7000-8000-000000000001",
    "evaluation.quality.recall": "019503a3-2222-7000-8000-000000000002",
    "helpful": "019503a3-3333-7000-8000-000000000003"
  }
}

The optimize parameter applies to all numeric and boolean feedback values in the request. If you need different optimization directions for different metrics, submit them in separate requests.

Next Steps

API Endpoints

Learn about the chat completions and messages endpoints that return episode and inference IDs.

Read more

Error Handling

Understand error codes and implement robust error handling in your application.

Read more