Feedback

Submit structured feedback on inferences.

The feedback endpoint accepts a JSON array of feedback events, each with flexible value types (booleans, numbers, strings, or structured scores) and support for nested keys that are automatically flattened with dot notation.

Submit Feedback

The /v1/feedback endpoint accepts an array of feedback events. Each event is tied to an inference by its id. Submit one or many events in a single request.

Endpoint

POST https://api.withmartian.com/v1/feedback

Request Body

The request body is a JSON array of feedback event objects. Each object has the following fields:

Required Fields

Name
id
Type
string (UUID)
Description
The inference ID to attach feedback to. This is the id field returned by any inference endpoint. See Obtaining IDs.
Name
feedback
Type
object
Description
A dictionary mapping user-defined keys to feedback values. Keys are arbitrary strings you define. Values can be booleans, numbers, strings, structured objects, or nested dictionaries. An empty object {} is valid and returns an empty feedback_ids map.

Optional Fields

Name
tags
Type
object
Description
Metadata key-value pairs to attach to all feedback entries in this event. Both keys and values must be strings.
Name
optimize
Type
string
Description
Optimization direction for numeric and boolean metrics in this event. "max" (default) means higher is better, "min" means lower is better.

Response

Returns a JSON array of result objects, one per input event, in the same order. Each result contains either feedback_ids (on success) or error (on failure). The endpoint always returns HTTP 200; inspect each item to determine success or failure.

Name
feedback_ids
Type
object | null
Description
Present on success. Maps each feedback key to its unique ID (UUID). null on failure.
Name
error
Type
object | null
Description
Present on failure. Contains status_code (integer) and message (string). null on success.

Success example

[
  {
    "feedback_ids": {
      "helpful": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "quality.accuracy": "019503a1-b2c3-7d4e-8f01-234567890def"
    },
    "error": null
  }
]

Partial failure example

If one event in the batch fails, it is reported per-item. Successful events are not affected.

[
  {
    "feedback_ids": { "helpful": "019503a1-b2c3-7d4e-8f01-234567890abc" },
    "error": null
  },
  {
    "feedback_ids": null,
    "error": { "status_code": 400, "message": "ID does not exist" }
  }
]

For nested feedback, the response keys use the flattened dot-notation form.

Example Request

curl https://api.withmartian.com/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '[
    {
      "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "feedback": {
        "helpful": true,
        "rating": 4.5
      },
      "optimize": "max"
    }
  ]'

Submitting Multiple Events

You can submit feedback for multiple inferences in a single request. Each event can target a different ID and have its own optimize direction and tags.

curl https://api.withmartian.com/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '[
    {
      "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "feedback": { "helpful": true }
    },
    {
      "id": "019503a2-d4e5-7f60-a1b2-c3d4e5f60718",
      "feedback": { "quality": 4.5 },
      "optimize": "max"
    }
  ]'

Failures are reported per-item. If one event fails (e.g., an invalid id), other events in the batch are unaffected. Check each item's error field to determine which events succeeded and which need to be retried.

Obtaining IDs

The id field is returned as a top-level field in all inference responses. For streaming, the ID is included in the first event.

{
  "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
  ...
}

Feedback Values

Each value in the feedback object can be one of the following types:

Name
boolean
Type
bool
Description
Thumbs up/down style feedback. Mapped to a boolean metric.
Example: "helpful": true
Name
number
Type
int | float
Description
Numeric score or rating. Integers are converted to floats. Mapped to a float metric.
Example: "rating": 4.5
Name
string
Type
string
Description
Free-text comment or note.
Example: "note": "Great response, very detailed"
Name
structured
Type
object
Description
A score with an optional reason. The metric type is determined by the type of score (boolean, number, or string). The reason is stored as metadata.
Example: "accuracy": {"score": 0.95, "reason": "Correct but missed one edge case"}
- Name
  score
  Type
  bool | int | float | string
  Description
  The feedback value. Determines the metric type.
- Name
  reason
  Type
  string
  Description
  Optional explanation for the score, stored as tag metadata on the feedback entry.
Name
nested object
Type
object
Description
A dictionary of feedback values. Keys at each level are joined with dots. See Nested Feedback.
Example: "quality": {"accuracy": 0.95, "clarity": 4.5}

Unsupported value types (such as arrays or null) return a 400 Bad Request error.

Example with Mixed Types

curl https://api.withmartian.com/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '[
    {
      "id": "019503a2-d4e5-7f60-a1b2-c3d4e5f60718",
      "feedback": {
        "thumbs_up": true,
        "relevance": 0.92,
        "comment": "Answered the question clearly",
        "accuracy": {
          "score": 4,
          "reason": "Correct but could include more detail"
        }
      },
      "tags": {
        "evaluator": "human",
        "task": "question-answering"
      }
    }
  ]'

Nested Feedback

Feedback values can be nested to arbitrary depth. Nested keys are automatically flattened with dot notation before submission.

[
  {
    "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
    "feedback": {
      "evaluation": {
        "quality": {
          "precision": 0.95,
          "recall": 0.87
        }
      },
      "helpful": true
    }
  }
]

This produces feedback entries with keys evaluation.quality.precision, evaluation.quality.recall, and helpful, each with its own feedback ID in the response:

[
  {
    "feedback_ids": {
      "evaluation.quality.precision": "019503a3-1111-7000-8000-000000000001",
      "evaluation.quality.recall": "019503a3-2222-7000-8000-000000000002",
      "helpful": "019503a3-3333-7000-8000-000000000003"
    }
  }
]

The optimize parameter applies to all numeric and boolean feedback values within a single event. If you need different optimization directions for different metrics, submit them as separate events in the array with different optimize values.