Skip to content

API Reference

Client

prodloop.client.ProdloopClient

ProdloopClient.evaluate_call(...)

Uploads a call recording for post-call evaluation.

Important arguments:

  • audio_file_path: local audio file to evaluate.
  • parameters: one or more EvaluationParameter values.
  • thresholds: optional thresholds for deterministic timing metrics.
  • extraction_schema: required when requesting extraction_variables.
  • bot_captured_variables: required when requesting extraction_variables.
  • input_prompt: required for hallucination and prompt-aware checks.

Prompt-aware checks compare the call against input_prompt and return a compact object:

{
  "passed": "true",
  "explanation": "..."
}

passed can be "true", "false", or "N/A". "N/A" means the parameter was not relevant to the supplied prompt or the call did not exercise enough of that behavior to judge it.

Models

prodloop.models.EvaluationParameter

Bases: str, Enum

prodloop.models.coerce_parameter(value)

Exceptions

prodloop.exceptions.ProdloopError

Bases: Exception

Base exception for SDK errors.

prodloop.exceptions.ValidationError

Bases: ProdloopError

Raised when request inputs are invalid.

prodloop.exceptions.APIError

Bases: ProdloopError

Raised when backend API returns an error response.

Prompt Simulation API

ProdloopClient.simulate_prompt(...)

Starts a prompt simulation.

Important arguments:

  • simulation_mode: SimulationMode.SELF_SIMULATION or SimulationMode.USER_ORCHESTRATED.
  • prompt: bot prompt under test.
  • parameters: exactly one EvaluationParameter per request.
  • bot_llm: required for self_simulation; use plugins.LiteLLM(...).
  • bot_turn_handler: required for user_orchestrated; use plugins.LiteLLMBot(...) or any callable that returns bot text.
  • max_turns: integer from 1 to 10; in adaptive simulations this is the turns-per-conversation limit.
  • adaptive_max_conversations: optional integer from 1 to 50; controls the maximum number of adaptive conversations.
  • scenario: optional tester scenario guidance.

ProdloopClient.get_simulation(chat_id)

Polls a simulation created by self_simulation and returns the current status, turns, transcript, and final result when complete. Adaptive final results include stop_reason and stop_message, including budget_exhausted when the configured adaptive budget is used before full coverage.

plugins.LiteLLM

Generic model connector for self_simulation.

plugins.LiteLLM(
    model="vertex_ai/gemini-2.5-pro",
    temperature=0.2,
    max_tokens=512,
    timeout_seconds=60,
)

plugins.LiteLLMBot

Callable local bot handler for user_orchestrated.

bot = plugins.LiteLLMBot(
    model="azure/<deployment-name>",
    system_prompt="You are a support bot.",
    options={"temperature": 0.2},
)

ProdloopClient.get_simulation_parameters()

Returns currently enabled simulation parameter keys from the backend runtime config.

params = client.get_simulation_parameters()
# {"version": "2026-05-07", "parameters": [{"key": "hallucination"}]}

The endpoint returns public parameter keys only. Internal execution type, such as deterministic metric vs LLM-judged, is intentionally not exposed to SDK users.

For prompt simulation, turn_by_turn_latency is emitted in every final result automatically, even if the selected parameter is different. It is the only timing parameter exposed for prompt simulation.

ProdloopClient.get_credit_balance()

Returns API-key credit/account status without creating a billable evaluation.

balance = client.get_credit_balance()
print(balance["credits_remaining"])

ProdloopClient.get_credits_remaining()

Alias for get_credit_balance().