Build and evaluate LLM function calling

An ML troubleshooting agent often needs facts the model cannot know: the Python runtime inside a failing environment, the result of a calculation on a fixture, or the status of a submitted operation. Function calling gives the application a structured request for that information.

With Polyaxon, the trusted application can resolve the request to a specific run or sandbox, execute a permitted action, and return an explicit result to the model. The same action can be evaluated in a repeatable job, with configuration, measurements, and case artifacts retained alongside the rest of the ML platform's work.

Define the action before its schema

Start with a narrow task. An agent investigating a dependency failure may need to inspect Python's version and implementation in its assigned sandbox. It does not need an unrestricted shell, a model-selected project, or a tool that accepts arbitrary source code.

Write down the tool's inputs, target, output format, timeout, and error behavior. Resolve owner, project, and run UUID from trusted session state after checking the user's access. Authenticate the host client through the normal Polyaxon configuration; do not put credentials or target selection into the model-visible schema.

The sandbox LLM guide demonstrates provider-specific function-calling loops. The provider's request format may differ, but the validation and execution function below can remain the application's common tool backend.

Implement a bounded Polyaxon tool

This host-side example requires the Polyaxon client and a running sandbox-enabled service containing Python. Set SANDBOX_OWNER, SANDBOX_PROJECT, and SANDBOX_RUN_UUID to the authorized target. It exposes a tool with no arguments and always executes the same inspection command.

import json
import os

from polyaxon.client import SandboxClient


TOOL_SPEC = {
    "name": "inspect_python_runtime",
    "description": "Read the Python runtime in this task's assigned sandbox.",
    "parameters": {
        "type": "object",
        "properties": {},
        "additionalProperties": False,
    },
}

INSPECT_SOURCE = (
    "import json, sys; "
    "print(json.dumps({"
    "'version': sys.version.split()[0], "
    "'implementation': sys.implementation.name"
    "}))"
)


def call_tool(name, arguments):
    if name != TOOL_SPEC["name"]:
        return {"ok": False, "error": "unknown_tool"}
    if not isinstance(arguments, dict) or arguments:
        return {"ok": False, "error": "invalid_arguments"}

    with SandboxClient(
        owner=os.environ["SANDBOX_OWNER"],
        project=os.environ["SANDBOX_PROJECT"],
        run_uuid=os.environ["SANDBOX_RUN_UUID"],
    ) as sandbox:
        sandbox.ping()
        result = sandbox.process.exec(
            command=["python", "-I", "-c", INSPECT_SOURCE],
            timeout_ms=5_000,
        )

    if result.exit_code != 0 or result.timed_out:
        return {
            "ok": False,
            "error": "execution_failed",
            "exit_code": result.exit_code,
            "timed_out": result.timed_out,
        }
    if result.stdout_truncated:
        return {"ok": False, "error": "incomplete_result"}

    try:
        payload = json.loads(result.stdout)
    except json.JSONDecodeError:
        return {"ok": False, "error": "invalid_result"}
    if (
        not isinstance(payload, dict)
        or set(payload) != {"version", "implementation"}
        or not all(isinstance(value, str) for value in payload.values())
    ):
        return {"ok": False, "error": "invalid_result"}

    return {"ok": True, "runtime": payload, "duration_ms": result.duration_ms}

The host decodes the provider's proposed arguments before passing them to call_tool(). It returns the resulting JSON using that provider's tool-result mechanism and lets the model continue. Catch authentication, transport, and readiness exceptions in the host loop, record them internally, and return a controlled error rather than exposing credentials or unlimited exception text.

The tool reads runtime facts, not package compatibility or application correctness. Its description should preserve that distinction so the model does not treat a successful inspection as proof that a training environment is valid.

Extend the interface deliberately

A second tool might calculate a statistic from an approved fixture. Keep the calculation code fixed and validate bounded numerical arguments. For a tool that submits an evaluation, accept only the supported component inputs and derive the project, queue, connections, and component version from application policy.

Keep write actions separate from inspection. Submitting a job, canceling an operation, or publishing an artifact needs explicit authorization and a receipt that identifies the resulting state. If submission times out, reconcile whether the operation was created before retrying; a model-requested retry must not duplicate costly work.

The process API returns execution results inside an existing service. A tool call does not automatically become an independent Polyaxon run. Create a separate operation when it needs its own scheduling, environment, tracked lifecycle, or qualification record.

Evaluate selection and execution independently

Build a case manifest that includes requests requiring runtime inspection, questions answerable without tools, invalid arguments, unknown tools, unavailable sandboxes, truncated output, and misleading instructions embedded in tool results. Define both the expected tool choice and the expected final answer.

Run the manifest through a versioned evaluation job component. Record model and prompt revisions, tool-schema revision, fixture image, and manifest digest as tracked inputs. Log selection accuracy, argument validity, task success, retries, and latency as separate metrics.

For every case, retain the proposed call, validation decision, sanitized receipt, and final result under the job's outputs path using artifact tracking. A syntactically valid call may still choose the wrong tool, while a correct call can encounter an infrastructure failure. Those cases need different fixes.

Compare changes in the platform

In Polyaxon run comparison, compare the baseline and revised tool descriptions on the same cases and target environment. Inspect false tool calls and false refusals as well as successful-task latency. Use case artifacts to determine whether a wording change improved routing or merely changed how errors were presented.

Move the stable harness into a reusable component so the same qualification runs when prompts, tool schemas, model settings, or execution images change. Keep model usage and application-recorded cost with those results.

Function calling then becomes an operated part of the Polyaxon application: the model proposes an action, the host validates its scope, the platform executes the permitted work, and evaluation evidence determines whether the interface is ready for use.