You can track the cost and token usage for any large language model (LLM) and endpoint in Polyaxon. There are two main ways to do this:

1. Automatic calculation by Polyaxon: For supported models, Polyaxon can infer the cost. This requires you to use the correct model names. We currently support OpenAI and Anthropic models out of the box. Refer to the Models tab in Polyaxon for the exact model names.

2. Explicit cost and token ingestion: You can ingest the cost and token counts for your LLM calls that you already track with Polyaxon. Some model providers return the cost and tokens as part of the response payload. You can pass these back to Polyaxon. Token counts are automatically captured across our native integrations, such as OpenAI SDK, LangChain, LlamaIndex, and others.

For more information and examples, please see the metadata logging documentation.