Skip to content

Make the figures match your invoice

Out of the box, Seylo prices your cluster at public list prices. If you pay something else — reservations, savings plans, an Enterprise Agreement, a negotiated discount — the figures overstate your cost. The Make the figures match your invoice card on FinOps → Cost & savings → Cost brings in what you actually pay.

The card is shown only to people who see the whole cluster in Seylo. Billing and an organisation’s spend belong to no namespace, so the card, and the Spend outside the cluster card, are not shown to anyone limited to some namespaces.

Row What it gives you What it needs
Your Azure bill What each machine type actually cost you, with reservations and savings plans spread over their hours, and your whole bill day by day, read from the subscription itself The subscription connected as a cloud account. One Reader role for Seylo’s app, no key, no export
Connect spend outside the cluster: one script Datadog usage and OpenAI spend, read by OpenCost in your cluster A key for each tool, in a Secret in your cluster. Shown while either is not connected on this cluster
Azure price sheet, Azure cost export OpenCost’s own Azure integrations Shown only on a cluster that already runs one. See if a cluster already runs them
  1. Connect the Azure subscription on Admin → Integrations → Cloud accounts. See cloud accounts. The Reader role you grant there is all Seylo needs: it reads Azure Cost Management with it. There is nothing to install in the cluster, no Cost Management export and no key.
  2. Open FinOps → Cost & savings → Cost. The Your Azure bill row says On and working, and for each subscription how many machine prices it read from the bill and the bill’s currency.

Seylo reads every Azure cloud account in your organisation whose last Test said Connected. It reads again every six hours, and the row says when it last read. Connecting, testing or removing a cloud account makes the next view read again.

With no Azure subscription connected, the row says No Azure subscription is connected, so Azure figures use public list prices.

What it is Where it goes
Machine prices For each machine type and region your nodes ran on, what those hours cost over the 7 complete days (UTC) before yesterday. Amortised: a reservation or savings plan is spread over the hours it covers. In US dollars as Azure states them; Seylo converts nothing The price book, and so every node figure on the Cost and Savings tabs
The daily bill The last 190 days, per service, meter category and location, as billed, in the bill’s own currency The whole-cloud panels on cost dashboards

How the machine prices are worked out:

  • Your bill wins. For the same machine type and region, a price from your bill replaces both an uploaded rate and the public list price.
  • The public list stays the fallback for anything the bill does not show: a machine type that did not run in those 7 days, a subscription that is not connected, and clusters on other clouds.
  • Spot is priced apart. A machine type’s spot rate is used only where that type ran nothing but spot in the window.
  • Several subscriptions are combined, weighted by the hours each ran.
  • Never invented. A resource with no hours, no US-dollar figure, or a machine type Azure does not state, is left out and never priced from a similar machine. The row counts them, with the reason, for example Azure Resource Manager did not say its machine type (it may have been deleted since).

The Price book card then reads Prices: from your Azure bill, and lists each rate with How it was worked out.

Until the first read finishes, nodes are priced from the public list.

You tick the tools, Seylo gives you one bash script, and you run it once against the cluster. The script asks for each key itself, hidden, in your own terminal: a key is never typed into the page, never shown on screen, never saved to a file and never sent to Seylo.

  • The seylo-observability chart 0.2.11 or later, with OpenCost. The script’s last command upgrades the chart and turns OpenCost on, so an older chart is fine as a starting point.
  • Agent 0.2.11 or later, so Seylo can read OpenCost’s status. See chargeback for the upgrade.
  • Helm 3.14 or later for --reset-then-reuse-values. On older Helm, save your values with helm get values seylo-observability -n seylo-observability -o yaml > mine.yaml and pass -f mine.yaml instead.
  • kubectl pointing at the cluster. The script prints the current context and asks before it writes anything.
  1. Under Connect spend outside the cluster: one script, tick Datadog, OpenAI or both. Only tools not yet connected on this cluster are offered.
  2. For Datadog, choose Your Datadog site: the site in your Datadog address bar, for example app.datadoghq.eu is EU1. There is no default, because a wrong site fails only later, inside OpenCost. The script appears once a site is chosen.
  3. Create each key in its own console. The card lists the step:
    • Datadog: in your parent Datadog organisation, Organization Settings → Application Keys → New Key, with scopes usage_read and billing_read. Datadog serves estimated cost to parent organisations only. This is not the key the Datadog connector uses, which is scoped to reading metrics and lives in the agent’s namespace. The script also asks for a Datadog API key.
    • OpenAI: on platform.openai.com, Settings → Organization → Admin keys → Create new admin key. It must be an admin key: the plugin reads what OpenAI billed, which project keys cannot.
  4. Copy the script and run it once, in bash: Linux, macOS, WSL, Git Bash or Azure Cloud Shell. Paste each key when it asks.

With both tools ticked, the script has this shape. The page fills in your Datadog site and the chart version:

Terminal window
# Connect Datadog and OpenAI spend to OpenCost on the cluster kubectl points at. Run in bash.
# Each key is typed here, hidden, and goes straight into a Secret in seylo-observability: never printed, never saved, never sent to Seylo.
(
set -euo pipefail
echo "Cluster: $(kubectl config current-context)"
read -rp "Connect Datadog and OpenAI spend on this cluster? [y/N] " OK; [ "$OK" = "y" ] || exit 1
DD_SITE="<datadog-site>"
read -rsp "Datadog API key (hidden): " DD_API; echo
read -rsp "Datadog application key, usage_read + billing_read (hidden): " DD_APP; echo
kubectl -n seylo-observability create secret generic seylo-opencost-datadog \
--from-literal=datadog_config.json="{\"datadog_site\":\"$DD_SITE\",\"datadog_api_key\":\"$DD_API\",\"datadog_app_key\":\"$DD_APP\"}" \
--dry-run=client -o yaml | kubectl apply -f -
read -rsp "OpenAI admin key (hidden): " OPENAI_ADMIN_KEY; echo
kubectl -n seylo-observability create secret generic seylo-opencost-openai \
--from-literal=openai_config.json="{\"openai_api_key\":\"$OPENAI_ADMIN_KEY\"}" \
--dry-run=client -o yaml | kubectl apply -f -
helm upgrade seylo-observability oci://ghcr.io/seylo-inc/charts/seylo-observability --version 0.2.22 \
--namespace seylo-observability --reset-then-reuse-values \
--set opencost.enabled=true \
--set opencost.plugins.datadog.enabled=true \
--set opencost.plugins.datadog.existingSecret=seylo-opencost-datadog \
--set opencost.plugins.openai.enabled=true \
--set opencost.plugins.openai.existingSecret=seylo-opencost-openai
)

What it does, and why it is safe to paste:

  • It runs inside ( ... ), so an error, or any answer but y at the prompt, ends the script, not your terminal, and the keys go with it.
  • It names the cluster kubectl points at and asks [y/N] before writing.
  • It writes one Secret per tool in seylo-observability, then runs one helm upgrade that turns on every tool you ticked.
  • Running it again is safe: each Secret is applied, not created, so a new key replaces the old one.

The chart downloads OpenCost’s Datadog and OpenAI plugins and refuses to run one unless its sha256 matches the one pinned in the chart.

Seylo reads spend outside the cluster only through OpenCost’s own plugins, which exist for Datadog and OpenAI. The card lists the others, folded under Anthropic and other tools: not supported, and why:

Tool Why
Anthropic OpenCost has no Anthropic plugin, so nothing in your cluster could read Anthropic’s cost report
Grafana Cloud, Splunk, Dynatrace, New Relic OpenCost has no plugin for their usage or billing APIs

OpenCost’s Datadog plugin prices each usage type by matching its name to a Datadog product, falling back to the closest-named product, and applies Datadog’s month-to-date estimated cost per unit of that product. A line can therefore carry another product’s rate, and OpenCost does not say which lines did. That is an estimate from a similar product, which Seylo never presents as a price.

So Spend outside the cluster shows Datadog’s own measured usage, and for the dollars says not priced, with that reason. Your Datadog invoice (Plan & Usage) has the billed amount.

Spend outside the cluster shows OpenAI’s lines as billed, over the last 7 or 30 days, labelled Billed by OpenAI (its Costs API) .... The plugin reports every amount as US dollars without reading the currency OpenAI returns, and the label says the currency is the plugin’s.

A configured plugin with no rows is never shown as $0. The card says the plugin holds no billed spend for the window, and that this may mean it has not finished its first run, or that it failed.

If a cluster already runs the Azure price sheet or cost export

Section titled “If a cluster already runs the Azure price sheet or cost export”

Before Seylo read your Azure bill through a cloud account, OpenCost could read your Enterprise Agreement price sheet, or a Cost Management export, from inside your cluster. Seylo no longer offers to set these up: a connected cloud account gives the same figures with no key, no storage account and no chart change.

A cluster that already runs one keeps its row on the card, with its state, so a failure stays visible until you turn it off. While the price sheet works, OpenCost’s node prices are your negotiated prices, labelled as from your Azure price sheet; a price from your Azure bill still wins in the price book.

To move to the cloud account: connect the subscription on Admin → Integrations → Cloud accounts, check that Your Azure bill says On and working, then turn the old integration off.

State Meaning
Off Not connected. Your Azure bill links to Cloud accounts; for Datadog and OpenAI, the script is offered
On and working Working. For Your Azure bill, each subscription’s machine prices and currency, and when Seylo last read the bill
On, failing Configured, and it reports an error. For Your Azure bill, the error is shown per subscription; for OpenCost’s integrations, under OpenCost says: in OpenCost’s own words
On, no data yet Configured, nothing wrong reported, no data yet, and why that can be normal
On, not confirmed Configured, but Seylo could not read OpenCost’s status on this cluster

The messages you are most likely to meet:

Message What to do
No Azure subscription is connected, so Azure figures use public list prices. Connect the subscription on Admin → Integrations → Cloud accounts
OpenCost fetches the price sheet only after it has loaded the subscription's rate card with the same service principal, and the rate card failed. ... A cluster that runs the price sheet: its key is wrong, or its principal lacks the rate-card role. Move to the cloud account, or fix the key
The rate card loaded and the price-sheet download has started. Azure can take several minutes to generate it; until then OpenCost uses list prices. Wait, then reload
OpenCost cannot read your cost export. With workload identity, check the federated credential's subject and that the identity has Storage Blob Data Reader on the storage account; with a key, check the key. A cluster that runs the cost export: check its identity. Or move to the cloud account
OpenCost reached your storage account but found no export data yet. A new Cost Management export can take up to 48 hours to write its first file. Wait for the export’s first run
OpenCost has not run the cost-export import yet; it runs every 6 hours. Wait
OpenCost's custom-cost pipeline did not start ... A plugin binary is missing or a config key is misnamed. The message names the two log commands to read
The plugin is configured but OpenCost holds no ... costs for the last 7 days. ... OpenCost keeps a plugin’s error only in its own log. Read it with the command below

OpenCost’s own log, which the card points to for plugin problems:

Terminal window
kubectl -n seylo-observability logs deploy/seylo-opencost -c opencost --since=6h

Your Azure bill needs nothing from your cluster or your network: Seylo reads it from Azure directly, through the cloud account.

The chart adds no egress policy for OpenCost, because it must reach the Kubernetes API. If your firewall allow-lists hostnames, OpenCost needs:

Integration Hosts
Datadog, OpenAI plugins github.com and objects.githubusercontent.com (the plugin download), then api.<your Datadog site> or api.openai.com
Azure price sheet (already running) management.azure.com, login.microsoftonline.com, and the download host Azure returns at run time
Azure cost export (already running) login.microsoftonline.com, <account>.blob.core.windows.net
  • Your Azure bill: remove the subscription on Admin → Integrations → Cloud accounts, and revoke its role in Azure. Node prices go back to your uploaded rates or the public list.
  • An OpenCost integration: run the same helm upgrade with that integration’s enabled=false, for example --set opencost.plugins.datadog.enabled=false, then delete its Secret (or, for the cost export, the federated credential and the identity). Seylo stops showing what OpenCost no longer reports.