Skip to content
NewFinOps, Kubernetes and AI, on one platform
Book a demo

GPU data centers

Owned GPUs, priced and put to work like cloud ones

GPU servers in your data center have no hourly bill, so their idle hours are invisible. A rate card gives every GPU an hourly price. From then on, owned GPUs are allocated to teams, shared with MIG, and offered as a burst location to your cloud clusters before anyone buys more.

Book a demoOn-premises pricing

Bought once, then invisible

  • No invoice:

    The servers came from a capital budget years ago, so the teams using them never see a cost.

  • Power is the hidden line:

    GPU servers draw far more than general servers, and the facility adds cooling and distribution on top.

  • Buying before using:

    New hardware gets ordered while owned GPUs sit allocated but barely used.

  • No like-for-like comparison:

    Cloud GPUs have an hourly price and owned ones do not, so the build-or-rent decision is a guess.

GPU-hour rates

A GPU-hour rate from what the servers really cost

Set a GPU-hour rate per node pool or label, or attach the purchase price, depreciation period and monthly power, space and support to the racks they cover. Rate cards are effective-dated, so a new support contract changes next month, not last year.

  • Unit rates or amortized hardware
  • Matched to nodes by pool or label
  • Reviewed with finance, versioned in Git
rate-cards/dc1-standard.yaml
# rate-cards/dc1-standard.yamlapiVersion: kubeon.io/v1kind: RateCardmetadata:  name: dc1-standardspec:  currency: USD  effectiveFrom: "2026-01-01"  # Option 1: unit rates, per node pool or node label  rates:    - match: { nodeLabel: "node.kubernetes.io/pool=general" }      vcpuHour: 0.0312      memoryGiBHour: 0.0041    - match: { nodeLabel: "node.kubernetes.io/pool=gpu" }      vcpuHour: 0.0312      memoryGiBHour: 0.0041      gpuHour: 1.18  storage:    - storageClass: ceph-rbd      gibMonth: 0.06  # Option 2: amortized hardware, spread across the nodes it covers  hardware:    - match: { nodeLabel: "kubeon.io/rack=r12" }      purchasePrice: 184000      depreciationMonths: 48      monthly:        power: 1450        space: 900        support: 610

Same method as cloud

Owned GPUs allocated by request, idle GPUs reported

Once a GPU has a rate, it is allocated exactly like a cloud GPU: by the GPUs pods requested each hour, with unrequested GPUs reported as idle owned capacity. Teams see their data center GPU cost next to their cloud spend.

  • Hourly GPU requests per pod
  • Idle owned GPUs with their monthly cost
  • Data center and cloud in one report

Before the next order

Use the GPUs you own before buying more

Idle owned GPUs, and allocated GPUs running at low utilization, are capacity you have already paid for. The GPU optimizer shares owned A100s and H100s with MIG, and offers idle owned GPUs as the first burst location for your cloud clusters, so a shortage in the cloud is met from the rack you already own.

  • MIG sharing on owned servers, GPUs freed priced at the rate card
  • Idle owned GPUs as the first burst location
  • Cloud and owned GPU-hour rates side by side

Where the GPU savings are

52%

average use of the GPUs in a large Microsoft training cluster

Jeon et al., USENIX ATC 2019
18%

average GPU use on Alibaba machines with 74% of GPUs allocated

Weng et al., USENIX NSDI 2022
$8,433/mo

saved by removing one GPU node after the peak at our sample company

Sample company, not a customer result

KubeOn GPU overview

What goes into a GPU-hour

Four inputs to an owned GPU's price

InputHow it becomes a monthly costWhere it comes from
HardwarePurchase price divided by the depreciation period in monthsFinance's asset register
PowerAverage IT power in kW x facility overhead (PUE) x hours x price per kWhFacilities, or a measured PUE
SpaceRack or floor cost for the server's share of the rackColocation contract or facilities
Support and softwareHardware support, operating system and GPU software licencesVendor contracts

Uptime Institute's 2024 survey put the average data center PUE at 1.56: total facility power about 56% above the IT load. See pricing Kubernetes in your own data center on the blog.

Owned GPU capacity with a price

GPU-hour rate cards

Unit rates or amortized hardware, matched by pool or label.

Team allocation

Owned GPUs split by request, like cloud ones.

Idle owned GPUs

Unrequested capacity with its monthly cost.

MIG on owned GPUs

A100 and H100 shared, freed GPUs priced at the rate card.

First burst location

Idle owned GPUs offered to cloud clusters first.

GPU data centers: your questions answered

KubeOn GPU

Keep exploring

Product

GPU burst

When a region has no GPUs left, attaches GPU nodes from the regions, clouds and data centers a policy allows, priced with data transfer, and moves the pods back home when capacity returns.

Product

GPU sharing

Finds workloads whose GPU compute and memory fit in part of a card, then partitions supported GPUs with MIG or time-slices development GPUs, drains the node in your change window and reschedules the pods onto their slices.

Product

GPU cost and utilization

Splits each GPU node's billed cost by the GPUs pods requested, reports GPUs nobody requested as idle GPU time, and shows how busy the allocated GPUs really are, from NVIDIA DCGM metrics.

Evaluate GPU data centers with your team

Bring your platform and FinOps priorities. We will walk through the data, controls and evidence for your evaluation.

Book a demoRead the guide