AI Costs in the First Half of 2026: Google Cloud, AWS, and Azure. Not Just LLMs!

How Much Does an AI Application Actually Cost?

It's simple to compare 1M Gemini, Claude, or ChatGPT tokens, but much harder to estimate the cost of an AI application used in production. AI in production can mean, among other things:

• LLM (generative AI)
• Grounding, other external tools, guardrails, and observability
• Semantic search, vector indexes, feature stores
• Document parsing, reranking
• Hosting, data ingestion, and querying

In the first half of 2026, the cost of some LLM models continued to drop, thanks to new cost-optimization techniques. At the same time, frontier (SOTA) models and agentic applications drove up total costs for the applications that use them. The cost per token can decrease even as total investment increases.

Note: Prices in this article were verified on August 7, 2026. The article will be updated periodically going forward.

1. LLM Prices Moved in Both Directions

It's often said that "AI models are getting cheaper," and this is true for fast, compact models. But in parallel, premium models and agentic execution drove up costs significantly in 2026.

In the first half of 2026, three things happened simultaneously:

• the minimum threshold for text processing stayed low
• the performance available at intermediate tiers increased
• SOTA models and advanced reasoning raised the cost ceiling

Costs per 1M Tokens

The table below estimates the cost of a request (or several requests, depending on the provider) of 1 million input tokens and 200,000 output tokens, using standard rates, without caching or batching and without a long-context surcharge.

Capabilities between models are not equivalent, which is exactly why the wide cost range for the same volume is worth noting.

On Google Cloud, Gemini 2.5 prices range between $0.10/$0.40 for Flash-Lite and $1.25/$10 for Pro. Flex and Batch can cut standard rates by roughly 50%, and cached input is billed at a much lower rate than fully processed input. (Source: Google Cloud)

AWS even estimates that cached input can cost up to 90% less, and moving simple tasks from Sonnet to Haiku can cut costs by roughly two-thirds. Batch inference is available for some models at 50% of the on-demand price, and Intelligent Prompt Routing is billed at $1 per 1,000 requests and can reduce the model cost by up to 30% on AWS for supported combinations. (Source: Amazon Web Services)

Execution Quality Versus Task Cost

For these reasons, in 2026 it became popular to classify requests and use the premium (SOTA) model only when the superior result justifies the cost (routing).

CHEAP MODEL
   │
   ├── correct zero-shot answer ──────────────────> low cost
   │
   └── incorrect answer
        │
        ├── retry
        ├── model switch
        ├── engineer intervention
        └── task redone ──────────────────────────> high total cost

SOTA MODEL
   │
   ├── simple request ────────────────────────────> unjustified cost
   │
   └── complex request solved correctly ──> justified cost

2. AI Workflow Costs Beyond Tokens

The cost of the language model (LLM) is broadly calculated as follows:

The cost of the AI application, however, is:

More importantly, retries are one of the most overlooked sources of cost, especially in agentic AI. AI application cost must identify the points at which an AI flow can fail and must be mandatorily retried. In emerging agentic applications, a retry can mean dozens or hundreds of repeated billable steps.

Google Cloud: Grounding Can Cost More Than the Model

Let's assume a Gemini 2.5 Flash request uses 2,000 input tokens and produces 500 output tokens.

If you want to use grounding (searching Google for current information that substantially improves answer quality), you can no longer calculate just the token price.

After exceeding the available free quota, grounding for Gemini 2.5 costs $35 per 1,000 grounded prompts, or roughly $0.035 for a single one. So the grounding service will cost nearly 19 times more than token processing. For Gemini 3 models, Google charges $14 per 1,000 queries for grounding after the first 5,000 per month. And in this case too, a single user request can involve multiple billable searches. (Source: Google Cloud)

AWS: RAG Can Cost More Than the Model Itself

AWS published an example for a 50 GB knowledge base with 100,000 monthly searches.

• With standard retrieval, the cost is $250 for index storage and $100 for searches, totaling $350 / month.
• With agentic retrieval and an average of two internal searches per call, the same volume reaches: $250 for index storage + $400 for agentic calls + $200 for underlying searches. Totaling $850 / month.

These are the costs before calling the model that generates the final answer, which is billed separately. (Source: Amazon Web Services)

Bedrock Guardrails (AWS's AI safety filters) also use their own distinct cost units, including text units, image processing, and input vs. output metering. For example, text content filters are billed at $0.15 per 1,000 text units, and contextual grounding checks at $0.10 / text unit. (Source: Amazon Web Services)

Azure: An Example of Dozens of Cost Variables for a Single Service

Foundry IQ is the evolution of Azure AI Search, building a managed knowledge base for the whole company for use with agentic AI. As such, it aggregates a wide range of AI-related technologies, resulting in very detailed pricing before you even reach the LLM cost. Dedicated plans are billed, among other things, by the hour, based on Search Units, replicas and partitions, and the use of special features like Semantic Ranker, agentic retrieval, or access to external sources.

Azure will also offer Serverless, i.e. usage-based models with the scale-to-zero savings feature (shutting down when not in use), but, as with other clouds, actual prices depend on region, tier, and commercial agreement. (Source: Microsoft Azure)

We listed 12 cost variables in two paragraphs, which is why we recommend configuring services in Azure's calculator or the other clouds' calculators.

3. Complex AI Products Are Not Priced in Tokens

Artificial intelligence also includes recommendation systems, e-commerce search, forecasting, computer vision, fraud detection, and OCR, which make up a large share of real-world AI use.

But predictive AI uses different billing units than LLMs. When you use a cloud OCR service, you pay per page or image, not per token as when transcribing with an LLM (with the inherent risk of hallucinations for an LLM).

Google AI Commerce Search

AI Commerce Search (formerly Google Retail / Google Vertex AI Search for Commerce) offers search and personalized recommendations for online stores and for B2B applications, as we detailed in AI Guide 1 for Sales.

Google manages the infrastructure, storage, and algorithms, acting as a recommendation engine into which the company uploads its data, while Google's experience feeds top-tier features back into the company's own applications.

Google Example for a Retailer with 10 Million Predictions per Month

Predictions account for approximately 85.7% of the cost, while training and tuning account for 14.3%. (Source: Google Cloud)

Amazon Personalize

For the competing solution Amazon Personalize v2, AWS charges:

• $0.05 per GB of ingested data;
• $0.002 per 1,000 interactions used for training;
• $0.15 per 1,000 recommendation requests.

In an official example with 200 GB of data, eight training runs, and approx. 12.3 million monthly requests, the cost is $2,016.80. Of this amount, approximately 91.6% is inference, 7.9% is training, and 0.5% is data ingestion.

However, a minimum threshold of one request / second also applies for real-time campaigns. A low-traffic service can be billed for the minimum capacity even if it doesn't use it (Source: Amazon Web Services). Other types of recommendation systems are billed per hour and based on the number of users.

Google Agent Search and Enterprise Search

This product, which enables search and assistant applications (like chatbots) over data, shows how several technologies combine into a managed variant:

(Source: Google Cloud)

So, even if the LLM cost can be exactly zero, a sophisticated AI product still has a considerable cost. Services are generally billed in usage units, not tokens, which provides predictability but increases complexity.

Conclusion: Investment and Optimization Move in Parallel

We found a Gartner study from May 2026 estimating that global AI-related spending will reach over $2.5 trillion in 2026, nearly +50% compared to 2025. Of this amount, the financial category "AI Models" is tiny, only $32.6 billion.

Gartner actually measures provider investment in infrastructure and software products where AI is embedded, while AI Models represent only about 1.3% of the broader AI market. (Source: Gartner)

So, beyond the cost of the AI technologies themselves, the infrastructure needed to adapt to them (e.g. a data warehouse like BigQuery) leads to rising costs that are investments.

For actual running costs, prices dropped in 2026 for compact models and through optimization mechanisms. But SOTA models and agentic systems raised the price ceiling. Planning must start from the type of application being developed:

• Simple chatbot: the LLM can be the main expense
• RAG: retrieval, indexing, and grounding can exceed the model
• Recommendation system: predictions and retraining can dominate the cost
• Agentic AI: retries, tools (tool calls), and rules make up a large part of the expenses