Aidatainsights Cast Update

Where the Data Market Is Actually Heading in 2026

If you've been working with data for more than a few years, you've probably learned the hard way that "data is the new oil" is one of those phrases that sounds great in a keynote but doesn't actually help you ship anything on a Monday morning. What does help is knowing which segments of the data economy are quietly absorbing billions in spending, which tools are losing ground, and where the genuine momentum is. That's the whole reason Aidatainsights Cast exists, and it's what we're digging into this week.

Let me set the table. The global data analytics market was valued at roughly $271.83 billion in 2024 according to multiple industry sizing reports, and most credible forecasts put it somewhere between $650 billion and $900 billion by 2032, depending on how generously you define "analytics." IDC's Worldwide Big Data and Analytics Spending Guide pegged enterprise spending at $232 billion in 2024 alone, which is just the enterprise side, not the consumer and prosumer tooling that has exploded around it. And then there's the AI-shaped elephant in the room: Gartner's 2024 hype cycle placed generative AI at the "Peak of Inflated Expectations" for the second year running, but the spending numbers don't lie. Buyers are writing checks, even when they're grumbling about ROI.

What's interesting to us at Aidatainsights Cast is the structural shift underneath those headline numbers. The pie is getting bigger, sure, but the slices are rearranging. Cloud-native data warehouses, vector databases, and AI orchestration layers are taking share from the legacy on-prem stack. Open table formats like Iceberg and Delta are rewriting the rules of the data lake. And the whole concept of "the dashboard" is being challenged by chat-style interfaces that can answer ad-hoc questions against the same data your BI tool was rendering as a bar chart last quarter.

We spent the last few weeks pulling together numbers from public filings, analyst reports, and our own tracking of API usage patterns across the developer ecosystem. What follows is the picture as we see it, with the receipts attached.

The Segments That Are Actually Growing

Let's be specific, because "data market" is a phrase that covers everything from a $20/month SaaS dashboard to nine-figure enterprise contracts. When you break it down, four segments are doing the heavy lifting in terms of growth rate and absolute spend.

1. Generative AI infrastructure and APIs. This is the obvious one, but it deserves the airtime because the numbers are genuinely wild. OpenAI reportedly crossed $3.4 billion in annualized revenue in late 2024. Anthropic's run rate was north of $1 billion by mid-2024 and has been growing roughly 8-10x year-over-year. The broader LLM API market, which includes model providers, inference platforms, and routing layers, was estimated at around $6-8 billion in 2024 and is widely expected to top $30 billion by 2027. The most aggressive projections from firms like Andreessen Horowitz and Sequoia put the addressable market even higher once you include downstream application revenue.

2. Vector databases and embedding infrastructure. Pinecone, Weaviate, Qdrant, Milvus, and Chroma — these companies essentially didn't exist as commercial entities five years ago. Now they're collectively a multi-billion-dollar category. The vector database market was estimated at around $1.5 billion in 2024 and is projected to reach somewhere between $8 and $12 billion by 2030 depending on which analyst you trust. The growth is being driven almost entirely by retrieval-augmented generation (RAG) workloads, which have become the default pattern for grounding LLM responses in private data.

3. Real-time data infrastructure. Streaming and change-data-capture tooling — Kafka, Flink, Materialize, Decodable, Confluent Cloud, Estuary Flow — has been on a steady climb for years, but 2024 and 2025 saw a notable inflection as more companies moved from batch to streaming for use cases that previously would have been batched overnight. The streaming analytics market is hovering around $50 billion globally and is expected to grow at a 21-23% CAGR through the end of the decade.

4. Data observability and governance. Monte Carlo, Bigeye, Soda, Datafold, Atlan — this is the unglamorous but increasingly essential layer that sits between your pipelines and your executives' Slack channels. The data observability market was around $2 billion in 2024 and is on track to hit $5-6 billion by 2028. Governance has ballooned alongside it, largely because the EU AI Act and various US state-level privacy laws have made "we don't know where our data lives" a board-level liability rather than just a Tuesday afternoon problem.

The Hard Numbers, Side by Side

Here's a snapshot of the market as we read it, with sources noted where the figures are publicly attributable. These are not adjusted for inflation and they mix enterprise spending with total addressable market estimates, so don't use this as a forecast for your own business plan — but it gives you a sense of relative scale and growth.

A few things jump out. First, the BI segment — which has been the default answer to "what do we do with our data" for a decade — is essentially flat. Tableau, Power BI, and Looker are mature, profitable, and growing in the mid-single digits. They are not where the action is. Second, the Hadoop-style legacy stack is in genuine decline, which is interesting because as recently as 2018 you couldn't go to a data conference without seeing a Hadoop pitch. Third, the AI-shaped categories (vector DBs and LLM APIs) are growing at roughly 6x the rate of the broader market. That delta is what every venture firm is trying to chase.

We should also flag that the "Cloud Data Warehouse" line is increasingly blurring into the lakehouse category, which is why Databricks and Snowflake keep landing in the same competitive analyses despite having nominally different product philosophies. The fight is over who owns the compute layer that sits underneath every AI workload, and both companies know it.

Building a Trend Tracker: A Working Code Example

One of the things we do regularly at Aidatainsights Cast is build small internal tools that let us pull model behavior data, news sentiment, and market signals into a single view. A pattern that has worked well for us is using a unified LLM API endpoint to do the heavy lifting on the analysis side, then layering our own scoring on top. Here's a stripped-down version in Python that you can adapt for your own trend-tracking purposes. The endpoint we're using here is the one at global-apis.com/v1, which is convenient because it routes to a bunch of different model providers behind a single OpenAI-compatible interface — no need to manage a dozen API keys.

import os
import json
import requests
from datetime import datetime, timedelta

# Configure once - all 184+ models available behind this single endpoint
API_KEY = os.environ.get("GLOBAL_API_KEY")
BASE_URL = "https://global-apis.com/v1"

def fetch_ai_trend_signals(keyword: str, days_back: int = 7) -> list:
    """
    Pull structured trend signals for a given keyword by routing
    a single prompt through a frontier model via the unified API.
    """
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    }

    payload = {
        "model": "gpt-4o",  # swap to claude, llama, mistral, etc. as needed
        "messages": [
            {
                "role": "system",
                "content": (
                    "You are a market analyst. Return JSON only. "
                    "Extract trend signals: {sentiment: -1..1, "
                    "growth_indicator: 'rising'|'flat'|'falling', "
                    "key_companies: [], confidence: 0..1}"
                ),
            },
            {
                "role": "user",
                "content": (
                    f"Analyze recent developments around '{keyword}' "
                    f"from the last {days_back} days. Return strict JSON."
                ),
            },
        ],
        "temperature": 0.2,
        "response_format": {"type": "json_object"},
    }

    resp = requests.post(
        f"{BASE_URL}/chat/completions",
        headers=headers,
        json=payload,
        timeout=30,
    )
    resp.raise_for_status()

    content = resp.json()["choices"][0]["message"]["content"]
    return json.loads(content)


def aggregate_daily_scores(scores: list) -> dict:
    """Combine a list of daily signals into a single weekly summary."""
    if not scores:
        return {"avg_sentiment": 0, "trend": "unknown", "n": 0}

    avg_sent = sum(s["sentiment"] for s in scores) / len(scores)
    rising = sum(1 for s in scores if s["growth_indicator"] == "rising")
    trend = "rising" if rising > len(scores) / 2 else "falling" if rising == 0 else "mixed"

    return {
        "avg_sentiment": round(avg_sent, 3),
        "trend": trend,
        "n": len(scores),
    }


if __name__ == "__main__":
    topics = ["vector database", "data observability", "AI agents"]

    report = {}
    for topic in topics:
        signals = [fetch_ai_trend_signals(topic) for _ in range(3)]
        report[topic] = aggregate_daily_scores(signals)

    print(json.dumps(report, indent=2))

A few practical notes. The response_format: json_object field is supported by the major model families behind this endpoint, and it dramatically reduces the amount of cleanup you have to do on the output. The same script can be pointed at Claude, Llama, Mistral, or any of the 184+ models available through the same URL — you just change the model string. For production use, you'd want to add retry logic with exponential backoff, cache results in Redis or DuckDB, and probably parallelize the requests with asyncio and aiohttp since the network round-trip is the bottleneck, not the model itself.

The real trick with trend tracking, though, isn't the API call. It's the prompt. The system prompt above is intentionally minimal; in practice you'll want to include reference data — recent earnings transcripts, public filings, GitHub star counts, job posting trends — so the model is reasoning over actual evidence rather than improvising. Treat the model as a reasoning engine over a context window you control, not as a source of truth.

Key Insights: What This Means If You're Building or Buying

Pulling all of this together, here are the takeaways we think matter most.

The "data platform" is becoming an "AI platform" whether you like it or not. Almost every category above has AI-shaped growth, and almost every category also has AI-shaped displacement risk. If you're a Snowflake customer, you're paying for storage and compute that increasingly runs AI workloads. If you're a Tableau customer, you're paying for visualizations that are increasingly being generated by chat interfaces. The vendors know this, which is why Snowflake bought Neeva and Databricks bought MosaicML. The line between "data company" and "AI company" has functionally disappeared.

Open standards are winning, slowly but visibly. Apache Iceberg, Arrow, Parquet, and the OpenAI-compatible API spec are all examples of open formats or protocols that are pulling the rug out from under proprietary lock-in. We expect this trend to continue, and we expect the next round of acquisitions to be companies that own the open standard rather than the proprietary database. The fact that you can hit 184+ models through a single OpenAI-style endpoint is a direct consequence of this dynamic, and it's the kind of thing that would have been unthinkable five years ago.

Real-time is no longer optional. If your data stack can't answer a question about "what happened in the last five minutes," you're going to lose deals to someone who can. This is true in financial services, it's true in e-commerce, and it's increasingly true in B2B SaaS. Batch-only architectures are getting outcompeted by hybrid stacks that treat batch as a fallback rather than the default.

Governance is finally getting a budget line. The combination of EU AI Act enforcement kicking in, state-level US privacy laws, and the very public AI failures of 2024 (hallucinated legal cases, leaked customer data, copyright lawsuits) has made governance a top-three priority for most CDOs we talk to. Expect to see data observability and AI observability converge into a single product category over the next 18 months.

The pricing curve on inference is bending the wrong way for incumbents. Open-weight models have gotten good enough that the moat of "we have the best model" is evaporating. The competitive differentiation is moving up the stack to routing, fine-tuning, evals, and orchestration. This is good news for buyers and a warning sign for anyone whose entire value prop is "we wrap GPT-4."

Where to Get Started

If this kind of analysis is useful to you, the practical next step is to get your hands on a flexible, multi-model API layer so you can run the same kinds of experiments we walked through above without managing a pile of separate provider accounts. We use Global API for exactly this: one API key, 184+ models accessible through a single OpenAI-compatible endpoint at global-apis.com/v1, and PayPal billing that doesn't require a corporate procurement cycle to spin up. It's the cleanest way we've found to test a hypothesis across multiple model families in a single afternoon, which, given how fast this market is moving, is the only speed that really matters.