Before You Scale the AI, Fix the Evaluation
Once an AI system makes measurable decisions in production, evaluation decides whether you can trust it. Five learning outcomes on metrics, operating points, and the handoffs between ML and generative AI.
Category
Engineering
Whose Authority Reaches the Tools?
65 minutes
Deciding, server by server, whether an ADK agent on Agent Runtime calls its MCP tools as the employee or as itself: on-behalf-of tokens for …
From a Leaked Pipeline Credential to Bulk Data
35 minutes
A service account key made before your org blocked key creation still works. How one legacy dbt key reaches BigQuery, and the inventory that …
What Your Agents Inherit From Your Data Engineering
25 minutes
An agent can only explain what your data platform can explain. How business rules, their history, and corrections are engineered from the …
From an Entra Sign-In to an Agent Session
43 minutes
How an employee's Microsoft Entra sign-in reaches an ADK agent on Agent Runtime: the token checks, the session binding, who else can call …
Native Tables, Managed Iceberg, or Iceberg in an Open Catalog
58 minutes
Choosing storage for a BigQuery lakehouse layer by layer: what native tables, managed Iceberg, and Iceberg in an open catalog each give up, …
Mapping the Agent Before Anyone Attacks It
45 minutes
A reference operations agent on Google Cloud, threat-modelled against OWASP and MITRE ATLAS, with a first control: a deny-by-default policy …
Keeping Raw Data You May Be Asked to Delete
91 minutes
Erasure in a BigQuery medallion lakehouse: keep emails out of bronze, delete in two stages, and compute when erased copies leave the …
Multi-Cloud Retail Without the Mess
21 minutes
Many QSR and retail estates run identity in Microsoft Entra ID, back-office systems on SQL Server in Azure, the data platform on Google …
Thousands of Sites, One Pipeline
22 minutes
Why operations and analytics report different sales for the same store-day, and how a medallion pipeline brings a multi-site network to one …
A 150B MoE on a Developer Laptop: An Upper Bound on What Faster Storage Buys You
18 minutes
A 150-billion-parameter mixture-of-experts model streamed from NVMe on an ordinary developer laptop with no usable GPU. Across a 200-token …
Tollgate v0.2.3: What Long-Context Pricing Does to a Spend Cap
11 minutes
Some models bill the whole request at a higher rate above a size threshold. What that does to a spend cap, and what changed in Tollgate …
Evidence That Looks Like Closure: Fuzzing, Fault Injection, and the GATE PARTIAL Checks
16 minutes
gate-fuzz v1.0.0 emits an artefact filed under Check15 that does not close Check15. Working out why is a useful way into which conformance …