- The Homebase AI
- Posts
- Prompt engineering shouldn’t be a guessing game...
Prompt engineering shouldn’t be a guessing game...
He helped scale Google Maps and YouTube. Now he's fixing the biggest failure mode in AI agents — observability.

Welcome to Homebase AI - A weekly newsletter where we share our interviews with founders & leaders building next-gen AI companies and curate interesting news, insights & trends in the AI space for our community members.
What’s on tap today:
Our interview with Nir Gazit
Weekly Headline Recap
What's trending in AI this week
Community Update
How Homebase Helps You
🎙️ This Week’s Interview
Listen on Spotify | Apple Podcasts
Watch on YouTube
AI Founder Story
The Black Box Recorder for AI Agents
Quick Background
Nir Gazit, an ex-Google ML engineer (Photos, Maps, YouTube) and former Chief Architect at Fiverr, co-founded Traceloop (YC W23) to tackle the pain of “flying blind” with LLMs. In early experiments (with GPT-3 in 2022), he and co-founder Gal Kleinman realized there were no tools to debug or monitor AI “agents,” only spreadsheets of prompts and outputs.
Traceloop’s mission: bring engineering discipline to AI. As Nir puts it, “Prompt engineering shouldn’t be a guessing game… It should be like the rest of engineering – observable, testable, and reliable”.
Traceloop’s product is a full observability stack for LLM-based apps built on OpenTelemetry. They first open-sourced OpenLLMetry, an SDK that captures prompts, completions and rich metadata at runtime. This “instrumentation framework” quickly took off: over 500,000 downloads per month and dozens of companies (Cisco, Dynatrace, IBM, etc.) using it. Those tools feed into Traceloop’s hosted platform, which lets teams monitor latency, cost, relevance, hallucinations and more – all with “just one line of code” to start collecting data.
In May 2025, Traceloop announced a $6.1M seed round (led by Sorenson Capital and Ibex) to expand the platform.
Key Achievements & Insights:
LLM Observability in Action: Traceloop offers an OpenTelemetry-native toolkit for LLM/agent apps. From one code change, it captures prompts, responses, latency, etc., turning “noisy LLM logs into clear insights”. Developers get real-time dashboards and alerts on model output quality (faithfulness, relevance, safety, drift).
OpenLLMetry traction: The open-source OpenLLMetry SDK has ~500K monthly installs, ~50K weekly active users, and 60+ GitHub contributors. Cisco, IBM, Dynatrace and others adopted it early, often for large-scale LLM monitoring.
$6.1M Seed Round: Traceloop closed a $6.1M seed in May 2025. The round was led by Sorenson Capital and Ibex Investors, with participation from Y Combinator (YC W23 alumni), Samsung NEXT, Grand Ventures, plus strategic angels (CEOs of Datadog, Elastic, Sentry, etc.). This funding will scale the core platform and enterprise features.
Enterprise Users: Major customers include Cisco, IBM, Dynatrace, and Miro. For example, Miro uses Traceloop to “monitor real-world performance at scale” and safely test new models (like GPT-4.1) in production without breaking their UX. In each case, Traceloop helps flag hallucinations or drift before users notice.
Product roadmap: After maturing data capture, Traceloop plans to “close the loop” – using observability data to automatically refine models and prompts. The vision is self-improving AI: collected errors and user feedback feed back into prompt versioning, automated evaluations, and model tuning in a continuous cycle (bringing code-like discipline to AI).
Founder lessons: Gazit stresses developer-centric design. Traceloop is like “Sentry for LLMs” – a lightweight SDK you plug in with one line of code, working with your existing tools and workflows (no heavy platform lock-in). It emphasizes open standards (built on OpenTelemetry) so teams aren’t locked to one vendor. Nir’s background – 15+ years in software (Google/Fiverr)– shows in Traceloop’s focus: measure everything and treat prompts like code.
Watch the full interview below! 👇
The Ultimate Lesson
LLMs are powerful, but unpredictable. Nir’s big realization: you can’t improve what you can’t observe…

Without proper tracing, debugging, or evals, teams are stuck vibecoding prompts and hoping for the best — a risky bet in production.
With Traceloop, AI builders get the same rigor they expect from traditional software: versioning, monitoring, testing, and continuous improvement. Treat AI like code — or it’ll break like magic.
🔗 Watch the full interview — How Miro Uses Traceloop to Scale GPT-4.1 Safely →
“At first, we thought cold outbound and marketing wouldn’t work – because it never worked on us. But we were wrong.”
HEADLINE ROUNDUP
Headline recap
OpenAI Eyes $500B Valuation
A mega share sale is in motion — and it may involve a secret AI device from Apple’s legendary designer.
👉How big will this bet go?200M New Users in 4 Months
OpenAI’s weekly user count has exploded — and this may be the fastest adoption curve in tech history.
👉What’s fueling the surge?GenAI Funding Hits $49.2B
Venture capital in 2025 has gone all-in on generative AI — and the money is pouring in at twice the speed of 2023.
👉Why is everyone betting on AI?AI Simulates 4B Atoms
Scientists used AI to simulate billions of atoms to reinvent concrete — and the impact could reshape global construction.
👉What are scientists building now?OpenAI Raises $8.3B
With revenue hitting $13B, OpenAI is expanding rapidly across industries — and investors can’t get enough.
👉Where is all that money going?
TRENDS
What's trending in AI
Retell AI
Key Player: Founded in 2023, Retell AI is led by CEO Bing Wu, alongside Todd Li (President), Evie Wang (CMO), Zexia Zhang (CTO), and Weijia Yu. The team brings deep expertise in conversational AI and enterprise tech, aiming to build voice agents that sound indistinguishably human..
Market Value: The company has secured $4.6M in seed funding from Y Combinator and top-tier investors. While its valuation is undisclosed, Retell is actively positioning itself in the booming AI voice market, projected to surpass $3B by 2028.
Adoption: Retell’s AI agents are already live in telehealth, logistics, and BPOs, helping companies manage high call volumes with minimal human involvement. Clients use it for inbound and outbound support, appointment scheduling, and issue resolution.
Recent Developments: In mid-2025, Retell was featured by OpenAI for its use of GPT‑4o, followed by a major product update enabling voice, SMS, and chat integration, along with real-time CRM syncing. The company made waves at CCW 2025, demoing hyper-realistic AI agents in live environments.
Growth: Since May 2025, Retell AI has experienced a +9800% spike in search interest, hitting 27.1K monthly searches. This growth reflects skyrocketing attention from the AI and SaaS communities, pushing it into the spotlight as one of the year’s breakout startups.
Why It Matters: Retell enables companies to cut support costs by up to 80%, while achieving industry-leading NPS scores (~90). Its agents don’t just answer — they listen, respond naturally, and learn with time. The result: faster resolution, happier customers, and scalable support that doesn’t break the bank.
The Big Picture: We’re moving toward a world where AI voice agents are always on, always available, and speak like us. Retell AI is at the forefront of this shift — transforming customer service from a cost center to a competitive advantage. This isn’t just about better bots. It’s about a new way for brands to speak at scale.

How Homebase Helps You
Discover tools, insights, and communities that help you build and scale your AI venture effectively.
Private Community
Private AI Founders & Executives Slack - Join our vetted community for exclusive insights & networking > Join Waitlist
Public Community
AI Enthusiasts Facebook Community - Join 1000+ founders sharing ideas & building in public > Join Super Founders
Product Development & Hiring
Building an AI product or need help Hiring? - Get advice on building AI products & scaling technical teams from > Here
Investor Database
Curated Investor Database - Access our continuously curated database of 40k+ active AI investors > Learn More
Join 10,000+ AI founders and leaders now reading our newsletter & part of our community.
What's your take on today's newsletter? |
Reply