Galileo: Observing GenAI So You Don’t Have To

We are excited to announce our investment in Galileo's $45M series B and partnering with amazing leaders like Vikram Chatterji, Yash Sheth and Atin Sanyal and the Galileo team.

‍

‍Galileo is the perfect fit in our investment puzzle

‍

At our early-stage fund, we’re keen to invest in businesses that excel in three key areas we call the "fit" questions: Founder-Market Fit (FMF), Product-Market Fit (PMF), and Go-To-Market Fit (GTMF). For FMF, we seek teams uniquely positioned to build their business, with founders who bring unmatched expertise and passion to the table. PMF is about undeniable customer demand—products that resonate deeply with their target market. GTMF focuses on how seamlessly the product integrates into the user's journey, with a clear and effective strategy to connect product and consumer. On all three fronts, Galileo doesn’t just meet these criteria—it surpasses them. Their team, product, and go-to-market strategy are perfectly aligned, setting them up for success.‍

‍

Galileo's team has lived the problem:

‍Each of the three founders of Galileo have lived and breathed the pains of getting probabilistic models into production. Vikram worked in Google's AI team long before AI was cool, using BERT style models to build vertical specific applications. Atin was an early member of Apple and Uber's AI teams, again using small models for evaluation. Yash had a similar tryst with small model evaluations for speech recognition, with his prior Google team now joining the powerhouse behind Gemini. Small models, purpose built for evaluation and security that can operate at low latency and cost is the best way to build out a scalable observability/security company and there couldn’t be a better team to pave the way forward for the industry.

‍

‍Galileo's customers can't buy fast enough:

With most of Galileo's sales coming inbound, rapid expansion within their customer base over the past 9 months and limited churn, it is reasonably clear that Galileo enjoys early signs of PMF. As we did our work into the company, we understood why. While many observability solutions in the market pigeon hole themselves into a specific problem statement - just in pre-production evals or just in prompt/response firewalls, Galileo stands out as a fully-featured, one-stop-shop platform for enterprises across pre- and post-production and across evaluation, monitoring and security. With the cycles between prototyping and production wicked small in genAI given the number of iterations needed to get a working product, having a single pane of glass is very beneficial for customers. Galileo's research-backed small models (its Luna models) work out of the box with limited tuning needed and don’t add significant latency or cost to the overall AI application. And just as Looker (a previous Premji Invest portfolio company) standardized the calculation of metrics across different teams, Galileo standardizes the metrics used across teams to evaluate genAI products, with users often times authoring custom metrics to track. Galileo's product is battle tested for the enterprise, with their models supporting applications that see loads up to 10,000 requests per minute, a huge relief for CIOs.

‍

Maybe the cherry on top was the excitement we saw from our in-house data science team that has created a fairly complex legal chatbot that reasons over Premji Invest legal documents (we know this is rare for a VC fund!). A single platform that could give them a hallucination or uncertainty score and allow for a data-backed approach to improve the product was the need of the hour for that team.

‍

‍Galileo appears exactly when users need it most:

This last 'fit' is the most rare for us to see in early teams that are still figuring out their GTM strategy and we were pleasantly surprised with the team's depth of thought. Galileo truly builds for the user, inserting the product at the right time in a user's journey, instead of building a product and hoping for the user to realize its importance and come to it. When they realized the biggest pain point was manual evaluation of apps that slowed down development times, they created the evaluation product, first with a larger LLM and over time, shrinking it to something custom-built. When customers moved into production, Galileo started offering a monitoring product that can evolve to a tracing product as more complex workflows get built. And when issues of security and PII cropped up, guess what, Galileo evolved and built a product around that to grow with their customer.

‍

Even more surprisingly is the team's knack to understand how to price and package products. Driven by deep understanding of the user and their job-to-be-done, Galileo prices appropriately to ensure that their product does not hinder experimentation, gives away products / features for free to ensure widespread adoption amongst certain roles but also knows when to switch to consumption based pricing to enjoy the upside and grow with their customer.

‍

Tech innateness leads to lasting durability:

‍

Let's take a walk down memory lane. Over the past two decades, we have seen software observability change continuously. Every language paradigm got its own Application Performance Monitoring (APM) solution, betting on a specific language taking hold. Wily captured the market initially and rode on the success of C and C++, closely followed by AppDynamics. With the advent of Java, we saw the rise of Dynatrace, becoming widely popular despite existing solutions like Wily and AppDynamics. With the advent of Ruby on Rails, we saw New Relic capture the hearts and minds. And with the advent of microservices and the need for tracing, we saw companies like Datadog. The various waves of software observability have taught us that nativeness to the technology is critical to build a large company. Companies like Galileo, that are born in a genAI-first world have a natural advantage over companies pivoting into this category.

‍

GenAI observability and security are the bright spot in a rough AI middleware landscape:‍

‍

The term 'AI middleware' can be defined as all tools that are used to get AI systems into production. They can include tools that optimize the cost and latency related to inference, tools that can label and curate datasets or get data 'RAG ready', fine tuning platforms, orchestration frameworks for rapid prototyping, model repositories and collaboration tools, observability and security products etc.

‍

Within each sub-category, one can see multiple players, with seemingly undifferentiated products and the right to become a platform unclear. Many products within the category are also directly in the product roadmap of a larger player, whether it be foundation models, hyperscalers or chip manufacturers, creating an imperative need to expand to multiple products early-on in the startup's journey.

‍

We feel genAI observability and security is different in many ways.

‍

GenAI observability and security products tread this fine line between being strategically significant to the enterprise while still being complex enough to prevent in-sourcing. The current best way to 'observe' AI applications is to create a golden-dataset (created by a human within the organization) and have a really large language model use that as context to judge whether an answer is hallucinatory. While this approach works pre-production, it breaks down in production given the latency and cost burden associated with it. Creating smaller models is the right approach but tweaking them for this use case, getting it to work correctly and the overhead associated with maintaining it is not worth it for most teams and hence they look for external vendors.

‍

The vendor tool that is then bought can price on the value it delivers (ie., providing hallucination or answer uncertainty scores) rather than a relatively simplistic cost+ pricing model. Not only that, observability coupled with security ends up being a stickier product as enterprises define their own custom metrics that they use in production and across applications, not too dissimilar from how we've seen other software 2.0 products get sticky user bases.

‍

‍Conclusion‍

‍

With a simple-to-get-started but hard-to-replace product like Galileo, we are excited with the potential growth opportunities ahead of the company. Potentially one of the biggest axes of growth is the rise of multi-agentic systems, basically AI applications that coordinate with one another to complete a task. The overall application flow (the steps that it must follow) can be pre-determined by a human being or by the AI system on the fly.

‍

Evaluating these systems is a hard task as errors developed by one application can quickly compound into larger errors across the chain. Not only that, tracing and monitoring these systems in real-time requires an incredibly nimble and cost effective solution. Lastly, with AI throwing up new threat vectors in simple AI applications, we can only imagine the complexity in a multi-agent system.

‍

We're excited to partner with Vikram, Yash, Atin and the rest of the Galileo team and are privileged to have a seat on this rocketship.