poolside: Software Reimagined Using First Principles

Premji Invest & poolside Partnership

‍

We are delighted to announce our investment in poolside's Series B, and to be partnering with Jason Warner, Eiso Kant, Paul St. John, Margarida Garcia, along with the rest of the poolside team.

‍

At Premji Invest, we firmly believe that vertically focused AI businesses will capture meaningful economic value by unlocking new markets and delivering products to customers in novel ways. poolside fits this mold perfectly: they are reimagining software development from the ground-up. From data curation to model training to post-model deployment, poolside is utilizing first-principles to attack these pillars in a unique manner. We know poolside is on a path to solve software engineers' most pressing problems, and we couldn't be more excited to partner with the team on their mission to revolutionize the broader software development industry.

‍

Why vertical approaches generate alpha

‍

poolside is building a foundation model for software development, with the goal of helping engineers design, plan, and generate high-quality code that executes as prompted. While general-purpose foundation models will continue to play a role, value will accrue to players like poolside that are taking a fundamentally different approach. Think about it this way: the data and techniques to train the first generation of large language models, i.e. the web and RLHF, were designed for language understanding. Computers can now "understand" English grammar and syntax. This is a very different problem than building workable code, which needs to execute, compile, and pass rigorous test harnesses. As such, copilots that use traditionally trained models produce buggy code that needs to be refactored. poolside's vision is much bigger: helping engineers both plan & generate code that works every time. The ability to "plan" enables engineers to also act as managers, not just ICs.

‍

As context, we led the Series A of Hippocratic AI with a similar bet that a vertical foundation model focused on nursing care would outperform general purpose foundation models over time. Hippocratic similarly rethought the entire model build, from data curation to model training from first principles. Hippocratic's latest Polaris 2.0 models uses a novel architecture, utilizing: #1) unique medical datasets such as deidentified patient conversions, #2) RLHF using trained nurses, #3) specialized "small" support models, as well as a number of other proprietary techniques. The result? Hippocratic's models far outperform foundation models across almost every task ranging from lab analysis to prescription adherence to human nurse escalation accuracy, by a wide margin. And today, Polaris 2.0 provides correct medical advice in 99% of scenarios vs. 81% for human nurses.

‍

We believe a similar dynamic will emerge in software development. poolside is experimenting with unique model architectures purpose-designed for long context windows, building in-house data pipelines optimized for generating high-quality code tokens, and training models using a novel reinforcement learning approach. The bet is that these unique approaches will yield superior results over time, a la the early signs of success we are seeing in Hippocratic AI.

‍

Model architecture

‍

poolside has made impressive progress on their RNN inspired non-transformer-based model architecture that allows for truly linear attention (i.e. a magnitude faster inference for similar size models), which already powers their code completion model in production. poolside has also built an in-house distributed training stack from the ground up that today powers training on their 10,000 GPU training cluster.

‍

Data curation

‍

poolside is taking a different approach to data curation than OpenAI/GitHub. Whereas OpenAI trains on the entire set of public Git repos, poolside has carefully selected a subset of the Git repos on the internet that takes the quality of the code in mind. poolside's goal is simple: ingest code that retains the highest quality code samples. They use deep metadata enrichments that include #1) a graph of developers across major code hosts, #2) contributions that relate to each other, #3) seniority of developers, and #4) experiences in languages/frameworks, among others.

‍

Additionally, poolside has built the world's largest synthetic code generation sandbox. The team believes there is a dearth of real-world high-quality code samples to power a world-class model. For context, there are roughly ~50T tokens in the world today, of which~3T tokens are code data. poolside aims to generate web scale tokens through its proprietary synthetic code generation techniques to augment existing datasets. The analogy here is AlphaGo. Google designed AlphaGo to beat the world's best human Go player, but they ran out of high-quality training samples. To solve this, Google simulated millions of synthetic games whereAlphaGo played against itself. Eventually, AlphaGo's abilities far surpassed that of humans. poolside believes a similar dynamic is playing out with code.

‍

Training

‍

The major difference between the English language and code is that code needs to be rigorously tested, compiled, and executed in different environments across tons of edge cases. To "test" the output of a language model on many prompts in most cases is only possible with experts in the area providing human feedback. Code, on the other hand, can be simulated and executed. As such, ensuring code executes as prompted is an extremely complex and multi-faceted problem. poolside believes the training process for building software and generating code needs to look very different than what it does for language models today. The bet is that these innovative approaches should yield superior results with time.

‍

Conclusion

‍

What excites us about poolside is that this might be the biggest opportunity in AI yet. While today's pricing models are fixated on $/user/month ("co-pilot"), if poolside's models become increasingly more performant, we believe the world will move towards value-based pricing ("co-worker"). In this universe, the sky is the limit. Total software engineering salary spend is $500B+ globally (4.4M software eng in U.S. * $100K salary on avg. = $440B), suggesting a massive opportunity.

‍

The bet on poolside is as much a bet on the founders, technology, and TAM, as much as it is on the strength of their go-to-market team. The poolside team is built with enterprise chops. While Jason, Eiso, and Paul St. John understand deeply that they need to win the hearts & minds of developers, they also understand the challenges of selling into enterprises. As such, poolside architected a three-tier distribution approach: developers (bottoms-up), direct sales (tops-down), and channel. poolside today provides VPC/self-hosted deployments, offering enterprise-grade privacy and security, and in the future will have a Cloud offering (single tenant and multi-tenant).

‍

The long-term vision is for poolside's models to serve as thought partners for software engineers. As their models continue to improve, poolside will move up the stack in the types of functional code they can produce - from APIs to functions to modules to mini-programs to entire programs. This is a market where a first-mover advantage is critical to mindshare, especially in the enterprise. We believe this is a land grab opportunity, so we're extremely excited about the road ahead for poolside and cannot wait to see this incredible team shape the future of software development!

‍