From a pandemic side project to delivering exabytes of web data to frontier AI labs, Andrej Radonjic has built a data business most people have never heard of. The co-founder and CEO of Wynd Labs, the team behind Grass, joins Yan on The Delphi Podcast in Miami for a conversation about why the open internet is closing its doors to AI, what labs actually pay for when they buy data, and how millions of people sharing spare bandwidth became one of the ways AI reads the web. Andrej shares how an arbitrage bot he built during his PhD pulled him into web scraping, why Grass was built on consent, and what it took to scale to exabytes, including a co-founder who flew to Ashburn and assembled servers overnight. He also explains how robots are learning from web video, why he compares AI data to defense tech, and how the new Contents API turns any public web page into a single API call. The conversation closes with the ownerless Grass Foundation behind the recent revenue attestation, what a moat really is, and his vision for Grass to become the way AI accesses the world’s information.

🔗 Connect with Delphi

🌐 Portal: https://delphidigital.io/
🐦 X: https://x.com/delphi_digital
📺Youtube: https://www.youtube.com/@Delphi_Digital

🎧 Listen on all podcast platforms: https://thedelphipodcast.buzzsprout.com

Yan: [00:00:00] What can’t agents do today and, and, and specifically what’s getting harder for them to access?

Andrej: If I’m being honest, most of the internet and AI kind of flipped that around where now the internet’s blocking everything, and this is what makes Grass actually almost magical. And from the internet’s perspective, it just looks like users of the internet accessing the internet as usual.

But in reality, it’s a massive data infrastructure platform feeding the frontier of AI. At this point, we’re at exabyte scale, so Grass has delivered exabytes of data to AI labs. Most people don’t know what an exabyte is. Flew into Ashburn and pulled an all-nighter, kind of slept on the floor, like assembled the servers.

The vision is for Grass to be the medium through which it accesses that information.

Yan: Full disclosure, Delphi Ventures are investors in Grass and we hold the token. Uh, but lo- would love to kickstart by kind of learning about you, Andrej, and, and, um, you know, what brought you here, and, and kind of a bit about [00:01:00] your background.

Andrej: Sweet. Yeah, I mean, what, what brought me here specifically was an invitation from you guys, right?

Um, uh, well, first of all, I’d, I’d actually like to start by thanking you for having me here. Um, this is pretty cool. Uh, I haven’t been in an in-person podcast before. I’ve done a bunch of these things where it’s on like a Zoom call or something, so th- this is pretty cool. Um, but yeah, yeah, uh, a bit about me.

I’m, uh, co-founder and CEO of Win Labs. Uh, we’re best known as a team that builds Grass and is actively contributing to it and, uh, building, uh, on top of it. Um, Grass is a network of millions of nodes that people opt into, uh, to share their bandwidth and, uh, get rewarded for it. Um, and then what happens on the other side of this is, uh, all kinds of data infrastructure gets built, uh, to use millions of people’s bandwidth to access web data at scale.

Uh, how I fell into this space is Uh, I don’t know, may- maybe it’s kind of random. I actually went to school, um, in Canada, and I did a [00:02:00] program called Engineering Physics, uh, where I specialized in two things, nuclear engineering and photonics engineering. Um, at the time, uh, that I graduated, most of the jobs in nuclear engineering, uh, they were in more remote parts of Canada that, you know, in my early 20s I didn’t exactly, uh, want to move to.

Um, and I discovered that I could actually build upon a lot of these skills I’d learned, uh, in physics, uh, working in the financial industry. So, so I did that for a few years, um, before going back to academia to pursue computational physics. Uh, during that time, the pandemic started and, um, during the pandemic, you know, I was working on this PhD in, uh, applied math and specifically, uh, modeling very complicated physics equations, um, and the math that goes behind that.

Um, and, well, when the pandemic starts and you’re a, a PhD student, you kind of find yourself with a decent amount of free time, and [00:03:00] a lot of that free time got channeled into a bunch of, uh, uh, I, I don’t want to, I don’t know what to call it, like, side quests or hobbies. Um, so, so I started building a bunch of random things.

Um, one of those was an arbitrage bot that, you know, would find all kinds of ar- arbitrage opportunities on the internet, uh, by looking at different prices across different, different, different websites. Um, this required a lot of scraping and that’s how I fell into this web scraping world. And fast-forward a few years later, I met my two co-founders and we talked about our experiences and this was a very, very obvious thing that we wanted to work on, and we’ve been doing it ever since.

Yan: Yeah. So, so, uh, yeah, very serendipitous in, in that sense where you, you basically are discovering that access to the public internet was beginning to deteriorate and, and kind of, so, you know, in that area, what did you kind of see that others missed, and what led you to, to really pursuing it?

Andrej: Yeah. I, I mean, the, the biggest signal really was the pandemic.

Um, during [00:04:00] the pandemic, most businesses went online. So during that time, um, a lot of businesses, the way that they would conduct intelligence, uh, so are things like price intelligence, for example, figuring out what your competitors are charging for a certain product, um, or, or even governments trying to calculate CPI.

Like, the way you would do that is actually send people physically into stores with a clipboard and, like, check the shelf price of each item and write it down. Uh, and then average that or, and then, like, do, run your analytics based on that data. When the pandemic- forced most of North America into lockdown, um, a lot of stores shut down and people were forced to spend more money online.

Because of that, web infrastructure became really advanced but then started the need for web intelligence. Um, so a lot of websites began noticing, hey, we really need our information to be public so people can go and discover us and use our products, buy our services, or pay for our products. Uh, however, we [00:05:00] want to limit and build kind of a walled garden around this data and make it such that only consumers can visit this website on the public web, but block any data centers or anything like that, right?

So, um, the need for automated web data infrastructure I, I, I don’t wanna use the word exploded, ’cause it explode- it really exploded later on, but, like, this revolution and, and, like, the need for this infrastructure really kicked off during the pandemic when all these things were forced online. Um, and what we were seeing at the time was, well, there’s no reason for this trend to slow down.

Like, it was very obvious that this is going to happen beyond just quantitative data, right? So e-commerce websites, uh, or any website that displayed prices at scale, uh, six, seven years ago, they were already blocking all data centers. They were already, you know, forcing all kinds of, like, browsers to show different things about their fingerprint in order to get access to their websites.

Um, [00:06:00] and we thought it was really obvious at the time, at least to us, um, that this trend is going to spread across the rest of the internet, uh, especially as machine learning algorithms become more powerful. Um, you know, we started working on this before ChatGPT really, like, blew up, when GPT-3 started this AI revolution at the end of 2022.

Um, uh, but, but despite that, you know, there were, there was already AI dominating most people’s lives. They just weren’t really noticing it, uh, and it was mostly in the form of recommendation algorithms on social media. So, like, this was already happening, and we were only seeing things that would continue to accelerate that.

Um, and it would become this huge problem that needed to be solved. Um, but then, you know, at, at the same time, the way that this was being solved back then was hundreds or thousands of legitimate businesses around the world, they needed the ability to access websites that only allow residential [00:07:00] IP addresses to access them.

Uh, and in order to get infrastructure that can do that, they were actually sneaking the software into people’s devices without their consent, right? Uh, and they were selling this to large enterprises. Uh, we saw that as a huge problem, and we saw it as, as unsustainable. So a, a core tenet of Grass is consent, so, like, the nodes on the Grass network are all consenting into the Grass network, um, and are being rewarded for it as well.

Yan: Yeah. No, it’s, it’s a great point. Can you kinda touch on the, the trade-offs of an opt-in network versus one where they kind of backdoor through VPNs, through smart TVs, and other devices?

Andrej: Yeah. I, I mean, the, the trade-offs are in the news, right? Um, these things get shut off very frequently, so despite just being completely unethical to do that without a user’s consent, um- It’s also unsustainable from a business perspective.

Uh, like, you know, it, th- there are obvious- there are [00:08:00] obviously, like, okay versions of this where businesses, um, will go and put something into a free screensaver, but a user has an explicit pop-up where they’ll go and click on, like, “Okay, I opt into this, and this is why I’m not seeing ads,” for example. Like, uh, that ad revenue’s being covered by my bandwidth or something like that, right?

Or i- in the case of Grass, you know, you download an app and you’re rewarded for your contri- contributions directly. Um, there, you know, the, the sketchier version of that is someone just sticks that into the software, into a software update or into a hardware update, the event which is scarier sometimes, and it just happens, and you, and you don’t really notice it.

Um, these things never really last, right? If you look up, you know, smart TV, residential proxy scandal, for example, you’re gonna see all kinds of examples of that. Um, and, and, uh, the, the unfortunate, you know, reality is that some companies are gonna continue doing those types of things. Um, but what gives me a lot of optimism [00:09:00] is the fact that, uh, you know, Grass has built something really, really powerful, um, that is, you know- Fully consented and, uh, done ethically

Yan: Absolutely.

So, yeah, you, you kind of get to this point where AI is consuming massive amounts of information just as kind of access to it becomes a bit restricted. So curious, I guess, what can’t agents do today and, and, and specifically what’s getting harder for them to access?

Andrej: Yeah. Um, if I’m being honest, most of the internet.

Uh, most of the internet that was designed for humans is relatively difficult for agents to access. Um, and there are a number of reasons why this is gonna continue to be the case for quite a while. Um, but, you know, in particular, if you, if you think about it, um, a lot of websites out there, they’re trying to block AI crawlers, uh, for a variety of their own reasons, um, or they’re trying to block their competitors scraping them.

Um, but whatever their reason [00:10:00] is, it’s very difficult for them to, uh, differentiate who’s doing what. So usually the answer is just a blanket block of any data center IPs, for example. So if your agent is running on a data center instance, so for example, on an AWS instance, um, every website will detect that agent as an AWS IP address, and that agent won’t look any different than, you know, their competitors scraping their prices to undercut them, for example.

They won’t really know the difference. Uh, and the default policy is normally just block it. Um, and, and we’re just seeing more and more of that. Uh, and, and, you know, obviously this doesn’t just apply to agents, it also applies to just standard LLM use cases or even in the process of building LLMs, which is where Grass is seeing most of its business right now.

Um, like when you’re crawling the web in order to get the data to train the model in the first place, um, like you, you see a lot of this as well. Um, in [00:11:00] order to get the best training data, you need to be able to access as much information as possible so that you can then filter that and, uh, you know, bake the right information into your weights.

Um, and yeah, like the, all, all of these things kind of come down to the same principle of you need the, the more, the more access to the public web, uh, your agent, your LLM, your crawler, whatever it is, like the more access it has, the better it’s fundamentally going to be. Um, and in order to have the best access possible- The only way to do that is to be able to route traffic through millions of residential IPs around the world.

Uh, and on top of that, being able to run it on sometimes custom browser infrastructure, like that’s what Grass does, for instance, uh, in order to do things like load JavaScript in websites, um, or, uh, appear with, like, various fingerprints as well. Yeah.

Yan: No, makes a lot of sense. So I think right now we’re kinda touching on, uh, both, uh, [00:12:00] training and inference data a bit.

So maybe if we, if we bifurcate it a bit, your business was kind of bootstrapped on providing training data, but it also puts you in a position to provide quality, uh, inferenced live data, right? Training is based on s- existing snapshots of the internet and, and, and, and, uh, inference, we need a lot of live data.

So maybe just talk about kinda the, the dynamics of the two markets and, and how you see them evolving. Sure,

Andrej: yeah. Um, so the property of modern AI or the way models are trained today is you go through this training cycle, which is often a long process. Um, most of this training cycle is actually spent on things like turning the data into a format that is useful for training.

Uh, and then after that, uh, you generate more data based on the model’s outputs. This is called reinforcement learning. Uh, and then based on that data, you can make tweaks to the model to make it useful at certain things in the real world Regardless, once the model [00:13:00] is trained, um, and put out into the real world, within seconds of it being deployed in the real world, um, its knowledge has a cutoff, and it can’t continue learning in real time.

At least that’s something that hasn’t been solved yet. Um, there is research going on in, like, continuous learning and, you know, having a model that’ll, like, continuously learn and update its weights, and I think this is an extremely exciting field, and it’s going to be solved eventually. But the way things work right now is you have a model that is trained up to some cutoff, and after that, it’s sort of blind unless it has access to the world’s information.

So, uh, a big part of, you know, the, the way Grass is currently being used for, uh, AI to access data is a- as you said, today there are two different processes, right? The training process where, you know, you crawl vast amounts of the Internet, uh, in order to create, like, very useful pre-training data sets. And often this involves targeting, uh, va- [00:14:00] I say vast because the scale is large, but in, you know, when you think about the scale of the entire Internet, sometimes this involves targeting a very large subsection of the Internet, uh, for specific types of files or specific types of content or the content in certain languages or things like that.

Um, after the model is trained, it needs the ability to go and see things on the Internet. A- and this ties into what we talked about earlier, right? You can deploy a model on an AWS instance, on some VM, on, like, basically any infrastructure you want. But if it’s in a data center, or even if it’s on your own residential IP address, it won’t be able to see content that’s only, um, visible to certain countries in many cases.

Uh, it won’t be able to access things at scale, at, at a real scale, u-unless it’s capable of routing through a network like Grass. Um, so a big emerging area, um, that we’re really excited about is actually [00:15:00] inference data. Um, and, and the idea behind inference data is you need the ability, uh, for models to, like, capture that context in real time.

So, like, you know, you’re using a model or the model’s, you know, being used by another model. That’s something that’s happening these days as well. But regardless, in order for the model to be useful, it needs the ability to tap into the up-to-date real time, like, information in the world, and, like, find those parts of the Internet or even outside of the Internet in the case of robotics, and, uh, use that in order to inform its decisions, uh Maybe several years from now, we’ll have continuous learning frameworks that are efficient enough to be running at scale.

Um, what this will end up doing is actually merging those two things. So models will be fed like a huge fire hose of real-time information from across the entire internet and constantly updating. Uh, that I think is personally pretty exciting because in the current framework, you do still get, you know, you throw a lot of context at [00:16:00] a model, eventually it’ll start hallucinating.

This is just a property of LLMs, whether you like it or not.

Yan: So do you imagine in, in that world training data becomes less relevant or is there still so much to be scraped and, and trained on that it’s just the, both of them will kind of grow in tandem?

Andrej: I, I wouldn’t say that it becomes r- less relevant. I think they become the same thing over, o- over time.

Um, I mean, e- even, even right now, um, you know, th- this morning, um, I had a look in like, a s- one of the smaller clusters that Grass is running, uh, collected like a few billion URLs over the course of a few hours, right? Um, and, and, and this is insane scale that’s pretty difficult to fathom. Uh, when you’re crawling a significant percentage of the internet, uh, in order to find training data, um, you’re actually building a huge web index.

Uh, that, like a web index is fundamental to being, being able to search the web and find the latest up-to-date information. So in [00:17:00] a way, a lot of the work that Grass is doing on training data, um, is going to be built on, uh, in the inference stack, right? So like you’re, you’re, you have to like, uh, you can think of training data as, I, I don’t want to use the word, term treasure hunting, but I can’t really think of something better, right?

Like, as a, a better analogy. But you’re, you’re fishing through the whole internet. You’re deploying crawlers all over and finding specific types of data. Uh, but in the process of doing that, you’re kind of mapping the whole internet. And once you’ve mapped the whole internet, you have the, um, y- you, you have the, uh, fundamentals, right, for running search on top of that.

So you can go and search things. And then what Grass is very, very good at is once you’ve searched the whole internet and found a URL, it can extract all the contents of that URL, um, and give it to a model in a format that is useful to that model. And, and most importantly, it can extract the contents of that URL without getting blocked, without being fed honeypotted information.

It’ll be data the way that a website would want [00:18:00] a human to see it, but an agent would be the one seeing it

Yan: Very interesting. And, and so I guess w- I’d love to better understand the degree of infrastructure needed to actually set something up like this. Obviously, you have your, the, the 8 million nodes, uh, I think that’s the most recent number, and then, that you publicly shared.

Um, but then also I know you guys went into the data ce- center side, and there’s a bunch of kind of underlying hardware that’s needed to both house and distribute all of this data. I would love to learn about that whole experience.

Andrej: Yeah. Um, Grass’s data center footprint is just constantly growing. Um, uh, we, we, we kind of laugh because every time new servers are ordered and set up, it’s not a matter of, like, weeks or days, um, before we’re at full capacity again.

It’s usually a matter of hours, um, because of just how much work there is to do. Um, so in the last year and a half, I think Grass has been reinvesting a lot into infrastructure, um, [00:19:00] and has set up, at this point, thousands of compute servers, um, and, like, many, many, many petabytes of storage servers. Uh, this involved buying tons of RAM, uh, luckily before the price exploded.

Um, uh, as, as well as, you know, compute and everything else that goes into this, uh, renting the data center space. Uh, actually, one of my co-founders, Chris, our CTO, um, famously, like, during a… We, we, we had a, you know, very tight deadline to deliver a lot of data to a major AI lab. Uh, we were short on compute, and we had a bunch of servers sitting in a warehouse, and he flew into Ashburn and, like, pulled an all-nighter, kind of slept on the floor, like, assembled the servers.

Um, so an- anyways, uh, a lot goes into this from an infrastructure perspective. Um, you’ve got- Like I said, thousands of compute servers. Uh, at this point we’re at exabyte scale, so Grass has delivered exabytes [00:20:00] of data, um, to AI labs. Most people don’t know what an exabyte is, uh, understandably. Like why on earth would a person ever have to deal with an exabyte of anything?

Um, the way you think about it is a terabyte is 1,000 gigabytes, a petabyte is 1,000 terabytes, and an exabyte is 1,000 petabytes. So, um, the, the scale at which Grass is operating today, you know, we can, we can literally say it’s operating at exabyte scale ’cause it’s delivered exabyte, uh, levels of content to major AI labs.

Um, so you’ve got compute, which is necessary for sending web requests. You’ve got the millions of Grass nodes around the world which are responsible for providing these requests with a network fingerprint. So, you know, instead, a- and this is what makes Grass actually almost magical, is the fact that you can have insane amounts of computing power accessing the internet at very, very large scale, and from the internet’s perspective, [00:21:00] it just looks like users of the internet accessing the internet as usual.

But in reality, it’s a massive data infrastructure platform feeding the frontier of AI. Uh, and then storage is important too, because, um, well, it- A lot of the time you might ask, “Well, what, who cares about storage?” Right? Why, why are you storing anything when AI labs can just buy the data and you can send it to them directly?

Um, well, one thing we’ve been spending some time on recently is, uh, filling a few of the gaps that a lot of labs are seeing, um, which is taking insane amounts of raw data, filtering it, uh, processing it, turning it into something more useful. Uh, this is increasingly becoming a necessity, uh, especially in the training data market.

So, yeah, you, you also need to store a lot of stuff on your own side.

Yan: Yeah. We’d love to, to dive into that a bit. I guess from the, the customer side, right, you’re sending all this data to, to both Frontier Labs, Mag 7, um, and, and a handful of others. Like what’s really been most important for them in terms of that [00:22:00] data?

Is it quantity, kind of freshness, scale, uh, quality? Uh, how do you kind of begin to understand what, what the customer really values and, and how you’re gonna build your business around that?

Andrej: Um, I’d say it varies, right? Um, different labs will buy different types of things depending on what they’re training, what their philosophies with respect to training are.

Um, the, I guess, most obvious way you can look at this is if a lab is training a video model, like video data is extremely valuable to them. Uh, whereas if a lab is training a coding model, code data is extremely valuable to them a- and, and so on and so forth then, right? And you can extrapolate that, you know.

If you w- if a lab that has general AI capability wants to make their AI models better in a specific dimension, let’s say it’s in a dialect of a language that is relatively rare and hard to find and on difficult to access websites, um, that’s what they’ll hone in on it and look for, right? A- and [00:23:00] what’s most important between quantity, quality, speed, like i- in an ideal world, you have all of these things, right?

Uh, but practically, uh, between labs, this will vary, right? Um, some lab… And, and, you know, a lo- a lot of the companies in this space, they’re under extreme amounts of pressure. It’s like every few months a new model is topping the benchmarks. If you don’t stop working, uh, sorry, if you, if you stop working, uh, you’ll very quickly fall behind.

Um, so one thing we’ve been seeing a lot of, uh, as a trend is the necessity for speed. Um, you know, if you can- A, a model will be better if it has more data, obviously. However, uh, the model will be more efficient if it has all the best parts of that huge amount of data and is trained with less weights, right?

Um, so the work that goes into taking huge amounts of data and turning it into what you wanna [00:24:00] actually train your model on is massive, and it takes time. And time is probably the most valuable resource right now in AI. Um, so th- that, that’s a big part of why we’ve, you know, seen that as al- almost a necessity in, in the crawling process for, for AI training data.

It’s like, okay, well, AI labs have their own philosophies with respect to how they’re gonna actually ingest this data, how they’re gonna train their model on it. But, um, what we can do to make their lives easier, um, and to provide a lot more value for them is to, uh, help identify some level of signal in that data.

Mm-hmm. So, so that, that, that’s a big part of what we’re doing at the moment.

Yan: Makes a lot of sense. And, and so just to touch on kind of the, the, the reference to robotics data and video data, it’s, we’ve started an area we, we’ve become highly interested in, uh, just naturally. There’s an immense amount of, of training data that needs to happen on the robotics front.

And, and so to kind of come back to the video data you guys [00:25:00] mentioned, you’ve collected over, I think it was a, a billion public web videos at this point. So curious to understand how, um, the robotics companies have been using that and, and, and how they find that valuable.

Andrej: Sure, yeah. Uh, though the figure might be a bit outdated.

Grass is indexed at this point. It, it’s in, it’s over 100 billion. Um, I, I say indexed though, right? Because you don’t just blindly download every single video or scrape the contents of every single video. Yeah. Uh, normally these things are somewhat targeted, um, because it’s a very, uh, dense medium, right?

It’s expensive to store, it’s expensive to process. Um, I think robotics and robotics data is such an interesting space right now. Um, are, are you familiar with the, the generalist AI release that came out a couple months ago?

Yan: No.

Andrej: Um, okay. Um, people were saying it’s like the GPT-3-

Yan: Oh, yeah, yeah.

Andrej: Sorry … AI models.

Yeah. Yeah,

Yan: yeah,

Andrej: yeah. [00:26:00] Yeah. I– Okay, so actually let, let, let me like backtrack this for, for a moment because I, I think this is super interesting, right? Um, but for maybe for some listeners that aren’t like us familiar with, uh, how machine learning research normally occurs, right? It’s like- Um, the, i-if, if you’ve ever done any kind of machine learning, um, the, you’re, you’re gonna have the most fun possible if you know what to optimize for and your data is already in a format that’ll make it really easy for you to go and test whether it’s optimal for that thing.

Uh, so what I’m, what I’m trying to say is, um, if you have a way, you know, if you have a bunch of data and you can just throw it at an algorithm that you’re familiar with, uh, and that algorithm is going to quickly start learning things from that data, um, you’re gonna have what, you know, what you call a loss curve.

[00:27:00] So, like, the loss curve is, um, high loss means it’s making a lot of mistakes. Um, and then as that curve comes down, the model’s making less mistakes, and that kind of asymptotically approaches zero. And, like, if you see, like, a nice loss curve, that’s usually a good thing. And when you talk about scaling laws in AI, um, you find a specific loss curve, right?

You want your model to get better at this benchmark. And if throwing more of a specific kind of data at that algorithm is, gives you that nice curve, then you want as much data like that as you can possibly find, right? So, um, a good example of that, for instance, is if you’re training a model to speak Spanish, right, you want a certain amount of Spanish data.

And it’s very obvious that Spanish data is what you need for that, so you go and find a bunch of, you know, Spanish forums on the internet, for example, or you filter, like, Common Crawl for anything that, you know, you detect is Spanish, and you throw that at the model and you’re, you’re probably gonna see good loss as [00:28:00] it starts to make less mistakes in Spanish.

Okay. Now, this gets a lit- little bit more complicated when you’re looking at the real world The ideal state for robotics researchers, um, is to be able to have a model watch millions of videos, listen to, or I should say billions of videos, listen to millions of podcasts, read trillions of tokens of internet data, and from that information learn how to behave.

Generalize that into like, okay, I’ve seen a guy using a lawnmower, uh, now I as a robot know how to do that from watching the way he like walks around and moves and like moves his arms and holds tools or whatever. Uh, and, you know, having seen that millions of times, you know, if you’ve got the right setup and the right algorithms, you’re gonna have a certain amount of loss where you just like have the model watch enough of that content and it’ll learn how to do that itself, right?

So [00:29:00] I think one of the more… What got people really excited, um, in, in generalist’s case with that model release was the fact that they showed what I believe to be, and, you know, if I’m wrong, have you correct it, right? Often people say, you know, credit’s like an important thing, right? But it’s what I believe to be the first example ever of a model being able to just watch a ton of videos And learn how to actually do things from those videos.

If I remember correctly, it like learned to take a ball and put it in a cup.

Yan: Yeah, yeah.

Andrej: You know? A

Yan: specific color and-

Andrej: Yeah, yeah. And then they showed it in real time, right? Some guy did it, and the model watched the guy doing it, and then the robot went and copied exactly the same motion. Since then, um, Roda AI, uh, another robotics company, put out a paper,

Yan: uh,

Andrej: proving that you can actually scale this behavior with web video.

So, um, there’s been kind of an ex- a, a really, really quickly growing need [00:30:00] for web video, um, for robotics. Now the, uh, I, I say the challenge that’s present for this that isn’t present for other forms of AI is the specificity, right? So like how precise, um, the data needs to be for these types of models. Uh, what, what I mean by that is if you’re training a video model, right, like you wanna train, uh, Sora, for example.

The more diverse video you give the model, the better. You know, you wanna give it cartoons, you wanna give it like people interacting with things, you wanna give it nature, space, like or whatever, whatever you can possibly think of. Because the way that model works is it’ll learn all these different environments and then kind of blend them all together, and then you can tell it, “Hey, give me a video of a cartoon cat in space.”

It’s gonna know how to do that, right? Um, so for those types of models

Yan: That’s good. Yeah, yeah. Every combination.

Andrej: Sure. Yeah, yeah. But like the– And then the more diversity you have, you’re exponentially increasing the amount of combinations. [00:31:00] And then the more unique the model outputs are, one could argue, um, the more valuable the model is, ’cause it’s capable of producing more differentiated, um, outputs.

Now- For robotics, you, you kind of have, uh, I shouldn’t say the opposite, you still need a lot of diversity in your data, but it’s a lot more specific. Like, you don’t need, you know, there, there’s no reas- there’s very little reason in today, this might change, this might become outdated, but there’s very little reason today for a robotics model to watch a music video, for instance, or some cartoon of something.

There is a good reason for it to maybe watch a video game gameplay, right? And watching how a character moves and interacts with an environment. Or there’s a good reason for it to understand how, you know, humans doing chores, for instance. Um, so, so I, I’d say, like, there’s that differentiation that is still being figured out.

Like, that’s a very difficult thing to do is filter [00:32:00] video at web scale that way. Um, a- and this is something that we’re actively working on, and I hope soon to be able to share a little bit more about that ’cause it’s, like, I personally find it extremely exciting.

Yan: And is it the same network doing a new job or do you need different infrastructure and, and kind of, are, are you running into the same constraints?

Is, uh, yeah, are the, the issues similar?

Andrej: I, you know, I, I, I think the really cool thing about Grass is that it can scale in so many different directions. Uh, it’s exactly the same underlying infrastructure, just being able to tackle yet another web data problem. Like, ultimately, um, if some data is on the public web and models need some capability that, um, emerges from that data, um, that’s a problem that Grass can solve, and that’s what’s really exciting.

Uh, now, like the, the work that, that’s left to do is, um, building on top of that infrastructure and verticalizing into a few different, uh, mediums and, uh, as far as the data’s [00:33:00] being consumed.

Yan: Makes sense. And, and so, you know, you- you’re providing all these various types of data. In, in the grand scheme of things, who, who kind of owns it after you provide it to the labs and is, is there this potential risk of, of kind of them not necessarily needing, uh, you to the same degree, or is there just kind of vast amounts of new data where, and, and so much of it is kind of untouched?

Andrej: Yeah. So, you know, with most data vendors, right, um- Modern data vendors, I think we’re probably going to see some consolidation in, in the industry. But a lot of the data vendors that are popping up these days, they’ll go and focus on a very, very specific reinforcement learning environment. They’re gonna identify a problem with all the existing models.

They’re gonna say, “Okay, we found that, you know, Claude, ChatGPT, Grok, um, Muse, like all the models out there today all suck at this very specific thing that we found, and [00:34:00] we can measure it with this benchmark.” So that we’ve built an environment that the model can play in, right? Kinda like a little jungle gym, right?

Um, it’ll go and like play in that environment and get a lot better at that specific benchmark, and they go and sell that as a capability, right? That, that, that’s how most modern data vendors operate. And, and not all of them are reinforcement learning environments, but many, many, many of them are. Um, what end- what ends up happening is there’s obviously recurring demand for these types of things, but a lot of these vendors have a need to very quickly stay on top of, uh, what the new demands are and what the new research directions are for various labs.

So, um, you know, a lab will have a lot of demand for a specific skill set that it’s training into a model. This is going to be demand for a while. It’s going to be recurring for a little while. At a certain point, um, either they’re satisfied with that and they wanna move in a different direction, or they abandon [00:35:00] it for whatever reason.

Maybe they determine, “Hey, it’s not as important as we thought. There’s no need to like spend this many resources to continue improving this thing,” right? Um, the thing with Grass is that if a lab changes its research direction, regardless of the capability that you’re trying to teach your model, uh, the vast majority of the world’s internet…

Or sorry, the vast majority of the world’s information lives on the internet, and that’s where it’s updated first most of the time. So you still need the ability to go and access that. So we, we’ve seen, uh, multiple Grass customers actually changing regime, if you will, right? Like focusing on something, uh, deciding that they’re happy with that, and then completely changing the nature of their asks.

Um, but it, it’s something that’s constantly evolving, uh, as, as far as the training data space.

Yan: So just to kind of go, uh, onto [00:36:00] new product that you guys have recently released, Live Context Retrieval, and the idea is that any, um, agent can now, or rather, th- this kind of goes back to the whole live inference situation of, of needing data that at real time and, and just having the inability to access it.

And I think most don’t even realize it’s an issue. Um, I think, you know, you, you run into it when you try and, and have your agent run something, and then X says you’re getting blocked. And, and so, uh, it, it happens in a, in a variety of ways, but curious to understand who the, the customer is and, and where you see most of the demand for something like that.

And, and also just love to better understand the, the product from, from your perspective.

Andrej: Yeah, absolutely. Um, so I, I think what’s really, really cool, so you’re referring specifically to the Contents API that was just released. Um, Live Context Retrieval is what we’re calling a suite of products that not just agents, but like any AI model, um, can use, like leverage to access the internet.

Um, we’re starting [00:37:00] with Contents, Search is next, and then we want to bake in a lot of other capabilities, uh, many of them centered around things like multimodal grounding, like being able to like listen to audio, for ex- for example, and like, or read and interpret images, like actually understand them semantically.

Um, so, so th- these, these are all things that are in the pipeline. But what was released was the Contents API. And, and what I think is really cool about the Contents API is the fact that this is a tool that was built Uh, originally for internal use, uh, in order to collect training data at scale. Um, you know, when the team began building all these crawlers to collect all kinds of training data and all kinds of modalities, you know, oftentimes just running all your requests through a, you know, residential IP network, uh, it, it’s often good enough.

But there are many cases where it isn’t, where you need to go and do [00:38:00] tricky things with your TLS fingerprint, or you need to go and render the content in a browser on a server in an efficient manner, right? Um, and sometimes you have to go and vary the, um, patterns in your web responses as well. Uh, like sometimes you can send, you know, one response then the, the next one, and then a website will block it, so you go with the next one and then the first one, and oftentimes the website will not block it.

This can happen for various reasons, but what I’m getting at is, um, in order to streamline a lot of processes internally with respect to collecting large amounts of web data quickly for training, um, the team built a really, really good web contents API so that engineers, you know, working on this problem can just plug that into, you know, they can just call that API instead of rewriting, you know, all the contents needs from scratch.

Um, you know, as we saw this product evolve, as we saw [00:39:00] this product evolve internally, we realized, hey, like this has quickly become state-of-the-art. It’s better than most of what exists out there in the world. This should be, um, the first thing released under the live context retrieval umbrella. Uh, so What it is, is like w- w- we basically boiled down the entire process of having to manage web requests across, you know, residential proxies around the world, load browsers, figure out what kind of anti-bot you wanna implement, what…

all this other stuff. We boiled all that down to a single API call. Um, all you have to do is give it the URL of the website, and very quickly it will just give you the contents of the website. Uh, and you can even specify what country you’d like to view it in. Um, you can have the contents rendered in markdown, for example, or plain text.

Uh, often websites are inefficient for LLMs. You wanna reduce [00:40:00] the number of tokens, so that’s built in as well. And we’re adding all kinds of other features as well that we already use internally, but will be available for public use soon. Um, so yeah, uh, personally super excited and, uh, what excites me most is the fact that we’re opening up the ability that we’ve had internally for a long time- Mm-hmm

to, like, the world’s businesses to use. Um, and yeah, in, in terms of, like, use cases-

Yan: Right. That’s, yeah, I was gonna say, “The world’s businesses”, let’s touch on that.

Andrej: Yeah. Yeah. Uh, b- you know, uh, increasingly, um, increasingly all of them, right? Uh, in many different industries it’s very difficult to remain competitive unless you have access to web data without getting blocked, right?

So like e-commerce, um, ad tech, like these are very, very obvious. Uh, also just general agentic use cases, right? And this can vary across all domains. You know, like pe- people will have an end objective, and in order to reach that end objective, their agent needs to touch all kinds [00:41:00] of different things that may not be accessible to it if it’s running in a normal data center environment.

So let’s say you’re building an agent that is testing a stock trading strategy based on information all over the world. Maybe you wanna go and scrape s- like the stock of various items on various websites, and those websites are only available in specific countries, and you’re, you know, and then you wanna go and calculate the fundamentals of some public company before, you know, earnings get released.

You’re not gonna… Your agent isn’t gonna be able to do that for you unless it has access to something like live context retrieval in order to do that. Um, so you know, uh, uh- I, I don’t like say, you know, you ask me what kind of businesses, like I don’t like saying things like the world’s businesses. Right.

But, you know, you kinda get the idea. It’s, I’d say like most of the Fortune 500 falls into that category already, and then anyone using agents in a sophi- in a sophisticated manner also falls into that category.

Yan: Makes sense. And, and so k- [00:42:00] the Contents API is the first product in, in the suite. Would love to kind of do a similar dive into the others, both in terms of the, the product itself and, and what it kind of changes from a, a use case perspective and, yeah.

Andrej: Yeah, yeah. Um, so s- search is the obvious next one. Um, what, what we see really accelerating the adoption of LCR, sorry, that’s just the abbreviation for… It’s easier to say, right? So what we see really accelerating it is the ability to search the web. Um, arguably, we, we’ve done a lot of the hard part, which is accessing any part of the web.

Uh, now it’s a matter of aggregating all that information and turning it into a search layer, right? Uh, why I think this will accelerate demand for LCR is, well, it’s great that if you have the URL of any website, you can go and access its content. Well, uh, you know, the average agentic call, right? The, uh, [00:43:00] the average, uh, cycle of an agent that’s like accessing the internet, it usually needs to access anywhere from 5 to 50 web pages, and oftentimes more if you have some sort of like deep reasoning going on, in order to collect the context that it needs.

How does it find which URLs to access in the first place, right? So the Contents API, as it stands today, is very useful for businesses that need to monitor, for example, the same website over and over again every single day, or know what they’re looking for, right? Um, once we release Search, this will become pretty generalized.

Like, your model won’t even need to know where to look for its content. Uh, it’ll be able to use the Search API, access that content, and then call the Contents API to extract the content. Um, what is really interesting about Search in general these days is, um, like historically, it was a [00:44:00] very daunting task to go up against some of the biggest companies in the world and build a search engine from scratch.

Um, a big, a big reason for this is, like those companies invested insane amounts of money into the top minds in the world who used to do things like build very sophisticated knowledge graphs that could, in a matter of milliseconds, connect concepts that are learned from massive amounts of data in their web corpus.

Um, nowadays a lot of the ancillary, ancillary, ancillary tooling of-

Yan: Yeah, ancillary.

Andrej: But anyway, a lot of the peripheral tooling, right, for, for from AI, um, things like ANNs, things like, uh, uh, embeddings and things like, you know, and being able to, like, retrieve things from those embeddings. Um, a lot of these things, I don’t wanna use the word com- commoditized because there’s still a lot of innovation to happen there, but they’re a lot more accessible, right?

Mm-hmm. Um, [00:45:00] like a lot of the models that are open source out there that are, you know, they’re, they’re machine learning models that are not LLMs, uh, and they’re responsible for turning text into a series of numbers that can be searched. Um, and you can, like, when, once you have things in number format, you can run quick computation to search things and rank it, right?

A lot of these things have become extremely accessible to anyone. And then what’s preventing most people from launching an actual search engine on the internet is the logic that goes into how often do you recrawl each website, um, how do you extract the content of, you know, from those websites. So nowadays, you know, i- it- it’s kinda flipped, right?

Previously, like there were companies that, you know, had the infrastructure to go and crawl all the internet because the internet wasn’t blocking anyone. Then the hard part was building a lot of the infrastructure on top of that, and a lot of the logic, and a lot of mind power went into that. And AI kinda flipped that around, where now [00:46:00] the internet’s blocking everything, and if you wanna become a new player in this space, you need the ability to go and actually extract that data without getting blocked.

And then the part that used to be the hardest part, like there are open source tools now. Like you can, there are startups that you can, uh, sign up with, uh, and it, it’ll cost you less than $100 a month, and immediately you can just dump 50 million documents into a, uh, AWS S3 bucket, and it’ll be searchable for you, right?

Now, obviously, I’m oversimplifying things in some

Yan: way,

Andrej: right? We’re not in a state yet where, like all the open source tooling in the world is gonna give you like a state-of-the-art search engine. Nowhere near that. What I’m getting at though is that the moat has definitely become smaller on that side of things.

So, uh, Grass being a network that is extremely good at what’s currently the hardest part of that makes me very, very excited [00:47:00] for that part of the opportunity.

Yan: Very interesting. And, and so- You were able to kind of use the, the initial token to bootstrap this network of nodes that allowed you to have the, the, the inferen- the training data that you, you, you know, you scrape and you sell to businesses to, to Frontier Labs rather.

And then on top of that, you were able to build this suite of products. Um, is the network currently equipped to service all of the needs of those products, or do you see the need for expansion?

Andrej: I, I think, um, that’s a good question. Um, for latency sensitive tasks, so inference in particular, um, it’ll definitely be important for the grass network to grow.

Um, and for, you know, more residential nodes to be joining the network on an ongoing basis. Uh, the need for this is basically, you know, when you’re collecting a training data set, latency doesn’t matter that much. You know, you can… I- if you’ve saturated [00:48:00] a specific geography of nodes, you can wait a little bit longer to collect that data.

Obviously, speed’s important, but there are other bottlenecks. When it comes to inference data, um, that’s no longer a convenience that you have. Mm. Right? So if, you know, there’s a lot of demand for agents or models or systems in general to access a lot of data from a very specific geography, for instance, that geography will become saturated, um, and then you need to add more nodes.

So we, we see there being, you know, an increasing need for nodes. Now, I, I, I used to say that we won’t really be able to launch LCR until the number of nodes in the grass network is an order of magnitude larger. Um, the reason that we’re launching LCR now and quickly expanding on it and will be launching the products on a much shorter timeline, shorter timeline than we originally intended, um, it’s, [00:49:00] it honestly mostly just comes down to a lot of infrastructure, um, optimization that we’ve been doing.

Uh, we found ways of doing a lot more with, you know, a smaller number of nodes than we just, than, than we initially thought we’d be able to. So, so that, that’s been super exciting.

Yan: Yeah, amazing. Yeah, clearly allows you to do that quite earlier than, than anticipated. I remember just speaking about timelines before, and then this was more of a 2027 thing, so obviously great to see that.

Um, do you anticipate that, that kind of scaling to continue and, and you getting the ability to squeeze out of, more out of each node?

Andrej: I, I, I think so. Yeah, I mean, obviously there’s a, there’s a ceiling, um, to scaling anything. Um, and, you know, you, you start hitting a point of diminishing returns. But for the foreseeable future, we’re going to be continuously, um, doubling down on finding as many optimization gains as we’re able to.

Yan: Yeah. I think you’re in a, in a unique position, uh, in terms of being, uh, where the Frontier Labs are a customer to you. [00:50:00] So I was curious to just learn a bit about, you know, what, what it’s like working with them and, and kind of what, what the demands look like.

Andrej: Sure. It’s exciting, man. I mean, like, I’m not, um- Uh, there, there’s no reason to dilute, you know, how, how cool it is.

Um, ultimately, AI is probably going to be looked back at, looked back on as the defining technology that shaped a generation, right? Or maybe even an era in, in human history. Um, and being able to, you know, have visibility into, like, what’s actively improving those models and helping human civilization get to a new level of capability, it’s extremely exciting.

A- a- and I think for a lot of our team as well, it’s one of the most rewarding things, like, one of the things that keeps everybody very motivated. Um, in terms of how it is as well, you know, like, just due [00:51:00] to the way that a lot of, um, like, resources are being concentrated, uh, there’s a massive concentration of, like, very strong talent at a lot of these labs too.

So, um, oh, every conversation tends to be intellectually stimulating. Um, you get to see really, really cool things. Um, I’d say the hardest part about it is not being able to, you know, share to the world, like, “Hey, you know, we had a, we, we played a small part in-” Mm-hmm … um, you know, this really, really cool thing that just got released.”

And we, we could kinda tell that it was coming for the last few months because, uh, you know, we, we to some extent contributed to making it happen. Um, i- i- in that way, I, I do, uh… I, I think I mentioned this. I was on Ansem’s podcast, like, a couple months ago, and, uh, I was telling him that, um, we, we sometimes draw the comparison to defense tech- Mm

uh, when we’re talking about AI data. Um, like a [00:52:00] defense company that’s helping the government find some terrorists or shut down some operation, like, obviously they’re not gonna go and, like, tell the world exactly which operations they’ve worked on, um, anything else, because you can go and back things out-

Yan: Right

Andrej: a- and act in an adversarial way towards their customers. It’s very similar in AI right now. Like, there’s a race to- in, you know, towards reaching best capabilities. Um, if a, if a model company, you know, i- if everyone finds out what type of data that model company is focusing on, uh, you can very easily back out what kind of capability they’re trying to build into that model, and you can go and s- try competing with them before they release it.

So, um, it, it’s one of those things where we’ll see, like, a certain direction, and, like, i- as a direction, that’s often reflective of what the AI industry’s gonna look like six to 12 months from now. Um, and for a lot of our team, that’s probably the, uh, the, the one part where you, sometimes you wish you could share, like, more exciting things, um, but you, you really can’t.

It’s just the nature of the industry.

Yan: Yeah. And, and I think models [00:53:00] have improved faster than most have anticipated, and you guys have been- Been a, a data provider for, uh, all those training runs. So curious to kind of understand how the requirements have changed across training runs as the models have gotten better.

Andrej: Yeah. Um, it, it, it really depends, you know? Um For, for some labs, like the requirements have, have kind of stayed pretty consistent. Um, for others, they’ve sort of, depending on, I guess, you know, things like internal resource allocation, um, and bandwidth play a huge impact, right? Like, you know, you’ll kind of yo-yo between huge, like quantities of raw data and then like more very specialized data sets, and then going back to like huge quantities of raw data.

Uh, usually the way, the, the reason you see this kind of swinging back and forth is, um, to, to a large extent, like having, collecting a huge quantity of data from a specific domain is seen as a discovery process, [00:54:00] right? Um, and a lot of the time, you know, even though we can do this to some extent, like many labs, um, want to get their own confirmation.

Like, a lot of, like data is sort of the alpha for AI models, right? So they’ll go and take, you know, okay, we’re exploring a new domain to build a certain capability into a new model. We’ll get us a ton of data from that specific area, and we’ll filter out what actually moves the needle for the evaluation that we’re working on.

And then they’ll come back and they’ll say, “Hey, we found that this,” you know, it can be anywhere from, uh, I’ve seen 0.16% of the data set all the way up to like all of it is useful for moving the needle. Can we get a lot more of that? And then we’ll kind of narrow it down, um, and then build, you know, deliver data based on those requirements.

Um, overall though, I’d say like that pattern has continued, but overall volume of data needs has just generally increased, uh, [00:55:00] as far as training goes.

Yan: And what are kind of some of the biggest hurdles you guys run into in, in constraints?

Andrej: Um- Just generally?

Yan: Yeah. With, with the data provision for, for these frontier labs or just, yeah, labs in general.

Andrej: Yeah. Y- you know, um, th- this might be kind of a random one, but sometimes, you know, newer labs that are working with us, uh, don’t realize how vast the capability of Grass is. You know, they’re used to working with, you know, for example, this is the vendor that helps us with specific type of images. This is the vendor that has like this data, right?

So sometimes we’ll come into contact with a lab that, you know, wants to improve their model in some way, and we were put in touch with them because, you know, they heard that we’ve got a lot of specific type of data that we’ve crawled. And they’ll say… And, and then, you know, sometimes you get bucketed as, okay, they’re the, you know, they’re the guys that can do this, right?

Um, in reality, you know, Grass is the infrastructure that can [00:56:00] collect all of the Internet. So like, not just that specific type of data, right? Um- So then, you know, and, and then in some labs, the data, uh, procurement teams, they’re kind of siloed off, uh, depending on, like, what they’re looking at. So like, you know, you’ll have in some labs they’ll call it the tokens team, which is looking at text data to make their models speak better in text, right?

Um, or, you know, or s- you know, learn different languages. And then there’ll be, like, uh, an audio team. They’re lo- they’re mostly focused on getting voice data. Um, so the procurement teams often end up being siloed, and finding your way through this, you know, what increasingly corporate maze on a lot of these labs, uh, in order to, I guess, tell them, “Hey, like, we know you’re also looking for this.

You’re currently using Grass for this very specific thing, but you’d probably… This, like, all these other things would probably be pretty helpful to you, too.” Um, you know, I, I wouldn’t say it’s, like, a huge challenge, but it’s definitely something that exists [00:57:00] because everyone’s short on time. Like, you get a certain amount of everyone’s attention.

Mm-hmm. Uh, and being able to, you know, navigate that sometimes can take a bit longer than you want it to.

Yan: Yeah. And so do these, uh, customers optimize? ‘Cause, you know, the, the assumption is that With the ability to provide across a variety of datasets gives you some, uh, cost, like, cost of scale, economies of scale benefits where, you know, you can, uh, procure the data at, at lower cost versus someone who’s kind of hyper-focused on it.

So do the labs end up caring more about kind of cost or, or quality? Curious what, what moves the needle more for that? Uh,

Andrej: it, it’s both. It’s definitely both. I mean, it… Sometimes you’ll pay for speed, right? Like, it, it, it’s like anything else, right? If you’re, if you’re a business and there’s something that you really need, um, if you can pay more money to get it tomorrow instead of getting it three months from now, and if you really need it tomorrow, uh, there’s a price that you’ll pay to get that tomorrow.

Um, in general, though, I, I think, [00:58:00] you know, most labs aren’t very keen on buying large amounts of slop.

Yan: Mm-hmm.

Andrej: Uh, I think sometimes that happens, and then vendors get blocklisted, uh, from, from what I understand, uh, or, or even just kind of, like, not reached out to for a while. Like, kind of put into a box until, like, maybe they’re, they become useful again.

Um, but yeah, like, usually there’s some, uh, there, there, there’s an optimization, I guess, on all, at a lot of these labs. Um, some of the more, I guess, at the most advanced ones, right? Um, so, like, the ones that have been around for a few years now that are thinking about things like profitability more than they are about, like, how do we reach the frontier?

Um, a lot of them actually built internal models that’ll tell them things like, “Okay, we’ve got a specific benchmark that we want to improve a model on. What is the expected economic benefit of making the model better on that benchmark?” Mm-hmm. “So if we make the model better at [00:59:00] doing this thing, how much more, how many more tokens are gonna be consumed because the model’s good at that thing?”

Uh, and then you can actually back out, like, per unit of data, um, what should that data be worth? Um, so, so that, that’s a very common way that it’s actually being done these days.

Yan: Makes sense. And so just to kind of, you know, you mentioned exabyte scale, uh, data provision. Um, I think it’s hard to kind of fully understand the, the size of these numbers.

I’m curious, I guess, what quantitative measure of, of what, what you guys are working on are you kind of most proud of? Or is, is, you know, some measurable, um, data point, and then kind of at the same time, what i- is, is a bar you’re aiming for? Um-

Andrej: Yeah. Uh, there are a few, for sure. Um, number of nodes on the Grass network is a, it was a big one, especially when we first started.

That was, like, the only thing that mattered, right? Um, and it’ll never stop being important, honestly. Um, I think not enough credit is actually given [01:00:00] to the average person that went and downloaded Grass because, you know, it’s very easy to overlook individuals downloading an app for passive, you know, like just to, like, join a network, uh, you know, and passively contribute to it.

Um, but at the end of the day, like, many, many individuals had to do that. Um, and without the first, you know, many that re- referred their friends and, like, kind of got Grass out there, like, this entire infrastructure just would not exist. That, that’s extremely important. Um, numbers that we care about, the number of pages that we’re crawling as well.

Um, but, you know, anybody these days can go and deploy a crawler, and with enough compute, you’re gonna, you know, you can go and advertise that you’re crawling- Mm-hmm … insane numbers of pages, right? Um, the thing is, like, there’s a lot of useless stuff on the internet. Um, and being able to, like, find that, you know, draw the line of what’s useful and, like, how to deploy crawlers as well ends up being more important than any quantitative metrics.

Um, [01:01:00] latency is a big one. Uh, what I’m, what I’m probably, like, really, um, like, really proud of is actually the success rate for the content API that was just released. Um, it’s something that, you know, a lot of vendors will advertise various numbers that are close to 100%, uh, and the Grass content API gets, like, very, very close to 100%.

Uh, 100% meaning no matter what URL on the public web you throw at it, um, it’s gonna be able to find a way to, uh, extract the contents of that URL Um, so that, that’s a metric I’m very happy with. In terms of improvement, I’d say latency is always a big one. Um, at the moment, you know, it does take, you add a certain amount of time, uh, to every web request when you’re routing traffic through millions of IPs all around the world.

It just, you know, your, your constraint ends up being the laws of physics, where how, you know, what’s the speed of light and how [01:02:00] long will it take for this web request to reach a server, um, by routing through, you know, a third party, which is like a, a person’s device where they downloaded Grass. Um, we want that to ultimately eventually be the actual latency of the system.

It isn’t quite there yet, uh, and that’s something we’re actively working on as well. Um, but yeah, I mean, lot, lot, lots of things.

Yan: Mm-hmm.

Andrej: You know, a, a lot of the way that we do things internally is data-driven. Um, uh, and so yeah, we, we were often looking at various numbers and saying, “Okay, how do we improve this?”

Like, what does it take to get into that?

Yan: Got it. And, and so I guess, um, changing it from, um, measures of, of, of data delivery and, and, and, uh, network quality to kind of more financial ones, thinking about, um, I think the estimates are labs are heading towards, uh, 100 billion of, of data spend by the end of the, of by, by 2030.

Curious how much of that you think is, is kind of addressable by what [01:03:00] you guys are building.

Andrej: I mean,

ultimately Grass can vertically in so many different areas. Um, you know- Grass is growing from just being a network that you can use to access, you know, some information on the web, um, to very quickly becoming a massive vendor of pre-training data. It is also now getting into inference data. Uh, beyond all of this, things like preference data, uh, synthetic traces, like all of these fall into that umbrella and, you know, u- ultimately we see all of them as addressable, right?

Like, these are all areas that Grass can expand into. Um, the really cool thing about being able to do this is that if you’re very, very good at something like what Grass is good at, which is accessing most of the world’s information from just a size perspective, um, it actually gives you huge advantages in other areas as well.

So, like, if you’re looking at synthetic data, for instance, [01:04:00] if you have tons of, you know, raw information, um, you’re gonna be best positioned to seed the best synthetic traces, for instance. Um, or if you’re able to search and access any parts of the web, you can actually improve reinforcement learning environments tremendously by giving a model better web access while it’s in the reinforcement learning loop.

Like, this is under-discussed, in my opinion. Um, and then, you know, the, that can go into real world data as well where, you know, right now, as far as Grass’ consumer distribution, there are millions of people that downloaded an app. You know, that app doesn’t just need to be an app that you download and, like, kind of forget about until rewards, right?

There are other things that can be done there, and that can be verticalized into as well. So, um, at the moment, like, I don’t see any area of the data space that is not addressable by Grass. There are certainly areas that we’re not prioritizing by any means just because, you know, we don’t see as much compounding value there, uh, based on where we’re, where [01:05:00] we are today.

Uh, but yeah, I, I’d say, like, the AI data market is, uh, is a market for Grass.

Yan: So recently you guys, uh, released a revenue attestation that was, uh, you know, market was, was very excited about naturally. And, and so it, it also Raised your, your, the typical questions you guys have gotten a bunch in terms of just understanding the, the structure and, and kind of the, the dynamics of everything.

So would love to kinda hear about it from, from yourself on that front.

Andrej: Uh, yeah. Um, Grass has a pretty unique structure, I’d say, um, that, uh, you know, sometimes might actually look kind of, uh, odd or, like, unusual to, um, a lot of customers of Grass. But, uh, it, it, it was structured that way for very important reasons that all have to do with, um, incentive alignment,

Yan: right?

Andrej: So, the attestation that got published actually, um, it, it was an accounting firm that went and looked [01:06:00] at all of Grass’s purchase orders, um, with large AI labs and other types of customers, um, tied those back to cash receipts of getting paid for those purchase orders. And, you know, there’s a tiny delta, right, where some things don’t get paid till 60 days after they’re invoiced, for instance, right?

But, um, in, in general, they, uh, published a, a reconciliation of those two things. And, uh, what a lot of people pointed out or I guess were, uh, relieved by was the fact that this attestation was, uh, in reference to the Grass Foundation and not some, uh, other equity vehicle. So, um, what a lot of people don’t realize is that the Grass Foundation is an ownerless foundation that faces all of Grass’s customers.

Uh, whether they’re buying pre-training data, uh, fine-tuning data, inference data, products, whatever it might be, ultimately it’s the Grass Foundation facing them, and it’s [01:07:00] an ownerless structure, so nobody’s owning it, uh, or no one owns it. Uh, it’s governed by independent directors who need to approve anything that goes through it.

Uh, I work at Wynd Labs, which is a service provider to the Grass Foundation, uh- No, Wynd Labs exists because in the, uh, you know, in the real world you still need to employ people. Um, you still need to, like, set up benefits, um, pay for things like insurance, uh, and there needs to be some equity-based vehicle that’s capable of doing that.

Um, however, for all intents and purposes, it does not own any intellectual property. Um, that was all transferred to the Grass Foundation around the time the token launched. Uh, it does not face any customers, and any interaction between Wynd Labs and the Grass Foundation needs to be, um, it, it needs to be approved by the Grass Foundation’s directors, who have no economic interest in Wynd Labs.

Uh, and you know, obviously, uh, [01:08:00] similarly, like the Wynd Labs team generally, uh, are token holders as well. So, um, like this was a relatively unique structure, um, but it had to happen that way, uh, in order to, uh, ensure incentive alignment.

Yan: Makes sense. And, and historically you guys have reinvested all the revenue back into the business, which is what you’d kind of expect for a company at your stage and, and is, is kind of what, you know, most startups do at, um, at, in their growth stage.

And historically you guys have raised cap- or the foundation has raised capital through, uh, uh, you know, token sales that, that one of which we participated in. Curious kind of what you would do with, with a war chest, with, with growth equity or growth capital rather, um-

Andrej: So like hypothetically?

Yan: Hypothetically.

Andrej: Hypothetically if Grass had like way more capital available?

Yan: Of course, hypothetically. Sure.

Andrej: Um, [01:09:00] so- Um, okay. I, I, I, I was out at dinner a couple months ago with one of the co-founders of, uh, a really exciting AI lab and, uh, I’m not gonna name drop him ’cause I don’t know if he’d want to be name dropped on a podcast.

But, um, I can’t take credit for this. Like, this is something he said, and I thought it was a very memorable quote, uh, that’s probably worth repeating. Um, and it was, you know, uh, he was telling me his response to a lot of VCs that ask him, “Hey, what’s your moat? All these other models are… All these other companies are training models.

Like, where is your moat?” Uh, and he said, “Well, you kinda have to take a step back and define what a moat is.” And he was telling me, like, in his view, and I quickly agreed with this, um, a moat is when your business is capable of reinvesting capital, and for every dollar that you reinvest, [01:10:00] um, it is creating more economic output than any of your competitors.

That is a moat. And that’s, that’s pretty good. That’s

Yan: good. Yeah, yeah. I thought about… I was wondering, it was like, “Yeah, reinvestment. Ah, more than the competitors.”

Andrej: Yeah. Yeah, yeah. Um, so the, the idea is, like, you’re able to compound growth. So as you grow and you reinvest that, um, you can compound that. Uh- I’d say Grass is very uniquely positioned in that it effectively owns all of its infrastructure.

There are tons of agentic tooling startups that deal with web access, and they have to pay for a bunch of third parties to get that web access. There are also, um, third parties that sell tools to help you with web access. So, you know, they’ll build residential proxy networks through various means, but ultimately they don’t own the underlying distribution.

They are paying app developers to put SDKs into things. Um, you, you’ve also got, uh, you know, companies doing all kinds of things with, uh, synthetic data [01:11:00] and compute who need to pay ever-increasing cloud prices because we’re in one of the biggest supply crunches that technology… Actually, the biggest one that technology’s ever seen.

Um, so, you know, the cost of running the same compute workload has been increasing, actually, for the first time ever, right? Um, so Grass is uniquely positioned where it’s built this foundation, right? Where everything is owned in this sort of controlled environment, right? Where the Grass foundation owns intellectual property for the Grass application and has exclusive access to the Grass app and is capable of routing web requests through it.

Um, it ends up giving you tons of pricing power, and then every dollar that you invest into this infrastructure is now being spent on making it better and being able to create more value as opposed to incrementally paying, um, vendors more money. So, so I, I would say, you know, to, now to answer your question, like, what would Grass actually do?

Um, [01:12:00] how would this be, you know, done in practice? The Grass network is, like, an amazing piece of technology. You can build so many verticals, and we’re currently tackling these verticals kind of one by one, right? We, we work on a few things at once, but at any given point, we’re hyper-focused on, like, one of these things, right?

So up until very recently, and, like, continues to be, but, you know, launching the Contents API, uh, building a very specific data set for a specific application. You talked about robotics earlier. We’re doing some work in that space. Um, then also multilingual content, uh, various domains like legal, finance, math, code, whatever it might be.

Um, if Grass had way more capital, I guess, to answer your question, um- All these initiatives would be parallelized, right? It would be kind of like, okay, well, you, now they’re the resources to not do these things, um, one at a time. But like they [01:13:00] could all happen at once. And you can very quickly realize, like you can, by doing all these things at once, you can on a much faster feedback loop determine which ones are worth continuing to invest in and compound, and which ones can be dropped, right?

So, but yeah, that, that’s probably the way I’d look at that.

Yan: And then I guess if, if all goes well and kind of according to your plan and your vision, what do you, what do you think Grass looks like in, uh, you know, 20, 30, or in five years or so?

Andrej: Yeah, I mean, with the rate of… Yeah, you know, what do you think the world looks like in fuckin’ in five or 10 years?

Yan: Right.

Andrej: Uh, with the rate, um, given where everything’s going at the moment, how exponentially, um, like things are progressing. Um, ultimately our, you know, our mission for Grass, um, is for Grass to become The way in which AI accesses information, like very fundamentally, whether it’s an agent accessing the web or some future sophisticated [01:14:00] AI system that needs to access information in a better way, um, in an unbiased way, right?

Um, what that looks like in five years, like who knows. You know, maybe that involves things like shooting satellites into the sky to be able to collect information in certain ways, um, versus today’s version of, okay, using all these I- IPs on the internet as sort of digital satellites- Mm … to view information from their perspective.

Um, whatever the world looks like, artificial intelligence systems are going to need the ability to access information, whether it is to train on and learn from, or to synthesize new data, um- Mm … or whether that becomes all one big continuous learning process. Um, the vision is for Grass to be the medium through which it accesses that information.

Yan: Really enjoyed having you here. Uh, you know, I, I think it’s just the, the in real IRL pods are, [01:15:00] are a different beast, and it was great to have this conversation and really excited about what you guys are building and, and just happy to be kind of following along. Um, so if there’s any way to kind of, yeah, what’s the best way for people to keep up to, keep up to date with everything that you guys have coming?

Andrej: Yeah. Um, Grass has a, the Grass website, grass.io, um, has a blog that regularly gets updated with, uh, new developments, uh, new things we’re working on. Uh, you can go to lcr.net, um, for the new live context retrieval suite that was just released. Um, and anything, I guess, if you, if you follow Grass on X, um, you’ll get pretty much all the updates that are happening from either of those things.

And thanks again for having me on this. This was a lot of fun.

Yan: Yeah, this was a blast. Thank you.