¶1You may not realize this, but there has been an ideological battle brewing in the AI industry for years. We have one group that believes artificial intelligence, potentially the most important invention in human history, should be free and open. And then we have another group who believes it's too dangerous to do that. And so in this video, I'm going to break down both sides arguments and I'm going to tell you what I think and also how China is affecting this entire conversation. This really does matter because artificial intelligence will affect every single person on Earth and whether it's free and open or held by just a few people affects everything.
¶2And if you like me breaking down these complex and oftentimes hairy topics, please like the video, subscribe to the channel. It does help. Thank you in advance. So over the weekend, Jensen Hong, the CEO of Nvidia, published a letter that is very much pro- open-source artificial intelligence, and he also got other major tech companies to co-sign it. Over the next few days, almost every single major tech company CEO came out for this letter.
¶3But there was really only one company that was silent that did not sign the letter and that's Anthropic. And at the same time that Jensen and all of these other major tech companies were coming out pro- opensource artificial intelligence, Anthropic was literally on the news campaigning against open-source talking about how dangerous AI can be in the wrong hands. All right, so first what is open source? Open source means that a piece of software is publicly available for anybody to use, modify, inspect, and share. An example of open- source is Android, one of the most popular pieces of software in the entire world.
¶4And if you use a Google Pixel phone, if you use a Samsung Galaxy, or really basically any phone except for an iPhone, you're probably using some flavor of Android. And that's the power of open-source. Now iOS by Apple which is closed source has a strong competitor in the market keeping them honest. But why is open source even important? I want to point to something that happened in the early to mid90s.
¶5The internet was just being born and at the time there were a few companies that really were the internet. It was AOL, it was Compuserve, it was Netscape with their browser and they were all closed source companies. So if you wanted to access the internet, you had to go to them. There was no internet as we know it today. And then two technologies were built that open-sourced everything.
¶6We had the Mozilla Foundation building a browser that was completely open- source. And then we had Apache building the backend to power the internet. They built a little thing called HTTP, which is one of the foundational technologies powering the internet today. And with that, we went from having to go to AOL or go to Netscape and basically abide by their rules of what the internet should and should not be to everybody being able to develop their own version of the internet. And all of the value of the internet got acred to the internet rather than just a handful or less of companies.
¶7And so now you can kind of see the analogy here. If we fast forward to today, we have some of the biggest AI companies in the world are closed source. Meaning, if you want intelligence, you have to go to them. And if they don't like what you're doing, if they change their terms of service, that's it. You're just cut off.
¶8Whereas, if we have an open-source alternative, that wouldn't happen. More competition. And I'm going to get more into the benefits of open-source shortly. And so, how does open source actually work? Are they different from closed source?
¶9Well, the answer is no. There's really no difference between an open- source AI model and a closed source AI model. The only difference is the business model. For open- source AI, it's released publicly. You can understand how a company built it.
¶10You can run it yourself if you want. But otherwise, there's no difference. how they're built, how they're trained, how they're fine-tuned, how they're served, it's all pretty much the same regardless of if it's closed source or open source. The difference with open source is once a company has developed the open- source model, they give it away for free. And you might be asking yourself, well, how does that make sense?
¶11Why would they do that? Well, as I said, the main difference between open source and closed source is that the business model is different. The raw intelligence of the model is no longer what captures the value. It's all of the services above and below the intelligence layer. When I say below, I'm talking about data centers.
¶12If you're powering one of these open-source models, that's your business. You can sell tokens. And when I say above the intelligence, I mean applications, software developers who build really cool things using artificial intelligence, including the model developers themselves. Now, you may have heard that open-source is not quite as good as closed source artificial intelligence. And although that is true, there is no fundamental law of physics that says open source can't be as good as closed source AI.
¶13And in fact, open source has gotten really close to date. We have a model from Moonshot AI, which is a Chinese company called Kimmy K3. And on many measures, it is as good as the closed source counterparts. But generally, closed source models like Claude and Chad GPT are better for now. And as you could tell, I'm a big fan of open ecosystems.
¶14And that's actually why I love the sponsor of today's video, Zapier, because they basically make any product work with any other product really well. Zapier allows you to build incredible automations right out of the box. They have over 9,000 integrations and then pair that with any model you want, closed source and more importantly open-source to create incredible automations for your business. They support things like Slack and Gmail and Google Sheets and Aana. All of the different tools that you already use.
¶15You can put them together in incredible ways and it plugs right into the agents you already use. Whether it's Chat GPT or Claude or Open Code. I've been using them for well over 10 years. I am a big fan. So join some of the biggest companies in the world using Zapier, including Nvidia, including Meta, Cursor, Samsung, and so many more.
¶16Go check them out. I'm going to drop a link down below. Click that link and let them know I sent you. So, this is the artificial analysis leaderboard. It basically shows which models are the most intelligent overall.
¶17And what we see is the number one, two, and three spots are given to Claude and Chachi BT, but Kimmy K3 is coming in right behind them. And that's important. So although overall closed source models today are generally better than their open source counterparts, open-source is not far behind. But why is that? Why is open-source even behind at all?
¶18If with open-source so many more people can use it, can play with it, can modify it, wouldn't you think it's better overall? Well, there's something very distinct about the creation of a new AI model that makes the open-source business model for AI much more difficult than any other industry. Now, there are two main reasons why closed source AI is better than open-source as of today. Number one is that the original innovations and scaling up of those innovations were all done by closed source companies. Specifically, Chat GPT came out first and they were already a closed source company.
¶19Anthropic wasn't far behind them. But the point is these two closed source companies developed and then scaled up the technology. And because they were able to do that, they had this momentum going for them. And then number two, because they were in the lead, they made so much money from selling intelligence. And with all of that revenue, they reinvested it into building data centers, scaling up their teams, scaling up their marketing.
¶20And what that allowed them to do is build better models after that. every subsequent model learned from the previous one and benefited from the increase in scale afforded to them by all of that revenue. Now, at the same time, open-source AI came about almost as an accident. The llama model being developed by Meta leaked. It got out and people realized, oh, I can just download this thing and put it on my computer.
¶21And so they leaned into it, they being meta, and decided, hey, we're building open-source models now. But the open-source business model had not been proven. Whereas the closed source business model was skyrocketing. And so you had these two companies, Anthropic and Open AAI, sucking up all of the oxygen in the room. And when I say oxygen, I mean revenue, compute, investment, researchers, everything was going to these two companies.
¶22And so opensource artificial intelligence hadn't been proven and just could not accelerate as fast as the closed source model companies could. Also, in the United States, especially, the open-source business model is very difficult to justify. We're still figuring out how that works. And there's a big reason for that. If you're a startup and you're developing an open-source model, it takes so much investment to do so.
¶23And then if you just give away that model for free, what's your business model? Well, you either need to build software and applications on top of that intelligence or you need to have incredible infrastructure data centers to serve the model efficiently and at scale. Both of those things take additional capital. So not only do you have to have this massive investment to build the initial model itself, you also have to have more investment to build applications on top of it or to build data centers to serve it. And so if you give away the model for free, another company only has to do one of those things really well.
¶24They can build apps on top of it or they can build data centers and serve it. and they don't have that initial investment of having to build the model. Now that's not to say that open-source can't be done well. There are a lot of examples as I mentioned Android being one of them, Apache, Mosilla. These are big companies that have developed products and services on top of the core thing that they give away for free.
¶25But there is again the major difference between all of those examples and AI which is AI takes orders of magnitude higher investment to actually build it. Now there is one company that can do it and has been doing it and that's Nvidia. Nvidia wins no matter what. Nvidia committed 20 billion dollar into building out open-source AI models. And again, they're able to do that.
¶26They are printing money by selling GPUs. So, investing $20 billion into making the entire ecosystem of artificial intelligence bigger, better, more efficient, it all helps Nvidia. Closed source models help Nvidia. They're all built on top of Nvidia's GPUs as well. And opensource is really good for Nvidia.
¶27That's why they signed the letter. I mean, they not only signed it because it's economically good for them, but also they probably do truly believe open- source is the way. And I'm going to explain all of this shortly. Now, you've probably heard China being talked about a ton with respect to open-source artificial intelligence. And that is because they are putting out the best open-source AI in the world.
¶28They have a strong lead on open-source AI right now, but they're constrained by one thing, and that's compute. They cannot build the same quality of chips that the United States can with Nvidia, with AMD, with Cerebrus, with Grock, all of these different companies. They cannot compete with yet. But they have all of the other ingredients necessary to win the AI race. They have tons of AI researchers.
¶29They have orders of magnitude more electricity generation than the United States. They have engineering talent, but I'm going to come back to China in just a moment. Now, I mentioned that Nvidia wins when open-source wins. So, let me explain why. But let me also tell you who else wins when open-source AI wins.
¶30And if you want to stay uptodate on the latest in AI, check out our newsletter, forwardfuture.com, link down below. So let's look at all of the layers of the AI stack. On the bottom we have chips. These are companies like Nvidia and AMD. They are building GPUs.
¶31They're building CPUs. They're building the things necessary to actually train and power the models. Then we have energy and data centers. We can think of some of the hyperscalers like Google cloud, Azure by Microsoft and AWS. Then we have the model providers.
¶32So those are both open-source and closed source companies like OpenAI and Enthropic. Next we have the software infrastructure layer and that's both tooling to build the models as well as tooling to help the application layer build on top of the models. And then finally we have applications and an application is like chat GPT that's not the model that's the application but we also have things like lovable and replet and cursor. All of these are at the application layer. Now, let's think about what happens when open-source wins.
¶33And then we're going to go back and look at each layer to see who wins. And so, the first thing is we have more options, which means increased competition. That also leads to more companies in the space, more options for businesses and consumers to choose from, and thus lower prices. When we have more options, we have more competition, we have lower prices. That's good for us, the consumer.
¶34The next thing you get when open source wins is increased efficiency. And that's because when you have so many more people looking at the code, at the models, at how they're built, what happens when they're used, how to make them work especially well with certain chips, then you're able to ek out so much more efficiency out of these models. And what happens when you have higher efficiency? You typically have lower prices. And again, that's good for us, the consumers of this intelligence.
¶35And then finally, when you have lower cost, more people using it, higher efficiency, you typically have higher quality as well, better AI. And so now it's time to talk about Javon's paradox. As the efficiency of a resource increases, the cost typically decreases as well. And you might think, okay, well, if the cost is decreasing, people are going to be paying less for it, right? Well, that's the paradox.
¶36It turns out as the cost of a resource decreases, people businesses actually use it more overall and actually pay more overall. And so that's what we're looking at here. As the cost goes down and efficiency goes up along this line, what we're seeing is that the demand along this line actually increases. So again, as intelligence decreases in cost, the overall usage will skyrocket. Now, let's go back to the AI stack.
¶37How do all of these dynamics affect each part of the stack? Well, first of all, if more people are using AI, more tokens are being consumed. Yes, they are less expensive per token, but so many more are being used, there needs to be so many more chips to actually serve them. And of course, who wins? The chip providers, AMD and Nvidia.
¶38Now, let's think about the energy and data centers. Well, if you need a lot more chips, you probably need many more data centers as well. And how do you power the data centers? Well, it's by energy. So, you need more energy.
¶39So, guess what? These companies win also. Again, two winners. This is good so far. Now, I'm going to come back to models.
¶40And you probably already know where I'm going with this, but let's skip over to the application layer. When developers and startups have more cheaper AI to use, of course, their margins go up. So, they're super happy. They can build more. They can deliver more value to their customers and for a cheaper price.
¶41Thus, their margins are higher. So, they win as well. ChatGpt, Replet, Cursor, Lovable, they all win. Now, same with software infrastructure. If you have so many more applications being built, you need more infrastructure to power them, to observe them, to analyze what's going on when something goes wrong.
¶42This is all good for every single layer except one, the model layer. If you are a closed source model provider, your margins are being compressed significantly by the fact that there are many more competitors on the market and they typically are less expensive than you. So, if your entire business model is selling tokens at high margins and I'm a business owner and I'm looking at Anthropics tokens at $50 per million and then I see an open-source version that is 95% as good at $5 per million, it's going to be very difficult to justify the $50 per million tokens. And so, if your entire business model is creating the model opensource is a real threat. So, OpenAI and Enthropic won't necessarily lose in this situation, but they'll have many more challenges ahead of them because of open-source, and they're going to need to change their business model from just having massive margins on selling tokens.
¶43But here's kind of a contrarian take. I actually think Chachi PT and Claude Banthropic are going to be just fine. Now, they already have the best models on the planet, but if they face increasing competition at the model layer, they have to rely on other parts of their business. And it turns out the other parts of their business are super strong. And so, let's think about what happens if the model layer isn't where the revenue is made anymore.
¶44Well, OpenAI and Anthropic are going to need to move up the stack. And what that means is they're going to have to build applications on top of intelligence. and they're already doing a great job. If you've ever tried to use an open-source model, whether you're running it yourself at home or whether you're buying inference from one of the hyperscalers, you know, it's a far worse user experience than just going to chat GPT or just going to Claude. And so, even though these are closed source companies, they can start to make money with their harness.
¶45They can start to charge for keeping track and managing your AI's memory. They can charge for the vastly superior user experience that they already have. And then of course they can build other products and services on top. And here's the part that I'm still quite confused about. Who's going to make the models in this world?
¶46If these companies profit margins are being squeezed at the model layer, what incentive do they have to build the models in the first place? Why don't they just use open-source models? And then for those open source model companies, who's building them? Because ultimately you need so much money to build one of these models that unless you're Nvidia and no matter if it's open source or closed source models, you win. It's really hard to justify spending billions and billions of dollars building a model.
¶47And then your competitor using an open- source model can compete in a hyperfocused way on these things directly and they're not having to worry about building the best model also. And so I'm confused about that. That's why I keep saying the business model of open-source seems very broken at least in the United States. And now for possibly the most important part of this conversation and the only argument that a company like Anthropic stands on when they rally against open-source AI and that is AI safety. Is open-source AI safe?
¶48If anybody can use it, if anybody can download it, is it safe to allow that? Now, here's the thing. I actually really do believe that Anthropic thinks open-source AI is not safe. Now, it also protects their business model because if open-source AI gets banned, then you're only getting your intelligence from those closed source companies. It becomes much more difficult as a startup to have to jump through all of these regulatory hoops to try to develop your own model.
¶49And since Anthropic has so much money, they already have the best models in the world. They already have the best researchers in the world. Of course, they want to protect their business model. But again, and maybe this is me being naive, I actually do believe they believe open source AI is bad. But many other people, including government officials, both former and current, believe Anthropic is trying to accomplish regulatory capture, which means they're going to make it so difficult for anybody else to compete in AI that they default just win.
¶50But I'm not going to talk about that today. I'm not going to try to convince you one way or the other. What I do want to talk about is whether I believe open- source AI is safe. And I also want to take some time to talk about why Anthropic believes it's unsafe. Now, I think there are four main reasons why someone may argue that open-source AI is in fact unsafe.
¶51So, here are the main reasons. Number one, anyone can remove the safeguards. Bad actors, rogue nations, they can all download this AI and remove any safety that is built into the model because if you have the model, you could do whatever you want with it. Now just picture this. If you go to Claude, if you go to Chad GPT and you ask it something dangerous or illegal, it will refuse.
¶52It won't tell you. Now, if you have an open- source model that has those same safeguards built in, it will also refuse. But since you have the model, you can train it to stop refusing. And so that's an argument against the safety of open source AI. Also, access can't be taken back once the model's out there.
¶53It's like something on the internet. Once it's on the internet, it's never being deleted. And the more you try to delete it, the faster it spreads. Actually, it also kind of more on the point of this first point lowers the barrier to misuse. If a bad actor has to go to anthropic or has to go to open AAI to get the intelligence to go do something bad, whether that's hacking somebody's system or asking it for boweapon recipes, whatever that thing is.
¶54If you have to go to one of these companies, the chances that you're going to be able to get the information out of the company is much lower than if there's an open-source model that you can do whatever you want with. And then finally, the responsibility is unclear. If somebody releases an open- source model, all these bad actors download it and do something bad with them. Who is ultimately responsible? Is it the model creator?
¶55Is it the inference provider, the hyperscaler? Is it the application layer? Maybe maybe it's the ISP who even allowed the internet connection to happen. So the responsibility is actually I do believe this less clear and we do need to figure this out. So what are the arguments for open source actually being more safe which to be clear is what I believe.
¶56Number one, there are more eyeballs on it. More people looking at the models, more people making sure there's nothing wrong with them. It's all out in the open. It's all very transparent. And that hardens these models more so than if they are just held by a single company.
¶57And we've seen this open-source software on average tends to be more secure than closed source software. This isn't a new argument, by the way. The whole open source versus closed source safety argument has been around since open- source has been a thing, since software has been a thing. More people can find weaknesses, biases, hidden behavior, and security flaws when it's all out in the open. Next, there's more transparency.
¶58This is super important. Less just trust me, bro. Right? So if you have anthropic completely closed source, not transparent about any of this stuff and they're saying, "Hey, this thing happened and we're going to fix it or we have all the security measures in place to make sure our AI is safe and secure." We just have to trust them and they are always going to be a subset of the total AI researcher population. So it's never going to be as safe as if everybody could look at it.
¶59Also, the safety tools improve more quickly. Again, it's the law of numbers. Anthropic has a subset of the total amount of people in artificial intelligence helping build these tools. And if everybody can contribute and collaborate and coordinate on an open-source AI model, on open-source tooling, this is good. It becomes more hardened more quickly.
¶60And for the big one, and this is what I am most concerned about with artificial intelligence generally, is the concentration of power. And we're already seeing that the vast majority of usage of AI comes from one of two companies, OpenAI and Anthropic. That gives them significant power over not only the AI industry, not only the GDP of the United States, but also the global economy. It gives them so much power. And that scares me because they get to decide the future of the most important technology ever created.
¶61And I don't want that. I want it to be for everybody. And so opensource allows that. It allows everybody to benefit and to contribute to artificial intelligence. And then last, local use, more privacy.
¶62If you're into the local AI thing, wonderful. It's fun. It's awesome. And whether you're a business or a consumer, you get a lot more privacy. The data never leaves your computer, and that's a good thing, especially if you are in really secure sectors of the economy.
¶63Now, this brings us to something that was just announced today. Nvidia on the heels of signing the pro opensource letter just announced they are introducing the open secure AI alliance. And what that means is all of these companies here, you can see a bunch of them. Here's OpenCloud, Palunteer, Microsoft, Door Dash, Cloudflare, Cisco, a bunch of great companies here. SpaceX, all of these companies are going to contribute to the safety of artificial intelligence.
¶64And of course, that makes everything more secure. All of these companies collaborating rather than working in isolation will make the world a safer place. And in fact, Jensen Wong, the CEO of Nvidia and really his second Twitter post ever, gave a prime example of open source coming to the rescue when closed source failed. Here it is. During the hugging face incident, closed AI blocked essential forensics.
¶65An openweight frontier model helped contain the intrusion. And so if you're not familiar with what he's talking about, an open AI model, the next generation model, while being tested, actually escaped containment and hacked Hugging Face. And so while Hugging Face was being attacked, they wanted to use an OpenAI model to diagnose what had happened. But because that diagnosis looked too much like a cyber attack because cyber defense and cyber attack are essentially identical because it looked too much like that the open AI models refused to do it. So Hugging Face used opensource they used open source to try to diagnose what had happened and they were successful in that and so that's what he's talking about here and that is why they created the open secure alliance.
¶66Now, this is Andrew Ning, one of the leading voices in artificial intelligence, who said, "Good move. It's well written, and we need open models and harnesses for defense. Let's stop believing the PR that closed models are safer. That's just regulatory capture." So, let's go through each one of these arguments that open source AI is unsafe and let's debunk them. So, first, anyone can remove safeguards.
¶67Yes, that's true. But safeguards in closed source models can also be bypassed. AI is non-deterministic and by nature are not perfect. And so there's always going to be a way to convince a model to give you information that it shouldn't to do something that it shouldn't. So whether we're talking about jailbreaks or stolen access or leaked model weights which are more likely to occur in a company that is closed because it has far fewer people.
¶68So we'll remove that. Next access cannot be taken back again. technically true, but same with closed source models. Once the harm is done with a closed source model, it can't be undone. The outputs also can be copied.
¶69That's called distillation, which I'm going to get to in a minute. That can be used to train future open- source models. Plus, the techniques to actually build the closed source models can just be used for open source. So, it's not like open source is just going to go away. Lowers the barrier to misuse.
¶70No, this is just not true. A lot of let's say sensitive or illegal data can already be found on the open internet. How do you think these closed source models were trained? There's existing software, closed source APIs. And so again, it's very much anthropic just saying, "No, no, trust me, bro." And then last, the responsibility is unclear.
¶71So I do think we need stronger definitions of who is responsible when open- source is misused. But that responsibility can be spread to a number of parties which is actually a good thing. Whether you're talking about the model creator, the inference provider, the application developer, this is all good. And thus the arguments against open-source AI safety fall short. So now hopefully you agree that open-source AI is beneficial for everybody and actually quite safe.
¶72And the next thing I want to talk about is China. Why is everybody talking about Chinese open-source AI? Well, they're making the best open-source models on the planet. And you might point back to earlier in the video and say, "Hey, Matt, you made a very strong point that the economics of open-source AI are kind of flawed. It's hard to make a business if you're just a model provider and giving away those models for free." And I don't have a counter to that.
¶73It's true. But China has incentive to give away these models for free. First, how are they even doing it? How are they able to create such great models even though they are missing one of the main ingredients for developing great models? Well, they have incredible energy infrastructure, right?
¶74They have all the electricity they could ever need. They have an incredible researcher base. Nvidia's Jensen Huang said, "Worldwide, 50% of all AI researchers are Chinese, but the thing that they're missing are chips. They cannot get their hands on as many or as powerful chips as US companies can. And so that is why Chinese AI companies are making open- source models because they have to.
¶75They're not doing it out of the goodness of their heart or because they believe in open source. Although, you know, maybe they do, but the real reason they're doing it is because this is a triedand-true strategy. If you do not have the best of something, you give it away for free or you give it away incredibly inexpensively to increase the competition and decrease the profit margins of the current leaders in the space. And because China, the Chinese government actually owns a piece of all of these Chinese AI labs, they can incentivize those Chinese labs to give it away for free. And then the question becomes, why does China want to give it away for free?
¶76Well, how does that benefit them? And I actually want to point to something that David Freedberg said on the All-In podcast this past week because I thought it was so preient and so accurate. So, I'm going to kind of summarize what he said. Now, for the past few decades, the United States has been really good at producing ideas. And these ideas we've monetized heavily.
¶77Everything from movies to software, music, company IP, all of these things we've been able to produce, but they are essentially bits. They are not physical goods. And the United States has basically offloaded our manufacturing capacity to China. And thus, China has gotten extremely good and extremely efficient at being able to produce physical goods. And so now we have this contrast.
¶78The United States really good at ideas, really good at the services economy, and then we have China really good at the manufacturing economy. And so if we game theory this out, China wants to win. China can then handicap our ability to monetize ideas if ideas are free. And where do ideas come from? Well, they come from intelligence.
¶79So, imagine this. China's sitting there. They're thinking, "We have all the energy in the world. We have all the manufacturing capacity in the world. We can produce physical goods better than anybody else on Earth.
¶80But the thing that we don't really do is export ideas. And so, let's make ideas. Let's produce intelligence, make it free, and let anybody produce ideas." That puts competitive pressure on the United States because then our main export of ideas decreases in value substantially and that puts us, the United States, in a very precarious situation. So that's one way to think about why China is willing to invest so much money and just give away the output of that investment for free. Is it a bad thing that China is giving away these incredible models for the world to use for free?
¶81I mean, we don't have to use Chinese companies to power the models. We can download them. We being the US, we can download them. We can run it ourselves. We can remove any guard rails that China might put on it.
¶82We can customize it for our exact needs. That sounds like a really good thing. And it is. I don't have an argument against it. It's awesome.
¶83I am very thankful for these Chinese AI companies that are putting in an incredible amount of work in developing these AI models that are so good. and then just giving them away. I love it. But there is that long-term kind of geopolitical risk there. And so the other risk and I think this is actually a shorter term risk is if companies in the United States decide to adopt and use Chinese AI models and that becomes the standard.
¶84Chinese AI becomes the standard. There's this notion that the model chip co-design when you really design the model and the chip tightly coupled together, they'll both be better. That if we're built on Chinese AI, we're going to start being dependent on Chinese chips. Now, it does require the Chinese to actually build chips that are really good, but I kind of have no doubt that they're going to get there. And if we're using their models, we will eventually use their chips, and that puts us at severe risk.
¶85So, if you've been hearing about Chinese AI, you've probably also been hearing about distillation. What is distillation? Well, it's a kind of scary sounding term that's actually quite simple. It generally means one AI model teaching a smaller AI model to be more capable. So, that's it.
¶86The student asks the teacher questions, the teacher answers those questions, and because of that back and forth, the student gets smarter. And that's essentially how distillation with AI works. And this isn't a new practice. Distillation has been happening for a long time. And it's not illegal.
¶87It's actually a really good thing because let's say you're an AI model company and you develop this incredible model, but it's huge and it's very expensive. You can distill that really big model into a much smaller model that is more efficient, faster, and cheaper. And that is good to have more model options because certain use cases need faster models and cheaper models. So distillation is very much legal. There have been a number of US government officials that have come out in the past few weeks saying yes, China is distilling our frontier models from anthropic and open AI.
¶88Here's Scott Bessant. We support open source AI, but open source is not open season on American IP. when PRC firms conduct covert industrialcale dissolation attacks that cross the line into IP theft. And he's not the only one. There has been a number of government officials coming out saying the same thing.
¶89China is stealing our IP. And Anthropic even put out a report a few months ago saying the same thing. And I'm not arguing. This is happening. China is actually distillation attacking our frontier labs.
¶90But it's more nuanced than that. And there are ways to stop distillation. I mean, at the scale that this distillation is happening against anthropic, they really should be able to stop it. And if they can't, that's on them. They should do things like KYC, which is know your customer.
¶91This is a triedand-true practice in the banking industry to prevent financial fraud and money laundering. So, it just means collecting information from your customers. Now, I want to reference the All-In podcast one more time. On the most recent version of the All-In podcast, they gave an analogy where distillation is kind of like when an auto company buys their competitor's car, disassembles it, looks what they did, and comes up with really cool ideas based on what they saw, and then applies those ideas to their own product. And I actually think that's an inaccurate analogy.
¶92What would be a more apt analogy is if one auto manufacturer bought their competitor's car, extracted different parts from that car, and then used those parts to build their car. That would be illegal. The former is very much legal and very common place in many industries. Auto manufacturers buy their competitors cars to get ideas all the time, and it's not illegal. But actually taking parts from the car and using those parts in the manufacturing process, that is illegal.
¶93And that's essentially what distillation is. One company is asking their competitor's model for parts from their model and then using those exact parts to build their model. Now, here's the fun thing to think about. What's the difference between a Chinese AI lab distilling from Anthropics models and Anthropic distilling from the open internet? Because that's what a lot of people think essentially happened.
¶94Anthropic scraped the entire internet. You know, the internet that you and me created for free and put out there for free. They scraped it and used that to create Claude. And now they're saying, "Hey, that's cool." But when Chinese AI labs distill from our models, that's not cool. And I'm going to actually have a very contrarian take on this one.
¶95I actually agree with Anthropic. It really all comes down to the laws. If the data that they took from the internet was truly open and free to use, they should be able to use it. Now, Anthropic just settled a billion plus dollar fine because they scraped books that they shouldn't have. Now, if Anthropics terms of service, say distillation from our model outputs are against our terms, then the companies who do that should be penalized and it should be illegal because those are their terms.
¶96Now, I still think Anthropic is being incredibly contradictory because they're saying, "Hey, even though all of this information was on the internet for free, we shouldn't have to abide by whether the author of that data says they don't want us to scrape it or not." And that's a whole different discussion I'll save for another time. And so, to be clear, distillation is very much legal. It might be against Anthropic's terms of service, but the practice of distillation is not illegal or bad. In fact, it's very good. And then the final question becomes, should we ban Chinese AI?
¶97And in short, no, I don't think so. Right now, it really only benefits the United States. We have competitive pressure on our companies to do better, lower prices, not capture all the value in the market. It creates more players. It allows for more people to use AI more cheaply, which is phenomenal.
¶98That is what I care about most. And so, the answer is no. Now, there is a trade-off. There is additional risk, which I talked about earlier. If we ban Chinese open-source models, China will continue to develop them and the rest of the world might adopt them.
¶99Because if we ban these Chinese AI models, the closed source models in the United States will have so much control over the industry, they will set the prices much higher than what the rest of the world will have available to them through Chinese open-source models, and we're going to be living on this little island. And whenever you try to control a market manually, it never really works out. Just let the market play out. And more importantly, we should bolster the US's open-source AI industry. Artificial intelligence and open-source is basic research, and we should fund it and incentivize it as such.
¶100So, open-source is good. It's important. It allows more people to use artificial intelligence. It allows them to use it more often and it disperses the most important technology of all time into more hands and that's what I care about. That's our mission at Forward Future.
¶101I want everybody to understand this stuff. I want everybody to use it and be excited about it. So, Anthropic is still the last hold out on that pro-open source letter and they do truly believe open source is dangerous and simultaneously they're wrong. And I also think a lot of people are conflating Anthropic's points of they simultaneously believe open source is not safe and they simultaneously believe distillation attacks are bad and should be prevented. And a lot of people are saying, well, they're rallying against open source as a way to prevent distillation.
¶102I don't actually think that's the case. These are two separate issues. But overall, I still am very disappointed in Enthropic's position. I really do think they should come around and join the awesome opensource initiatives. So, I had just finished recording and then I see this.
¶103Anthropic just responded to all of this discussion about open-source and their stance against it. Let me show you what it says. So, this is a short letter from Dario Amade, the CEO of Anthropic. And in it, he clearly states that they are not against open source and I call BS. So, here are the important parts.
¶104Anthropic has never advocated for a ban on open weights models. However, they don't have to explicitly advocate for a ban when they so frequently talk about how open- source models are an increased security risk. They don't actually need to go lobby against open- source, but every time they go on any news network and talk about how dangerous open source models are, it's effectively doing the same thing. And even the parts that I do agree with have kind of a subtext to them that I do not agree with. So open weights models that don't have dangerous capabilities are a public good.
¶105They don't cost anything besides the compute needed to run them and they provide value to businesses, developers, and researchers. Sounds pretty benign, right? Sounds like something I could agree with, right? Well, here's the nuance that don't have dangerous capabilities. And then throughout this entire letter, they say that it's basically impossible to have an open weights model that doesn't have dangerous capabilities unless it's a really bad AI model.
¶106That's what they're saying. So they're basically talking out of both sides of their mouth here. They do address the protectionism that they've been accused of. Protectionist bans would not address my most serious national security concerns. However, they go on to say banning the use of these models by US businesses does nothing to address this risk, but it would protect US AI companies from competition, but that has never been my goal.
¶107Okay, so Daario lays out his primary concerns with both open-source openweights models and distillation. My primary concern is the risk that authoritarian governments, not solely the CCP, Chinese government, would build AI models that are more powerful than those built by the US and use them to achieve permanent military superiority or perpetrate incredibly deepression of their own people. Okay, I can agree with that. I want the US to win. I'm a US citizen.
¶108And he even goes on to say it's irrelevant whether these models are released with open weights, which is true. The whole open weights close weights discussion does not matter. If the main fear is that an authoritarian government will have more powerful AI than a democratic government, open weights, close weights, it doesn't matter. Then he goes on to say his secondary concern is the risk that powerful AI models may be misused to carry out cyber attacks or biological attacks. But here is where I start to really disagree.
¶109Open weights models potentially present a higher risk than closed models because it is very difficult to apply guardrails to them or monitor their usage. And once weights are released, they cannot be withdrawn. Now, here's the thing. To run a really powerful AI model, you need a lot of compute. And rather than saying it's very difficult to apply guardrails or to monitor, what he's really saying is it's not within our control anymore.
¶110And that makes me nervous because ultimately it takes a lot of compute to run a really powerful big model. And we're talking about AWS level compute, Google cloud level compute. And at that point, if those companies are serving open- source models and allowing them to be used for cyber attacks, they're responsible, right? It's not like some individual hacker can download a massively capable model, put it on their little desktop computer, and run a model that's as powerful than what a company with a 100,000 GB300's can run, the most powerful GPUs on the planet. It's just not possible.
¶111So, it's not really about just having the model. It's about having the model, have it be incredibly capable, and actually being able to run it. That's just as important. So, I don't agree open- source models do not present a higher risk. Now, he goes on to talk about the biological weapon risk.
¶112So, it is true, especially for asking about biological weapons, you don't really need a big powerful model to do so. However, these models were trained on publicly available data for the most part, including whatever was necessary to have the recipe for a biological weapon. So if somebody really wanted to get it, they could. And once again, if they're using the model for testing and iterating on this biological weapon, they're going to need a lot of compute. They're going to need the biggest best model in the world.
¶113So this whole argument that somehow if you can just easily remove the guardrails, you get everything that a frontier closed source model has with just no guardrails is just false. You either have a not as capable model that you can run locally or you have a highly capable model in which you have to pay a company to serve it to you because it's so big and so powerful. It requires so much compute. And one last thing about the whole biological weapon argument, it's not enough just to have the information and it's certainly not enough to have massive compute to be able to power a model to help with the creation of biological weapons. you still need to acquire all of the ingredients.
¶114You need to have the facilities. You need to have the actual human researchers to help progress discovery. And so, again, it's not as simple as just, well, the model told me what to do. I'm going to go do it. Now, one thing I do agree with is he says we should crack down on industrialcale distillation operations.
¶115I don't know exactly how severe these distillation attacks have been, but if it's against Anthropic's terms of service, we should apply the law just like we would any other law. But he even admits it here. Just being able to distill the data does not bring the distillation attackers model to the frontier. It brings them closer and it allows them to do so in a much cheaper way than it would if they were actually baking the model from scratch. But ultimately, there's still going to be a gap between the absolute frontier and whatever model was distilled.
¶116But if it's against the terms of service, if it's against United States law, we should apply the law. And then he says, "A blanket ban on open weights models is neither the correct remedy nor something we have called for." I hope he's not saying a blanket ban as though some kind of ban on open weights would be applicable and relevant in this case. And then again, here's where kind of the gray area comes in. All sufficiently capable models, open and closed, should go through mandatory safety testing. I don't disagree with this, but the devil's in the details.
¶117How much safety testing is required? How much money does it cost to do that safety testing? Because Anthropic already has the best model in the world. They already have a ton of money in their bank account. They already have lawyers and researchers and basically everything they need to be able to achieve this safety testing pretty easily.
¶118Now, think of that startup. The startup doesn't have any of that. They're competing on a different plane with limited resources. And so if you make the barrier to entry into model creation, especially frontier model creation that high, that is effectively regulatory capture. And so that's really where I want you to pay attention in this letter.
¶119He does say safety testing. I do agree, but it needs to be easily achieved by anybody who wants to compete in AI model creation. And so then he says that brings me to the open letter. I agree with much of it. Open weights expand access to the AI economy, exactly what we talked about.
¶120They strengthen competition, at least for some use cases. I don't know what use cases they don't, but fine. And they give customers greater control. These are all really good things, and he's right about each and every one of them. These are the benefits of open source.
¶121Here's where he disagrees. I disagree with the letter's assertions that open weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. This is where I disagree. I do think that broad access to artificial intelligence does allow more people to look at the models, build tooling more quickly, all of which will make us safer. And then broad access to capabilities does help defenders and attackers equally.
¶122If you just think about kind of the intelligence leveling the playing field, but there are other parts of the playing field that make it uneven and benefit open source benefiting the defenders. One, hopefully there are many more defenders than there are attackers. And so you have more good guys looking at the models, building new tooling to help cyber defense. We've had multiple major companies come out with open-source cyber defense capability models in the last like two weeks. And then again, it takes a lot of capital to run these models.
¶123And there is so much more capital available to the good guys than there are to the bad guys. That's just a fact. Now, China is not a bad guy, but they're our adversary. And they're an actor that does have the same resources as the United States. So, he's pretty clear.
¶124Although I don't agree with his arguments, he's saying a ban on open source is not what he's asking for. This is the most important conversation happening in AI today. And if you want to go even deeper, look at this video right