This is going to be a pretty long answer, but just to preface it: if you really disagree with this, or you feel strongly that you have data that proves me wrong, let’s discuss it properly. We can do a public roundtable with you, your research team, engineering guys, whoever else you think would add value to the discussion. My community is almost at 2 million people monthly now, and I think it would be a great way to show people how you guys are thinking about this, why you’re so confident in the approach, and where you think my analysis is wrong. The point would not be "debate" (I think that's corny af), but to have a good-faith discussion about our differences in opinion/technical thesis in a way that everyone can walk out with a better understanding of things. Happy to also talk privately if that's better. Or just argue with code, as they say. We'll define a task, define a budget, we build it in our own ways, and let’s compare the results. Now onto the answers. TLDR: training custom models would be wasteful for law firms and most challenges around sensitive agentic deployments are not a model layer problem. Trying to solve them in the model layer creates major security issues that law firms will not want to deal with. 1-- when you talk about helping law firms build their own models/building on their private data, you're likely suggesting post-training, because obviously they will not have the amount of money and data required to train a completely new intelligent model from scratch. Especially one that’s competitive against the really good open models out there currently. This is not useful to them because stacks of research have already shown us that most of the knowledge of an LLM comes from the pre-training stage, not from the post-training. Post-training is essentially just a way to change how the model samples from its latent space, but the way concepts are organized and information is actually learned is mostly coming from the pre-training stage. This is a problem for law firms because they don’t want their own custom knowledge, their client data, their own proprietary knowledge, to be confused with generic off-the-shelf internet insights (since you're not changing the pre-training). The whole point of training a model on your custom data would be that it’s much more precise, much more grounded in your data (including private data that isn't available in generic models). If you’re only post-training and not pre-training, most of your knowledge is still coming from that generic base. You can actually confirm this for yourself with your tenant models. Compare outputs on identical tasks between the tenant model and the base model it was post-trained from. You’ll see that the distributions are actually quite similar, especially when you account for the amount of resources you would have had to spend getting there. 2) let’s say you realize this and you’re post-training the model not to be more knowledgeable, but specifically to navigate a law firm’s spaces better: learn how to navigate through Google Drive connectors, any enterprise software they’ve bought, DISCO, Clio, etc. The problem there is most generic off-the-shelf LLMs, especially the really cutting-edge ones, will already have had access to the same tools and will know how to navigate these environments themselves. It’s not like these environments are alien technology that intelligent models will not be familiar with. So then you’re again stuck on the same issue: what is the ROI? Especially when you consider the fact that once you start getting into post-training, what happens when the next generation of model comes out? Are you just going to start post-training again? You’ll have to post-train that and redo all of your investment. So really all that gets left are the very, very specific law firm things: ways they want to practice, where the templates are, how to find their templates, how to use their templates, etc. That is incredibly valuable. But any generic intelligent model plus a well-defined harness and a well-defined customized context map can handle this much better. Customizing the harness (this includes setting up things like access controls, which we will talk about in a sec) and context map will be 10x cheaper and much better than training a model. If you're skeptical about this, let's back both of our claims with code. 3) Now let’s say for the sake of argument that your law firm is committed to investing a lot of money into pre-training and post-training a model. They have the data, they have the compute, and they have the stubbornness to go ahead with this. The final problem here is that agent deployment in sensitive enterprise settings is not blocked by intelligence. In most cases, it’s blocked by other important aspects, such as data access control, strict edit permissions, etc. Certain employees should be able to access certain kinds of data; other employees should not. For some questions, answering them might require the system to access data that the employee should not see directly. It should be able to use that data to produce an allowed answer without exposing the raw data, or simply say that at their level of access control it cannot do this. Both are valid, depending on the org rules, but monitoring and enforcing these rules should not be purely left to LLM calls. Similarly, what about edit controls? You should be able to edit your files, not edit files above your permission level and absolutely not touch client data. If you try to solve these kinds of problems in the AI model layer, you are just introducing unnecessary stochasticity. You will have mistakes. Look no further than coding models for prood. despite tens of billions of dollars invested into post-training coding models, coding models continue to do stupid shit like delete production databases and rewrite access controls. And coding models have orders of magnitude more investment behind them. Coding is more verifiable, so it’s easier to train with post-training techniques. Coding is also a domain AI researchers and engineers are familiar with, so there’s much greater domain understanding there compared to something like law, where your AI researcher and your AI engineer will usually not know law particularly deeply. Despite all of those advantages, coding models still have not solved these problems because these are not AI model-layer problems. Anybody trying to solve these problems by just training a law-firm AI model is introducing unnecessary security risks and unnecessary costs. All of which will compund when the agents spins out subagents in swarms. 4) And I haven’t even started to mention the fact that even if you do train your own model, you still have to worry about inference. There’s a reason AI inference companies and providers that optimize GPU utilization and inference utilization make so much money. What are you going to do? Go to a cloud/neo cloud provider and tell them to host your model? Host it in a VPC? Because otherwise you’re asking law firms to invest not just in research teams and model teams, but also inference engineers, cloud-serving engineers, DevOps guys, and everything required to make sure one person’s query doesn’t get messed up with another person’s query. These problems continue to compound. You’re adding a lot of unnecessary headache onto people, and you’re doing all of this for basically no gain. It is much simpler for a law firm to take, if they want, an open model hosted in their environment or through something like OpenRouter. Fine. Or if they want a closed model, they can use an API with ZDR and no training guarantees. Then worry about setting up the harness, the context map, the access controls, and building a system with the security and access-control structure in place. That will give you 10 times the results. Happy to hear where you see things differently

We serve those as well! Sorry if using nuclear energy is a real blocker

This is going to be a pretty long answer, but just to preface it: if you really disagree with this, or you feel strongly that you have data that proves me wrong, let’s discuss it properly. We can do a public roundtable with you, your research team, engineering guys, whoever else you think would add value to the discussion. My community is almost at 2 million people monthly now, and I think it would be a great way to show people how you guys are thinking about this, why you’re so confident in the approach, and where you think my analysis is wrong. The point would not be "debate" (I think that's corny af), but to have a good-faith discussion about our differences in opinion/technical thesis in a way that everyone can walk out with a better understanding of things. Happy to also talk privately if that's better. Or just argue with code, as they say. We'll define a task, define a budget, we build it in our own ways, and let’s compare the results. Now onto the answers. TLDR: training custom models would be wasteful for law firms and most challenges around sensitive agentic deployments are not a model layer problem. Trying to solve them in the model layer creates major security issues that law firms will not want to deal with. 1-- when you talk about helping law firms build their own models/building on their private data, you're likely suggesting post-training, because obviously they will not have the amount of money and data required to train a completely new intelligent model from scratch. Especially one that’s competitive against the really good open models out there currently. This is not useful to them because stacks of research have already shown us that most of the knowledge of an LLM comes from the pre-training stage, not from the post-training. Post-training is essentially just a way to change how the model samples from its latent space, but the way concepts are organized and information is actually learned is mostly coming from the pre-training stage. This is a problem for law firms because they don’t want their own custom knowledge, their client data, their own proprietary knowledge, to be confused with generic off-the-shelf internet insights (since you're not changing the pre-training). The whole point of training a model on your custom data would be that it’s much more precise, much more grounded in your data (including private data that isn't available in generic models). If you’re only post-training and not pre-training, most of your knowledge is still coming from that generic base. You can actually confirm this for yourself with your tenant models. Compare outputs on identical tasks between the tenant model and the base model it was post-trained from. You’ll see that the distributions are actually quite similar, especially when you account for the amount of resources you would have had to spend getting there. 2) let’s say you realize this and you’re post-training the model not to be more knowledgeable, but specifically to navigate a law firm’s spaces better: learn how to navigate through Google Drive connectors, any enterprise software they’ve bought, DISCO, Clio, etc. The problem there is most generic off-the-shelf LLMs, especially the really cutting-edge ones, will already have had access to the same tools and will know how to navigate these environments themselves. It’s not like these environments are alien technology that intelligent models will not be familiar with. So then you’re again stuck on the same issue: what is the ROI? Especially when you consider the fact that once you start getting into post-training, what happens when the next generation of model comes out? Are you just going to start post-training again? You’ll have to post-train that and redo all of your investment. So really all that gets left are the very, very specific law firm things: ways they want to practice, where the templates are, how to find their templates, how to use their templates, etc. That is incredibly valuable. But any generic intelligent model plus a well-defined harness and a well-defined customized context map can handle this much better. Customizing the harness (this includes setting up things like access controls, which we will talk about in a sec) and context map will be 10x cheaper and much better than training a model. If you're skeptical about this, let's back both of our claims with code. 3) Now let’s say for the sake of argument that your law firm is committed to investing a lot of money into pre-training and post-training a model. They have the data, they have the compute, and they have the stubbornness to go ahead with this. The final problem here is that agent deployment in sensitive enterprise settings is not blocked by intelligence. In most cases, it’s blocked by other important aspects, such as data access control, strict edit permissions, etc. Certain employees should be able to access certain kinds of data; other employees should not. For some questions, answering them might require the system to access data that the employee should not see directly. It should be able to use that data to produce an allowed answer without exposing the raw data, or simply say that at their level of access control it cannot do this. Both are valid, depending on the org rules, but monitoring and enforcing these rules should not be purely left to LLM calls. Similarly, what about edit controls? You should be able to edit your files, not edit files above your permission level and absolutely not touch client data. If you try to solve these kinds of problems in the AI model layer, you are just introducing unnecessary stochasticity. You will have mistakes. Look no further than coding models for prood. despite tens of billions of dollars invested into post-training coding models, coding models continue to do stupid shit like delete production databases and rewrite access controls. And coding models have orders of magnitude more investment behind them. Coding is more verifiable, so it’s easier to train with post-training techniques. Coding is also a domain AI researchers and engineers are familiar with, so there’s much greater domain understanding there compared to something like law, where your AI researcher and your AI engineer will usually not know law particularly deeply. Despite all of those advantages, coding models still have not solved these problems because these are not AI model-layer problems. Anybody trying to solve these problems by just training a law-firm AI model is introducing unnecessary security risks and unnecessary costs. All of which will compund when the agents spins out subagents in swarms. 4) And I haven’t even started to mention the fact that even if you do train your own model, you still have to worry about inference. There’s a reason AI inference companies and providers that optimize GPU utilization and inference utilization make so much money. What are you going to do? Go to a cloud/neo cloud provider and tell them to host your model? Host it in a VPC? Because otherwise you’re asking law firms to invest not just in research teams and model teams, but also inference engineers, cloud-serving engineers, DevOps guys, and everything required to make sure one person’s query doesn’t get messed up with another person’s query. These problems continue to compound. You’re adding a lot of unnecessary headache onto people, and you’re doing all of this for basically no gain. It is much simpler for a law firm to take, if they want, an open model hosted in their environment or through something like OpenRouter. Fine. Or if they want a closed model, they can use an API with ZDR and no training guarantees. Then worry about setting up the harness, the context map, the access controls, and building a system with the security and access-control structure in place. That will give you 10 times the results. Happy to hear where you see things differently