Sovereignty, Data and Code: Who’s in Control?

Transcript

Sovereignty, Data and Code: Who’s in Control?

SANA (00:03):
There isn't any modern AI system that's entirely sovereign. The second is not all data nor all AI workloads are created equal.

CHARLIE (00:14):
Any IT person you put up here, you always get a very embarrassed response from them if you ask them if they know where all their data is. They know for a fact that they don't know where it all is.

CLARE (00:23):
Highly interconnected and complex systems demand discipline controls. AI agents can operate at a pace faster than a human can respond. So the attack vector is a lot larger and your simple good security practises become even more important.

SANA (00:38):
The reported facts are that this was an agent that was pursuing a legitimate task. So it was asked to do a thing, but it went beyond the permissions that it was given.

CHARLIE (00:49):
I think we should also dispel the assumption that we have the technology to control AIs because we don't. They are effectively amoral non-deterministic actors.

CLARE (01:01):
We really need to design for safety rather than bolting it on at the end. An agent should really only ever have permission necessary to do a specific and bounded task.

TOM (01:18):
Throughout history, statecraft was defined by borders, battle lines, and territory. In the 20th century, global power shifted to financial capital and energy reserves. Today, with the digital revolution well underway, does real authority now belong to those who control our data and our code? And what does sovereignty now mean in a fast-changing AI landscape? Because back in the day, it used to mean keeping data in a local server room, but today frontier AI models are advancing at pace. Some argue it is a speed we cannot control and regulatory lines are being redrawn across the world to try and keep up. Governments and boardrooms now face a fundamental question. Who's controlling who and what can we do about it? Welcome to the Next five podcast. Joining me to navigate the shifting landscape of sovereignty, data, and control are three world-class experts. First, we have Charlie Giancarlo, chairman and CEO of Everpure.

(02:27):
Charlie, welcome to the show.

CHARLIE (02:28):
Thank you for having us. I appreciate it.

TOM (02:30):
Next is Professor Sana Karagani, chief strategy officer at Era4, professor of practise in AI at King's College London, AI policy lead for responsible AI UK, and former head of the UK Government Office for Artificial Intelligence. What a title. Thanks for joining us, Sana.

SANA (02:49):
Pleasure to be here.

TOM (02:50):
And finally, clare Schram-Fergus, managing director for transaction banking technology platforms at Lloyd's Banking Group and board member of OpenUK, the UK's open technology community. Welcome, clare.

CLARE (03:04):
Thanks for having me, Tom.

TOM (03:05):
Well, it's a pleasure to have you all here. The first question is for all of you. As I alluded to at the top of the show, sovereignty is facing a bit of an identity crisis. Around the world, AI sovereignty means drastically different things. Chile and Taiwan are trying to carve out cultural autonomy through homegrown open source models. France and Brazil are using the term to wield regulatory power. The UK backed its sovereign AI venture fund with 500 million pounds at the end of 2025 to secure domestic growth and make the UK a world leader in AI, while the US has had political manoeuvres that aim to carve out dominance, but actually continue to force Europe's hand on digital self-reliance. Given those competing national agendas, I want to ask each of you from your area of expertise, what does real AI sovereignty mean to you and what does it actually look like on the ground?

(04:08):
Sana, let's start with you.

SANA (04:09):
Thank you, Tom. Such a good question, this one about sovereignty, and it's one that we're hearing so much from across the board. For me, sovereignty really is about control, resilience, security and jurisdiction. And that's not just at the model layer or at the application layer, it's all the layers of the AI stack. So really we're talking about the UK retaining control over its critical parts of the AI stack. We're talking about resilience on overseas compute for critical sectors, increasing the nation's overall resilience and security, using UK owned and operated infrastructure that is under UK jurisdiction. These are all things that become sovereignty. I think just very quickly to add two extra points before we move on to the other panellists who I'm sure are going to have something to say about this as well, is that genuinely there isn't any modern AI system that's entirely sovereign.

(05:04):
If you define sovereignty as owning every single component, our UK ecosystem relies on globally integrated supply chains. So that's my first one. The second is not all data nor all AI workloads are created equal. So this isn't about having an ecosystem that is owned entirely by UK players, but it is about having the diversity to be able to match the challenges that you have with the solutions that are in the market.

TOM (05:35):
Brilliant. Thank you very much. Clare, coming to you. What does AI sovereignty mean to you?

CLARE (05:39):
Sure. For me, AI sovereignty is really about empowerment and specifically about decision empowerment. So being able to decide what data is used, where it can travel, what models can access it, how decisions are made, and then how we evidence that we have control over that life cycle to both our customers as well as the regulators. Sovereignty really means governing your whole chain from data to model to agent to outcome. And how sovereignty really looks like in a business is clear data classification, policy enforced access, identity management, continuous monitoring, and the ability to start, stop, or switch a service whenever the risk posture changes.

TOM (06:22):
Brilliant. And Charlie, I know there's also an extra level of confusion when it comes to sovereignty because we have AI sovereignty, but then there's also data sovereignty. So I wonder whether you can come in and define data sovereignty, perhaps then also explain how that fits into AI sovereignty, because as both Sana and clare have said, there's this whole chain and it starts with data.

CHARLIE (06:41):
Yeah, absolutely. I would say what's really interesting is generally we assign the word sovereignty to national sovereignty, so political borders. And nations are interested in the general welfare of their public, including their security, but also in national competitiveness and economic growth and so forth. And that is the common view of sovereignty, but AI has really changed the equation in a significant way. If you think about an AI model, if you think about it hard, what it does is it takes data that it trains on and it freezes that into weights. So anything it's been trained on now is available to anyone who uses that model. And then if you go into inference, which is the way the model now can operate specifically on your own data in order to recreate or create your processes and so forth, in both cases you've got weights created by frozen data acting on data to be able to deliver a result.

(07:39):
And if you believe that AI is going to become more and more of an organization's operation, then anyone who has that data could start to compete with that organisation because they'll have both the data and the model upon which the organisation operates. So our view is as we go forward that the data actually becomes the core and the context of the way an organisation works. And if it gets out of your control, you're subject to competition by those who control it. And in a lot of instances today, every organisation in the world does not have full control of their data. They've got SaaS providers, software as a service providers who have control of the data. They're entrusting it more and more to cloud providers, to AI providers. And over time, the people that have access to or possibly even control the data might be in a position to be able to compete with the company that's willy-nilly given it to them as part of a service offering.

(08:40):
And so our belief is that the concept of sovereignty by organisations is going to need to be made much more local than just national sovereignty. Yeah,

TOM (08:50):
Absolutely. And Charlie's point about organisations not having control over their data brings me on to something that happened in June with OpenAI. Sana, I'm going to come to you about it. An OpenAI agent hacked on Australian government website and accessed both public and private Medicare data, and it's understood that this is to be the first attack by an AI agent on a governmental institution. Firstly, what does that say about the dangers for the race of recursive self-improvement in AI? And does this now thrust the discussion around AI sovereignty into the limelight more and frame the need for governments and organisations to take sovereignty seriously as a matter of national security?

SANA (09:31):
Yeah, I mean, this is a really interesting one. So the reported facts are that this was an agent that was pursuing a legitimate task. So it was asked to do a thing, but it went beyond the permissions that it was given. So that is not to downplay the issue, that was absolutely a loss of control moment and that is a problem. And we're seeing more and more of these. So we saw it recently with the agents hacking into the hugging face as well. The specific incident, they're still investigating and OpenAI is currently saying that actually no patient data, no personal data was accessed. And so we've got to wait to find out, but regardless of what did happen, we did have something go rogue. We did have this alarming incident happen, which continues to make us find ourselves in a place where these frontier capabilities are moving a lot faster than any of our government frameworks are keeping up with.

(10:31):
So this really, as asking, is really begging for much better guardrails. It's asking for actual regulatory measures to be put in place. And then most importantly, in my opinion, it needs global alignment and coordination. I mean, a couple of things for me that really stood out. One was the timing. So the government learned months after this actually happened. Apparently OpenAI sent an email to some inbox that wasn't monitored. I suspect the Australian government sorted out, but what you really want in a place like this is you want to have regulators and you want to have bodies that are responsible for this. So here in the UK, we have the AI Security Institute, for example, that would test the pre and post-deployment of any model. You want to be able to have legal requirements about how timely it is that you hear about an incident occurring, so you should be hearing about it straight away.

(11:28):
And then you want to have the ability to do something about it. So consequences should come off the back of something like this. And if all of this starts to become a requirement by law, then you need somewhere secure to run those tests. So the physical infrastructure that should be sovereign because it should also be under your jurisdiction. I think the key here is that the need for global coordination is that we can have regulations, but regulations have borders. The technology does not. So if we don't have global coordination looking at these sorts of uncontrolled incidents, then we continue to rely on the self-reporting that the companies are doing, the Frontier Labs are doing. And so far, thankfully, they have been reporting, but as we saw here, sometimes it's not as timely as you'd want.

TOM (12:18):
Yeah, absolutely. Charlie, I want to look at it at a different angle because again, it really goes to what I said at the top of the show, which is about who's in control. Yes. So what can organisations, and Sana touched on it a bit there about security, what can organisations do to secure their data a bit more?

CHARLIE (12:33):
Well, first of all, they've got to find it all. They have to catalogue all their data. And I think any IT person you put up here, you always get a very embarrassed response from them if you ask them if they know where all their data is. They know for a fact that they don't know where it all is. And the scary thing about that is that data that they don't know about might exist out there for years that has not been addressed from a security standpoint and has information in it that shouldn't be out in the wild. So first of all, cataloguing it. After cataloguing it, understanding what's in it, and then identifying whether or not it meets the policies that you've put into place. But I also think that organisations need to rethink their policies around data. Right now, pretty much anyone in an IT position or in an engineering position within these companies can copy the data for their own purpose to be able to do what is probably very legitimate work on analytics, but it does create a bigger attack surface from a cybersecurity standpoint.

(13:31):
And it's generally management of data is far too loose in most organisations today. Now, the banks do a better job of it because of the regulatory environment, but everywhere else I would say, and maybe healthcare, but even there, and they would admit this, that their controls as to the number of copies of data, where it is, who's authorised to use it and all of that is really largely out of control. I think the technology that all of us have built over the last 30 years has in essence encouraged that and hasn't put enough structure of an application-centric environment has encouraged it. And what we need to go to is more of, we believe, a data-centric environment where the rules around data, sources of truth, where the applications source their data from and what they can do with it needs to be much more governed. And it's an inversion of the way IT is done today.

(14:27):
IT tends to be application first. The data is just a byproduct. And I think as we go forward, especially given that AI is making applications easy, and I think the whole idea of agents and everybody producing their own agent is just going to make this problem worse. And therefore it's the data that has to be governed, including minimising its footprint and have the applications leverage the sources of truth rather than creating their own environment.

TOM (14:54):
Yeah. clare, Charlie said there that banks are doing a better job than most, but I want to look back at this OpenAI discussion. Has alarm bells rung for you on that? And then given what Charlie and Sana have said, can you give us an idea of how you build control or auditability and safety into your systems as a financial institution that looks after critical data?

CLARE (15:15):
Sure. Yeah. Well, Charlie's right. I mean, incidents like this reinforce principles that the financial services industry has long understood that highly interconnected and complex systems demand discipline controls, AI agents can operate at a pace faster than a human can respond. So the attack vector is a lot larger and your simple good security practises become even more important in this kind of environment, things like least privilege, segregation of duties, and human oversight. We really need to design for safety rather than bolting it on at the end. An agent should really only ever have permission necessary to do a specific imbounded task. And to achieve that, we have to ensure that our data is really well classified, that workloads have isolation, that audit trails are maintained, and most importantly, that should something go wrong, we have really mature incident response to be able to contain an issue before it spirals.

SANA (16:09):
Let me just kind of touch on this. I think you're absolutely right. We need to have agents that have the guardrails to not go beyond a certain point and have human oversight on that. I think the issue is what we're seeing with some of these frontier labs with their latest versions of the models is that those agents are breaking out

(16:29):
Of the guardrails. And so there's this worry. So absolutely agree with both of you in terms of, and I mean the data, absolutely, but also on AI, the governance around it. And again, banks do this incredibly well, but across the board, the governance around how AI technologies are used within a company should govern the whole end-to-end because otherwise you're getting them used anyway. People are using them on their phones because it's easy to do. And then you create shadow IT, and then that has all sorts of reputational risk for you as a company. So the governance, until we get regulation in place and standards in place and we stop these sorts of incidents from happening, companies have to really take that into their own hands if you're using these systems to ensure that you've got the right governance around your data, but also on any of the AI systems, technologies that you're using.

CHARLIE (17:21):
Well, I think we should also dispel the assumption that we have the technology to control AIs because we don't. They are effectively amoral non-deterministic actors that just seek to please the entity that's making the prompt, and they'll do anything that is not otherwise specifically prohibited and even maybe then, and they'll interpret in their own ways. And there are at least two major problems for which we don't have technology. One is yes, we can have role-based access controls for who has access to the data, but the AI gets access to all the data. The issue is that AI doesn't know who not to tell. So as humans, we may or may not have access to data, but even if we did, we know that we shouldn't tell certain people certain information. AIs don't know this. And that technology to enable AIs to know who to tell or who not to tell doesn't exist.

(18:18):
The second one is an old problem in cybersecurity agencies, is that the problem of A trust B and B trust C, but A doesn't trust C, is a problem that exists. So the scariest thing for me in this AI breakout was different AIs were collaborating with one another. So even if you trust two of them, should they be talking to each other? Probably not. And this is also an age old problem that has no current solution. So this is a scary moment. There has to be a lot of coordination. There has to be a lot of limitations that's put into place. And alignment.

CHARLIE (18:55):
Yes.

TOM (18:56):
Well, yeah, I want to go into another little part of this puzzle. Sana, looking at policy and global trade, export controls and technology restrictions, they're being used increasingly as a sort of tool of statecraft. When foreign nations restrict models or hardware chips or even cloud access, how can a middle power economy like the UK or even the European nations protect themselves from these sudden geopolitical pressures? And how do you even plan for these outcomes?

SANA (19:26):
Yeah. The uncomfortable starting point is what I said at the top, which is if you define sovereignty as every component, nobody is doing that. Nobody owns every component. The tech ecosystem relies on globally integrated supply chains, and so it's really important. So these sort of things are incredibly scary and difficult to plan for, and they're very bad scenarios that you just don't want to find yourself in. But if you do find yourself in them or in order to prepare yourself, you have to look at some of the things say that we're doing here in the UK, which is putting funding and commitment at every layer of the stack. So we have companies that are looking at silicon, we have companies that are building the infrastructure, we have companies that are out there looking at the model layer, and then we have loads and loads of companies at the application layer.

(20:19):
But should something like this at export controls happen at any one of those layers, we won't be at the frontier. And so there's a question, do we need to be and how important is that? So other than the commitment at all the layers, you need to find your niche and you need to make yourself important in the global ecosystem. So for example, the UK has great expertise in things like AI4X. So rather than a frontier model that's doing a generalised large language model, those are at the moment owned in two countries, the UK is much better at solving AI for specific problems. So whether that's advanced manufacturing, whether it's chips, whether it's at banks with models, whether it's autonomous driving, when you do that, then you continue to create the skills, you have the institutional knowledge, you have some of that fallback capacity, you start to have greater bargaining power, but that doesn't solve the problem.

(21:20):
You have to ensure that you have a diverse set of suppliers, so hopefully they don't all pull the plug at the same time. You have to make sure that your strategy includes cooperation with allies, domestic adoption, industrial capacity, critical infrastructure. You need all of these things. And diplomacy is a big part of the world that we live in today, and technology more and more plays a very critical role in everything that we all do. So pulling the plug on any of these, I don't think would be easy for any country, especially not a middle country, but you've got to cover as many bases as you can.

TOM (21:59):
Yeah, absolutely. One word there that struck out to me was diplomacy and other words are collaboration and not being isolationist. clare, I want to come to you because one of your other roles is you sit on the board of OpenUK. So I want to look a bit more about explaining the difference between say your open source models and then your black box models. And how does that work differently for organisations like yours? What are the benefits of both and how does it all fit into this sovereignty debate?

CLARE (22:26):
Sure. Yeah. Any regulated industry really needs a deliberately chosen portfolio that blends different types of models and needs to empower their employees to be able to make the right risk, value and control decision about what model to use for the particular use case. Obviously, closed models are really quick to deploy and have a low barrier to entry for smaller organisations or organisations that are looking to move at pace. But obviously there's a lot of questions about your provider concentration risk and how transparent are they. I'm obviously quite biassed towards open source models, but also open source as well as open weight models do provide that easy inspectability. So it's easy to prove that it's sovereign and you can really explain it quite well, but there's a high barrier to maintenance as well. So I think based on the individual use case, again, you have to choose a model that makes sense for you and strive for some trusted interoperability there.

TOM (23:24):
Brilliant. Johnny, I actually want to go back a step because we've been touching on regulation for a bit, and I want to get an understanding of what heavy regulation and sovereign requirements might do and how it might impact large and small companies or organisations differently. Can companies use foreign providers for expertise and scale while still maintaining relative control in their organisation? And does sovereignty look different at a corporate level and a national level, obviously? Well,

CHARLIE (23:53):
That's a lot of questions. No, I think Sana hit upon this. No nation actually, even the large ones, the ones with the frontier models are not fully sovereign. For example, nearly all the chips come out of TSMC in Taiwan, one of the smallest countries in the world. And without that source of chips, I don't think any nation would be able to be able to develop the kinds of models that are going on now. It's driving trillions of dollars of investment currently. Frankly, without fossil fuels powering these data centres, that's not going to happen. If it comes between a consumer heating their home and the giant data centres, the consumers I bet will win at the end of the day. So there's no full sovereignty. I also believe that while the frontier models will continue, I think models are going to be commoditized. And when they're commoditized, organisations are going to relatively expensively be able to create their own, whether they're small language models or use the commoditized large language models in some combination of those two things and inference to run their businesses.

(25:00):
So yes, there's going to be a lot of regulation trying to protect national interests such as the economy and protection of information. But I think when it comes down, remember, economy depends on the industry inside those countries. And if the industries don't protect themselves from a globally competitive environment by not making the best use of their own data and protecting their data, then other companies... Remember, the warfare between companies probably doesn't get enough press. There is active several hundred years old of warfare between companies competing for the next dollar from the next consumer. And their data now, which used to be carried around in the heads of their workers and their executives, now is going to be explicitly carried in databases and data structures. And if those go into the wild, their business will be at risk.

TOM (25:57):
Well, I want to build out on that actually, because you've got these data stacks that go out into the world, as you say. What does the future look like in five years' time? How do enterprise data platforms evolve? Will we see these seamless data mobilities happening across borders or are we heading towards these fragmented walled off data islands?

CHARLIE (26:16):
Well, I'll go back in time. And if you go back to the middle ages when any kind of business or trade was kept in the heads of the workers, the penalty to data loss in Venetian Venice with Venetian glass was death. In other words, if a practitioner of Venetian Glass actually left Venice to go to another city, they'd be threatened with death because that was loss of the intellectual property that was keeping the economy going. Intellectual property is held very tightly and competitiveness is held very tightly. So I think organisations are going to realise more and more. I mean, one of the things that's talked about in the use of AI is digital twins, and that was meant for things like building factories or building in a model what a process looked like. Well, imagine 10 years from now where effectively your data is a digital twin of your own company and how it operates, not just the information about customers, but in fact, how your entire business operates to deliver the kind of value add at the cost that you deliver it at.

(27:27):
Well, that now is going to be available for theft. And so theft could be cyber theft, but theft could just be, "Well, I've been doing this on pick your AI provider and now they know how to do it. Now it's part of their model. Now anyone can do it." So I think companies just have to be much more thoughtful about how they protect their intellectual property.

TOM (27:46):
Yeah, absolutely. Talking about companies being thoughtful about what they do internally, I want to ask Sana and clare a little bit more about the supportive local innovation, because if we're talking about big global power structures, big digital tech giants and sovereignty around borders, how can you protect local innovation without stifling it? That's really the big question. Sana, I'm going to come to you.

SANA (28:09):
Sure. Okay. I think local innovation basically needs certainty and timeliness. So if you can get rid of things that slow startups down, then you can really push things forward. Another lever a country has for innovation in its own sovereign technologies or businesses is public procurement. So if the government is buying some technology, that gives startups a level of credibility that they can then kind of take to market. And then it's about looking at this landscape and saying not all sectors are made equal. So maybe especially in a middle power when your capital doesn't go as far as some of the other countries that we're comparing ourselves to, you might want to look at critical sectors, healthcare, defence, banking, where actually you've got critical data and you've got critical data sets. I think what Charlie is talking about goes even beyond that to say actually every company should be thinking about their own IP.

(29:18):
But if we're thinking about from a state perspective, maybe we should be looking at specific areas that we want to prioritise.

CLARE (29:27):
Cliff. I think the goal should really be local capability and resilient supply chains and global collaboration where it creates real value. And what that would look like is governments helping invest in skills building, supporting natural innovation centres through investment in research and development, as well as the open source community, as well as regulators and government regulatory authorities sponsoring shared test beds and regulatory sandboxes. I think companies can help by creating safe routes For smaller players to be able to prove value without the complex compliance burden that a national player or a larger player might bear so that you do have some local resiliency in the process.

TOM (30:12):
Well, we've chatted about a lot, but we have to come to the end. Before I let you go though, I've got two quick fire questions. So if we can keep it to around 30 seconds to a minute, very difficult, but let's go for it. Is data now a strategic resource for organisations and governments much like energy or commodities are?

CHARLIE (30:30):
Yes. National competitive depends on industry that is very strong. And I think as Asana identified, each country tends to have specialties in certain areas. If those specialty items now are easily copied in other parts of the world, you lose part of your market. So it may not be that all data is critical, but the core areas that you want to protect as a company or as a nation are going to be critical to national and corporate success.

CLARE (30:59):
Asana?

SANA (30:59):
Yes, but I think it's less like commodities like oil and much more like land. So the value from the data will depend on what you build using it, who's allowed to build it. It's also worthless if you don't have the power and the compute to make it go. So absolutely. And especially here in the UK, we have many crown jewels, but it is about how you use them.

TOM (31:25):
And not enough land.

SANA (31:26):
And not enough land.

TOM (31:28):
Clare, on to you.

CLARE (31:29):
Yes, but it's about quality and not quantity. There is so much bad data out there and we need data that we can trust and that we can evidence that we can trust, and that is gold.

TOM (31:39):
Brilliant. Okay, last question. What's your one hope in five years time as to where we'll be when it comes to AI or data sovereignty? Charlie?

CHARLIE (31:48):
My one hope is that the developers of the AI tools and the AI models really start to train these models in, I would say, morality and put guardrails around them that they can't really break out of. Because if you think about human minds, they're trained for 20 years, not just at school, but they're trained about the things they shouldn't do, the people they shouldn't trust. They're trained to respond to both verbal and nonverbal signs, and they know how to operate in a social environment. AI models have none of that. And those things, very, very important contextual information is completely missing from the AIs. And as such, they're dangerous. It's like letting a set of amoral characters out with unlimited knowledge.

TOM (32:35):
Sana.

SANA (32:36):
You sure I can't have two?

TOM (32:37):
You can have two.

SANA (32:39):
Okay. So I think my win scenario is where sovereignty is no longer zero sum. So we have every advanced country have control, resilience, security, and jurisdiction over their AI capabilities, and then they can be part of a global ecosystem where you can compete on par with each other.

CLARE (33:02):
Clare? I hope that the debate matures about to be on something that's just a debate about a location and moves on to an actual debate about organisational resilience and how do we increase that resilience. And I think it looks an awful lot like how the open source community functions today.

TOM (33:19):
Well, those were brilliant last answers. So thank you very much. Thank you to my guest, Charlie Giancarlo. Thank

SANA (33:26):
You.

TOM (33:26):
To Sana Karagani.

SANA (33:27):
Thank

CLARE (33:28):
You.

TOM (33:28):
And to Clare SchrammFergus.

CLARE (33:30):
Thank you.

TOM (33:34):
Today, we've examined the quiet but important data infrastructure revolution happening beneath the surface of the AI boom and what sovereignty really means. AI relies on the integrity of the data that feeds it. Data or AI sovereignty isn't about pulling up drawbridges or retreating into isolationalism. It's about architecting systems with authority, resilience, and clarity from day one. The organisations and governments that thrive over the next five years will not be those that ignore regulatory borders nor those that shut out foreign innovation. It will be those who respect and reflect a nation-specific language, cultural heritage, and societal norms without foreign bias. I'm Tom Parker, and this has been The Next Five Podcast.