---

Katrina Manson is a reporter for Bloomberg News covering technology and national security. She is also the author of a new book, Project Maven: A Marine Colonel, His Team, and the Dawn of AI Warfare that tells the story of the making of the Defense Department’s flagship military artificial intelligence system. Today, Maven Smart System is integrated into all of the U.S.’s military commands and is deployed all over the world. 

In this lightly edited Q&A, we discussed some of Manson’s research findings for this book, including the role of China’s rising military in Project Maven’s genesis, how the technology made its way from being deployed in the desert biomes of the Global War on Terror to the South China Sea, and the state of Pentagon-Silicon Valley relations today.

Q: What is Project Maven and why did you decide to write a book about it? 

A: Project Maven was the Pentagon’s effort to put AI at the heart of how America makes war. It was very narrowly defined in a memo that came out launching it in 2017 to bring AI, machine learning and computer vision to drone video feed footage at a time the U.S. was collecting so much video footage from drones and not being able to analyze it. 

But it was always about much more than that. Although for several years there was no mention of the word “targeting” in even internal discussions, ultimately it became an AI targeting effort. And when I spoke to some of the founders, they always had that in mind. That was controversial for a number of reasons: one, because AI targeting in itself is a controversial pursuit, very untested at the time. And two because it started as an intelligence effort and targeting is about operations, so there was a kind of internal bureaucratic split. They were trying to really create it for the first time and encountering a number of problems, and didn’t want it to fail before it had even got started.

I had been writing on the future of war since 2017, when I was at the Financial Times and a Pentagon reporter. And then in 2022, I moved to Bloomberg and was focused narrowly on national security and was trying to understand the way the U.S. military was using AI. It was becoming a little bit more public, but not that much more public.

I wrote a magazine story on Maven Smart System — a name I hadn’t known when I started reporting and researching the magazine article. I came to understand this was a platform that was bringing in lots and lots of different data feeds, intelligence feeds, and then trying to identify, select, and ultimately to prosecute or execute targets. 

I actually became focused on this before the launch of generative AI. But the launch of ChatGPT [in November 2022] meant that everybody else became interested as well. I found the moral and ethical dimension to this really important. Of all the worries about AI, whether AI will have a role in the taking of life seems to me very crucial and immediate, rather than some of those more existential, long-term risks that other people look at.

I was also interested in the changing nature of war and the debate over it. Why were people pursuing this with so much passion? I wanted to tell a story that brought together the different perspectives: from groups like Stop Killer Robots, a coalition of 270+ NGOs that thinks the world should ban autonomous lethal weapon systems, to the U.S. military that was arguing, often against some of their own, that AI and autonomy is what will deliver deterrence and victory for the U.S. and save lives.

It felt to me like a very important set of claims that needed tire-kicking, and that the debate was happening really in the absence of any concrete information about what AI was doing on the battlefield. None of those raw examples of how an algorithm fares in war or what an operator actually does with an algorithm were public. And so I really wanted to dig those out.

A central character in the history of Project Maven is Colonel Drew Cukor. Could you tell us about him? Why is he so important? 

Everyone I asked about Project Maven said, “You have to go and speak to Cukor, he’ll never speak to you.” That combination of two contrasting ideas weighed on me, and I found it very important to understand why he would never talk to me and why he was so important.

A lot of U.S.-China friction has driven big tech leadership to change their position. There has been a very deliberate effort from the U.S. military and Congress to finger-point at big tech and say, “This is the patriotic position to take.”

Cukor was the chief of Project Maven. Project Maven had a director who I had spoken to before for interviews, General Jack Shanahan, a retired Air Force general. But I was told that to understand the day-to-day and the details, I should find Cukor.

I spent some time trying to meet him and really learn what his vision was, and he decided to share it with me. He never shared operational details with me, but he did talk about what he was pursuing and why.

You wrote that the genesis of Project Maven came from a group known as the Breakfast Club. What is the Breakfast Club and what was its connection to China? 

At the time, the U.S. was fighting the Global War on Terror — the GWOT as it’s called — a series of what people in the Pentagon described to me as very rudimentary wars. And despite bringing to bear incredible firepower and intelligence capabilities, it was still finding them very difficult, with many, many U.S. military personnel killed and maimed by improvised explosive devices. They were losing more people and not gaining the kind of ground they might like. 

The Deputy Defense Secretary at the time was Robert Work. He had not fought the GWOT. He had been in the military, then left, then came back as Deputy Defense Secretary and had a long-term interest in the future of war and maintaining America’s technological edge. He saw that China had been studying America’s weak points at war for 10 years and was beginning to find ways to circumvent America’s firepower. China was beginning to spend more on defense, although nothing like as much as the U.S. military, and U.S. aircraft carriers and other elements were starting to become vulnerable.

It was Work’s vision that really enabled AI. He told me that he saw AI as a stepping stone always toward autonomy. He talked about autonomy in two different ways: autonomy at rest and autonomy on the move. Autonomy at rest was this idea that you could bring a digital system together that, with the help of AI, would find targets and act as mission control. And autonomy on the move was about putting AI onto drones and unmanned fleets and weapon systems.

Work would pop in on the Breakfast Club, which was a group of intelligence and defense higher-ups who came together to work out how the U.S. should keep its technological edge and seek dominance. The way he saw it, the nuclear bomb, which the U.S. is the only country to have detonated in wartime, had kept the U.S. ahead. Then, precision-guided weapons had kept the U.S. ahead. The third era, he thought, would be AI and autonomy. So the Breakfast Club was meeting trying to work out what to do in practice, and also to get the culture of the U.S. military up to speed.

When Project Maven got started and Drew Cukor became chief, Cukor was focused on this idea that people were too used to the traditional ways of buying tanks and other big pieces of military kit and they needed to completely change the way they thought. Cukor saw software as one of the main tools of war. And he was ready for a knife fight over that. He was prepared to stake his career on this pursuit. He was one of the people who ended up attending the Breakfast Club, and toyed around with ideas and then tried to think of concrete ways to actually build new tech.

While the original impetus may have been the rise of China’s military, you wrote about how Maven was field tested first in the Middle East. How did the technology make its way to Indo-Pacific Command and eventually get deployed against China? 

The reason for the initial focus on the Middle East was that the U.S. was in active combat and drones were overflying those areas. So if they wanted to use AI on data, that’s where the data was coming from. And it was where there was an opportunity to save lives if the AI could be useful. So they wanted to work within existing military structures in places like Somalia, Iraq and Afghanistan.

They actually had enormous difficulty convincing Indo-Pacific Command to use AI. I found that very surprising. The group that was meant to establish AI, as you say, for a U.S.-China rivalry, wasn’t initially able to infiltrate its own Indo-Pacific Command. That might have been for reasons besides AI, to do with contractors and existing programs. Maven wasn’t the only AI effort at the time and it wasn’t the only effort to bring together data in one place in a common operating picture. 

But Indo-Pacific Command now uses the Maven Smart System, so there has been a change, it’s just taken many years. Something retired Colonel Cukor told me stayed with me, which is that he always thought it would take 20 years for the U.S. military to change its bureaucracy, its culture and its idea of embracing AI. Today, the U.S. military is 10 years into the experiment that he started. And so there’s another 10 years to go. Within that timeframe, they have got it to INDOPACOM now.

One of the challenges to developing AI in this context is even securing the right training data. So much of the video data used to train Maven at the start came from drone footage of the Middle East. So how accurate is Maven in other climates and environments, given the different geographies of say, Somalia versus Taiwan? 

When [the Maven team] started, they had access to several years of video footage but it was unusable. It was old, stored in cupboards. Even when they were offered the privilege of going through these old cupboards of video footage, they couldn’t integrate it with the current systems. Part of what Maven achieved, from their perspective, was getting different groups to grant them permission to collect a pipeline of data. For example, video footage from helicopters. 

Another problem was that even if you had the data, the data labeling may not have been up to speed. They went through several different iterations of trying to label the data correctly, first using volunteer labor from the U.S. military services, which didn’t work — they weren’t good enough, they weren’t consistent. Sometimes [volunteers] would scrawl expletives on the very imagery that they were supposed to be labeling because they got fed up. So then they tried a series of commercial entities with varying degrees of success. Even today, there are problems with labeling, and it’s now worth a tremendous amount of money — the [National Geospatial-Intelligence Agency, which oversees part of Project Maven] has a $700 million contract out just for labeling, for example. 

To your point about accuracy, they struggled a lot to work out how to test how accurate an algorithm was, partly because they didn’t have a baseline for humans. They found the question of, “are algorithms better or worse than humans?” hard to answer. They also would argue that they weren’t comparing apples with apples because what algorithms could do at scale was look at vast amounts of territory that no human could, and then surface things that a human could then check and verify — that was the ideal way in which it would work.

What often happened, though, was that the algorithms were very poor at detecting the objects they were meant to. And so even once the algorithms got good enough in, say, desert conditions, when they were used in the Philippines, they couldn’t recognize anything. The objects were different, the background was largely green, and so the capabilities completely collapsed to 30 percent accuracy or lower.

Then in Ukraine, where it really, really counted, the algorithms couldn’t recognize tanks, or differentiate tanks with their turrets blown off, so they couldn’t do battle damage assessments. And they also couldn’t recognize objects in the snow. Everything that had been trained in the desert or in the Philippines quickly failed. 

If you look at Maven Smart System today, I’ve reported that it’s going to become a program of record… which means it’s getting a dedicated funding stream [from Congress]. It’s being used in every combatant command, and NATO also is adopting a version of it…

So they brought over the satellites, collected more data, sent all those images of the Russian tanks lined up on the route to the Ukrainian capital, and got the algorithm vendors to retrain the algorithms. Detection scores that had collapsed down to 10 to 30 percent started to come back up. But that took time. And if you’re thinking about using AI to speed up your delivery of war, particularly in the opening days, not having algorithms that are adjusted to your biome, as they call it, or the kind of weapons your enemy will be using, is mission-critical.

In the case of China and Taiwan that you mentioned, they began to think about that. In 2023, they started this big effort to collect imagery of Chinese vessels and to train algorithms on detecting boats. It got so good that they eventually demonstrated this recently to the Chairman of the Joint Chiefs of Staff, General Dan Caine, to show, “Look, we have a computer vision algorithm that can recognize a Chinese destroyer.” But they also have encountered, I’ve learned, many problems, and accuracy is not where anyone would want it to be, even though they’ve collected an enormous amount of data.

For some of the conditions under which the algorithms have to operate, sometimes the algorithms are being put over feeds from vessels that only have one camera. Even just a drop of ocean spray can get in the way of that tracking. That has implications not only for AI, but also for the design of drone boats — a drone boat that has only one lens is going to be more susceptible to ocean splashes than a drone boat with multiple lenses. They’re still trying to work on that accuracy. 

What’s going on with AI jet skis? 

Over the course of reporting the book, I learned that the U.S. wanted to create a series of autonomous platforms. One of these — to my surprise — was an autonomous jet ski. The theory was that China has the advantage in shipbuilding, and the U.S. is obviously trying to catch up to that from a position very far behind. But the U.S. has actually cornered the market for jet ski production. The idea was that putting weapons on jet skis and having them be able to navigate and target autonomously could be useful for a new uncrewed weapons fleet. 

One thing that emerged clearly from Ukraine is that drones are susceptible to jamming. It’s one of the reasons that drones in Ukraine these days are run by fiber optic cable. That is not expected to work in a Taiwan invasion scenario. That’s where the argument for autonomy comes from. It also feeds into the ‘Hellscape’ idea that Admiral Sam Paparo, commander of the United States Indo-Pacific Command, has spoken about: that with enough autonomous systems on water, underwater and in the air, the U.S. could fend off an attack and buy him what he said is a month of time. My understanding is that these jet skis could be part of that. 

The program is called Whiplash. I found it in Navy budget documents, and then subsequently under the second Trump administration, that name was removed, but a similar program that seems to match the exact details of what had previously been called Whiplash is still under development. 

I also discovered that the CIA smuggled rudimentary versions of some of these autonomous jet skis into Ukraine and that subsequently a jet ski washed up on the shores of Turkey, sparking consternation inside the Pentagon that their contribution had been uncovered. It obviously raises the question about how much control anyone has over an autonomous vehicle with weapons on it, with explosives packed into it, if it can be lost.

Chinese drone swarms versus American jet ski fleets. It almost beggars imagination.

But it is within our imaginations now, because the U.S. now has a Defense Autonomous Warfare Group, which is known as DAWG. They are trying to produce autonomous weapon systems in air, on the sea, under the sea. In January, they launched a hundred-million-dollar contest for voice-controlled autonomous drone swarming tech. Many of the LLM companies submitted for this and are, I’ve reported, working on this today. And so they really are leaning into these very hard-to-imagine forms of new warfare technology. 

One of your findings is that many now highly-recognizable AI companies actually owe their existence to Project Maven, like Palantir. Could you say more about that?

The actual AI was produced by companies like AWS and Microsoft, companies that obviously could stand without Project Maven. But even for them, Project Maven was a stepping stone to a big cloud contract in the Defense Department, and that was worth real money. And so everyone was keen to work on Project Maven, even if they were the biggest of the big tech companies.

Palantir was a startup at the time Project Maven got started. This is contested, but my reporting shows they were on a smaller foothold inside the Defense Department than they wanted to be. As some people told me, they were on their way out of the Defense Department because the Global Wars on Terror were in theory winding down. They hadn’t been interested in AI. And so what a senior Palantir official told me is that Cukor kind of forced them to reassess their position that AI wouldn’t be useful in the future, saying, “No, it will be, and I want you to make me this digital interface.” Palantir didn’t even want to particularly make that, they didn’t want to make a pretty graphical user interface, they wanted to do their data crunching.

Eventually they got it, but not at the beginning. Cukor sketched out his ten year vision to Palantir and said, “Make me this,” particularly in the wake of Google dropping out [of Project Maven]. I think Cukor would have most liked Google to make that interface, but he never got the team at Google that he wanted. 

If you look at Maven Smart System today, I’ve reported that it’s going to become a program of record by September this year, which means it’s getting a dedicated funding stream [from Congress]. It’s being used in every combatant command, and NATO also is adopting a version of it. That is money for Palantir. So it has become a tremendous boon for Palantir and it has established it as a government contractor that is on the up. I have people in the book tell me that it’s entirely fair to say that Palantir owes its fortunes to Project Maven, certainly at the Defense Department. At one point Palantir was valued at over $400 billion, although it’s come down quite a way now, but they’re still one of the biggest defense companies in the world, and that dates back to Project Maven.

The relationship between the U.S. military and Silicon Valley has changed dramatically since the early days of Project Maven, when Google withdrew from the project after facing a mutiny from its own staff. In your view, where has Silicon Valley landed on cooperation with the military today?

A lot of U.S.-China friction has driven big tech leadership to change their position. There has been a very deliberate effort from the U.S. military and Congress to finger-point at big tech and say, “This is the patriotic position to take.” Now, even a company like Anthropic — which is in court after being called a supply chain risk because it refused to do certain types of work with the Pentagon — is still being used on classified networks and is being used today in U.S. operations in Iran. So even where you do see very calamitous dissent, it is a different shade. It is not about being averse to the business of war, which is what Google workers protested about.

If you look at the workforce, it’s different. It’s divided. I recently reported on a protest letter from hundreds of workers at Google urging leadership not to allow the company’s AI to be used for classified defense work. And that included DeepMind workers. One of the things that happened as a result of Project Maven is companies started to segment their work, creating sections of their companies that worked with the government and hired from the veteran community, which obviously was already comfortable doing defense work.

Apple is a very interesting case. I learned that the algorithms that Google had [developed for Project Maven] actually survived. A small startup called Xnor.ai kept them on and was working on Project Maven. But when Xnor was bought by Apple, it stopped providing work for Project Maven. So there are some companies that have a different position on working with the U.S. Defense Department, and Apple is obviously a huge one.

Within the workforce, there are multiple different protest movements, not just against Palantir. There are concerns within workforces about the way in which data may be used by ICE in immigration and deportation. That bleeds into concerns about defense work. It is of course not transparent. So even companies that say they’re comfortable doing work for the Defense Department do not get to know precisely how their tech is used. And so there are huge questions that workers are organizing around. It’s important to note a huge amount of variety in perspective within big tech. And it’s still playing out.

---