Bezos and DoD Dream of AI, but Invoke Terminator Nightmare - International Prose

Bezos and DoD Dream of AI, but Invoke Terminator Nightmare
by Emily Manna via claire - American Conservative Thursday, Mar 21 2019, 12:54am
international / prose / post

But it may be Amazon’s activities in Washington that warrant more scrutiny, as Open the Government notes in our new report, Government, Inc.: Amazon, Government Security & Secrecy. Much of the more recent media drama has been over Amazon’s planned headquarters (thanks to the close to $780 million in state and local subsidies that it’s been guaranteed) just outside the Pentagon in Crystal City, Virginia. But really, Amazon has been deeply entrenched as a dominant Beltway contractor in Washington for years, providing, among other government services, artificial intelligence to the Department of Homeland Security for their customs and border protection detention centers. The FBI has recently agreed to pilot Amazon’s facial recognition technology.

But for Bezos, his work here has just begun. He is now seeking to expand Amazon’s reach even further by providing cloud services and artificial intelligence to the military. And he’s leaps ahead of the competition for what may be the largest single source government IT contract in history—the $10 billion Joint Enterprise Defense Infrastructure (JEDI) program with the Department of Defense.

Amazon’s advantage in this regard is that Amazon Web Services (AWS) already provides cloud computing services across the federal government. Analysts predict the company’s total U.S. government business for 2019 could rise to as much as $4.6 billion. Even when AWS is not the direct provider, it’s often partnered with other contractors. In 2013, half of the 10 vendors that were part of a $10 billion Interior Department contract partnered with Amazon.

The most significant contract for AWS, however, was with the CIA in 2013—a massive $600 million deal to provide secure cloud services to the entire intelligence community. The contract not only means substantial revenue for AWS, but an advantage in competing for future government cloud contracts, particularly within the national security state.

The Department of Defense’s JEDI contract will be worth up to $10 billion over 10 years, and will be awarded to a single company. From the start, the contract has come under fire from Amazon’s competitors, who argue that the competition is rigged in Amazon’s favor. JEDI arose from former defense secretary James Mattis’s interest in developing artificial intelligence for the military. His 2017 trip to Amazon and Google convinced him that a single, department-wide cloud infrastructure would be the best way to accomplish that goal. Google later chose not to compete for JEDI following its own controversy over its participation in the Defense Department’s Project Maven, and while Microsoft subsequently emerged as a contender, Amazon is still the clear favorite to win the contract.

The fight for JEDI has turned ugly, as old-guard government IT contractors pull out all the stops to protest the bidding process. One contender, Oracle, took its complaint about the JEDI process to Federal Claims Court, alleging that multiple DoD officials who worked on the contract were former Amazon employees. Earlier this month, Federal News Network reported that the Pentagon Inspector General’s office and the FBI Anti-Corruption squad are investigating possible conflicts of interest in the JEDI process.

We know that Amazon has drastically scaled up its lobbying efforts in Washington over the past few years, far outpacing its competitors. But the JEDI process has also highlighted how little we know about corporate influence on federal procurement. In 2017, Amazon fought for legislation that would allow the DoD to set up an online portal for acquisition of commercial products. Congress passed the so-called “Amazon amendment” as part of the final bill, and critics say the specifications are such that only large companies like Amazon and Walmart can realistically compete for the contracts. These lobbying efforts are public knowledge because companies are required to report on them. But no such requirements exist for lobbying federal agencies on contracts and procurement, leaving the extent of Amazon’s (and other companies’) influence over the JEDI contract process unknown.

Looking beyond the contentious bidding process, the technology that the Pentagon hopes to get out of the JEDI cloud program is even more troubling. While many of us are used to thinking of the “cloud” as simple data storage, DoD’s cloud strategy reveals that a primary purpose of the JEDI cloud infrastructure will be to enable advances in AI and machine learning. The military is already using AI on the battlefield, and the Pentagon’s Defense Innovation Board website says that “the impact of AI and ML will be felt in every corner of the Department’s operations.”

The problem is that complex AI systems, in their current state, are virtually immune to human oversight. It’s a cautionary tale no different than the Terminator movies. In its recently released AI strategy, the Pentagon did commit to developing adequate testing and evaluation processes for AI systems. However, researchers and experts warn that testing to ensure that AI systems behave in a predictable way in all scenarios may not currently be feasible, and that even AI engineers often can’t discern how and why a complex system makes the decisions that it does. That means, crucially, that they have trouble figuring out why an AI system fails to produce a desired result. These issues also make it difficult to know when an AI system has been hacked.

So while DoD may intend to make its use of AI safe, predictable, and explainable, that may not even be technologically possible. But that hasn’t stopped the Pentagon from moving full steam ahead with developing and operationalizing AI systems, citing the need to stay ahead of rivals like Russia and China. And while Google employees made headlines by demanding that the company back away from Project Maven, the Pentagon considers the project to be a huge success.

Through Maven, the DoD used AI to assist in targeting for U.S. airstrikes in Iraq and Syria beginning in 2018. In late 2018, the military suddenly stopped releasing information about targeting of airstrikes in those two countries, despite a surge in strikes and civilian casualties. Maven is just a sample of what DoD hopes to do with AI, but it is cause for significant concern. When the military refuses to release even basic information, the public has no hope of knowing what role AI is playing and what damage it’s causing.

The tech companies, for their part, don’t seem terribly concerned about how their technology will be used. Despite facing employee protests regarding government use of AI services, Microsoft and Amazon declared their intent to continue working with the military and law enforcement. One Amazon Web Services vice president affirmed that the company has “not drawn any lines” in terms of the government’s use of its technology, despite the fact that the company “doesn’t know everything they’re actually utilizing the tool for.”

The layers of secrecy here are vast. First, overclassification and excessive secrecy are typical of the military and national security agencies. Second, the AI technology itself is difficult to understand and explain even for its creators, much less to oversight bodies or the public. Finally, private government contractors, who will be doing much of this work, do not have nearly the same transparency requirements as government agencies. Congress should work to rectify this problem by mandating greater transparency from the Pentagon both in our conflicts overseas as well as on their current and planned use of AI in those conflicts, and by ensuring better public access to information from private government contractors. Companies must, in turn, commit to disclosing more information about how they are ensuring their technology is safe and accessible to oversight bodies. They must get firm commitments from government agencies as to how their AI services will be used before signing contracts.

For more, read the full report from Open the Government: Government, Inc.: Amazon, Government Security & Secrecy.

Copyright applies.

Follow link below for additional embedded information: