Librarians of Babel: Intelligence’s Three Big Problems in the Information Age
Data Saturation, Source Inversion and Disengagement
“This much is known: For every rational line or forthright statement there are leagues of senseless cacophony, verbal nonsense, and incoherency.” - Jorge Luis Borges, The Library of Babel[i]
The U.S. Intelligence Community faces major challenges resulting from the information revolution. First, an exponential increase in the volume of data being generated threatens to overwhelm an already over-tasked collection and analytic structure. Secondly, the vast majority of useful information today is publicly-available, a fact the IC has struggled to come to terms with and that could obviate its classified collection model. Lastly, both of these problems contribute to a growing disengagement by the end-users of finished intelligence, American policymakers.
In 1941, Argentine poet Jorge Luis Borges wrote “The Library of Babel,” a short story about an enormous library, its numerous rooms full of randomly- assorted books. The library, its inhabitants believed, contained every book that had ever been written, and every book that could ever be written—that is, every possible combination of the thirty-six alphanumeric characters theoretically existed somewhere in its vast network of honeycombed rooms. On the downside, this meant that the vast majority of the library’s books were gibberish. But there were also, theoretically, books of unimagined brilliance.
Over time, the librarians divided into factions. Most of them contentedly lived their lives tending the books around them, grew old and died, not far from the same room where they were born. Others, however, degenerated into savagery. “I know of districts in which the young men prostrate themselves before books and kiss their pages in a barbarous manner, but they do not know how to decipher a single letter,” Borges wrote. Some, calling themselves ‘purifiers,’ burned books they deemed harmful. Others, called ‘inquisitors,’ traveled from room to room searching for meaning among the millions of volumes. “Obviously,” Borges concluded sadly, “no one expects to discover anything.”[ii]
If you’ve been online recently, Borges’ Library may seem like an eerily prescient portrayal of modern digital life. Our library is a vastly expanding datasphere[iii] that contains a wealth of information—both exquisite and banal. In 2009, this datasphere began doubling every two years, a rate of growth that shows no signs of slowing. The volume of data generated in just two recent years alone is estimated to be equivalent to nine times all the data generated over the span of human history.[iv] By 2025, that number will increase by a factor of ten.[v]
The information revolution is having profound effects on social, economic, and political life. It has already challenged once-reliable frameworks, testing some and dismantling others. It has led to the emergence of radically different systems of engagement and collaboration for organizations and individuals. Rigid bureaucracies, in particular, have found it difficult to keep up.[vi]
Unfortunately, it has also supercharged the basest of human foibles, such as our tribal nature[vii] and our preference for lies.[viii] Henry Kissinger observed that in some ways this new age makes manifest Thomas Hobbes’ proverbial state of nature—the war of all against all.[ix] Consequently, even though we have more information available, certainty about anything has decreased—and like Borges’ librarians, we are collectively more confused than ever.[x]
The United States Intelligence Community (IC), the constellation of agencies charged with providing decision advantage[xi] for American policymakers, is an enterprise that cost the American taxpayers $80 billion this year.[xii] The IC was established in the middle of the last century, at the height of a hierarchically-dominated industrial age in which knowledge work was analog, production models were streamlined for standardization, and the most useful information was kept secret and thus difficult to collect.[xiii]
The IC’s standard operating model,[xiv] though well-suited for its time, faces three major challenges today. The first is that now that information has been digitized and moves at the speed of light, it is growing at an exponential rate—a problem of data saturation which the IC was never designed for. The second is that today most information is freely available, and nothing remains secret for long—a problem of source inversion. The third is that given this abundance of new sources, and their greater availability, end-users of intelligence have many more options available to them—in other words, competition—a problem of customer disengagement.
In contrast to the scarcity of information during the Cold War, we are now virtually drowning in it. Unfortunately, even though the pendulum has swung from paucity to inundation, insight remains in short supply. The IC struggles against an insidious Red Queen[xv] effect as it tries to make this mass of data coherent for policymakers, a distinctly Sisyphean task.[xvi]
More collection! – Has been the answer to intelligence failures in the past. But today, more is the problem, not the solution.[xvii] One observer warns that the next big intelligence failure “won’t come from a failure to connect the dots, but instead from missing dots buried in a mountain of noise.”[xviii]
Though equipped with the most technically sophisticated suite of collection platforms, there exists what some call a ‘success catastrophe,’[xix] but what I think of as an ‘analysis gap’—an unquantified delta between the volume of ‘things knowable,’ ‘things collected,’ and things known. Analysts are essentially running faster to stay in the same place. No matter how many of them are hired or mission centers established, the analysis gap delta will continue to grow. It thus becomes much harder to find proverbial needles within haystacks.
Sherman Kent, the ‘father’ of American intelligence analysis,[xx] hinted at the magnitude of this problem when he wrote that “to be able to deliver [intelligence] in the fashion apparently expected…would demand a research staff large enough to codify and keep up-to-date virtually the sum-total of universal knowledge.”[xxi]
Which is of course far more challenging today than it was in 1949 because, by 2025, the global datasphere will exceed163 zettabytes, a ten-fold increase from just 2016.[xxii] In the same period, there will very likely be more machine-to-machine communication taking place on the world’s wireless networks than all human communication.[xxiii] For an idea of just how much data that is, all of the words ever spoken by human beings throughout history constitutes much less than a single zettabyte.[xxiv] Put another way; it would be like watching Netflix’s entire digital catalog 489 million times (which, if you’re curious, would take nearly two billion years).
That’s a big haystack.
As the number of hard targets shrinks and the need to understand a complex, interconnected world grows, intelligence becomes less about collecting the haystack to find needles and more about understanding the shape of the haystack itself and the collective properties of hay.[xxv] Intelligence should be judged “…less by whether it was right or wrong and more by whether it helped the government come to grips with difficult questions.”[xxvi] Because the world is connected today and connectivity changes objects, there will be far fewer ‘golden nuggets’ of intelligence reporting because single items will be less informative against a vast horizon of interconnected background data.
In response, in a reversal of its cultural heritage, the IC needs, to embrace rather than eschew uncertainty—because uncertainty is a persistent fact of life in this complex era.[xxvii] Rather than spending its analytic resources on niche specialists, it desperately needs broad-minded, holistic thinkers with conceptual grounding across multiple domains who grasp the shape and magnitude of global connections rather than “…sufficing with analysts who perform primarily as data sifters and arrangers.”[xxviii]
A common proposal embraced by IC leaders is to accelerate the IC’s adoption of cross-domain, multi-disciplinary machine learning (ML) software and Artificial Intelligence (AI).[xxix] But while many have embraced this need, some are dubious of the promise of big data.[xxx] Data analytics is not magic, after all, because data can be tricked, and data can lie.[xxxi] An algorithm is only as good as the coder who wrote it, and many very mundane human biases are essentially ‘baked in’ from the outset.[xxxii] Statistics express averages, not individual cases.[xxxiii] And it is the individual cases—the crises—that intelligence is most concerned with.
Even some leading AI experts believe the promise of big data “overhyped.”[xxxiv] As National Intelligence University Fellow Bowman Miller observed: “If the IC continues to wait in vain for the advent of ‘bionic analysts’ or rely on the presumed marvels of technology to fix its shortcomings, those working in and relying upon intelligence analysis are in trouble.”[xxxv]
The second major challenge is a sharp reversal in the places where useful information can be found. In a source inversion from the Cold War, today the vast majority of information is available through open sources. While much—most—of it is dross, the percentage that is insightful is nonetheless much greater than in decades past. And as the information revolution continues to ripple across the world, the amount of data being generated increases in a super-linear fashion. Former Defense Intelligence Agency chief analytic methodologist and NIU research faculty-member Josh Kerbel characterizes this burgeoning trove of data as U3 (ubiquitous, useful, and unclassified)—a take on the data science world’s ‘V3’ for volume, velocity, and variety.[xxxvi]
The IC hasn’t quite figured out how to incorporate open source material it into its production model because of its organizational culture. That culture is still fixated (Kerbel calls it nostalgia) on the security compartmentalization that promotes narrowly reductive thinking, and many intelligence officers exhibit a cognitive bias that prefers classified information to anything available from outside the community.[xxxvii]
Tearing down organizational walls and opening the floodgates that prevent information sharing between disciplines would threaten decades of this compartmentalization and “…force the collection and analytic communities—with their brethren in counterintelligence and security offices—to reconcile threats from outside with threats from within.”[xxxviii]
In other words: good luck with that.
If this assessment seems grim, then the old saw about intelligence officers seeing flowers and thinking funerals must be apt. But there is room for optimism, right out of the IC’s history. It was Sherman Kent himself, who in 1942 led an Office of Strategic Services team equipped with only publicly-available information to “produce in record time a series of studies on the ports and railways of North Africa” to support Operation TORCH, the allied invasion. “The military customers,” Kent reported, “ … couldn’t believe so much useful information existed, much less could be written up with authority so quickly.”[xxxix]
Embracing open sources is in this sense a return to the IC’s roots. Imagine what he’d think today.
The third, and perhaps greatest, challenge is that of increasing competition driving customer disengagement. The fact is that with more things to notice and the same amount of time in which to notice them, our attention spans are shrinking, and policymakers—the end users of intelligence—have less inclination and time to read intelligence reports. We are continuously deluged with new, seemingly-important data-points in high-definition granularity, but have scant time to spend addressing any of them.[xl] As one author put it, “a wealth of information creates a poverty of attention.”[xli]
Annually, the IC produces more than 50,000 intelligence reports, of which many—if not most—are ignored.[xlii] Either because they failed to ‘stick’ with the intended customer,[xliii] or, perhaps more likely, because they never made it to that customer’s desk in the first place. Many of these reports are insightful, well-sourced, and expertly-written, but unfortunately are only read by other analysts. If its mission is to speak truth to power, today the IC is often simply preaching to the choir.
Former CIA Director Michael Hayden relates how a senior IC officer referred to the Obama administration as the first ‘Google’ White House, meaning staffers were accustomed to searching for their own information. The appeal of Google, he reflected, was that it “lets you wander and explore. They [the staffers] really didn’t know how to search intelligence, and we [the IC] didn’t know how to make it available to them that way.”[xliv]
The most successful companies today, in response to this sort of challenge, do not push standardized products. Instead, they are pulled by their customers to where the customers want to go, moving from a model of flow dependency to one of shared dependency in which clients and providers co-create value.[xlv] Increasingly, policymakers will expect the same from the IC.[xlvi] The further we get into the information age, more leaders will expect an interactive and customized relationship with their intelligence officers.
Not only have our attention spans diminished; our very perception of time and willingness to wait have been altered.[xlvii] Simply put, decision-makers expect more today: on-demand insight, customized to their tastes. Good enough stopped being good enough a long time ago. They assume that they will have access to all of the information required to make well-informed decisions, anywhere, at any time. They expect to be able to recall any fact or figure and to find the answer to any question almost instantaneously.[xlviii]
In conclusion, while the refinement of data into knowledge[xlix] is central to analysis, analysts don’t create knowledge for its own sake.[l] The purpose of analysis is to turn knowledge into value—in other words, advantage—for policymakers. The information revolution is having profound effects on how intelligence is collected, processed, analyzed, and disseminated as valuable knowledge. It will likely irrevocably alter the relationship between intelligence producers and policymakers—blurring the lines for example, between recipient and participant and between collector and analyst.
The problems identified and briefly discussed here are ‘wicked’—that is, they have no simple solutions, and each is a symptom of the others.[li] Fortunately, there are brilliant and dedicated professionals within the IC that have recognized them.[lii] While there have been calls for reform of the IC literally since its establishment,[liii] the ranks of those calling for major changes in doctrine have grown larger—and more influential in recent years.[liv] Raising awareness of these challenges is critical to creating a sense of urgency in the IC’s stakeholders. Only when they recognize looming failure are normally stolid institutions more pliable to reform and more likely overcome the institutional inertia that prohibits the implementation of necessary changes.[lv]
The IC’s officers are working in the most dynamic competitive environment in the enterprise’s 70-plus-year history. They have, for nearly two decades of war, performed admirably all over the world—often in austere conditions and not infrequently under fire. Several of them have paid the ultimate price. As a community, they have deservedly derived satisfaction from various succès d'éclat, such as the 2011 raid into Pakistan which resulted in the death of Al Qaeda founder Osama bin Laden.
But past pedigree counts for little in today’s world, and previous success guarantees nothing. Together, these three wicked problems facing the IC in the information age place its standard operating model in crisis. If the community insists on retaining this model, it risks becoming obsolete.
Zachery Tyson Brown is a career intelligence analyst and consultant who currently serves as an intelligence advisor to the Department of Defense. He can be found on Twitter @Zaknafien_DC. Zach previously served in the United States Army and as a civilian intelligence officer in the Department of Defense. Zach is most recently a graduate of the National Intelligence University, where his thesis "Adaptive Intelligence for an Age of Uncertainty" was awarded the LTC Michael D. Kuszewski Award for Outstanding Thesis on Operations-Intelligence Partnership. He also holds a Master’s Degree in History from American Military University and is currently enrolled in the Masters of International Service Executive Program at American University.
The views expressed herein do not reflect the official policy or position of the National Intelligence University, the Department of Defense, the Intelligence Community, or the U.S. Government.
[i] Jorge Luis Borges, trans. James E. Irby, “The Library of Babel,” in Labyrinths (New York, NY: Modern Library, 1962), 5.
[ii] Ibid., 9.
[iii] The term datasphere is defined as the sum of all data created, captured, or replicated in a given year globally. See David Reinsel, John Gantz, and John Rynding, “Data Age 2025,” International Data Corporation (April, 2017). Retrieved from https://www.seagate.com/files/www-content/our-story/trends/files/Seagate-WP-DataAge2025-March-2017.pdf
[iv] Leo Leung, “99.8 Percent of the World’s Data Was Created in the Last Two Years,” Tech Expectations (2014). Retrieved from https://techexpectations.org/2014/10/23/99-percent-of-the-worlds-data
[v] David Reinsel, John Gantz, and John Rydning, “Data Age 2025,” IDC White Paper, (April 2017). Retrieved from https://www.seagate.com/files/www-content/our-story/trends/files/Seagate-WP-DataAge2025-March-2017.pdf
[vi] Niall Ferguson, The Square and the Tower: Networks and Power from the Freemasons to Facebook, (New York, NY: Penguin Press, 2018); Michael J. De La Merced, “Eastman-Kodak Declares Bankruptcy,” The New York Times (January 19th, 2012). Retrieved from https://dealbook.nytimes.com/2012/01/19/eastman-kodak-files-for-bankruptcy/; WSJ Staff, “Toys ‘r’ Us Files to Wind Down US Business,” The Wall Street Journal, (March 14th, 2018). Retrieved from https://www.wsj.com/articles/toys-r-us-files-to-wind-down-its-u-s-business-1521092181?mod=searchresults&page=2&pos=8; Daniel Kreps, “Columbia House Files for Bankruptcy, Blames Streaming,” Rolling Stone (August 11th, 2015). Retrieved from https://www.rollingstone.com/music/news/columbia-house-files-for-bankruptcy-blames-streaming-20150811
[vii] P.W. Singer and Emerson T. Brooking, LikeWar: The Weaponization of Social Media (New York, NY: Houghton-Mifflin-Harcourt, 2018), 166; Francis Fukuyama, Identity: The Demand for Dignity and the Politics of Resentment (New York, NY: Farrar, Straus, & Giroux, 2018).
[viii] Soroush Vosoughi, Deb Roy, and Sinan Aral, “The Spread of True and False News Online,” Science 359/6380 (March 18th, 2018), 1146-1151. Retrieved from http://science.sciencemag.org/content/359/6380/1146; Eli Saslow, “Nothing on This Page is Real: How Lies Become Truth in Online America,” Washington Post (November 17th, 2018). Retrieved from https://www.washingtonpost.com/national/nothing-on-this-page-is-real-how-lies-become-truth-in-online-america/2018/11/17/edd44cc8-e85a-11e8-bbdb-72fdbf9d4fed_story.html?fbclid=IwAR2ht-o4-TLdyDUv-ze5UGBfnOtWfRvF59wz2fFcGOD4a4f2m7aNuWcctz4&utm_term=.ab6e4e0a7ff2;
[ix] Henry Kissinger, World Order, (New York, NY: Penguin Press, 2014), 344.
[x] Kevin Kelly, The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future (New York, NY: Viking Press, 2016), 279.
[xi] The Office of the Director of National Intelligence, Vision 2015: A Globally Networked and Integrated Intelligence Enterprise (Washington, DC: Government Printing Office, 2008), 1;
[xii] The Office of the Director of National Intelligence, “U.S. Intelligence Community Budget,” Office of the Director of National Intelligence, (November 17th, 2018). Retrieved from https://www.dni.gov/index.php/what-we-do/ic-budget;
[xiii] Niall Ferguson, The Square and the Tower: Networks and Power from the Freemasons to Facebook, (New York, NY: Penguin Press, 2018), 256.
[xiv] Stephen Marrin, “Intelligence Analysis and Decisionmaking: Methodological Challenges,” in Peter Gill, Stephen Marrin and Mark Phythian (eds.) Intelligence Theory: Key Questions and Debates (Abingdon: Routledge 2008), 131–50; Paul R. Pillar, Intelligence and U.S. Foreign Policy: Iraq, 9/11, and Misguided Reform (New York, NY: Columbia University Press, 2006), 3; Uri Bar-Joseph and Rose McDermott, “The Intelligence Analysis Crisis,” in Loch K. Johnson, ed. The Oxford Handbook of National Security Intelligence (New York, NY: Oxford University Press, 2010), 359-374.
[xv] In Lewis Carrol’s Through the Looking Glass, the Red Queen explains to a confused Alice that in her kingdom, you must run twice as fast just to stay in the same place, let alone to get anywhere. Evolutionary biologists refer to the same phenomena in species competition as the ‘Red Queen effect.’ See Shane Parrish, “The Red Queen Principle: Avoid Running Faster to Stay in the Same Place,” Farnham Street, (October, 2012). Retrieved from https://www.fs.blog/2012/10/the-red-queen-effect/
[xvi] Stephen M. Trzaskoma, R. Scott Smith, and Stephen Brunet, eds. Anthology of Classical Myth: Primary Sources in Translation, (Indianapolis, IN: Hackett Publishing, 2004), 234.
[xvii] Cortney Weinbaum, John V. Parachini, Richard S. Girven, Michael H. Decker, Richard C. Baffa, “Perspectives and Opportunities in Intelligence for U.S. Leaders,” Perspectives (Washington, DC: The RAND Corporation, September 2018), 19. Retrieved from http://www.rand.org/t/PE287
[xviii] Phil Nolan, “A Curator Approach to Intelligence Analysis,” International Journal of Intelligence and Counterintelligence 25 (2012), 786. Retrieved from https://doi.org/10.1080/08850607.2012.678698
[xix] Cortney Weinbaum and John N.T. Shanahan, “Intelligence in a Data-Driven Age,” Joint Forces Quarterly 90 (3rd Quarter 2018), 6; Cheryl Pellerin, “Project Maven to Deploy Computer Algorithms to War Zone by Year’s End,” DOD News, (July 21st, 2017). Retrieved from www.defense.gov/News/Article/Article/1254719/project-maven-to-deploycomputer-algorithms-to-war-zone-by-yearsend/;
[xx] Sam Tanenhaus, “The DNA Problem in American Spying,” The New York Times (January 1st, 2010). Retrieved from http://www.nytimes.com/2010/01/03/weekinreview/03tanenhaus.html;
[xxi] Sherman Kent, Strategic Intelligence for American World Policy (Princeton, NJ: Princeton University Press, 1949), 183.
[xxii] David Reinsel, John Gantz, and John Rynding, “Data Age 2025,” International Data Corporation (April, 2017).
[xxiii] Kevin J. Obrien, “Talk to Me, One Machine Said to the Other,” The New York Times (July 29th, 2012). Retrieved from http://www.nytimes.com/2012/07/30/technology/talk-to-me-one-machine-said-to-the-other.html
[xxv] Phil Nolan, “A Curator Approach to Intelligence Analysis,” International Journal of Intelligence and Counterintelligence 25 (2012), 786. Retrieved from https://doi.org/10.1080/08850607.2012.678698
[xxvi] Robert Jervis, Why Intelligence Fails: Lessons from the Iranian Revolution and the Iraq War (New York, NY: Cornell University Press, 2011), 182.
[xxvii] Alex R. McQuade, “2016 Worldwide Threat Assessment of the US Intelligence Community,” Lawfare (February 12th, 2016). Retrieved from https://www.lawfareblog.com/2016-worldwide-threat-assessment-us-intelligence-community; Tim Castelle, “Few People Understand the Difference Between Risk and Genuine Uncertainty,” Business Insider (March 27th, 2013). Retrieved from http://www.businessinsider.com/difference-between-risk-and-uncertainty-2013-3
[xxviii] Bowman H. Miller, “Improving All-Source Intelligence Analysis: Elevate Knowledge in the Equation,” International Journal of Intelligence and Counterintelligence, 28/2 (June 1st, 2008), 339.
[xxix] Emily Dreyfuss, “Top U.S. Intelligence Official Sue Gordon Wants Silicon Valley on Her Side,” Wired (November 9th, 2018). Retrieved from https://www.wired.com/story/sue-gordon-us-intelligence-public-private-google-amazon/; Robert Cardillo, “Racing to Secure Our Future,” The Cipher Brief (May 22nd, 2018). Retrieved from https://www.thecipherbrief.com/column_article/racing-secure-future; Mark Pomerleau, “Here’s How Intelligence Agencies Will Take Advantage of Machine Learning and AI,” C4ISRNet (May 1st, 2018). Retrieved from https://www.c4isrnet.com/intel-geoint/2018/05/01/heres-how-intelligence-will-take-advantage-of-machine-learning-and-ai/;
[xxx] Mark Lowenthal, The Future of Intelligence (Cambridge, UK: Polity Press, 2018), 21-33.
[xxxi] Sydney J. Freedburg, Jr., “Big Bad Data: The Achilles Heel of Artificial Intelligence,” BreakingDefense (November 13th, 2018). Retrieved from https://breakingdefense.com/2018/11/big-bad-data-achilles-heel-of-artificial-intelligence/;
[xxxii]Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner, “Machine Bias,” ProPublica (May 23rd, 2016). Retrieved from https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing; Sascha Eder, “How Can We Eliminate Bias in Our Algorithms?” Forbes (June 27th, 2018). Retrieved from https://www.forbes.com/sites/theyec/2018/06/27/how-can-we-eliminate-bias-in-our-algorithms/#1eab5e3337eb; Louise Matsakis, “What Does a Fair Algorithm Actually Look Like?” Wired (October 11th, 2018). Retrieved from https://www.wired.com/story/what-does-a-fair-algorithm-look-like/;
[xxxiii] Jerome Groopman, How Doctors Think (New York, NY: Houghton Mifflin, 2008), 6.
[xxxiv] Andrew Ng, quoted in Remco Zwetsloot, Helen Toner, and Jeffrey Ding, “Beyond the AI Arms Race,” Foreign Affairs (November 16th, 2018). Retrieved from https://www.foreignaffairs.com/reviews/review-essay/2018-11-16/beyond-ai-arms-race?cid=soc-tw;
[xxxv] Bowman H. Miller, “Improving All-Source Intelligence Analysis: Elevate Knowledge in the Equation,” International Journal of Intelligence and Counterintelligence, 28/2 (June 1st, 2008), 339.
[xxxvi] Josh Kerbel, “The US Intelligence Community Wants Disruptive Change as Long as It’s Not Disruptive”, War on the Rocks, (January 20th, 2016). Retrieved from https://warontherocks.com/2016/01/the-u-s-intelligence-community-wants-disruptive-change-as-long-as-its-not-disruptive/; Phil Nolan, “A Curator Approach to Intelligence Analysis,” International Journal of Intelligence and Counterintelligence 25 (2012), 786. Retrieved from https://doi.org/10.1080/08850607.2012.678698
[xxxvii] Jeffrey R. Cooper, Curing Analytic Pathologies: Pathways to Improved Intelligence Analysis (Washington, DC: Center for the Study of Intelligence, December, 2005), 39; Alfred S. Hulnick “The Dilemma of Open Sources Intelligence: Is OSINT Really Intelligence?” in in Loch K. Johnson, ed. The Oxford Handbook of National Security Intelligence (New York, NY: Oxford University Press, 2010), 230-241.
[xxxviii] Cortney Weinbaum and John N.T. Shanahan, “Intelligence in a Data-Driven Age,” Joint Forces Quarterly 90 (3rd Quarter 2018), 6.
[xxxix] Jack Davis, “Sherman Kent and the Profession of Intelligence Analysis,” Studies in Intelligence, () 3.
[xl] Madeleine Bunting, “Disarming the Weapons of Mass Distraction,” The New York Review of Books (March 15th, 2018). Retrieved from http://www.nybooks.com/daily/2018/03/15/disarming-the-weapons-of-mass-distraction
[xli] Herbert Simon, “Designing Organizations for an Information-Rich World,” in Martin Greenberger, Computers, Communication, and the Public Interest (Baltimore, MD: Johns Hopkins University Press, 1971), 40-41, 176.
[xlii] Dana Priest and William H. Arkin, “Top Secret America,” The Washington Post, (July 19th, 2010). Retrieved from http://projects.washingtonpost.com/top-secret-america/articles/a-hidden-world-growing-beyond-control/; Brian Katz, “Intelligence and You: A Guide for Policymakers,” War on the Rocks (November 14th, 2018). Retrieved from https://warontherocks.com/2018/11/intelligence-and-you-a-guide-for-policymakers/;
[xliii] Jack Davis “A Policymaker’s Perspective on Intelligence Analysis,” Studies in Intelligence 38/5, (1994), 8. Retrieved from https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/95unclass/Davis.html
[xliv] Michael V. Hayden, The Assault on Intelligence (New York, NY: Penguin Press, 2018), 37.
[xlv] Thomas P. Malone, The Future of Work (Boston, MA: Harvard Business School Press, 2004), 140-145; Niolfer Merchant, “Why Porter’s Model No Longer Works,” Harvard Business Review, (February 29th, 2012). Retrieved from https://hbr.org/2012/02/why-porters-model-no-longer-wo?referral=03759&cm_vc=rr_item_page.bottom
[xlvi] Courtney Weinbaum, Richard Girven, and Jenny Olberholtzer, “The Millenial Generation: Implications for the Intelligence and Policy Communities,” The RAND Corporation (2016), 35-38.
[xlvii] Aoife McLaughlin, “Wired Society Speeds Up Brains and Time,” James Cook University (Queensland, AU: November, 2015). Retrieved from https://www.jcu.edu.au/news/releases/wired-society-speeds-up-brains-and-time
[xlviii] Tom Nichols, The Death of Expertise: The Campaign Against Established Knowledge and Why it Matters (New York, NY: Oxford University Press, 2017), 106; Matthew Finnegan, “IBM’s New Watson Assistant Targets Businesses with White Label Service,” Computer World (March 26th, 2018). Retrieved from https://www.computerworld.com/article/3265690/artificial-intelligence/ibms-new-watson-assistant-targets-business-with-white-label-service.html;
[xlix] George S. Pettee, The Future of American Strategic Intelligence (Washington, DC: Infantry Journal Press, 1946), 185-187.
[l] Sherman Kent, Strategic Intelligence for American World Policy (Princeton, NJ: Princeton University Press, 1949), 180.
[lii] See, Among Others: Carmen Medina "What To Do When Traditional Models Fail." Studies in Intelligence 46/3, (2001), 2. Retrieved from https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol46no3/article03.html; Carmen Medina, “The New Analysis,” in in Roger Z. George and James B. Bruce, eds. Analyzing Intelligence: Origins, Obstacles, and Innovations. (Washington, DC: Georgetown University Press, 2008), 239-248; Richard K. Betts, “Fixing Intelligence,” Foreign Affairs 81/1 (February 2002); Gregory Treverton, Reshaping National Intelligence for an Age of Information (Cambridge, UK: Cambridge University Press, 2003); Deborah G. Barger, Toward a Revolution in Intelligence Affairs (Santa Monica, CA: The RAND Corporation, 2005); John Michael McConnell, “Overhauling Intelligence,” Foreign Affairs 86/4 (August, 2007); Josh Kerbel “The US Intelligence Community’s Kodak Moment,” The National Interest, (May 15th, 2014). Retrieved from http://nationalinterest.org/feature/the-us-intelligence-communitys-kodak-moment-10463?page=show; Cortney Weinbaum, John V. Parachini, Richard S. Girven, Michael H. Decker, Richard C. Baffa, “Perspectives and Opportunities in Intelligence for U.S. Leaders,” Perspectives (Washington, DC: The RAND Corporation, September 2018). Retrieved from http://www.rand.org/t/PE287
[liii] Michael Warner and J. Kenneth McDonald, “US Intelligence Community Reform Since 1947,” Center for the Study of Intelligence, (2005), 6.
[liv] James Kitfield, “Rethinking Intel in the Age of Trump: DNI Dan Coats and PDDNI Sue Gordon,” Breaking Defense (March 21st 2018). Retrieved from https://breakingdefense.com/2018/03/exclusive-rethinking-intel-in-the-age-of-trump-dni-coats-pdni-gordon/;
[lv] Francis Fukuyama, The Origins of Political Order: From Pre-Human Times to the French Revolution (New York, NY: Farrar, Strauss, & Giroux, 2011), 9.