Decoding Official Secrecy: Computational Analysis of Hundreds of Thousands of Declassified Documents
The Robert S. Strauss Center for International Security and Law welcomed Dr. Matthew Connelly on November 5, 2013 to discuss the nature and origins of official secrecy and the declassification process. Dr. Frank Gavin, Director of the Strauss Center, introduced Dr. Connelly, who spoke about his current project, the Declassification Engine. The project is a collaborative effort between historians, data scientists, and other stakeholders, which attempts to use data analytics to produce new tools to address, process, and analyze the growing corpus of official secrets.
In his talk, Decoding Official Secrecy: Computational Analysis of Hundreds of Thousands of Declassified Documents, Dr. Connelly discussed the history and evolution of the classification process, which he argued is currently experiencing a bottleneck. Due to the long crisis of over-classification, recent decades have seen more and more secrets created while fewer documents are being declassified. Dr. Connelly explained that due to the increased volume of secrets, it is becoming harder both for the government and historians who use this material, to identify what is important and process the information.
Dr. Connelly’s current project, the Declassification Engine, attempts to develop tools to confront this problem. The project applies advances in machine learning and natural language processing to discover patterns in declassified texts, in the hopes of identifying larger patterns in official secrecy. More broadly, Dr. Connelly hopes the project will result in useful tools that historians and others can use in the future to deal with Big Data.
The√Ǭ†New Yorker recently published an article on Connelly’s Declassification Engine project, which can be found online, here. Connelly further discusses the project in a brief video interview, below:
Matthew Connelly, professor, works in international and global history. He received his B.A. from Columbia (1990) and his Ph.D. from Yale ( 1997). His publications include A Diplomatic Revolution: Algeria’s Fight for Independence and the Origins of the Post-Cold War Era (2002), and Fatal Misconception: The Struggle to Control World Population (2008). He has written research articles in Comparative Studies in Society and History, The International Journal of Middle East Studies, The American Historical Review, The Review francaise d’histoire d’Outre-mer, and Past & Present. He has also published commentary on international affairs in The Atlantic Monthly and The National Interest.