Can you help ONA do a better job?

I attended a fascinating roundtable last week on ‘open-source intelligence’ at the Office of National Assessments (ONA), a body whose mandate to ‘provide all-source assessments on international political, strategic and economic developments to the Prime Minister’.  As a recent US report describes the topic:

Open source information (OSINT) is derived from newspapers, journals, radio and television, and the Internet. Intelligence analysts have long used such information to supplement classified data, but systematically collecting open source information has not been a priority of the U.S. Intelligence Community. In recent years, given changes in the international environment, there have been calls, from Congress and the 9/11 Commission among others, for a more intense and focused investment in open source collection and analysis. However, some still emphasize that the primary business of intelligence continues to be obtaining and analyzing secrets.

As a tech-savvy bunch of readers, let me ask the question: if you were in the business of using publicly available data to fulfill ONA’s mandate, are there any novel or unusual sources that you would draw upon? This includes sources that would provide new information, as well as those that might help sift the wheat from the chaff.

(xposted @ andrewleigh.com)

11 thoughts on “Can you help ONA do a better job?”

  1. However, some still emphasize that the primary business of intelligence continues to be obtaining and analyzing secrets.
    Perhaps I’m an ignorant git, but surely the key job of a foreign  intelligence agency (leaving aside covert action for a moment) is to provide information that it useful for the government in its foreign policy/defence goals.  Whether the information comes from a modern-day Mata Hari, or the New York Times, is very much a second-order issue.
    As to where you might want to look, who knows?  University and academic publications are a potential goldmine.  Scientists who do research with military applications, who suddenly disappear, or whose grad students disappear, is one example.
     

    Like

  2. Along similar lines to the “missing people” is looking at job advertisements. In my previous role this was one way of keeping tabs on what competitors are up to. Indeed, there was a good example of this recently when Unwired/Vivid advertised for WiMAX engineers in Perth and not long after announced the plans for Vivid’s rollout.

    Whilst UNwired wasn’t named as the recruitign company, anyone familiar with the industry should have been able to narrow it down.

    Another source I like to use is informaiton on tenders and govt contracts… though this should be pretty obvious and I’d imagien this is standard fare for mil/int agencies.

    Like

  3. Another possibility, partly of a crowdsourcing bent  – find internet fora that post on areas of interest to the intelligence agency, and get involved in the discussions 🙂
    Yes, there is the signal-to-noise factor, and there’s the risk you’ll give something away, but it’s a chance to get some of your research and analysis done for you.

    Like

  4. A fascinating idea – and one that seems to be very worthy.  Most of the information sources are well known.
    Given these sources and the quality and volume of these data the only way to comprehensively sift, sort, mine, analyse and prioritise it – and to extract the intelligence gems – is by using some very smart technology.
    Fortunately the technology exists to do the job – the problem is that the nation doesn’t have the skills because we have deployed our brains trust into the property game or the mining boom – or let them wander oiffshore.
    If we were to resurrect the concept of *the clever country* and put appropriate resource into developing that then we might have the people to do the job – in a decade ot two.
    Only question is – can ONA wait that long?

    Like

  5. Two thoughts:

    (1) I use Google Trends and Google Insights data all the time (and Google Analytics, for that matter). While this is aggregated information, it does provide unique insights into changes in the momentum of traffic over any time period in any given geography (eg, people searching in Brisbane, Australia) surrounding (a) any chosen search term or (b) any website (eg, Al Jazeera) subject to their being a minimum volume of traffic. In fact, one potentially quite cool idea would be to pay Google to construct special “intelligence indices” (ie, more focused than that which you can get from Google Trends/Insights) that track changes in key search terms from specific locations and/or traffic to particular websites as a proxy for shifts in activity and/or public sentiment; and

    (2) Why not hold semi-open source think-tank sessions with academic and, crucially, practitioner subject matter experts to provide the agencies with an expanded yet flexible human resource base upon which to draw (perhaps this is what Andrew participated in). You could structure these sessions to avoid revealing classified information. But the agencies would be silly to think that they have a monopoly on intelligence horsepower. They could even pay participants for their time and sign them up to NDAs to the extent there were concerns about information dissemination. Again, they might already do this, but the bureaucracy does have a tendency to be rather closed.

    Like

  6. United States’ experience is instructive in understanding the costs of non-disclosure of sources, selective release of information, maintaining secrecy and distorting resource allocations through unrealistic risk assessments in matters of public safety. In proposing the Protection and Reduction of Government Secrecy Act of 1994, the United States Congress noted that extensive government secrecy had developed in government in relation to the Cold War in particular.[1] This Act authorised the formation – and funding for two years – of a bi-partisan Commission on Protecting and Reducing Government Secrecy. The enabling statute recorded Congressional findings as follows: (1) During the Cold War an extensive secrecy system developed which limited public access to information and reduced the ability of the public to participate with full knowledge in the process of governmental decision making. (2) In 1992 alone 6,349,532 documents were classified and approximately three million persons held some form of security clearance. (3) The burden of managing more than 6 million newly classified documents every year has led to tremendous administrative expense, reduced communication within the government and within the scientific community, reduced communication between the government and the people of the United States, and the selective and unauthorized public disclosure of classified information. (4) It has been estimated that private businesses spend more than $14 billion each year implementing government mandated regulations for protecting classified information. (5) If a smaller amount of truly sensitive information were classified the information could be held more securely. (6) In 1970 a Task Force organized by the Defense Science Board and headed by Dr. Frederick Seitz concluded that “more might be gained than lost if our Nation were to adopt — unilaterally, if necessary—a policy of complete openness in all areas of information.” (7) The procedures for granting security clearances have themselves become an expensive and inefficient part of the secrecy system and should be closely examined. (8) A bipartisan study commission specially constituted for the purpose of examining the consequences of the secrecy system will be able to offer comprehensive proposals for reform.[2] The Commission reported to the Senate in 1997. On issues of national security, the Commission’s report noted especially the mistake in thinking that classified information was necessarily more reliable than openly available information. It argued that the US might have been better served by more open analysis and debate, given the considerable costs of consistently overestimating the Soviet Union as a threat. The taking down of the Red Flag over the Kremlin for the last time on Christmas Day 1991 marked the formal collapse of the Union. Official analysis failed to see inherent tensions within the former Union and foresee emerging conflict along ethnic lines among former states.[3] 4.1.1.1 [1] Commission on Protecting and Reducing Government Secrecy, Report to the Senate, Document 105-2 pursuant to Public Law 236, 103rd Congress, United States Government Printing Office: Washington DC, 1997. Documents accessible at <http://www.gpo.gov/congress/commissions/secrecy/index.html> [2] ibid., Appendix B, Protection and Reduction of Government Secrecy Act, Public Law 103-236, 30 April 1994, accessed at URL <http://www.gpo.gov/congress/commissions/secrecy/pdf/13stat1.pdf> [3] ibid., Appendix A, A brief account of the American experience, pp.A73-75, accessed at URL <http://www.gpo.gov/congress/commissions/secrecy/pdf/12hist1.pdf>

    Like

  7. Chris, it’s my understanding that at least some intelligence agencies already consult outside experts on a regular basis.
    That said, you put smart people in a room and start asking them to discuss a topic and just by the nature of the questions you ask you’re likely to disclose information to them.

    Like

  8. RM: I would have thought that it would be very easy to carefully construct ostensibly anodyne questions that nevertheless force participants to drill into specific subjects that would be of interest to the agencies. In my experience the bureaucracy is highly insular and risk-averse. I would be surprised if they engaged in genuinely open-source cross-fertilisation—or thought experiments/simulations—with independent third-parties (as opposed to consulting experts on telescoped topics of interest). Perhaps I am being heroic here…And perhaps there is little to be gained from such activities.

    Like

  9. Chris, as you’ve already stated the people you’re asking for help are people a) with background knowledge in the area of concern, and b) analytical skills and an inquisitive nature.
    If they’re not going to actively try and figure out what prompted ONA to want to chat to them, I’ll eat my hat.
    That said, a lot of the time no particular brilliance in searching for clever sources is required.  If I recall correctly, there was a story in Crikey about an ONA analyst who wrote a report for the government about climate change and its strategic consequences.  Nothing remarkable about that, except it was done in 1982.
    But then, the fundamentals of the science were well established in the academic literature by then.  All that was required was an analyst who read and could understand the basic science, and conduct a fairly simple thought experiment as to the consequences.

    Like

  10. A big challenge is synthesis of significant relationships from a large mass of data. With hindsight a number of occasions have been discovered where 9/11 terrorists came to the notice of US authorities. Whether with smarter synthesis the attack could have been averted we will never know.
    There may be enhancements that can be made to social network mapping tools that map not just on who communicates with whom, but also linking people whose only known relationship is that, say, they all bought sacks of ammonium nitrate recently.
    I assume that intelligence agencies are already making extensive use of social network analysis. Some of the required data for building terrorist network structures comes from clandestine surveillance but much of the rest is public data (who came from the same Irish village and worshipped at the same church or drank at the same pub?)
    Even so it does not necessarily lend itself to crowd-sourcing. You don’t want to put into the public arena the question, “Which pub does Paddy O’Toole drink at these days and who else regularly goes there?”

    Like

  11. The community of technically-minded arms control & anti-nuke advocates do a great job of digging up open-source data of interest in the WMD area – for example, a lot of interesting data and ideas are kicked around at ArmsControlWonk.

    Like

Comments are closed.