The subject of bias in intelligence analysis has been widely studied and a lot of effort has gone into education and quality assurance. Can open source intelligence (OSINT) mitigate bias in traditional analysis or does it come with its own challenges?
There has been potential for bias in conventional (closed source) classified intelligence in a number of places:
Relying on a single source, however special, is like looking at the world through a straw. The resulting analysis can yield an incomplete picture or shaped perspective.
We can find ourselves looking in the wrong place for the right information. Like searching for answers about transnational criminal groups cyber hacking Canadian critical infrastructure from Asia when the only collection available is from high arctic high-frequency radio sensors monitoring Russian military manoeuvres. Re-tasking and pivoting conventional collection can be a bit of an extreme sport and not as agile as open source intelligence (OSINT).
For a risk-sensitive organization, there is always a temptation to collect what is easy rather than what is necessary. Without the strict client-driven-requirements and feedback, there is a tendency to push intelligence to potential consumers without checking if it is actionable, relevant or worth the price. Many public sector consumers are getting intel for free, so may not provide the same feedback they would if they were paying the full price.
Mandates create bias in that they limit the types of sources and methods a government agency may use or restrict the location, jurisdiction or topic of collection. What if a threat actor is a paramilitary transnational criminal organization operating globally who are implicated in state-sponsored espionage disinformation, cyber warfare, crime, terrorism and war crimes. This crosses all jurisdictions that it falls across multiple mandates but not single lead agency. The fear of transgressing mandates often means missed coverage.
Conversely, a mandate may drive an agency to produce intelligence for which there is no current client interest because trends and threats change faster than legislation.
There is also a practice of consolidating or white-labeling[1] intelligence from other producers especially from partners, commercial, academic and open sources by removing references. Classification is a form of branding insomuch as it is a security handling label. Such a repackaging practice can unduly elevate the veracity of the intelligence by creating feedback loops and making it appear that multiple sources are confirming the same thing. I have seen similar phenomena in the media where major news stories have been build-up on a single tweet by some random person online. We also see this when journalists interview other journalists and build up a story. It is critical that all information in an intelligence report be fact checked and referenced to primary sources.
That brings is to foreign bias. The vast quantity of classified intelligence consumed by Canada comes from foreign sources and is ingested without question. The same is true for OSINT and news media. We have seen a trend from secret intelligence that has spilled into OSINT involving political partisanship or even subtle colouring of the analysis based upon cultural and social norms - left or right and liberal or conservative.
One also needs to consider that the foreign information was collected under different legal and ethical frameworks perhaps inconsistent with Canadian values or law. Conversely, something could be legal in Canada but not in other jurisdictions, and you can have your favorite source suddenly shut down, but you will still be billed. We are seeing this happen right now.
Subscribing to foreign data sources, tasking providers or using foreign managed attribution systems represents an unnecessary exposure to one’s operational security especially where trusted data brokers and vetted sovereign capability exists. Your supply chain is susceptible to foreign ownership, control and influence especially if the adversary gets inside that chain through surreptitious means or mergers and acquisitions.
Using foreign suppliers establishes critical dependency that may not be there when you need it, or you only get left over answers to similar questions allies have asked. Foreign providers working in much more lucrative markets may not entertain Canada’s unique collection priorities. An then there are ITAR and no-foreign restrictions that come with many high-end commercial and OSINT data products and tools.
Finally, there is the risk of politicizing the reporting. Either by only reporting what leadership wants to hear, or hyping a threat to justify action or fund programs. This occurs less often these days but there is always a risk of speaking truth to power.
Most politicizing still happens by political staff. Cherry picking intelligence to fit an agenda or taking narrative out of context, compiling DIY intelligence using Google or ChatGPT, making stuff up or ignoring good intelligence all together. We are currently watching an highly-politicised debate around Foreign Interference here in Canada.
There is an unfortunate perception amongst some that Open Source Intelligence is less reliable than traditional closed sources. I have heard statements like OSINT is all hearsay or “free is worth what you pay for it.”
The truth is that 90% of classified reporting is already padded with or informed by open source material. The wide-aperture and diverse nature of OSINT mitigates much of the biases we spoke about because OSINT is all-domain and multi-source. It is not driven by a single collection method, target or mandate.
The manner in which one validates and verifies the reliability of the source and information remain the same whether one uses classified or open sources. The thing is in the classified world, often the reliability of the information is taken at face value by the analyst because it has already been labeled top secret and appears on the system.
The notion that commercial intelligence can’t be trusted and is intentionally biased, because companies are only interested in profits, is generally without substance.
Commercial intelligence companies only produce intel that someone is willing to pay for. Furthermore, clients pay for relevant facts and informed analysis. Intelligence must be unique, timely, accurate and cost effective. It is subject to verification of the facts, quality assurance function and assessment of value by the consumer. Mess up on this and you don’t get repeat business. There is market competition, checks and balances which keep the process fair, honest and fugal.
On the other hand, getting open raw information directly off social media and news channels needs to be scrutinized. Social media is ripe with misinformation. Media companies themselves are more predisposed to bias because they are in the business of selling ads and chasing ratings for revenue - printing big headlines and tuning content to play to audiences. Herein, there is a danger of popularized investigative reporting turning into tabloid journalism or misinformation for entertainment – like history channel broadcasting shows about Ancient Aliens and other conspiracy theories.
Finally, monocultures are fertile ground for analytical bias. It is critical that we create diversity in the analytical team to include: age, gender, experience, ethnicity, language, culture, education, open world views. We ought to look for qualities of intelligence, critical thinking, honesty, integrity, empathy, communications skills and neutrality. We need an internal challenge function within the analytical cell.
My conclusion is that open source and commercial intelligence have the potential to significantly reduce bias, when sound analytical practices are followed.
AUTHOR
Dave McMahon is the Chief Intelligence Officer of Sapper Labs Group (SLG). He has forty-years experience in the Intelligence enterprise.
Sapper Labs Group (SLG) is an all-source Intelligence Company providing Open Source Intelligence (OSINT) technology, training, talent and finished intelligence analysis for human rights investigations, national security, defence, law enforcement, critical infrastructure protection, privacy, peace and prosperity.
[1] https://www.latimes.com/archives/la-xpm-2003-feb-08-fg-plagiarism8-story.html [2] https://www.rand.org/blog/2021/04/the-intelligence-communitys-deadly-bias-toward-classified.html