Just before the release in June of the much-anticipated Pentagon report on unidentified aerial phenomena (UAP), I sat down to try to create a list of the greatest hurdles to UAPs' scientific analysis. What I came up with were five major challenges that are described here, together with a cross-comparison with some of the statements made in the published government report. Although only nine pages long, that report turns out to be thorough, careful and scientifically accurate in that it fully expresses how little certainty can be drawn from the data to hand. As the saying goes: the more things change, the more they stay the same.
Challenge No. 1: All UAP/UFO incidents are nonrepeatable: we can’t go back and perform the “experiment” of that exact observation again.
For science in general, this kind of thing is a big headache. A lack of repeatability or replication poses a very significant challenge for the interpretation of data (especially if those data are noisy and incomplete); for filling in obvious gaps; and for eliminating or supporting any hypotheses. As the Pentagon report states: “Limited data leaves most UAP unexplained….” Limited, anecdotal and nonrepeatable are hardly the words you want to use, but they apply here.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Challenge No. 2: There is nothing systematic in how incidents are recorded or reported. Different camera systems, radar systems, data processing, observers and environmental circumstances mean that each incident is, in effect, an uncontrolled experiment, with few ways to ascertain the real quality and sensitivity of data.
Again, the Pentagon report states effectively the same point: “The limited amount of high-quality reporting on unidentified aerial phenomena (UAP) hampers our ability to draw firm conclusions about the nature or intent of UAP.” The report then goes on to suggest a potentially useful task of: “Consistent consolidation of reports from across the federal government, standardized reporting, increased collection and analysis, and a streamlined process for screening.”
This is really important; the report is very, very specific about the lack of appropriateness of typical military sensor equipment for this sort of analysis. “The sensors mounted on U.S. military platforms are typically designed to fulfill specific missions. As a result, those sensors are not generally suited for identifying UAP.”
Challenge No. 3: There is no easy way to account for “cherry-picking” of data. We don’t know how often pilots or other observers see something unexpected but then, a minute later, figure out what they’re witnessing (or at least convince themselves they’ve done so) and consequently don’t report anything. There could be thousands of such incidents, or very few. We don’t know, and those “mundane” cases could actually represent all cases.
The report does discuss the “stigma” surrounding personnel or observers reporting UAPs, but it also states that out of the 144 reports that were studied, only 18 incidents (covered in 21 of the reports) appeared to demonstrate “advanced technology,” inasmuch as there was an appearance of unusual aeronautical behavior in movement.
In a small (unspecified) number of cases there was even evidence of military aircraft systems “processing radio frequency (RF) energy”—whatever that really means; presumably there was some increased radio noise. But, as for all the times that nothing was reported, either because something was quickly identified, or a pilot just chose not to, that remains a total unknown.
Challenge No. 4: If any incidents or observations are genuinely associated with something tangible and physical, we don’t know whether we’re looking at a single underlying phenomenon or many. It’s a bit like going into a zoo blindfolded and trying to understand what you’re hearing and smelling. If there’s only one species you might figure it out, but if there are 100 species, then decoding your experience is going to be very difficult.
Again, the report hits this nail right on the head, with an entire section titled “UAP probably lack a single explanation.” Some of the possibilities offered are: “Airborne clutter … birds, balloons, recreational unmanned aerial vehicles … debris like plastic bags … that muddle a scene,” as well as natural atmospheric phenomena (ice crystals, thermal fluctuations that can register on infrared and radar systems), classified aircraft and the like, and foreign “adversary systems.”
The Pentagon report also provides an outline of ongoing efforts, and possible future directions, for trying to improve all analyses. This includes a more systematic collection of military aircraft sensor data, along with FAA data, and applying machine learning to sift through current and historical information to look for “clusters,” patterns and associations with known phenomena like weather balloons, wildlife movements and other Earth-monitoring databases.
Challenge No. 5: The popular association of UAP with hypotheses involving alien technology creates a severe analysis bias. Usually, science tries to move stepwise towards finding support for a given hypothesis or for eliminating hypotheses, and weighs those options as evenly as possible. But in this case a hypothesis that would require extraordinarily robust evidence in order to be supported (as with Carl Sagan’s famous dictum “Extraordinary claims require extraordinary evidence”), regardless of what some people say, hangs heavily over any analysis or discussion, and there is a vocal community who feel that the answer is already known. That’s a problem.
In fact, and rather ironically, the “sociocultural stigmas” around recording surprising observations mentioned in the report are undoubtedly exacerbated by elements of the UFO community that express ideas or beliefs that are, well, fantastical in nature.
Consequently, observers such as highly trained, professional pilots are likely going to be reticent to mention things they are very surprised by. This relates to point No. 3 and creates bias because the unreported incidents, if further analyzed, could provide significant insight—especially as to how often human observers are simply confused, as opposed to witnessing genuinely unusual phenomena.
Where does all of this leave us? Well, the Pentagon report does suggest ways to improve data collection and analysis, as I’ve described. It also points out that if some UAP do represent physical hazards, or security challenges, it would be important to figure that out. In that sense, there is some possible risk mitigation to be had by investigating UAP further, irrespective of an eventually mundane or extraordinary explanation.
As a scientist who studies the possibilities of life elsewhere in the cosmos, I find myself saying “Well, it seems worth having some more work done on this.” But that’s not because I think it’s likely that extraterrestrials or their probes could be dropping into Earth’s atmosphere. Although as a rational thinker I can’t, and shouldn’t, permanently exclude such possibilities, my point No. 5 bothers me enough that I’d rather follow the stepwise approach. There are other benefits to that strategy too.
In particular, I think that the idea of a vastly more systematic collection of data (from things like state-of-the-art camera systems placed on aircraft or in monitoring locations) would be an interesting activity regardless of what is actually taking place in our skies.
New kinds of high-resolution time-lapse data and high-fidelity monitoring of our planetary environment could have many additional benefits as we try to navigate our way through a perilously changing world. From atmospherics to animal migration to human-generated garbage floating in the air and on the sea, seeing what’s actually going on is always going to help.
This is an opinion and analysis article; the views expressed by the author or authors are not necessarily those of Scientific American.