The pursuit of objectivity
There has been controversy recently over the use of spectrograms to analyse the spectral (frequency) features of birdsong. This greatly unsettled me. Using spectrograms to analyse birdsong is the basis of my PhD. A spectrogram is a visual representation of sound and has been used for decades to analyse the characteristics of animal (and human) vocalisations. But when it is used in the field, there are certain aspects to that use that require a great deal of care. Field conditions cannot be standardised in the same way that laboratory conditions can be. The weather, ambient noise, the sounds of other animals, wind, the rustle of vegetation, all can distort or mask the sound of interest. Furthermore, the bird of interest, the focal individual as we say in the business, may be at varying distances, orientations or heights, so all these factors contribute to a less than perfect recording situation, which could potentially undermine valid comparisons between individuals of the same species, never mind individuals of different species.
Fortunately, my work has been experimental, so the treatment I expose the birds to, is, at the very least, the greatest if not the only immediate change in their environment during the recording session. Also, the acoustic analysis software that I use is nothing short of miraculous, in its ability to adjust to varying quality of recordings while nonetheless allowing a fair comparison between them. However, there is inevitably a degree of subjectivity to the assessment. Firstly, the software has to be set up, its boundaries of acceptability and sensitivity are decided and determined upon by a human, ie, me. Secondly, even with the software being set up to near perfection for a given set of recordings, it still can’t reliably distinguish the differences in species or individual song, especially when birds sing over the top of each other. That depends on my visual recognition system, not the computer’s. However, these issues can be addressed statistically as well as by the precaution of regularly getting my assessments checked. Additionally, with another student, I’ve also run validation experiments on the accuracy of the assessment process. In other words, in keeping with the literature, I’ve put into practice all the precautions I possibly can to ensure that I’m measuring what I’m supposed to and to allow a valid comparison across recordings.
Last week there was a bombshell. A paper was published taking issue with the objectivity of such measurements. While a good deal of what the authors said made sense, and may be, in the strictest sense, correct, it also called into question just the sort of “measuring” issues I’ve been dealing with. I felt scared and somewhat defeated. It boiled down to an issue that has actually plagued me for much of my life: does a thing have to be perfect to be good enough? Does measurement, assessment, have to be so specifically spot-on, that any slippage renders the whole thing meaningless? Meaning. Reality. It comes down to that. What is statistically meaningful? What is biologically meaningful? What is real? What actually matters?
This being me, I lost sleep. I worried that the entire basis of my PhD was a nonsense. Until this week. Hey la, a reply has been published, which while conceding some minor points, is wholly robust in arguing back. And utterly convincing. Sighs of relief all round. Fieldworkers in avian acoustics world-wide – relax! It’s ok. We can still rise in the pre-dawn! We can deal with the shortcomings. We can deal with the ambiguities. With the right care, consistency, and by addressing the limitations appropriately, we can still formulate hypotheses, get data, find meaning within it. Asking the right questions, we can still get good enough answers.