Full Width Blog

On 26 February 2015, Cates Holderness, a BuzzFeed community manager, posted a picture of a dress, captioned: ‘There’s a lot of debate on Tumblr about this right now, and we need to settle it.’ The post was accompanied by a poll that racked up millions of votes in a matter of days. About two-thirds of people saw the dress as white and gold. The rest, as blue and black. The comments section was filled with bewildered calls to ‘go check your eyes’ and all-caps accusations of trolling.

Vision scientists were quick to point out that the difference in appearance had to do with the ambiguity of ambient light in the photograph. If the visual system resolved the photograph as being taken indoors with its warmer light, the dress would appear blue and black; if outdoors, white and gold. That spring, the annual Vision Sciences Society conference had a live demo of the actual dress (blue and black, for the record) lit in different ways to demonstrate the way the difference of ambient light shifted its appearance. But none of this explains why the visual systems of different people would automatically infer different ambient light (one predictive factor seems to be a person’s typical wake-up time: night owls have more exposure to warmer, indoor light).

Whatever the full explanation turns out to be, it is remarkable that this type of genuine difference in visual appearance could elude us so completely. Until #TheDress went viral, no one, not even vision scientists, had any idea that these specific discrepancies in colour appearance existed. This is all the more remarkable considering how easy it is to establish this difference. In the case of #TheDress, it’s as easy as asking ‘What colours do you see?’ If we could be oblivious to such an easy-to-measure difference in subjective experience, how many other such differences might there be that can be discovered if only we know where to look and which questions to ask?

Take the case of Blake Ross, the co-creator of the Firefox web browser. For the first three decades of his life, Ross assumed his subjective experience was typical. After all, why wouldn’t he? Then he read a popular science story about people who do not have visual imagery. While most people can, without much effort, form vivid images in their ‘mind’s eye’, others cannot – a condition that has been documented since the 1800s but only recently named: aphantasia. Ross learned from the article that he himself had aphantasia. His reaction was memorable: ‘Imagine your phone buzzes with breaking news: WASHINGTON SCIENTISTS DISCOVER TAIL-LESS MAN. Well, then, what are you?’

Ross went on to ask his friends about what it’s like for them when they imagine various things, quickly realising that, just as he took his lack of imagery as a fact of the human condition, they similarly took their presence of visual imagery as a given. ‘I have never visualised anything in my entire life,’ Ross wrote in Vox in 2016. ‘I can’t “see” my father’s face or a bouncing blue ball, my childhood bedroom or the run I went on 10 minutes ago… I’m 30 years old, and I never knew a human could do any of this. And it is blowing my goddamn mind.’

There is a moral imperative for us to study and understand these kinds of differences

There is a kind of visceral astonishment that accompanies these types of hidden differences. We seem wedded to the idea that we experience things a certain way because they are that way. Encountering someone who experiences the world differently (even when that difference seems trivial, like the colour of a dress) means acknowledging the possibility that our own perception could be ‘wrong’. And if we can’t be sure about the colour of something, what else might we be wrong about? Similarly, for an aphantasic to acknowledge that visual imagery exists is to realise that there is a large mismatch between their subjective experiences and those of most other people.

Studying hidden differences like these can enrich our scientific understanding of the mind. It would not occur to a vision scientist to ask whether being a night owl might have an impact on colour perception, but a bunch of people on the internet comparing notes on how they saw a dress inspired just such a study. The study of aphantasia is helping us understand ways in which people lacking imagery can accomplish the same goals (like remembering the visual details of their living room) without using explicit imagery. How many other such examples might there be once we start looking? There is also, arguably, a moral imperative for us to study and understand these kinds of differences because they help us understand the various ways of being human and to empathise with these differences. It’s a sobering thought that a person might respond differently to a situation not just because they have a different opinion about what to do or are in possession of different knowledge, but because their experience of the situation is fundamentally different.

For most of my research career, I didn’t really care about individual differences. Like most other cognitive scientists, my concern was with manipulating some factor and looking to see how this manipulation affected the group average. In my case, I was interested in the ways that typical human cognition and perception is augmented by language. And so, in a typical experiment, I would manipulate some aspect of language. For example, I examined whether learning names for novel objects changed how people categorised, remembered and perceived them. These were typical group-effect studies in which we compare how people respond to some manipulation. Of course, with any such study, different people respond in different ways, but the focus is on the average response.

For example, hearing ‘green’ helps (most) people see the subtle differences between more-green and less-green colour patches. Interfering with language by having people do a concurrent verbal task makes it harder for (most) people to group together objects that share a specific feature, such as being of a similar size or colour. But most people aren’t everyone. Could it be that some people’s colour discrimination and object categorisation is actively aided by language, but other people’s less so? This thought led us to wonder if this could be another hidden difference, much like aphantasia. In particular, we began to look at inner speech, long thought to be a universal feature of human experience.

Most people report having an inner voice. For example, 83 per cent (3,445 out of 4,145 people in our sample) ‘agree’ or ‘strongly agree’ with the statement ‘When I read I tend to hear a voice in my mind’s ear.’ A similar proportion – 80 per cent – ‘agree’ or ‘strongly agree’ with the statement ‘I think about problems in my mind in the form of a conversation with myself.’ This proportion goes up even more when asked about social problems: 85 per cent ‘agree’ or ‘strongly agree’ with the statement ‘When thinking about a social problem, I often talk it through in my head.’

On average, those who report having more visual imagery also report experiencing more inner speech

But 85 per cent is hardly everyone. What about those who disagree with these statements? Some of them report experiencing an inner voice only in specific situations. For example, when it comes to reading, some say that they hear a voice only if they deliberately slow down or are reading something difficult. But a small percentage (2-5 per cent) report never experiencing an inner voice at all. Like those with aphantasia who assume their whole lives that visual imagery is just a metaphor, those with anendophasia – a term Johanne Nedergaard and I coined to refer to the absence of inner speech – assume that those inner monologues so common in TV shows are just a cinematic device rather than something that people actually experience. People with anendophasia report that they never replay past conversations and that, although they have an idea of what they want to say, they don’t know what words will come out of their mouths until they start talking.

It is tempting to think that there is a trade-off between thinking using language and thinking using imagery. Take the widespread idea that people have different ‘learning styles’, some being visual learners and others verbal learners (it turns out this idea is largely incorrect). When it comes to imagery and inner speech, what we find is a moderate positive correlation between vividness of visual imagery and inner speech. On average, those who report having more visual imagery also report experiencing more inner speech. Most who claim to not experience inner speech also report having little imagery.

This raises the question of what their thoughts feel like to them. When we have asked, we tend to get answers that are quite vague, for example: ‘I think in ideas’ and ‘I think in concepts.’ We have lots of language at our disposal that we can use to talk about perceptual properties (especially visual ones) and, of course, we can use language to talk about language. So it is not really surprising that people have trouble conveying what thoughts without a perceptual or linguistic format feel like. But the difficulties in expressing these types of thoughts using language don’t make them any less real. They merely show that we have to work harder to better understand what they are like.

Differences in visual imagery and inner speech are just the tip of the iceberg. Other hidden differences include synaesthesia, Greek for ‘union of the senses’, in which people hear lights or taste sounds, and Eigengrau, a German word for the ‘intrinsic grey’ we see when we close our eyes. Except not all of us experience Eigengrau. About 10 per cent in our samples claim their experience is nothing like Eigengrau. Instead, when they close their eyes, they report seeing colourful patterns or a kind of visual static noise, like an analogue TV not tuned to a channel.

Our memory, too, seems to be the subject of larger differences than anyone expected. In 2015, the psychologist Daniela Palombo and colleagues published a paper describing ‘severely deficient autobiographical memory’ (SDAM). A person with SDAM might know that they went on a trip to Italy five years ago, but they cannot retrieve a first-person account of the experience: they cannot engage in the ‘mental time travel’ that most of us take for granted. As in other cases of hidden differences, these individuals tend not to realise they are unusual. As Claudia Hammond wrote for the BBC about Susie McKinnon, one of the first described cases of SDAM, she always ‘assumed that when people told in-depth stories about their past, they were just making up the details to entertain people.’

What is it about differences in imagery, inner speech, synaesthesia and memory that render them hidden? It is tempting to think that it’s because we don’t directly observe them. We can see that someone is a really fast runner. But having direct access only to our own reality, how are we to know what another person imagines when they think of an apple, or whether they hear a voice when they read? Still, while we can’t directly experience another person’s reality, we can compare notes by talking about it. Often, it’s remarkably easy: for #TheDress, we just needed to ask one another what colours we see. We can also ask whether letters always appear in colour (a grapheme-colour synaesthete will say yes; others will say no). People without imagery will tell you they cannot visualise an apple, and those without inner speech will say they do not have silent conversations with themselves. It is not actually difficult to discover these differences once we start systematically studying them.

Paradoxically, although language is what allows us to compare notes and learn about differences between our subjective experiences, its power to abstract may also cause us to overlook these differences because the same word can mean many different things. We use ‘imagine’ to refer to forming an image in the mind’s eye, but we also use it when referring to more abstract activities like imagining a hypothetical future. It is perfectly reasonable for an aphantasic to not realise that, in some cases, people use ‘imagine’ to mean actually forming mental images that have a perceptual reality.

Much of our understanding of hidden differences relies on people’s self-report. Can we trust it? Modern psychology is sceptical about self-report, a scepticism I’ve inherited as part of my academic training. Recent reports of large individual differences in imagery and inner speech have often been accompanied by incredulity. How do we know that these differences reflect something real? Can we really just take people at their word when they say they don’t have an inner voice?

If the goal is to understand what a person feels, self-report trumps objective measurement

Before tackling the more complex question of whether we should trust self-reports about internal subjective states like imagery and inner speech, let’s consider some simpler cases. When someone says they dislike cauliflower, they are reporting on their subjective experience, and we tend to take them at their word. But we don’t have to. We can easily set up an experiment where we observe how likely they are to eat cauliflower when given alternatives. It would be surprising if someone claimed to not like cauliflower but chose to eat it at every opportunity. There are, of course, cases where such ‘stated-vs-revealed preference gaps’ occur. Many researchers have made their careers studying these gaps. For example, if one lives in a culture where cauliflower-eating is associated with higher status, people may be compelled to say they like it even though they don’t. Conversely, someone might eat cauliflower only to avoid offending their host. Such situations call for caution in interpreting people’s preferences – both stated and revealed – but they do not negate the observation that, in ordinary circumstances, taking people at their word regarding their preferences is a very good guide to their behaviour.

Let’s take another case. You are in a shared office and your office-mate says they feel cold when the thermostat is set to 72°F (22°C). Do you take them at their word, or do you say ‘But 72°F is the proper indoor temperature? How can you feel cold?’ Suppose you take measurements of their skin temperature, core temperature, even an fMRI scan showing activation of their insula. None of these would allow you to claim that they don’t feel cold. None of these measures would negate their self-report. If one was concerned about hypothermia, relying on objective measurements may well be appropriate but, if the goal is to understand what a person feels, self-report trumps objective measurement.

The same logic applies to other inherently subjective states such as loneliness, pain and awe. To measure loneliness, it is not sufficient to count how many people someone talks to or is friends with because one person’s active social life may be another person’s depth of loneliness. We can tell if there is a flu epidemic by using objective tests, but diagnosing a ‘loneliness epidemic’ requires taking into account whether people feel lonely. This is also why, despite all the available technology we have to measure people’s physiological states, when it comes to pain, we continue to rely on pain scales, a simple form of self-report. If we take introspective judgments seriously when it comes to preferences, emotion and pain, why would we be more sceptical about them in cases of phenomenal differences such as imagery and inner speech?

One possibility is that we are able to reliably introspect about some things and not others. Perhaps we can reliably report on ‘basic’ states like pain and whether we like cauliflower (though, even here, there may well be differences in people’s ability to self-report), but in other cases our introspection fails. For example, most people think they are above-average drivers – one of the many examples of the so-called ‘Lake Wobegon Effect’. We can also be wrong in the other direction. In a typical implicit learning study, participants are exposed to sequences of flashing lights, sounds or shapes that obey a certain rule. They subsequently have to identify whether new sequences obey the same rule or not. Participants often feel like they are just guessing, that is, they think they have not learned anything. Their behaviour, however, can be far above chance level, indicating that they in fact have learned something. In such cases, the ‘incorrect’ self-report is still informative: it gives us insight into the person’s subjective reality (they think they are in the 80 percentile of driving ability, they think they are just guessing, they think they haven’t learned something that they, in fact, have). But at the same time, these self-reports do not reflect objective reality. They are poor guides to predicting what a person can or is likely to do.

Lastly, consider dreams. In a 1958 survey, Fernando Tapia and colleagues reported that only about 9 per cent of respondents indicated that their dreams contained colour. Other surveys done around this time reported similarly low proportions. A decade later, the tide turned and a large majority reported dreaming in colour. The philosopher Eric Schwitzgebel considers several explanations for this discrepancy. One possibility is that black-and-white photographs and television changed the content of dreams. As colour TV came to dominate, colour returned to people’s dreams (‘returned’ because, in a few studies from the more distant past, people did not claim to dream in black and white).

The problem with this is that there is no reason to think TV should have such an outsized impact on the phenomenology of our dreams. After all, the world never ceased to be in colour. The alternative, argues Schwitzgebel, is that ‘at least some people must be pretty badly mistaken about their dreams.’ Our ability to report on the perceptual content of our dreams may simply be unreliable. And with no objective measures against which to measure the subjective report, we can’t really know whether these reports reflect anyreality, subjective or not. Why then would there be any consistency in people’s reports from a given time? Perhaps because, in the absence of having good access to their phenomenal states, people go with the response they think is most reasonable. In the 1950s, the dominant popular and scientific view was that dreams lack colour. And so, when queried, participants simply mirrored that dominant view. The same happened as the dominant view later changed. Neither case, Schwitzgebel argues, reflects ‘correct’ phenomenology because we simply do not have valid introspection when it comes to the colour of our dreams.

If reports about phenomenal states like imagery and inner speech are like reports about dreams, we have every reason to remain sceptical of whether differences in introspection report actual differences in people’s actual experiences. If they are more like reports about our preferences and emotions, then we can (mostly) take people at their word. Even then, we must consider social pressures to respond in a certain way. If having vivid imagery were a requirement for admission to art school, we should not be surprised if aspiring artists all claim to have very vivid imagery. If hearing a voice when one reads were considered a sign of mental illness, people would be less likely to say they hear a voice when they read.

Establishing the validity of self-report can be done in several ways. First, we must show consistency. If one day people claimed they experience inner speech constantly and the next day they claimed they never did, we have a problem. As it turns out, people’s reports are highly consistent. Inner speech questionnaires taken months apart show high correlations. (At the same time, Russell Hurlburt’s work using descriptive experience sampling, which probes people’s thinking at random points during the day, does show that people overestimate how much of their thinking is in the form of inner speech.)

We can also see whether differences in reported phenomenology predict differences in objective behaviour. This is not an option when it comes to dreams, but we can make specific predictions about behavioural consequences of having more or less visual imagery and inner speech based on existing theories of imagery and language. Differences in self-reported phenomenology can be linked to differences in objective behaviour. Those with less inner speech have a harder time remembering lists of words; those with less visual imagery report fewer visual details when describing past events.

There are also reported differences in more automatic physiological responses. More light entering the pupil causes it to constrict. But simply imagining something bright like the Sun also causes (a smaller, but still measurable) constriction. Aphantasics show perfectly typical pupillary responses to actual changes in light. However, their pupils do not change to imagined light. At the same time, many hypothesised differences in behaviour are notobserved because, it seems, people compensate by, for example, discovering ways of remembering detailed visual content without engaging explicit imagery. Such compensation can prove beneficial. People with poor autobiographical memory find other ways of keeping track of information that can help stave off some of the cognitive decline in ageing.

It’s harder to brush aside self-reports of someone who says they could imagine things, and now can’t

Another way to establish validity is that we can ask whether there are neural and physiological correlates of reported phenomenal differences. If differences in reported imagery were mere confabulations or the results of people just telling researchers what they think the researchers want to hear, it would be surprising if they had different brain connectivity and functional activation as measured by fMRI. Yet this is what we are finding. Fraser Milton and colleagues scanned groups of people identifying as aphantasics and hyperphantasics (those with unusually vivid visual imagery). When asked to lie in the scanner and stare at a cross on a screen, the brain responses of the hyperphantasic group had greater connectivity between prefrontal cortex and the occipital visual network, compared with the aphantasic group. Participants were also asked to look at and imagine various famous people and places. The difference in activation between perception and imagery (in a left anterior parietal region) was larger in hyperphantasic compared with aphantasic participants. Those with typical imagery tended to fall in between the aphantasics and hyperphantasic group on many of the measures. Less is known about neural correlates of differences in inner speech. In work presented at the 2023 meeting of the Society for the Neurobiology of Language, Huichao Yang and colleagues found a relationship between how much inner speech people reported to experience and resting-state functional connectivity in the language network.

Lastly, even though we don’t know what it’s like to be someone else, we can compare how our phenomenology differs from one time to another. There are numerous reports of people with brain injuries that cause them to lose visual imagery, and some cases of losing inner speech. It is much harder to brush aside self-reports of someone who says they used to be able to imagine things, and now they can’t (especially when these are confirmed by clear differences in objective behaviour).

Holderness’s caption introducing the world to #TheDress had a second part. ‘This is important,’ she wrote, ‘because I think I’m going insane.’ The idea that the same image can look different to different people is alarming because it threatens our conviction that the world is as we ourselves experience it. When an aphantasic learns that other people can form mental images, they are learning that something they did not know was even a possibility is, in fact, many people’s everyday reality. This is understandably destabilising.

And yet, there is a scientific and moral imperative for learning about the diverse forms of our phenomenology. Scientifically, it prevents us from making claims that the majority experience (or the scientist’s experience) is everyone’s experience. Morally, it encourages us to go beyond the ancient advice to ‘know thyself’ which can lead to excessive introspection, and to strive to know others. And to do that requires that we open ourselves up to the possibility that their experiences may be quite different from our own.

Original article here