Report

Bumble bees display cross-modal object recognition between visual and tactile senses

See allHide authors and affiliations

Science  21 Feb 2020:
Vol. 367, Issue 6480, pp. 910-912
DOI: 10.1126/science.aay8064

These bees have “seen” that before

Humans excel at mental imagery, and we can transfer those images across senses. For example, an object out of view, but for which we have a mental image, can still be recognized by touch. Such cross-modal recognition is highly adaptive and has been recently identified in other mammals, but whether it is widespread has been debated. Solvi et al. tested for this behavior in bumble bees, which are increasingly recognized as having some relatively advanced cognitive skills (see the Perspective by von der Emde and Burt de Perera). They found that the bees could identify objects by shape in the dark if they had seen, but not touched, them in the light, and vice versa, demonstrating a clear ability to transmit recognition across senses.

Science, this issue p. 910; see also p. 850

Abstract

Many animals can associate object shapes with incentives. However, such behavior is possible without storing images of shapes in memory that are accessible to more than one sensory modality. One way to explore whether there are modality-independent internal representations of object shapes is to investigate cross-modal recognition—experiencing an object in one sensory modality and later recognizing it in another. We show that bumble bees trained to discriminate two differently shaped objects (cubes and spheres) using only touch (in darkness) or vision (in light, but barred from touching the objects) could subsequently discriminate those same objects using only the other sensory information. Our experiments demonstrate that bumble bees possess the ability to integrate sensory information in a way that requires modality-independent internal representations.

Humans can easily recognize, through touch alone, objects they have previously only seen (1). This is demonstrated when we search and find objects on a high shelf or inside a cluttered bag. The ability to recognize objects across different senses increases the flexibility of any object-recognition system because the amount of noise and available information within different senses can vary dramatically across situations.

Cross-modal recognition requires the combination of information from multiple sensory modalities—for example, vision and touch. In humans, visual and tactile abilities are closely linked from birth, but their combination seems incomplete and limited at birth and develops to maturity over many years (2). Cross-modal object recognition has been shown across vision and touch in humans (3), apes (4), monkeys (5), and rats (6); across vision and hearing in dolphins (7); and across vision and electric sense in fish (8).

The ability to recognize objects across modalities entails some type of internal representation of an object’s shape or its characteristic features (7, 9, 10). In humans, cross-modal recognition seems to require mental imagery (11), an internal representation that occurs in the absence of sensory stimulation in a given sensory modality and that functions as a weak form of perception (1214). Theoretical analyses and empirical evidence support the idea that mental imagery involves an internal representation that is not only available to awareness but is a basic building block and integral part of consciousness (1417).

The ability to recognize objects across modalities is beneficial and adaptive, allowing for enhanced perceptual monitoring of an animal’s environment, flexible recognition of objects between senses, and richer representations of objects across multiple senses. Cross-modal recognition indicates that an animal possesses coherent “mental images” of objects. Whether the small brains of invertebrates perceive the world by storing internal representations of objects is unknown.

Because bumble bees naturally forage in the light but will also forage in the dark in laboratory conditions (18), they constitute an ideal system to examine whether an invertebrate is capable of cross-modal recognition across vision and touch. To do this, we trained 44 bumble bees in a dark room (a windowless room with lights turned off and vents, door edges, and all light diodes covered completely) (supplementary materials) to find 50% sucrose solution in one of two differently shaped objects (cube or sphere; 33 training trials) (Fig. 1A). To ensure that bumble bees could learn to discriminate objects in the dark without any visual information in our paradigm, we examined the behavior of 21 of the trained bumble bees in a nonreinforced test in the same dark setup as that in training. During the test, bumble bees spent more time in contact with the previously rewarding object [generalized linear mixed-effect model (GLMM): 95% confidence interval (CI) = 2.60 (1.95 to 3.25), n = 21 bees, P = 4.51 × 10–15] (Fig. 1C). To assess bumble bees’ ability for tactile-to-visual cross-modal recognition, we tested the other 23 trained bees in a lighted arena where they were unable to touch the objects (Fig. 1B). In the cross-modal test situation, bumble bees spent more time in contact with the previously rewarding object [GLMM: 95% CI = 1.41 (1.06 to 1.75), n = 23 bees, P = 9.81 × 10–16] (Fig. 1D).

Fig. 1 Cross-modal recognition in bumble bees.

(A and B) Setups for training and testing. Bumble bees were trained to find 50% sucrose solution in one of two differently shaped objects (sphere or cube) in one setup and then tested in the other. (A) In the dark setup, bees entered a dark arena and found two Petri dishes containing four spheres each and two Petri dishes containing four cubes each. (B) In the lighted setup, bees found the same objects, but placed under the Petri dishes so that the bees could see but not touch the objects. Bees accessed the reinforcement solution (rewarding sucrose solution or aversive quinine solution) through small holes in the top of each shape. (C and D) After being trained in the dark, bumble bees that were tested in the dark [(C) uni-modal] or in the light [(D) cross-modal] spent more time in contact with the previously rewarding object. (E and F) Similarly, after being trained in the light, bumble bees that were tested in the light [(E) uni-modal] or in the dark [(F) cross-modal] spent more time in contact with the previously rewarding object. Bars indicate mean; vertical lines indicate SEM; open circles indicate individual bees’ data points (random x axis displacement for individual discernment).

We then evaluated whether bumble bees were capable of visual-to-tactile cross-modal recognition. To do this, we trained 43 bumble bees first to discriminate cubes and spheres in a lighted arena where they could not touch the objects (Fig. 1B). Subsequently, to ensure that our setup was conducive to visual discrimination learning, the behavior of 22 of the trained bumble bees was tested in the same setup as in training but unreinforced (no reinforcements were present). Bumble bees spent more time in contact with the previously rewarding objects [GLMM: 95% CI = 2.00 (1.61 to 2.40), n = 22 bees, P = 1.36 × 10–23] (Fig. 1E). To test bumble bees’ ability for visual-to-tactile cross-modal recognition, the behavior of the other 21 trained bees was examined in a dark arena (Fig. 1A). In the test, bumble bees again spent more time in contact with the previously rewarding object [GLMM: 95% CI = 1.85 (0.54 to 3.15), n = 21 bees, P = 5.63 × 10–3] (Fig. 1F). These results suggest that after learning to discriminate objects by using only one sensory modality, bumble bees can discriminate these same objects by using a different sensory modality.

This interpretation rests on the assumption that bumble bees could not touch the objects in the lighted situation and could not see the objects in the dark situation. In the lighted situation, bees could only access the holes of the objects (which were all the same dimensions) with their proboscis but could not touch the outsides of the objects. In the dark situation, the measured light levels within the arena were less than 0.01 lux, and the infrared lights were the only source of illumination that was not covered (supplementary materials). Bumble bees were unable to find their way back to the tunnel that led to the hive without following the walls. Furthermore, the proportion of bees’ first entries onto Petri dishes during the tests in the dark was no different between the positively reinforced and negatively reinforced objects [GLMM: trained in dark, 95% CI = 0.69 (–0.21 to 1.60), n = 21 bees, P = 0.13; trained in light, 95% CI = 0.69 (–0.21 to 1.60), n = 21 bees, P = 0.13]. As an additional measure, we tested bees’ ability to discriminate the two objects in the dark without being able to touch the objects. One group of bees (n = 20 bees) was trained to discriminate the two objects visually in a lighted arena (Fig. 1B), and another group of bees (n = 21 bees) was trained to discriminate the two objects tactilely in the dark (Fig. 1A). Subsequently, during an unreinforced test in the dark and without being able to touch the objects (Fig. 2A), both groups showed no difference in the duration in contact with the two different objects [GLMM: trained in light, 95% CI = –1.81 (–8.98 to 5.35), n = 20 bees, P = 0.62; trained in dark, 95% CI = –0.85 (–1.90 to 0.21), n = 21 bees, P = 0.12] (Fig. 2, B and C).

Fig. 2 Bumble bees were unable to see in the dark experimental conditions.

(A) Setup for testing in control experiments. Bumble bees had no tactile information regarding the objects during these tests in the dark. (B and C) After being trained in the light (Fig. 1B) or in the dark (Fig. 1A), bumble bees that were tested in the dark, while not being able to touch the objects, had no difference in the amount of time they were in contact with the two different objects. Bars indicate mean; vertical lines indicate SEM; open circles indicate individual bees’ data points (random x axis displacement for individual discernment).

Perception of objects results from the integration of information from various sensory modalities (12, 19). Two high-order centers in the insect protocerebrum, the mushroom bodies and central complex, have been shown to receive both visual and mechanosensory information (2022). We thus surmise that these areas are possible candidates for the multisensory integration necessary for the cross-modal object recognition behavior observed here.

Our results indicate more than just direct associations formed between two stimuli across senses, as is the case in simpler forms of cross-modal information transfer (2326). Cross-modal object recognition requires that the two different senses provide information about the same object property (such as shape); that the information provided is encoded in such a way that it can be identified as related, even though it is temporally and physically distinct; and that this information is stored in a neuronal representation that is accessible by both senses (8).

Whether bumble bees solve the task by storing internal representations of entire object shapes (cube or sphere) or local object features (curved or flat edge) remains unknown. In either case, our experiments show that bumble bees are capable of recognizing objects across modalities, even though the received sensory inputs are temporally and physically distinct. Bumble bees show a kind of information integration that requires a modality-independent internal representation (7, 9, 10). This suggests that similar to humans and other large-brained animals, insects integrate information from multiple senses into a complete, globally accessible, gestalt perception of the world around them (12, 26, 27).

Supplementary Materials

science.sciencemag.org/content/367/6480/910/suppl/DC1

Material and Methods

Supplementary Text

Database S1

Reference (28)

References and Notes

Acknowledgments: Funding: This work was supported by European Research Council (ERC) grant SpaceRadarPollinator (grant 339347) and Engineering and Physical Sciences Research Council (EPSRC) grant Brains-on-Board (grant EP/P006094/1) awarded to L.C. Author contributions: C.S. conceived and designed the study, with input from L.C.; S.G.A.-K. and C.S. performed experiments and behavioral data analyses; C.S. performed statistical analyses; and C.S. and L.C. wrote the paper. Competing interests: None declared. Data and materials availability: Data are available in the supplementary materials.

Correction (3 March 2020): The statistical values reported for proportion of first entries for the dark test when trained in light were incorrect. The correct values for the statistical test are as follows: CI = 0.69 (–0.21 to 1.60), n = 21 bees, P = 0.13. The results and interpretation of these analyses and the rest of the manuscript remain unchanged.

Stay Connected to Science

Navigate This Article