Special Reviews

Evidence for Ecological Speciation and Its Alternative

See allHide authors and affiliations

Science  06 Feb 2009:
Vol. 323, Issue 5915, pp. 737-741
DOI: 10.1126/science.1160006


Natural selection commonly drives the origin of species, as Darwin initially claimed. Mechanisms of speciation by selection fall into two broad categories: ecological and mutation-order. Under ecological speciation, divergence is driven by divergent natural selection between environments, whereas under mutation-order speciation, divergence occurs when different mutations arise and are fixed in separate populations adapting to similar selection pressures. Tests of parallel evolution of reproductive isolation, trait-based assortative mating, and reproductive isolation by active selection have demonstrated that ecological speciation is a common means by which new species arise. Evidence for mutation-order speciation by natural selection is more limited and has been best documented by instances of reproductive isolation resulting from intragenomic conflict. However, we still have not identified all aspects of selection, and identifying the underlying genes for reproductive isolation remains challenging.

It took evolutionary biologists nearly 150 years, but at last we can agree with Darwin that the origin of species, “that mystery of mysteries” (1), really does occur by means of natural selection (25). Not all species appear to evolve by selection, but the evidence suggests that most of them do. The effort leading up to this conclusion involved many experimental and conceptual advances, including a revision of the notion of speciation itself, 80 years after publication of On the Origin of the Species, to a definition based on reproductive isolation instead of morphological differences (6, 7).

The main question today is how does selection lead to speciation? What are the mechanisms of natural selection, what genes are affected, and how do changes at these genes yield the habitat, behavioral, mechanical, chemical, physiological, and other incompatibilities that are the reproductive barriers between new species? As a start, the many ways by which new species might arise by selection can be grouped into two broad categories: ecological speciation and mutation-order speciation. Ecological speciation refers to the evolution of reproductive isolation between populations or subsets of a single population by adaptation to different environments or ecological niches (2, 8, 9). Natural selection is divergent, acting in contrasting directions between environments, which drives the fixation of different alleles, each advantageous in one environment but not in the other. Following G. S. Mani and B. C. Clarke (10), I define mutation-order speciation as the evolution of reproductive isolation by the chance occurrence and fixation of different alleles between populations adapting to similar selection pressures. Reproductive isolation evolves because populations fix distinct mutations that would nevertheless be advantageous in both of their environments. The relative importance of these two categories of mechanism for the origin of species in nature is unknown.

In this review, I summarize progress in understanding the general features of speciation by selection. I do not differentiate speciation by sexual selection here because natural selection drives the divergence of mate preferences, by either ecological or mutation-order mechanisms, in most theories of the process (8, 11). I leave out discussion of sympatric and allopatric speciation but instead identify the likelihood of ecological and mutation-order speciation when there is gene flow. I ignore reinforcement, a special type of natural selection thought to favor stronger premating reproductive isolation once postzygotic isolation has evolved. I also ignore speciation by polyploidy, even though selection might be crucial in the early stages.

Speciation and Adaptation from Darwin to Dobzhansky

Appreciation of the connection between adaptation and speciation began with Darwin when a morphological concept of species largely prevailed. In On the Origin of Species, Darwin wrote that “I look at the term species, as one arbitrarily given for the sake of convenience to a set of individuals closely resembling each other...” and “The amount of difference is one very important criterion in settling whether two forms should be ranked as species or varieties” (1). Under this view, speciation is defined as the accumulation of sufficiently many differences between populations to warrant their classification as separate taxonomic species. Darwin understood the importance of reproductive barriers between species (1), but the study of speciation after the publication of this work focused mainly on the evolution of species differences, particularly of morphological traits but also of behavioral and other phenotypic traits.

Under this Darwinian perspective, linking speciation with adaptation was relatively straightforward, requiring only a test of whether phenotypic differences between species were caused by natural selection. For example, at the American Association for the Advancement of Science 1939 speciation symposium [the last major symposium on speciation before the biological species concept (7)], an extensive comparative and biogeographic study showcased instances in which derived morphological and life history forms of fishes had arisen over and over again from the same ancestral type (12). The repeated, parallel origin of nonparasitic lamprey in streams from the same migratory, parasitic ancestor showed that “Again and again the parasitic lampreys have evolved into nonparasitic forms...correlated with life in small streams, where a suitable food supply in the way of large fish is scarce or seasonal” (12). When correlated with environmental factors, such repetition is unlikely to result from chance; environmental selection pressures must therefore be the cause of speciation. “As a result of our recent studies on fishes...weight is constantly being added to the theory that speciation is...under the rigid control of the environment” (12). However, this case is only referring to the origin of morphological species.

The turning point for speciation studies came with the modern concept of speciation “Species separation is defined as a stage of the evolutionary process at which physiological isolating mechanisms become developed” (6) (here, “physiological” is interpreted to mean evolved reproductive isolation between populations, as distinct from geographical barriers to interbreeding). Subsequently, species were defined as “groups of interbreeding natural populations that are reproductively isolated from other such groups” (7). From this point on, the study of speciation was the study of the evolution of reproductive isolation (3). Progress up to then in understanding the link between morphological speciation and adaptation was largely forgotten, its contributions uncertain under the new concept.

The biological species concept must surely have made it more difficult to investigate any link between speciation and natural selection. T. Dobzhansky (13) suggested that the genes underlying differences between populations in ordinary phenotypic traits were unlikely to be the basis of reproductive isolation. He later changed his mind, but at the time this viewpoint, and the generally greater difficulty of studying reproductive isolation than morphology, must have discouraged many from pursuing the connection. Virtually no research effort followed that tested the role of adaptation in speciation.

Models of Speciation by Selection

The topic of natural selection in speciation is once again receiving attention. The two most general hypotheses involving selection are ecological and mutation-order speciation. Ecological speciation is defined as the evolution of reproductive isolation between populations by divergent natural selection arising from differences between ecological environments (2, 8, 9, 14). It predicts that reproductive isolation should evolve between populations adapting to contrasting environments but not between populations adapting to similar environments. The basic idea has been around for a while (7), although it was tested only recently. The agents of divergent selection are extrinsic and can include abiotic and biotic factors such as food resources, climate, habitat, and interspecies interactions such as disease, competition, and behavioral interference. Ecological speciation can lead to the evolution of any type of reproductive isolation, including premating isolation, hybrid sterility, and intrinsic hybrid inviability as well as extrinsic, ecologically based pre- and postzygotic isolation. Speciation by sexual selection is ecological speciation if ecologically based divergent selection drives divergence of mating preferences, for example by sensory drive (15).

In accordance with (10), mutation-order speciation is defined as the evolution of reproductive isolation by the fixation of different advantageous mutations in separate populations experiencing similar selection pressures. Whereas different alleles are favored between populations under ecological speciation, the same alleles would be favored in different populations under mutation-order speciation. Divergence occurs anyway because, by chance, the populations do not acquire the same mutations or fix them in the same order. Divergence is therefore stochastic but the process is distinct from genetic drift. It can occur in both small and large (though not infinite) populations. Selection can be ecologically based under mutation-order speciation, but ecology does not favor divergence as such. It can lead to the evolution of any type of reproductive isolation, with the exception of ecologically based pre- and postzygotic isolation.

Speciation resulting from intragenomic conflict such as meiotic drive or cytoplasmic male sterility (Fig. 1B) is likely to be mutation-order speciation because, by chance, the initial mutations causing drive and those countering it are unlikely to be the same in separate populations. Speciation by sexual selection is mutation-order speciation if divergence of mate preferences or gamete recognition occurs by the fixation of alternative advantageous mutations in different populations, as by sexual conflict (16). Divergence in song and other learned components of behavior under purely social selection, not molded by selection for efficient signal transmission (5), is the cultural equivalent of the mutation-order process. Additional scenarios are elaborated in (5).

Fig. 1.

(A) Example of ecological speciation. Repeatedly and independently, the mosquito fish, Gambusia hubbsi, inhabiting blue holes in the Bahamas has evolved a larger caudal region and smaller head in the presence of predators (top) than in their absence (bottom) (29). In laboratory trials, the probability of two individuals mating was higher when they were from different populations having the same predation environment (and similar body shape) than when they were from opposite predation environments. [Photo credit: Brian Langerhans (29)]. (B) Example of reproductive isolation evolving under the mutation-order mechanism. Male-fertile (left) and male-sterile (right) flowers of F2 hybrids between an Oregon population of monkey flowers (M. guttatus) having a cytoplasmic male sterility element and nuclear restorer and a closely related species (M. nasutus) having neither (46, 47). Both flowers shown have M. guttatus cytoplasm. The flower on the left also has the nuclear restorer, whereas the one on the right, with undeveloped anthers, lacks the restorer. [Photo credit: Andrea Case (47)]

Both models of speciation, ecological and mutation-order, are theoretically plausible, and only data can determine their relative importance in nature. The key is to figure out by which mechanism reproductive isolation first evolved (3). Once the earliest genetic differences have accumulated between populations by either process, subsequent mutations might be favored in one population and not the other because of epistatic interactions with genetic background (10). Hence, epistasis, including that producing Dobzhansky-Muller incompatibilities in hybrids between species (3), can result from either ecological or mutation-order speciation.

Speciation can be rapid under both speciation models, because alleles are driven to fixation by natural selection in both cases. However, under the mutation-order process, the same alleles, if present, would be favored in every population, at least in the early stages of divergence. For this reason, mutation-order speciation is difficult when there is gene flow, because gene flow increases the possibility that favorable mutations occurring in one population will spread to other populations, preventing divergence (17, 18). Any process resulting in low levels of gene flow, including selection, facilitates subsequent divergence by the mutation-order process (19). In contrast, ecological speciation can proceed with or without gene flow, although it is easiest when gene flow is absent.

Experiments with laboratory populations of Drosophila and yeast demonstrate the plausibility of ecological speciation. In those instances when measurable pre- and postmating reproductive isolation evolved, it was greater between lines subjected to different environments than between lines raised under homogeneous conditions (20, 21). Laboratory experiments on various microbes maintained under homogeneous conditions for many generations have detected genetic divergence consistent with the mutation-order process (22), but effects on reproductive isolation have not been explored.

Two approaches investigate the mechanisms of speciation by natural selection in nature. The bottom-up approach involves (i) genetic mapping of reproductive isolation between closely related species, (ii) testing whether discovered genes exhibit a genomic signature of positive selection, and (iii) identifying the phenotype and source of fitness effects of alternative alleles at selected loci. The approach has been hugely successful in identifying major genes implicated in hybrid inviability (Hmr, Lhr, Nup96), sterility (Odsh, JYAlpha), and sexual isolation (ds2) between Drosophila species. Most of these genes show molecular signatures of positive selection, proving natural selection's role (3), provided that fixation occurred before complete reproductive isolation rather than afterward. The top-down approach involves identifying (i) the phenotypic traits under divergent selection, (ii) those traits associated with reproductive isolation, and (iii) the genes underlying traits and reproductive isolation. Step (iii) has been challenging under both approaches but is needed to understand how selection has led to reproductive isolation.

Ecological Speciation

Evidence for ecological speciation has accumulated from top-down studies of adaptation and reproductive isolation [reviewed in (2, 8, 9)]. We now know of many real species that have, at least in part, evolved by divergent natural selection between environments. The connections between selection on ordinary phenotypic traits and reproductive isolation are often strong and straightforward. It follows that much of the genetic basis of reproductive isolation should involve ordinary genes that underlie differences in phenotypic traits. But we still know little about the genetics of ecological speciation.

One line of evidence comes from tests of parallel speciation, whereby greater reproductive isolation repeatedly evolves between independent populations adapting to contrasting environments than between independent populations adapting to similar environments (20, 23). A major challenge in applying the test to natural populations is to eliminate the possibility that each ecotype has originated just once and has spread to multiple locales. This is difficult because gene flow of neutral markers between closely related but nearby populations can result in the false appearance of multiple independent origins of these populations when evaluated by phylogenies (3, 24). However, multiple origins are supported in several examples of parallel speciation, including the sympatric benthic-limnetic species pairs of threespine stickleback in young lakes of British Columbia (25, 26), the repeated origin of divergent marine and stream populations of threespine stickleback around the Northern Hemisphere (27), ecotypes of Timema walking stick insects living on different host plants (28), Littorina marine snail ecotypes inhabiting different zones of the intertidal (24), and mosquito fish inhabiting blue holes with and without fish predators in the Bahamas (29) (Fig. 1A). In these studies, it was shown that males and females are more likely to mate if they are of the same ecotype, regardless of relatedness as indicated by phylogenetic affinity.

Ecological speciation is also supported by examples of premating reproductive isolation in which individuals choose or preferentially encounter mates on the basis of phenotypic traits that are under ecologically based divergent selection. Examples include assortative mating by host choice in insects, body size and coloration in fish, beak size in birds, pollinator preferences for specific phenotypic floral traits, and variation in flowering time—traits inferred to be under divergent selection between environments [see examples in (8, 30, 31)].

Ecologically based divergent selection has also been directly measured, as shown by reduced fitness of each ecotype in the environment of the other [immigrant inviability; reviewed in (31, 32)] and by reduced fitness of hybrids in the parental environments [extrinsic postzygotic isolation (33)]. For example, each of the coastal perennial and inland annual races of the monkey flower (Mimulus guttatus) along the west coast of North America has low fitness when transplanted to the habitat of the other (31). This is an example of active selection on phenotypic differences, and it also constitutes direct reproductive isolation because it is an evolved barrier to gene flow between parental populations. Multiple traits are probably involved, including flowering time and tolerance of salt and drought. This type of reproductive isolation is context-dependent and is weakened in intermediate environments. On the other hand, active selection favors the evolution of ever-greater differences between populations, which may strengthen the barrier to gene flow (20).

It is unclear how much reproductive isolation typically evolves by ecologically based divergent selection in nature. We can approximate an answer from estimates of the combined contribution of active selection on traits and trait-based assortative mating, as compared with other forms of reproductive isolation (Fig. 2 and table S1). These estimates are incomplete because individual studies may lack data on components of reproductive isolation, separate components may not be independent, and the strength of barriers between species may not be symmetric (34). Nevertheless, compilation of the data shows that the amount of reproductive isolation attributable to active selection and trait-based assortative mating is at least as strong, on average, as the amount from components of reproductive isolation lacking identifiable causes (Fig. 2). The unidentified component of speciation, if built by selection and not genetic drift, could be the result of either ecological or mutation-order mechanisms.

Fig. 2.

Estimates of the magnitude of reproductive isolation resulting from divergent selection components (top), compared with other components lacking identifiable causes (bottom). Divergent selection components include those attributable to active selection on traits (immigrant inviability and extrinsic postzygotic isolation) and to trait-based assortative mating (habitat preference, floral isolation, and breeding time). The unattributed components include intrinsic hybrid inviability, sexual selection against hybrids, pollen competition, and reduced hybrid fecundity. Data were taken from (32, 31) (table S1). A negative value indicates that hybrids had higher fitness than the parental species for at least one component of postzygotic isolation. One data value of –2.66 was left out of the bottom panel.

These examples indicate a growing knowledge of the mechanisms of selection and its consequences for reproductive isolation. At this point, the most glaring deficiency is our knowledge of the impact of selection on genes. Optimistically, progress is being made with genetic mapping to identify quantitative trait loci (QTLs) and genes or regulatory control regions that affect individual phenotypic traits on which components of reproductive isolation depend. Examples include the yup QTL, which affects flower color differences between the monkey flowers, Mimulus cardinalis and M. lewisii (35). Swapping alleles of this QTL between the species with repeated backcrossing resulted in shifts in pollinator preference and, hence, indirectly affected premating isolation. Survival and salt tolerance of second-generation hybrids between the sunflowers Helianthus annuus and H. petiolaris transplanted to the salt marsh habitat of their hybrid descendent species (H. paradoxus) mapped strongly to a QTL identified as the salt tolerance gene CDPK3 (36).

Another form of investigation involves the analysis of genome scans of ecologically different populations and species. These scans compare allelic variation within and between populations at many marker loci spaced throughout the genome (37). Markers that show excessive differentiation between populations (outliers) may indicate selection on nearby genes. The method is particularly informative when applied to populations with ongoing hybridization, because outlier markers may identify points in the genome that resist the homogenizing influence of gene flow, perhaps indicating genomic regions under divergent selection. However, sets of genes that diverged under a mutation-order process can produce the same pattern (17, 18), which makes analysis of such studies more difficult. Clues to whether ecologically based divergent selection is involved are gained if outliers at the same genomic locations turn up repeatedly in scans between populations that inhabit contrasting environments (38) and by identifying phenotypic traits under divergent selection that map to those locations in the genome (36, 39, 40). As genomic resources increase for more species, it will be possible to measure natural selection directly on genomic regions of interest by transplanting otherwise relatively homogenous experimental populations containing alternative alleles into the environments of the parent species (35).

Mutation-Order Speciation

Mounting evidence for divergent selection in speciation does not diminish the potential role of mutation-order divergence. It may be that the mutation-order process is more difficult to detect, or perhaps we have not looked hard enough at species with only small ecological differences (5). We still do not know much about the selective factors causing mutation-order speciation.

Evidence for mutation-order speciation comes from instances in which reproductive isolation apparently evolved as a by-product of conflict resolution between genetic elements within individuals (intragenomic conflict), such as cytoplasmic male sterility in hermaphroditic plants (Fig. 1B), and genetic elements conferring meiotic drive. Under both mechanisms, a mutation arises that can distort representation in gametes and spreads in a selfish manner, even though such an element reduces overall fitness of the organism that bears it. This, in turn, places selection on mutations in other genes that counter the selfish element's effects and restore more equal genetic representation in gametes. Distorter and restorer mutations are unlikely to be the same in different populations regardless of environment; thus the process leads to divergence. The mismatch between the distorter in one population and the restorer in the other can result in hybrid sterility or inviability and, thus, reproductive isolation (3, 41). Numerous examples of selfish elements, such as those observed in cytoplasmic male sterility of plants, support these hypotheses (42, 43). In addition, partial reproductive isolation generated by meiotic drive has been identified in Drosophila [reviewed in (3, 41)]. Sexual conflict is also expected to lead to mutation-order speciation, but there are few compelling examples (3). The contribution by these mechanisms to speciation is still uncertain, however. The alleles responsible for meiotic drive and cytoplasmic male sterility may be prevented from spreading to fixation because selection on such elements is frequency-dependent (43) and because restorer alleles arise and weaken selection on the distorter elements (44). Second, if divergent populations come into secondary contact, the alleles within each population causing cytoplasmic male sterility or meiotic drive (and the corresponding restorer alleles) will spread between the populations by gene flow, eliminating that component of reproductive isolation (43). Hence, for these mechanisms to contribute to speciation, the fitness of hybrids must be reduced to very low levels, or other incompatibilities must arise that interact with these genes to prevent their spread after secondary contact.


Our understanding of the role of natural selection in speciation has come a long way since Darwin's time. If he were here to witness, he would most likely be staggered by the discoveries of genes and molecular evolution and astonished at the prospect that evolutionary conflict between genes could generate reproductive isolation (45). Mostly, I expect that he would be chuffed by mounting evidence for the role of natural selection on phenotypic traits in the origin of species. This is really what On the Origin of Species was all about. Between 1859 and the present, the general acceptance of the biological species concept altered the focus of speciation studies. Yet, the discovery that reproductive isolation can be brought about by ecological adaptation in ordinary phenotypic traits bridges Darwin's science of speciation and our own.

The most obvious shortcoming of our current understanding of speciation is that the threads connecting genes and selection are still few. We have many cases of ecological selection generating reproductive isolation with little knowledge of the genetic changes that allow it. We have strong signatures of positive selection at genes for reproductive isolation without enough knowledge of the mechanisms of selection behind them. But we hardly have time to complain. So many new model systems for speciation are being developed that the filling of major gaps is imminent. By the time we reach the bicentennial of the greatest book ever written, I expect that we will have that much more to celebrate.

Supporting Online Material


Tables S1 to S3


References and Notes

View Abstract

Stay Connected to Science

Navigate This Article