## Editor's Summary

Topological insulators owe their exotic properties to the peculiarities of their band structure, and one can induce a transition between a topologically trivial and nontrivial material by chemical doping. Now, J. Zhang *et al*. (p. 1582) have gone a step further—simultaneously observing that a magnetic quantum transition as the ratio of Se and Te is varied in Bi_{2}(Se_{x}Te_{1-x})_{3} thin films grown by molecular beam epitaxy and doped with magnetic Cr. Photoemission and transport experiments, as well as density functional calculations, imply that the topological transition induces magnetism

## Abstract

The breaking of time reversal symmetry in topological insulators may create previously unknown quantum effects. We observed a magnetic quantum phase transition in Cr-doped Bi_{2}(Se_{x}Te_{1-x})_{3} topological insulator films grown by means of molecular beam epitaxy. Across the critical point, a topological quantum phase transition is revealed through both angle-resolved photoemission measurements and density functional theory calculations. We present strong evidence that the bulk band topology is the fundamental driving force for the magnetic quantum phase transition. The tunable topological and magnetic properties in this system are well suited for realizing the exotic topological quantum phenomena in magnetic topological insulators.

The metallic surface states of three-dimensional (3D) topological insulators (TIs) are protected by time reversal symmetry (TRS) (*1*–*3*). Although breaking the TRS is generally detrimental to these states, it may also lead to exotic topological quantum effects. Examples include image magnetic monopoles (*4*), a quantized anomalous Hall effect (*5*, *6*), giant magneto-optical effects (*7*), and a dissipationless inverse spin-Galvanic effect (*8*). A key step for realizing these previously unknown quantum states is to tune the magnetic ordering in TIs in a controlled manner and investigate the interplay between magnetism and topological order.

With their large bulk gap and a single-surface Dirac cone, Bi_{2}Te_{3} and Bi_{2}Se_{3} (*9*–*11*) are widely used as hosts for TRS-breaking perturbations. In Mn-doped Bi_{2}Te_{3} single crystals, magnetization measurements demonstrate a ferromagnetic (FM) state with a Curie temperature (*T*_{C}) of up to 12 K (*12*). On the cleaved surface of an Fe-doped Bi_{2}Se_{3} single crystal, angle-resolved photoemission spectroscopy (ARPES) reveals the opening of an energy gap at the Dirac point (*13*). Iron (Fe) atoms deposited on the surface of a Bi_{2}Se_{3} single crystal are found to create odd multiples of Dirac-like surface states (*14*). Scanning tunneling microscopy on the surface of magnetically doped TI crystals demonstrates that breaking TRS can lead to previously unknown quasiparticle interference patterns (*15*) and strong spatial variations for the helical surface states (*16*).

Most of these previous studies focused on the effect of magnetism on the topological surface states; however, little is known about how the magnetic ordering is affected by the topological property. Because the Z_{2} bulk topology is the most fundamental identity of a TI, it probably also plays a role in determining the phases and phase transitions in magnetically doped TIs. To test this conjecture, we fabricated chromium (Cr)–doped Bi_{2}(Se_{x}Te_{1-x})_{3} TI films using molecular beam epitaxy (*17*). By varying the mixing ratio of Bi_{2}Se_{3} and Bi_{2}Te_{3}, we can actively modify the strength of spin-orbit coupling (SOC), which is essential for the band inversion of TIs. In the resulting Bi_{2-y}Cr_{y}(Se_{x}Te_{1-x})_{3} (Fig. 1A), the Cr dopants substitute Bi sites, and Se/Te atoms are randomly mixed. All of the films have the same thickness *d* = 8 quintuple layers (QLs), so that they are in the 3D regime with decoupled surfaces (*18*). The Cr content is fixed at *y* = 0.22 because at this doping level, the density of local moments is high enough to sustain long-range magnetic order, and the SOC strength is reduced to the verge of a topological phase transition (*19*).

Magneto transport measurements on the TI films (Fig. 1B) are made in the presence of an external magnetic field (*H*) perpendicular to the film plane. At the base temperature *T* = 1.5 K, the Hall effect of the 8-QL Bi_{1.78}Cr_{0.22}Te_{3} film (Fig. 1C) shows a hysteretic loop and a nearly square-shaped positive jump, the hallmarks of anomalous Hall effect (AHE) in FM conductors (*20*). The total 2D Hall resistivity ρ_{yx} can be expressed as ρ_{yx} = *R*_{A}*M*(*T*, *H*) + *R*_{N}*H*, where *M*(*T*, *H*) is the magnetization and *R*_{A} and *R*_{N} are the anomalous and normal Hall coefficients, respectively. Both the anomalous Hall resistivity and the coercive force (*H*_{coer}) decrease as *T* rises. The Hall traces become fully reversible at *T* > 20 K; thus, the *T*_{C} of this film is around 20 K. The normal Hall effect at high *H* has a negative slope for the entire temperature range (fig. S2), indicating the existence of electron-type charge carriers. The magnetoconductivity (MC) curves taken at *T < T*_{C} show butterfly-shaped hysteresis at weak *H* (Fig. 1D), as commonly observed in FM metals. MC keeps increasing at higher *H*, which is indicative of the weak localization (WL) of charge carriers instead of the weak antilocalization (WAL) in pristine Bi_{2}Te_{3} (*21*).

The Cr-doped Bi_{2}Se_{3} exhibits a very different transport behavior. At *T* = 1.5 K, the Hall trace of Bi_{1.78}Cr_{0.22}Se_{3} film (Fig. 1E) has a pronounced negative curvature at weak *H* but shows no observable hysteresis, which is consistent with the field-induced AHE in paramagnetic (PM) materials without spontaneous magnetization. The MC curves also show no sign of hysteresis even at the base temperature (Fig. 1F). MC changes from positive at low *T* to negative at high *T*, which is consistent with the WL-to-WAL crossover reported previously (*22*). The possible existence of in-plane ferromagnetism in the Bi_{1.78}Cr_{0.22}Se_{3} film is ruled out by magnetization and magneto transport measurements with *H* applied along the film plane (fig. S12).

To uncover the origin of the sharp contrast between the magneto transport properties of Cr-doped Bi_{2}Te_{2} and Bi_{2}Se_{3}, we fabricated Cr-doped Bi_{2}(Se_{x}Te_{1-x})_{3}, an isostructural, isovalent mixture of Bi_{2}Te_{3} and Bi_{2}Se_{3}. The Hall traces are displayed in Fig. 2, A to E, measured on five Bi_{1.78}Cr_{0.22}(Se_{x}Te_{1-x})_{3} films with 0.22 ≤ *x* ≤ 0.86. The three Te-rich samples with *x* ≤ 0.52 all show a hysteretic response at low *T*, reflecting FM ordering. As the Se content is increased to *x* = 0.67, however, the hysteresis disappears, and the system becomes PM. The sign of the AHE also reverses to negative right at this doping. With further increase of *x* to 0.86, the system remains PM, and the negative AHE becomes more pronounced. The FM-to-PM phase transition can also be seen in the MC curves (fig. S4) (*17*).

The Hall curves of all the Bi_{1.78}Cr_{0.22}(Se_{x}Te_{1-x})_{3} films measured at *T* = 1.5 K are summarized in Fig. 2F, revealing a highly systematic evolution of *H*_{coer} and the intercept Hall resistivity ρ_{yx}^{0} (fig. S6A). Because the magnetism of the system can be characterized by the AHE, we can construct a magnetic phase diagram by plotting the ρ_{yx}^{0} values of each sample and at each *T* (Fig. 2G). At the base temperature, the phase diagram is separated into two distinct regimes: an FM phase with positive ρ_{yx}^{0} and a PM phase with negative ρ_{yx}^{0}. Because the transition between the two magnetic phases occurs at the ground state, it is a quantum phase transition (QPT) driven by the change of chemical composition. The quantum critical point (QCP) *x*_{c} ~ 0.63 can be estimated from the interpolated *x* value when ρ_{yx}^{0} changes sign (fig. S6B). The solid symbols in Fig. 2G indicate the *T*_{C} of each sample determined by the temperature when *H*_{coer} becomes zero.

ARPES measurements on Bi_{1.78}Cr_{0.22}(Se_{x}Te_{1-x})_{3} uncover a surprising feature. The ARPES band maps displayed in Fig. 3A were taken at *T* = 120 K, when all the samples are in the PM state. The Fermi level of all the samples lies above the Dirac point, which is consistent with the negative slope of the normal Hall effect. The three samples with *x* ≤ 0.52 show well-defined gapless surface states with linear dispersions. The surface-state features can be better identified by the dual-peak structures around the Γ point in the momentum distribution curves (MDCs) (Fig. 3, G to I). As *x* is increased to *x* = 0.67 (Fig. 3D), however, the surface state features can no longer be resolved, and a small energy gap starts to appear at the Γ point of the band structure. With further increase of *x* up to *x* = 1, the surface states are always absent, whereas the gap amplitude keeps increasing. Because the surface states derive from the nontrivial bulk topology, their absence for *x* ≥ 0.67 suggests that the bulk band structure in this regime is topologically trivial. Therefore, the ARPES results reveal a topological QPT, a transition from the TI to trivial band insulator, accompanying the magnetic QPT. The ARPES patterns are very similar to that in BiTl(S_{1-δ}Se_{δ})_{2} TIs, showing the topological QPT induced by S substitution of Se (*23*, *24*).

The topological QPT is also corroborated through density functional theory (DFT) calculations (*17*). The signature of a topological QPT in the bulk band structure, a gap closing point at the critical SOC strength, can be seen clearly in the DFT results on Cr-doped Bi_{2}Se_{3} (fig. S7). The topological QPT is caused by the reduced SOC strength resulting from the Cr substitution of Bi. At sufficiently high Cr doping, the SOC is not strong enough to invert the bands, leading to a trivial bulk topology (*19*). In contrast, for Cr-doped Bi_{2}Te_{3} our DFT calculations show that the bulk band remains inverted for Cr content up to *y* = 0.25 (figs. S8 and S9). The more robust band inversion is a result of the larger SOC strength of Te as compared with Se. The calculated band structures of Bi_{1.75}Cr_{0.25}(Se_{x}Te_{1-x})_{3} with varied *x* (Fig. 4, A to E) show a transition from inverted to normal bands caused by the reduced SOC strength with increasing Se/Te ratio. The calculated bulk gap at the Γ point is summarized in Fig. 4F, which clearly shows a topological QPT near *x* ~ 0.66, which is in agreement with the experiments.

With the correlation between the magnetic and topological QPTs firmly established, we turned to a more fundamental question: Which phase transition is the driving force, and which one is the consequence? Two pieces of evidence support the scenario that topology determines the magnetic ordering. First, in our Bi_{1.78}Cr_{0.22}(Se_{x}Te_{1-x})_{3} samples the Cr content is fixed, and only the Se/Te ratio is varied. This provides a knob for fine-tuning the SOC strength—hence, the bulk band topology—but the magnetic property is not directly affected. Therefore, the magnetic QPT should be a secondary effect of the topological QPT. Second, the ARPES results show that even at high *T* when all the samples are in the PM state, the two regimes separated by the QCP already develop different topologies. At low *T*, the magnetic ground states form following the preformed topological character, with the FM phase resulting from the nontrivial topology and transitioning to the PM phase when the bulk turns topologically trivial.

The topological origin of the magnetic QPT is further supported by the effective model calculations (*17*). We calculated the *z*-direction spin susceptibility (χ_{zz}) of eight QL magnetically doped TI films using an effective four-band model (Fig. 4G) as a function of the chemical potential (μ) and the mass term (*M*_{0}). In the inverted regime with *M*_{0} < 0, χ_{zz} always remains a large value when μ is around the gap, as a consequence of the van Vleck mechanism (*5*); the second-order matrix element is strongly enhanced when the bulk bands become inverted. The topologically nontrivial phase thus strongly favors an FM ordering, which naturally explains the topology-driven magnetic QPT discovered in the experiments. The van Vleck mechanism is further supported by the magnetization measurements (fig. S10), which show that the ferromagnetism occurs in the bulk rather than on the surfaces (*25*, *26*). The out-of-plane magnetic anisotropy (fig. S11) is also consistent with the van Vleck–type FM order in TIs (*5*).

To reveal the physical origin of the AHE sign change at the QCP, we calculated σ_{xy} based on the four-band model, with two additional Zeeman splitting terms, *G*_{z1} and *G*_{z2}, from the exchange coupling between the electrons and magnetic impurities (*17*). The σ_{xy} value is summarized in Fig. 4H as a function of *M*_{0} with fixed chemical potential, which clearly uncovers a sign change when the band gap is reversed, which is in good agreement with the experimental observation. The close correlation between the sign of AHE and topological QPT suggests that it can be used as a transport fingerprint for the bulk topology. This is not unexpected given the growing recognition of the topological nature of the intrinsic AHE in recent years (*27*, *28*). The extrinsic AHE, which may be present in realistic materials, is ignored here because it typically dominates in highly metallic materials, whereas the disordered TI films studied here are poorly conductive (*20*).

The topologically nontrivial FM states with tunable magnetic properties revealed here provide an ideal platform for realizing the exotic magnetoelectric effects proposed by theory. The topology-driven magnetic QPT may also inspire new ideas for topological-magnetic phenomena and spintronic applications in TIs with broken TRS. We cannot completely rule out all other possibilities for the disappearance of FM ordering across the topological QPT. For example, the ARPES results (Fig. 3) show that the properties of itinerant carriers also change with Se content, which may affect an itinerant-driven FM mechanism.

## Supplementary Materials

www.sciencemag.org/cgi/content/full/339/6127/1582/DC1

Materials and Methods

Supplementary Text

Figs. S1 to S17

## References and Notes

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵Materials and methods are available as supplementary materials on
*Science*Online. - ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
**Acknowledgments:**We thank X. Dai, Z. Fang, and S. C. Zhang for helpful discussions. We thank Y. G. Zhao, P. S. Li, X. L. Dong, and Y. Wu for assistance with superconducting quantum interference device measurements. This work is supported by the Natural Science Foundation and Ministry of Science and Technology of China and the Chinese Academy of Sciences.