## Abstract

The anomalous Hall effect is a fundamental transport process in solids arising from the spin-orbit coupling. In a quantum anomalous Hall insulator, spontaneous magnetic moments and spin-orbit coupling combine to give rise to a topologically nontrivial electronic structure, leading to the quantized Hall effect without an external magnetic field. Based on first-principles calculations, we predict that the tetradymite semiconductors Bi_{2}Te_{3}, Bi_{2}Se_{3}, and Sb_{2}Te_{3} form magnetically ordered insulators when doped with transition metal elements (Cr or Fe), in contrast to conventional dilute magnetic semiconductors where free carriers are necessary to mediate the magnetic coupling. In two-dimensional thin films, this magnetic order gives rise to a topological electronic structure characterized by a finite Chern number, with the Hall conductance quantized in units of *e*^{2}/*h* (where *e* is the charge of an electron and *h* is Planck’s constant).

The anomalous Hall effect (AHE) (*1*, *2*), in which a voltage transverse to the electric current appears even in the absence of an external magnetic field, was first detected in ferromagnetic (FM) metals in 1881 and later found to arise from the spin-orbit coupling (SOC) between the current and magnetic moments. Recent progress on the mechanism of AHE has established a link between the AHE and the topological nature of the Hall current by adopting the Berry-phase concepts (*3*–*5*) in close analogy to the intrinsic spin Hall effect (*6*, *7*). Given the experimental discovery of the quantum Hall (*8*) and the quantum spin Hall (QSH) effects (*9*, *10*), it is natural to ask whether the AHE can also be quantized.

A simple mechanism for a quantum anomalous Hall (QAH) insulator has been proposed in a two-band model of a two-dimensional (2D) magnetic insulator (*11*). In the limit of vanishing SOC and large enough exchange splitting, the majority spin band is completely filled and the minority spin band is empty. When the exchange splitting is reduced, the two bands intersect each other, leading to a band inversion. The degeneracy at the interaction region can be removed by turning on the SOC, giving rise to an insulator state with a topologically nontrivial band structure characterized by a finite Chern number and chiral edge states characteristic of the QAH state (*11*). Alternative mechanisms of realizing the QAH state include bond currents on a honeycomb lattice (*12*) and the localization of the band electrons (*13*). However, these mechanisms may be harder to realize experimentally.

The crucial criteria for realizing a QAH state are (i) a FM 2D insulator that breaks the time-reversal symmetry and (ii) a band inversion transition with strong SOC. QSH insulators are a good starting point for the search for the QAH effect because they satisfy the second criterion. Magnetically doped HgMnTe has been proposed as a candidate for the QAH insulator (*14*); however, because the Mn moments do not order spontaneously in HgMnTe, an additional, small Zeeman field is required.

Recently, tetradymite semiconductors Bi_{2}Te_{3}, Bi_{2}Se_{3}, and Sb_{2}Te_{3} have been theoretically predicted and experimentally observed to be topological insulators (TIs) with the bulk band gap as large as 0.3eV in Bi_{2}Se_{3} (*15**–**17*). As the thickness is reduced, a three-dimensional (3D) TI crosses over to a 2D TI in an oscillatory fashion (*18*). We predict that thin films made out of this family of compounds doped with proper transition metal elements (Cr or Fe) support the QAH state. Recent experimental progress has shown that well-controlled layer-by-layer molecular beam epitaxial thin-film growth can be achieved (*19*, *20*), and various transition metal elements (such as Ti, V, Cr, and Fe) can be substituted into the parent compounds with observable ferromagnetism even above 100 K (*21*–*23*).

We first discuss minimal requirements for the FM insulator phase in a semiconductor system doped with dilute magnetic ions under the assumption that the magnetic exchange among local moments is mediated by the band electrons. The whole system can then be divided into two subsystems describing the local moments and band electrons, respectively, coupled by a magnetic exchange term. If we only consider the spatially homogeneous phase, the total free energy of the system in an external magnetic field *H* can be written as_{L}_{/}* _{e}* is the spin susceptibility of the local moments/electrons,

*M*

_{L}_{/}

*denotes the magnetization for the local moment and electron subsystem, and*

_{e}*J*

_{eff}is the magnetic exchange coupling between them. In order to have a nonzero FM transition temperature (

*T*

_{c}), a sizable χ

*is needed [for a detailed analysis, see (*

_{e}*24*)]. In most dilute magnetic semiconductors, for example, (Ga

_{1−}

*Mn*

_{x}*)As, the electronic spin susceptibility is negligible for the insulator phase, and finite carrier concentration is required to mediate the magnetic coupling among local moments. The insulating Bi*

_{x}_{2}Se

_{3}, however, gains considerable spin susceptibility through the Van Vleck paramagnetism (

*25*), which is caused by the nonzero matrix element of the spin operator,

*S*, between the valence and conduction bands (

_{z}*24*). For the Bi

_{2}Se

_{3}family, the semiconductor gap is opened by the SOC between the bonding and antibonding

*p*orbitals, leading to large matrix elements (

*24*). Such a mechanism is absent in the GaAs system, which has

*s*-like conduction and

*p*-like valence bands.

The Van Vleck type spin susceptibility can be further enhanced by band inversion. To demonstrate this argument, we perform first-principles calculations (*24*) for the electronic structures and the spin susceptibility of Bi_{2}Se_{3} bulk (Fig. 1). The band inversion at the Γ point occurs when the the relative SOC strength λ/λ_{0} exceeds 0.5, and the spin susceptibility starts to increase appreciably. The susceptibility tensor is anisotropic, with χ* _{e}^{zz}*/χ

*≈ 1.1 estimated for the parent compound (without magnetic dopants). In the presence of dopants, the single-ion magnetic anisotropy will dominate and favor the off-plane orientation (*

_{e}^{xx}*23*,

*26*).

We show, by first-principles calculations, that the insulating magnetic ground state discussed above can be indeed obtained by a proper choice of magnetic dopants. Experiments (*21*–*23*) have suggested that the magnetic dopants, such as Ti, V, Cr, and Fe, will mostly substitute the Bi ions; we therefore concentrate on this situation. Because the nominal valence of Bi ions is 3+, a general rule is to find a transition metal element that may have a stable 3+ chemical state, so that no free carriers are introduced by this isoelectronic substitution. In addition, because of the coexistence of orbital and spin degrees of freedom of magnetic dopants (due to the partially filled *d* shells), we need a mechanism to quench these degrees of freedom and stabilize the insulating state.

The results shown in Fig. 2 suggest that an insulating magnetic state is obtained for Cr or Fe doping, whereas the states are metallic for Ti or V doping cases (*24*). To understand the results, we first point out that, for all the cases, the dopants are nearly in the 3+ valence state, and we always obtain the high-spin state because of the large Hund’s rule coupling of 3*d* transition metal ions. This will directly lead to the insulating state of Fe-doped samples, because the Fe^{3+} has five 3*d* electrons, favoring the *d* ^{5↑}*d* ^{0↓} configuration in a high-spin state and resulting in a gap between the majority and minority spins. For the Cr-doped case, the local environment of dopants, which substitute the Bi sites, is an octahedral formed by six nearest neighboring Se^{2−} ions. Such a local crystal field splits the *d* shell into *t*_{2}* _{g}* and

*e*manifolds. This splitting is large enough to stabilize the

_{g}*t*

_{2}

_{g}^{3↑}

*e*

_{g}^{0↑}

*t*

_{2}

_{g}^{0↓}

*e*

_{g}^{0↓}configuration of a Cr

^{3+}ion, resulting in a gap between the

*t*

_{2}

*and*

_{g}*e*manifolds. For the case of Ti or V doping, even the

_{g}*t*

_{2}

*manifold is partially occupied, leading to the metallic state. We note that, although the local density approximation (LDA) in the density functional theory may underestimate the electron correlation effects, the inclusion of electron-electron interaction*

_{g}*U*(such as in the LDA +

*U*method) should further enhance the gap [it may also reduce the

*p*-

*d*hybridization and

*T*

_{c}(

*27*)].

The energy gain because of the spin polarization is about 0.9 eV per Fe, 1.5 eV per Cr, 0.7 eV per V and 0.02 eV per Ti, respectively, which are large numbers except for the case of Ti substitution. From the spin splitting of *p* orbitals of the band electrons, the estimated effective exchange coupling *J*_{eff} between the local moments and the *p* electrons (*28*) is around 2.7eV for Cr and 2.8eV for Fe in Bi_{2}Se_{3}, comparable to that in GaMnAs (*28*, *29*). The exchange splitting introduced by the magnetic dopants can be estimated as Δ*E* = *xJ*_{eff}〈*S*〉, where *x* is the doping concentration and 〈*S*〉 is the mean field expectation value of the local spin. By using this value of *J*_{eff} and applying a mean field theory, we can estimate the FM Curie temperature to be of the order of tens of K, in the range accessible by experiments (*24*).

Once the FM order is achieved in TI, the QAH effect can be realized in 2D thin films of such systems. Because the bulk states are always gapped, we focus on the simplest low-energy effective Hamiltonian consisting of Dirac-type surface states only,*t*↑〉, |*t*↓〉, |*b*↑〉, and |*b*↓〉, where *t*, *b* represent the surface states sitting on the top and bottom surfaces and ↑, ↓ represent the spin up and down states, respectively. *v*_{F} is the Fermi velocity, *m _{k}* describes the tunneling effect between the top and bottom surface states,

*g*is the effective

*g*factor,

*k*

_{±}=

*k*±

_{x}*ik*, and

_{y}*M*represents the exchange field along the

*z*axis introduced by the FM ordering. For simplicity, spatial inversion symmetry is assumed, which requires that

*v*,

_{F}*g*, and

*M*take the same values for top and bottom surfaces. In the thick slab geometry (

*m*≈ 0), the spatially separated two pairs of surface states are well defined for the top and bottom surfaces. However, with the reduction of the film thickness, quantum tunneling between the top and bottom surfaces becomes more and more pronounced, giving rise to a finite mass term,

_{k}*m*, which can be expanded up to the second order as

_{k}*m*=

_{k}*m*

_{0}+

*B*(

*k*

_{x}^{2}+

*k*

_{y}^{2}), and the Hamiltonian can be rewritten in terms of the symmetric and antisymmetric combination of the surface states on top and bottom surfaces as

*h*(

*k*) =

*m*σ

_{k}*+*

_{z}*v*(

_{F}*k*σ

_{y}*−*

_{x}*k*σ

_{x}*), similar to the Bernevig-Hughes-Zhang model describing the low-energy physics in a HgTe/CdTe quantum well (*

_{y}*9*). When

*m*

_{0}

*B*< 0, band inversion occurs, and the system will be in the QSH phase if this is the only band inversion between two subbands with opposite parity. Regardless of whether this condition is satisfied, a strong enough exchange field will induce the QAH effect in this system, thanks to the presence of the σ

*matrix in the exchange field (*

_{z}*gM*σ

*) and the opposite signs of the Zeeman coupling terms in the upper and lower blocks of the effective Hamiltonian (Eq. 3). The exchange field increases the mass term of the upper block and reduces it for the lower block, breaking the time reversal symmetry. More importantly, a sufficiently large exchange field can change the Chern number of one of the two blocks. As illustrated in Fig. 3, if the four-band system is originally in the topologically trivial phase, the exchange field will induce a band inversion in the upper block and push the two subbands in the lower block even farther away from each other. Therefore, the 2D model with a negative mass in the upper block contributes*

_{z}*e*

^{2}/

*h*(where

*e*is the charge of an electron and

*h*is Planck’s constant) for the Hall conductance. On the other hand, if the system is originally in the topologically nontrivial phase, both blocks have inverted band structures. In this case, a sufficiently large exchange field can increase the band inversion in the upper block and release it in the lower block. Again, the negative mass in the upper block contributes

*e*

^{2}/

*h*for the Hall conductance. Such a mechanism is general for the thin-film TI systems with FM ordering; it is guaranteed by the fact that the surface states on the top and bottom surfaces have the same

*g*factor. In HgMnTe, a further assumption that the electron and hole subbands have opposite signs of the exchange splitting (

*14*) is required. This further justifies the robustness of the present proposal.

We carried out quantitative first-principles calculations for the Hall conductance in Bi_{2}Se_{3} films, based on the Kubo formula (*24*). A spatial uniform exchange field is included to take into account the effect of magnetization in the FM state at the mean field level. In Fig. 4, A to C, we plot the lowest four subband levels at Γ point as a function of the exchange field. The level crossings between the lowest conduction bands (blue lines) and valence bands (red lines) are found for all three values of the layer thickness, signaling a quantum phase transition to the QAH state. In an insulator, where the chemical potential is located inside the energy gap between conduction and valence subbands, the Hall conductance is determined by the first Chern number of the occupied bands and must be an exact integer in the unit of *e*^{2}/*h*. The calculated Hall conductance for Bi_{2}Se_{3} thin films with three typical thicknesses (Fig. 4D) has a jump from 0 to 1 at the corresponding critical exchange field for the level crossing.

Experimentally, the best way to see the QAH effect is to measure the Hall conductance as the function of gate voltage that tunes the chemical potential. A quantized plateau in Hall conductance should be observed when the chemical potential is inside the gap (fig. S2). In real samples, a small amount of bulk carriers would always be present; however, if the concentration is low enough, they will be localized by disorder in two dimensions and will not affect the precise quantization of the Hall plateau.

## Supporting Online Material

www.sciencemag.org/cgi/content/full/science.1187485/DC1

Materials and Methods

SOM Text

References and Notes

## References and Notes

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
Materials and methods are available as supporting material on
*Science*Online. - ↵
- ↵
- ↵
- ↵
- ↵
- We acknowledge valuable discussions with C. X. Liu, X. L. Qi, Q. Niu, and S. Q. Shen and support from the National Science Foundation of China, the 973 Program of China (nos. 2007CB925000 and 2010CB923000), and the International Science and Technology Cooperation Program of China. S.C.Z. is supported by the U.S. NSF under grant number DMR-0904264.