Information Transduction Capacity of Noisy Biochemical Signaling Networks

See allHide authors and affiliations

Science  21 Oct 2011:
Vol. 334, Issue 6054, pp. 354-358
DOI: 10.1126/science.1204553


Molecular noise restricts the ability of an individual cell to resolve input signals of different strengths and gather information about the external environment. Transmitting information through complex signaling networks with redundancies can overcome this limitation. We developed an integrative theoretical and experimental framework, based on the formalism of information theory, to quantitatively predict and measure the amount of information transduced by molecular and cellular networks. Analyzing tumor necrosis factor (TNF) signaling revealed that individual TNF signaling pathways transduce information sufficient for accurate binary decisions, and an upstream bottleneck limits the information gained via multiple integrated pathways. Negative feedback to this bottleneck could both alleviate and enhance its limiting effect, despite decreasing noise. Bottlenecks likewise constrain information attained by networks signaling through multiple genes or cells.

Signaling networks are biochemical systems dedicated to processing information about the environment provided by extracellular stimuli. Large populations of cells can accurately sense signaling inputs, such as the concentration of growth factors or other receptor ligands, but this task can be challenging for an individual cell affected by biochemical noise (13). Noise maps an input signal to a distribution of possible output responses, which can cause loss of information about the input. For example, a cell cannot reliably distinguish different inputs that, because of noise, can generate the same output (Fig. 1A).

Fig. 1

Information theoretic analysis of cell signaling fidelity. (A) Schematic showing information loss due to overlapping noisy response distributions. (B) Diagram of the TNF–NF-κB signaling pathway represented in biochemical form (left) and as a noisy communication channel (right). (C) Experimental flowchart for using immunocytochemistry to sample the conditional response distribution at single-cell resolution and resulting four-dimensional compendium of multiple responses in cells of multiple genetic backgrounds to multiple TNF concentrations, at multiple time points. The data were collected in a single experiment, allowing controlled, quantitative comparisons along each dimension. (D) Distributions of noisy NF-κB nuclear translocation responses to 30-min TNF exposure (examples shown at top) used to compute the channel capacity of the TNF–NF-κB pathway. Scale bars, 20 μm.

Conventional metrics related to the standard deviation or variance of the response distribution measure noise magnitude (48), but fail to elucidate how noise quantitatively affects the accuracy of information processing in single cells. By contrast, an information theoretic approach (Fig. 1B), and the metric of mutual information in particular, can quantify signaling fidelity in terms of the maximum number of input values that a cell can resolve in the presence of noise. Such methods have been commonly used to evaluate man-made telecommunication systems (9) and more recently in computational neuroscience and in analyses of transcriptional regulatory systems (1014), but have not been applied to biochemical signaling networks. We developed a general integrative theoretical and experimental framework to predict and measure the mutual information transduced by one or more signaling pathways. Applying this framework to analyze a four-dimensional compendium of single-cell responses to tumor necrosis factor (TNF) (Fig. 1C, see also SOM section 1), an inflammatory cytokine that initiates stochastic signaling at physiologic concentrations spanning about four orders of magnitude (1521), shows that signaling via a network rather than a single pathway can abate the information lost to noise. Furthermore, an information bottleneck can restrict the maximum information a network can capture, and negative feedback potentially but not always relieves this limitation.

The mutual information, I(R;S), measured in bits, is the binary logarithm of the maximum number of input signal values (S), such as ligand concentrations, that a signaling system can perfectly resolve on the basis of its noisy output responses (R) (9). One bit of information can resolve two different signal values, 2 bits resolves four values, etc. More generally,


The joint distribution P(R,S) determines the marginal distributions P(R) and P(S), and hence also the mutual information, and can be decomposed as P(R,S) = P(S) P(R|S). The response distribution, P(R|S), is experimentally accessible by sampling responses of individual isogenic cells to various signal levels (Fig. 1C), and its spread reflects the noise magnitude given any specific input. The signal distribution, P(S), reflects potentially context-specific frequencies at which a cell experiences different signal values. Although the amount of information might thus vary from case to case, one can also determine the maximal amount of transducible information, given the observed noise (see SOM section 2). This quantity, known as the channel capacity (9), is a general characteristic of the signaling system and the signal-response pair of interest and can thereby be experimentally measured without making assumptions about the (possibly nonlinear) relation between R and S, signal power, or noise properties.

Using immunocytochemistry, we assayed nuclear concentrations of the transcription factor nuclear factor κB (NF-κB) in thousands of individual mouse fibroblasts 30 min after exposure to various TNF concentrations (Fig. 1D). We chose this time point because NF-κB translocation peaks at 30 min regardless of the concentration used, initiating expression of early-response inflammatory genes (1922). The NF-κB response value in a single cell could yield at most 0.92 ± 0.01 bits of information, which is equivalent to resolving 20.92 = 1.9, or about 2, concentrations of the TNF signal, thus essentially only reliably indicating whether TNF is present or not. (See SOM sections 2.2 and 3 regarding the low experimental uncertainty.) A bimodal input signal distribution, P(S), with peaks at low and high TNF concentrations, maximizes the information (fig. S1), supporting the notion of essentially binary (digital) sensing capabilities of this pathway (18), although we did not observe bimodal output responses, P(R|S).

Noise also limits other canonical pathways, including signaling by platelet-derived growth factor (PDGF), epidermal growth factor (23), and G protein–coupled receptors (24), to ~1 bit (fig. S2, A to C, and table S1). Even the most reliable system we examined, morphogen gradient signaling through the receptor Torso in Drosophila embryos (25), was limited to 1.61 bits (fig. S2D and table S1), corresponding to about three distinguishable signal levels.

The pathways examined above are examples of individual biochemical communication channels (Fig. 1B) that capture relatively low amounts of information about signal intensity, which would allow only limited reliable decision making by a cell. However, information in biological systems is typically processed by networks comprising multiple communication channels, each transducing information about the signal. For instance, a transcription factor often regulates many genes, a receptor many transcription factors, and a diffusible ligand many cells. The integrated outputs of such multiple channels can provide more information about the signal than the output of any one channel (see SOM section 4). Subsequently, downstream signaling processes that converge to co-regulate common effectors, biological processes, or physiologic functions can provide the point needed to integrate the multiple outputs to realize the benefit of increased aggregate information (fig. S3). To provide a unified framework for analyzing such various networks, we first theoretically investigated the information gained by network signaling in general, then experimentally tested the predictions made by the theory when applied to a specific system.

We considered two information theoretic models, similar to models of population coding in neural systems (2628), for transmitting a signal S through multiple channels to the responses R1, R2, …, Rn, under the assumption of Gaussian variables (see SOM section 5). The bush model uses independent channels (topologically resembling an upside-down shrub) (Fig. 2A), whereas the tree model signals through a common channel (“trunk”) to the intermediate, C, before diverging into independent branches (Fig. 2B). The information resulting from the bush model is

Fig. 2

Information gained by signaling through a network comprising multiple communication channels. (A) Schematic of a bush network with independent channels lacking an information bottleneck. (B) Schematic of a tree network with channels sharing a common trunk that forms an information bottleneck. Circles represent noise introduced in the indicated portions of the signaling network; see text for definition of symbols. (C) Comparison of bush and tree model predictions for the capacity of the TNF network to experimental values. At 30 min, the NF-κB and ATF-2 pathways together capture more information about TNF concentration than either pathway alone (bars 1 to 3), and the tree rather than bush model accurately predicts this increase (bars 3 to 5). The tree model further predicts a receptor-level bottleneck of 1.26 ± 0.13 bits (bar 6). (D) Joint distribution of NF-κB and ATF-2 responses to 30-min stimulation of TNF. Each data point represents a single cell, and each concentration of TNF examined is shown with a distinct color.


where σS2 is the variance of the signal distribution, and σSR2 is the noise (variance) introduced in each branch. Thus, the information can grow logarithmically with the number of branches without an upper bound. In contrast, the information resulting from the tree model is


where σSC2 and σCR2 are the trunk and branch noises, respectively (see SOM section 3.3). As the number of branches increases, the information asymptotically approaches an upper limit equal to the mutual information between the input signal and the common intermediate. Thus, the information lost to noise in the trunk determines the maximum throughput of a tree network.

The key difference between bush and tree networks is the absence or presence of this trunk-based information bottleneck. The biochemical structure of a network can resemble a tree, but if there is little loss of information upstream, the bush model lacking a bottleneck might best estimate the capacity of the network. Additionally, the bush and tree models make various semiquantitative predictions (see SOM section 6), such as the information captured by a network based on the capacities of its component pathways. For example, for a bush network comprising two pathways each with 1-bit responses, Eq. 2 implies σS2/σSR2=3 and that together they should yield 12log2(1+2(3))=1.4  bits.

TNF activates the NF-κB and c-Jun N-terminal kinase (JNK) pathways, stimulating nuclear localization of NF-κB and phosphorylated activating transcription factor–2 (ATF-2) (fig. S4), respectively (29). To determine if the TNF signaling network contains an appreciable upstream information bottleneck limiting the information captured by these pathways, we examined whether the bush (bottleneck absent) or tree (bottleneck present) network model better approximates the network (fig. S5). The models are applicable because the NF-κB (Fig. 1D) and ATF-2 (fig. S6) response distributions are approximately Gaussian at all TNF concentrations. We found that NF-κB alone yielded at most 0.92 bits of information about TNF concentration, and ATF-2 alone yielded at most 0.85 ± 0.02 bits (fig. S1B and table S1). Together, the bush model predicts that these pathways jointly yield 1.27 ± 0.01 bits (Fig. 2C), and a similar model assuming independent pathway responses that are not necessarily Gaussian likewise predicts an increase to 1.13 ± 0.01 bits. The actual information determined by dual-staining immunocytochemistry (Fig. 2D) was 1.05 ± 0.02 bits, much lower than both predictions (Fig. 2C), demonstrating that the bush model does not approximate the TNF network well. In contrast, the tree model predicts 1.03 ± 0.01 bits, matching the experimental value within error (Fig. 2C), and also correctly predicts the statistical dependency between the responses given the signal (fig. S7).

The correspondence between the tree model predictions and experimental measurements strongly indicates that the network contains an information bottleneck. The tree model predicts that the maximum information that can pass through the bottleneck is 1.26 ± 0.13 bits (Fig. 2C), corresponding to just 21.26 = 2.3 distinguishable TNF concentrations. The known biochemistry of TNF signaling implies that the bottleneck (trunk) comprises the steps of TNF receptor complex activation common to both pathways, including ligand binding, receptor trimerization, and complex formation and activation. Because all TNF signaling passes through the receptor complex, multiple pathways in the TNF signaling network, activated at the 30-min time point, only modestly increase the information about TNF concentration regardless of the number of pathways or their fidelity.

We next explored whether negative feedback, which can reduce noise (12, 30, 31), might alleviate the receptor-level signaling bottleneck. The information captured by a single channel (Eq. 2, n = 1) can be written as 12log2(σR2/σSR2). Thus, negative feedback can have equivocal effects on information, depending on the balance of the tendencies for negative feedback to reduce both the dynamic range of the signaling response (32), represented by the response variance σR2, and noise, represented by σSR2. Indeed, comparison of wild-type cells and cells lacking A20 (fig. S8), an inhibitor of TNF receptor complexes whose expression is up-regulated by NF-κB (33) (Fig. 3A), showed that A20-mediated negative feedback increases information at the 30-min time point, but decreases it at 4 hours (Fig. 3B).

Fig. 3

Effect of negative feedback to the bottleneck on information transfer. (A) TNF signaling network diagram showing A20-mediated negative feedback to the information bottleneck. (B) Comparison of information about TNF concentration captured with and without A20 negative feedback. The information is larger at 30 min but smaller at 4 hours in wild-type cells as compared to A20−/− cells. (C and D) Schematic of NF-κB dynamics in wild-type and A20−/− mouse fibroblasts exposed to saturating concentrations of TNF. Average dynamics (black) and the expected magnitudes of the dynamic range (double arrow) and noise (single arrow) are shown. See fig. S9 for experimental support. (E) Comparison of NF-κB responses to zero (basal) or saturating concentrations of TNF. Differences in the means with and without TNF indicate the dynamic range, and error bars (SD) indicate the noise.

To understand these different outcomes, we examined how A20 affects the dynamic range and noise at either time point. At the early time point, constitutively expressed A20 inhibits basal NF-κB activity, but TNF does not induce A20 expression rapidly enough to affect saturating levels of NF-κB at 30 min (Fig. 3, C and D, and fig. S9) (17, 34). Hence, A20 negative feedback decreases noise, primarily at low TNF concentrations, and also increases the dynamic range by lowering basal NF-κB levels (Fig. 3E and fig. S10A), explaining why information at 30 min is higher for wild-type than for A20−/− cells (Fig. 3B). In contrast, at the late time point, A20 is increased in wild-type cells (17, 34). The negative feedback decreases noise at all TNF concentrations but also decreases the dynamic range by strongly suppressing the maximum inducible NF-κB activity (Fig. 3E and fig. S10A). The net effect is lower information for wild-type versus A20−/− cells at 4 hours (Fig. 3B).

We observed that A20 negative feedback similarly both improves and limits information at the early and late time points, respectively, for ATF-2 alone, or together with NF-κB (Fig. 3B and fig. S10B), consistent with A20 affecting the portion of the network common to both pathways. Nevertheless, the maximal information about TNF concentration acquired with or without A20-mediated negative feedback was still ~1 bit, suggesting limited advantages for mitigating the information bottleneck in this pathway by using negative feedback.

We next considered whether networks comprising multiple target genes can capture substantial amounts of information through time integration. If the target gene product lifetime is long compared to its transcription and translation time scales, the accumulated protein concentration is approximately proportional to the time integral of signaling activity, thereby averaging out temporal fluctuations (35, 36). However, the biochemical readout of protein synthesis can introduce extra noise, confounding determination of the information contained in the time integral. Fortunately, the maximum information captured by a tree network, in which the time integral of transcription factor activity is the intermediate signal activating multiple independent target genes (Fig. 4A, inset), is determined by the trunk (time integration) rather than branch noise (readout mechanism). We measured the information captured by such tree networks in cells stably transfected with different copy numbers (1.8-fold difference, as determined by polymerase chain reaction) of the gene coding for a stable green fluorescent protein (GFP) (37) reporting on NF-κB activity (Fig. 4B). Using the tree model to extrapolate the extent of the bottleneck, under the assumption that ~10 hours of TNF exposure induces similar expression level and noise for each gene, indicates that 1.64 ± 0.36 bits is the maximum information that integrating NF-κB activity over the experimental time period can yield about TNF concentration (Fig. 4A), regardless of the readout mechanism.

Fig. 4

Information gained by signaling through networks of multiple genes. (A) Plot shows the unique curve (solid black) determined by the tree model (inset), passing through the experimentally determined values (circles), for information as a function of the number of copies of a NF-κB reporter gene. The upper limit, corresponding to the maximum information captured by integrating NF-κB activity over time, is 1.64 ± 0.36 bits (blue dashed line). (B) Expression-level distributions of clonal cell lines containing different numbers of copies of an NF-κB reporter gene in response to ~10 hours of TNF exposure. (C) Time courses corresponding to individual cells showing cell-to-cell differences in the onset and rate of NF-κB reporter gene expression (left). In each cell, expression is nearly linear and deterministic in time, as quantified by the correlation coefficient (right) of the time course after onset of expression (shown schematically in inset on left).

To understand why information was only moderately higher compared to a single time point (1.64 versus 0.92 bits), we monitored GFP reporter gene expression in individual cells, finding that, for any given cell, GFP accumulated linearly in time in a nearly deterministic fashion, although its onset and accumulation rate varied from cell to cell (Fig. 4C). This is consistent with observations made with live cell probes (1820) showing NF-κB dynamics to be essentially deterministic over the experimental time scale within each cell, but distinct across cells. We thus conclude that the ability of time integration to increase the information about TNF concentration is limited by the lack of rapid temporal fluctuations that would otherwise be suppressed by integration over the 10-hour response.

Finally, we considered signaling via multiple cells, each considered as separate information channels within a network (Fig. 5A, inset). An ensemble of cells resembles a bush network if each cell directly and independently accesses the same signal, and because bush networks do not contain trunk-based bottlenecks, substantial increases in information might be obtained. To test this hypothesis, we analyzed the collective TNF response of different numbers of cells, as measured by immunocytochemistry. We varied cell number by considering cells within nonoverlapping circular regions of variable diameter (Fig. 5B) and used the average NF-κB response within each region to simulate cells contributing to a collective response in proportion to their NF-κB activity. The bush model predicts (Eq. 2), and the data confirm (Fig. 5A), that the information should increase logarithmically with the number of independently signaling cells functioning collectively.

Fig. 5

Information gained by signaling through networks of multiple cells. (A) Comparison of experimentally measured information obtained by collective cell responses (circles) versus logarithmic trend (solid black line) predicted the bush model (inset). (B) Schematic of methodology used to measure collective cell responses.

Moreover, we found that networks of just 14 cells can yield up to 1.8 bits of information, far greater than the other network types analyzed above. Because ensembles of this size can plausibly experience a similar concentration of a diffusing signal such as TNF and function collectively (21, 38) [e.g., TNF-activated blood vessel endothelial cells (39)], collective cell behavior can effectively increase the information gained and produce responses that can discriminate between many TNF concentrations. Nonetheless, networks relying on cell-cell communication can still contain bottlenecks. For instance, TNF can be secreted by macrophages stimulated by lipopolysaccharide (LPS) from invading bacteria, with the information about the initial LPS dose lost within the macrophage signaling networks before secretion of TNF.

By treating biochemical signaling systems as information theoretic communication channels, we have rigorously and quantitatively shown that, in a single cell, noise can substantially restrict the amount of information transduced about input intensity, particularly within individual signaling pathways. The bush and tree network models, which provide a unified theoretical framework for analyzing branched motifs widespread in natural and synthetic signaling networks, further demonstrated that signaling networks can be more effective in information transfer, although bottlenecks can also severely limit the information gained. Receptor-level bottlenecks restrict the TNF and also PDGF signaling networks (fig. S11) and may be prevalent in other signaling systems.

We explored several strategies that a cell might use to overcome restrictions due to noise. We found that negative feedback can suppress bottleneck noise, which can be offset by concomitantly reduced dynamic range of the response. Time integration can increase the information transferred, to the extent that the response undergoes substantial dynamic fluctuations in a single cell over the physiologically relevant time course. The advantage of collective cell responses can also be substantial, but limited by the number of cells exposed to the same signal or by the information present in the initiating signal itself.

Responses incorporating the signaling history of the cell might also increase the information (40, 41). For instance, responses relative to the basal state (fold-change response) might be less susceptible to noise arising from diverse initial states (23), although this does not necessarily translate into large amounts of transferred information (table S1). Similarly, for the reporter gene system described here (fig. S12), ~0.5 bits of additional information can be obtained if a cell can determine expression levels at both early and late time points. However, noise in the biochemical networks that a cell uses to record earlier output levels and to later compute the final response may nullify the information gain potentially provided by this strategy. Overall, we anticipate that the information theory paradigm can extend to the analysis of noise-mitigation strategies and information-transfer mechanisms beyond those explored here, in order to determine what specific signaling systems can do reliably despite noise.

Supporting Online Material

Materials and Methods

SOM Text

Figs. S1 to S12

Table S1

References (4255)

References and Notes

  1. Acknowledgments: We thank A. Hoffmann, M. Simon, S. Shvartsman, C. Cohen-Saidon, and U. Alon for sharing data and materials; A. Ganesan and H. Chang for experimental assistance; and P. Iglesias, Y. Qi, and A. Feinberg for insightful discussions and reviewing drafts of the manuscript. This work was supported by NIH grants GM072024 and RR020839 (R.C., A.R., C.J.W., and A.L.) and CA132629 (I.N.), the Medical Scientist Training Program at the Johns Hopkins University (R.C.), and, in early stages of the work, the Los Alamos National Laboratory Directed Research and Development program (I.N.).

Stay Connected to Science

Navigate This Article