Skip to main content
  • Original Articles
  • Published:

Lipopolysaccharide-Induced Cytokine Cascade and Lethality in LTα/TNFα-Deficient Mice

Abstract

Background

Tumor necrosis factor α (TNF-α) is often considered the main proinflammatory cytokine induced by lipopolysaccharide (LPS) and consequently the critical mediator of the lethality associated with septic shock.

Materials and Methods

We used mice carrying a deletion of both the lymphotoxin α (LT-α) and TNF-α genes to assess the role of TNF in the cytokine cascade and lethality induced by LPS.

Results

Initial production of IL-1α, IL-1β, IL-6, and IL-10 is comparable in wild-type and mutant mice. However, at later times, expression of IL-1α, IL-1β, and IL-10 is prolonged, whereas that of IL-6 decreases in mutant mice. Expression of IFN-γ is almost completely abrogated in mutants, which is in agreement with a more significant alteration of the late phase of the cytokine cascade. We measured similar LD50 (600 µg) for the intravenous injection of LPS in mice of the three genotypes (+/+, +/−, −/−), demonstrating that the absence of TNF does not confer long-term protection from lethality. However, death occurred much more slowly in mutant mice, who were protected more efficiently from death by CNI 1493, an inhibitor of proinflammatory cytokine production, than were wild-type mice.

Discussion

Thus, while TNF-α is not required for the induction of these cytokines by LPS, it modulates the kinetics of their expression. The lethality studies simultaneously confirm a role for TNF as a mediator of early lethality and establish that, in the absence of these cytokines, other mediators take over, resulting in the absence of long-term protection from LPS toxicity.

Introduction

The cytokine network is defined by cascades of cytokines, positive and negative feedbacks, autocrine effects, and synergistic actions. Knowledge of this network has been essentially achieved by the use of recombinant cytokines and specific antibodies. More recently, transgenic and knockout animals have been successfully employed to further characterize the relative contribution of each cytokine. Tumor necrosis factor a (TNF-α) is considered a key cytokine within the network of pro- and anti-inflammatory cytokines. It has been well established that in vitro TNF-α can trigger its own production (1) as well as the release of interleukin 1 (IL-1) (1), IL-6 (2), IL-8 (3), IL-10 (4), macrophage colony stimulating factor (5), and many other cytokines by different cell types, including monocytes and macrophages, fibroblasts, endothelial cells, and epithelial cells. Similarly, in vivo, TNF-α is able to induce in the bloodstream the appearance of IL-1 (6), IL-6 (7), IL-8 (8), IL-10 (9), IFN-γ (10), and other cytokines.

Endotoxins (lipopolysaccharide; LPS) from gram-negative bacteria are potent TNF-α inducers. Following LPS or bacteria injection, the peak of circulating TNF occurs rapidly within 90 min (1114). The TNF-α peak precedes that of IL-1, IL-6, and IL-8. TNF-α mRNA levels peak in vivo at 15 min after LPS injection and at 1 hr in in vitro assays (15). This has led to the proposal that TNF-α could orchestrate the production of other cytokines. This model has been supported by experiments with anti-TNF-α. For example, in the baboon model, in which a lethal injection of Escherichia coli led to the appearance of circulating TNF-α, IL-1β, and IL-6, pretreatment of the animals with a monoclonal antibody anti-TNF-α 2 hr before the E. coli infusion significantly reduced the levels of any detectable cytokines in plasma (16). This observation has been confirmed following injection of endotoxin and gram-positive bacteria (1719) and extended to IL-8 (20). Similar findings were obtained using soluble TNF receptor, soluble TNF receptor-Fcγ chimeric molecule, and two TNF receptors bound covalently to polyethylene glycol to neutralize TNF-α (2123).

However, not all in vivo experiments support a central role for TNF-α in the induction of the cytokine cascade. This is particularly the case for interleukin-1α (24), circulating IL-10 (9,25), and interferon gamma (IFN-γ) (24,26). Moreover, LPS can probably directly induce the production of several cytokines. For example, in a whole-blood assay, suboptimal doses of LPS did not lead to the release of detectable TNF-α, but they induced significant amounts of IL-8 (27). In addition, different in vitro as well as in vivo inhibition of TNF by either specific antibodies, pentoxifylline, or adenosine kinase inhibitor failed to modify the levels of other cytokines, including IL-1α, IL-1β, and IL-6 (2831).

Thus, even within the simple model of the response to LPS, the role of TNF-α in the induction of the cytokine cascade has not been clearly established. To further analyze the role of TNF-α in this cascade, we investigated the production of various cytokines in mice that were rendered deficient for the lymphotoxin-α (LT-α) and TNF-α genes (32,33). We studied the in vitro LPS activation of peritoneal macrophages and in vivo levels of circulating cytokines following LPS injection.

In conjunction with a central role for TNF-α in the cytokine cascade, and on the basis of the pioneering work of Cerami’s group (34,35) using specific anti-TNF-α antibodies, it has been proposed that TNF is a major actor responsible for the lethality in endotoxin shock in animal models as well as in models of bacteria-induced lethality (17,19,36,37). However, the galactosamine model of septic shock has often been employed to demonstrate the beneficial effects of neutralizing TNF-α. Although LPS-induced lethality is essentially mediated by TNF in this model (38), it may not be the case in other models. For example, carrageenan-sensitized mice were not protected by anti-TNF antibodies against a lethal dose of LPS (39). In addition, p55 TNF receptor-deficient mice were insensitive to the lethal effect of LPS in galactosamine-injected mice but were as sensitive as wild-type animals to lethal doses of LPS in the absence of galactosamine (40). Furthermore, the absence of any beneficial effects of treatment with monoclonal anti-TNF-α antibodies or soluble receptor chimeric molecules in septic patients (41,42) illustrated that blocking TNF-α may not be sufficient for preventing death in these patients. Thus it is of interest to analyze the resistance of LT-α/TNF-α-deficient mice when treated with lethal doses of LPS in the absence of any galactosamine treatment.

Materials and Methods

Mice

The generation of LT-α/TNF-α-deficient mice has been described elsewhere (33). The animals of the three genotypes used in this study were of a comparable 129sv × C57BL/6 genetic background. Mice were bred under specific pathogenfree conditions at the animal facility of the Institut Gustave Roussy or of the Institut de Recherches sur le Cancer. Experiments were performed with 8- to 12-week-old mice of both sexes unless otherwise indicated.

Cytokine Induction by LPS In Vivo

For each time point, groups of 8 to 10 mice were injected in the retro-orbital sinus with 100 µg of S. typhimurium LPS (Difco, Detroit, MI) in 100 µl 0.9% NaCl. Mice were bled retro-orbitaly and sacrificed at the end of the indicated period. One hundred microliter-aliquots of the plasma were kept at −70°C until assayed for cytokine content.

In Vitro Macrophage Stimulation

Female mice were injected intraperitonealy with 1.5 ml of thioglycollate broth 5 days before they were sacrificed. Cells were harvested by peritoneal lavage with cold phosphate-buffered saline (PBS), washed three times in cold RPMI-1640 medium, resuspended at a concentration of 106 cells/ml, and allowed to adhere to plastic as described elsewhere (43). Cells (5 × 105 cells per well in 0.5 ml of culture medium) were stimulated with the indicated concentration of purified Escherichia coli 0111:B4 LPS (Sigma). After 4 or 18 hr of incubation, the culture medium was recovered and frozen or immediately analyzed for cytokine content. For the analysis of IL-1α and IL-1β production, cell lysates were prepared by repeated freezing and thawing.

Cytokine Immunoassays

ELISAs specific for murine TNF-α, IL-1α, IL-1β, IL-6, IL-10, and IFN-γ were obtained from Endogen (Cambridge, MA). Serial dilutions were used to determine the cytokine concentration by comparison with the internal standard according to the manufacturer’s instructions. For the production of IL-1α and IL-1β by macrophages, the supernatants and cell lysates were assayed separately.

LPS-Induced Lethality

Mice were injected in the retro-orbital sinus with 100 µl of 0.9% NaCl containing the indicated amount of S. typhimurium LPS, except for the highest doses of LPS (1.2 and 1.6 mg) which were injections of 120 and 160 µl, respectively. Lethality was monitored for 7 days. For the CNI 1493 experiments, mice were injected intraperitonealy with 200 µl of water containing the indicated amount of CNI 1493, 90 min before the injection of LPS in the retro-orbital sinus.

Results

Cytokine Expression In Vivo

To analyze the cytokine cascade induced by LPS in mice devoid of TNFs, wild-type, heterozygous, and homozygous LT-α/TNF-α-deficient mice were injected intravenously with 100 µg of LPS. The levels of circulating cytokines were determined with specific ELISAs (see Materials and Methods) and the results presented as the mean ± SEM (8 to 10 animals were analyzed for each time point). Figure 1 presents the results for TNF-α. As expected, no TNF-α could be detected in homozygous mutant mice. In wild-type and heterozygous mice, no TNF-α could be detected at the 3-hr time point or later, which is in agreement with the transient induction of TNF-α by LPS. In heterozygotes, the mean level at 1 hr 30 min was 5-fold lower than in wild type, which confirms our previous observation of a reduction in expression that exceeds the gene dosage (33).

Fig. 1
figure 1

Cytokine plasma levels following LPS injection in wild-type and LT-α/TNF−α mutant mice

Mice of the three genotypes (8 to 10 for each time point; wild-type mice: white bars; heterozygous mice: gray bars; LT-α/TNF-α−/− mice: black bars) were injected with 100 µg of S. typhimurium LPS and bled 1.5 hr thereafter. The concentration of TNF-α, IL-1α, IL-1β, IL-6, IL-10, and IFN-γ in the plasma were determined with specific immunoassays. Results are presented as the mean and SEM. When relevant, the statistical significance of the differences between genotypes is indicated (*p < 0.05; **p < 0.01 by a t test).

Figure 1 presents the circulating levels of IL-1α and β, IL-6, IL-10, and IFN-γ. In wild-type and heterozygous mice, the maximal level of circulating IL-1α was observed at 1 hr 30 min, whereas it occurred at 3 hr in homozygous mutants. The same change in kinetics was observed for IL-1β. Thus, in the absence of TNF, expression of IL-1α and β occurs later. Our data further suggest that there is no major IL-1 deficit in the homozygous mutants, as the lower circulating levels at 1 hr 30 min could be compensated by the subsequent increase at 3 hr.

Comparable levels of IL-6 were detected at 1 hr 30 min in wild-type, heterozygous, and homozygous mutants, establishing that TNF is not required for early IL-6 induction by LPS. At 3 hr and later, the levels of circulating IL-6 were three to four times lower in homozygous mutants than in wild-type mice (p < 0.01 and 0.02 at 3 and 6 hr, respectively). This profile indicates a biphasic expression with a TNF-independent early phase and a TNF-dependent later phase. The high level of IL-6 at 1 hr 30 min in heterozygotes was associated with an important dispersion and was not statistically significant.

Plasma levels of IL-10 at 1 hr 30 min were comparable for the three genotypes. At 3 hr, however, this level was unchanged in homozygotes but had decreased 4- to 5-fold in wild type and heterozygotes (p < 0.05). No significant levels of IL-10 were observed later in any of the genotypes, probably reflecting the limited sensitivity of the assay. Thus the kinetics of IL-10 expression is prolonged in the LT-α/TNF-α−/− mice.

High levels of circulating IFN-γ were detected in wild-type mice at 6 hr 30 min and 10 hr and absent at 1 hr 30 min and 3 hr. By contrast, the levels in homozygous mutants were 30-fold lower than those in wild-type mice at both 6 hr 30 min and 10 hr (p < 0.01 and 0.05, respectively). Thus production of IFN-γ in response to LPS is strongly TNF-dependent. This was confirmed by the IFN-γ levels in the heterozygotes which were three to four times below those of the wild type and thus closely followed the TNF levels.

Cytokine Expression In Vitro

Macrophages are one major target of LPS and are responsible for most of the TNF-α production in vivo. To assess how much of the difference in the cytokine cascade among the three genotypes is attributable to the macrophage response, we stimulated elicited peritoneal macrophages in vitro with LPS. Figure 2 presents the results of an ELISA analysis of the culture supernatants for TNF-α and IL-6 after 4 and 18 hr of incubation with 100 ng/ml or 10 µg/ml of LPS. For IL-1α and IL-1β, the data presented are the sum of the intracellular and secreted amounts of proteins. Each analysis was performed in triplicate and the results are presented as the mean ± SEM.

Fig. 2
figure 2

Cytokine production by elicited macrophages stimulated in vitro by LPS

Thioglycollate-elicited macrophages (5 × 105 cells in 0.5 ml of culture medium) from the three genotypes (same symbols as in Fig. 1) were stimulated with the indicated amount of purified Escherichia coli 0111:B4 LPS. After 4 or 18 hr of incubation, the culture medium was harvested and analyzed for cytokine content with specific immunoassays. The experiment was performed in triplicate and the results are presented as the mean and SEM. For IL-1α and IL-1β, the activity presented is the sum of that present in the culture medium and in the cell lysate.

No immunoreactive TNF-α could be detected in the supernatant of macrophages of homozygous mice. The difference between wild-type and heterozygous macrophages increased with time and exceeded the prediction of a simple gene dosage, confirming our previous analysis of TNF-α expression (33).

IL-6 release in the culture supernatants of heterozygous and homozygous cells was lower than that for wild-type cultures. This was true for both concentrations of LPS as well as at 4 and 18 hr.

IL-1α production by homozygous macrophages was 7- and 4-fold less than that of the wild type at 4 and 18 hr, respectively. Most of the activity was present in cell lysates, as about 1 % and 2% of the total immunoreactive material could be detected in the supernatant at 4 and 18 hr, respectively. A similar profile with comparable differences between wild-type and homozygous cells was observed for IL-1β. The proportion of the IL-1β activity present in the supernatant was 5% and 25% at 4 and 18 hr, respectively.

No detectable IL-10 production could be observed in cultures of elicited macrophages.

For all the analyzed cytokines, the levels produced by heterozygous cells were between those of the wild-type and homozygous cells, confirming a TNF dose-responsive expression of cytokines.

LPS-Induced Lethality

TNF-α has been proposed to play a critical role in the lethality induced by endotoxin shock. Accordingly, mice unable to produce any TNF should be resistant to high doses of LPS. To test this prediction, groups of 10 mice were injected intravenously with doses of LPS ranging between 200 µg and 1.6 mg. Survival was monitored daily for 7 days (Fig. 3), with no death occurring after this period. In wild-type mice of 129sv × C57BL/6 genetic background, an LD50 of 600 µg was observed (Fig. 3A). In homozygous LT-α/TNF-α−/− mice, the same LD50 of 600 µg was observed (Fig. 3C), establishing that the absence of TNF did not render these mice resistant to LPS. The observation that at the highest doses of LPS (1200 and 1600 µg) one homozygous mouse survived, could suggest, however, a slight modification in the dose response. In contrast with these results on long-term survival is the observation that a major role for TNF was apparent in the death kinetics. In wild-type mice, as is usually observed, all deaths occurred during the first 3 days (Fig. 3A). In homozygous mutants, only one-third of the deaths took place during the first 3 days, whereas one-fifth of the deaths (8/39) did not occur until the sixth day. Moreover, although the death kinetics is not much influenced by the dose of LPS in wild-type mice, a clear dose dependence can be observed in homozygous mutants. Thus, at intermediate doses of LPS such as the LD50, the difference between the two models is particularly apparent; all the deaths occurred within the first 2 days in wild-type mice and between days 4 and 6 in the mutants. In heterozygous mice, the same LD50 of 600 µg was observed (Fig. 3B). As can be judged from the general pattern, death occurred at time points between those of wild type and homozygotes, confirming the implication of TNF in the death kinetics. Thus, these lethality studies establish an essential role for TNF in the rapid deaths induced by LPS. The absence of long-term protection, however, indicates that in the absence of TNF, other mediators can induce lethality.

Fig. 3
figure 3

LPS-induced lethality in wild-type and LT-α/TNF−α mutant mice

Groups of 10 mice for each genotype (A, wild type; B, heterozygotes; C, homozygotes) were injected intravenously with the indicated amount of S. typhimurium LPS (O, 200 µg; , 400 µg; , 600 µg; ▲, 800 µg; □, 1000 µg; ■, 1200 µg; ♦, 1600 µg). Lethality was monitored daily for 7 days and the percentage of surviving animals is presented. No death occurred after day 6.

Protection from LPS Lethality by CNI 1493

Although the cytokine cascade induced by LPS in homozygous mutants differs in several respects from that of wild-type mice, our data clearly indicate the significant induction of major proand anti-inflammatory cytokines. Thus it is reasonable to assume that the lethality induced by LPS in homozygous mutants is mediated by this cytokine cascade. To test this proposition, we investigated the ability of CNI 1493, an inhibitor of translation of several cytokines (44), including IL-1, IL-6, MIP-1, as well as TNF, to protect the mutant mice from LPS-induced lethality. Groups of 11 wild-type or homozygous mutants were first injected with CNI 1493 (0, 2, or 10 mg/kg) and 1 hr 30 min later with 600 µg of LPS. As previously reported, a dose of 10 mg/kg of CNI 1493 is required to significantly reduce lethality in wild-type mice (Fig. 4A). In mutant mice, doses of 2 and 10 mg/kg conferred an even better protection (Fig. 4B). Thus an inhibitor of cytokine production can more easily protect mutant mice against LPS-induced lethality, confirming both the implication of the other cytokines in their death and the importance of TNF in the death of wild-type mice.

Fig. 4
figure 4

Protection by CNI 1493 from LPS-induced lethality

Groups of 11 wild-type (A) and LT-α/TNF-α−/− mutants (B) were injected intraperitoneally with the indicated dose of CNI-1493 (▲, 0 mg/kg; ■, 2 mg/kg; , 10 mg/kg) 1 hr 30 min before an intravenous injection of 600 µg of S. typhimurium LPS. Lethality was monitored daily for 7 days, no death occurring after day 5.

Discussion

TNF and LPS-Induced Cytokine Cascade

The use of mice carrying a deletion of both the LT-α and TNF-α genes provides the opportunity to investigate the role of TNFs in the cascade of cytokines induced by LPS without the limitations of using inhibitors in vivo. We observed delayed kinetics for both plasma IL-1α and IL-1β, a longer persistence of circulating IL-10, a very significant reduction in detectable IFN-γ, and a decrease in late levels of circulating IL-6. Altogether, these data clearly illustrate that TNF-α is involved in the cytokine network and in the regulatory loops induced in vivo by LPS. However, the major contribution of TNF for the appearance of a given cytokine was only demonstrated for the late levels of plasma IL-6 and IFN-γ, whereas early induction of IL-6 and IL-10 were TNF-independent in this model. Thus, our observations do not support a major role for TNF in the early expression of IL-1β, IL-6, and IL-10. Previous studies with anti-TNF reagents have yielded divergent results. Some studies (1623) showed an overall reduction of detectable IL-1β, IL-6, and IL-8, but not IL-1α (24), without significant modification of the kinetics. Other models did not lead to a reduction of the various detectable cytokines. For example, pentoxifylline, a phosphodiesterase inhibitor known to inhibit TNF production, reduced the levels of circulating TNF-α following injection of LPS but did not reduce those of circulating IL-1β (31). Similarly, an adenosine kinase inhibitor that lowered the plasma levels of TNF-α in LPS-injected mice did not significantly modify the levels of circulating IL-1α and IL-6 (45). Finally, the effects of anti-TNF treatment on the levels of circulating IL-10 (9,25) and IFN-γ seemed to depend on the nature of the stimuli used (i.e., LPS, whole gram-negative bacteria, Streptococci, anti-CD 3) (24,26). In a recent report on TNF-α−/− mice, Marino et al. (46) did not find any significant difference between mutant and wild-type animals in terms of IL-1β, IL-6, IL-10, and IFN-γ production. The differences between this study and ours may reflect either the exact nature of the deletion (TNF-α versus LT-α/TNF-α) or the route of LPS administration. Furthermore, the use of a larger number of mice in our study is instrumental in establishing the significance of our observations. In summary, our results conclusively establish that the presence of the TNFs is not required for the induction of IL-1, IL-6, and IL-10 by LPS. Nevertheless, TNFs do affect the cytokine cascade, mostly by modulating the later phase of expression.

The decrease in IL-6 levels after the initial peak, the late peaks of IL-1α and IL-1β, and the persistence of circulating IL-10 in knock-out animals as compared with wild-type mice suggest that regulatory loops involving TNFs may affect cytokine production. If the expression of a cytokine is in part controlled by a positive regulatory loop associated with TNFs, its expression in LT-α/TNF-α-deficient mice should be reduced during the later phase of induction. Thus the low levels of IL-6 at 3 hr and 6 hr 30 min may be directly due to the absence of TNF, which is known to induce the production of IL-6, as well as to the reduced levels of TNF-inducible regulatory mediators known to enhance IL-6 production such as prostaglandin E2 (47), thrombin (48), platelet-activating factor (PAF) (49), leukotriene B4 (50), and IL-2 (51). Conversely, if there is a negative regulatory loop associated with TNF, the expression in mutant mice of the corresponding cytokine should be increased during the later phase. This could be case for the enhancement of IL-1 observed at 3 hr or for the maintenance of high levels of IL-10. This could involve heat shock proteins known to limit IL-1 production (52) and corticoids that inhibit IL-1 (53) as well as IL-10 production (54). The enhancement of IL-10 at 3 hr in deficient mice injected with LPS is in agreement with the observations obtained with anti-TNF treatment (55), but this was not observed in TNFα−/− mice (46). Thus the network of regulations taking place during the cytokine cascade provide multiple mechanisms for the control of the kinetics of cytokine expression.

As illustrated above, the injection of LPS initiates a complex pattern of mediators and regulatory loops involving many cell types and tissues. As macrophages are a major source of cytokines upon activation by LPS, it was of interest to investigate the contribution of TNF-α in the production of IL-1α, IL-1β, and IL-6 in this cell population. The in vitro analysis of peritoneal macrophage-derived cytokines demonstrates a clear involvement of TNF-α in the production of these cytokines. A lower production of IL-1α, IL-1β, and IL-6 was observed in both early (4 hr) or late (18 hr) cultures from deficient mice. These results are in agreement with the fact that TNF-α can trigger the production of these cytokines by macrophages (1,2,4,5,8). Thus, whereas LPS alone can induce the release of IL-1α, IL-1β, and IL-6 by macrophages, TNF amplifies it. The intermediate levels obtained with heterozygous animals producing low amounts of TNF further illustrate the dose-dependent effect of TNF on IL-1α, IL-1β, and IL-6 production. In cell cultures, as in the in vivo studies, the use of TNF inhibitors has provided divergent results. Inhibition of TNF by specific antibodies failed to modify the levels of released IL-1α, IL-1β, and IL-6 in human whole-blood assays upon activation by LPS (28,56), whereas IL-8 release was shown to be biphasic. LPS at optimal concentrations induces a first wave of IL-8 and TNF-α, whereas a second wave of IL-8 appears following TNF-α acting within a positive feedback loop. Pentoxifylline, a phosphodiesterase inhibitor known to inhibit TNF production, did not affect the release of IL-6 by LPS- or Staphylococcus-activated human peripheral blood mononuclear cells (29,30). Finally, metalloproteinase inhibitors that prevent TNF processing did modify the release of IL-1β, IL-6, or IL-8 (57,58). Further studies are required to fully analyze the reasons for the low capacity of peritoneal macrophages from LT-α/TNF-α-deficient mice to produce IL-1 and IL-6. It might reflect different macrophage subpopulations, as deficient mice possess profiles of leukocytes different from those of wild-type animals (32), or different levels of receptor expression, such as CD14. However, heterozygous mice exhibit only very subtle differences from wild-type animals in their cell populations; nonetheless, a reduction of cytokine expression was observed in heterozygous macrophage cultures. Altogether, these in vitro patterns, which are quite different from those obtained in vivo, confirm that the in vivo situation following LPS injection is not a simple reflection of macrophage activation.

TNF and LPS-Induced Lethality

For more than a decade, TNF has been considered a major actor in LPS-induced lethality and tissue injury. These conclusions were reached following experiments in animals models in which injection of anti-TNF (17,19,3437) or soluble TNF receptors (21,5961) protected the animal against endotoxin-induced shock or gram-negative bacteriemia. Furthermore, injection of TNF was able to mimic most of the deleterious effect associated with LPS challenge or gram-negative bacteria infection. However, the use of galactosamine in some animal models, the differences in protection as a function of the nature of the infectious agent (18,62,63), and the fact that blocking TNF in human sepsis did not lead to significant improvement (41,42) has dampened the optimistic concept that blocking TNF would be sufficient for preventing death and organ injury in sepsis patients.

Although we and others have previously shown that LT-α/TNF-α-deficient animals are highly resistant to LPS-induced shock in the galactosamine model (33,64), we now report that in the absence of galactosamine treatment, the LD50 of LPS is similar in wild-type and LT-α/TNF-α-deficient mice (i.e., 600 µg). It is worth noting that histological analysis of the tissues (lung, adrenal, kidney, liver, gut) sampled shortly before death did not reveal any significant differences between wild-type and knockout animals (data not shown). While these observations definitively establish that LPS-induced shock can be lethal in the absence of TNF, we have also obtained indications that TNF contributes to the severity of LPS-induced shock. Specifically, the marked difference in the death kinetics of wild-type and homozygous mutant mice indicates that TNF is required for the rapid death observed in LPS-treated wild-type mice. In addition, although the LD50 is identical in both groups, 1 mutant mice out of 10 survived at the two highest doses (1200 and 1600 µg), suggesting that 100% lethality could be difficult to achieve in the absence of TNF. Similarly, Marino et al. (46) showed that TNFα−/− mice appeared somewhat more resistant to the lethal effect of LPS. We have previously reported (33) that, because of the regulation of TNF production, the phenotype of heterozygous mice challenged with high doses of LPS could be more similar to that of homozygous mutants than to that of wild-type mice. This is again apparent in the present lethality study, as both the delayed death kinetics and the survival of a few mice at the highest doses were observed with heterozygous mice.

In mutant mice, the LPS-induced cytokine cascade still has an important proinflammatory component. Because IL-1, leukemia inhibitory factor (LIF), macrophage migration inhibitory factor (MIF), and IFN-γ have overlapping activities with TNF, they could contribute to the deleterious effects following LPS injection. Indeed, in wild-type mice, blocking IL-1 with IL-1 receptor antagonist (65) or passive immunization against LIF (66), MIF (67), and IFN-γ (10) protects against LPS-induced mortality. Furthermore, in addition to cytokines, the blocking of several other mediators (including PAF, tissue factor, anaphylatoxins, nitric oxide, prostaglandins, or leukotriene) has also been shown to be beneficial in animal models, indicating the multiplicity of potential mediators of LPS-induced lethality. CNI 1493 is a translation inhibitor of several proinflammatory cytokines, including TNF-α and IL-1 (44, 68), which can protect mice from the lethality induced by LPS alone (69). The fact that LT-α/TNF-α-deficient mice could be protected with lower doses of CNI-1493 than those required by wild-type mice and that death in deficient mice occurs later than in wild-type mice confirms that although other mediators can substitute for TNFs, LPS-induced shock is less severe in mutant than in wild-type mice.

Our data are in agreement with previous reports on TNF receptor (TNFR) knock-out mice. In the absence of galactosamine, LPS-induced lethality in p55 TNFR and p75 TNFR-deficient mice was similar to that in wild-type animals, whereas in the galactosamine model, p55 TNFR−/− but not p75 TNFR−/− mice were protected (40,70). This illustrates the importance of the model and suggests that the claim that p55 TNFR−/− mice are resistant to endotoxin shock might be misleading (71). Nevertheless, transgenic mice for a fusion protein (p55 TNFR-Fcγ3) have been reported to be resistant to endotoxininduced shock in the absence of galactosamine (72). However, in this study, comparisons between wild-type and transgenic mice were only performed at a unique dose of LPS (LD50 = 250 µg in this model).

LT-α/TNF-α-deficient animals are important tools for analyzing the contribution of TNF that circumvent the difficulties inherent to the use of pharmaceutical reagents. However, the specific features of these knock-out mice should be kept in mind when trying to transpose the results to normal situations, as the absence of secondary lymphoid organs (32,64) and hyperleukocytosis create a biological context that differs from normal ones. Nevertheless, the observation that LPS can induce lethal shock in these deficient animals further suggests that caution should be taken when considering TNF as a unique target for future sepsis therapy.

References

  1. Smith DM, Lackides GA, Epstein LB. (1990) Coordinated induction of autocrine tumor necrosis factor and interleukin 1 in normal human monocytes and the implications for monocyte-mediated cytotoxicity. Cancer Res. 50: 3146–3153.

    PubMed  CAS  Google Scholar 

  2. Van Damme J, Opdenakker G, Simpson RJ, et al. (1987) Identification of the human 26-kD protein, interferon beta 2 (IFN-beta 2), as a B cell hybridoma/plasmacytoma growth factor induced by interleukin 1 and tumor necrosis factor. J. Exp. Med. 165: 914–919.

    Article  PubMed  Google Scholar 

  3. Cassatella MA, Guasparri I, Ceska M, Bazzoni F, Rossi F. (1993) Interferon-gamma inhibits interleukin-8 production by human polymorphonuclear leucocytes. Immunology 78: 177–184.

    PubMed  PubMed Central  CAS  Google Scholar 

  4. Wanidworanum C, Strober W. (1993) Predominant role of tumor necrosis factor-α in human monocyte IL-10 synthesis. J. Immunol. 151: 6853–6861.

    Google Scholar 

  5. Yamada H, Iwase S, Mohri M, Kufe D. (1991) Involvement of a nuclear factor-κB-like protein in induction of the macrophage colony stimulating factor gene by tumor necrosis factor. Blood 178: 1988–1995.

    Google Scholar 

  6. Debs RJ, Fuchs HJ, Philip R, et al. (1988) Lung-specific delivery of cytokines induces sustained pulmonary and systemic immunomodulation in rats. J. Immunol. 140: 3482–3488.

    PubMed  CAS  Google Scholar 

  7. Brouckaert P, Spriggs DR, Demetri G, Kufe DW, Fiers W. (1989) Circulating interleukin 6 during a continuous infusion of tumor necrosis factor and interferon gamma. J. Exp. Med. 169: 2257–2262.

    Article  CAS  PubMed  Google Scholar 

  8. Sheron N, Williams R. (1992) IL-8 as a circulating cytokine: Induction by recombinant tumour necrosis factor-alpha. Clin. Exp. Immunol. 89: 100–103.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. van der Poll T, Jansen J, Levi M, ten Cate H, ten Cate JW, van Deventer SJ. (1994) Regulation of interleukin 10 release by tumor necrosis factor in humans and chimpanzees. J. Exp. Med. 180: 1985–1988.

    Article  PubMed  Google Scholar 

  10. Doherty GM, Lange JR, Langstein HN, Alexander HR, Buresh CM, Norton JA. (1992) Evidence for IFN-gamma as a mediator of the lethality of endotoxin and tumor necrosis factor-alpha. J. Immunol. 149: 1666–1670.

    PubMed  CAS  Google Scholar 

  11. Michie H, Manogue K, Spriggs D, et al. (1988) Detection of circulating tumor necrosis factor after endotoxin administration. N. Engl. J. Med. 318: 1481–1486.

    Article  CAS  PubMed  Google Scholar 

  12. Cannon J, Tompkins R, Gelfand J. (1990) Circulating interleukin 1 and tumor necrosis factor in septic shock and experimental endotoxin fever. J. Infect. Dis. 161: 79–84.

    Article  CAS  PubMed  Google Scholar 

  13. Redl H, Schlag G, Bahrami S, Schade U, Ceska M, Stutz P. (1991) Plasma neutrophilactivating peptide-1/interleukin-8 and neutrophil elastase in a primate bacteremia model. J. Infect. Dis. 164: 383–388.

    Article  CAS  PubMed  Google Scholar 

  14. Martich GD, Danner RL, Ceska M, Suffredini AF. (1991) Detection of interleukin 8 and tumor necrosis factor in normal humans after intravenous endotoxin: The effect of antiinflammatory agents. J. Exp. Med. 173: 1021–1024.

    Article  CAS  PubMed  Google Scholar 

  15. Wollenberg GK, DeForge LE, Bolgos G, Remick DG. (1993) Differential expression of tumor necrosis factor and interleukin-6 by peritoneal macrophages in vivo and in culture. Am. J. Pathol. 143: 1121–1130.

    PubMed  PubMed Central  CAS  Google Scholar 

  16. Fong Y, Tracey KJ, Moldawer LL, et al. (1989) Antibodies to cachectin/tumor necrosis factor reduce interleukin 1 beta and interleukin 6 appearance during lethal bacteremia. J. Exp. Med. 170: 1627–1633.

    Article  CAS  PubMed  Google Scholar 

  17. Emerson T, Jr, Lindsey DC, Jesmok GJ, Duerr ML, Fournel MA. (1992) Efficacy of monoclonal antibody against tumor necrosis factor alpha in an endotoxemic baboon model. Circ. Shock 38: 75–84.

    PubMed  CAS  Google Scholar 

  18. Teti G, Mancuso G, Tomasello F, Chiofalo MS. (1992) Production of tumor necrosis factor-alpha and interleukin-6 in mice infected with group B streptococci. Circ. Shock 38: 138–144.

    PubMed  CAS  Google Scholar 

  19. Hinshaw LB, Emerson T, Jr, Taylor F Jr, et al. (1992) Lethal Staphylococcus aureus-induced shock in primates: Prevention of death with anti-TNF antibody. J. Trauma 33: 568–573.

    Article  CAS  PubMed  Google Scholar 

  20. van der Poll T, Levi M, van Deventer SJ, et al. (1994) Differential effects of anti-tumor necrosis factor monoclonal antibodies on systemic inflammatory responses in experimental endotoxemia in chimpanzees. Blood 83: 446–451.

    PubMed  Google Scholar 

  21. Van Zee KJ, Kohno T, Fischer E, Rock CS, Moldawer LL, Lowry SF. (1992) Tumor necrosis factor soluble receptors circulate during experimental and clinical inflammation and can protect against excessive tumor necrosis factor alpha in vitro and in vivo. Proc Natl Acad Sci U.S.A. 89: 4845–4849.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Suffredini AF, Reda D, Banks SM, Tropea M, Agosti JM, Miller R. (1995) Effects of recombinant dimeric TNF receptor on human inflammatory responses following intravenous endotoxin administration. J. Immunol. 155: 5038–5045.

    PubMed  CAS  Google Scholar 

  23. Espat NJ, Cendan JC, Beierler EA, et al. (1995) PEG-BP-30 monotherapy attenuates the cytokine-mediated inflammatory cascade in baboon Escherichia coli septic shock. J. Surg. Res. 59: 153–158.

    Article  CAS  PubMed  Google Scholar 

  24. Teti G, Mancuso G, Tomasello F. (1993) Cytokine appearance and effects of anti-tumor necrosis factor alpha antibodies in a neonatal rat model of group B streptococcal infection. Infect. Immun. 61: 227–235.

    PubMed  PubMed Central  CAS  Google Scholar 

  25. Junger WG, Hoyt DB, Redl H, et al. (1995) Tumor necrosis factor antibody treatment of septic baboons reduces the production of sustained T-cell suppressive factors. Shock 3: 173–178.

    Article  CAS  PubMed  Google Scholar 

  26. Ferran C, Dautry F, Merite S, et al. (1994) Anti-tumor necrosis factor modulates anti-CD3-triggered T cell cytokine gene expression in vivo. J. Clin. Invest. 93: 2189–2196.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Cavaillon JM, Marie C, Pitton C, Fitting C. (1995) The production of TNF alpha and IL-8 in whole blood assays are differently regulated upon activation by LPS. Prog. Clin. Biol. Res. 392: 433–439.

    PubMed  CAS  Google Scholar 

  28. Vasilescu C, Berger D, Buttenschön K, Seidelmann M, Beger HG. (1996) Endotoxin-induced release of interleukin-6 and interleukin-1β in human blood is independent of tumor necrosis factor-α. Eur. Surg. Res. 28: 55–62.

    Article  CAS  PubMed  Google Scholar 

  29. Schandené L, Vandenbussche P, Crusiaux A, et al. (1992) Differential effects of pentoxifylline on the production of tumor necrosis factor alpha and interleukin-6 by monocytes and T cells. Immunology 76: 30–34.

    PubMed  PubMed Central  Google Scholar 

  30. Mandi Y, Farkas G, Ocsovszky I, Nagy Z. (1995) Inhibition of tumor necrosis factor production and ICAM-1 expression by pentoxifylline: Beneficial effects in sepsis syndrome. Res. Exp. Med. 195: 297–307.

    Article  CAS  Google Scholar 

  31. Netea MG, Blok WL, Kullberg BJ, et al. (1995) Pharmocologic inhibitors of tumor necrosis factor production exert differential effects in lethal endotoxemia and in infection with live microorganisms in mice. J. Infect. Dis. 171: 393–399.

    Article  CAS  PubMed  Google Scholar 

  32. Amiot F, Bellkaid Y, Lebastard M, Ave P, Dautry F, Milon G. (1996) Abnormal organisation of the splenic marginal zone and the correlated leukocytosis in lymphotoxin-α and tumor necrosis factor-α double deficient mice. Eur. Cytokine Netw 7: 733–739.

    PubMed  CAS  Google Scholar 

  33. Amiot F, Boussadia O, Cases S, et al. (1997) Mice heterozygous for a deletion of the TNFα and LTα genes: Biological importance of a non linear response of TNFα dosage. Eur. J. Immunol. 27: 1035–1042.

    Article  CAS  PubMed  Google Scholar 

  34. Beutler B, Milsark IW, Cerami AC. (1985) Passive immunization against cahectin/tumor necrosis factor protects mice from lethal effect of endotoxin. Science 229: 869–871.

    Article  CAS  PubMed  Google Scholar 

  35. Tracey KJ, Fong Y, DG. H, et al. (1987) Anticachectin/TNF monoclonal antibodies prevent septic shock during lethal bacteraemia. Nature 330: 662–664.

    Article  CAS  PubMed  Google Scholar 

  36. Fiedler VB, Loof I, Sander E, Voehringer V, Galanos C, Fournel MA. (1992) Monoclonal antibody to tumor necrosis factor-alpha prevents lethal endotoxin sepsis in adult rhesus monkeys. J. Lab. Clin. Med. 120: 574–588.

    PubMed  CAS  Google Scholar 

  37. Jesmok G, Lindsey C, Duerr M, Fournel M, Emerson Jr, T. (1992) Efficacy of monoclonal antibody against human recombinant tumor necrosis factor in E. coli challenged swine. Am. J. Pathol. 141: 1197–1207.

    PubMed  PubMed Central  CAS  Google Scholar 

  38. Freudenberg MA, Galanos C. (1991) Tumor necrosis factor alpha mediates lethal activity of killed gram-negative and gram-positive bacteria in D-galactosamine-treated mice. Infect. Immun. 59: 2110–2115.

    PubMed  PubMed Central  CAS  Google Scholar 

  39. Franks AK, Kujawa KI, Yaffe LJ. (1991) Experimental elimination of tumor necrosis factor in low-dose endotoxin models has variable effects on survival. Infect. Immun. 59: 2609–2614.

    PubMed  PubMed Central  CAS  Google Scholar 

  40. Rothe J, Lesslauer W, Löscher H, et al. (1993) Mice lacking the tumour necrosis factor receptor 1 are resistant to TNF-mediated toxicity but highly susceptible to infection by Listeria monocytogenes. Nature 364: 798–802.

    Article  CAS  PubMed  Google Scholar 

  41. Abraham E, Wunderink R, Silverman H, et al. (1995) Efficacy and safety of monoclonal antibody to human tumor necrosis factor alpha in patients with sepsis syndrome. A randomized, controlled, double-blind, multicenter clinical trial. TNF-alpha MAb Sepsis Study Group. JAMA 273: 934–941.

    Article  CAS  PubMed  Google Scholar 

  42. Fisher CJ, Agosti JM, Opal SM, et al. (1996) Treatment of septic shock with the tumor necrosis factor receptor:Fc fusion protein. N. Engl. J. Med. 334: 1697–1702.

    Article  CAS  PubMed  Google Scholar 

  43. Sansonetti PJ, Arondel J, Cavaillon JM, Huerre M. (1995) Role of interleukin-1 in the pathogenesis of experimental shigellosis. J. Clin. Invest. 96: 884–892.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Bianchi M, Bloom O, Raabe T, et al. (1996) Suppression of pro-inflammatory cytokines in monocytes by a tetravalent guanylhydrazone. J. Exp. Med. 183: 927–936.

    Article  CAS  PubMed  Google Scholar 

  45. Firenstein GS, Boyle D, Bullough DA, et al. (1994) Protective effect of an adenosine kinase inhibitor in septic shock. J. Immunol. 152: 5853–5859.

    Google Scholar 

  46. Marino MW, Dunn A, Grail D, et al. (1997) Characterization of tumor necrosis factor-deficient mice. Proc. Natl. Acad. Sci. U.S.A. 94: 8093–8098.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Hinson R, Williams JA, Shacter E. (1996) Elevated interleukin-6 is induced by prostaglandin E2 in a murine model of inflammation: Possible role of cyclooxygenase-2. Proc. Natl. Acad. Sci. U.S.A. 93: 4885–4890.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Sower LE, Froelich CJ, Carney DH, Fenton II, JW, Klimpel GR. (1995) Thrombin induces IL-6 productin in fibroblasts and epithelial cells. J. Immunol. 155: 895–901.

    PubMed  CAS  Google Scholar 

  49. Thivierge M, Rola-Pleszczynsky M. (1992) Platelet activating factor enhances interleukin-6 production by alveolar macrophages. J. Allergy Clin. Immunol. 90: 796–802.

    Article  CAS  PubMed  Google Scholar 

  50. Brach MA, de Vos S, Arnold C, Gruβ H-J, Mertelsmann R, Herrmann F. (1992) Leukotriene B4 transcriptionally activates interleukin-6 expression involving NF-κB and NF-IL6. Eur. J. Immunol. 22: 2705–2711.

    Article  CAS  PubMed  Google Scholar 

  51. Musso T, Espinoza-Delgado I, Pulkki K, Gusella GL, Longo DL, Varesio L. (1992) IL-2 induces IL-6 production in human monocytes. J. Immunol. 148: 795–800.

    PubMed  CAS  Google Scholar 

  52. Schmidt JA, Abdulla E. (1988) Down-regulation of IL-1β biosynthesis by inducers of the heat shock response. J. Immunol. 141: 2027–2034.

    PubMed  CAS  Google Scholar 

  53. Snyder DS, Unanue ER. (1982) Corticosteroids inhibit murine macrophage Ia expression and interleukin-1 production. J. Immunol. 129: 1803–1805.

    PubMed  CAS  Google Scholar 

  54. Fushimi T, Okayama H, Seki T, Shimura S, Shirato K. (1997) Dexamethasone suppressed gene expression and production of interleukin-10 by human peripheral blood mononuclear cells and monocytes. Int. Arch. Allergy Immunol. 112: 13–18.

    Article  CAS  PubMed  Google Scholar 

  55. Barsig J, Küsters S, Vogt K, Volk HD, Tiegs G, Wendel A. (1995) Lipopolysaccharide-induced interleukin-10 in mice: Role of endogenous tumor necrosis factor-α. Eur. J. Immunol. 25: 2888–2893.

    Article  CAS  PubMed  Google Scholar 

  56. De Forge LE, Kenney JS, Jones ML, Warren JS, Remick DG. (1992) Biphasic production of IL-8 in lipopolysaccharide (LPS)-stimulated human whole blood. Separation of LPS- and cytokine-stimulated components using anti-tumor necrosis factor and anti-IL-1 antibodies. J. Immunol. 148: 2133–2141.

    Google Scholar 

  57. Gearing AJH, Beckett P, Christodoulou M, et al. (1994) Processing of tumour necrosis factor-α precursor by metalloproteinase. Nature 370: 555–557.

    Article  CAS  PubMed  Google Scholar 

  58. McGeehan GM, Becherer JD, Bast Jr, RC, et al. (1994) Regulation of tumour necrosis factor-α processing by a metalloproteinase inhibitor. Nature 370: 558–561.

    Article  CAS  PubMed  Google Scholar 

  59. Lesslauer W, Tabuchi H, Gentz R, et al. (1991) Recombinant soluble tumor necrosis factor receptor proteins protect mice from LPS-induced lethality. Eur. J. Immunol. 21: 2883–2886.

    Article  CAS  PubMed  Google Scholar 

  60. Ashkenazi A, Marsters S, Capon D, et al. (1991) Protection against endotoxic shock by a tumor necrosis factor receptor immunoadhesin. Proc. Natl. Acad. Sci. U.S.A. 88: 10535–10539.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Mohler KM, Torrance DS, Smith CA, et al. (1993) Soluble tumor necrosis factor (TNF) receptors are effective therapeutic agents in lethal endotoxemia and function simultaneously as both TNF carriers and TNF antagonists. J. Immunol. 151: 1548–1561.

    PubMed  CAS  Google Scholar 

  62. Silva A, Bayston K, Cohen J. (1990) Prophylactic and theurapeutic effects of a monoclonal antibody to tumor necrosis factor-α in experimental Gram negative shock. J. Infect. Dis. 161: 1148–1152.

    Article  Google Scholar 

  63. Martin RA, Silva AT, Cohen J. (1993) Effect of anti-TNF-alpha treatment in an antibiotic treated murine model of shock due to Streptococcus pyogenes. FEMS Microbiol. Lett. 110: 175–178.

    PubMed  CAS  Google Scholar 

  64. Eugster HP, Muller M, Karrer U, et al. (1996) Multiple immune abnormalities in tumor necrosis factor and lymphotoxin-alpha double-deficient mice. Int. Immunol. 8: 23–26.

    Article  CAS  PubMed  Google Scholar 

  65. Ohlsson K, Bjökk P, Bergenfield M, Hageman R, Thompson R. (1990) Interleukin-1 receptor antagonist reduces mortality from endotoxin shock. Nature 348: 550–552.

    Article  CAS  PubMed  Google Scholar 

  66. Block MI, Berg M, McNamara MJ, Norton JA, Fraker DL, Alexander HR. (1993) Passive immunization of mice against D factor blocks lethality and cytokine release during endotoxemia. J. Exp. Med. 178: 1085–1090.

    Article  CAS  PubMed  Google Scholar 

  67. Bernhagen J, Calandra T, Mitchell RA, et al. (1993) MIF is a pituitary-derived cytokine that potentiates lethal endotoxaemia. Nature 365: 756–759.

    Article  CAS  PubMed  Google Scholar 

  68. Cohen PS, Nakshatri H, Dennis J, et al. (1996) CNI-1493 inhibits monocyte/macrophage tumor necrosis factor by suppression of translation efficiency. Proc. Natl. Acad. Sci. U.S.A. 93: 3967–3971.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  69. Bianchi M, Ulrich P, Bloom O, et al. (1995) An inhibitor of macrophage arginine transport and nitric oxyde production (CNI-1493) prevents acute inflammation and endotoxin lethality. Mol. Med. 1: 254–266.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Erickson SL, de Sauvage F, Kikly K, et al. (1994) Decreased sensitivity to tumour-necrosis factor but normal T-cell development in TNF receptor-2-deficient mice. Nature 372: 560–563.

    Article  CAS  PubMed  Google Scholar 

  71. Pfeffer K, Matsuyama T, Kundig TM, et al. (1993) Mice deficient for the 55 kd tumor necrosis factor receptor are resistant to endotoxic shock, yet succumb to L. monocytogenes infection. Cell 73: 457–467.

    Article  CAS  PubMed  Google Scholar 

  72. Garcia I, Miyazaki Y, Araki K, et al. (1995) Transgenic mice expressing high levels of soluble TNF-R1 fusion protein are protected from lethal septic shock and cerebral malaria, and are highly sensitive to Listeria monocytogenes and Leishmania major infections. Eur. J. Immunol. 25: 2401–2407.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

We thank the staff of the animal facilities at the Institut Gustave Roussy and the Institut de Recherches sur le Cancer for their help throughout this project. This work was supported by the Association pour la Recherche sur le Cancer (grant 6307 to F.D.) and the Institut Pasteur (J.-M.C). F.A. was the recipient of a Ligue Nationale Contre le Cancer fellowship.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to François Dautry.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Amiot, F., Fitting, C., Tracey, K.J. et al. Lipopolysaccharide-Induced Cytokine Cascade and Lethality in LTα/TNFα-Deficient Mice. Mol Med 3, 864–875 (1997). https://doi.org/10.1007/BF03401722

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03401722

Keywords