Obesity and diabetes are increasingly common diseases that can lead to other complications such as fatty liver disease. Fatty liver disease affects one in five people and is caused by a built-up of fat in the liver, which can result in scarring of the liver tissue and other serious complications.

There is currently no cure for fatty liver disease. Drugs that have been effective in treating the condition in mice, lack efficacy in humans. To better understand why this is the case, Hunter, de Gracia Hahn, Duret, Im et al. conducted a review of over 5,000 published studies, analysing over 600 experiments.

Hunter et al. asked which drugs improved fatty liver in mice the most and if they had the same effect in humans. They also tested whether the age of the mice affected the outcome of the experiments. The analyses revealed that the drugs that work best in mice are different to the ones that show some effect in humans.

In mice, many of the drugs reduced their weight or lowered their blood sugar levels, which also improved the fatty liver condition. Moreover, drugs appeared to be less effective the older the mice were. However, most of these drugs do not cause weight loss or lower blood sugar levels in humans, suggesting that factors other than the intended action of these drug could affect the outcome of a mouse study.

These findings will help shape future research into obesity, diabetes and fatty liver disease using mice. They highlight that results obtained from studies with mice so far do not predict if a drug will work in humans to treat fatty liver disease. Moreover, weight loss seems to be the most important factor linked to how efficiently a drug treats fatty liver disease.

Interventional studies in animals are an integral component of drug development. If a disease can be suitably modelled in an animal, then the therapeutic response to a treatment observed in animals should inform its potential efficacy in humans (Howells et al., 2014). However, there is a well-documented translational gap between preclinical studies and subsequent outcomes in humans (Hackam and Redelmeier, 2006; Landis et al., 2012; Perel et al., 2007). Multiple factors contribute to this, including bias within study design (Macleod et al., 2015), insufficiently powered preclinical studies (Macleod et al., 2005), and biological differences between species (Mestas and Hughes, 2004; Rangarajan and Weinberg, 2003).

Systematic analyses of preclinical studies have found that publication bias may account for at least a third of the estimate of efficacy in trials (Henderson et al., 2015; Sena et al., 2010; van der Worp et al., 2010). In addition, other variables of animal model design can influence the magnitude of the treatment response (Watzlawick et al., 2019) and reporting of model design is often incomplete (Flórez-Vargas et al., 2016). These findings are highly relevant in the context of the ‘reproducibility crisis’ (Baker, 2016; von Herrath et al., 2019) as well as having ethical implications for the use of animals in research that is not of optimum quality (Prescott and Lidster, 2017).

Non-alcoholic fatty liver disease (NAFLD) is a highly active field of animal research (Brenner, 2018; Farrell et al., 2019). NAFLD is a common condition characterised by increased liver fat (hepatic steatosis) that may progress to inflammation in the form of non-alcoholic steatohepatitis (NASH) and fibrosis (Sanyal, 2019). Cirrhosis, end-stage liver disease, and hepatocellular carcinoma develop in a small proportion of patients. However, due to the high prevalence of obesity, NAFLD is the second most common indication for liver transplant in the United States (Younossi et al., 2018), predicted to overtake hepatitis C virus. NAFLD is intricately related with insulin resistance and therefore usually coexists with other features of the metabolic syndrome, such as type 2 diabetes and its recognised complications including cerebrovascular disease, coronary artery disease, and chronic kidney disease (Byrne and Targher, 2015).

There are currently no approved pharmacological therapies for NAFLD (Chalasani et al., 2018). Several Phase three trials are ongoing (Ratziu et al., 2019), but many interventions that appeared to have substantial efficacy in preclinical models have failed to be replicated in humans (Budas et al., 2016; Harrison et al., 2018; STELLAR-3 and STELLAR-4 Investigators et al., 2020; Sanyal et al., 2014). These studies have used a wide range of preclinical NAFLD models, including genetically modified animals (e.g. leptin deficient ob/ob mice), hypercaloric diets (e.g. high-fat diet), and toxic insults (e.g. streptozocin injections), all of which may be used in varying combinations and with different parameters (Anstee and Goldin, 2006). It is not known if, or which of, these variables influence treatment response to therapeutic agents in preclinical models of NAFLD, and which models are better predictors of response in humans.

Therefore, we performed a meta-analysis of interventional rodent studies of NAFLD to describe which drug classes were associated with improvement in NAFLD and whether any study characteristics (or biases) were linked to the magnitude of effect.

We performed a systematic search to identify interventional studies in rodent models of NAFLD. Our searches yielded 8621 articles, which after screening gave 5458 articles for full-text review (Figure 1). Studies were included in the meta-analysis if they used a pharmacological class that had been used in Phase 2 or three trials for NAFLD in humans (Supplementary file 1) and reported at least one of: hepatic triglyceride content, NAFLD Activity Score (NAS, or any of its components), portal inflammation, or fibrosis stage. After adjustments made for shared controls, 414 studies were included in the meta-analysis, comprising 603 cohorts of rodents (10,364 animals). Studies were predominantly performed in male animals (527/578, 91%). The median age at the start of intervention was 9-weeks old (range 0.6–80 weeks) for a median duration of 6 weeks (range 1 day – 60 weeks).

Figure 1

Download asset Open asset

Figure 1—source data 1

Dataset used in this meta-analysis.

Details and raw data of all studies included in the meta-analysis. This data can be used with the R code found in Supplementary Methods to run all analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig1-data1-v2.xlsx

Download elife-56573-fig1-data1-v2.xlsx

Hepatic triglyceride content was the most widely reported measure: 474/603 (79%) cohorts. Steatosis grade was the most frequently reported histological measure (174/603 (29%) cohorts), compared to: NAS 144/603 (24%), lobular inflammation 143/603 (24%), ballooning 106/603 (18%), and fibrosis in 58/603 (9.6%) cohorts. Portal inflammation was only reported in 8 cohorts from three studies, therefore meta-analysis was not possible for this outcome.

Meta-analysis of hepatic triglyceride content

We used random-effects meta-analysis to estimate the mean difference (MD) in hepatic triglyceride (TG) content between intervention and control groups (Figure 2A). The overall mean difference in hepatic TG content was −29.9% (95% CI −33%, −27%) with considerable between-study heterogeneity (I² = 90% (95% CI 89%, 90%), P_Q <1×10⁻³⁰⁰). Exclusion of outliers minimally affected the overall estimate (−30.2% (95% CI −33%, −27%), Figure 2—source data 1).

Figure 2 with 2 supplements see all

Download asset Open asset

Figure 2—source data 1

Results of meta-analysis and meta-regression of hepatic triglyceride content in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of hepatic triglyceride with subgroup by drug class. Tab 2. Results from meta-analysis of hepatic triglyceride with subgroup by individual drug. Tab 3. Results from meta-analysis of hepatic triglyceride with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses. Tab 5. Results from model 1 (without drug) and model 2 (including drug used) multivariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig2-data1-v2.xlsx

Download elife-56573-fig2-data1-v2.xlsx

For comparison, a relative decline of liver fat by ≥30%, as measured by magnetic resonance imaging proton-density fat fraction (MRI-PDFF), has been determined as the reduction required to achieve histological response in humans with NAFLD (Jayakumar et al., 2019; Loomba et al., 2020; Stine et al., 2020).

We hypothesised that much of this heterogeneity would be due to the different drug class interventions, with some classes having a greater effect than others. On meta-analysis using drug class as a subgroup, 22/28 (79%) of drug classes demonstrated a significant reduction in hepatic TG (i.e. the upper limit of their 95% CI was negative). If we were to use ≥30% reduction as a benchmark for clinical significance (analogous to change in MRI-PDFF), only 3/28 (11%) of drug classes passed this cut-off: fibrates, omega-3 polyunsaturated fatty acids (mixtures), and DPP-4 inhibitors.

The 95% CI of 24/28 drug classes overlapped with the CI of the overall effect estimate. Two drug classes, thiazolidinediones and vitamin E, were found to have a smaller mean reduction in hepatic TG and two classes had a greater reduction: fibrates and mixtures of omega-3 polyunsaturated fatty acids (PUFA). However, ‘PUFA mixtures’ was a comparatively broad drug class, and many PUFA mixtures included eicosapentaenoic acid (EPA) or docosahexaenoic acid (DHA), which individually showed no significant reduction in hepatic TG. There remained substantial or considerable heterogeneity within drug class subgroups (P_Q <0.05 for 21/28 drug classes, Figure 2—source data 1).

In order to investigate whether this heterogeneity was due to variation between individual drugs within classes we repeated the meta-analysis with subgroup by individual drugs (Figure 2—figure supplement 1). There was sufficient data for meta-analysis of 28 individual drugs (from the original 28 drug classes). 22/28 (79%) individual drugs were found to have a significant reduction in hepatic TG. Vitamin E was associated with a smaller mean reduction in hepatic TG than the 95% CI of the overall estimate, whilst fenofibrate was the only drug with a greater mean difference than the overall estimate. There remained considerable heterogeneity within subgroups for 20/28 drugs (I² = 75–100%, P_Q <0.05).

We then performed univariable meta-regression to investigate which variables accounted for the heterogeneity in results (Figure 2—source data 1). Though individual drug used was the single variable that accounted for most heterogeneity (adj R² = 4.9%, p=0.02), the majority of variation in results was unaccounted. An association was also observed for weight difference (adj R² = 3.3%, p=6.4×10⁻⁴), where greater weight loss in the intervention group was associated with a greater reduction in hepatic TG. This association was stronger after removal of NAFLD models that induce weight loss (e.g. methionine-choline deficient diet (MCD), Figure 2B) and similar results were obtained for difference in fasting insulin levels (Figure 2C).

When these study characteristics were combined for multivariable meta-regression using an unbiased method, 10 variables were predicted to substantially contribute to the variation in hepatic TG difference (Table 1). In final model 1, weight difference was the only variable to be significantly associated with MD in hepatic TG (p=0.003). Including drug used in model two was able to account for all heterogeneity in results (Figure 2—source data 1) in a small subset of cohorts (k = 42), though neither of these models were significantly predictive of outcome following permutation tests (p-value*>0.05).

Table 1

	Meta-analysis with subgroup by drug class		Multi-variable meta-regression – model 1		Multi-variable meta-regression – model 2
Outcome	Drug classes associated with outcome	Differential efficacy	Top predictors	Final model	Top predictors	Final model
Hepatic TG	22/28 (79%): SCD1-i, PUFA-mix, Fibrates, Bifidobacterium sp., DPP4-i, Curcumin, EPA, Silymarin, TUDCA, Polyphenol, GLP1 agonist, ARB, FXR agonist, SGLT2-i, PPARα-δ agonist, Cholesterol Absorption Inhibitor, Berberine, Statin, Biguanide, Lactobacillus sp., Vitamin E	Greater reduction: Fibrates, PUFA-mix Smaller reduction: Thiazolidinediones, Vitamin E	Weight, Insulin, Fat (%kcal), Model, Age at start, Background, Glucose, Sex, Duration, Quality score (k = 333)	R2 = 48.9%, P-val*=0.22 K = 67	Insulin, Fat (%kcal), Weight, Glucose, Age at start, Sex, Drug (k = 222)	R2 = 100%, P-val*=0.26 K = 42
Steatosis	9/22, (41%): Fibrates, GLP-1 agonist, DPP4-i, Probiotic (mix), Curcumin, Thiazolidinediones, Lactobacillus sp., Statin, ARB	Greater reduction: Fibrates	Glucose, Fat (%kcal), Sex (k = 94)	R2 = 91.8%, P-val*=0.03 K = 19	Fat (%kcal), Sex, Weight (k = 62)	R2 = 60.3%, P-val*=0.098 K = 27
Lobular inflammation	9/16 (56%): Fibrates, Probiotic (mix), Statin, ARB, FXR agonist, DPP4-i, Biguanide, Thiazolidinediones, Vitamin D	-	Glucose, Fat (%kcal) (k = 81)	R2 = 49.8%, P-val*=0.43 K = 19	-	-
Ballooning	8/14 (57%): Fibrates, Biguanide, Thiazolidinediones, Vitamin D, DPP4-i, ARB, FXR agonist, Probiotic (mix)	Greater reduction: Fibrates Smaller reduction: Probiotic (mix)	Glucose (k = 56)	R2 = 8.1%, P-val*=0.38 K = 26	-	-
NAFLD Activity Score	10/14 (71%):Fibrates, DPP4-i, GLP1 agonist, Probiotic (mix), Vitamin D, Silymarin, Biguanide, Thiazolidinediones, FXR agonist, ARB	Greater reduction: Fibrates	Glucose, Fat (%kcal), Age at start, Weight (k = 89)	R2 = 78.0%, P-val*=0.03 K = 19	Fat (%kcal), Weight, Background, Age at start, Sex (k = 58)	R2 = 63.1%, P-val*=0.001 K = 30
Fibrosis	2/5 (40%): FXR agonist, Statin	-	Model, Weight, Glucose, Fat (%kcal), Duration, Age at start (k = 58)	R2 = 100%, P-val*=0.67 K = 16	-	-

Given that meta-regression implicated weight loss and improved insulin sensitivity in results, we explored how these traits were distributed by drug class (Figure 3A). Including all available data, we observed that 12/33 (36%) drug classes showed a significant reduction in weight (i.e. the upper limit of their 95% CI was below 1, Figure 3—source data 1). 17/32 (53%) and 15/25 (60%) of drug classes were associated with reductions in fasting glucose (Figure 3B) and insulin (Figure 3—figure supplement 1A), respectively. There was a positive correlation between weight, glucose, and insulin differences (Figure 3—figure supplement 1B). In addition, there was a negative correlation between weight difference and study duration or the age of mice at the end of intervention, that is longer studies (or those in older mice) were associated with greater weight loss in interventional groups.

Figure 3 with 1 supplement see all

Download asset Open asset

Figure 3—source data 1

Results of difference in weight, glucose, and insulin for each drug class.

Mean, standard deviation, and 95% confidence intervals for the percentage difference in weight, fasting glucose, and fasting insulin between interventional and placebo animals. ACC, acetyl-CoA carboxylase; ACE, angiotensin-2 converting enzyme; CB1, cannabinoid receptor 1; DPP4 Dipeptidyl peptidase-4; FXR, Farnesoid X receptor; GLP-1, glucagon-like peptide-1; LXR, liver X receptor; PDE, phosphodiesterase; PPAR, peroxisome proliferator-activated receptor; SCD1, stearoyl–CoA desaturase-1; SGLT2, sodium-glucose co-transporter-2; TUDCA, tauroursodeoxycholic acid; and UDCA, ursodeoxycholic acid.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig3-data1-v2.xlsx

Download elife-56573-fig3-data1-v2.xlsx

We then explored whether these results showed study distribution (publication) bias or were heavily influenced by individual outliers (Figure 2—figure supplement 2). There was an uneven distribution of studies with a bias towards a reduction in hepatic TG, which was supported by Egger’s test (β = -.83 [95% CI −1.3, −0.4], p=2.2×10⁻⁴). Using the trim-and-fill method to account for this bias, we estimated that the true overall mean difference in hepatic TG would be −18.7% (95% CI −21%, −16%), over a third smaller than the original estimate.

Meta-analysis of histological steatosis grade

Whilst hepatic TG was the most widely reported measure, histological assessment of disease is considered the gold standard for patients with NAFLD. Therefore, we performed a meta-analysis of MD in steatosis grade (Figure 4A). The overall MD in steatosis was −0.7 (95% CI −0.8, −0.5) again with considerable heterogeneity (I² = 94% (95% CI 93%, 95%), P_Q <1×10⁻³⁰⁰). Compared to hepatic TG, fewer drug classes were identified to be associated with a significant reduction in steatosis grade (8/22, 36%), though again fibrates showed the largest effect size. Similar results were obtained when performing subgrouping by individual drugs, rather than classes (Figure 4—source data 1).

Figure 4 with 1 supplement see all

Download asset Open asset

Figure 4—source data 1

Results of meta-analysis and meta-regression of steatosis grade in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of steatosis grade with subgroup by drug class. Tab 2. Results from meta-analysis of steatosis grade with subgroup by individual drug. Tab 3. Results from meta-analysis of steatosis grade with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses. Tab 5. Results from model 1 (without drug) and model 2 (including drug class used) multivariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig4-data1-v2.xlsx

Download elife-56573-fig4-data1-v2.xlsx

Univariable meta-regression found a marked association between difference in plasma glucose levels and MD in steatosis grade (Figure 4B, adj R²21%, p=2.4×10⁻⁶). Similar associations were observed for difference in weight and insulin levels, particularly after removal of weight-loss inducing models (Figure 4C). In addition, the sex of animals (adj R²7%, p=0.01) and genetic background were associated with MD in steatosis grade (Figure 4—source data 1). When factors were combined in multivariable meta-regression (Table 1), a model using sex, fasting glucose difference, and fat (%kcal) in diet accounted for 92% of variability in a small subset of cohorts (k = 19), which remained robust after a multiple permutation test (p-value*=0.03).

Meta-analysis of lobular inflammation

9/16 (56%) drug classes were associated with a reduction in MD of lobular inflammation (Figure 5A). Again there was considerable heterogeneity within drug classes and when subgrouping by individual drugs (Figure 5—source data 1).

Figure 5 with 1 supplement see all

Download asset Open asset

Figure 5—source data 1

Results of meta-analysis and meta-regression of lobular inflammation in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of lobular inflammation with subgroup by drug class. Tab 2. Results from meta-analysis of lobular inflammation with subgroup by individual drug. Tab 3. Results from meta-analysis of lobular inflammation with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses. Tab 5. Results from multivariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig5-data1-v2.xlsx

Download elife-56573-fig5-data1-v2.xlsx

Univariable meta-regression identified an association with difference in weight (Figure 5B, adj R²15%, p=4.0×10⁻⁴), as had been observed for steatosis grade and hepatic TG content. In addition, an association was found for fat %kcal in diet and MD in lobular inflammation: a higher %kcal fat in diet was associated with a smaller difference in lobular inflammation (Figure 5C, adj R²21%, p=1.7×10⁻⁵), indicating that study design was associated with size of treatment response. The bubble plot of fat content in diet also illustrated that the majority of studies reporting fat content in diet used either 40–45% or 60% kcal fat (Figure 5C).

Meta-analysis of hepatocellular ballooning

8/14 (57%) drug classes were associated with a reduction in hepatocellular ballooning (Figure 6A). Fibrates showed greater reduction in ballooning than other studied drug classes, however this could not be replicated at an individual drug level (Figure 6—figure supplement 1).

Figure 6 with 1 supplement see all

Download asset Open asset

Figure 6—source data 1

Results of meta-analysis and meta-regression of hepatocellular ballooning in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of hepatocellular ballooning with subgroup by drug class. Tab 2. Results from meta-analysis of hepatocellular ballooning with subgroup by individual drug. Tab 3. Results from meta-analysis of hepatocellular ballooning with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig6-data1-v2.xlsx

Download elife-56573-fig6-data1-v2.xlsx

Similar to previous analyses, difference in fasting glucose (adj R²17%, p=9.0×10⁻⁴) and weight (adj R²8%, p=0.01) were associated with the magnitude of treatment effect. Study design characteristics also influenced difference in ballooning, namely percentage of fat in diet (Figure 6B, greater reduction in ballooning where a lower %kcal was used) and percentage of fructose/glucose in diet (Figure 6C); however, there were only 12 studies contributing to this analysis. In addition, longer studies were associated with larger reductions in ballooning severity (Figure 6D).

Meta-analysis of NAFLD activity score (NAS)

The NAFLD activity score is a composite of steatosis, lobular inflammation, and ballooning scores. The results largely reflected those observed for the previous three meta-analyses (Figure 7A). 10/14 (71%) drug classes were associated with a significant reduction in NAS, with fibrates being the most beneficial drug class. Meta-regression found associations for difference in weight (Figure 7B) and glucose (Figure 7C) to account for 11% and 12% of heterogeneity in results, respectively.

Figure 7 with 1 supplement see all

Download asset Open asset

Figure 7—source data 1

Results of meta-analysis and meta-regression of NAFLD Activity Score (NAS) in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of NAS with subgroup by drug class. Tab 2. Results from meta-analysis of NAS with subgroup by individual drug. Tab 3. Results from meta-analysis of NAS with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses. Tab 5. Results from model 1 (without drug) and model 2 (including drug used) multivariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig7-data1-v2.xlsx

Download elife-56573-fig7-data1-v2.xlsx

multiple-variable meta-regression models were able to account for more than 60% of variation in results (in a small subset of cohorts) using genetic background, fat in diet, age at start of intervention, weight and glucose difference, but without requiring drug or drug class (Table 1).

Meta-analysis of fibrosis stage

Fibrosis stage is the histological feature that most strongly correlates with liver-related outcomes in humans with NAFLD (Angulo et al., 2015; Ekstedt et al., 2015), and was therefore pre-specified as the primary outcome measure for this study. However, it was reported in only 58/603 (9.6%) of cohorts. Only FXR agonists and statins (2/5, 40% drug classes) were associated with a significant reduction in fibrosis stage (Figure 8A), where the overall mean difference was −0.5 (95% CI −0.6, −0.3) stages. Meta-regression replicated previous findings for other traits, showing that difference in weight was associated with reduction in fibrosis stage (Figure 8B, adj R²27%, p=0.004).

Figure 8 with 1 supplement see all

Download asset Open asset

Figure 8—source data 1

Results of meta-analysis and meta-regression of fibrosis stage in rodent studies of NAFLD.

Tab 1. Results from meta-analysis of fibrosis stage with subgroup by drug class. Tab 2. Results from meta-analysis of fibrosis stage with subgroup by individual drug. Tab 3. Results from meta-analysis of fibrosis stage with subgroup by drug class, after removal of outlier studies. Tab 4. Results from univariable meta-regression analyses. Tab 5. Results from multivariable meta-regression analyses.

https://cdn.elifesciences.org/articles/56573/elife-56573-fig8-data1-v2.xlsx

Download elife-56573-fig8-data1-v2.xlsx

Bias analyses of histological outcomes and study quality

Funnel plots for steatosis grade, lobular inflammation, fibrosis stage, and NAS were asymmetric (Figure 9), supported by the results of Egger’s test for each analysis.

Figure 9 with 1 supplement see all

Download asset Open asset

Using the trim-and-fill method to account for these differences substantially altered the overall effect estimates: for steatosis grade, there was an 79% reduction in estimated effect size to −0.14 (95% −0.3, +.01); for lobular inflammation, a 70% reduction in effect size to −0.18 (95% −0.32, −0.05); for fibrosis, 72% reduction to −0.12 (95% −0.33, +.08); and NAS, 55% reduction in effect size to −0.82 (95% −0.1.1, −0.5).

We used a four-item scale to estimate study quality (Figure 9—figure supplement 1). We found that 497/603 (82%) cohorts were at high risk of bias due to either absence of randomisation or absence of blinding. In addition, we used post-hoc power calculations to estimate the proportion of studies that were adequately powered. For analysis of hepatic TG, 39% (185/474) cohorts had a power of 80% or greater on post-hoc calculation. However, using the results from this meta-analysis, to achieve a power of 80% with significance set as p=0.05, group size would need to be n = 16. 4.2% (20/474) cohorts included 16 or more animals and would have met sufficient power to detect associations, based on these data.

Similar results were obtained for histological steatosis grade: 70/174 (40%) reported results consistent with >80% power but only 27/174 (16%) had a group size large enough to be expected to reach 80% power.

Summary of findings across traits

The majority of drug classes (or individual drugs) were found to show a significant reduction in severity of NAFLD. Fibrates (for which most data were available for fenofibrate) demonstrated the greatest improvement in several outcome measures (Table 1).

Univariable meta-regression found that weight loss and lower fasting glucose were associated with a greater improvement in multiple outcomes (Figure 10). In addition, diet composition influenced the magnitude of treatment response for lobular inflammation, ballooning, and fibrosis.

Figure 10

Download asset Open asset

Heatmap illustrating the results of univariable meta-regression analyses using continuous variables. Beta-regression co-efficient was normalized within each outcome analysis (e.g. steatosis grade) to mean = 0, standard deviation = 1. Rows (variables used as predictors in meta-regression) and columns (outcome measures for NAFLD) are clustered for similarity.

Through meta-analysis and meta-regression we have illustrated that weight loss and alleviation of insulin resistance are consistently associated with treatment response in interventional trials for NAFLD in rodents. This extends beyond drugs that cause weight loss in humans. In addition, we have found that study design characteristics (e.g. diet composition) can influence the magnitude of treatment response. These findings suggest that factors other than the pharmacological mechanism of the trialled drug may confound the results observed in such studies.

All stages of NAFLD show a strong, positive correlation with severity of insulin resistance in humans and type 2 diabetes is a major risk factor for the presence of advanced fibrosis (Younossi et al., 2019). Consistent with this, weight loss and improvement in insulin sensitivity are associated with histological improvement in NAFLD (Koutoukidis et al., 2019), particularly evident from studies of bariatric surgery (Lassailly et al., 2015; Lee et al., 2019) and liraglutide (Armstrong et al., 2015). Therefore, it is not a surprising observation to see this replicated in our meta-regression analyses and it is consistent with previous observations (Hui et al., 2015). On multiple-variable inference, weight loss or fasting glucose were the most important variables across several outcome metrics. This provides strong evidence that (in rodents) alleviation of insulin resistance, usually mediated by weight loss, improves features of NAFLD, independent of the drug used.

Some drug classes that caused weight loss in rodents are also well established to cause weight loss in humans (e.g. GLP-1 agonists and metformin), whilst others are not (e.g. vitamin D and statins). The findings for insulin sensitivity were similar, with over 50% of drugs reducing fasting glucose. Again, some drugs were consistent with their effect in humans (e.g. thiazolidinediones, DPP4-inhibitors) but not others (e.g. ezetimibe). It is not clear whether this is due to reduced food intake or other toxic effects of the drugs. It should be noted that some individual studies faithfully recapitulated observations in humans, for example weight gain, adipose expansion, and improved insulin sensitivity with thiazolidinedione use. However across the dataset as a whole, these observations suggest that ‘off-pharmacological-target’ effects, causing changes in weight and glucose homeostasis, may account for some of the translational gap between agents efficacious in rodents but not humans.

Though there are no licensed therapies for NAFLD, drug development is a highly active field (Friedman et al., 2018) and there have been over 30 drugs used in Phase 2 or three trials. Some have demonstrated potential efficacy in well-conducted randomized controlled trials, most notably GLP-1 agonists (Armstrong et al., 2015) and pioglitazone (Cusi et al., 2016; Sanyal et al., 2010). However, the majority of early phase trials did not find substantial benefit from the trialled interventions (Supplementary file 1). Whereas in animals, a large number of drugs (and classes) demonstrated significant efficacy across several outcome measures. This did not appear to be consistent with the results from human trials, for example we observed that vitamin D was associated with a significant reduction in NAS, however several trials have not found any benefit from its use in humans (Barchetta et al., 2016; Dabbaghmanesh et al., 2018). In addition, the magnitude of effect observed in rodents was not consistent with human data. For example, there is reasonably convincing evidence that pioglitazone improves NAFLD in humans, however it had one of the smallest improvements in hepatic TG. Similarly, GLP-1 agonists, which met their primary outcome in a human Phase two study (Armstrong et al., 2015), rank in the middle for most outcomes in this analysis. Fibrates had one of the largest treatment effects across multiple analyses but this does not appear to be consistent with human evidence to date (Fabbrini et al., 2010; Oscarsson et al., 2018). Fibrate use was also associated with a median 10% wt loss in these analyses, which has not been observed in large randomised trials in humans (Keech et al., 2005). Even though we found evidence for efficacy of the majority of drugs included in this analysis, the 95% CI for treatment effect size overlapped for most drug classes. This is generally consistent with findings reported in preclinical models of spinal cord injury where the effect size of several different types of treatment overlapped (Watzlawick et al., 2019). Overall, the trends observed are not consistent with findings in humans and there does not appear to be any clear patterns that indicate potentially successful translation.

Several study design characteristics affected treatment response across multiple outcome measures, including the age of animals, sex, genetic background, and dietary composition. There are a huge number of variables in the design of an interventional animal study and many were simplified for the input into analyses. For example, the ‘model’ used was simplified to a ‘core’ model (e.g. leptin deficient (ob/ob) mice) and separated from the genetic background of the animals for this analysis. Similarly, we studied several dietary components in isolation, which could have led to the observation that a higher proportion of dietary fat (e.g. 60% kcal) was associated with a smaller treatment response. This may be because lower fat containing diets (e.g. 40% kcal) may be combined with added cholesterol or other components, such as fructose. However these data do illustrate the concept that multiple factors associated with model design influence not only animal phenotype but magnitude of treatment response. This was demonstrated using multiple-variable meta-regression models where in some analyses the majority of variation in results could be accounted for (in a small subset of cohorts) without including drug as a covariate, particularly for NAS and steatosis grade.

It should be noted that there have been more systematic analyses of genetic background on NAFLD (Chella Krishnan et al., 2018; Hui et al., 2015) as well as in other fields, including immunology (Martin et al., 2017) and behavioural neuroscience (Homanics et al., 1999; Liu and Gershenfeld, 2001). We were surprised to find that genetic background was a top variable in comparatively few of our multivariate models. Based on observations from the Hybrid Mouse Diversity Panel (Chella Krishnan et al., 2018; Hui et al., 2018; Hui et al., 2015), we anticipate that the true impact of genetic background may be greater than we could quantify, due to our inclusion of a narrow range of backgrounds that had been used in multiple studies and our exclusion of mixed genetic backgrounds from analysis.

The vast majority of included studies demonstrated an improvement in NAFLD, which could be partly accounted for by a trend towards reporting positive results that is publication bias. Using the trim-and-fill method, we estimated that study distribution bias (most likely publication bias in this case) may have substantially increased the reported magnitude of effect (e.g. overall reduction in hepatic TG of 19% compared to 30%). The presence of publication bias did not come as a surprise (Tsilidis et al., 2013) and this dataset provides useful replication of the strong evidence base for this in preclinical neurological studies. A previous work on preclinical models of sunitinib calculated the overestimate from potential publication bias at 45% (Henderson et al., 2015). The results from power calculations are also likely to reflect publication bias: based on the overall effect summary, a minority of cohorts were of sufficient size to be predicted to achieve the power of 80%. Similarly, we have replicated previously described low rates of randomisation and blinding in animal studies (Bahor et al., 2017).

We found very few studies to report portal inflammation severity. In humans, (peri-)portal inflammatory activity has been shown to correlate with severity of fibrosis in both adults and children with NAFLD (Brunt et al., 2009; Mann et al., 2016; Rakha et al., 2010). Therefore, this remains a relatively unexplored area worthy of investigation, as targeting portal inflammation may be beneficial in slowing disease progression.

There are several implications of these results. Firstly, it is not surprising that there are multiple reports of difficulty in reproducing preclinical studies in the field of metabolism (von Herrath et al., 2019) given that study design has a considerable effect on treatment response. Variations in what may appear to be small details (such as age at the start of study diet) influence results and therefore could silence subtle differences or generate false positives.

Secondly, these results also help to explain the difficulty in bridging the preclinical to human translational gap (Denayer et al., 2014), which might be relevant beyond the field of metabolism research. For example, we did not observe an association between drug dose and treatment effect size. In addition, studies were overwhelmingly performed in male animals, whereas human studies are more evenly balanced (e.g. 60% female in the ‘STELLAR-3/–4’ trial [STELLAR-3 and STELLAR-4 Investigators et al., 2020]). Sex was a top predictor of several multivariable inference models and therefore the lack of inclusion of female mice may hinder identification of drugs for translation. Similarly, studies were almost uniformly done on young mice who were growing, unlike the focus on adult patients in all major phase 3 NAFLD trials.

The main strength of this work is the number of included studies, interventions, and variables. This has facilitated a detailed analysis of a single disease area. However this study has simplified some study characteristics to facilitate meta-regression analyses, which may have under-estimated the impact of particular variables on outcome measures. One such simplification was grouping of drugs into classes, some of which (e.g. ‘Probiotics (mix)’) were comparatively vague, compared to those with well-defined mechanisms (e.g. thiazolidinediones). Similarly, we used a simplified categorisation of rodent models (e.g. high-fat diet), combined with individual continuous metrics (e.g. fat %kcal), which will not capture the full variation of models used. We used fasting glucose and insulin as proxies for insulin resistance, however these are not direct measures of insulin resistance. This would require results from hyperinsulinaemic-euglycaemic clamps, or at least insulin tolerance tests, but these were performed in comparatively few studies. Similarly, we elected to record histological outcomes only where it was reported according to standard criteria for reporting human biopsies of NAFLD. There are a wide variety of other methods of interpreting liver histology, some of which are more quantitative (e.g. collagen proportionate area), though again these were less frequently reported. It should also be noted that this study did not have a pre-specified statistical analysis plan, which increases its risk of bias.

There is a wide range of other variables that were not considered in this analysis. Some were unreported variables, such as technique of animal handling. A further factor of potential relevance is the bacterial status of rodents, which is known to affect liver phenotypes (Kaden-Volynets et al., 2019), potentially via intestinal dysbiosis (Balmer et al., 2014; Mazagova et al., 2015). Furthermore, many studies did not report certain variables, for example genetic background of animals was not reported in 5.3% (32/603), which reduced the number of studies included in meta-regression analyses. This was most obvious for multiple-variable meta-regression, where some final models included fewer that 20 data points. However this meta-analysis has included a large number of articles, which gives considerable confidence in the findings we have replicated across several outcome measures.

The raw dataset used for analysis, including references to individual studies, are available Figure 1-source data 1 and deposited in the Dryad repository at https://doi.org/10.5061/dryad.pzgmsbcgc. R code used for analysis are available in Source code 1. Source data files have been provided for Figures 2-8.

1. 1. Mann JP

   (2020) Dryad Digital Repository

   Data from: Weight loss, insulin resistance, and study design confound results in a meta-analysis of animal models of fatty liver.

   https://doi.org/10.5061/dryad.pzgmsbcgc

1. 1. Angulo P
   2. Kleiner DE
   3. Dam-Larsen S
   4. Adams LA
   5. Bjornsson ES
   6. Charatcharoenwitthaya P
   7. Mills PR
   8. Keach JC
   9. Lafferty HD
   10. Stahler A
   11. Haflidadottir S
   12. Bendtsen F

   (2015) Liver fibrosis, but no other histologic features, is associated with Long-term outcomes of patients with nonalcoholic fatty liver disease

   Gastroenterology 149:389–397.

   https://doi.org/10.1053/j.gastro.2015.04.043

   • PubMed
   • Google Scholar
2. 1. Anstee QM
   2. Goldin RD

   (2006) Mouse models in non-alcoholic fatty liver disease and steatohepatitis research

   International Journal of Experimental Pathology 87:1–16.

   https://doi.org/10.1111/j.0959-9673.2006.00465.x

   • PubMed
   • Google Scholar
3. 1. Armstrong MJ
   2. Gaunt P
   3. Aithal GP
   4. Barton D
   5. Hull D
   6. Parker R
   7. Hazlehurst JM
   8. Guo K
   9. Abouda G
   10. Aldersley MA
   11. Stocken D
   12. Gough SC
   13. Tomlinson JW
   14. Brown RM
   15. Hübscher SG
   16. Newsome PN

   (2015) Liraglutide safety and efficacy in patients with non-alcoholic steatohepatitis (LEAN): a multicentre, double-blind, randomised, placebo-controlled phase 2 study

   The Lancet 387:679–690.

   https://doi.org/10.1016/S0140-6736(15)00803-X

   • Google Scholar
4. 1. Bahor Z
   2. Liao J
   3. Macleod MR
   4. Bannach-Brown A
   5. McCann SK
   6. Wever KE
   7. Thomas J
   8. Ottavi T
   9. Howells DW
   10. Rice A
   11. Ananiadou S
   12. Sena E

   (2017) Risk of Bias reporting in the recent animal focal cerebral ischaemia literature

   Clinical Science 131:2525–2532.

   https://doi.org/10.1042/CS20160722

   • Google Scholar
5. 1. Baker M

   (2016) 1,500 scientists lift the lid on reproducibility

   Nature 533:452–454.

   https://doi.org/10.1038/533452a

   • PubMed
   • Google Scholar
6. 1. Balmer ML
   2. Slack E
   3. de Gottardi A
   4. Lawson MA
   5. Hapfelmeier S
   6. Miele L
   7. Grieco A
   8. Van Vlierberghe H
   9. Fahrner R
   10. Patuto N
   11. Bernsmeier C
   12. Ronchi F
   13. Wyss M
   14. Stroka D
   15. Dickgreber N
   16. Heim MH
   17. McCoy KD
   18. Macpherson AJ

   (2014) The liver may act as a firewall mediating mutualism between the host and its gut commensal Microbiota

   Science Translational Medicine 6:237ra66.

   https://doi.org/10.1126/scitranslmed.3008618

   • PubMed
   • Google Scholar
7. 1. Barchetta I
   2. Del Ben M
   3. Angelico F
   4. Di Martino M
   5. Fraioli A
   6. La Torre G
   7. Saulle R
   8. Perri L
   9. Morini S
   10. Tiberti C
   11. Bertoccini L
   12. Cimini FA
   13. Panimolle F
   14. Catalano C
   15. Baroni MG
   16. Cavallo MG

   (2016) No effects of oral vitamin D supplementation on non-alcoholic fatty liver disease in patients with type 2 diabetes: a randomized, double-blind, placebo-controlled trial

   BMC Medicine 14:92.

   https://doi.org/10.1186/s12916-016-0638-y

   • PubMed
   • Google Scholar
8. 1. Baujat B
   2. Mahé C
   3. Pignon JP
   4. Hill C

   (2002) A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials

   Statistics in Medicine 21:2641–2652.

   https://doi.org/10.1002/sim.1221

   • PubMed
   • Google Scholar
9. 1. Brenner DA

   (2018) Of Mice and Men and Nonalcoholic Steatohepatitis

   Hepatology 68:2059–2061.

   https://doi.org/10.1002/hep.30186

   • Google Scholar
10. 1. Brunt EM
    2. Kleiner DE
    3. Wilson LA
    4. Unalp A
    5. Behling CE
    6. Lavine JE
    7. Neuschwander-Tetri BA
    8. NASH Clinical Research NetworkA list of members of the Nonalcoholic Steatohepatitis Clinical Research Network can be found in the Appendix

    (2009) Portal chronic inflammation in nonalcoholic fatty liver disease (NAFLD): a histologic marker of advanced NAFLD-Clinicopathologic correlations from the nonalcoholic steatohepatitis clinical research network

    Hepatology 49:809–820.

    https://doi.org/10.1002/hep.22724

    • PubMed
    • Google Scholar
11. 1. Brunt EM
    2. Kleiner DE
    3. Wilson LA
    4. Belt P
    5. Neuschwander-Tetri BA
    6. NASH Clinical Research Network (CRN)

    (2011) Nonalcoholic fatty liver disease (NAFLD) activity score and the histopathologic diagnosis in NAFLD: distinct clinicopathologic meanings

    Hepatology 53:810–820.

    https://doi.org/10.1002/hep.24127

    • PubMed
    • Google Scholar
12. 1. Budas G
    2. Karnik S
    3. Jonnson T
    4. Shafizadeh T
    5. Watkins S
    6. Breckenridge D
    7. Tumas D

    (2016) Reduction of liver steatosis and fibrosis with an Ask1 inhibitor in a murine model of nash is accompanied by improvements in cholesterol, bile acid and lipid metabolism

    Journal of Hepatology 64:S170.

    https://doi.org/10.1016/S0168-8278(16)01686-X

    • Google Scholar
13. 1. Byrne CD
    2. Targher G

    (2015) NAFLD: a multisystem disease

    Journal of Hepatology 62:S47–S64.

    https://doi.org/10.1016/j.jhep.2014.12.012

    • PubMed
    • Google Scholar
14. 1. Chalasani N
    2. Younossi Z
    3. Lavine JE
    4. Charlton M
    5. Cusi K
    6. Rinella M
    7. Harrison SA
    8. Brunt EM
    9. Sanyal AJ

    (2018) The diagnosis and management of nonalcoholic fatty liver disease: practice guidance from the american association for the study of liver diseases

    Hepatology 67:328–357.

    https://doi.org/10.1002/hep.29367

    • PubMed
    • Google Scholar
15. Software

    1. Champely S

    (2018) pwr: Basic functions for power analysis, version 1.2-2

    R Package.

    https://cran.r-project.org/web/packages/pwr/pwr.pdf
16. 1. Chella Krishnan K
    2. Kurt Z
    3. Barrere-Cain R
    4. Sabir S
    5. Das A
    6. Floyd R
    7. Vergnes L
    8. Zhao Y
    9. Che N
    10. Charugundla S
    11. Qi H
    12. Zhou Z
    13. Meng Y
    14. Pan C
    15. Seldin MM
    16. Norheim F
    17. Hui S
    18. Reue K
    19. Lusis AJ
    20. Yang X

    (2018) Integration of Multi-omics data from mouse diversity panel highlights mitochondrial dysfunction in Non-alcoholic fatty liver disease

    Cell Systems 6:103–115.

    https://doi.org/10.1016/j.cels.2017.12.006

    • PubMed
    • Google Scholar
17. Book

    1. Cohen J

    (1988) Statistical Power Analysis for the Behavioral Sciences

    Routledge.

    https://doi.org/10.1016/C2013-0-10517-X

    • Google Scholar
18. 1. Cusi K
    2. Orsak B
    3. Bril F
    4. Lomonaco R
    5. Hecht J
    6. Ortiz-Lopez C
    7. Tio F
    8. Hardies J
    9. Darland C
    10. Musi N
    11. Webb A
    12. Portillo-Sanchez P

    (2016) Long-Term pioglitazone treatment for patients with nonalcoholic steatohepatitis and prediabetes or type 2 diabetes mellitus: a randomized trial

    Annals of Internal Medicine 165:305–315.

    https://doi.org/10.7326/M15-1774

    • PubMed
    • Google Scholar
19. 1. Dabbaghmanesh MH
    2. Danafar F
    3. Eshraghian A
    4. Omrani GR

    (2018) Vitamin D supplementation for the treatment of non-alcoholic fatty liver disease: a randomized double blind placebo controlled trial

    Diabetes & Metabolic Syndrome: Clinical Research & Reviews 12:513–517.

    https://doi.org/10.1016/j.dsx.2018.03.006

    • PubMed
    • Google Scholar
20. 1. de Vries RB
    2. Hooijmans CR
    3. Tillema A
    4. Leenaars M
    5. Ritskes-Hoitinga M

    (2014) Updated version of the embase search filter for animal studies

    Laboratory Animals 48:88.

    https://doi.org/10.1177/0023677213494374

    • PubMed
    • Google Scholar
21. 1. Denayer T
    2. Stöhr T
    3. Van Roy M

    (2014) Animal models in translational medicine: validation and prediction

    New Horizons in Translational Medicine 2:5–11.

    https://doi.org/10.1016/j.nhtm.2014.08.001

    • Google Scholar
22. 1. Duval S
    2. Tweedie R

    (2000) Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication Bias in meta-analysis

    Biometrics 56:455–463.

    https://doi.org/10.1111/j.0006-341X.2000.00455.x

    • PubMed
    • Google Scholar
23. 1. Ekstedt M
    2. Hagström H
    3. Nasr P
    4. Fredrikson M
    5. Stål P
    6. Kechagias S
    7. Hultcrantz R

    (2015) Fibrosis stage is the strongest predictor for disease-specific mortality in NAFLD after up to 33 years of follow-up

    Hepatology 61:1547–1554.

    https://doi.org/10.1002/hep.27368

    • PubMed
    • Google Scholar
24. 1. Fabbrini E
    2. Mohammed BS
    3. Korenblat KM
    4. Magkos F
    5. McCrea J
    6. Patterson BW
    7. Klein S

    (2010) Effect of fenofibrate and niacin on intrahepatic triglyceride content, very low-density lipoprotein kinetics, and insulin action in obese subjects with nonalcoholic fatty liver disease

    The Journal of Clinical Endocrinology & Metabolism 95:2727–2735.

    https://doi.org/10.1210/jc.2009-2622

    • PubMed
    • Google Scholar
25. 1. Farrell G
    2. Schattenberg JM
    3. Leclercq I
    4. Yeh MM
    5. Goldin R
    6. Teoh N
    7. Schuppan D

    (2019) Mouse models of nonalcoholic steatohepatitis: toward optimization of their relevance to human nonalcoholic steatohepatitis

    Hepatology 69:2241–2257.

    https://doi.org/10.1002/hep.30333

    • PubMed
    • Google Scholar
26. 1. Flórez-Vargas O
    2. Brass A
    3. Karystianis G
    4. Bramhall M
    5. Stevens R
    6. Cruickshank S
    7. Nenadic G

    (2016) Bias in the reporting of sex and age in biomedical research on mouse models

    eLife 5:e13615.

    https://doi.org/10.7554/eLife.13615

    • PubMed
    • Google Scholar
27. 1. Friedman SL
    2. Neuschwander-Tetri BA
    3. Rinella M
    4. Sanyal AJ

    (2018) Mechanisms of NAFLD development and therapeutic strategies

    Nature Medicine 24:908–922.

    https://doi.org/10.1038/s41591-018-0104-9

    • PubMed
    • Google Scholar
28. 1. Hackam DG
    2. Redelmeier DA

    (2006) Translation of research evidence from animals to humans

    Jama 296:1727–1732.

    https://doi.org/10.1001/jama.296.14.1731

    • Google Scholar
29. Report

    1. Harrer M
    2. Cuijpers P
    3. Furukawa TA
    4. Ebert DD

    (2019)

    Doing Meta-Analysis in R: A Hands-on Guide

    PROTECT Lab Erlangen.

    • Google Scholar
30. 1. Harrison SA
    2. Abdelmalek MF
    3. Caldwell S
    4. Shiffman ML
    5. Diehl AM
    6. Ghalib R
    7. Lawitz EJ
    8. Rockey DC
    9. Schall RA
    10. Jia C
    11. McColgan BJ
    12. McHutchison JG
    13. Subramanian GM
    14. Myers RP
    15. Younossi Z
    16. Ratziu V
    17. Muir AJ
    18. Afdhal NH
    19. Goodman Z
    20. Bosch J
    21. Sanyal AJ
    22. Us G
    23. Investigators G-U-

    (2018) Simtuzumab Is Ineffective for Patients With Bridging Fibrosis or Compensated Cirrhosis Caused by Nonalcoholic Steatohepatitis

    Gastroenterology 155:1140–1153.

    https://doi.org/10.1053/j.gastro.2018.07.006

    • Google Scholar
31. 1. Henderson VC
    2. Demko N
    3. Hakala A
    4. MacKinnon N
    5. Federico CA
    6. Fergusson D
    7. Kimmelman J

    (2015) A meta-analysis of threats to valid clinical inference in preclinical research of sunitinib

    eLife 4:e08351.

    https://doi.org/10.7554/eLife.08351

    • Google Scholar
32. Book

    1. Higgins JPT
    2. Green S

    (2011)

    Cochrane Handbook for Systematic Reviews of Interventions

    The Cochrane Collaboration.

    • Google Scholar
33. 1. Homanics GE
    2. Quinlan JJ
    3. Firestone LL

    (1999) Pharmacologic and behavioral responses of inbred C57BL/6J and strain 129/SvJ mouse lines

    Pharmacology Biochemistry and Behavior 63:21–26.

    https://doi.org/10.1016/S0091-3057(98)00232-9

    • PubMed
    • Google Scholar
34. 1. Hooijmans CR
    2. Tillema A
    3. Leenaars M
    4. Ritskes-Hoitinga M

    (2010) Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed

    Laboratory Animals 44:170–175.

    https://doi.org/10.1258/la.2010.009117

    • Google Scholar
35. 1. Howells DW
    2. Sena ES
    3. Macleod MR

    (2014) Bringing rigour to translational medicine

    Nature Reviews Neurology 10:37–43.

    https://doi.org/10.1038/nrneurol.2013.232

    • PubMed
    • Google Scholar
36. 1. Hui ST
    2. Parks BW
    3. Org E
    4. Norheim F
    5. Che N
    6. Pan C
    7. Castellani LW
    8. Charugundla S
    9. Dirks DL
    10. Psychogios N
    11. Neuhaus I
    12. Gerszten RE
    13. Kirchgessner T
    14. Gargalovic PS
    15. Lusis AJ

    (2015) The genetic architecture of NAFLD among inbred strains of mice

    eLife 4:e05607.

    https://doi.org/10.7554/eLife.05607

    • PubMed
    • Google Scholar
37. 1. Hui ST
    2. Kurt Z
    3. Tuominen I
    4. Norheim F
    5. C.Davis R
    6. Pan C
    7. Dirks DL
    8. Magyar CE
    9. French SW
    10. Chella Krishnan K
    11. Sabir S
    12. Campos-Pérez F
    13. Méndez-Sánchez N
    14. Macías-Kauffer L
    15. León-Mimila P
    16. Canizales-Quinteros S
    17. Yang X
    18. Beaven SW
    19. Huertas-Vazquez A
    20. Lusis AJ

    (2018) The Genetic Architecture of Diet-Induced Hepatic Fibrosis in Mice

    Hepatology 68:2182–2196.

    https://doi.org/10.1002/hep.30113

    • Google Scholar
38. 1. Jayakumar S
    2. Middleton MS
    3. Lawitz EJ
    4. Mantry PS
    5. Caldwell SH
    6. Arnold H
    7. Mae Diehl A
    8. Ghalib R
    9. Elkhashab M
    10. Abdelmalek MF
    11. Kowdley KV
    12. Stephen Djedjos C
    13. Xu R
    14. Han L
    15. Mani Subramanian G
    16. Myers RP
    17. Goodman ZD
    18. Afdhal NH
    19. Charlton MR
    20. Sirlin CB
    21. Loomba R

    (2019) Longitudinal correlations between MRE, MRI-PDFF, and liver histology in patients with non-alcoholic steatohepatitis: analysis of data from a phase II trial of selonsertib

    Journal of Hepatology 70:133–141.

    https://doi.org/10.1016/j.jhep.2018.09.024

    • PubMed
    • Google Scholar
39. 1. Kaden-Volynets V
    2. Basic M
    3. Neumann U
    4. Pretz D
    5. Rings A
    6. Bleich A
    7. Bischoff SC

    (2019) Lack of liver steatosis in germ-free mice following hypercaloric diets

    European Journal of Nutrition 58:1933–1945.

    https://doi.org/10.1007/s00394-018-1748-4

    • PubMed
    • Google Scholar
40. 1. Keech A
    2. Simes RJ
    3. Barter P
    4. Best J
    5. Scott R
    6. Taskinen MR
    7. Forder P
    8. Pillai A
    9. Davis T
    10. Glasziou P
    11. Drury P
    12. Kesäniemi YA
    13. Sullivan D
    14. Hunt D
    15. Colman P
    16. d'Emden M
    17. Whiting M
    18. Ehnholm C
    19. Laakso M
    20. FIELD study investigators

    (2005) Effects of long-term fenofibrate therapy on cardiovascular events in 9795 people with type 2 diabetes mellitus (the FIELD study): randomised controlled trial

    Lancet 366:1849–1861.

    https://doi.org/10.1016/S0140-6736(05)67667-2

    • PubMed
    • Google Scholar
41. 1. Kleiner DE
    2. Brunt EM
    3. Van Natta M
    4. Behling C
    5. Contos MJ
    6. Cummings OW
    7. Ferrell LD
    8. Liu Y-C
    9. Torbenson MS
    10. Unalp-Arida A
    11. Yeh M
    12. McCullough AJ
    13. Sanyal AJ
    14. Nonalcoholic Steatohepatitis Clinical Research Network

    (2005) Design and validation of a histological scoring system for nonalcoholic fatty liver disease

    Hepatology 41:1313–1321.

    https://doi.org/10.1002/hep.20701

    • Google Scholar
42. 1. Koutoukidis DA
    2. Astbury NM
    3. Tudor KE
    4. Morris E
    5. Henry JA
    6. Noreik M
    7. Jebb SA
    8. Aveyard P

    (2019) Association of weight loss interventions with changes in biomarkers of nonalcoholic fatty liver disease: a systematic review and Meta-analysis

    JAMA Internal Medicine 179:2248.

    https://doi.org/10.1001/jamainternmed.2019.2248

    • Google Scholar
43. 1. Landis SC
    2. Amara SG
    3. Asadullah K
    4. Austin CP
    5. Blumenstein R
    6. Bradley EW
    7. Crystal RG
    8. Darnell RB
    9. Ferrante RJ
    10. Fillit H
    11. Finkelstein R
    12. Fisher M
    13. Gendelman HE
    14. Golub RM
    15. Goudreau JL
    16. Gross RA
    17. Gubitz AK
    18. Hesterlee SE
    19. Howells DW
    20. Huguenard J
    21. Kelner K
    22. Koroshetz W
    23. Krainc D
    24. Lazic SE
    25. Levine MS
    26. Macleod MR
    27. McCall JM
    28. Moxley RT
    29. Narasimhan K
    30. Noble LJ
    31. Perrin S
    32. Porter JD
    33. Steward O
    34. Unger E
    35. Utz U
    36. Silberberg SD

    (2012) A call for transparent reporting to optimize the predictive value of preclinical research

    Nature 490:187–191.

    https://doi.org/10.1038/nature11556

    • PubMed
    • Google Scholar
44. 1. Lassailly G
    2. Caiazzo R
    3. Buob D
    4. Pigeyre M
    5. Verkindt H
    6. Labreuche J
    7. Raverdy V
    8. Leteurtre E
    9. Dharancy S
    10. Louvet A
    11. Romon M
    12. Duhamel A
    13. Pattou F
    14. Mathurin P

    (2015) Bariatric surgery reduces features of nonalcoholic steatohepatitis in morbidly obese patients

    Gastroenterology 149:379–388.

    https://doi.org/10.1053/j.gastro.2015.04.014

    • PubMed
    • Google Scholar
45. 1. Lee Y
    2. Doumouras AG
    3. Yu J
    4. Brar K
    5. Banfield L
    6. Gmora S
    7. Anvari M
    8. Hong D

    (2019) Complete resolution of nonalcoholic fatty liver disease after bariatric surgery: a systematic review and Meta-analysis

    Clinical Gastroenterology and Hepatology 17:1040–1060.

    https://doi.org/10.1016/j.cgh.2018.10.017

    • PubMed
    • Google Scholar
46. 1. Liu X
    2. Gershenfeld HK

    (2001) Genetic differences in the tail-suspension test and its relationship to imipramine response among 11 inbred strains of mice

    Biological Psychiatry 49:575–581.

    https://doi.org/10.1016/S0006-3223(00)01028-3

    • PubMed
    • Google Scholar
47. 1. Loomba R
    2. Neuschwander‐Tetri BA
    3. Sanyal A
    4. Chalasani N
    5. Diehl AM
    6. Terrault N
    7. Kowdley K
    8. Dasarathy S
    9. Kleiner D
    10. Behling C
    11. Lavine J
    12. Van Natta M
    13. Middleton M
    14. Tonascia J
    15. Sirlin C
    16. Allende D
    17. Dasarathy S
    18. McCullough AJ
    19. Penumatsa R
    20. Dasarathy J
    21. Lavine JE
    22. Abdelmalek MF
    23. Bashir M
    24. Buie S
    25. Diehl AM
    26. Guy C
    27. Kigongo C
    28. Kopping M
    29. Malik D
    30. Piercy D
    31. Chalasani N
    32. Cummings OW
    33. Gawrieh S
    34. Ragozzino L
    35. Sandrasegaran K
    36. Vuppalanchi R
    37. Brunt EM
    38. Cattoor T
    39. Carpenter D
    40. Freebersyser J
    41. King D
    42. Lai J
    43. Neuschwander‐Tetri BA
    44. Siegner J
    45. Stewart S
    46. Torretta S
    47. Wriston K
    48. Gonzalez MC
    49. Davila J
    50. Jhaveri M
    51. Kowdley KV
    52. Mukhtar N
    53. Ness E
    54. Poitevin M
    55. Quist B
    56. Soo S
    57. Ang B
    58. Behling C
    59. Bhatt A
    60. Loomba R
    61. Middleton MS
    62. Sirlin C
    63. Akhter MF
    64. Bass NM
    65. Brandman D
    66. Gill R
    67. Hameed B
    68. Maher J
    69. Terrault N
    70. Ungermann A
    71. Yeh M
    72. Boyett S
    73. Contos MJ
    74. Kirwin S
    75. Luketic VAC
    76. Puri P
    77. Sanyal AJ
    78. Schlosser J
    79. Siddiqui MS
    80. Yost‐Schomer L
    81. Brunt EM
    82. Fowler K
    83. Kleiner DE
    84. Doo EC
    85. Hall S
    86. Hoofnagle JH
    87. Robuck PR
    88. Sherker AH
    89. Torrance R
    90. Belt P
    91. Clark JM
    92. Dodge J
    93. Donithan M
    94. Isaacson M
    95. Lazo M
    96. Meinert J
    97. Miriel L
    98. Sharkey EP
    99. Smith J
    100. Smith M
    101. Sternberg A
    102. Tonascia J
    103. Van Natta ML
    104. Wagoner A
    105. Wilson LA
    106. Yamada G
    107. Yates K
    108. Covarrubias Y
    109. Gamst A
    110. Hamilton G
    111. Henderson W
    112. Hooker J
    113. Lavine JE
    114. Loomba R
    115. Middleton MS
    116. Schlein A
    117. Schwimmer JB
    118. Shen W
    119. Sirlin C
    120. Wolfson T

    (2020) Multicenter validation of association between decline in mri‐pdff and histologic response in NASH

    Hepatology 72:1219–1229.

    https://doi.org/10.1002/hep.31121

    • Google Scholar
48. 1. Macleod MR
    2. O'Collins T
    3. Horky LL
    4. Howells DW
    5. Donnan GA

    (2005) Systematic review and metaanalysis of the efficacy of FK506 in experimental stroke

    Journal of Cerebral Blood Flow & Metabolism 25:713–721.

    https://doi.org/10.1038/sj.jcbfm.9600064

    • PubMed
    • Google Scholar
49. 1. Macleod MR
    2. Lawson McLean A
    3. Kyriakopoulou A
    4. Serghiou S
    5. de Wilde A
    6. Sherratt N
    7. Hirst T
    8. Hemblade R
    9. Bahor Z
    10. Nunes-Fonseca C
    11. Potluru A
    12. Thomson A
    13. Baginskaite J
    14. Baginskitae J
    15. Egan K
    16. Vesterinen H
    17. Currie GL
    18. Churilov L
    19. Howells DW
    20. Sena ES

    (2015) Risk of Bias in reports of in vivo research: a focus for improvement

    PLOS Biology 13:e1002273.

    https://doi.org/10.1371/journal.pbio.1002273

    • PubMed
    • Google Scholar
50. 1. Mann JP
    2. De Vito R
    3. Mosca A
    4. Alisi A
    5. Armstrong MJ
    6. Raponi M
    7. Baumann U
    8. Nobili V

    (2016) Portal inflammation is independently associated with fibrosis and metabolic syndrome in pediatric nonalcoholic fatty liver disease

    Hepatology 63:745–753.

    https://doi.org/10.1002/hep.28374

    • PubMed
    • Google Scholar
51. 1. Martin MD
    2. Danahy DB
    3. Hartwig SM
    4. Harty JT
    5. Badovinac VP

    (2017) Revealing the complexity in CD8 T cell responses to infection in inbred C57B/6 versus outbred swiss mice

    Frontiers in Immunology 8:1527.

    https://doi.org/10.3389/fimmu.2017.01527

    • PubMed
    • Google Scholar
52. 1. Mazagova M
    2. Wang L
    3. Anfora AT
    4. Wissmueller M
    5. Lesley SA
    6. Miyamoto Y
    7. Eckmann L
    8. Dhungana S
    9. Pathmasiri W
    10. Sumner S
    11. Westwater C
    12. Brenner DA
    13. Scmiabl B

    (2015) Commensal Microbiota is hepatoprotective and prevents liver fibrosis in mice

    The FASEB Journal 29:1043–1055.

    https://doi.org/10.1096/fj.14-259515

    • Google Scholar
53. 1. Mestas J
    2. Hughes CC

    (2004) Of mice and not men: differences between mouse and human immunology

    The Journal of Immunology 172:2731–2738.

    https://doi.org/10.4049/jimmunol.172.5.2731

    • PubMed
    • Google Scholar
54. 1. Oscarsson J
    2. Önnerhag K
    3. Risérus U
    4. Sundén M
    5. Johansson L
    6. Jansson PA
    7. Moris L
    8. Nilsson PM
    9. Eriksson JW
    10. Lind L

    (2018) Effects of free omega-3 carboxylic acids and fenofibrate on liver fat content in patients with hypertriglyceridemia and non-alcoholic fatty liver disease: a double-blind, randomized, placebo-controlled study

    Journal of Clinical Lipidology 12:1390–1403.

    https://doi.org/10.1016/j.jacl.2018.08.003

    • PubMed
    • Google Scholar
55. 1. Ouzzani M
    2. Hammady H
    3. Fedorowicz Z
    4. Elmagarmid A

    (2016) Rayyan—a web and mobile app for systematic reviews

    Systematic Reviews 5:210.

    https://doi.org/10.1186/s13643-016-0384-4

    • Google Scholar
56. 1. Perel P
    2. Roberts I
    3. Sena E
    4. Wheble P
    5. Briscoe C
    6. Sandercock P
    7. Macleod M
    8. Mignini LE
    9. Jayaram P
    10. Khan KS

    (2007) Comparison of treatment effects between animal experiments and clinical trials: systematic review

    BMJ 334:197.

    https://doi.org/10.1136/bmj.39048.407928.BE

    • PubMed
    • Google Scholar
57. 1. Prescott MJ
    2. Lidster K

    (2017) Improving quality of science through better animal welfare: the NC3Rs strategy

    Lab Animal 46:152–156.

    https://doi.org/10.1038/laban.1217

    • PubMed
    • Google Scholar
58. Software

    1. R Core Development team

    (2019)

    A language and environment for statistical computing

    R Foundation for Statistical Computing, Vienna, Austria.
59. 1. Rakha EA
    2. Adamson L
    3. Bell E
    4. Neal K
    5. Ryder SD
    6. Kaye PV
    7. Aithal GP

    (2010) Portal inflammation is associated with advanced histological changes in alcoholic and non-alcoholic fatty liver disease

    Journal of Clinical Pathology 63:790–795.

    https://doi.org/10.1136/jcp.2010.079145

    • Google Scholar
60. 1. Rangarajan A
    2. Weinberg RA

    (2003) Comparative biology of mouse versus human cells: modelling human Cancer in mice

    Nature Reviews Cancer 3:952–959.

    https://doi.org/10.1038/nrc1235

    • Google Scholar
61. 1. Ratziu V
    2. Sanyal AJ
    3. Loomba R
    4. Rinella M
    5. Harrison S
    6. Anstee QM
    7. Goodman Z
    8. Bedossa P
    9. MacConell L
    10. Shringarpure R
    11. Shah A
    12. Younossi Z

    (2019) REGENERATE: design of a pivotal, randomised, phase 3 study evaluating the safety and efficacy of obeticholic acid in patients with fibrosis due to nonalcoholic steatohepatitis

    Contemporary Clinical Trials 84:105803.

    https://doi.org/10.1016/j.cct.2019.06.017

    • PubMed
    • Google Scholar
62. 1. Sanyal AJ
    2. Chalasani N
    3. Kowdley KV
    4. McCullough A
    5. Diehl AM
    6. Bass NM
    7. Neuschwander-Tetri BA
    8. Lavine JE
    9. Tonascia J
    10. Unalp A
    11. Van Natta M
    12. Clark J
    13. Brunt EM
    14. Kleiner DE
    15. Hoofnagle JH
    16. Robuck PR
    17. NASH CRN

    (2010) Pioglitazone, vitamin E, or placebo for nonalcoholic steatohepatitis

    New England Journal of Medicine 362:1675–1685.

    https://doi.org/10.1056/NEJMoa0907929

    • PubMed
    • Google Scholar
63. 1. Sanyal AJ
    2. Abdelmalek MF
    3. Suzuki A
    4. Cummings OW
    5. Chojkier M
    6. EPE-A Study Group

    (2014) No significant effects of ethyl-eicosapentanoic acid on histologic features of nonalcoholic steatohepatitis in a phase 2 trial

    Gastroenterology 147:377–384.

    https://doi.org/10.1053/j.gastro.2014.04.046

    • PubMed
    • Google Scholar
64. 1. Sanyal AJ

    (2019) Past, present and future perspectives in nonalcoholic fatty liver disease

    Nature Reviews Gastroenterology & Hepatology 16:377–386.

    https://doi.org/10.1038/s41575-019-0144-8

    • PubMed
    • Google Scholar
65. Software

    1. Schwarzer G O

    (2007) Meta: an R package for meta-analysis, version 4.9-1

    R News.

    https://cran.r-project.org/package=meta
66. 1. Sena ES
    2. van der Worp HB
    3. Bath PM
    4. Howells DW
    5. Macleod MR

    (2010) Publication Bias in reports of animal stroke studies leads to major overstatement of efficacy

    PLOS Biology 8:e1000344.

    https://doi.org/10.1371/journal.pbio.1000344

    • PubMed
    • Google Scholar
67. 1. STELLAR-3 and STELLAR-4 Investigators
    2. Harrison SA
    3. Wong VW
    4. Okanoue T
    5. Bzowej N
    6. Vuppalanchi R
    7. Younes Z
    8. Kohli A
    9. Sarin S
    10. Caldwell SH
    11. Alkhouri N
    12. Shiffman ML
    13. Camargo M
    14. Li G
    15. Kersey K
    16. Jia C
    17. Zhu Y
    18. Djedjos CS
    19. Subramanian GM
    20. Myers RP
    21. Gunn N
    22. Sheikh A
    23. Anstee QM
    24. Romero-Gomez M
    25. Trauner M
    26. Goodman Z
    27. Lawitz EJ
    28. Younossi Z

    (2020) Selonsertib for patients with bridging fibrosis or compensated cirrhosis due to NASH: results from randomized phase III STELLAR trials

    Journal of Hepatology 73:26–39.

    https://doi.org/10.1016/j.jhep.2020.02.027

    • PubMed
    • Google Scholar
68. 1. Stine JG
    2. Munaganuru N
    3. Barnard A
    4. Wang JL
    5. Kaulback K
    6. Argo CK
    7. Singh S
    8. Fowler KJ
    9. Sirlin CB
    10. Loomba R

    (2020) Change in MRI-PDFF and histologic response in patients with nonalcoholic steatohepatitis: a systematic review and Meta-Analysis

    Clinical Gastroenterology and Hepatology 1:61.

    https://doi.org/10.1016/j.cgh.2020.08.061

    • Google Scholar
69. 1. Tsilidis KK
    2. Panagiotou OA
    3. Sena ES
    4. Aretouli E
    5. Evangelou E
    6. Howells DW
    7. Al-Shahi Salman R
    8. Macleod MR
    9. Ioannidis JP

    (2013) Evaluation of excess significance Bias in animal studies of neurological diseases

    PLOS Biology 11:e1001609.

    https://doi.org/10.1371/journal.pbio.1001609

    • PubMed
    • Google Scholar
70. 1. van der Worp HB
    2. Howells DW
    3. Sena ES
    4. Porritt MJ
    5. Rewell S
    6. O'Collins V
    7. Macleod MR

    (2010) Can animal models of disease reliably inform human studies?

    PLOS Medicine 7:e1000245.

    https://doi.org/10.1371/journal.pmed.1000245

    • PubMed
    • Google Scholar
71. 1. Viechtbauer W

    (2010) Conducting Meta-Analyses in R with the metafor package

    Journal of Statistical Software 36:1–48.

    https://doi.org/10.18637/jss.v036.i03

    • Google Scholar
72. 1. von Herrath M
    2. Pagni PP
    3. Grove K
    4. Christoffersson G
    5. Tang-Christensen M
    6. Karlsen AE
    7. Petersen JS

    (2019) Case reports of Pre-clinical replication studies in metabolism and diabetes

    Cell Metabolism 29:795–802.

    https://doi.org/10.1016/j.cmet.2019.02.004

    • PubMed
    • Google Scholar
73. 1. Watzlawick R
    2. Antonic A
    3. Sena ES
    4. Kopp MA
    5. Rind J
    6. Dirnagl U
    7. Macleod M
    8. Howells DW
    9. Schwab JM

    (2019) Outcome heterogeneity and Bias in acute experimental spinal cord injury: a meta-analysis

    Neurology 93:e40–e51.

    https://doi.org/10.1212/WNL.0000000000007718

    • PubMed
    • Google Scholar
74. 1. Younossi Z
    2. Anstee QM
    3. Marietti M
    4. Hardy T
    5. Henry L
    6. Eslam M
    7. George J
    8. Bugianesi E

    (2018) Global burden of NAFLD and NASH: trends, predictions, risk factors and prevention

    Nature Reviews Gastroenterology & Hepatology 15:11–20.

    https://doi.org/10.1038/nrgastro.2017.109

    • PubMed
    • Google Scholar
75. 1. Younossi ZM
    2. Golabi P
    3. de Avila L
    4. Paik JM
    5. Srishord M
    6. Fukui N
    7. Qiu Y
    8. Burns L
    9. Afendy A
    10. Nader F

    (2019) The global epidemiology of NAFLD and NASH in patients with type 2 diabetes: A systematic review and meta-analysis

    Journal of Hepatology 71:793–801.

    https://doi.org/10.1016/j.jhep.2019.06.021

    • Google Scholar

Acceptance summary:

Your systematic analysis of animal studies of fatty liver disease as they compare to human disease is timely. In particular, your studies will be of interest to colleagues in the pharmaceutical industry who are working to develop treatments for the growing problem of NAFLD.

Decision letter after peer review:

Thank you for submitting your article "Multiple drug classes show similar treatment effect sizes in animal models of fatty liver disease" for consideration by eLife. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Eduardo Franco as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Sarah McCann (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

As the editors have judged that your manuscript is of interest, but as described below that additional experiments are required before it is published, we would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). First, because many researchers have temporarily lost access to the labs, we will give authors as much time as they need to submit revised manuscripts. We are also offering, if you choose, to post the manuscript to bioRxiv (if it is not already there) along with this decision letter and a formal designation that the manuscript is "in revision at eLife". Please let us know if you would like to pursue this option. (If your work is more suitable for medRxiv, you will need to post the preprint yourself, as the mechanisms for us to do so are still in development.)

Summary:

This study provides a comprehensive overview of the field of preclinical NAFLD research. These reviews are important for summarizing and evaluating current evidence and identifying gaps and areas for improvement for future research. It is clearly structured and follows widely accepted methods. The lack of a pre-specified statistical analysis plan and deviations from the protocol represent possible risks of bias in the review and the search is out of date. I'm unsure whether the relatively strong emphasis on the weak association between drug class and treatment effect is justified. Importantly, the data are discussed in relation to current issues in preclinical research around reproducibility and translation.

Essential revisions:

A very major concern is the overly broad inclusion and grouping criteria used to structure this analysis. It seems to be too broad for a biologically meaningful and productive meta analysis. NAFLD is a heterogenous mix of states (steatosis, steatitis, cirrhosis). Liver triglyceride per se is not a robust measure of NAFLD. A reduction in liver triglyceride alone would likely not be sufficient for regulatory approval for a NAFLD drug. Were the studies that were pooled looking at liver triglyceride as their primary metric, or were they a mix, including other (inflammation, fibrosis) end points more relevant to NAFLD?

How were the drug classes chosen? There seem to be four types of classes, which are not equally valid groupings. 1) Some drug classes include only are a single compound (e.g. Vit E, curcumin,.…), 2) others are groups with a common primary mechanism (e.g. DPP4 inhibitor, SGLT2 inhibitor, FXR agonist, etc), but include different compounds so off-target activities presumably vary, 3) others are poorly defined entities that one could argue should constituent each be treated as separate single compounds (e.g. probiotics, Protoberberine alkaloids, polyphenols), and 4) well defined classes which can have individual members with very different biological activities (e.g., bile acids, ), again, suggesting the more subgroups might be needed for biological sense.

A major determinant of liver triglyceride is body triglyceride. Why was body weight not included in the analysis? I do not think using % overcomes the massive differences in liver triglyceride due to massive differences in body weight.

'Model' includes both genetic models and diet models, finding only 10 categories. This seems like an overly small number of categories. There are many kinds of high-fat diets that have very different compositions and that give very different results.

The analyte in this study is % loss of liver triglyceride. How robust a metric is this for a meta analysis? Liver triglyceride correlates with total body triglyceride and body weight; loss of enough body weight will cause an experiment to be stopped for humane reasons. So the % liver triglyceride would seem to me to be a potentially truncated/limited/bounded metric. Is this affecting the variation that is needed/used for the meta analysis?

It is great to see that there was a published study protocol, however there was no statistical analysis plan specified. There are also several deviations from the protocol that have not been addressed e.g. lack of secondary outcomes, not addressing key research questions as planned, analysis of subgroups. The search is 2.5 years old and while there are over 200 included studies, representing a substantial body of work, the authors note this is a highly active field and thus it is could be a substantial limitation if the data are out of date.

It is reported that there is weak evidence of difference between drug classes and p values are given (p=0.014 subgroup analysis; p=0.002 meta-reg) but as there is no pre-specified statistical analysis plan and no description in the Materials and methods, we don't know what the authors consider to be significant effects. The authors conclude quite strongly that there is limited difference between classes and describe overlapping CIs but e.g. the mean effect of the most effective class is ~4 times higher than the least effective, with no overlapping CIs. It would be useful to know when interpreting these results what a clinically relevant treatment effect would be. The wide CIs limit strong conclusions but as the authors note, the number of animals in each group is low and some classes have few comparisons.

I found the description that "The confidence intervals of 20 out of 21 drug classes overlapped" slightly confusing (Figure 2) when Thiazolidinediones CIs do not overlap with at least 3 other classes but do overlap with many others.

There is also substantial within-class heterogeneity for some drug classes. Could there be differences in efficacy between individual drugs not captured at the class level? How many individual drugs are represented by each class? (Post hoc) subgroup analysis of classes with sufficient comparisons might provide more nuanced information on treatment response and factors affecting efficacy.

While it is dangerous to over-interpret results, I would probably recommend softening the argument of limited differences slightly.

Does it make sense to look at drug dose as a potential source of heterogeneity across different drugs and classes? We would most likely expect individual drugs to have different effective doses.

The multivariable regression model explains ~50% of the heterogeneity in a subset of the studies but is not discussed anywhere in the manuscript (10 variables/71 studies?) Is there something about these studies that can provide information on the broader dataset or is this analysis of limited value?

As only drugs/classes that have been used in clinical trials have been included, it would be interesting to note any similarities/differences in results e.g. GLP-1 agonists demonstrate a robust effect in the current data and the authors report that they demonstrated potential efficacy in clinical trial. Is there anything we can learn from examples of potentially successful translation?

Essential revisions:

A very major concern is the overly broad inclusion and grouping criteria used to structure this analysis. It seems to be too broad for a biologically meaningful and productive meta analysis. NAFLD is a heterogenous mix of states (steatosis, steatitis, cirrhosis). Liver triglyceride per se is not a robust measure of NAFLD. A reduction in liver triglyceride alone would likely not be sufficient for regulatory approval for a NAFLD drug. Were the studies that were pooled looking at liver triglyceride as their primary metric, or were they a mix, including other (inflammation, fibrosis) end points more relevant to NAFLD?

Thank you for this suggestion. We agree that liver triglyceride content, particularly in isolation, is not the optimum indicator of NAFLD severity. Therefore we have now added meta-analyses for histological outcomes: steatosis, lobular inflammation, ballooning, overall NAFLD Activity Score, and fibrosis stage. (We also collected data on portal inflammation severity but there was insufficient data for meta-analysis.) We consider this now to appropriately reflect the spectrum of disease in NAFLD.

The included studies rarely identified a primary outcome and they were difficult to categorise into, for example, those focusing on a ‘liver phenotype’ versus those focusing on the systemic ‘metabolic syndrome’. Therefore we believe that reporting both hepatic TG content as well as histology is the least biased method.

Reassuringly, we observed similar trends for multiple outcomes. We have focused the main conclusions of our study on those that were consistent across several meta-analyses and meta-regressions.

Though histological outcomes are more directly comparable to human data, we have retained hepatic TG as an outcome because it is the most widely reported measure.

How were the drug classes chosen? There seem to be four types of classes, which are not equally valid groupings. 1) Some drug classes include only are a single compound (e.g. Vit E, curcumin,.…), 2) others are groups with a common primary mechanism (e.g. DPP4 inhibitor, SGLT2 inhibitor, FXR agonist, etc), but include different compounds so off-target activities presumably vary, 3) others are poorly defined entities that one could argue should constituent each be treated as separate single compounds (e.g. probiotics, Protoberberine alkaloids, polyphenols), and 4) well defined classes which can have individual members with very different biological activities (e.g., bile acids, ), again, suggesting the more subgroups might be needed for biological sense.

We appreciate that the grouping of drugs into classes is a simplification and it was challenging to be consistent as not all the drugs neatly fit into pharmacological classes. We had originally aimed to group drugs based on their mechanism of action. We have now refined the grouping, being more stringent to separate drugs where possible and where their mechanism is less well established. We have added detail to the Materials and methods to describe the process of grouping drugs into classes.

We have also run each meta-analysis with sub-grouping by individual drugs and see consistent trends for the principal findings.

We appreciate that there are some drug groups that remain broad, in particular “Probiotics (mix)” and “Omega-3 polyunsaturated fatty acids (mix)”. Where possible we have separated groups within them, specifically Lactobacillus / Bifidobacterium and eicosapentaenoic acid / docosahexaenoic acid. However we have retained these broader groups to reduce the number of studies being excluded.

We have also added a section in the limitations of the Discussion to address that this is a simplification and grouping is variably imprecise. We have also specifically highlighted to that conclusions from less well-defined classes may need to be interpreted with caution.

A major determinant of liver triglyceride is body triglyceride. Why was body weight not included in the analysis? I do not think using % overcomes the massive differences in liver triglyceride due to massive differences in body weight.

Thank you for this suggestion and we apologise for not including this initially. Given the body weight often influences liver triglyceride via insulin resistance, we have now performed a series of analyses to address this. We felt it was important to include glucose (and insulin) as there are some models (particularly lipodystrophic models) that may develop marked hepatic steatosis without becoming obese but are insulin resistant.

We have collected data for, and calculated, the difference in body weight, fasting glucose, and fasting insulin for all included studies, then included these as variables in meta-regression. We have found highly consistent associations across multiple outcomes where weight loss and lower insulin/glucose are associated with greater treatment response.

We noted that many drugs were associated significant differences in weight, glucose, or insulin, in animals though they do not cause such responses in humans (e.g. vitamin D or statins). Our interpretation is that the weight loss and change in insulin sensitivity seen in animals may be confounding results and contributing to the lack of translation from animals to humans.

We do appreciate the fasting insulin and glucose are technically not direct measures of insulin resistance, however they are the most widely reported metrics. We have added a section in the limitations to acknowledge that other data (e.g. hyperinsulinaemic-euglycaemic clamps) would be more accurate.

'Model' includes both genetic models and diet models, finding only 10 categories. This seems like an overly small number of categories. There are many kinds of high-fat diets that have very different compositions and that give very different results.

We realise that this is one of several simplifications we have performed in an attempt to synthesise the data. We performed a categorical grouping of models as a ‘high-level’ descriptor. This identified 137 different ‘core’ model categories. They are found in Column E (“Model_simple”) of Figure 1—source data 1 (i.e. the raw data used in the meta-analysis). These are highly variable, for example: “SREBP-1c transgenic overepression + High-fat and high-fructose diet”, “ApoE knockout (ApoE -/-) + high-fat, high-cholesterol diet + T0901317 (LXR agonist)”, and “Pemt-/- + high-fat diet”.

In order for these Models to be included as variables in the meta-regression we required a minimum of 3 unique studies to use them. Due to variation and the high number of studies that used a few common models (e.g. “high-fat diet”, “Leptin deficiency (ob/ob)”), comparatively few different models were included in the meta-regression. We appreciate that our analysis will not have captured the full variation of Model and we have now addressed this as a limitation in the Discussion.

In addition, we have now added data on other dietary constituents (cholesterol, sucrose, and fructose/glucose), which have been included as variables in the meta-regression. Again, we appreciate that this is simplifying the complexity of dietary models however we now use these data to draw general conclusions that diet can affect treatment response. We have specifically commented that analyses of fat %kcal in isolation may be misleading because lower fat %kcal is more frequently coupled with added cholesterol (and/or fructose) than 60% kcal fat diets.

Whilst updating our analyses, we noted that many continuous variables used in the meta-regression were heavily skewed. We have therefore log-transformed them prior to analysis.

The analyte in this study is % loss of liver triglyceride. How robust a metric is this for a meta analysis? Liver triglyceride correlates with total body triglyceride and body weight; loss of enough body weight will cause an experiment to be stopped for humane reasons. So the % liver triglyceride would seem to me to be a potentially truncated/limited/bounded metric. Is this affecting the variation that is needed/used for the meta analysis?

Thank you for highlighting this point. To our knowledge, the most systematic previous analysis of liver fat in animals was from the Hybrid Mouse Diversity Panel (Hui et al., 2015) who found >30 fold variation in hepatic TG across strains.

It is difficult to know whether studies were stopped early due to animal health (including weight loss). We have now included weight difference between placebo and intervention as a variable, which demonstrated a strong association with hepatic TG.

As described above, we have now added multiple histological outcome measures, therefore we have increased confidence in our findings given that we are no longer basing our conclusions on a single metric.

Whilst we did not formally record this, we can anecdotally report that fewer than five of all included studies reported specifics of harm to animals or protocol changes due to animal welfare.

We also note that the range of percentage change in hepatic TG is similar to that reported in another pre-clinical meta-regression analysis in a different field (Watzlawick et al., 2019). Therefore we believe that the dynamic range is appropriate for the analyses performed, especially in combination with the other metrics used.

It is great to see that there was a published study protocol, however there was no statistical analysis plan specified. There are also several deviations from the protocol that have not been addressed e.g. lack of secondary outcomes, not addressing key research questions as planned, analysis of subgroups. The search is 2.5 years old and while there are over 200 included studies, representing a substantial body of work, the authors note this is a highly active field and thus it is could be a substantial limitation if the data are out of date.

We appreciate that the lack of a pre-specified statistical analysis plan is a potential risk of bias. We believe that through sharing our full raw data, code for analysis, the results of all analyses conducted (i.e. not selective reporting) we have minimised the bias introduced from this. We have also specifically commented on it in the Discussion to highlight it as a limitation.

In addition, we have improved the methodology we have used for multivariable meta-analysis to use an approach less open to bias, which considers all variables for importance. We now present results from multi-model inference to determine the top model, which is then subjected to a multiple permutation test to determine the fit of the final models.

With this revised version of the study we have now been able to address several of our secondary outcomes regarding liver histology and insulin resistance/weight.

In the future we hope to build on this initial database to further study our secondary outcomes with a more qualitative approach but we would consider that to be a separate project and anticipate that would take a substantial duration. We trust that the revisions to the current study have made it a more complete analysis to address one research question.

We have now updated our search to January 2019, which required screening a further 1118 articles (991 for full-text review). This, combined with addition of studies reporting histology, has increased the number of included studies from 244 (414 cohorts) to 414 studies (603 cohorts). We appreciate that this remains out of date however our interpretation is that we have now accumulated sufficient evidence to support the general conclusions we have reached, particularly given that we are focusing on observations that were replicated across multiple outcome measures.

It is reported that there is weak evidence of difference between drug classes and p values are given (p=0.014 subgroup analysis; p=0.002 meta-reg) but as there is no pre-specified statistical analysis plan and no description in the Materials and methods, we don't know what the authors consider to be significant effects. The authors conclude quite strongly that there is limited difference between classes and describe overlapping CIs but e.g. the mean effect of the most effective class is ~4 times higher than the least effective, with no overlapping CIs. It would be useful to know when interpreting these results what a clinically relevant treatment effect would be. The wide CIs limit strong conclusions but as the authors note, the number of animals in each group is low and some classes have few comparisons.

We acknowledge that it was not helpful to make non-specific statements about the magnitude of effects, such as ‘weak’ or ‘large’ without pre-specifying criteria. We have added to our Materials and methods to detail how all analyses were assessed for significance.

We have also tempered our discussion regarding the similarity of drug classes. We have commented where confidence intervals have overlapped with the 95% CI for the overall estimate and stated which drugs had deviated from the others based on their 95% CI.

There is a recently established benchmark for change in hepatic TG: a reduction of ≥30% on MRI-measured liver fat is linked to a significant improvement in histology. We have now included this comparison in our manuscript (Jayakumar et al., 2019; Loomba et al., 2020; Stine et al., 2020).

We have also added a table (Supplementary file 1) to provide a narrative summary of the evidence for each drug class studied, with references to the key published RCTs.

I found the description that "The confidence intervals of 20 out of 21 drug classes overlapped" slightly confusing (Figure 2) when Thiazolidinediones CIs do not overlap with at least 3 other classes but do overlap with many others.

We apologise for not being clear. As described above, we have removed non-specific statements about overlapping drug classes and have provided further clarity on the criteria against each analysis was assessed.

There is also substantial within-class heterogeneity for some drug classes. Could there be differences in efficacy between individual drugs not captured at the class level? How many individual drugs are represented by each class? (Post hoc) subgroup analysis of classes with sufficient comparisons might provide more nuanced information on treatment response and factors affecting efficacy.

Thank you for this suggestion. We have now performed a sub-analysis by individual drugs for each outcome metric.

We found that considerable heterogeneity remains even when analysing by individual drug, a trend which was replicated for multiple outcomes.

There is variation of effect size within drug classes where more than one drug contributed (e.g. statin), however it is difficult to make firm conclusion about superiority individual drugs within classes because their confidence intervals overlap. We have provided the full results for all analyses to allow readers to examine results for individual drugs that may be of interest.

While it is dangerous to over-interpret results, I would probably recommend softening the argument of limited differences slightly.

We acknowledge that our original phrasing was too bold. We have edited the language and our conclusions significantly in light of our new results.

Does it make sense to look at drug dose as a potential source of heterogeneity across different drugs and classes? We would most likely expect individual drugs to have different effective doses.

This is an important suggestion and we had also expected to observe an effect of drug dose, however we did not find one in our original submission. We have checked and re-analysed the data to ensure we weren’t missing an effect and excluding any studies that used unusual drug doses or units. We then scaled all drug doses by each individual drug to facilitate maximum power for meta-regression. With this methodology we still did not observe any effect using our standard minimum of 3 studies per drug for inclusion. We also tried re-running the analysis where at least 10 studies were required for each drug and still did not observe any effect.

For example, this bubble plot reflects scaled drug dose (x-axis) and mean difference in hepatic TG (y-axis) for fenofibrate, the drug with the largest treatment effect size and had a suitable number of data points for meta-regression:

Author response image 1

Download asset Open asset

We interpret this observation to be a further explanation for challenging translation from animals to humans and have added a comment regarding this in the Discussion.

The multivariable regression model explains ~50% of the heterogeneity in a subset of the studies but is not discussed anywhere in the manuscript (10 variables/71 studies?) Is there something about these studies that can provide information on the broader dataset or is this analysis of limited value?

Apologies for not having discussed these results thoroughly. Following the updates we have described above, we have more robust multivariable meta-regression results and improved performance of models. Several models (particularly for NAS and Steatosis grade) account for >60% of variation, without including drug as a variable, and remain robust after a permutation test.

Whilst we could not replicate the same model accuracy across all outcomes, we have used multimodel inference to provide additional evidence for the importance of individual variables (e.g. fat %kcal, glucose difference). We now conclude these analyses illustrate how the combination of study design and weight loss / insulin resistance are central to determining treatment response in NAFLD.

As only drugs/classes that have been used in clinical trials have been included, it would be interesting to note any similarities/differences in results e.g. GLP-1 agonists demonstrate a robust effect in the current data and the authors report that they demonstrated potential efficacy in clinical trial. Is there anything we can learn from examples of potentially successful translation?

This is a key question and we have now attempted to address it in more detail in the Discussion. The drugs that show good efficacy in humans (thiazolidinediones, GLP-1 agonists) do not stand out at the top of any outcome measure. Conversely, fibrates are consistently the drug class with the most efficacy in animals but do not show efficacy in humans.

Unfortunately, our interpretation of these findings is that there is no clearly discernible pattern.

Author details

1. Harriet Hunter

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Formal analysis, Investigation, Writing - original draft, Writing - review and editing

   Contributed equally with

   Dana de Gracia Hahn, Amedine Duret and Yu Ri Im

   Competing interests

   No competing interests declared

2. Dana de Gracia Hahn

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Formal analysis, Investigation, Writing - original draft, Writing - review and editing

   Contributed equally with

   Harriet Hunter, Amedine Duret and Yu Ri Im

   Competing interests

   No competing interests declared

3. Amedine Duret

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Formal analysis, Methodology, Writing - original draft, Writing - review and editing

   Contributed equally with

   Harriet Hunter, Dana de Gracia Hahn and Yu Ri Im

   Competing interests

   No competing interests declared

4. Yu Ri Im

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Formal analysis, Investigation, Writing - original draft, Writing - review and editing

   Contributed equally with

   Harriet Hunter, Dana de Gracia Hahn and Amedine Duret

   Competing interests

   No competing interests declared

5. Qinrong Cheah

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Investigation, Writing - review and editing

   Competing interests

   No competing interests declared

6. Jiawen Dong

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Investigation, Writing - review and editing

   Competing interests

   No competing interests declared

7. Madison Fairey

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Investigation, Writing - review and editing

   Competing interests

   No competing interests declared

8. Clarissa Hjalmarsson

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Investigation, Writing - review and editing

   Competing interests

   No competing interests declared

9. Alice Li

   School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

   Contribution

   Data curation, Investigation, Writing - review and editing

   Competing interests

   No competing interests declared

10. Hong Kai Lim

    School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Data curation, Investigation, Writing - review and editing

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-7266-7790

11. Lorcan McKeown

    School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Data curation, Investigation, Writing - review and editing

    Competing interests

    No competing interests declared

12. Claudia-Gabriela Mitrofan

    School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Data curation, Investigation, Writing - review and editing

    Competing interests

    No competing interests declared

13. Raunak Rao

    School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Data curation, Investigation, Writing - review and editing

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-6954-575X

14. Mrudula Utukuri

    School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Data curation, Investigation, Writing - review and editing

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0003-1510-469X

15. Ian A Rowe

    Leeds Institute for Medical Research & Leeds Institute for Data Analytics, University of Leeds, Leeds, United Kingdom

    Contribution

    Investigation, Methodology, Writing - review and editing

    Competing interests

    No competing interests declared

16. Jake P Mann

    Institute of Metabolic Science, University of Cambridge, Cambridge, United Kingdom

    Contribution

    Conceptualization, Data curation, Formal analysis, Supervision, Funding acquisition, Investigation, Methodology, Writing - original draft, Project administration, Writing - review and editing

    For correspondence

    jm2032@cam.ac.uk

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0002-4711-9215

• 1,634

  Page views

• 214

  Downloads

• 1

  Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.