Response to comment on 'AIRE-deficient patients harbor unique high-affinity disease-ameliorating autoantibodies'

[This document contains the response from Hertel et al. to specific points in the decision letter sent on 9 April 2019; the full decision letter is available at the following URL: https://elifesciences.org/articles/45826#SA1]

We repeat the reviewers’ points here in italic, and include our replies point by point in Roman.

Reviewer #1:

The formal response by Hertel et al is appropriate in its criticism of one point of contention - the issue of type 1 IFN neutralizing autoantibodies and diabetes development. As mentioned above and in the response, Landegren have reached saturation in their type 1 IFN neutralizing autoantibody assay and, therefore, cannot make their conclusion that there are no differences in type 1 IFN neutralizing autoantibodies between the APECED patients with and without diabetes. The formal response and new data convincingly demonstrate the original claim by Meyer. One point to note, however, is that the majority of GAD antibody positive individuals (healthy or diseased) do not develop type 1 diabetes (unless they have other type 1 diabetes autoantibodies) and it is highly unlikely that many of these have high titers of type 1 IFN neutralizing autoantibodies - be careful about over-generalizing the findings from the APECED patients.

REPLY: We agree with the reviewer, and make clear in our response that neither we nor anyone has suggested that natural Type I IFN neutralization is a widespread means of naturally limiting disease. This notwithstanding, the comparison by Meyer et al. of GAD-reactive T1D⁺ and T1D^- patients, together with the animal studies conducted, provides a powerful illustration of the potential of autoantibodies to be disease-ameliorating, and a justification for examining such possibilities in other settings.

Unfortunately, on their second point of evidence, the Fishman paper shows correlations between the array and the LIPS assay Z scores (note that the LIPS Z scores are actually not very high), which is not entirely convincing as a validation of these antigens and there were only 20 controls. Thus, the formal response is subject to the same flaws - too few controls for Z scores in what they use as evidence to their original claim. Also, I have concerns about Table 1 in the formal response as I don't find the data it is derived from sufficiently conclusive.

REPLY: We absolutely do not agree that relatively low z scores for LIPS assays are of concern. For example, CYP21, which is a very well defined, broadly accepted autoantigen also has a low z score in LIPS. Importantly, these scores are significant. We do not show raw data in Table 2, because the relevant information is provided in reference 4. However, we are happy to provide the raw data, should it be requested. Importantly, the many autoantigen specificities that Meyer et al described and that are further studied by Fishman et al., 2017 have many key properties (e.g. correlation with specific clinical disease course) that are laid out in our response and that could not be expected for false positives.

I do not believe that the authors of the original work should publish a correction, but I do believe that in their response they should have recognized the flaw in defining positivity with Z scores derived from 21 controls and that they should have stated even more clearly that their method of analysis is biased towards identifying false-positives. I believe that it would be fair and objective to say validation was performed for many of the cytokine targets in the original paper and only partially done for some of the other specificities in the Fishman paper.

REPLY: We extensively discussed the issue of false positives in our paper, and have devoted additional attention to this in our response, acknowledging that our choice of statistical method risked false positives as a cost of limiting false negatives. By contrast, the original report by Landegren et al., 2016 is very vulnerable to false negatives, leading them to claim that private reactivities do not exist, despite the convincing demonstration of several of them by us and, more recently, by co-authors of Landegren et al (Sng et al., 2019).

I also suggest that they should have been a little less aggressive in their response.

REPLY: We believe that we have made our case firmly but not aggressively.

However, it was inadequate to make the claim that thousands of antigens can be targeted in these patients. The ProtoArray is just not ideal for such a conclusion without testing all the antigens in a different system. It is also way too expensive to be used with hundreds (a minimum) of controls necessary to appropriately define Z scores, and APECED patients are simply too rare to have a sufficiently large cohort to be conclusive about low frequency positive targets. The follow-up studies that did test some of the antigens by LIPS don't do a lot to dampen these limitations since they also don't have many controls and patients.

REPLY: This critique is intuitive rather than evidence-based. ProtoArrays have proved extremely important, hypothesis-generating guides to the status of sera and other biological samples, provided that those hypotheses are followed up with downstream analyses, as were conducted for cytokine reactivities by Meyer et al¹ and for additional reactivities by Fishman et al⁴. When that is the case, it is fair to draw the community’s attention to conclusions that may be drawn from them. Nonetheless, we have never stated that we have independently validated all the proto-array “hits”, just as we cannot exclude the existence of additional false negatives that lead to an under-estimate of serum reactivity.

In rebuttal, the authors of the Cell paper claim that a sampling of the thousands of private specificities were validated. There was no evidence for this in the published study, and unconvincing preliminary evidence here.

REPLY: We repeat our response to the previous point.

Reviewer #2:

The analysis performed by Landegren provide compelling evidence that the original publication in Cell employed a flawed normalization of the analysis of antibody binding to protein microarrays. As a result, the main conclusion in the Cell paper about the much wider breakdown of tolerance in AIRE deficiency, with antibodies recognizing private specificities unique to one or two patients, is erroneous.

REPLY: First, a thorough reading of Meyer et al makes clear that the issue of private reactivities was not the major conclusion but one of four. Beyond this point, both the biologists and the statisticians on our team completely disagree with this Reviewer’s assessment. Just because the statistical approach knowingly adopted makes our study vulnerable to false positives does not mean that the identified private reactivities are false positives. This point is most convincingly made by our validation of several of the reactivties, to the point of cloning the autoantibodies responsible, and by our demonstration that the properties of many such specificities correlate with key features of disease progression and of AIRE-deficiency. For example, the autoreactivity of female patients toward male antigens, and the response to tissue-restricted MAGE proteins but not to ubiquitously expressed ones with similar physicochemical properties! Such properties confound the reviewers convictions over false positives. Moreover, a recent paper (Sng et al., 2019)authored by a prominent co-author of Landegren et al reports a wide-ranging breakage of tolerance in the B cell compartment, consistent with which they describe additional autoantigen specificities.

We should also note that while Landegren and the reviewers are able to criticize our statistical approach (as do we) for its vulnerability to false positives, none of them offers a constructive suggestion of a method to limit false positives while permitting the detection of private reactivities: the simple fact is that there is no silver bullet method.

In rebuttal, the authors of the Cell paper claim that a sampling of the thousands of private specificities were validated. There was no evidence for this in the published study, and unconvincing preliminary evidence here.

REPLY: As we responded above to Reviewer 1, this critique is not evidence-based. ProtoArrays have proved extremely important, hypothesis-generating guides to the status of sera and other biological samples, provided that those hypotheses are followed up with downstream analyses, as were conducted for cytokine reactivities by Meyer et al¹ and for additional reactivities by Fishman et al⁴. When that is the case, it is fair to draw the community’s attention to conclusions that may be drawn from them. Nonetheless, we have never stated that we have independently validated all the proto-array “hits”, just as we cannot exclude the existence of additional false negatives that lead to an under-estimate of serum reactivity.

In rebuttal, the original authors point out that the data in the Cell paper were from serum titrations, and that if they repeat the experiments with serum diluted 1/10 they also now find neutralizing antibodies in both groups. In the Cell paper, the mean IC50 of neutralizing antibodies in patients without type 1 diabetes was on the order of 1/100,000, at which point there was a difference in titre. However, in the data in the rebuttal, the difference in inhibitory activity is only apparent at one dilution (1/50) and not at 1/250 or 1/10. Since the rebuttal now also shows neutralizing antibodies are present in Type 1 diabetes patients at relatively small differences in titre, and since serum is undiluted in vivo, this appears to warrant a revision to the original conclusions in Cell that "those with T1D showed only low or negligible neutralization.”

REPLY: Our response now provides expanded data-sets that use independent assays to provide unequivocal support for our claims. It is of course the case that the value of serum dilution required to see neutralizing activity will be different depending on the assay used, and will be higher for a less sensitive assay such as the STAT1 phosphorylation employed by Landegren et al. Indeed, because of its relative insensitivity, the pSTAT1 assay is not suitable for the generation of full dose-response curves that are necessary for precise estimation of neutralizing capacity. Nonetheless, whether we employed our highly sensitive assay for IFN-dependent alkaline phosphatase production or the pSTAT1 assay, we could demonstrate a clear and significant capacity to segregate the patients with T1D from those that did not. Hence, our conclusions are fully supported. Finally, we are extremely surprised by the Reviewer’s seeming assertion that IC50 may be irrelevant given that serum is undiluted in vivo. The key site of IFN activity may be the tissues, and it is completely unclear to what degree serum autoantibodies reach that site in “serum concentrations”.

In conclusion, Meyer et al¹ draw the community’s attention to four major aspects of APECED biology. Two of these were unchallenged by Landegren et al., and the challenges made to the other two are insufficient to undermine them and the conclusions that we draw based on them. Those conclusions have provoked numerous follow up studies of benefit to biological understanding and to the elucidation of human pathophysiology.