Modelling freedom from disease - how do we compare between countries?
A modelling framework for the prediction of the herd-level probability of infection from longitudinal data
Recommendation: posted 24 August 2021, validated 02 September 2021
In this paper, Madouasse et al. (2021) present a generalisable Bayesian method for calculating the probability that a herd is free from disease, based on its prior disease status, and using data (herd status over time over a sufficient number of herds to inform the model) and reasonable prior estimates of the sensitivity and specificity of tests being used to determine animal infection status. Where available, the modelling approach can also include relevant additional risk factors.
By bringing all these factors together, it allows for most countries to use the same analytical approach on their data, with differences across datasets expressed in terms of the uncertainty around the central estimates.
Having a single methodology that generates both a central estimate of disease freedom, and uncertainty thus provides the opportunity (given typically available data) to compare the probability of freedom across different systems. This is relevant in terms of the context of trade (since international trade of livestock in many cases depends on disease freedom). It is also important when evaluating, for example, transnational burdens of disease - and with different regulations in place in different countries, this is invaluable and can be used, for example, to assess risks of zoonotic infection including for zoonotic infection emergence. In the BVD example provided, the point is made that, since regular testing would probably pick up infection rapidly, the addition of risk factors is most valuable where testing is infrequent. This emphasizes the advantages of direct incorporation of risk factors into a single modelling framework.
From a technical point of view, the analysis compares two different packages for the Markov Chain Monte Carlo (MCMC) implementation necesary to run the model. They show that, while there are some slight systematic differences, the estimates provided by the two methods are similar to each other; as one method is approximate but substantially more stable and generally much more computationally efficient, this is an important outcome. Both implementations are freely available and with relevant additional software made similarly available by the authors. This is extremely welcome and should encourage its general adoption across different countries.
No single model can of course account for everything. In particular, the reliance on past data means that there is an implicit assumption common to all purely statistical methods that the underlying risks have not changed. Thus projections to altered circumstances (changing underlying risk factors or systematic changes in testing or test performance) cannot so easily be incorporated, since these factors are complicated by the dynamics of infection that lie outside the modelling approach. Of course the well known quote from George Box that "all models are wrong" applies here - the generality of approach, statistical robustness and open source philosophy adopted make this model very useful indeed.
Madouasse A, Mercat M, van Roon A, Graham D, Guelbenzu M, Santman Berends I, van Schaik G, Nielen M, Frössling J, Ågren E, Humphry RW, Eze J, Gunn GJ, Henry MK, Gethmann J, More SJ, Toft N, Fourichon C (2021) A modelling framework for the prediction of the herd-level probability of infection from longitudinal data. bioRxiv, 2020.07.10.197426, ver. 6 peer-reviewed and recommended by PCI Animal Science. https://doi.org/10.1101/2020.07.10.197426
Rowland Raymond Kao (2021) Modelling freedom from disease - how do we compare between countries?. Peer Community in Animal Science, 100007. https://doi.org/10.24072/pci.animsci.100007
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article. The authors declared that they comply with the PCI rule of having no financial conflicts of interest in relation to the content of the article.
Evaluation round #2
DOI or URL of the preprint: https://doi.org/10.1101/2020.07.10.197426
Version of the preprint: v4
Author's Reply, 11 Aug 2021
Decision by Rowland Raymond Kao, posted 07 Jul 2021
First of all, very sorry for the delays in replying to you on your paper.
However, I am pleased to say that we have now had replies from your two reviewers. While they have made some comments, and responding to them I think will improve the paper, there is nothing there to prevent recommendation - therefore I suggest that you take a look at those comments, and decide for yourselves how best to further to revise it. Once this is done, I will certainly be more than happy to recommend it without need for further review. It is a very useful addition to the literature.
Reviewed by Arata Hidano, 22 Jun 2021
Reviewed by anonymous reviewer, 27 May 2021
Evaluation round #1
DOI or URL of the preprint:
Version of the preprint: v3
Author's Reply, 16 Apr 2021
Decision by Rowland Raymond Kao, posted 01 Dec 2020
Two reviews have been received for this preprint. Both reviewers have highlighted the importance of the problem - i.e. the estimation of herd "freedom from infection" status based on a series of partial observations with imperfect testing, which is particularly a problem where disease incidence is long, and the disease itself with long and variable latent periods. This is the case for BVD (the disease observed here) but also problematic long duration bacterial diseases such as paratubeculosis and bovine Tuberculosis. The development of a general framework, described here, is therefore welcome. Both reviewers have indicated the importance of the problem and relevance of the approach. Both have also been very positive about the execution of the approach, but both have also highlighted methodological issues that, while they do not fundamentally compromise this analysis, do require addressing. A particular issue is the choice and use of priors, highlighted by both reviewers though with different perspectives. It also seems to me that the choice of prior is one area where other adopters of the framework would have to exercise the greatest individual judgement (also, as indicated in their discussion of this from lines 635 onwards, they are very aware of this) and therefore the reviewers comments require considerable attention. I also fully agree with one reviewer on the justification of the use of narrow priors - it suggests possible issues with the model specification itself or with the data, and is not necessarily an issue of the prior. While some of the other reviewer comments are outside my expertise, they all seem sensible and should be responded to by the authors. I also include here some comments from a 3rd reviewer who did not feel it sufficiently within their expertise to offer a full review: "The submitted manuscript aims to predict herd status using all prior information over time. This is an interesting new approach and worth being published. However, author imply in the introduction that they consider the number of animals tested (a frequent feature of herd testing) whereas they don’t consider this option. Only tank milk testing is included. Moreover, I believe there is a mistake in formula 15 for the probability of infection for test-negative herds (I think this should read: (1-T+).[Sp.(1-pS+) / ( Sp.(1-pS+) + (1-Se).pS+) i.e. the negative predictive value for negative test results). Having said that, my advice is to consult a statistician for quality checking of the code. I recommend consulting my colleagues Prof Geoff Jones (firstname.lastname@example.org) or Prof Wesley Johnson (email@example.com). The text requires an overhaul for clarity and typos. The manuscript is definitely worth reviewing as it is topical at a time of emerging infectious diseases, so it would likely attract citations. I cannot review this submission myself as it would need considerable more time and more statistical expertise than I can offer." Overall, this is likely to be a very good contribution to the literature and I look forward to seeing a revision.