Wikipedia for better computer models

GMP – some readers know the acronym as standing for Good Manufacturing Practice, a term frequently used in the pharmaceutical, food and nutrition industries. GMP is a quality assurance system in which all the steps and conditions during the production process are documented in detail and monitored. The development of a computer model should not be any different.

In the world of computer modelling, GMP stands for Good Modelling Practices. Bert Annevelink and Koen Meesters, both senior researchers at Wageningen Food & Biobased Research, work on quality assurance for computer models. In the past few years, they have helped develop the GMP Wiki platform.

What are good modelling practices?

Bert Annevelink: “Models are used everywhere, for instance in weather forecasts, to predict maximum water levels in rivers or the spread of infectious diseases, or for the nitrogen problem. Models are becoming increasingly important, including when designing policies. Such policies have a major impact on society — and on individual farmers, for example — so it’s crucial to have sound models.”

“GMP is about what you need to do to end up with a high-quality model. There must be a good description of the purpose of the model, the data used must be documented in detail, the assumptions made when developing the model must be specified, the modelling process must be evaluated continually, and this must all be clearly reported.”

Koen Meesters: “If you work in accordance with GMP and record properly what you have done, you’ll be better able to assess whether the model can make useful predictions in certain new situations. Or you’ll be in a better position to see whether certain effects have been taken into account in another researcher’s model that you are interested in using.

We’ve seen how important good models are with the nitrogen issue. It would be quite unacceptable to force a farmer to sell and then conclude one year later it wasn’t necessary after all because there was an error in the model.”

The Wageningen Research Modelling Group focuses on the quality of computer models at WUR. In the past few years, you have worked on a Wikipedia for Good Modelling Practices?

Meesters: “Everyone knows Wikipedia, the online encyclopaedia that anyone can contribute to. We’ve built a similar environment that modellers can use as an aid when building their own models. The idea with the GMP Wiki is to learn from the methods known from the literature but also from techniques that others at WUR have developed to improve the quality of their models. As such, the various pages are a combination of theoretical knowledge and working methods taken from the literature and tips and tricks from fellow researchers.”

Annevelink: “We don’t want to issue edicts saying ‘Modellers of Wageningen Research & University, thou shalt not do this or that!’ The GMP Wiki is an aid and a source of inspiration. You should consult the GMP Wiki if you’re not quite sure how to tackle the following phase of your model development.”

Meesters: “Our GMP is a guide. You can use it as a manual and run through all the phases of the modelling process, step by step. But you may also have reached a certain stage in the modelling process and wonder whether there might be a method documented in the GMP Wiki that could help you at that stage.”

Annevelink: “Like in Wikipedia, the information can be accessed via multiple routes. The researcher can find the information they need by going to the list of chapters, by entering keywords in the search bar or by clicking the links. The idea is also that anyone can add information to the Wiki — like in Wikipedia. That will make it a dynamic part of the Wageningen modelling community.”

So all computer modellers can add content in the Wiki environment?

Annevelink: “Yes. Visitors to the GMP Wiki can post comments and suggestions for improvements at the bottom of each page. We will then get an alert and deal with the input. Fortunately, we got a budget again last year that lets us take action in response to the comments. We can also ask other colleagues to come up with a solution if necessary. Ultimately, the idea is that frequent users of the GMP Wiki will be able to make their own additions, where they enter their suggestions themselves.”

Does the GMP Wiki also contain the finished computer models?

Meesters: “No, but that’s not necessary. We do plan to link the GMP Wiki to the Model Gallery that we have also created. The Model Gallery stores models developed by WUR. The intention is that all researchers at WUR will be able to use the Gallery, to check it out and see whether there is already a usable model for their idea or research plan. It may not do exactly what they want, but if they then talk to the researcher who built it, they may find the model can be tweaked to make it work for their purposes.”

Annevelink: “It is important for Wageningen researchers to be able to make more use of the models that are already available on campus. You shouldn’t be reinventing the wheel and wasting money and resources if there’s already a usable model for that research question. There are hundreds of models on campus that could be used far more broadly. But one researcher may not realise that another already has a valuable model.”

Meesters: “That means it’s also important to keep the models permanently up to date. Then the model will always be ready at the start of a new project and you don’t have to worry whether it will still work with the latest version of the software. A stable source of funding is crucial for this. Such continuity cannot be taken for granted in our project-oriented organisation.”