Models Need Scenarios
I was reading about a Covid-19 model being ‘rejected’ because it didn’t ‘give the right answer’.
First thought: no models ‘give the right answer’. But they DO give some clues as to possible futures and tend to rule out impossible ones.
Then, I read that this model had been set up with a number of possible Scenarios. Each of them showing what would happen if you did different things. The model was there to guide the best decisions.
Models do not just predict one future. They allow you to make choices, change assumptions and so predict different futures. The different scenarios that you imagine are as important as the models themselves.
So, scenarios are as important as the Model. Choosing a Scenario sets your course. The Model shows where you end up.
When should you reject a model? When you have a better one! In general, your ‘gut feeling’ is not a better one, although it is a natural human tendency to believe so. That ‘gut feeling’ is really nothing more than an unconscious belief. The model also represents beliefs. But they are made visible. You can challenge something visible in a way that you cannot challenge something unconscious. To reject a model, you must say what assumptions are wrong. Just to say you ‘don’t believe it’ is irrational and gets you nowhere.
You might also say that you have not suggested the best Scenarios. That’s valid. But only if you can suggest others that make sense. Then you can use the model to see where it goes. Or maybe force you to modify the model. But not to reject it.