Algorithms and Complexity in Mathematics, Epistemology, and Science by Nicolas Fillion & Robert M. Corless & Ilias S. Kotsireas

Author:Nicolas Fillion & Robert M. Corless & Ilias S. Kotsireas
Language: eng
Format: epub
ISBN: 9781493990511
Publisher: Springer New York

2 The Problem of Validation

2.1 Verification and Validation

Any model—if it is to be useful for predicting or controlling its target system—needs both verification and validation. Verification is the process of assessing whether a given model possesses the intended properties. That is, does the actual instrument or mechanism for generating predictions instantiate that which was intended; do the outputs instantiate the intended mapping from inputs? Verification is not a significant concern in the case of analytic models, since there is generally little doubt that a set of equations is in fact the set intended. It becomes critical, however, when numerical approximations are used in extracting predictions or solutions from the equations, and even more pressing in the case of complex computational models. It is not at all obvious that a program correctly implements the numerical integration of a set of equations, or represents the intended set of functional relationships between variables. It is even less clear whether a multi-physics or agent-based model captures the intended set of approximations of law-like interactions amongst constituents. Verification poses a fascinating collection of epistemic problems, and there exist large literatures on model verification in engineering, software development, and mathematics.1 However, for the purposes of this essay, I’ll set aside the problem of verification, and assume that models are correctly implemented.

Validation concerns the accuracy of the model in representing the intended aspects of the target. There are three principal senses in which a model can be accurate. First, it can more or less successfully describe the target system. That is, it can reproduce the known data with varying degrees of fidelity. Second, it can more or less accurately predict passively observed features of the target system at later times, or for different boundary conditions or inputs. And finally, it can more or less accurately predict the behavior of the target system under interventions or alterations of the environment, boundary conditions, or input. The first two tend to be the focus of many standard statistical methods. The novel method described below concerns the third.

There are, of course, many kinds of models with distinct epistemic aims, and the suitability of a particular method of validation will depend on the kind of model under consideration. The method below is appropriate for models in which each variable varies continuously, possibly as a function of other variables in the system. Of particular interest are dynamical systems, i.e., systems that change through time. The bulk of the examples below will deal with systems for which the values of variables change continuously over time.

A brief word on terminology is also important. The term “model” tends to be used in subtly different ways in different disciplines—ecologists tend to use the term differently than machine learning practitioners, and the same group tends to use the term differently depending on context. For clarity, I will use the term as follows: a model is the specification of a class of mappings from input to output.2 The mappings may be via an explicit function or set of equations (as in differential models of thermodynamic phenomena) or via simulations of varying complexity.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.