Looking under a model’s hood
One of the biggest difficulties for a regulator in assessing the accuracy of groundwater model predictions is that the models themselves are generally not made available and in any case cannot be readily examined by non-expert modellers. Instead, regulators are presented with a report about the model and its outcomes, but these reports often do not accurately convey all of the important information about how the model has been constructed, how key assumptions have been made and what uncertainties remain. It is too easy and too tempting to hide a model’s weak points or a modeller’s uncertainty about how to simulate a groundwater regime by omission in a report if you are confident that the model will not be examined.
This situation is somewhat improved if the model has been independently peer-reviewed as recommended by the Australian Groundwater Modelling Guidelines (Barnett et al, 2013) and their equivalents in other countries, but as a regulator I have been frequently disappointed by the tick-box quality of these peer reviews. I’m aware of numerous instances where professional peer-reviews have been dutifully undertaken in models that later turned out to be very poor for their purpose.
One solution to this impasse is for the model report to be accompanied by a set of exported model layers and other information in a standardised electronic format that would enable the regulator (or any other well-informed stakeholder) to visualise how the model has been constructed and whether key outputs appear sensible. The model layers and information would sometimes need to be tailored depending on the decision(s) that the model is seeking to inform, but it is still possible to nominate a “routine” set of model layers to export as a starting point.
I’m not sure why this hasn’t been attempted as far as I’m aware, but one reason is how much format matters. The format for these export files needs to be sensibly restricted to enable the regulator/ stakeholder to import them into GIS-compatible visualisation software, such as xxx. I have had some experts provide some guidance on formatting requirements and will make that available to anyone interested by email, or maybe in a future post if I can get the advice debadged.
An informed observer can quickly learn a lot from the layers and information set out below. By importing the required layers into visualisation software, one can see for example how aquifer recharge has been spatially interpreted and whether this makes sense in view of the known topography and geology.
The following is a list of my nominations for the “Minimum Model Export” list which I hope one day to get into government regulations – they can of course be used by anyone and can be adjusted for local/regulatory/stakeholder needs.
The list is broken into two tables below; Table 1 contains the metadata and information that can be better communicated outside a graphical environment, whilst Table 2 lists those attributes that are better understood in a spatial context, and are able to be imported into a groundwater visualisation package.
Table 1. Standard model information to be provided in electronic text or spreadsheet filesProperty/ ParameterDescriptionModel ObjectivesA clear statement of the purpose of the model, as discussed in the Australian Modelling Guidelines (Barnett et al, 2012).Model MetadataSingle text file that stores the high level model metadata as a .json object file in accordance with formatting requirements (separately stipulated). Any number of attributes may be specified in this file, e.g. model/run name, created date, author etc.Grid DefinitionMultiple text files exported as .json files in accordance with formatting instructions , describing the 3D grid used by the model and its position in space.Groundwater Pumping/Extraction Rates AppliedExcel table summarising predicted groundwater pumping or other discharge rates applied in modelScenario PropertiesExcel table explaining variables used in each scenario presented in report.ReceptorsExcel table summarising locations and types of key receptors considered in modellingWater BudgetExcel table summarising key volumetric flows (water budget components) across the model space at key time slices*, both within model domain and through external boundaries.
Table 2 presents the layers and parameters which could be routinely exported from the model.
Table 2. Minimum model layers and parameters to be exported from modelProperty ParameterDescriptionStratigraphy3D representation of key stratigraphic layers (with labels based on Geoscience Australia stratigraphic names)Hydraulic PropertiesKey hydraulic properties generated following calibration, specifically vertical and horizontal hydraulic conductivity (Kv & Kh), storativity [s] at key time slices*.Rainfall & EvapotransA map summarising annual rainfall and evapostranspiration rates if areally varied.G’water RechargeMap layer showing inferred recharge rates and spatio-temporal variability (if any applied) for shallowest aquifer for each main time step*.Borehole DataStratigraphic or lithological logs and temporal water level measurements of key monitoring wells (see Tab C for format requirements).Structural GeologyMajor faults, shear zones or joint sets incorporated into model (if any applied).TopographySurface topography (presented as a digital elevation model)Boundary ConditionsGraphical representation of the boundary conditions applied to each edge of the model.Surface Water FeaturesStream/lake bottom elevations and heads, streambed conductance.Surface Water InteractionFluxes at base of stream/lake/drain features at key time slices*.ReceptorsMap layer showing location of main receptors, with legend denoting receptor types.Predicted HeadsSeparate contoured layers showing groundwater (and surface water if relevant) heads at pre-mining and current conditions, and predicted at key time slices*.CertaintyLevel of model certainty/confidence associated with spatial data. As described in Section 8.5.7 and Figure 8-6 of the Australian Groundwater Modelling Guidelines (Barnett et al, 2012), one option is to present certainty as a colour density or by varying the transparency of a layer to indicate the level of uncertainty.
Where the parameters are requested in terms of key time slices, the default set of events for which those parameter should be provided are as follows:
Predicted conditions at commencement of the activity
Predicted conditions at “peak impact” for that parameter
Predicted conditions at completion of the activity
Predicted conditions once steady-state water conditions have returned following project closure.