71st SWCS International Annual Conference July 24-27, 2016 Louisville, KY

1. Lessons learned from applying APEX
to field‐sized agricultural watersheds
underlain by a restrictive layer
SWCS Annual Meeting – July 24‐27, 2016 – Louisville, KY

2. Regional P Index Assessment Project Team
•Modeling team
 Claire Baffaut, USDA‐ARS, Univ. of Missouri
 Nathan Nelson, Kansas State Univ.
 John Lory, Univ. of Missouri
 Anomaa Senaviratne
 Ammar Bhandari
 Mike Van Liew, Univ. of Nebraska
• Runoff study managers
 Matt Helmers, Iowa State Univ.
 Ranjith Uddawatta, Univ. of
Missouri
 Dan Sweeney, Kansas State Univ.

3. Study Objectives
 Determine if APEX can be used to evaluate and
improve P Indices
• Step 1 ‐ Determine the capacity of various parameterization
strategies to successfully replicate water quality data.
• Step 2 ‐ Assess the impact of calibration strategy on the
capacity of APEX to evaluate management systems.

4. What parameterization strategy is required to
model P loss?
Best Professional Judgement – no calibration,
parameters selected based on literature survey and best‐
professional judgement
Full calibration – parameters selected on a site‐specific
calibration/validation procedure. Options were adjusted
in control file and apex parm file.
Regional calibration – single parameter set for sites
across a broad region. Multi‐site calibration/validation.

5. Event‐based Model Assessment
(Calibration/Validation)
Water Quality
Parameter
Coefficient of
determination (r2)
Percent bias
(p‐bias)
Nash‐Sutcliffe Eff.
(NSE)
Runoff > 0.5 ± 35% > 0.3
Sediment Loss > 0.5 ± 60% > 0.3
Phosphorus Loss > 0.5 ± 60% > 0.3
Comparison of observed vs. simulated data

6. Regional Model Datasets
Size: 0.4 – 5 ha.
Crops
• Corn/Soybean/Sorghum
Tillage
• No‐till/Reduced till
Fertility
• Fertilizer
• Poultry litter
Structures
• Grass waterways
• Buffers
• Terraces
Soils with a
restrictive layer
at all sites.
Kansas

7. Development of Regional Calibration
 Compiled a list of all options/parameters that were
used in calibration (sensitive)
• 3 options in the control file
• 21 parameters in the parm file
 Identified the “optimal” parameter set

8. Identifying the “optimal” parameter set
 Control file options
• Most common value
 Parameter file
• Additional sensitivity analysis
• Equal values among most models and non‐sensitive for the
others  select that value.
• All values close to each other and low sensitivity within that
range  select average
• Different values and high sensitivity  9 parameters
selected for regional calibration

9. Regional Calibration Process
 Multi‐site Calibration: identify sets of good values for
the 9 parameters
 Identify a parameter set that met performance criteria
for most of the data sets  4 parameters
 Manual calibration using all data sets together and
selecting parameter values that led to best
performance values.

10. Lessons Learned…

11. 1. Accurate parameterization of APEX requires a
high level of training.
 Users must be informed about many model options
 Some published parameter ranges may not apply to
the specific locations
 “blind” calibration can lead to un‐realistic parameter
sets or incorrect model options.
• Example: selecting the correct method for determining peak
rainfall rate (ITYP)

12. 2. Review models results thoroughly at each step
of a study
 APEX is an actively developing model
• We have fixed or updated model code several times:
− Improved the method for tracking water balance in the watershed.
− Improved the method for simulating flood‐plain infiltration
− Improved the methods for simulating P leaching in the model.
• We have gone through 8 modifications of our code

13. 3. Best Professional Judgement:
unacceptable water quality estimates.

14. PBIAS r2 NSE
Runoff Total Phosphorus Sediment
Full symbol: meets performance criteria
Hollow symbol: does not meet criteria
BPJ distributions of performance indicators

15. 4. Full calibration excellent for runoff and P loss,
marginal for sediment.
 Data sets used for calibration need to include useful
information.
• Do not use a data set from a no‐till system with filter strips to
calibrate sediment loss processes.

16. 5. Include data from multiple management
systems for model calibration.
P/F
Runoff Total Phosphorus
r2 NSE PBIAS r2 NSE PBIAS
Calibration with one single management system
P 0.81 0.76 22 0.68 0.60 35
P 0.63 0.55 24 0.41 0.37 21
F 0.77 0.45 49 0.64 ‐0.17 92
F 0.62 0.18 ‐52 0.36 ‐0.56 89
F 0.66 0.38 ‐24 0.50 ‐0.99 ‐53
F 0.70 0.52 ‐13 0.51 0.13 ‐7
Calibration across several management systems
F 0.70 0.63 4 0.57 0.07 ‐8
P 0.59 0.38 43 0.47 0.30 49
P 0.82 0.70 30 0.71 0.68 28
Results from Franklin County site

17. 6. Regional parameter set excellent for runoff,
marginal for P loads at each site but excellent over
the region. Unacceptable for sediment.
Validation Validation

18. 7. BPJ not consistent with fully calibrated
models for effectiveness of some BMP.
Runoff Relative Effect
(%)
Total Phosphorus Relative
Effect (%)
Scenario LC ALT SITE BPJ LC ALT SITE BPJ
GWW ‐3.5 ‐2.8 ‐2.4 ‐7.2 ‐10.1 ‐14.3
WCC ‐9.7 ‐18.9 ‐2.1 ‐17.1 ‐19.4 ‐1.9
WCC&GWW ‐12.8 ‐20.7 ‐4.4 ‐23.4 ‐28.1 ‐16.0
TR&GWW 1.4 1.1 ‐0.6 ‐50.3 ‐48.9 ‐51.7
All three ‐8.1 ‐17.3 ‐2.5 ‐58.9 ‐59.0 ‐52.6
LC: local calibration, ALT SITE: calibration using alternative site.

19. Overall outcome: Success with regional calibration
presents opportunities to use APEX to generate P
loss values beyond the calibrated watersheds
 This study was limited to:
• Soils with a restrictive layer.
• Low sediment loss management systems.