The 3SAQS project performed photochemical grid modeling (PGM) for the year 2008 using the Comprehensive Air Quality Model with Extensions (CAMx) version 5.41. The 3SAQS 2008 Modeling Protocol details the CAMx configuration and justification for why it was chosen for the 3SAQS. This document presents the CAMx 2008 model performance evaluation (MPE) for the 3SAQS 2008 base year simulation version B (CAMx_3SAQS_Base08b). We conducted the MPE for ozone, fine particulate matter (PM2.5), and wet deposition species. We evaluated the performance of hourly ozone as well as daily maximum 1-hour (MDA1) and daily maximum 8-hour average (MDA8) ozone. In addition to ozone, we also included carbon monoxide (CO), nitrogen oxides (NOx), and sulfur dioxide (SO2) gas-phase species in the evaluation. We did not include volatile organic compounds (VOC) or ammonia (NH3) in the evaluation because observational data for these species were either not readily available or sparse in the Three-State Region. The PM2.5 evaluation includes total PM2.5 along with the component species sulfate (SO4), nitrate (NO3), ammonium (NH4), elemental carbon (EC), organic carbon (OC), and other PM (PM Other). The deposition evaluation focused on total sulfur and oxidized and reduced nitrogen species. We did not include dry deposition species or visibility metrics in the MPE.
We evaluated the 3SAQS 2008 base case model performance against concurrent measured ambient concentrations using graphical displays of model performance and statistical model performance measures. We compared these measures against established model performance goals and criteria, following the procedures recommended in EPA’s photochemical modeling guidance documents. The evaluation included sub-regional evaluations for Colorado, Utah, and Wyoming, and evaluations by month and season.
The 3SAQS 12-km domain-wide ozone model performance meets the performance goals for both MDA1 and MDA8. CAMx has a positive bias for ozone at both the AQS and CASTNet networks; with higher fractional bias at the AQS sites (1-hour: 7.8%, MDA8: 12.6%) compared to the CASTNet sites (1-hour: 4.15%, MDA8: 6.5%). On days with elevated O3 measurements (> 60 ppb), CAMx has a negative bias, with lower fractional bias at the AQS sites (1-hour: -8.8%, MDA8: -7.4%) compared to the CASTNET sites (1-hour: -11.7%, MDA8: -11.0%).
The CAMx O3 estimates for Colorado meet the bias (NMB: 6.0-14.2%) and error (NME: 12.0-30.3%) performance goals on an annual basis for hourly O3, MDA1, and MDA8. The CAMx biases for Colorado are lowest during the spring and summer months at the AQS sites and during the winter and spring at the CASTNet sites. The simple O3 performance bias goal of ±15% is exceeded in September-December for the AQS sites and in September for the CASTNet sites. In general, CAMx performance at the Colorado AQS monitors is better during periods of elevated ozone.
The CAMx O3 estimates for Utah are within the bias (NMB: 0.2-17.6%) and error (NME: 9.0-30.3%) performance goals on an annual basis for hourly O3, MDA1, and MDA8. Like at the Colorado sites, the CAMx biases are lowest during the spring and summer months at the Utah AQS sites and during the winter and spring at the CASTNet sites. The simple O3 performance bias goal of ±15% is exceeded at the AQS sites in January, October, and December; the criteria bias of ±35% is exceeded in November. CAMx has a low bias (NMB and FB
The CAMx O3 estimates for Wyoming are within the bias (NMB: 0.26-11.4%) and error (NME: 10.8-22.8%) performance goals on an annual basis for hourly O3, MDA1, and MDA8. The CAMx biases are lowest during the summer months at the Wyoming AQS sites and during the winter and summer at the CASTNet sites. The simple MDA8 performance bias goal of ±15% is only exceeded February at the AQS sites; CAMx meets the performance goals at the Wyoming CASTNet sites in every month. The CAMx MDA8 underestimates at the Boulder site, along with the two other Sublette County sites (Jonah and Daniel South) in February, indicate winter O3 formation related to oil and gas operations. The 3SAQS Base 2008 CAMx modeling platform is not configured to simulate the conditions that lead to winter O3 formation. Additional work on the components of the modeling platform is required to reproduce the radiative, dynamic, and emissions that to lead to high wintertime O3 concentrations.
Modeled PM2.5 meets or exceeds the performance goal in the summer months over the 12-km domain relative to IMPROVE and CSN, but falls outside the criteria at the IMPROVE sites in the winter. The performance is within the criteria in UT at CSN sites in all months. In general, the urban (CSN) sites, where PM concentrations are larger, show better performance than in the remote sites in the 12-km domain, and in CO and UT. Spatially across the 12-km modeling domain CAMx is overbiased in the western part of the domain and the four corners states in the winter and spring, and in most of these regions in the fall. The model tends to be underbiased in summer in some of these same regions, as well as in the upper Midwest. On a seasonal basis, CAMx significantly overpredicts observations from both the IMPROVE and CSN networks across the 12-km domain, in all seasons except for the summer. The overprediction is greatest in the winter at the IMPROVE sites, and much less in the summer at the CSN sites.
Domain-wide, sulfate wet deposition model performance is outside the performance criteria for all but the spring months, and moderately overbiased. In CO, the fall months show an underbias in SO4, but are within the performance criteria; the fall and winter months are overbiased, with the latter being beyond the plot range. The errors are smaller in WY and UT; overall SO4 performance in UT appears to be the best of the three states. Nitrate deposition performance for the 12-km domain shows an underbias in most of the year, although the bias and error values fall within the performance criteria except in December. The performance for NH4 wet deposition shows the model being underbiased in several months in the 12-km domain as a whole, and in the three states although the performance in UT is within the acceptable range in the spring and fall. Exceptions to the underbias are the late fall and early winter in the 12-km domain, and May, June and October in CO, wherein the overbias falling outside the plot range. WY also shows sporadic occurrences of NME outside the performance criteria.
CAMx 2008b Final Model Performance Evaluation Report