Not logged in - Login
< back

2014_1216 DW Operations

Date: 2014_1216 2:30pm MST

Time: 1-888-394-8197 and access code 299990

Overview: First bi-weekly (every other week) call to discuss WAQDW operations. Meetings will focus on:

1.) Transfer of files from data providers to the data warehouse

2.) Data requirements and back end systems to support data warehouse visualization tools.

3.) Other topics as needed.

Agenda:

1. Minutes from last meeting 2014_1204 DW Ops

2. DW data holdings and modeling I/O file transfer status 3SDW Data Tracker

  • 2008b plots
  • 2008b 2020a
  • 2011a

3. DW Disk Tracker 3SDW Disk Tracker

4. Projects

  • Current DW projects overview Projects Overview
  • New project/data request interface (Dustin)
  • Ralph's follow up requests for Hiawatha and SRP NGS projects

5. DW model I/O file categories (Dustin)

  • Categories overview categories
  • UNC scripts to generate XML file metadata

6. Open discussion

  • file transfers to support DW visualization products
  • model-to-obs datasets Studies
  • etc.

7. AOB

Wrap up

Attendees: Ralph, Zac, Rodger, Shawn, Dustin

Call duration: 2:30-3:33 MST

Action Items

1. DW file holdings: 3SDW Data Tracker
  • 2008b_Base Case: Dustin will recreate previous /data1/3SDW/3SAQS/2008/Base08b/ directory and file structure from current holdings on viking /data2/3SDW/deliverables/ Base08b Recovery Progress
  • 2008b_2020: 2x4Tb disks at UNC. Zac will mail to CIRA - Completed, mailed 12/17
  • 2008b_Base Case CAMx output: 2x4Tb disks at ENVIRON. Ralph will mail to CIRA
  • 2011a CAMx scripts/code: CAMx Inputs. Zac will copy these files directly to viking - Completed, files located on Viking: /mnt/data2/3SDW/3SAQS/2011/Base11a/AQ/CAMx/software
  • 2011a: Rodger will mail 6x4Tb disks to UNC for transfer of bulk of 2011a platform. - Completed, received by UNC 12/22

2. DW disk tracker: 3SDW Disk Tracker

  • Rodger will update disk tracking table to reflect Project disks currently involved in file transfer and those on deck for future file transfers. - Page updated

3. DW projects:

  • Dustin will roll out updated projects and data request interface. After the roll out, updated data requests for ENVIRON's Hiawatha and SRP NGS projects can be added to the DW project tracking system. -Dustin is on track to roll out updated version this week

4. Modeling I/O categories Categories

  • Dustin will send out list of categories to Zac for input. Dustin will look over UNC's python scripts for generating file metadata and lay out a plan for dovetailing UNC's system with his category based system. -Completed new interface for browsing categories is here http://views.cira.colostate.edu/DEMO1/DataRequest/DataBrowser.aspx

5. Data visualization

  • Rodger will draft a paragraph listing criteria for including observational data sets in the DW Query Wizard tool. -Completed

Air quality monitoring data, including paired observational data and modeling results for the 2008 and 2011 modeling platforms, are available through the Data Warehouse. Observational data sets that can potentially inform modeling studies supported by the DW can be found on the Studies page under Monitoring Data. These data sets have monitoring sites within the 3SAQS 12km modeling domain, and include measurements of gas phase species and particulates that map to corresponding PGM output.

In the meantime, the 2008b and 2011a paired data sets should be filtered from the QW pending final approval of the group. - Completed

  • Shawn will publish the model-to-obs visualization tool on the DW website. This will allow the development group to look at the obs-model output paired data sets for 2008b and 2011a with an eye toward future tool development, and allow for Ralph and others to conduct initial MPE.

  • Dustin will continue development work on his dynamic tool for displaying slices of 3D model output files. visualization tool -Ongoing Recent update published at end of 2014.