Not logged in - Login
< back

2014_1216 DW Operations

Date: 2014_1216 2:30pm MST

Time: 1-888-394-8197 and access code 299990

Overview: First bi-weekly (every other week) call to discuss WAQDW operations. Meetings will focus on:

1.) Transfer of files from data providers to the data warehouse

2.) Data requirements and back end systems to support data warehouse visualization tools.

3.) Other topics as needed.

Agenda:

1. Minutes from last meeting 2014_1204 DW Ops

2. DW data holdings and modeling I/O file transfer status 3SDW Data Tracker

  • 2008b plots
  • 2008b 2020a
  • 2011a

3. DW Disk Tracker 3SDW Disk Tracker

4. Projects

  • Current DW projects overview Projects Overview
  • New project/data request interface (Dustin)
  • Ralph's follow up requests for Hiawatha and SRP NGS projects

5. DW model I/O file categories (Dustin)

  • Categories overview categories
  • UNC scripts to generate XML file metadata

6. Open discussion

  • file transfers to support DW visualization products
  • model-to-obs datasets Studies
  • etc.

7. AOB

Action Items

1. DW file holdings: 3SDW Data Tracker

  • 2008b_Base Case: Dustin will recreate previous /data1/3SDW/3SAQS/2008/Base08b/ directory and file structure from current holdings on viking /data2/3SDW/deliverables/
  • 2008b_2020: 2x4Tb disks at UNC. Zac will mail to CIRA
  • 2008b_Base Case CAMx output: 2x4Tb disks at ENVIRON. Ralph will mail to CIRA
  • 2011a CAMx scripts/code: CAMx Inputs. Zac will copy these files directly to viking
  • 2011a: Rodger will mail 6x4Tb disks to UNC for transfer of bulk of 2011a platform.

2. DW disk tracker: 3SDW Disk Tracker

  • Rodger will update disk tracking table to reflect Project disks currently involved in file transfer and those on deck for future file transfers.

3. DW projects:

  • Dustin will roll out updated projects and data request interface. After the roll out, updated data requests for ENVIRON's Hiawatha and SRP NGS projects can be added to the DW project tracking system.

4. Modeling I/O categories Categories

  • Dustin will send out list of categories to Zac for input. Dustin will look over UNC's python scripts for generating file metadata and lay out a plan for dovetailing UNC's system with his category based system.

4. Data visualization

  • Rodger will draft a paragraph listing criteria for including observational data sets in the DW Query Wizard tool. In the meantime, the 2008b and 2011a paired data sets should be filtered from the QW pending final approval of the group.

  • Shawn will publish the model-to-obs visualization tool on the DW website. This will allow the development group to look at the paired data sets for 2008b and 2011a and enable initial MPE.

  • Dustin will continue development work on his dynamic tool for displaying slices of 3D model output files. visualization tool