Not logged in - Login

2014_1204 DW Ops

2014_1204 8:30am Check in on DW ops (Tom, Shawn, Dustin, Rodger)

Future meetings:

  • Set up a bi-weekly (every other week) call to discuss WAQDW data management. Meeting will focus on 1.) Transfer of files from data providers to the data warehouse and their physical organization on Viking's file structure; 2.) Data requirements and backend file and db organization to support data warehouse visualization tools. 3. Other topics as needed. Bring in Ralph and Zac and find time slot for meeting. Rodger will follow up.

  • Set up recurring meeting developer meeting with Shawn, Dustin, Rodger. Shawn will follow up.

Website modifications:

  • Shawn has been working on migrating demo web pages into main TSDW website. He has developed a db table to serve the current static HTML documents page. He has developed 3 master pages for three generic page types; with sidebar, without sidebar, and wide format. Shawn is also looking into a .NET forum “yet another forum”. Plans to migrate new pages into production site by the end of the week.
  • Shawn proposed modifications to the data use agreement, for example, pulling out key points such as “the data user agrees to provide attribution”, and highlighting these on the data request.
  • Tom requested need for one click access to 2008b and 2011a modeling platforms. Follow up in future meeting.

Data request trial run:

  • Dustin worked with Tammy on a research based data request. She was able to submit the initial request, then add an additional request for new data.
  • Discussion on developing a dynamic version of Zac’s 3SDW data tracker successbox:(http://vibe.cira.colostate.edu/wiki/wiki/1033/3sdw-data-tracker) based on the data manifest tool currently integrated into the data request form. Follow up in future meeting.

Action items:

  • Set up meeting time for bi-weekly DW ops call. Rodger
    1st meeting 12/16/14 at 2:30pm MST, 2nd meeting 1/6/15 then every other Tuesday.

  • Publish website modifications for revised docs, meetings and home pages to production server. Shawn
    Shawn published updated pages and forum feature on 12/5

  • Rebuild 2008b modeling platform on Viking based on original /data1/ directory structure. Dustin will generate and run file copy script.
  • Resolve issue of duplicate WestJump PGM output. Work with Mike Barna to create external disk copies of his “updated” PGM output. Rodger
    Mike estimates he needs 8TB for offline storage. Disks are in house, RA will coordinate file copy with MB

  • Create delivery tracking spread sheet to transfer of WAQDW external disks. The spread sheet could be expanded to indicate future storage requirements and hardware cost projections for DW operations. Rodger
    prototype page 3SDW Disk Tracker

  • purchase additional disks for 2011 platform file transfer from UNC to DW. Rodger
    10x4TB disks in house; RA will send 6 blank disks to UNC for 2011 platform