Data Request Procedure

Overview

This page is intended to document the procedure for requesting data from the Western Air Quality Data Warehouse.

The basic procedure is defined as follows:

1. Register for a user account
2. Access the Data Request Form
3. Provide Basic Parameters
4. Provide Details regarding intended use
5. Select Data
6. Submit Request. Request submission does not need to happen right away. Your progress is saved and you can return to a given request at any time.
7. Monitor Request Status

Data Request Form

The Data Request form can be accessed at the following URL: http://views.cira.colostate.edu/DEMO1/DataRequest/DataRequest.aspx

The Request form consists of:

1. A status overview showing the current status of the request.
2. A number of actions available based on the current status of the request and the role of the user currently viewing the request. Actions affect the status of the request. Hovering your mouse over an action button will provide a definition of that action.
3. A number of collapsible sections containing various input fields and information. Each section is described in detail within this document.
   

   Data Request Form

Basic Parameters

The Basic Parameters input section consists of the following fields

1. Project: Projects are detailed in another section of this page. This field shows any projects that you have been assigned to. If this does not mean anything to you simply leave the default option of "Unspecified. If you are participating in a known project and wish to issue a request within the context of that project you should select the appropriate project here.
2. Requestor: You should see your name and organization listed in this field. Nothing to do here.
3. Data Use Agreement: You should read the data use agreement. This document describes the conditions of using any data you retrieve from the warehouse.
4. "I Agree" Checkbox: Checking this box signifies that you agree with the data use agreement. You must explicitly check this box before you submit your request so you should save this until immediately before submitting your request.

Request Details

Data Selection

The Data Selection section is divided into three regions. On the left is a tree showing a hierarchical view of data available to be requested. On the right is a table showing the data elements currently attached to the request. At the bottom is an Information section which displays additional info about nodes in the tree.

Data can be added to or removed from the request by checking or un-checking tree nodes. Data can also be removed by clicking the "Remove" links in the table on the right. The header at the top of the section indicates the volume in bytes and number of files for data currently attached to the request.

The divider between the Tree and Table can be moved by clicking and dragging on the light grey sections of the divider bar. The sections can be toggled by clicking the dark grey section in the center of the divider bar.

Data Browser Tree

The data housed in the warehouse is arranged in a categorical hierarchy like a computer file system.

Clicking the + or - icon on a tree node will expand that node to show sub categories.

Clicking on the icon for a tree node will display additional information about that node in the Information section below the tree, including the data volume for that category.

Clicking the check box on a node will add or remove that category and its subcategories from the request.

Clicking on underlined text of a node will open another web page with more details about the data within that node.

Data Availability

Some nodes within the Data Browser Tree or rows in the Data Selection table may display red text which says UNAVAILABLE. There are many reasons this may occur. Unavailable data may still be requested but it cannot be delivered until it is made available.

Data Selection Table

The table at the right side of the data selection section shows an overview of data attached to the current request. The table is divided into sections by Category. Categories are nodes in the tree and contain files and directories on the warehouse file system.

The Data selection table allows removing data from the request.

• Clicking the Remove link next to a category (A grey section header) will remove all files within that category from the request.
• Clicking the Remove link next to a file or directory will remove only that file or directory from the request.
• Clicking the Remove All link in the upper left will remove all data from the request.

The data selection table only shows an overview of the data attached to the request; it does not show every individual file. To get a detailed, file level listing of selected data click the Download Detailed Manifest button.

Download Detailed Manifest

The Data Selection Table only displays an overview of the data attached to a given request. Clicking the Download Detailed Manifest button will download a csv file listing every individual file attached to the request.

Download Transfer Script

This option is only available to Administrators. Clicking this button will download a shell script consisting of copy statements to allow warehouse administrators to transfer the requested data to a disk local to the warehouse.

Download WGET Script

This option is only available to Administrators. Clicking this button will download a shell script consisting of wget statements to allow a remote user to download the requested data via FTP. If the requester wishes to receive data via FTP warehouse administrators will use this button to generate the script and deliver the script to the requester. Note to administrators: anonymous FTP is currently disabled on the warehouse server.

Data Request Lifecycle

From the time a data request is started through to its completion it will transition through a number of relevant statuses. Also the responsibility for handling the request step will change hands. The current status of a Data Request is shown near the top of the Data Request form. The following diagram shows all of the statuses which a request may assume as well as who is responsible for setting that status. For example a Data Requestor is responsible for Staring a request (Status Started) as well as submitting it (Status Submitted).

Actions

Near the top of the Data Request Form is a section titled Actions. This section contains action buttons which modify the request status and transition the responsibility for the request. The actions that are visible depend upon the role of the person viewing the page, as well as the current status of the request. The following table defines the various request statuses and the actions which trigger them.

Request Status and History

The following table describes the life cycle of a data request in terms of the request status. Each status has a definition and an indicator of which user roles can set a request to that status

Status	Definition	Requestor	Project Manager	Admin
Started	The Data Request has been initiated on the TSDW website by a user who has completed one or more fields on the request form, but the request has not yet been submitted. A “started” (unsubmitted) request can be returned to and built over time by the user until he/she decides to actually submit it.	x	x	x
Submitted	A user has clicked the “Submit” button on a completed request. The request can now appear in any lists of submitted requests that are shown to administrators and project managers on the website.			x
Approved	The request has been officially approved. We have yet to formally define the approval process.			x
Disapproved	The request has been explicitly denied. We probably need a “Notes” or “Comments” field (or multiple records in a separate, linked “DataRequestNotes” table) for specifying the reasons for disapproval. As one possible example, let’s say that the request is denied because the user didn’t sufficiently and/or properly fill out all the required fields. In that case, the request could be potentially submitted again after the user corrects/completes his entries.			x
Filling	The request is being filled. Right now, and for NEPA requests in particular, this takes the form of someone copying the desired files onto a hard disk. In the future, this may be expanded to mean that the Wget script for downloading the data has yet to be generated.			x
Filled	The request has been completely filled and is pending delivery. For hard disk fulfillment, this means that all the necessary files have been copied to set of hard disks and that the disks are awaiting packing/shipping/delivery. For Wget fulfillment, this means that the appropriate Wget scripts have been generated.			x
Data Delivered	The requested data has been sent to the requesting user (i.e. either the hard disks have been shipped or the Wget script has been sent)			x
Data Received	The requesting user has acknowledged that the data has been received.	x	x	x
Data Incomplete	The requesting user has indicated that there is an issue with the received data (it’s incomplete, unreadable, erroneous, etc.) If this is the case, the request would revert to a previous status that’s relevant to the reason for which it has been flagged. For instance, if the disk was unreadable by the user, the request might simply revert to a status of “Filling” while we recopy the data onto a new disk.	x	x	x
Data Accepted	The data has been explicitly “signed off on” by the user.	x	x	x
Complete	The life cycle of the Data Request is over and the request is considered historical			x

Finding Your Data Requests

To find a Data Request that you have navigated away from log in to the warehouse and click the "Your Account" link in the upper right. Your profile page will display a listing of your requests.

Projects

Roles

Registered User

Any user registered for the site

• View (Read-only) all projects
• View (Read-only) all Data Requests

Project Collaborator / Team Member

A Registered User who has been added to a project by the Project Manager

• View (read-only) all projects
• Issue data requests within a Project they have been assigned to.
• Edit Data Requests they have created
• Submit Data requests they have created
• Sign-off Data requests they have created

Project Manager

A Registered user who is also a member of a 3SDW cooperating agency (BLM, EPA, NPS, etc). Project Managers regulate access to warehouse data and monitor the data transfer progress. Project Managers may also submit Data Requests

• Create new projects
• Edit projects they manage
• View (read-only) all projects
• Issue data requests within a project they manage
• Assign registered users to a project they manage

Administrator

Administrators are warehouse staff who are responsible for monitoring and fulfilling data requests.

• Create new projects
• Edit any project
• Edit any data request