Not logged in - Login
< back

File System Scanner

The file system scanner is a collection of scripts hosted on viking which are run on a daily basis to update the 3SDW relational database with the current state of viking's file system.

  1. Scan the file system to a file
    1. /var/DWUtils/FileSystemScanner/doScan.py
      This is a top level script which executes scans of several directory trees. This script needs to be parameterized. The main function of this script is to connect specific directory trees to be scanned to instances of ScannerToFile described below.
    2. /var/DWUtils/FilesystemScanner/pyFSScan/fsScan.py
      This file contains numerous classes to handle the scanning of directory tree contents into various forms. The specific class used is ScannerToFile. Methods of the scanner to file class are called in the doScan.py script. The results of the file system scan are stored in a binary file whose form is defined within the ScannerToFile methods.
  2. Process the file to the database
    1. /var/DWUtils/FilesystemScanner/doIngest.sh
      This shell script processes the binary scan result files generated in the last step to web services. This shell script contains arguments given to buildHistoryWeb.py
    2. /var/DWUtils/FilesystemScanner/pyFSScan/buildHistoryWeb.py
      This file parses a file system scan result binary file and publishes the contents to the TSDW web services.
    3. /var/DWUtils/FileSystemScanner/TSDWConnection.py
      This file contains a class which represents the connection to TSDW web services.
    4. \TSDW\TSDWAPP\DataRequest\DataCatalog.asmx.vb
      StoredThis procsis calledthe server side code for the web services. There are many web services methods contained within this file but only a few are used for file system scanning. The services utilized in this case are all mapped directly to sql server stored procedures with the same name. Listed below are a description of the web services / stored procedures.
      1. SetfileSystemScanTime
      2. StartFileSystemScan
      3. CreateOrUpdateFileItems
      4. CompleteFileSystemScan
      5. UpdateFileSystemHIDs
      6. UpdateFileSystemTreeTotals