DW Utilities

This page details the installation and configuration of various data and user management utilities installed on the warehouse file servers. All of these utilities are stored in a git repositories hosted on bitbucket. All of these repositories belong to a group project on bitbucket called IWDW. The IWDW landing page on bitbucket is here: https://bitbucket.org/iwdw/

Each repository may have a specific wiki hosted on bitbucket. This page provides a description of each system and the steps to install and configure them.

Configure Utilities Directory

Linux IP Blocker

This system consists of a script and cron job which scans a security log for repeated login attempts and blocks the source ip address in iptables

Setup

Clone git repository

cd /var/DWUtils/
git clone git@bitbucket.org:iwdw/linuxipblocker.git
mv ./linuxipblocker/ IPBlocker

Activate cron job

0 * * * * /var/DWUtils/IPBlocker/blockIPs.py

File System Scanning

cd /var/DWUtils/
git clone git@bitbucket.org:iwdw/FileSystemScanner.git
cd FileSystemScanner
mkdir pyFSScan
cd pyFSScan
git clone git@bitbucket.org:iwdw/pyFSScan.git

Cron Overview

0 19 * * * /var/DWUtils/FileSystemScan/tarCheck.sh
0 22 * * * /var/DWUtils/FileSystemScan/doScan.py
0 23 * * * /var/DWUtils/FileSystemScan/doIngest.sh
0 * * * * /var/DWUtils/IPBlocker/blockIPs.py
@reboot /var/DWUtils/FTPAccount/mount_on_boot.sh