Smart Server Defense
Powered by BlueIce3
Errata
Contents
Root Mail - Large File Size   top
Depending on your scheduling for the small_interval_blueice3 cron job and your local system setup the file /var/mail/root can get quite large over time. To remedy this we recommend adding a cycle script into your cron.weekly folder. From the blueice2 home directory 0_4_0_9 run the following commands.
Version 0_5_0_9 will include the proper adjustments in the install script and by default in the AMI.
wget http://middlemind.net/scratch/cycle_root_mail.txt
mv cycle_root_mail.txt cycle_root_mail
sudo chmod +x ./cycle_root_mail
sudo ln -s /home/ubuntu/blueice3/0_4_0_9/cycle_root_mail /etc/cron.weekly/cycle_root_mail
This will ensure that the root mail file isn't much larger than 64MB.
Automated apt-get Error   top
If your apt-get updates haven't been working in a little while. Check to see if this error message is present in your system logs by running the commands below. If so, no worries just run the update commands by hand an reboot the server. The full error message is as such
dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem
You may also see errors similar to the following in you /var/mail/root file, If your apt-get updates haven't been going through in a while check the root mail file for messages such as the following.
Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
These errors are usually caused by apt-get updates that require a user's interaction.
cd /var/log
grep "dpkg was interrupted" -R ./
cd /var/mail
grep "Could not get lock" -R ./
If you find the message in the server logs and/or root mail file and suspect there is an issue run the following commands and reboot the server.
sudo dpkg --configure -a
sudo shutdown -r now
Tensorflow Error Switching To/From NoUrl   top
If you've been running out of the box with the '_nourl' option set in your 'small_interval_blueice3' file and you've switched to a configuration that supports URLs, or vice-versa you'll get an error in your T ensor Flow execution. Check the 'small_interval_blueice3_lastrun', or the root mail file for the results of the cron execution. This is most likely because the checkpoints are based on the previous settings from the previous run. To fix the problem clear out the checkpoints folder using the commands below from the blueice3 home directory. Command #32 is only available in the newer software suite so you may need to run the delete commands by hand if you're using version 0.4.0.9.
[full path]/blueice3_install -i -int -apkg -j 32
or...
cd [full path]/apps/python/BlueIce2
rm ./checkpoints/*
rm ./checkpoints/*.*
Message of the Day Error   top
Version 0.4.0.9 AWS AMIs have a typo in the message of the day that lists the version as '0404' instead of '0409'.
BlueIce2 Utils v0.4.0.9 Bug   top
The job command 'processApacheLogFile' is broken in utility version 0.4.0.9, use the direct BlueIce2 CLI call. Alternatively you can adjust the method code by pasting the following code in replace of the version 0.4.0.9 method.
def processApacheLogFile(targetFile, dbValidWebFiles=False, dbTrainingFiles=False, otherArgs=None):
    if os.path.exists(targetFile):
        lp("Processing apache log file entry: %s" % (targetFile,))
        info = os.stat(targetFile)
        filePath = targetFile
        fileName = os.path.basename(targetFile)
        size = info.st_size
        # lastModified = datetime.datetime.fromtimestamp(info.st_mtime)

        data = getApacheLogFileByFileNameFilePath(filePath, fileName)
        print(data)

        csize = data[0][1]
        lastLogRow = str(data[0][3])

        if(size < csize):
            lastLogRow = 0
        # eif

        print("")
        print("")


        args = ""
        if(dbValidWebFiles):
            args += " -dbValidWebFiles"
        # eif

        if(dbTrainingFiles):
            args += " -dbTrainingFiles"
        # eif

        print("Setting working directory: " + cfgs["exeDir"])
        os.chdir(cfgs["exeDir"])

        if(not otherArgs is None and not otherArgs.strip() == ""):
            cmd = cfgs['blueice2exe'] + args + " -lastLogRow=" + str(lastLogRow) + " -apacheLogFile=" + targetFile + " " + otherArgs
        else:
            cmd = cfgs['blueice2exe'] + args + " -lastLogRow=" + str(lastLogRow) + " -apacheLogFile=" + targetFile
        # eif

        print("Running command: '" + cmd + "'");
        os.system(cmd)
    else:
        lp("Target file does not exist: %s" % (targetFile,))
    # eif
# edef    
                                
Operating System Hooks   top
This section lists all the operating system connection points for both the AWS AMI version and the stand alone version of blueice3.
  • $home/.profile: The profile aliases included after running blueice3_install command 22. Similarly .profile.bak_bi3 is a backup of the original .profile file.

    alias updatebi="/home/ubuntu/blueice3/0_4_0_9/blueice3_install -u -int -apkg"
    alias bih="cd /home/ubuntu/blueice3/0_4_0_9"
    alias biha="cd /home/ubuntu/blueice3/0_4_0_9/apps/"
    alias bihap="cd /home/ubuntu/blueice3/0_4_0_9/apps/python"
    alias whatami="echo bi-3-drone-master"
    alias updatebir="bih; updatebi;"
    /home/ubuntu/blueice3/0_4_0_9/blueice3_install -q > .bi3_login_tmp 2 > .bi3_login_tmp


  • $home/.bi3_login_tmp: The output of the process that makes sure the hostname is properly set for the system. Has more of a purpose on AWS AMIs where this causes an error message when running sudo.

  • $home/.bi3-mysql-init-complete: The result from running the mysql user account initialization found in the AWS AMI release. The initialization service, found in [full path]\apps\scripts\SmartServerDefenseCfg\blueice3_mysql_prep.service, the service initializes custom mysql db user account information based on blueice3_install commands 26 and 30.

  • /etc/crontab: There are two entries placed in the crontab file for running the short term and longer term scheduled blueice3 tasks, blueice3_install command 27.

  • /etc/cron.weekly: A symbolic link to the [full path]\cycle_root_mail script for re-cycling the root mail file if it gets too large.

Securing BlueIce3   top
General steps to increase the security for blueice3 installation is to make sure your server uses a non-standard SSHD port. We also recommend setting tighter permissions on the 'blueice3' filesystem. You can try running things with root ownership and/or with 644 permissions.
Fixing CORS Errors   top
If you run into a client side javascript error similar to the following, "The 'Access-Control-Allow-Origin' header contains the invalid value," followed by a URL you may need to setup CORS on your apache instance to allow access to the web service tier via javascript. To do so run the following command on your server.
sudo a2enmod headers
sudo service apache2 restart
Next you'll want to add in a 'Header' command into the virtual host entry for your BlueIce3 website. You should use the URL listed in the client side javascript error to set the CORS value. See below.
Header always add Access-Control-Allow-Origin "http://yoururl.com"
Don't forget to restart your server one more time after adding the configuration change.
sudo service apache2 restart
Copyright © 2018    Middlemind LLC.    Victor G. Brusca