F5 BIG-IP v10.2 ASM Policy Change Monitoring

1. Monitor ASM polices on F5 BIG-IP v10.2 for changes
2. Add ASM policy files to CVS or SVN repositories
3. Alert appropriate staff
**Tools & References**
I, and quite possibly you, already use RANCID for configuration change monitoring of your network equipment. It already supports F5’s so this is a natural progression.
**Dumping ASM Policies to Clear Text**
ASM configuration and policies are stored in policy archive (.plc) files, they’re binary so while we can back them up in that form they’re not terribly useful for diff’ing or entering into CVS and SVN repositories. So we need a way to export them as plain text.
You can do this via the web interface as XML. So I tracked that back to **/usr/share/ts/bin/import_export_policy.pl**
So while there is already an official F5 script to do this, it actually requires you already know the ASM policy ID. There doesn’t appear to be anything already written that can dump a list of policies.
The F5’s web interface is written in PHP and it knows how to get a list. By creating a php script and including the right files we can piggy back off the same code and the unit’s configuration without having to know how to decode the encrypted MySQL password and other components.
So yes, while these two scripts are a little immature, they are doing the job for now. I’ll be posting updates to them as I make major improvements. They’ve been created based on the .php files used for the F5’s WebGUI based management of the ASM policies. Hopefully they won’t change significantly over time, such that this script will break, however it would be best to manually check/test it after any upgrades.
By dumping them individually to file, in exactly the same way that the web interface does, we also gain another avenue for backup and restore of policies.
1) It dumps *ALL* policies to file in /var/local/asm, you need to mkdir /var/local/asm before using it
2) Based on /ts/dms/policy/pl_export_import_policy.php
3) It takes approximately 8 minutes to run on my development F5 3600. With approximately 110 ASM policies.
4) It adds HTML/XML style comments to the beginning of each line of the produced files, this is to permit easy identification of the modified policy and version. ie.  5) Each policy file is typically between 600KB and 1MB. If you have a lot of policies the size of the resulting config dump that RANCID stores will increase considerably. 6) Scripts are, I believe, best installed in /var/local/bin/ – you need to create the directory.
**Automating Export of Policies** ** **
Cron Config – you can increase or decrease the frequency of the dump and cleanup based on the number of policies your system/s run. 

# /etc/cron.d/asm_policy_export.cron
# Automatic ASM policy export and cleanup
*/30 * * * * root /usr/bin/php /var/local/bin/asm_export_all_policies_as_files.php 1>/var/tmp/asm_export_files.log 2>&1
5 9 * * * root /bin/bash /var/local/bin/asm_export_all_policies_cleanup.sh 1>/var/tmp/asm_export_cleanup.log 2>&1

One script to find them and in the darkness export them….
**// ****asm_export_all_policies_as_files.php**
**// Based on F5’s /ts/dms/policy/pl_export_import_policy.php**
**// Whatever copyright is applicable to that is applicable to this**
** $_SERVER[“DOCUMENT_ROOT”] = ‘/ts’;include_once(‘/ts/dms/common/settings.php’); include_once(‘/ts/dms/common/global_init.php’);

// ######################### change db ############################
// mostly DCC is used here

$query=$db->query(“SELECT id, account_id, name,
        flg_disable_rejects, modifier_name,
        FROM_UNIXTIME(modifytime) AS modifytime, policy_version
        FROM $policy_db_name.PL_POLICIES
        ORDER BY id”) or die(“Error querying database.”);


if($num != 0) {

while($policy = $db->fetch_array($query)) {
                $policy_name = htmlspecialchars(string_wrap($policy[“name”], 45,40,5));
                $policy_version = iif($policy[‘policy_version’],”v{$policy[‘policy_version’]}”,””);

$output = array();
                $result_code = ”;

$file = “/var/local/asm/” . $policy[‘id’] . ‘-‘ . $policy_version . ‘.xml’;

$result = exec_noshell(array(TOOLS_PATH . ‘import_export_policy.pl’, ‘-a’, ‘export’, ‘-p’, $policy[‘id’], ‘-f’, $file, ‘–json’, ‘–format’, ‘xml’), $result_code);

$sed_string = “-i ‘s:^:<!– ” . $policy[‘id’] . ” ” . $policy_name . ” ” . $policy_version . ” –>:’ ” . $file;

exec(‘/bin/sed ‘ . $sed_string, $output, $result_code);

** **
**                 print $file . ” ” . $policy[‘id’] . ” ” . $policy_name . ” ” . $policy_version . “n”;         } }**


** **
And the cleanup script to make sure you don’t wind up with too many files in there. This also ensures that policies which no longer exist are removed within 24 hours, which will in turn be picked up by RANCID.** **


find ${OUTPUT_DIR} -type f -mtime +${AGE} -exec rm -f {} ; 1>/dev/null 2>&1


And a front-end script to dumping them all at once. This helps with RANCID running the necessary commands over the SSH shell; and provides room for extension/modification at a later date.

# /var/local/bin/asm_policy_dump.sh

cat /var/local/asm/*.xml


**Enabling ASM Policy Change Monitoring by RANCID **
RANCID implements vendor/hardware specific checks using individual scripts. The script that collects config/info from F5’s is ‘f5rancid’, usually located in /usr/bin or /usr/libexec/rancid/
Down towards the end of it you’ll find this array of commands which it will run and process via a specific function that each references,

# Main
@commandtable = (
        {‘bigpipe version’              => ‘ShowVersion’},
        {‘bigpipe platform’             => ‘ShowPlatform’},
        {‘cat /config/bigip.license’    => ‘ShowLicense’},
        {‘bigpipe monitor list all’     => ‘ShowMonitor’},
        {‘bigpipe profile list’         => ‘ShowProfile’},
        {‘bigpipe base list’            => ‘ShowBaseRun’},
        {‘bigpipe db show’              => ‘ShowDb’},
        {‘bigpipe route static show’    => ‘ShowRouteStatic’},
        {‘ls –full-time –color=never /config/ssl/ssl.crt’ => ‘ShowSslCrt’},
        {‘ls –full-time –color=never /config/ssl/ssl.key’ => ‘ShowSslKey’},
        {‘bigpipe list’                 => ‘WriteTerm’}

So basically all we’re going to do here is extend f5rancid so that it also runs asm_policy_dump.sh

Here’s a unified diff  patch with the full function, use against f5rancid from rancid 2.3.2 thru 2.3.4,

— f5rancid.original   2010-12-03 15:50:38.000000000 +1000
+++ f5rancid    2010-12-03 16:35:29.000000000 +1000
@@ -245,6 +245,30 @@

+# This routine parses “/bin/bash /var/local/bin/asm_policy_dump.sh”
+sub ShowASMPolicyFiles {
+    my($line) = (0);
+    print STDERR ”    In ShowASMPolicyFiles: $” if ($debug);
+    while () {
+        tr/15//d;
+        # just in case we don’t have CR at EOF ?
+        s/^#-+($prompt.)/$1/;
+        last if (/^$prompt/);
+        next if (/^(s
+        return(1) if /^s*^s*$/;
+        return(1) if /(Invalid input detected|Type help or )/;
+        return(-1) if (/command authorization failed/i);
+        if (!$line++) {
+            ProcessHistory(“ASMPOLICYFILES”,””,””,”#n#/var/local/asm/*.xml:n”);
+        }
+        ProcessHistory(“ASMPOLICYFILES”,””,””,”# $
”) && next;
+    }
+    return(0);
 # This routine parses “bigpipe monitor list all”
 sub ShowMonitor {
     my($line) = (0);
@@ -516,6 +540,7 @@
        {‘bigpipe version’              => ‘ShowVersion’},
        {‘bigpipe platform’             => ‘ShowPlatform’},
        {‘cat /config/bigip.license’    => ‘ShowLicense’},
+       {‘/bin/bash /var/local/bin/asm_policy_dump.sh’  => ‘ShowASMPolicyFiles’},
        {‘bigpipe monitor list all’     => ‘ShowMonitor’},
        {‘bigpipe profile list’         => ‘ShowProfile’},
        {‘bigpipe base list’            => ‘ShowBaseRun’},

And here it is in action.

[rancid@server ~]$ f5rancid -d devf5
executing clogin -t 90 -c”bigpipe version;bigpipe platform;cat /config/bigip.license;/bin/bash /var/local/bin/asm_policy_dump.sh;bigpipe monitor list all;bigpipe profile list;bigpipe base list;bigpipe db show;bigpipe route static show;ls –full-time –color=never /config/ssl/ssl.crt;ls –full-time –color=never /config/ssl/ssl.key;bigpipe list” devf5
PROMPT MATCH: [root@devf5:Active] config #
HIT COMMAND:[root@devf5:Active] config #  bigpipe version
    In ShowVersion: [root@devf5:Active] config #  bigpipe version
HIT COMMAND:[root@devf5:Active] config # bigpipe platform
    In ShowPlatform: [root@devf5:Active] config # bigpipe platform
HIT COMMAND:[root@devf5:Active] config # cat /config/bigip.license
    In ShowLicense: [root@devf5:Active] config # cat /config/bigip.license
HIT COMMAND:[root@devf5:Active] config # /bin/bash /var/local/bin/asm_policy_dump.sh
    In ShowASMPolicyFiles: [root@devf5:Active] config # /bin/bash /var/local/bin/asm_policy_dump.sh
HIT COMMAND:[root@devf5:Active] config # bigpipe monitor list all
    In ShowMonitor: [root@devf5:Active] config # bigpipe monitor list all
HIT COMMAND:[root@devf5:Active] config # bigpipe profile list
    In ShowProfile: [root@devf5:Active] config # bigpipe profile list
HIT COMMAND:[root@devf5:Active] config # bigpipe base list
    In ShowBaseRun: [root@devf5:Active] config # bigpipe base list
HIT COMMAND:[root@devf5:Active] config # bigpipe db show
    In ShowDb: [root@devf5:Active] config # bigpipe db show
HIT COMMAND:[root@devf5:Active] config # bigpipe route static show
    In ShowRouteStatic: [root@devf5:Active] config # bigpipe route static show
HIT COMMAND:[root@devf5:Active] config # ls –full-time –color=never /config/ssl/ssl.crt
    In ShowSslCrt: [root@devf5:Active] config # ls –full-time –color=never /config/ssl/ssl.crt
HIT COMMAND:[root@devf5:Active] config # ls –full-time –color=never /config/ssl/ssl.key
    In ShowSslKey: [root@devf5:Active] config # ls –full-time –color=never /config/ssl/ssl.key
HIT COMMAND:[root@devf5:Active] config # bigpipe list
    In WriteTerm: [root@devf5:Active] config # bigpipe list
[rancid@server ~]$
[rancid@server ~]$ grep -i ‘<policy bigip_version’ devf5.new | wc -l
[rancid@server ~]$

Now RANCID should start picking up your policy changes too.

—–Original Message—–
From: rancid@dev [mailto:rancid@dev]
Sent: Sunday, 5 December 2010 12:10 AM
To: rancid-waf@dev
Subject: waf router config diffs
Index: configs/devf5

– — configs/devf5    (revision 5)
@@ -359485,7 +359485,7 @@
  # <!– 205 webapp-policy v1 –>  <web_application>Web-Application</web_application>
  # <!– 205 webapp-policy v1 –>  <maximum_http_length>8192</maximum_http_length>
  # <!– 205 webapp-policy v1 –>  <maximum_cookie_length>8192</maximum_cookie_length>
– # <!– 205 webapp-policy v1 –>  Test change detection

  • <!– 205 webapp-policy v1 –>  Test change detection again

# <!– 205 webapp-policy v1 –>  <trigger_irule>false</trigger_irule>
  # <!– 205 webapp-policy v1 –>  <trust_xff>false</trust_xff>
  # <!– 205 webapp-policy v1 –> 

*** *** *** *** *****EDIT Mon Dec 06 218, added below.***
** ** **Including ASM Policy Files and Scripts in UCS Backups**
Modifying the UCS backup policy is the preferred way of including the produced files and the scripts used above in backups. This should then work in seamlessly with your existing backup methodology. Follow F5 SOL4422, 


And add the following or similar to the end of /usr/libdata/configsync/cs.dat

# local scripts
save.9000.dir          = /var/local/bin
# asm exports
save.9001.dir          = /var/local/asm

**# cron configs** **save.9002.dir****          ****= /etc/cron.d**
The entire directory and contents should now be included in your .ucs archives.
Author image
About colin-stubbs