LibreNMS and Oxidized

 Install LibreNMS on Ubuntu 12.04 LTS

Rancid is an American band! Oops ! Sorry! Here I mean Rancid “Really Awesome New Cisco config Differe” ;-) ;-)
LibreNMS ‘Libra Network Monitoring System’ is a auto discovering PHP/MySQL-based monitoring system similar to the erstwhile Observium and one could say LibreNMS is the copy of Observium and is a OpenSource software.
Oxidized is a great tool for automating backups of device configs like ASA Firewalls, IOS Routers and all types of switches. Unlike Rancid, Oxidized can be integrated with LibreNMS and has a very useful and handy Graphical User Interface (GUI).  Note: # prompt shows that you will have to run the command as root         

Note: # prompt shows that you will have to run the command as root
          $ prompt shows that you will have to run the command as normal user
(Update/Upgrade the System)
#apt update
#apt upgrade

(Install the packages)
#apt install software-properties-common
#add-apt-repository universe
#add-apt-repository ppa:ondrej/php
#apt update
#apt install acl curl apache2 fping git graphviz imagemagick libapache2-mod-fcgid mariadb-client mariadb-server mtr-tiny nmap php-cli php-curl php-fpm php-gd php-gmp php-json php-mbstring php-mysql php-snmp php-xml php-zip rrdtool snmp snmpd whois python3-pymysql python3-dotenv python3-redis python3-setuptools python3-systemd python3-pip


(Add librenms User)
#useradd librenms -d /opt/librenms -M -r

(Install Librenms)
#cd /opt
#git clone https://github.com/librenms/librenms.git

(Set the Permission)

#chown -R librenms:librenms /opt/librenms
#chmod 771 /opt/librenms
#setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
#setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

(Install the PHP dependencies)
#su - librenms
$./scripts/composer_wrapper.php install --no-dev
$exit

(Set timezone)

#vi /etc/php/8.1/fpm/php.ini
And add the following line

date.timezone = Asia/Thimphu

#tzselect
Select Asia, then Thimphu and then add to the file .profile under root

#vi ~/.profile
TZ='Asia/Thimphu'; export TZ

#source ~/.profile

(Optimize MySQL/MariaDB)

#vi /etc/mysql/mariadb.conf.d/50-server.cnf
Add the following two lines
innodb_file_per_table=1
lower_case_table_names=0

#systemctl enable mariadb
#systemctl restart mariadb

(Login into mysql)

#mysql -u root

(Create the databases, credentials and the set the privileges)
MariaDB [(none)]> CREATE DATABASE librenms CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
MariaDB [(none)]> CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'password';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'localhost';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> exit

(Configure PHP-FPM(FastCGI Process Manager))
#cp /etc/php/8.1/fpm/pool.d/www.conf /etc/php/8.1/fpm/pool.d/librenms.conf
#vi /etc/php/8.1/fpm/pool.d/librenms.conf
And change the file content as follows

[www] to [librenms]

User and group as follows
User = librenms
Group = librenms

Listen = /run/php-fpm-librenms.sock

(Configure Web Server)
Create the following file
#vi /etc/apache2/sites-available/librenms.conf

<VirtualHost *:80>
  DocumentRoot /opt/librenms/html/
  ServerName  

  AllowEncodedSlashes NoDecode
  <Directory "/opt/librenms/html/">
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews
  </Directory>

  # Enable http authorization headers
  <IfModule setenvif_module>
    SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
  </IfModule>

  <FilesMatch ".+\.php$">
    SetHandler "proxy:unix:/run/php-fpm-librenms.sock|fcgi://localhost"
  </FilesMatch>
</VirtualHost>

(Configure snmpd)
First make the backup of the snmpd.conf file installed by the Ubuntu system and now copy the snmpd.conf file from the
Librenms directory as follows

#cp /etc/snmpd/snmpd.conf /etc/snmpd/snmpd.conf.back
#cp /opt/librenms/snmpd.conf.example /etc/snmp/snmpd.conf
#vi /etc/snmp/snmpd.conf
com2sec readonly  default         ThisIsSecret

#curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro
#chmod +x /usr/bin/distro
#systemctl enable snmpd
#systemctl restart snmpd

(Configure cronjob)

#cp /opt/librenms/librenms.nonroot.cron /etc/cron.d/librenms

Copy logrotate config
LibreNMS keeps logs in /opt/librenms/logs. Over time these can become large and be rotated out. To rotate out the old logs you can use the provided logrotate config file:
#cp /opt/librenms/misc/librenms.logrotate /etc/logrotate.d/librenms

(Web Installer)
http://10.0.2.242/install

And if you get the following error

Time between this server and the mysql database is off blah blah

Run the following commands
#mysql –e “SELECT @@global.time_zone;”
SYSTEM

Check the time of the mysql
#mysql -e “select now();”
Check if the value of the time is same as the value from the following command
#date
If they are not same, run the following command (for Bhutan ie., +6)
#mysql -e "SET GLOBAL time_zone = ‘+6:00’;"

Again compare the value of #mysql -e “select now();” and #date -> They should be the same and go back to the web installer screen.

(Performance Tuning LibreNMS using rrdcached; and MySQL tuning)
#apt install rrdcached
#vi /etc/default/rrdcached  

# Full path to daemon
DAEMON=/usr/bin/rrdcached

# Optional override flush interval, in seconds.
WRITE_TIMEOUT=1800

# Optional override maximum write delay, in seconds.
WRITE_JITTER=1800

# Optional override number of write_threads
WRITE_THREADS=4

# Where database files are placed.  If left unset, the default /tmp will
# be used.  NB: The daemon will reject a directory that has symlinks as
# components.  NB: You may want to have -B in BASE_OPTS.
BASE_PATH=/opt/librenms/rrd/

# Where journal files are placed.  If left unset, journaling will
# be disabled.
JOURNAL_PATH=/var/tmp/

# FHS standard placement for process ID file.
PIDFILE=/var/run/rrdcached.pid

# FHS standard placement for local control socket.
SOCKFILE=/var/run/rrdcached.sock

# Optional override group that should own/access the local control
# socket
SOCKGROUP=librenms

# Optional override access mode of local control socket.
#SOCKMODE=0660

# Optional unprivileged group to run under when daemon.  If unset
# retains invocation group privileges.
DAEMON_GROUP=librenms

# Optional unprivileged user to run under when daemon.  If unset
# retains invocation user privileges.
DAEMON_USER=librenms

# Any other options not specifically supported by the script (-P, -f,
# -F, -B).
BASE_OPTIONS="-F -B"

#systemctl restart rrdcached

(Configuring rrdcache in librenms)

#vi /opt/librenms/config.php
Change the line
#$config['rrdcached']    = "unix:/var/run/rrdcached.sock";

$config['rrdcached']    = "unix:/var/run/rrdcached.sock";

Add/Change line from

$config['rrdtool_version'] = ;

To

$config['rrdtool_version'] = ‘1.7.2’ ; // This is in our case during the time of installation

Then restart the apache2 service

#systemctl restart apache2

(Fine tuning MySQL)

#cd /etc/mysql/conf.d/
#vi librenms.cnf
[mysqld]

innodb_file_per_table = 1
Sql-mode=“”
innodb_flush_log_at_trx_commit=0

*save the file*

#systemctl restart mysql

Install Oxidized on Ubuntu 12.04 LTS

*Enable universe repository by running the following command
#add-apt-repository universe (which is required for libssh2-1-dev)

#apt-get install ruby ruby-dev libsqlite3-dev libssl-dev pkg-config cmake libssh2-1-dev libicu-dev zlib1g-dev g++
#gem install oxidized
#gem install oxidized-script oxidized-web

#useradd oxidized -s /bin/bash -d /home/oxidized/ -m
#passwd oxidized
#usermod -aG sudo oxidized (adding oxidized to the shudders group - this is optional)

Now login as oxidized user
#su - oxidized
$chown -R oxidized:oxidized /home/oxidized
$mkdir -p ~/.config/oxidized/configs
$mkdir -p ~/.config/oxidized/router.db

Run oxidized command
$oxidized
Edit ~/.config/oxidized/config (is the output you will see and default config file should be created)

Note: take the backup of the /home/oxidized/.config/oxidized/config to /home/oxidized/.config/oxidized/config.back before editing the file (for you may lose mess the syntax in ruby)

$vi /home/oxidized/.config/oxidized/config

*Make necessary changes to the username/password and the sources such as csv, http, file, git, etc.
In my case, I will go for csv (router.db) file and add the following lines to the files

source:
  default: csv
  csv:
    file: ~/.config/oxidized/router.db
    delimiter: !ruby/regexp /:/
    map:
      name: 0
      model: 1
For my case; I am testing with the following file

*Beginning of the file*
---
username: admin
password: ********
model: huawei
resolve_dns: false
interval: 3600
log: ~/.config/oxidized/log
use_syslog: false
debug: false
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
rest: 192.168.1.2:8888
next_adds_job: false
vars: {}
groups: {}
models: {}
pid: ~/.config/oxidized/oxidized.pid192.168.1.2
crash:
  directory: ~/.config/oxidized/crashes
  hostnames: false
stats:
  history_size: 10
input:
  default: ssh, telnet
  debug: false
  ssh:
    secure: false
  ftp:
    passive: true
  utf8_encoded: true
output:
  default: file
  file:
    directory: ~/.config/oxidized/configs
source:
  default: http
  debug: false
  http:
    url: http://192.168.1.2/api/v0/oxidized
    delimiter: !ruby/regexp /:/
    map:
      name: hostname
      model: os
      group: group
    headers:
      X-Auth-Token: '71c1be0609d6e3b6fdf21e1bfffdb0f0'
model_map:
  huawei: vrp
  juniper: junos
  cisco: ios
*End of the File*

(Now integrate oxidized with librenms) with the user nms)
$lnms config:set oxidized.enabled true
$ lnms config:set oxidized.url http://192.168.1.2:8888
$lnms config:set oxidized.features.versioning true
$ lnms config:set oxidized.group_support true
$ lnms config:set oxidized.default_group default
$lnms config:set oxidized.reload_nodes true

(Verify the return of groups by querying the API:)
curl -H 'X-Auth-Token: YOURAPITOKENHERE' https://librenms.org/api/v0/oxidized

Add the following lines to the librenms config.php file

 Oxidized configuration

$config['oxidized']['enabled']                  = TRUE;
$config['oxidized']['url']                      = 'http://192.168.1.2:8888';
$config['oxidized']['features']['versioning']   = true;
$config['oxidized']['group_support']            = true;
$config['oxidized']['default_group']            = 'default';
$config['oxidized']['reload_nodes']             = true;
$config['oxidized']['ignore_os'] = array('linux','windows');
$config['oxidized']['ignore_types'] = array('server','power');

(To add oxidized service to the Ubuntu system, do the following)

#cp /var/lib/gems/2.7.0/gems/oxidized-0.28.0/extra/oxidized.service /etc/systemd/system

(Setup /var/run/)

#mkdir /run/oxidized
Chown oxidized.oxidized /run/oxidized

(Make oxidized start on boot)

#systemctl enable oxidized.service