BGP monitoring playground

Monitoring BGP can be done in various ways. First thing people want to know is if there is a way to get notifications in case a BGP peering session goes down. A pretty common way monitoring the BGP peering session state is using SNMP and the RFC1269 MIB. In OpenNMS Horizon we have the BGP session monitor which allows to track the state using the BGP peer table. The downside is, you need to configure for every peering session a monitor and this can be cumbersome and hard to maintain.

OpenNMS Horizon, Docker, Traefik and Let's Encrypt

I work from home for over 6 years now and especially when you like networking, want to get stuff up and running and breaking it - you start looking around :) You’ve heard about k8s, k{0,3}s or Microk8s but you don’t want to use it to run your private blog and you find yourself in a spot where the benefits running stuff in containers justify the pain - this article might be something for you :)

Quick manual Kafka OpenNMS stack

We have gathered a few ready-to-run Docker stacks in our public stack-play GitHub repository. But sometimes you need Kafka, Zookeeper and OpenNMS quickly on a baremetal deployment without Docker. Here a few quick notes how to get the bare minimum up and running.

Zookeeper

Install OpenJDK 11 JRE

sudo apt install -y openjdk-11-jre

Create a user for zookeeper

sudo adduser --system --home /opt/zookeeper --disabled-login zookeeper

Create a logs directory

sudo mkdir /var/log/zookeeper
sudo chown zookeeper:nogroup /var/log/zookeeper -R

Install zookeeper

Setting up a VMware Test environment

To test functions like importing OVA files in VMware ESXi and with vCenter the trial phase and a local deployment can be used. You need the following requirements:

• VMware Workstation on Windows or VMware Fusion on Mac OSX
• VMware Hypervisor ISO image to install the ESXi host system
• VMware vCenter ISO image for local deployment

If you just want to learn how it works you can get a 60 days trial license registering an account on VMware. I had to manually select a Bridged network for the ESXi server. The vCenter appliance used a Bridge network by default. connected them to a bridged network. You can use DHCP if you want, but you need to make sure they get the same IP addresses afterwards.

VMware multipathd log spam

While I was deploying Loki with Promtail I’ve seen a lot of log spam from Ubuntu virtual machines in my VMware environment. As a note to myself and for some others who want cleaner system logs – here is what I’ve found to get rid of it.

The log entries look like these here:

2021-04-07 20:14:21 opennms-bgp multipathd[693]: sda: failed to get sgio uid: No such file or directory
2021-04-07 20:14:21 opennms-bgp multipathd[693]: sda: failed to get sysfs uid: Invalid argument
2021-04-07 20:14:21 opennms-bgp multipathd[693]: sda: failed to get udev uid: Invalid argument
2021-04-07 20:14:21 opennms-bgp multipathd[693]: sda: add missing path

The best article I’ve found was from SUSE describing the problems source. In a nutshell, VMware doesn’t provide information needed by udev to generate the /dev/disk/by-id. To solve the problem you have to set in the Virtual Machine the attribute disk.EnableUUID=true. For the reason I have a few VM’s it’s pretty tidious to do all these things manually.

Installing Node Exporter on Linux

In OpenNMS Horizon 28+ is now a PrometheusCollector available. It scrapes the metrics from the provided exporter pages and allows to add data collections. As of speaking today it is not 100% feature complete, scraping data types like histograms is not implemented yet. If you want to play around here is a quick way to get the Linux Node_Exporter installed.

The following steps are executed in a root shell with sudo -i.

Hardening SSH for audit

Running a server in the public requires some additional work. Especially if you want management access via SSH for Ansible or if you want break stuff manually with fiddeling around :)

You can run an SSH audit of your public server using https://www.sshaudit.com. This section here is a very condensed way to get an A rating.

Just use strong host key for authentication of the host

# file: /etc/ssh/sshd_config
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

Delete existing keys and re-generate the RSA and ED25519 keys

UCARP and High Availability

If you have ever played with BSD you probably ran into CARP. It allows you to build a high available service which is provided by two physical servers behind a virtual shared IP address. The CARP nodes define a master and a backup system. A master serves the content and if the master crashes, the backup system takes over automatically the virtual IP (VIP) and the client won’t notice.

Disclaimer: You should be aware this setup will not share load and increase your network throughput. It just used to increase availabilty and room to do maintenance without bringing your service down.

Continue reading

Send notifications with Signal

In some cases it is nice to have notifications from OpenNMS in a separate channel on a smartphone and you don’t want to pay for SMS. Here is a tutorial where I use Signal using the signal-cli.

This Howto will describe how to download the latest signal-cli tool, link it to your existing Signal account and how to configure OpenNMS to use it as a notification target. You should have already an OpenNMS Horizon or Meridian running and you need a Signal account with the Signal app installed and configured on your smartphone.

Everyone can change it and why you shouldn't

Open-source software is great. If there is something you don’t like, you can at least - try to change it. A lot of open-source software out there is not primarily used by private people. There are many companies who provide professional services around open-source software. Some of them try to enhance the appearance with custom User Interfaces, their company logo to fit their own Corporate Identity. There are several motivations, mostly they want to be distinguished on the marked or need some easy to maintain little customization which allows their sales guys easier to sell a project and not using the software from the community projects website. How hard could it be to make something pretty, right?