Docker build and cache invalidation

Right now I’m working with my work mates @opennms integrating the docker image building in our CI/CD environment. We build our container image based on CentOS and we noticed the caching doesn’t work for ${reasons}.

Running a docker build -t myimage . ended up always in installing packages from the official yum repositories even we haven’t changed anything in the Dockerfile.

To understand things better, I went back to drawing board and started with a simple example and rebuilding things step by step to understand when gets the docker build cache unnecessarily invalidated.

Docker, Java, Signals and Pid 1

Running a Java application in a container seems to be very easy. The devil is in the details and I want to shed some light on the PID 1 problem when you run Java applications in containers. Theoretical, processes in containers should not have state so you just don’t care, but reality is different.

Signals are used to message running process to behave in certain ways. A common case is to terminate a process nicely sending a SIGTERM to the process identified by the process id using the tool ps. People can implement logic shutting down an application gracefully. When you issue the command kill <pid> you send a SIGTERM to a process. We have other signals you can use, especially when application provide terminals. Hitting CTRL + C sends SIGINT to the terminal process. Some signals are quite out of time, the signal to hangup SIGHUP comes from terminal applications. It is often used nowadays telling a running process to re-read configurations.

SSL and Java

Running applications with a current Java is not a big deal thanks Let’s Encrypt. This article describes what happens if you want to authenticate your OpenNMS against LDAP using SSL with a self-certified certificate.

First of all I assume you have confiured verything so you can authenticate against LDAP in plaintext and you got a role mapping as you wanted it. If not you can have a look here.

So the naive approach would be, just changing the line in your activeDirectory.xml from

Monitoring Websites with OpenNMS

Monitoring websites is a common requirement. Using OpenNMS to monitor websites can be done by using the built in HTTP/HTTPS based monitors.

While a “Node” can be pretty much everything in a network, the internal model to monitor something is pretty old-fashioned and static. Monitoring a service requires to assign a service to an IP address.

This article describes a pattern how you can monitor web sites with low maintenance and without the need to maintain for each website a monitor which is cumbersome in maintenance.

Authenticate OpenNMS Horizon with FreeRADIUS

Centralized authentication is a core service as soon you have a network with more than 3 computers. This article is about how to authenticate a OpenNMS Horizon 22.0.2 using RADIUS provided by a FreeRADIUS service.

In this example the FreeRADIUS server is configured to provide 3 users. A dictionary is configured which returns 2 roles, ROLE_USER and ROLE_ADMIN which can be used to decide which security role is assigned in the OpenNMS Horizon Web UI.

Authenticate OpenNMS Horizon with Active Directory on Windows Server 2016

Centralized authentication is a core service as soon you have a network with more than 3 computers. This article is about how to authenticate a OpenNMS Horizon 22.0.1 against an Active Directory provided on a Microsoft Windows Server 2016. I’ll focus here on the Active Directory and Spring configuration parts, securing the connection with LDAPS and using self-signed certificates in Java is another topic and not covered here.

In this example the domain is called labmonkeys.local and the Microsoft Windows Server has the IP address 192.168.178.220. The user name for the bind user is OpenNMS Bind User with the password MyPass123!.

Build OpenNMS with Docker

Being able to compiling an open-source project is important. You can change the code, so you should also able to build it.

Why is there a dedicated Docker image for the build environment? The dependencies running a pre-build OpenNMS Horizon distribution and compiling from source are different. To build OpenNMS Horizon you need Apache Maven and to compile JICMP, JRRD you need a C compiler environment. This is nothing you want to carry when you just want to run OpenNMS Horizon.

Scanning for SNMP communities

Adding devices into monitoring system is easy. Getting all the right SNMP communities for them is harder. People don’t give you the right community string or forget to open firewall ports.

If you have to test a lot of IP’s against various IP addresses you can use nmap and a community list file as an input.

Be aware you talk about permission to run this test otherwise somebody can get angry when you try to brute-force community strings against their devices.

Centralized Logging with Graylog2

How many times do you connect with SSH to your remote server and cat, grep, tail and awk through your logs? It probably works for 3 servers and running a handful services, but if you have more, you should definitely spend some time to centralize your logs.

I personally prefer Graylog2 which can deal very well with different log formats like GELF, Syslog RFC’s. Just start some listener with the format and forward them to your Graylog2 instance.

Cleaner log with Docker and SNMP

Centralizing logs is important as soon you have more than 2 servers. In my environment the bare metal is monitored with Net-SNMP and my services are deployed as containers with Docker. All system logs are sent to a Graylog2 instance and I quickly noticed a few ugly entries caused by snmpd.

Cannot statfs /run/docker/netns/...: Permission denied

You will notice a few of them. First approach try to increase the logging level in /etc/default/snmpd from SNMP daemon with