How Docker Broke the Internet for Me

We have seen two major outages in the last weeks caused by AWS and Azure networking issues. Both outages had a significant impact on services running in the cloud and affected a large number of users. The self-hosting people were laughing about it, but some got struck with the recent Docker 29.0.0 release. It simply increased the minimal API version from 1.24 to 1.44.

This change broke Traefik and some tooling, like the testcontainers-java. The maintainers of Traefik already released a fix, and probably created some stress for them. A lot of issues poped up in their GitHub repository, like this one https://github.com/traefik/traefik/pull/12256. I felt sorry for them, many users use their stuff - including me - to drive HTTP/HTTPS traffic to their websites. It was great to see how quickly shared workarounds. The Traefik maintainers were able to provide and ship a fix in a new release very quickly. All you need to do is upgrade to 3.6.1, and you are good to go again.

macOS with Apple Silicon and x86-64-v3 support

I’m maintaining container images in the OpenNMS ecosystem. My local machine is a Mac with Apple Silicon. While I was upgrading some base images using RHEL UBI 10 / CentOS 10 images, I noticed the following error message.

0.112 Fatal glibc error: CPU does not support x86-64-v3

That happens if you try to run on an emulated x86 architecture running on ARM. After some investigation, I’ve found an article in the Red Hat developer forum talking about upgrading the Microarchitecture level from v2 to v3. The problem introduced is that v3 isn’t supported. I’m using Orbstack and you can reproduce this problem simply running the following docker command.

Building container images for OpenNMS

The previous article described how you can build and compile OpenNMS Horizon from source. This section explains how you build container images (OCI) from the source artifacts.

Deploy base image as foundation

Running OpenNMS Horizon core, Minion, or Sentinels in a container requires shared components. Some of them are a) the JDK base image, b) some useful tools, and c) JICMP, and JICMP6.

The JDK is shared with Core, Minion, and Sentinel. JICMP, and JICMP6 are required for Core and Minion. To manage these dependencies, we have a deploy-base image created which covers the main requirements running the Core, Minion, and Sentinel server processes. Getting an efficient size was a goal and a multi-stage build approach was chosen to address it. The fist

Hello Containerlab with Orbstack

I still remember signing up on DockerHub 11 years ago. Learning how to build container images on real world projects is definitely a plus. Having a software and some use cases in the back of your mind, helps you to achieve things quicker and with some purpose. I’ve started to work with containerlab to build some network test environments mainly for three use cases:

• Layer 2 network topologies for network monitoring tests using LLDP, CDP and Bridge-MIB
• Routing topologies with protocols like BGP, OSPF or IS-IS in general
• Some vendor specific network gear for monitoring with SNMP and streaming telemetry

Playing with OpenNMS from this perspective opens some use cases around Netflow, IPFIX, BGP monitoring, and SNMP in general. I was using Docker4Mac for a very long time. At some point in time I have switched to colima which was slim and easy to use. With switching to ARM on my Mac it got a bit more complicated.

Docker build and cache invalidation

Right now I’m working with my work mates @opennms integrating the docker image building in our CI/CD environment. We build our container image based on CentOS and we noticed the caching doesn’t work for ${reasons}.

Running a docker build -t myimage . ended up always in installing packages from the official yum repositories even we haven’t changed anything in the Dockerfile.

To understand things better, I went back to drawing board and started with a simple example and rebuilding things step by step to understand when gets the docker build cache unnecessarily invalidated.

Docker, Java, Signals and Pid 1

Running a Java application in a container seems to be very easy. The devil is in the details and I want to shed some light on the PID 1 problem when you run Java applications in containers. Theoretical, processes in containers should not have state so you just don’t care, but reality is different.

Signals are used to message running process to behave in certain ways. A common case is to terminate a process nicely sending a SIGTERM to the process identified by the process id using the tool ps. People can implement logic shutting down an application gracefully. When you issue the command kill <pid> you send a SIGTERM to a process. We have other signals you can use, especially when application provide terminals. Hitting CTRL + C sends SIGINT to the terminal process. Some signals are quite out of time, the signal to hangup SIGHUP comes from terminal applications. It is often used nowadays telling a running process to re-read configurations.

Authenticate OpenNMS Horizon with FreeRADIUS

Centralized authentication is a core service as soon you have a network with more than 3 computers. This article is about how to authenticate a OpenNMS Horizon 22.0.2 using RADIUS provided by a FreeRADIUS service.

In this example the FreeRADIUS server is configured to provide 3 users. A dictionary is configured which returns 2 roles, ROLE_USER and ROLE_ADMIN which can be used to decide which security role is assigned in the OpenNMS Horizon Web UI.

Authenticate OpenNMS Horizon with Active Directory on Windows Server 2016

Centralized authentication is a core service as soon you have a network with more than 3 computers. This article is about how to authenticate a OpenNMS Horizon 22.0.1 against an Active Directory provided on a Microsoft Windows Server 2016. I’ll focus here on the Active Directory and Spring configuration parts, securing the connection with LDAPS and using self-signed certificates in Java is another topic and not covered here.

In this example the domain is called labmonkeys.local and the Microsoft Windows Server has the IP address 192.168.178.220. The user name for the bind user is OpenNMS Bind User with the password MyPass123!.

Build OpenNMS with Docker

Being able to compiling an open-source project is important. You can change the code, so you should also able to build it.

Why is there a dedicated Docker image for the build environment? The dependencies running a pre-build OpenNMS Horizon distribution and compiling from source are different. To build OpenNMS Horizon you need Apache Maven and to compile JICMP, JRRD you need a C compiler environment. This is nothing you want to carry when you just want to run OpenNMS Horizon.

Scanning for SNMP communities

Adding devices into monitoring system is easy. Getting all the right SNMP communities for them is harder. People don’t give you the right community string or forget to open firewall ports.

If you have to test a lot of IP’s against various IP addresses you can use nmap and a community list file as an input.

Be aware you talk about permission to run this test otherwise somebody can get angry when you try to brute-force community strings against their devices.