no42

... search for answer?

Docker build and cache invalidation

2019-05-15 3 min read Ronny Trommer

Right now I’m working with my work mates @opennms integrating the docker image building in our CI/CD environment. We build our container image based on CentOS and we noticed the caching doesn’t work for ${reasons}.

Running a docker build -t myimage . ended up always in installing packages from the official yum repositories even we haven’t changed anything in the Dockerfile.

To understand things better, I went back to drawing board and started with a simple example and rebuilding things step by step to understand when gets the docker build cache unnecessarily invalidated.

The section Leverage build cache with especially this section was important to me:

Aside from the ADD and COPY commands, cache checking does not look at the files in the container to determine a cache match. For example, when processing a RUN apt-get -y update command the files updated in the container are not examined to determine if a cache hit exists. In that case just the command string itself is used to find a match.

So this means the outcome of a RUN command doesn’t invalidate a cache it’s purely checked against the command string itself to find a match.

So lets start with a simple example:

Ok testing the build, the initial build works as expected and downloads wget from CentOS mirrors and installs it. The second run hits the caches and finishes in under a second.

Caching yum install

We can now change the output in our last echo command and it will use the cache for the yum install wget command. Only the last layer would be rebuild which is quick and works as expected.

We have added some build arguments when running in CI/CD which injects some information like the build number which ends up in labels, so we can give some hints which build created this artefact you have running.

Running without any changes is not a big deal and caching works as expected.

Caching with build argument

Ok what happens if we inject a build argument like the build number and what is the effect on the cache.

Effect on cache with argument

By observing the behaviour, you can see changing the build argument invalidates the layer above the RUN directive, which has as consequence to rebuild all following layers.

If you move the Line 3 ARG BUILD_NUMBER to the end right before the LABEL you can change it and you get cache hits from the more expensive tasks at the top.

In conclusion order in Dockerfile matters and it make sense to group logical commands in RUN statements to make caching more efficient while working on your Dockerfiles.

Happy caching.

Docker, Java, Signals and Pid 1

2019-02-20 9 min read Ronny Trommer
Running a Java application in a container seems to be very easy. The devil is in the details and I want to shed some light on the PID 1 problem when you run Java applications in containers. In a general running applications in containers should not have any state so you just don’t care, but reality is different forces you to have to. Signals are used to a running process to behave in a certain ways. Continue reading

SSL and Java

2018-11-26 3 min read Ronny Trommer
Running applications with a current Java is not a big deal thanks Let’s Encrypt. This article describes what happens if you want to authenticate your OpenNMS against LDAP using SSL with a self-certified certificate. First of all I assume you have confiured verything so you can authenticate against LDAP in plaintext and you got a role mapping as you wanted it. If not you can have a look here. Continue reading

Send notifications with Signal

2018-11-01 3 min read Ronny Trommer
In some cases it is nice to have notifications from OpenNMS in a separate channel on a smartphone and you don’t want to pay for SMS. Here is a tutorial where I use Signal using the signal-cli. This Howto will describe how to download the latest signal-cli tool, link it to your existing Signal account and how to configure OpenNMS to use it as a notification target. You should have already an OpenNMS Horizon or Meridian running and you need a Signal account with the Signal app installed and configured on your smartphone. Continue reading

Everyone can change it and why you shouldn't

2018-09-27 3 min read Ronny Trommer
Open Source software is great. If there is something you don’t like, you can at least - try to change it. A lot of Open Source software out there is not primarily used by private people. There are many companies who provide professional services around Open Source software. Some of them try to enhance the appearance with custom User Interfaces, their company logo to fit their own Corporate Identity. There are several motivations, mostly they want to be distinguished on the marked or need some easy to maintain little customization which allows their sales guys easier to sell a project and not using the software from the community projects website. Continue reading

Guidance to Survive Monitoring

2018-08-08 5 min read Ronny Trommer
While working in the monitoring field for a long time, here are some rules I try to follow when requirements go awry. Rule #1: Only create an alert when human interaction is required When you setup a monitoring, it tends to get noisy very quickly. The problem is, people want to know everything and want to monitor everything. You tend to build a system which sends you a lot of alarms and you will get alarm fatique. Continue reading

There are only two hard things in Computer Science: cache invalidation and naming things.

Phil Karlton

If you’re brave to say’ goodbye’, life will reward you with a new ‘hello’.

Paulo Coelho

He's dead, Jim

2018-07-30 5 min read Ronny Trommer
If you operate networks there is a big chance you had to deal with SNMP - the Simple Network Management Protocol. If you ever wondered where it came from, it started with a big bang. On October 27, 1980, there was an unusual occurrence on the ARPANET. For a period of several hours, the network appeared to be unusable, due to what was later diagnosed as a high priority software process running out of control. Continue reading

Monitoring Websites with OpenNMS

2018-07-27 4 min read Ronny Trommer
Monitoring websites is a common requirement. Using OpenNMS to monitor websites can be done by using the built in HTTP/HTTPS based monitors. While a “Node” can be pretty much everything in a network, the internal model to monitor something is pretty old-fashioned and static. Monitoring a service requires to assign a service to an IP address. This article describes a pattern how you can monitor web sites with low maintenance and without the need to maintain for each website a monitor which is cumbersome in maintenance. Continue reading
Older posts