I spent some time with preparing the OUCE 2015 and we use the free an open source software frab as conference management system. I do most of my server automation with Chef and my playground is running with Vagrant. For this reason I decided to spend some time an built a Vagrant / Chef environment and contributed it back to the frab project. This guys do a really good job and IMHO it is the best free conference management system you can get. If you want to play with it feel free and give it a try and contribute.

The last event of the year is the famous 31c3 in Hamburg. It’s always amazing hanging around on this unconference. We sponsor some bandwidth by mirroring the video streams and we can definitely recognized the beginning of the conference ;)

I spend a lot of time in the OpenNMS project and I love to work in free software and the workflows around it. We moved with our project from SourceForge to GitHub a few years ago and I think it was the right decision. There are now some established workflows in this ecosystem and they tear down borders between different open source projects and here is an example of it.

I develop with IntelliJ IDEA and spend currently some time working in documentation of OpenNMS. We have migrated from Docbook XML format to AsciiDoc and started with the help of a few brave community volunteers a new documentation environment. I’ve found a plugin for IDEA which renders AsciiDoc and gave it a try to have a better workflow working on documentation and navigating through source code in just one program.

Unfortunately I run in an error message. The project is hosted on GitHub and I’ve opened an issue with the version and the behavior. They responded my question just a few hours later.

They where really helpful and asked how it was installed to avoid basic mistakes. The issue still existed and they asked me if its possible to provide the .adoc files to reproduce the error. This was quite easy, the documentation I work with is also public on GitHub, so I just sent them a link to the files. The guys figured out, the problem occurred whenever I use the source code formatting from AsciiDoc, so it was easy to create a small test file to reproduce the issue. Beside the fact finding the reason of the problem, they created also a new release of the plugin and sent me a link in the public comments and asked if I can verify the solution. I’ve installed the new plugin and ran unfortunately in a different error, I’ve provided a screenshot and the detailed error message.

Just a few hours later they figured out, there was a reference in AsciidoctorJ a dependency of the IDEA plugin which pointed to a already fixed issue in JRuby dealing with white spaces in a file path. They updated to the latest version of JRuby and provided another version of the IDEA plugin just a few hours later which fixed my problem.

Please let imagine the procedure with a similar issue in a closed source software where the issue is in a dependency of a dependency. This issue was fixed in 5 days, they provided two releases of the plugin and they responded to my comments in maximum a day! I paid nothing – just the time providing information of my issue – which I had anyways even I had to pay for the software and the support. There was not one moment blaming, finger pointing and complete transparency about three different open source projects to identify and fix the problem.

Thank you for this open source experience

– or my friend Q would sing: ♪ Everything is awesome … ♪

If you run a centralized monitoring system in large environment you can run in some issues regarding file descriptor limits. Linux gives you very detailed information in the kernel control and information center in /proc. The soft and hard limits have effect for file and network sockets, which can end up in a too many files open exception in OpenNMS.

The default values for soft and hard limits can be checked with

ulimit -a
ulimit -a -H

The value is per user and each new process inherits these limits. OpenNMS changes the hard limit during the start with the default init script and changes with

ulimit -n 20480

from normally 4096 to 20480. You can change this value by adding the following line to your /etc/opennms/opennms.conf.


If you start OpenNMS you can see the limits for the OpenNMS JVM with

cat /proc/$(cat /var/run/opennms.pid)/limits

You can see how much file descriptors OpenNMS has allocated with:

ls -l /proc/$(cat /var/run/opennms.pid)/fd | wc -l

If you use lsof with the process id of OpenNMS you will see a larger number than in /proc/pid/fd

lsof -p $(cat /var/run/opennms.pid) | wc -l

The reason is memory mapped .so files are listed and don’t count for the configured limits and are listed with lsof.

lsof | grep $(cat /var/run/opennms.pid) | wc -l

If you want to see how many filesystem handles OpenNMS uses, you can run:

cat /proc/sys/fs/file-nr
4128    0   262144

you can see three values:

number of allocated file handles: 4128
number of used file handles:      0
maximum number of file handles:   262144

To help the OpenNMS project I spent a year together with Alexander Finger and Klaus Thielking-Riechert writing an OpenNMS book. We started with a development version of for 1.8 and tried to find a mixture between consistent slow changing and new concepts in 1.8. It was kind a complicated thing and from my point of view not perfectly done. Nevertheless I’ve thought about something like: "It’s about time writing a second edition covering the topics in coming 1.14?" The problem I’ve with books, it doesn’t allow contribution and hasn’t a really long life cycle. So I’ve decided instead of spending a year again writing a book, I want something which is more helpful to the project itself.

During my visits on different conferences like FroSCON, Linuxtage in Chemnitz and LinuxTag in Berlin, I tried to get in talks and met people working in the community area. Two things I’ve quickly recognized, one was the Open Stack project, which did a really cool job communicating about how to contribute in their project. The second one was a talk from Mikey Ariel a technical writer and documentarian at Red Hat, Inc. on LinuxTag in Berlin.

To get some feedback, I started an open discussion about documentation in the OpenNMS project with attendees at OUCE 2014 in Southampton. I was very happy and we had a very constructive conversation and people seems interested to help the project here. The status quo of our documentation is currently MediaWiki based collection of unstructured sometimes category tagged pages. On one side it makes it easy for people creating content and there is quite a lot of good stuff in the Wiki, but on the other side it is hard to find things. You have always exactly to know what term you have to search for and makes it hard to get an idea what OpenNMS is capable of. To address this issue I decided to spend the week at DevJam on OpenNMS documentation and my friend Markus von Rüden helped me a lot. He is a developer from hell and helped me a lot evaluating technologies and gave important input how to establish a process in an agile environment.

A few weeks later I got in contact with a guy worked at Juniper with OpenNMS and he asked to help us with this topic. I was amazed and used his positive feedback as trigger to get a documentation meeting set up. We used the opennms-discuss and move now this topic to opennms-docs mailing list. I’ve suggested a Google Hangout where we could explain a little about the experience we had what we have figured out during DevJam.

We have basically divided the documentation in two parts:

  • Wiki based documentation for tutorials and integration of OpenNMS with external devices, applications etc.
  • Source code based documentation which gives a complete reference how to configure things which is tightly based on a version of OpenNMS

For each of this components we created a Jira component to track issues against:

We started with a simple doable task like, document all existing monitors in OpenNMS with default, values and possible configuration parameters. It is our first little project we use to establish and model the process and also building the "How to contribute to documentation guide" which is currently a work in progress. The guys helping at the docs suggested to have a bi-weekley meeting and so I’ll drop a doodle at the opennms-docs mailing list to get a next appointment set up. If you are interested in join, the opennms-docs is a low traffic mailing list and this is the place to be in the loop if you are interested on this topic.

Thanks for reading and I’m excited about the help you guys who want to help

Have a nice day.

I started in 1998 in IT-Services and I did all the funny stuff – was involved building new networks for companies, migrated software from commercial vendors from version A to B to C to D and I have wasted too much lifetime on broken RAID 5s, backups and restores.

Air Conditioning improvisation at CERN.

If you want to learn how to operate a computer network – this is a good place. You will be always called for the complicated problems nobody can solve with a simple Google search. You get also a good feeling, which solution is maintainable over a longer period of time. Fortunately people developing software experienced enough pain and started thinking about how to make code maintainable. Good developers realized years ago, people spend more time maintaining software than writing it. In my point of view an underestimated acceptance criteria in an "Agile" process ;) Since I spend more and more time in the OpenNMS project, I get much better an idea what the meaning about testing, documenting, Continuous Integration and Test Driven programming is. What people realize is a nameless magical connectivity between the guys who develop Things™ and the ones who bring them to production and the poor souls which have to keep it productive. In the late 90s there wasn’t a name for it. To make a good job Ops-Guys and the Dev-Guys have to work together – which wasn’t/isn’t a default behavior by nature.