The OpenNMS Project

So it’s 10:05 here in the UK, and I’m sitting in the kitchen, listening to verity coughing on the baby monitor, drinking beer and watching Bamboo chug through my latest commit. Not very productive use of a Friday night, I think an Amateur Developer SITREP is in order.

At Dev-Jam 2006, David was kind enough to hand me an assignment use case that, at the time, was more flattering than productive. I spent the best part of the week baffled by maven, Spring and the time zone change, and didn’t make much progress. I did make some lasting friends though, went home fitter than I arrived and also found out why urandom gets through so much lotion.

Anyway, the problem has been bugging me ever since. I think I’ve got it nailed now, but it’s been nearly 4 years. I’ve had intermittent goes at it, including an aborted attempt to completely rewrite the report code. At one point, it started to resemble my own small-scale Project Xanadu. In February last year, whilst “resting” between jobs, I changed focus and started work on an API that could eventually allow the existing reports to be replaced using Jasper Reports or BIRT. Some of the work made it into the 1.7.9 release, but I was effectively stalled.

Interestingly, the impetus to finish the work came from others. Ronny and Jay have been working on Jasper Reports integration, which is now up in the main 1.7 branch. Their efforts prompted me to get my act together, and with some help from Ben, Alejandro and others got the branch cleaned up and tracking 1.7.

I’ve got a deadline now, too. I’ve been accepted as a speaker at the OpenNMS User Conference Europe in May, so I’d better get the code merged up into a release before then. Early-Bird registration is still open, and on the plus side, there will be some proper developers and actual users present. There may even be beer.

In the meantime, it would appear that Bamboo has finally decided that the branch passes all our tests. So it’s probably time to say hi to everyone on #opennms before they quit for the weekend.

Expect another update from me sometime in 2011….

Yesterday I saw this article, which lays out a recipe for basic Asterisk monitoring using Munin, in Matt Riddell’s daily Asterisk news. OpenNMS has included out-of-the-box support for Asterisk management for some time now, but the functionality is often hidden for reasons I’ll get to in a moment. This article provides a set of steps for switching it on.
Read the rest of this entry »

If you follow TWIO, you might have noticed that I’ve resurrected the reporting feature that I was working on a while back. There’s a lot of useful management information waiting to be unlocked in the database, and at the moment it’s difficult to do that in a way thats tightly integrated with the rest of OpenNMS.
Read the rest of this entry »

I attended a great talk at last weekend’s Ohio Linux Fest by Jorge Castro with Ubuntu about “Building a Community Around Your Project” based on his experiences with gwibber. My rough notes are below.

Read the rest of this entry »

We are preparing to reorganize the OpenNMS website to make it easier to use, and part of that process is to move my blog to its own site and to merge the developer blogs into one place: http://www.opennms.org/blog.

Sorry for any inconvenience.

Well, The last day or two has been quite a challenge. I’ve made some significant progress on the refactoring of collectd, but got held up for two key reasons. One is that I had gotten past the completely mechanical refactorings to the more sophisticated and I realized that the tests I had (or didn’t have) were not going to do it. Soooo… I had to start figuring out how to write some collectd tests. The second hindrance to progress was the fact that I had been ‘pussy footing’ around the ConfigFactories. They all but stopped my efforts to write tests and were making the refactoring difficult because some of the functions in the factory need to be on other objects.

So all this to say that today, I started refactoring the CollectionConfigFactory to simplify it. This effort has been very educational and has helped to solidify in my mind exactly what a ‘configuration DAO’ needs to look like and how it should be implemented. Hopefully I will be able to complete this soon as this is a prerequisite to abstracting the RRD code out.

Anyway, stay tuned as there is much more to come.

Matt

Well I didn’t make it on Thursday and I had Friday off for the holiday weekend. So it’s now Tuesday and I’m finally getting back to things. I spent some more time Monday working on Collectd and feel like things have gotten quite a bit simpler in there. I’ve been working primarily with Collectd.java and CollectableService.java trying to ‘find’ a good set of objects that describes what is going on in this set of code. I’ve introduced a few objects that make the code seem quite a bit simpler. Still need to work on the event handling though as well as the scheduling. THEN there’s SNMPCollector. That will be where the fun really begins.

Oh yeah, I also wanted to mention that I found a few places in the main collector that assume we are only collectiong SNMP data so I’ll have to figure out what to do about that.

Anyway, more to come,

Matt

I just recently got contacted about a post I made around 01:34, 1 December 2006 to be exact, at the time I was working quite a bit with FreeBSD. Things change, I moved on, I guess this made rounds on IRC and mailinglists.
So I got emailed and pinged on IRC about this, it suddenly wasn’t building – prior to that I just found out the community has been updating the page….

Now Venture37 went ahead after we spoke about it and completed a port for FreeBSD!

You have no idea how happy that makes me as a geek!

The link to it is here, I asked Venture to either replace or update the old manual post.

And no, I haven’t tested it, I’m just plain happy people prove that people are good </end of hippie moment>

Following up on my previous post (sadly now over a year old), I wanted to talk about how to write good error messages in Java. If you haven’t read the previous post, go read it, including Eric Schrock’s Designing for Failure post. Go on, I’ll wait …..

Now that you’ve done your homework, let’s get started!

As I mentioned in my previous post, a comment on Eric’s blog covers one of my Java pet peeves:

try {
// do something which throws an exception
…
} catch (RuntimeException ux) {
throw new ExceptionHiddenException(“Something bad happened”);
}

This is error hiding. It doesn’t tell you what is wrong, in fact it explicitly discards all information in the caught RuntimeException–information that will likely tell you what is wrong. It would almost always be better to let this exception bubble up, even to the top-level, than to do what is done above. If you need to do some cleanup, like close a (possibly) open database connection, at least wrap it in a finally clause and drop information-hiding catch clauses like the one above. Or better–fix it…

What’s better? Maybe something like this:

try {
// do something which throws an exception
…
} catch (RuntimeException ux) {
throw new ModeratelyUsefulException(“Something bad happened”, ux);
}

So, we have a new exception that includes the cause exception, but we didn’t give any context, and we still fall into the trap of two of my lesser Java pet peeves: we still have getMessage() results that are largely devoid of information and useful exception data split across multiple lines. If you want to search through your application logs for errors, you might find the “Something bad happened” line, but you’d have to see the adjacent lines to get more details, making grep less handy that it could be and making you lean towards using much more heavyweight tools to look at logs. If you include the nested exception’s message in your message along with any additional context you can add, you end up with a very detailed (albeit fairly long) error message on line that is very greppable. Your support staff will love you. And so will I.

An even better example is in a patch for a bug I submitted to Maven’s Mojo project after being irritated that part of my OpenNMS build failed with no details whatsoever. The original re-thrown exception:

throw new MojoExecutionException( “Unable to copy dependency to staging area” );

Great! No nested exception, nor does it tell me what the dependency was, nor where it was trying to copy it to the staging area. The only way I could figure this out was to fire up a system call tracer (truss/strace/etc.) and see what system call failed, or fire up Maven inside of a debugger.

Here’s the improved message with context and the nested exception:

throw new MojoExecutionException( “Unable to copy dependency to staging area. Could not copy ” + artifact.getFile() + ” to ” + newLocation, ioe );

My only complaint about the new code is that it doesn’t include the nested exception’s message in the new exception’s message. This isn’t so bad with Maven, because it will show nested exceptions if there are any. If that was going to end up in a log4j log message, you can bet that I would include the nested exception’s message. Speaking of log4j, this is how I like to log exceptions:

log().warn(“Could not connect to host ” + host + “:” + port + ” due to: ” + ex, ex);

I both include the exception in the String log message and also pass it as a second argument. This gets support staff a pretty darn good amount of information on one line, as well as a stack backtrace for developers (and those super-duper support staff in the world).

I’ll stop blabbering on and share one of the simple little classes that makes me love Spring: NestedExceptionUtils, in particular its buildMessage method. This method is used by the getMessage() methods in the Exception classes that are the root of all Spring Exceptions to return their own message along with the message for a nested exception, if any. This can create huge, long messages, but they sure are chock-full-o-information. Take a peek at the NestedExceptionUtils.java source in its elegant simplicity along with the source of NestedRuntimeException to see it in action.

Happy coding, and good luck improving your logging!

It is amazing to see all of the great stuff going on here at DevJam. After getting the RESTful stuff into the code on Tuesday night, Craig Miskell and Alejandro Galue took off like a shot adding RESTful interfaces to all or our main objects. Things were going so quick that we had to stop and take some time to review together the URLs we wanted use so we were consistent with things. We sat together along with Mike Huot, Rob Moore and a few others that stayed in or out and came up with a set of URL that will describe how to access OpenNMS model objects via REST. We posted the results on our up on the wiki so everyone could see them and start contributing. After we get these done we just need to go through and hit the configuration work. A totally awesome day even though I personally hardly did anything. These guys rock and just like Stymie from the Little Rascals… I can’t keep up!