Update, Eventum/MySQLTicketing Integration

Well I know I haven’t updated in a while. I have a whole bunch of links that I’d like to comment on, but things have been horribly busy. You can find the links in my “1-toblog” folder on del.icio.us (prefixed with “1-” so it shows up at the beginning of my bookmark menu).

In monitoring land, I’ve paused my Hyperic HQ VM as I wasn’t too pleased with how the features panned out. I was invited to beta test Groundwork Open Source 5.2b, but I’m not crazy about the open-ness of a non-public Beta, and am honestly not that intrigued by the small feature set (though, admittedly, they do need more documentation on the F/OSS version). I’d still like to try them all, especially Zenoss Core, but I’m pretty busy with class, and things are heating up at work and with a few consulting projects.

In my “spare time” (read: staying up until 5 AM and somehow still getting up for work at 9) I’ve been working on something that’s been bugging me for a while – getting Nagios to automatically open and update tickets in Eventum, the ticketing system that I (and MySQL) use. The general idea is to use a “glue” script, written in PHP (Eventum’s native language). It will (hopefully) keep track of which hosts/services it has opened tickets for (and what the ticket ID is), and decide from that whether to open a new ticket or, if one already exists for that host/service, update it. It should also handle changes to assigned user/group, update categories, priorities, etc. This will all be based on a DB table that maps problem severities and hosts/services to the users, groups, categories, and priorities that they should be assigned.

The biggest problem is that I’m not a whiz at object-oriented PHP, and like any good OO program, Eventum is broken down into dozens of objects, classes, and files. With the help of the Xdebug debugging extension for PHP, which prints full debugging output including stack and function call traces, I’ve been able to *finally* – after about four hours of work – write a simple little 15-line script that uses ONLY existing Eventum classes, unmodified (except for a separate init.php with some stuff commented out), which gets a list of users assigned to an issue. From here, it shouldn’t be difficult to get full issue information and then, hopefully, add and update issues.

I have a basic description of the project on my wiki, and the current (development, so could be broken) source code in CVS, which can be seen through ViewVC on my site.

Stay Tuned!

Bug/Issue Tracking – Update / Review of Eventum

So, after many hours of investigating potential bug/issue tracking packages, I have chosen one. And gone live with it, all in one weekend. After much evaluation, my final choice was Eventum from MySQL (I’m using the current version, 2.0.1). I will admit that it is not perfect. There are some features I wanted that weren’t there, and the documentation is awful. But it’s written in PHP.

That being said, it is very much a community-driven project. The support on the mailing list, both from other community members and from Bryan Alsdorf, has been wonderful. I had some issues viewing the help documents, specifically those in pop-ups, but I’ve been told that most new documentation has been moved to the wiki at the link above.

The project provides for advanced bug- and issue-tracking including time tracking, file attachments, and customizable statuses, priorities, and categories. It also provides email integration, both in terms of sending email alerts and opening issues (as well as updating and tracking) by parsing incoming emails. The email alerts worked out-of-the-box, but I did not configure parsing of emails.

One of my main requirements which was met perfectly by Eventum is its’ ability to easily handle multiple projects. It also has a built-in capability to allow anonymous issue submission (enabled on a project-by-project basis). You can define custom fields for issues on a project-by-project basis, and set them as required fields for either registered users, anonymous users, or both.

One feature that I found lacking was the possibility for a user to view all of the open issues assigned to them. Currently, all user interface is on a project-by-project basis. Therefore, listing of open issues is only available for the currently selected project. To cope with this, I hacked together a little PHP script that just queries the database for issues by user and displays it in a simple little page.

One of the major features about Eventum that caught my eye was integration with a version control system (SCM, as far as the Eventum docs are concerned). The feature list stated integration with CVS and SVN. When I actually looked into integrating it with CVS, however, the problems began. Firstly, the javascript-based help popup would not display anything, let alone the proper page. Installation was otherwise perfect under Apache2. I was forced to browse to the included HTML file manually and check it out. The overview seemed simple enough – throw a script in your CVSROOT loginfo file, update a few variables in the web-based Eventum configuration, and you’re off to the races. Reading on, I found that the installation page was a confusing jumble of references to a deprecated perl script and the current PHP script to call from loginfo. Furthermore, database access is provided by having the script called from loginfo parse the logging information and then *run a HTTP GET* on a local script served in the Eventum web directory. This added level of abstraction not only confuses me to no end, but also introduces the possibility for malicious users to insert data in the Eventum SCM database simply by visiting a well-known URL.

More importantly, the script provided to be called by loginfo seems to expect the old CVS logging format, not the new one being provided by my installation of CVS 1.12.12. While annoying, this ended up being a minor fix in the provided “process_cvs_commits.php” – I simply had to rewrite the argument parsing code so that it no longer expects the file, old version, and new version (%{sVv}) information to be space-separated on the command line in the form of s,V,v tuples, but expects everything to be space-separated. I should be cleaning up my fix a bit and submitting it for inclusion in the next release.

Once patched, CVS integration works perfectly. Simply append an identifier to the end of your commit log message, such as “(issue: 21)” or “(bug: 21)” and the commit will be automatically associated with the issue of that number. When viewing an issue, a list of associated CVS commits can be viewed.

It must be remembered that, as I have read, Eventum is used internally by MySQL. It is, therefor, a mature project that is well tested in one circumstance. I believe that it is mature and generally well-working (though I’ve heard reports that the 2.x tree isn’t as stable as the older versions, which are most likely still in use at MySQL). It must also be noted that the issue with CVS integration is most likely only with the newer CVS versions using the new logging format (I don’t know when the switch was made) and will probably not be noticed in older projects which have established CVS systems.

Now, for the opinion section. Eventum has thousands of features. I have detailed every issue that I have, which total about five. I found it to be a stable system, ready-to-run out of the box. Overall, I think it has the best feature set of the open-source bug tracking systems that I surveyed, which are probably most of the ones out there. It’s a great project which I’d recommend to anyone, though if you want more advanced features (like integration with CVS, or things not offered such as anonymous issue viewing) you should be comfortable with coding in PHP until someone makes patches available.

Bug / Issue Tracking Software

I recently posted the following to the DevShed forum in hope of an answer:

I’ve spent the last 48 hours reading up on bug tracking software. At the moment, I’m using the SourceForge tracker for my public projects and a custom (lightweight kludge) system for internal and customer-specific stuff.

First off, I’m looking for something that’s F/OSS and no-cost. Beyond that, it should have a web-based interface, preferably coded in PHP as that’s what I’m most comfortable writing code for.

I’ve looked into many possibilities. At the moment, they’re pretty much narrowed down to RT, FlySpray, Codetrack, Trac, and a few other america’s promise rorabaugh3possibilities. I’ve immediately disregarded anything without an online demo, so some may have slipped by me.

I can do some coding to add a few features as needed, but here’s what I’m looking for:

1) Web-based interface
2) Ability to handle multiple projects
3) Ability to handle modules/branches of a project – specifically to be used for projects which also have custom, user-specific modifications on a branch.
4) *multiple* bug submission forms – ideally, one for team members/technical users, one (simplified) for non-technical end users, and one for non-technical users which also collects browser/OS information, as many of my projects are web-based.
5) If possible, integration with CVS or SVN so that I can track commits related to a specific bug.
6) Fine-grained access control. There should be a way to submit an anonymous (unregistered user) bug. Anonymous submissions and registered user submissions should be authorized on a project-by-project (or branch-by-branch) basis. There should also be access control on viewing *any* information at all for certain projects/modules (so the same system can be used for internal-only projects).

I’m sure that I have a lot of other requirements, but these are the ones that stuck out in my mind. Also, though I don’t have a system up and running yet, there’s a possibility I may be moving to LDAP for all auth, so something where I could drop in a custom auth module would be a plus.

Any suggestions?

I know I probably won’t find *all* of this in one place, but I’d like to minimize the amount of programming I have to do. After all, starting a new programming project to track bugs in an existing one isn’t especially efficient.”

We’ll see what happens out of it.

I recently received a few emails from someone attempting to get PHP EMS Tools up and running for his EMS agency. I did a lot of correspondence via email, and ended up in a bit of a predicament. First off, since CVS is down, I had to make a copy of the latest development sources from their live location on my web server, and revert to using RCS for the development work. Secondly, he wasn’t familiar with the SourceForge trackers and didn’t have an account with them, so I had to manually enter the bug reports. This was not good. I need something better.

I know that a lot of ticketing systems use e-mail interfaces, but I have no interest in that – I want to be able to require certain information for certain people. Moreover, I have a few main issues with most of the ticketing software I’ve seen:

  1. It’s either designed for development team members or non-technical end users. I want a system that can present an end-user with a simple, non-jargon-filled issue submission form, yet put that right in with issues entered by developers, which would have a greater depth of technical content. Developers should be able to follow-up on users’ tickets and add technical details as needed. Most importantly, since most of my projects are web-based, I want to add JS to the form which will include the users’ browser and OS information in the ticket.
  2. I don’t have a server farm. I don’t expect to be getting hundreds of bugs a year, let alone week. As such, it makes no sense for me to run a separate system for every project. I want a system that can handle multiple projects.
  3. Furthermore, I want it to handle internal projects as well – server upgrades, hardware issue tracking, etc. As such, it should support fine-grained access control, both for viewing and submitting bugs. Some projects should allow anyone to view bugs, and allow a non-registered (provided the email address is valid) user to submit. Some projects shouldn’t let anyone but me see *anything*.
  4. I need support for branches within projects. While PHP EMS Tools only has the trunk publicly available, I now have two branches, one customized for my organization and one for this gentleman’s organization. I want a system that can understand that these are branches of the main project, and that bugs may pertain to only one of these branches, but that all of the bugs from the trunk will pertain to them.
  5. I would love a system that can somehow integrate with CVS or SVN, so that I can associate a specific commit with a specific bug(s).

So far, I’m considering FlySpray, RT, and Trac. I’m also looking at Eventum, Codetrac, and BugZero. I’m interested in OTRS as I’ve heard good things about it, but I can’t seem to find a web-based demo. Most likely, I’ll probably give two or three of them a whirl, and then add in the custom code to do what I need. Of course, none of this will happen until I either get CVS back up or migrate to SVN.