27 Aug 2015 - Introducing the Archon K2 Log Watcher log watcher general
Have you ever had the need to monitor K2's log files, and gotten tired of manually reloading the file in Notepad or refreshing Event Viewer? We run into this situation on a daily basis, and have often felt the need for a lightweight, stable utility which can monitor K2's logs for us.
Archon K2 Log Watcher is our solution to this problem. It can watch as many K2 log folders as you throw at it, giving you both a summary of the folders as well as any new entries which K2 appends. You can download it for at the K2 Community Site. And, perhaps best of all, this utility is completely free.
A bit of background info on K2 logs
K2 blackpearl offers a flexible logging system, which allows it to log events to a number of different log targets (such as the file system, the Windows Event Log, the database, etc). From our perspective, the most lightweight log to monitor is the one that resides on the file system, so we will focus on that.
By default, K2 is set up to log into the Host Server's directory (typically under
C:\Program Files (x86)\K2 blackpearl\Host Server\Bin). If you watch this folder in Explorer, you will notice K2 creating log files here, with names such as
HostServer150813_2.log. The name essentially represents the date on which the log file was started by K2, as well as the file number for that day. The file number can increment if the log fie reaches a maximum size (5 MB by default), or K2 is restarted.
These log files are comma separated (CSV) text files, which can be opened with any text editor. Each log event has a certain level associated with it, which represents the level of severity of the event. By default, K2 only logs
error events, although it can be reconfigured all the way up to the
debug level. This produces extremely verbose logs and can severely impact the performance of your server. As a result, it is recommended that you use debug logging with extreme care; it should not be enabled for extended periods of time, or on servers which are mission critical without careful monitoring.
Where the Archon K2 Log Watcher fits in
Regardless of whether you have K2 set up to only log
error level items, or want to monitor full
debug logs as they happen, our log watcher utility can be of help.
Automatic monitoring: The first tab in the utility is the Servers tab; this is where you register one or more K2 log folders to be monitored. Folders in this area will automatically be monitored for any changes, unless they are currently marked as disabled. The time of last activity is shown, and highlighted in green for very recent updates. The File Count and File Sizes values give an indication of whether the logs have grown excessively and need to be cleaned up.
As soon as the utility is notified of a file system change in any of the log files it is monitoring, it will perform a lock-free read of the last bit of the file. This gives us the newly added rows, which are then parsed and appended to the bottom grid.
Manual log browsing: You can also open an entire existing log file in the utility. To do this, either click the Open File... tab, and then drag the log file in, or highlight the log folder in the summary grid and click the Open Latest File button. In both cases, a new tab will open with the full file shown as a grid. The utility will attach itself to this file, and import any new rows that K2 adds, as with the Combined View.
Viewing log details: It is also possible to view the detail entry or raw text for one or more log entries by enabling the Show Details option, and selecting the rows in the log entry grid:
The first tab shows the detail entry; this can often be a bit too long to comfortably read in the grid, and can also often span multiple lines. The raw text tab shows the log entries as they were in the log file, in case you need to forward specific lines to support personnel.
With this utility, a number of technical aspects were quite critical. Since it's meant to be run on (or at least near) K2 servers, it is as non-intrusive and lightweight as possible.
In terms of memory usage, we limit the default Combined View to 500 records and have stayed away from any fancy 3rd party controls or graphical components. This means that under normal usage, memory usage stays under 20 MB even after weeks of staying open on a server. This figure does goes up if large log files are opened manually by the user, although memory usage still typically stays under 200 MB.
In terms of IO, the two primary goals were to stay away from polling the log folders for changes, as well as never locking the log file (and thus preventing K2 from writing a record). The .Net
FileSystemWatcher object is used to notify of any file system changes we're interested in; this works around the need for polling. We've found this method to be extremely reliable and lightweight. For the reading of log files, the utility is aware of the previous file size, and only performs a read from that point onward using a lock-free read. To ensure this does not affect K2's ability to write to the log file, we've intentionally kept the log file open in this way to ensure it could still be written to by K2.
The utility rarely uses much of the processor; the most intensive activity is the parsing of the CSV, but under normal circumstances this only happens for a relatively small number of records. Even manually opening a 5 MB log file happens in less than one second, and taking up a fraction of one CPU core.
So, if you find yourself needing a lightweight and stable tool to do spot-checks of K2 log files, or happen to be in the middle of some
debug level K2 log tracing, the Archon K2 Log Watcher is a great tool for the job. It's completely free, and available on K2 Community Site.