tldr; Console app to translate datetime stamps in a logfile into time offsets.
Get it here, with a precompiled release version here.
I often find myself comparing log files to identify discrepancies or variations in execution timing. And a big problem with doing so is time stamps.
Most modern logging in .net apps happens through log4net, or nlog, or any of a variety of their clones.
In almost all cases, log files will be standard text files that look something like so:
…
2020-03-03 12:34:22 PM ERROR A problem has occurred…..
2020-03-03 12:34:25 PM INFO Informational message….
…
This works just fine under normal circumstances, but, when you’re trying to compare two log files that were captured at different times, but for the same sequence of actions, there’s a big problem.
Those timestamps will be different for every single line!
Couple that with comparing multi-megabyte log files, and locating real differences starts to look like finding that one whisperer in a herd of walkers.
In my case, what I really needed were time deltas instead of timestamps in the log files.
So instead of the above, we’d have something like:
... 00:00:00 ERROR A problem has occurred..... 00:00:03 INFO Informational message.... ...
In other words, instead of literal datetime stamps on each log entry, I’d see a time delta from each entry to the next.
This solves two major issues:
- Log entries written at approximately the same time would all have a delta of 0, so they won’t show up as differences in a file diff tool
- When the time delta is non-zero, the difference is actually meaningful. It represents a differing amount of time elapsed between the two log entries. When comparing two different runs of the same code, this can really help identify specific places in code where performance has been impacted.
Google it!
So, of course, I head to Google to see if there’s anything out there already for this. I did turn up a couple of promising log viewers, but most were either commercial, or required capturing log files into a database for review, which was a bit of overkill in my case.
I just needed something to massage two log files into a more comparable format.
Enter LogShift
After coming up empty with Google, I decided to put a little utility together just for this purpose.
I’d heard that .net core 3.x allowed for “single file executables” but I’d never had a chance to try it out. Bingo! Perfect opportunity!
And so, LogShift was born.
Get it here, and a precompiled release version is here.
First off, a big thank you to Giacomo Stelluti Scala and his CommandLineParser nuget. If you have a need for a simple and powerful command line parser that makes your console app look every bit as flexible as GIT, check it out!
The Nitty Gritty Usage Details
Just download the LogShift.exe, throw it on your desktop, and drag just about any text-format log file onto it.
When the dust settles, there should be a file in the same folder as your original log file, but with an additional “.shifted” extension.
And if you look at that *.shifted file, you’ll find that most if not all date time stamps in it have been converted to “(DURATION)hh:mm:ss”
Type LogShift –help for a typical usage screen:
LogShift 1.0.0 (c) 2020 drVenture -f, --file Log Filename to process. Can also omit the -f and just put the filename as the only argument. -d, --durationtag (Default: (DURATION)) String used to preface duration values inserted into shifted log file -s, --outputsuffix (Default: shifted) The Suffix to append to file name when writing the output file -m, --monthfirst (Default: true) true if the month should be assumed to be before the day in date stamps -y, --yearfirst (Default: true) true if the year should be assumed to be before the month and day in date stamps --help Display this help screen. --version Display version information. value pos. 0 The Filename(s) to process. Specify either one or more filenames or use the -f option.
The options should be pretty straightforward.
One question I’ve gotten is “How does it recognize datetime stamps?”
Basically, the program scans each line for a REGEXE that will match most typical date time formats.
It will just work if the datetime stamps have a 4 digit year, esp if the year comes first, because that virtually guarantees that the month comes second and the day third.
If you happen to have log files with oddly formatted date time stamps, you can use the -yearfirst and -monthfirst options to indicate year comes before the day and month, and that the month comes before the day, respectively.
The time portion should automatically get recognized with just hh:mm or with hh:mm:ss. Additionally, fractional seconds will be recognized as hh:mm:ss:fff, with a separator of “:”, “.” or “,”.
Finally, AMPM will also automatically be recognized.
Note that the first recognized timestamp in the file is recorded and all subsequent timestamps are compared to it. Any date time values that are BEFORE the first detected date time stamp will be considered “data” and as such, will not be converted to a time difference.
This is so that any logging of actual date-oriented data values will likely not get translated, which is generally a good thing. Since most data values in the file will be in the past, this should just work. Of course, if your log file contains date data in the future, well… there’s always a pull request!
Multiple files at once
To process multiple files at once, just multi-select 2 or more log files in explorer and drag them onto LogShift.exe. Either that or run it from the command line with more that one filename… for instance:
LogShift a.txt b.txt c.txt
Wrapping up
And that’s about it for this little utility. I hope it helps someone out there as much as it’s helped me.
As usual, feel free to post with any questions or comments! I look forward to hearing from you!
