Powershell script troubleshooting

[HCHTech]

This is a simple script someone posted for running in Solarwinds RMM to count the number of specific event log entries and fail if the count exceeds a threshold. I'm trying to troubleshoot this on a system here and it's not really working as desired. I'm sure it's because I don't understand dates well enough in powershell.

$startdate = ((Get-Date).Adddays(-14))
$ID = get-EventLog system | where {$_.eventid -match "6008" -and $_.TimeCreated -lt $startdate} | measure-Object
If ($ID.Count -lt 3)
{Write-Output "System Normal"
Exit 0 }
Else
{Write-Output "> 3 Unexpected Shutdowns in 2 weeks"
Exit 1001}

So, just reading through the script,

Line one sets the variable "startdate" to the value of today minus 14 days.

Line two seems to have a mistake, it sets the variable "ID" (presumably to "True") for every line in the system event log where the event ID is equal to 6008 and the timestamp on the event is less than the date set in line one. It seems to me that this line SHOULD use the -gt comparison operator so the the test is "(eventid = 6008) and (timestamp is greater than the date set in line one). Note - This comparison might also be done using either the -before or -after operators, but that escapes me as well currently.

Line three compares the total count of the True ID variables to 3. If the total is less than 3, then output "System Normal" and exit with a zero code. If the total is greater than or equal to 3, then output the error text and exit with a 1001 code.

SO, to test the logic of the script, I exported the system log on a system I have here and opened it in Excel so I could count the entries. I didn't have any eventid 6008s, but I had a bunch of eventID 7000s, so I'm testing that. In the last 7 days, I had 5 event 7000s in my system log. In the entirety of the system log, there are 144 event 7000 entries.

So I open a powershell window, and type in the first command as:

$startdate = ((Get-Date).Adddays(-7))

No errors, so to test, I type just the name of the variable, $startdate. I get the following

Tuesday, December 19, 2017 5:07:01 PM

Ok, this is exactly 7 days prior to the time I entered the command to set this variable. So far, so good.

Next, I type the second command as originally posted:

$ID = get-EventLog system | where {$_.eventid -match "7000" -and $_.TimeCreated -lt $startdate} | measure-Object

It thinks a while, but no errors, so to test, I type $ID.count

I get 144. In other words, this is the total count of all eventID 7000 events in the log. It looks like it thinks all events have a timestamp of less than $startdate.

So now I try it again but replace the 2nd -lt with a -gt, which makes more sense to me. I want a count of events that happened SINCE $Startdate, not PRIOR TO $Startdate.

So I type

$ID = get-EventLog system | where {$_.eventid -match "7000" -and $_.TimeCreated -gt $startdate} | measure-Object

Again, it thinks a while, but no errors, so to test, I type $ID.Count

I get 0, even though I know there are 5 event log entries newer than $startdate with eventID 7000. So, I don't think the value $_.TimeCreated is being picked up correctly by the EventLog call. Or, the variable $Startdate, as created here, isn't in a form that can be used in the comparison. I couldn't land on a way to output the TimeCreated value for a specific event, so I couldn't test this theory.

Maybe the timestamps for event log entries are stored in a non-date format? Remember Lotus 123? It stored dates as sequential numbers starting with January 1st 1900. So January 1, 1979 was stored as 28,854. Just a thought.

Also, giving the original poster of the script some credit, it must have worked on their system as written....

I've read through the get-help entries on get-eventlog, but of course, none of the examples are quite on point. Can someone point me in the right direction?

 

[trevm999]

Take measure-object off and then play with ID to see what you're getting

 

[HCHTech]

Well, it appears the original coding using -lt is correct. rather than trying to run it locally, I just found a system I'm monitoring with some unexpected shutdowns in the log and ran the script in various forms from the dashboard.

So....when testing event log events, NEWER timestamps are "less than" OLDER timestamps. I don't know yet exactly how eventlogs store their data, but when you export an event log to a csv file, the "Date and Time" field is shown as

12/26/2017 15:33

In the script, the $startdate variable is shown as

Tuesday, December 19, 2017 5:07:01 PM

And based on the now-correct counts I'm getting, it appears that for powershell calculation purposes anyway "12/26/2017 15:33" is less than "Tuesday, December 19, 2017 5:07:01 PM"

 

[trevm999]

I don't have a Windows computer handy currently to test this script out, but my guess is that
the TimeCreated property is a string rather than a datetme object. For the reliable results you should compare a datetime object to a datetime object.

 

[HCHTech]

Perhaps you're right. I'll look into converting that object.

Haha- - love your link. I'm thinking "Who in the world would name their company Fascist Computers?" Hook, line & sinker, boy - hook, line & sinker.

 

[HCHTech]

Further notes from a helpful soul on the LinkedIn group (yea for helpful communities) - it appears that the TimeCreated property is not, in fact, a date object, so that's the problem. The TimeGenerated property is, so that should work. I'll test it out this weekened and post the final result.

 

[HCHTech]

As promised, and with Daniel Eggleston from the Linked In SW group doing most of the heavy lifting, here is the final script. It looks for a particular event (in this case 6008), and fails if it finds more than X of these events in Y days. If so, it fails the check and prints the most recent Z events to the dashboard. This is a great accomplishment, and can be used as a template for any event.

In this example, X = 2, Y=14 and Z=3

Note also that the "-First" variable is what pulls the most recent Z events to the dashboard. You can use "-Last" if you want to pull the oldest Z events of those found.

<#
.NOTES
===========================================================================
Created with: SAPIEN Technologies, Inc., PowerShell Studio 2017 v5.4.145
Created on: 1/2/2018 11:46 AM
Created by: Dan Eggleston
Organization: IT Right
Filename:
===========================================================================
.DESCRIPTION
Flags excessive eventlog id's within a given time frame, for use in SolarWinds MSP.
#>
$startdate = ((Get-Date).Adddays(-14))
$Result = get-EventLog system | where { $_.eventid -match "6008" -and $_.TimeGenerated -Gt $startdate }
$Current = $Result | Select -First 3
$ID = $Result | measure-Object
If ($ID.Count -lt 3)
{
Write-Output "System Normal"
Exit 0
}
Else
{
Write-Output "Total Excessive Unexpected Shutdown(s) = $($ID.count)" $Current.Message
Exit 1001
}