Tag Archives: Fun with PowerShell

How do I: Create an Event View that excludes a particular Event ID

I had a large enterprise customer recently who was monitoring ADFS with the default management pack. They liked being able to glance at the event view which gave them a single place where they could look at the ADFS events occurring across their environment. They were using this event data as part of their correlation and tuning process to determine if there were additional actionable events that were being missed for their unique infrastructure. The eventual goal being to stop collecting the events altogether and only have alert generating rules/monitors in place for patterns of events that they cared about.

01

They quickly found that at least for their environment some of the events being collected were essentially noise, and they asked how to adjust the view so it would exclude one particular event.

This is one of those sounds really easy and of course the product should do this out of box questions that SCOM has never really had a great answer for.

If we take a look at the view it is populated by the following criteria:

02

And if we dig into the corresponding rule that collects the events we find a wildcard regex-style collection rule targeted at the ADFS log:

03

04

05

Since the collection rule is part of a sealed MP the best we could do at the rule level is to shut off this collection rule, and create a new collection rule with a modified wildcard expression such that it would collect everything the old rule did with the exception of the event ID the customer doesn’t like.

The problem with this solution is it isn’t particularly efficient/self-service friendly. If next week the customer realizes there is an additional event they want excluded the AD team has to contact the SCOM team and request further modifications.

In an ideal world the exclusion would be possible at the View level, but if you ever dig into modifying the classic OpsMgr views you will find that while you can use WildCards for some fields like Event Source to perform exclusions:

06

The same is not true for event ID’s, where wildcard exclusions are not allowed:

07

I briefly toyed with the idea of making modifications to the MP at the XML level to allow exclusions as I have occasionally done in the past to hack a subscription into meeting a customer need, but in this case such a solution doesn’t really fit. The customer needed something that was easy for them to change as they gradually winnow down the list of events they see to only the ones they care about.

They needed something that was extremely easy to edit.

Enter PowerShell and the SCOM SDK.

The first solution I put together for them to test was the following:

PowerShell Grid Widget

08

with a where-object {$_.Number -ne 31552 -and $_.PublisherName -eq “Health Service Modules” } I used a SCOM publishername since I didn’t have any ADFS events in my test environment and I wanted to use something that I could confirm that the exclusion was working as expected: 

11

Everything looked good the event I wanted excluded was dealt with properly  (Description dataObject is commented out in the code for this screenshot to make it easier to view. With Description uncommented each event takes up more lines of screen real-estate. I recommend creating two views, one with description commented out, and one where it is uncommented so customers can easily toggle between views.)

12

And if we remove the -ne $_.Number 31152 I get results as below with the event present:

10

In theory this should be all we needed, but when my customer tested out the script nothing happened. After a little bit of head scratching it became apparent what the problem was.

We were calling Get-SCOMEvent | Where-Object

which means we were telling the OpsMgr SDK to please go retrieve every single event in the OpsDB, and then once you are done with that we are going to pipe the results to a Where-Object and tell you what we really need.

In my relatively small test environment this wasn’t that big of an ask and the results returned quickly.

In my customer’s environment with thousands of servers and friendly event generating MP’s like the Exchange 2010 MP, getting every event in the OpsDB was basically a great way to enter an endless loop of dashboard timeouts with nothing ever being displayed.

So we needed to filter things down a bit up front, before piping to the Where-Object.

If you search the blogs you will find that Stefan Stranger has a nice post describing how to deal with this issue when calling the Get-SCOMAlert cmdlet with a Where-Object. Basically you use Get-SCOMAlert -criteria and then pipe to a Where-Object if still needed.

Unfortunately, Get-SCOMEvent doesn’t have a -criteria parameter because that would make things too easy and intuitive.

It does, however, have a -rule parameter which looked promising:

13

First I tried passing it a rule Name, followed by a second try with a rule GUID for an event collection rule I was interested in. In both cases I got a nice red error message:

14

While a little a cryptic it is saying that I am passing a parameter of the type string, and it wants a special SCOM specific rule type.

To give it what it wants we need to first retrieve the -rule parameter using the get-scomrule cmdlet and then pass it to get-scomevent as a variable:

$rule = get-scomrule -DisplayName “Operations Manager Data Access Service Event Collector Rule”

15

$rule = get-scomrule -DisplayName “Operations Manager Data Access Service Event Collector Rule”

get-scomevent -rule $rule

16

So our final script would look something like this: (I have added some additional filtering to be able to allow if you just want events from the past hour. *Keep in mind this date/time filtering doesn’t increase the efficiency of the script since it occurs after the Where-Object, the only thing making this script more efficient is that we are first only pulling back events collected from a specific rule*)

$rule = get-scomrule -DisplayName “Operations Manager Data Access Service Event Collector Rule”

$DateNow = date

#Modify the .AddMinutes below to determine how far back to pull events

$DateAgo = $DateNow.AddMinutes(-60)

#$_.Number -ne(not equals) is used to indicate the event number that you want to exclude from the view

$eventView = Get-scomevent -rule $rule |where-object {$_.Number -ne 17 -and $_.TimeGenerated -ge $DateAgo -And $_.TimeGenerated -le $DateNow}|Select Id, MonitoringObjectDisplayName,  Number, TimeGenerated, PublisherName, Description| sort-object TimeRaised -descending

foreach ($object in $eventView){

     $dataObject = $ScriptContext.CreateInstance(“xsd://OpsConfig!sample/dashboard”)

     $dataObject[“Id”] = [String]($object.Id)

     $dataObject[“Event Number”] = [Int]($object.Number)

     $dataObject[“Source”] = [String]($object.MonitoringObjectDisplayName)

     $dataObject[“Time Created”] = [String]($object.TimeGenerated)

     $dataObject[“Event Source”] = [String]($object.PublisherName)

     $dataObject[“Description”] = [String]($object.Description)

     $ScriptContext.ReturnCollection.Add($dataObject)

}

And then the ADFS code would look like this, though event 17 was not the event they wanted to exclude:

$rule = get-scomrule -DisplayName “Federation server events collection”

$DateNow = date

#Modify the .AddMinutes below to determine how far back to pull events

$DateAgo = $DateNow.AddMinutes(-60)

#$_.Number -ne(not equals) is used to indicate the event number that you want to exclude from the view

$eventView = Get-scomevent -rule $rule |where-object {$_.Number -ne 17 -and $_.TimeGenerated -ge $DateAgo -And $_.TimeGenerated -le $DateNow}|Select Id, MonitoringObjectDisplayName,  Number, TimeGenerated, PublisherName, Description| sort-object TimeRaised -descending

foreach ($object in $eventView){

     $dataObject = $ScriptContext.CreateInstance(“xsd://OpsConfig!sample/dashboard”)

     $dataObject[“Id”] = [String]($object.Id)

     $dataObject[“Event Number”] = [Int]($object.Number)

     $dataObject[“Source”] = [String]($object.MonitoringObjectDisplayName)

     $dataObject[“Time Created”] = [String]($object.TimeGenerated)

     $dataObject[“Event Source”] = [String]($object.PublisherName)

     $dataObject[“Description”] = [String]($object.Description)

     $ScriptContext.ReturnCollection.Add($dataObject)

Hopefully this helps save a little bit of time for anyone else who comes across a question like this one.

Tagged , , , , , ,

SCOM Management Pack Authoring Training: A different approach (Part I)

It would be wrong to start off any discussion of SCOM authoring without pointing out some of the great resources that do exist on this topic:

-The MSDN Operations Manager Development Kit  (Excellent for those of the developer persuasion, but less friendly for those whom PowerShell is much more comfortable than C#)

-Steve Wilson’s classic  AuthorMP blog posts. (A bit dated, but they remain a source of some the most insightful posts on various aspects of OpsMgr)

-Jonathan Almquist’s posts both in his old, and new home

-Brian Wren has put together great video content + some awesome posts over the years

-Graham Davies and the Manageability Guys in the UK have some awesome posts: 1, 2 , 3,  4, 5, 6, 7, 8

-If you happen to be a Microsoft Premier customer there is a great workshop on SCOM Authoring with Visual Studio that came out last year

-And countless members of the community like Tao Yang &  Raphael Burri who have written high quality MPs that can serve as a primer to those who want to dig in and start authoring. (Since I started writing this post awhile back I believe Tao has hosted some MP authoring training, I haven’t gotten a chance to look at it yet, but once I do I will add a link.)

-There are of course many other worthwhile posts throughout the SCOM community both inside and outside of Microsoft, but that is what Bing, Google, and DuckDuckGo are for.

But despite all that great content out there. Management Pack authoring can be an extremely difficult skill to acquire. At least from my own perspective – starting out in MP Authoring was really hard. Even after having read the vast majority of the published info on MP authoring, and watching all the videos that are out there I can’t say that I felt particularly confident to wade into Visual Studio and start writing management packs. I understood the basic mechanics, but I lacked the ability to fill in the inevitable gaps of knowledge to be able to author custom MPs that met real enterprise level business needs.

Unfortunately, a lot of the best how-to examples and step-by-step tutorials tend to be a little generic. I suspect this is done intentionally to minimize complexity as much as possible. The hope being that a budding MP author can learn the fundamentals and then extrapolate from the excellent guide on “how to author a custom MP that no one would ever import into their real environment”, and later apply this knowledge to some real-world problem.

My brain tends to not work that way. It is easily distracted by shiny objects and Wikipedia. To learn I need concrete real-world examples, problems, deadlines.  For me the most valuable training in MP authoring came not from all the guides and links referenced above, but from a single one hour Lync call with a senior colleague at Microsoft. I had a specific question that I didn’t know how to solve and step-by-step over the course of an hour he walked me through how to build an MP that addressed that request.

I really wish there were a ton of Authoring videos like that call out there following the simple formula:

Real-world enterprise monitoring problem that is not currently addressed by a Management Pack + Screen capture video that walks through the process every step of the way.

Sadly as fun as it would be to simply complain, I think there might be some small value in me adding what little I know about MP authoring to the general ether following the format above.

I had toyed with the idea of doing Livecoding.tv or live Twitch.tv sessions, but my home internet connection these days is DSL so the upload streaming experience is lacking, so these will be pre-recorded sessions with some light editing.

The first installment can be found here:

MP Authoring 001

I have about 10 videos planned so far and if the first few are of any use to the community I will shoot for publishing a new video every two weeks until I run out of ideas.

Tagged , , ,

Best Practices: Agent Remediation Tool

This is a proof of concept script consisting of a mix of PowerShell with some .NET for a GUI that can serve as an automated playbook for agent remediation.

Typically I prefer to remediate agents via the SCOM console, but there are instances where an agent is locked down such that remote management is not possible, and the SCOM Team may not have access to remote a server and fix an agent. This script empowers non SCOM sysadmins, DBA’s et cetera to be able to perform basic troubleshooting on their agents without the fear of accidentally deleting the wrong thing.

-The script must be run in PowerShell as an administrator

-This script is compatible with PowerShell 2.0, 3.0 & 4.0

*There are some dependencies on the .NET Framework. It is designed for .NET 3.5, but in testing it does work with .NET 2.0, though it will throw some fun red errors for certain non critical display elements which will not be able to load.

This script is designed for SCOM 2012 R2 Agents

opsconfigtool

Functionality:

1. Restart SCOM Agent (This restarts the Microsoft Monitoring Agent)

2. Flush SCOM Agent Cache (This stops the SCOM agent, Clears the Health Service State Agent Caches, Starts the agent and rebuilds the Agent Cache)

3. Uninstall SCOM Agent (This queries WMI to determine appropriate GUID that is associated with the SCOM agent installation and then passes this GUID to an automated uninstall.)

4. Install SCOM Agent (This is a placeholder for either manual agent install instructions, or it can be adapted to call a function to kick off a command-line based agent install assuming agent media is on an accessible UNC file share)

*It appears I neglected to include the link to the script, it can be downloaded here: TechNet Gallery *

 

Tagged , ,

Comic Relief: PowerShell 5.0

powershell

Happy Friday All!


The original unaltered Comic can be found here: Abstruse Goose (Please be mindful of AG’s CCommons license)

Tagged , ,

Troubleshooting: SCOM reports yield weird data/what the heck does 9.221E+07 mean?

Eventually when running a report in SCOM you are going to end up with a report like the one below.

01

At first glance everything looks okay. But then you start looking at the data that was returned and it can sometimes be a little confusing.

02

Usually the questions I get from customers ranges from “I think this report is broken” to “what the heck does 9.221E+07 mean?”

Fear not, reporting is not broken and 9.221E+07 is not nearly as confusing as it may seem. Basically, what is going on is that the dataset you have returned is so large in regards to the number of digits that in order to display it in a meaningful way the system is presenting the data using some shorthand commonly known as scientific notation. All you need to understand is that +07 indicates the number of times the decimal point would need to be moved to the right to display the full number.

So 9.221E+07 = 92210000

And if we look at the top of the chart we will note that the particular performance counter that we are reporting on is being returned in Bytes so we are dealing with:

92210000 Bytes

For those of you who like me are not particularly mathematically inclined and prefer to leave conversions to someone else I recommend using the wonderful built-in functionality of PowerShell.

If you enter 9.221E+07 and hit enter it will automatically output the full value for you:

03

If the original unit–in this case Bytes–is not your unit of choice and you want to know what the value is in MB  just enter the value in scientific notation form and then divide by 1 MB:

9.221E+07  / 1MB

04

Same goes for GB

9.221E+07  / 1GB

05

 

Tagged , , , , , ,