Comic Credit: Abstruse Goose
For some background. The question that led to this post was in regard to being able to access properties that SCOM was discovering in order to detect config drift in some networking hardware. Normally when I get a question like this my first answer is don’t use SCOM for this–use OMS, SCCM, or some other tool designed specifically for this purpose. With that said, they had a specific use case that made sense, and SCOM was already collecting all the properties they cared about as part of a 3rd party Management Pack so the primary goal became giving the customer a better picture of where this data gets stored and the easiest way to access it.
First way of getting at discovered property data is via the OperastionsManager Database (The usual caveats about directly querying the OpsDB not being recommended or supported apply.)
There are tables called Dbo.MT which contain the various properties associated with a certain class of object.
If I look at something like SQL 2014 Databases I find the following: (There are more properties, but they get truncated off screen)
Select * from dbo.MT_Microsoft$SQLServer$2014$Database
To make this a little more meaningful we need to pick which tables we are interested in and join FullName from BaseManagedEntityID so we can understand which systems these databases are associated with. For this I wrote the following query:
MT.DatabaseName_3AD1AB73_FD77_E630_3CDE_2CA224473213 As ‘DB Name’,
MT.DatabaseAutogrow_E32D36C4_7E11_62BE_D5B4_B77C841DCCA1 As ‘DB Autogrow’,
MT.RecoveryModel_772240AD_E512_377C_8986_E4F8369BDC21 As ‘DB RecoveryModel’,
MT.LogAutogrow_75D233F6_0569_DB26_0207_8894057F498C As ‘LogAutogrow’,
MT.Collation_4BC5C384_34F3_4C3F_A398_2298DBA85BCD As ‘Collation’,
FROM dbo.MT_Microsoft$SQLServer$2014$Database MT
JOIN dbo.BaseManagedEntity BME On BME.BaseManagedEntityID = MT.BaseManagedEntityId
Which gives this output:
You could also get at similar data through the SDK via PowerShell (This would technically be the officially supported technique, though sometimes not as flexible as SQL). To do this you would use something like:
$WindowsServerClass= Get-SCOMClass -Name Microsoft.SQLServer.2014.Database
$ServerObjects = Get-SCOMClassInstance -Class $WindowsServerClass | Select Fullname, *.DatabaseName,*.RecoveryModel,*.DatabaseAutogrow,*.LogAutogrow,*.Collation
This will give you results that look as follows: (I just arbitrarily picked a few properties, there are more available that you can look at with either I get-member or | Select *
From there we can make things a little more readable with the following:
$WindowsServerClass= Get-SCOMClass -Name Microsoft.SQLServer.2014.Database
$ServerObjects = Get-SCOMClassInstance -Class $WindowsServerClass
$ServerObjectsB = $ServerObjects | Select *.DatabaseName, *.RecoveryModel, *.DatabaseAutogrow, *.LogAutogrow, *.Updateability, *.UserAccess, *.Collation, *.Owner, *.ResourcePool | FT
From there we started playing around with ways to quickly identify differences:
This is still a work in progress, but I figured I would share in case this can be of use to anyone.
Fascinating talk from Jeffrey Snover on the road from the Monad Manifesto to PowerShell to Nano Server.
I recently had a question from a customer on how to create Wildcard Service Monitors + Recoveries. The Service Monitor from the Monitoring template and a simple Unit Monitor for services both require an explicit service name, no wildcards allowed. For most services this is fine, but there are some applications which do fun and interesting things like concatenate computername + service to create a unique service name. This creates a bit of a problem for monitoring. You can create individual monitors, but if you have hundreds of services each with unique service names that follow a particular pattern creating hundreds of corresponding monitors could get a little time consuming.
Brian Wren has a great article from back in the SCOM 2007 R2 days that answers part of this question, but when I went through the steps I found it needed some slight tweaking and updating for SCOM 2012 R2. Once that was complete I also needed to come up with a simple low overhead wildcard service recovery for when one of the services stops and needs to be brought back online.
Below are my steps:
Launch SCOM Console
Tasks – Create a Management Pack
Enter Name + Description – Next – Create
Right Click Windows Service
Add Monitoring Wizard
Enter a Name, Save to the MP you just created
Enter Service Name. Use % for a wildcard representing multiple characters. As I don’t have any unique services in my environment I am using m% to demonstrate how this can work. For the rest of these instructions wherever you see m% keep in mind that you need to modify this value to match your unique service name wildcard value. Be careful using too broad a wildcard could create a lot of noise and load very quickly in your environment.
Pick a Target Group. In this case I am using All Windows Computers. Generally you would want to target this as precisely as possible. Leave Monitor only automatic service checked
Select Management Packs
Select the Custom Management Pack you just created
Select Export Management Pack
Select a location to save the unsealed xml file
Open the File in your XML editor of choice (Notepad will do, but Visual Studio or Notepad+++ will make it a bit easier to read)
Search the file for your wildcard in my case this is M%
We’ll be making a few replacements in the code.
You will be modifying:
<DataSource ID=”DS” TypeID=”MicrosoftWindowsLibrary7585010!Microsoft.Windows.Win32ServiceInformationProviderWithClassSnapshotDataMapper”>
To: (remember to also swap the m% with the appropriate value)
<DataSource ID=”DS” TypeID=” MicrosoftWindowsLibrary7585010!Microsoft.Windows.WmiProviderWithClassSnapshotDataMapper”>
<Query>select * from win32_service where name like ‘m%'</Query>
Save the .xml file
Go back to the SCOM console – ADministration
Import Management Packs
Add from disk
Select the newly modified .xml file
To check and confirm that the discovery associated with the wildcard monitor is working.
Change Target Type
Select Custom Target
A few minutes after importing the updated pack you should see services discovered.
Now we need to create a wildcard recovery. If this was a single service recovery I would create a standard SCOM recovery and call net.exe and pass a start command with the service name. Since this is a wildcard service we have to do things a little differently as I don’t know of a way to pass wildcards to net.exe. (We could use PowerShell, but for this I want to try to be as light weight as possible from an overhead perspective even if that means sacrificing some more advanced error handling that we could easily add in with PowerShell.)
Go to Authoring:
Select Windows Service
Right Click your custom Service Monitor – View Management Pack Objects – Monitors
Expand Entity Health -Availability – Right Click the Basic Service Monitor Stored in your custom MP – Properties
Select the Diagnostics and Recovery Tab
Under Configure recovery tasks select Add – Recovery for critical health state
Select Run Command
Name your Recovery – Check the Boxes for run recovery automatically and recalculate monitor state after recovery finishes
Enter Full path to file
Parameters: (originally I used slightly different param, but found that while it worked in the command line it failed when run as recovery. This method works consistently)
/interactive :off service where “name like ‘m%'” call startservice
You should now be all set to test out and validate your new monitor.
Just keep in mind that a wildcard discovery if targeted incorrectly (too broad a wildcard, too broad a target group, or both) you could have the recipe for a single monitor that can cause a lot of churn/perf issues/and noise in your environment. So be cautious and test very carefully. Make sure you have a good sense of the number of objects this monitor will pick up not just in your test environment, but once you move it into production. To be clear I would never recommend using a wildcard as broad as m% in production. This picks up way too many services that you likely don’t care about.
Also please note that the recovery is equally general as the monitor if not more so. It is also not checking to see if the services that apply to it are already started. In the case of my example m% picks up a bunch of services. If a single service matching that criteria goes down, the recovery will attempt to recover/start every single service that matches that criteria m%. So if you are building your wildcard service monitor to pickup multiple services on single system that follow a common pattern, a failure of one will result in an attempt to recover all.
In theory this shouldn’t be a problem. The method I am using is extremely lightweight and if the service is already started in the background the service will just output an exit code of “I’m already started” and remain started. With that said this is only a sample, and its still worth testing in your environment to confirm the behavior and make sure you understand exactly how the recovery is working before you consider implementing.
An example of running the recovery for an instance of m% being stopped:
Two things you need to know about Charles:
1. He is one of the best resources on Unix/Linux monitoring with SCOM outside of Kris Bash & and the PG.
2. He has an awesome new MP he just released that fixes the age old problem of Monitor based alerts being closed while the monitor is still unhealthy. I have seen customers solve this in the past with Green Machine, but the problem with Green Machine is unless it is applied carefully it becomes a state churn monster in your environment. Sky Blue has some slick PowerShell filtering logic to go only after those pesky monitors/alerts that need it while leaving the rest alone.
Check it out:
SKY BLUE “management asking about “missing alerts” always feels like a rainy day.”