Management Packs: SharePoint 2013 Extended MP

I had a question recently on whether or not the SharePoint 2013 Management Pack monitors when Services are stopped?

As with most questions this is not quite as straightforward as you would expect. With SharePoint there are two kinds of services. There are your standard services that you access via services.msc which are easy to make custom monitors for:

10

And there are also those services that are only accessible via the SharePoint Central Administration console:

11

So the native monitoring available in the pack is different depending on what type of SharePoint service we are talking about. The services of the .msc persuasion are easy enough to create custom monitoring around, but does the pack monitor the specialized internal services? If we look at the 2013 pack we find the SharePoint specific services are split up into three state views. Service Front End, Services, & Shared Services:

12

Each is populated as follows:

13

14

15

So this is normally the point that I would put up my feet and bask in the glory of the product team. The SharePoint Services that can only be accessed via the Central Administrative interface are all enumerated above and they are healthy. Question answered, all is well, and there was much rejoicing throughout the land. Unfortunately, I am prone to a nasty habit of testing things. So before breaking into celebratory dance I took one of these services and stopped it waiting impatiently for the inevitable alert to roll in.

16

I waited for a long time. I refreshed many times. No alert came. I was sad. I also noticed that the state view for the relevant service now showed “Not Monitored” whereas before I stopped the service it showed healthy.

17

{Insert appropriate Expletive Here}

So what you find if you open the Health Explorer for any of these SharePoint specific services is that while there are monitors associated, it’s just that they don’t actually monitor the status of the service. So if  it is stopped you lose monitoring around that service and you will never know it unless you go to the view and note the Not Monitored.

18

Part of me really wants to know why this is the case. My completely uninformed guess would be that since SharePoint services can exist on multiple members of the Farm and by design you may only want them running on certain members it is hard to code a solution that isn’t going to be extremely noisy when looking at service status since you as the MP author have no idea that the service being stopped on one server is a critical situation whereas on an identical server it could be by design and non-actionable.

So how do we fix this so that you can selectively monitor service status for certain SharePoint servers and get alerts? First we have to know a little bit about how SharePoint Services work– which at the time I didn’t know much. I started by stopping services via the Central Administrative site and then trawling all relevant event logs on the SharePoint servers hoping for a nice event ID that I could use.

I will hopefully save you some time by telling you that there is no such event ID. There are some events that get generated that are related and will say hopeful things like X Service is Stopping or Starting, but you will find these won’t actually correlate with the time you stopped the service.

So this left me with PowerShell.

The basic mechanics of dealing with SharePoint via PowerShell as it pertains to Services are below. First to do anything you need Add the relevant Snap-in. Then you can use Get-SPServiceInstance to get a list of the services:

Add-PSSnapin -Name “Microsoft.SharePoint.PowerShell” -ErrorAction SilentlyContinue

Get-SPServiceInstance | ft TypeName, Identity

19

Then you can use the script below to check status/start/stop the service as you wish:

#To Stop a Service:

$ServiceName = “Visio Graphics Service”

Get-SPServiceInstance -server $env:COMPUTERNAME | where-object {$_.TypeName -eq $ServiceName} | Stop-SPServiceInstance -confirm:$false > $null

#To get Service Status:

Get-SPServiceInstance -server $env:COMPUTERNAME | where-object {$_.TypeName -eq $ServiceName} | Select Status

#To Start a Service:

Get-SPServiceInstance -server $env:COMPUTERNAME | where-object {$_.TypeName -eq $ServiceName} | Start-SPServiceInstance -confirm:$false > $null

From my testing I found that there are 4 different States that a SharePoint Service can be in:

When a service is initially stopped it will temporarily be listed as = Unprovisioning

Once fully stopped the Service Status will = Disabled

Service Started initially = Provisioning

Sevice Started will eventually = Online

The reason this is important is that if you create a custom script based PowerShell monitor you need to either account for each of these states in your script or alternatively you could get around this with something like any state other than Online would be considered Unhealthy. From here I had enough info to start writing a custom management pack. My original script treated ‘Provisioning’ and ‘Online’ as Healthy and ‘Unprovisioning’ and ‘Disabled’ as Unhealthy. This was then changed in a later revision so that only Online would be considered healthy since you could theoretically have a situation where a service is stuck in a provisioning state for a protracted period of time. I then explicitly gave ‘Provisioning’, ‘Unprovisioning’, and ‘Disabled’ a Status of Unhealthy, this allowed me add one additional status whereby if the script returns anything other than an expected state it will flip to a warning state indicating that either the Run As account is not properly configured or the service isn’t actually present on the server you are targeting.

The basic script is below:

20

I will skip over the details of how I converted this into a functional management pack, but the end result is an extended MP. By default all monitors are disabled and targeted against the SharePoint 2013 Server Class. You will need to override and enable on a case by case basis as it is likely that your SharePoint team will only care about monitoring certain services on certain members of the Farm.

02

Since most SharePoint environments are not fully accessible via the default action account I have added a custom Run As Profile:

01

Just add/associate your Run As account that has SharePoint Farm Admin privileges and you should be all set.

If you neglect to configure the Run As or you enable the monitor on a server that doesn’t have the target service you will get the following:

03

I tweaked the associated Alert Status for the Run As Warning to critical just so it won’t get missed.

04

Other than that the pack is fairly self explanatory, when a service is stopped within 15 minutes you will get an alert as below:

08

As with all MP’s of this nature this is a proof of concept “As-Is” pack. Script based monitoring (Particularly PowerShell based monitoring) can be some of the most intensive monitoring you can perform on a system in terms of overhead so you should test thoroughly first and insure that the extra scripts running is not negatively impacting performance.

Currently the pack does not implement cookdown. Normally you would want to limit impact by having a module that runs a single script that queries the status of all relevant services at once and then is referenced by multiple monitors. Currently there is a 1:1 relationship for each monitor, so there is a separate script for each monitor and thus a higher impact on the system. If time allows I will rewrite to properly implement cookdown but in the meantime feel free to take apart the MP and build something better/more efficient, but this should at least help get you started.

Download from: TechNet Gallery

Tools: New OpsConfig Tool v1.1 released

OpsConfigTool

Download Here

This is an AS-IS proof-of-concept GUI tool for core maintenance tasks that some of my customers perform on a regular basis in Operations Manager. There are a number of great OpsMgr PowerShell scripts, MP’s, and maintenance tools out there–this tool is not intended to replace, merely to compliment them while giving me the opportunity to get better acquainted with PowerShell.  

Features:

1. Bulk Backup of all UnSealed MP’s

2. Unseal and Backup of all Sealed MP’s

3. Return a list of all systems where Proxy is not enabled

4. Enable Proxy on all systems in your Management Group 

5. Return a list of all systems in Maintenance Mode

Requirements: The tool is intended to be run on a Management server by an account that had admin access in SCOM & read access to the OperationsManager DB.

Compatibility: SCOM 2012, 2012 SP1, 2012 R2

On The Importance Of Building Test Environments

One of the things I didn’t quite grasp when I first started using SCOM a few years back was the importance of test environments. SCOM was this bright and shiny new tool that was going to help proactively monitor our servers, increase uptime, and as long as I only installed Microsoft approved Management Packs everything would be alright. This was admittedly extremely naive– but it was good starting point. I was enthusiastic as well as fortunate enough to learn that this was a terrible idea long before making a critical mistake.

SCOM is an incredibly powerful tool, but it has to be used and implemented intelligently:

-Installation guides must be read.

-MP’s should be evaluated in Test or Dev environments first (If you don’t have a test environment build one)

-Blogs should be scoured for relevant info.

-Management Packs should be installed in production because they provide value not just because you happen to have the associated product installed.

Anytime an engineer or admin asks to have a shiny new management pack installed in production and doesn’t want to test it first I remember this slide from a talk I stumbled across from Microsoft’s Management Pack University entitled “Getting Manageability Right” given by Nistha Soni, a program manager on the Ops Manager team at Microsoft:

Getting Managability Right Nista Soni

The talk was for the different Microsoft product teams to help them think about how to build better management packs that are useful to their customers. If a MP reduces total cost of ownership this is a good thing, if it increases TCO then we have a problem. This slide was referencing an iteration of a Microsoft MP–name omitted to protect the guilty– which provided feedback that while potentially useful for a developer at Microsoft, was also inundating their customers/operators with alerts.

Building a useful MP is a delicate balancing act and its important to remember that even the ones made by Microsoft are essentially a work in progress. Each successive iteration tends to get better, but if you just import into production without testing and research you are asking for trouble.

The talk itself is an interesting look at how Microsoft thinks about monitoring and building management packs and is still available here.

The contents of this site are provided “AS IS” with no warranties, or rights conferred. Example code could harm your environment, and is not intended for production use. Content represents point in time snapshots of information and may no longer be accurate. (I work @ MSFT. Thoughts and opinions are my own.)