Archive | Maintaining RSS feed for this section

How To fix the 500 Error using Favorite Reports in the SCOM web console.

22 May

First mention this is a non official solution.

The Problem

You have installed SCOM 2012 Sp1 UR2 and have implemented the scom webconsole and reporting service to be running under HTTPS mode. You have created using the native scom console a favorite report and now when you try to open this favorite report in the scom webconsole you get a error 500.

image

image

Analyzing

To see the real error we have to do some web.config changes. So open the web.config file on this location: C:\Program Files\System Center 2012\Operations Manager\WebConsole\MonitoringView

Now we enable the SCOM error logging

image

And to get it displayed on the user page we do

image

Now when you run the favorite report again we get in the webconsole  the real error

image

Okay looks like the reportviewer web component binary dll can’t be found. Hmm but wait wasn’t this a prereq at installation time. So I checked if the 2010 ReportViewer components where installed and yes it was and the dlls where also spotted in the assembly cache. It looks like the webconsole has problems finding the correct version of the Microsoft.ReportViewer.WebForms.dll in the assembly cache.

The Quick non Official Solution

Copy the missing dlls to the correct directory will force the web runtime to first look in this directory for the dlls and then go to the assembly cache. So that’s what i did.

Copy the Microsoft.ReportViewer.WebForms.dll file from the assembly cache to path : C:\Program Files\System Center 2012\Operations Manager\WebConsole\MonitoringView\bin

Come on give me some script to do that ! Okay open PowerShell as admin and run

Copy-Item c:\Windows\assembly\GAC_MSIL\Microsoft.ReportViewer.WebForms\10.0.0.0*\*.dll “C:\Program Files\System Center 2012\Operations Manager\WebConsole\MonitoringView\bin”

And now you try to run the favorite report again in the webconsole …

image

… and Yes its working!

The End.

For me this looks  like a bug and I will address this to Microsoft.

Happy Scomming!

Michel Kamp

SCOM Console Hidden Feature ??

21 May

A really short post. A member of a NOC operations team reported a problem with his SCOM console.

The Problem.

Using the operations native console and opening a windows computer state view resulted in gray computer targets.

image

Analyze

Normally when for example the computer targets are gray you know it’s a gent connectivity problem or the MS health service is stalled / crashed.  But this time non of this was true. Looking at the agents and MS states its was all green and okay.

The solution

First the root cause was assumed to be the computer of the NOC operator. Yes I have rebooted it twice and I have installed all the patches was the first response Knipogende emoticon  . But however I am running Windows 8. Okay .. could this be the case ?… After a while of figuring out what could be the problem we detected that on other NOC computers the problem was also reproduced. And yes now we had found the problem .. And what do you think ? If you have a target state view (for example the windows computer view) and you personalize this view and remove the State column you will facing the problem above. So we added the State Column again and the problem was fixed.

image

The End

Of course you can ask you’re self why remove the state column it’s a very important one. Lets say the most important one in a state view…. I have ask it my self’s but I think it was a human error by mistake and not noticed earlier… But the question remains this still looks like a bug in the SCOM console.. What do you think ?

Happy Scomming

Discovery’s at your demand , yes sir!

27 Apr

Hi,

This time a short post. But I think this could be useful  for SCOM admins.

The challenge.

We all know that one of the big powers of SCOM is the self maintaining of the monitor targets. SCOM uses discovery’s for this that run at regular intervals. Lets say you install a new SQL database instance on a server that has already a SCOM agent on it. Normally you have to wait for 4 hours before the new database instance is discovered. Yes you can speed this up to restart the SCOM agent but now we have a better way.

Analyze

First, all the credits go to the SCOM product team it self’s. It seems the feature was already build in but making it globally known was somehow left behind. There is a agent task called ‘Trigger On Demand Discovery’ that can help you out. But how to operate this task can be painful.

The solution

I have written a PowerShell script that does the hard work for you. Running this script and supplying the correct Discovery and target will result in a instantly run of that discovery. So now you don’t have to wait for the discovery interval of 4 hours to trigger.

How it works:

1) You fill in the $OMserver with the SCOM SDK server FQDN.

2) You fill in the $discoveryname with the display name of the discovery rule you want to trigger. Just copy and paste the displayname from your author pane in the scom console. See picture below.

image

3) You fill in the $targetdisplayname with the name of the main target where this discovery should run. You can find this name by looking at the target from the discovery rule you got from step 2.

image

And fill this in the inventory view.

image

The name “servicemanager.systemcenter.local” is the target display name to use.

btw. of course you can use PowerShell to do this for you…

Below the script:

It triggers the discovery task and then waits for the results and displays it. Be sure to look at the output results property because it only is okay when it contains :

image

The script.

## =======================================================
## Trigger SCOM discovery for a discovery rule and target
## ======================================================
## Michel Kamp

Import-Module operationsmanager
## OM sdk server
$Omserver=”scom01.systemcenter.local”
## discovery display name
$discoveryname=”Service Manager Management Server Properties Discovery”
## target display name
$TargetDisplayName=”servicemanager.systemcenter.local”

## —————————————————-
## MAIN
## —————————————————-
# connect to OM server
$credentials = get-Credential
new-ScommanagementGroupConnection -Computer $Omserver -Credential $credentials

# get task to execute
$task=get-scomtask -name Microsoft.SystemCenter.TriggerOnDemandDiscovery
# make override params
$discovery=get-scomdiscovery -DisplayName $discoveryname
$TargetInstanceId= (Get-SCOMClass -Id   $discovery.target.id  | Get-SCOMClassInstance | ?{$_.displayname -eq $TargetDisplayName}).ID.Tostring()
$DiscoveryID=$discovery.id.tostring()
$override=@{DiscoveryId=$DiscoveryID;TargetInstanceId=$TargetInstanceId}
$instance=get-scomclass -name Microsoft.SystemCenter.ManagementServer | get-scomclassinstance | ?{$_.displayname -eq $Omserver}
# run the task
$task_run=start-scomtask -task $task -instance $instance -override $override

# wait for result
while ( (get-SCOMTask -Id $task_run.TaskId).Status -eq “Started” )
{
    write-Output “Waiting…”
    Sleep -Seconds 2
}
# show task output
get-SCOMTaskResult -BatchID $task_run.BatchId

## —————————————————-
## end script
## —————————————————-

The End.

I already did some more investigation on this topic because I think when you can do it for a discovery you can also do it for every workflow that contains a timed interval trigger module. Can you imagine that you can now trigger every rule or monitor at your demand… so cool and so handy while debugging.  When I have it working I will of course share it with you “the community”.

Happy SCOMMING

Michel Kamp

Let SCOM check for Updated Management Packs

21 Apr

The challenge

Using the SCOM native console the import from the Microsoft Management Pack Catalog is a nice feature. I like also the feature to check and import updated MPs that you have already imported in your management group. But what I really miss and don’t understand : why did the product team removed the monitor that gives us a alert when a new MP version is in the MP catalog ?. This monitor was build in MOM 2005 but removed in the begin of SCOM 2007.

The solution

So since we are SCOM author diehards we are going to build our own MP update monitor. I am going to use VSAE to build it all. But wait even if you aren’t a SCOM author diehard it still worth reading this post because this time I will share the VSAE project and even the MP with you at the end!!!

Analyze

So I used my good old friend ‘Fiddler’ to backward-engineer what the scom console is doing when I press the ‘check for updated management packs’ button. It seems it sends a SOAP request to a webservice. The SOAP request contains a MP list of the MPs that you have already imported. The answer result of this request will be a MP list with the updated MP versions or an empty list if there aren’t any updates for you.

Building time

Below I’m going to give you a overview what I have done. You can look in the VSAE project for details on it. If you have any questions just let it know and I will help you out.

1) The datasource

So now we are going to make a datasource that runs a PowerShell script. This PowerShell script is simulating the webservice request.

Below a snippet of the code. (the full code is in the VSAE project). What I am doing here are 3 steps:

1) Build a SOAP request message that contains all my MP version meta data from all MPs that I have already imported in my management group.

2) I call the “ManagementPackCatalogWebService.asmx” and execute the method “FindManagementPacks”

3) as last step I check if there are any MPs returned and set the $Status flag according the result. And I return the scom property bag.

# step 1

$MPSoap = get_MP_List
$ret = Do-SOAPRequest -SOAPRequest $MPSoap -URL $MPCatalogURL -SOAPAction $SOAPAction

# step 2

## show MPs that have a Update
$MpList = $ret.Envelope.Body.FindManagementPacksResponse.FindManagementPacksResult.CatalogItem | where { $_.IsManagementPack -eq $true} | select-Object DisplayName

# step 3

## check MP returned

if ( $MpList.Count -eq 0)
{
$Status=”UPTODATE”
}
else
{
$Status=”NOTUPTODATE”
}

  # Create the property bags
$pb = $oAPI.CreatePropertyBag()
$pb.AddValue(“Status”,$Status)
$pb.AddValue(“MpList”,($MpList | Out-String))
$pb

The script above we are going to use in the datasource below

image

2) The Monitor

Now we are going to compose a 2 state UnitMonitorType that uses this datasource. The health state check is done with the “Status” value in the returned property bag.

image

Having this UnitMonitorType composed we can now use it in the real monitor KPI. See below for the KPI. The target is the Management server. I choose this target because I have only one MNG server in my test lab but if you have more it’s better to choose the RMS emulator target.

image

Now when the monitor is unhealthy it will generate an alert message constructed below:

image

 

The result

Building and importing the MP in your SCOM management group will show you the result below:

image

And of course a nice ALERT message also:

image

So now the part you are waiting for..

As promised I will share the VSAE project and the MP it self. Please notice that it is a show case alias prototype MP and so it is far from complete. For example not all display strings are applied and no knowledge is supplied. But that’s up to you to complete…. In my production version I have even build in a recovery/console task that also automatically imports the updated MPs.. Just a idea for you to work out…

MP download and VSAE project download: https://1drv.ms/f/s!Au2euLDFD_ovilTTrLl-XqSmd1tj

The End

Feel free to comment or contact me if you have any questions.

Happy SCOMMING

Michel Kamp

SC 2012 SP1 UR2

12 Apr

 

Hi ,

It isn’t any new news , since I twitted also 2 days ago the UR2 release of System Center 2012 SP1 is released.  ( http://support.microsoft.com/kb/2802159 )

As I write now not all software download links are working jet. So don’t get frustrated as I did…

For Installation experience see here for again an excellent post from Kevin Holman:

http://blogs.technet.com/b/kevinholman/archive/2013/04/11/applying-update-rollup-2-ur2-to-opsmgr-2012-sp1.aspx

O wait what’s missing ?:

If you read the release notes you will notice that it mentions patching the Gateways but if you look at the software downloads you won’t see any gateway patch.

I don’t know if the gateway patch is simply forgotten to publish but if you have a environment that uses a gateway you will be stuck for now… because your agents behind the gateway will not be patched using the pending actions in SCOM.

Or not…

The solution is however not far away.
1) Just copy the Agents msp binarys to the agent management directory on the GW servers.

Path : C:\Program Files\System Center Operations Manager\Gateway\AgentManagement

image

1) So copy file KB2826664-AMD64-Agent.msp to

C:\Program Files\System Center Operations Manager\Gateway\AgentManagement\amd64

2) copy file KB2826664-AMD64-Server.msp to

C:\Program Files\System Center Operations Manager\Gateway\AgentManagement\x86

And at last the KB2826664-ia64-Agent.msp file to the ia64 directory.

3) approve the pending actions in your scom console. And you will see the agents behind your GW will be patched.

The end

Hope Microsoft will clear this confusion soon. Because I can’t imagine that the GW it self’s doesn’t have any fixes…

Happy scomming

Michel

Aggrr missis SCOM sometimes you drive me nuts!

12 Mar

Here we are again. I am for sure you will remember a situation where you were sure everything was configured correctly and you even had restarted the server but still it doesn’t work…… NO ??

Okay so this time it was my turn…

The problem:

I was trying to discover a new network device. Network discovery won’t find my newly added network device. Why ? I checked everything and it isn’t connectivity or security. Normally when those issue apply you will see a pending failure discovery attempt. What could be wrong..

Analyze:

1) First I checked the discovery rule. The network devices are explicit specified.

image

I made sure the device was listed in the explicit discovery.

image

2) I started the discovery and checked the progress events.

image

Hey what’s that ! One device excluded ?? Strange this option can only be applied to recursive discovery’s….

So it must be something in the discovery rule….

The solution:

So this excluded event triggered my attention.

1) open the discovery rule.  And change the Discovery method to Recursive discovery

image

2) Now select the Exclude Filters tab

image

And what do we see here . The network device I was trying to discover.

3) Now remove this exclude filter entry and save the discovery template without RUNNING IT

image

4) Reopen the discovery rule and change the discovery mode back to ‘explicit’ and save it again.

image

5) now run the discovery.

6) and what do we see here … the device is discovered successfully !

image

Conclusion

It looks like this rule was initially created in recursive discovery mode. After a while the operator decided to remove a device from the discovered device inventory. If you do that a exclude filter entry is automatically created. At the end the operator decided to change the discovery mode back to explicit (don’t ask why) and that’s causing the problem I faced…

The end

Hope I saved you a lot of time almost hating misses SCOM ; – )

Happy SCOMMING

Michel Kamp MVP System Center Operations Manager

Hey SCOM Agent just install your self , please !

11 Feb

Hi,

The last time I notice more and more SCOM 2012 Agent installation fails. Where the SCOM 2007 agent installs fine the SCOM 2012 agent has troubles.

There are plenty of guides on the web but none fixed my problem.

1. Problem:

During the SCOM 2012  agent install (push or manual) the install fails with error 25211 Failed to install performance counters. Error Code: -2147024809

image

2. Analyze

When looking at the deployment log (search for value 3) I noticed that the component failing was the APM module. Secondly it has something to do with registering performance counters. ; – )

3. Solution

When you look on the web you will find as a possible solution KB2554336. Following this article you are rebuilding your performance counters. At my case this rebuild was successful but the agent install still failed. So …

My analyze concluded the setup stopped at the APM module. So why ‘not’ install this module ? I know the server where I am going to install the agent never is going to use this module. How do we do this ? You can do this by supplying the parameter: NOAPM=1

1) On the failed sever open CMD as admin.

2) Copy the agent install files to this server.

3) start the install with :

msiexe /i momagent.msi NOAPM=1

You will notice the installation was successful!!

4. The end

Finally you can use the ops hack in scom db to set this agent from manual to pushed install so it will be patched also.

Happy Scomming,

Michel Kamp

Bridgeways VMWARE ESX/VC subscriptions fails

28 Nov

 

Problem:

You want to add a new VC server to the Bridgeways ESX monitoring using the “Bridgeways VMware Management Administrator” tool. But the connection always fails. You checked everything user / pwd / IP , all okay

Analyze:

1) First check if the network routing is okay. Open IE and type https://<ip>/sdk

You should get a SSL warning. If not you will have to check networking.

2) If you checked networking and you can connect to the VC but still get no SSL warning you have a SSL RSA problem.

Solution:

On Windows 2008 and above the RSA minimum bits are 1024. All other certificate length will be blocked. See http://support.microsoft.com/kb/2661254?wa=wsignin1.0 .

So check you certificate properties and look for the RCA bits length. I think it will be 512Bits.

To solve this:

Open under admin a CDM console and execute this.

Certutil -setreg chain\minRSAPubKeyBitLength 512

Now retry your add action.

 

Happy Scomming

Michel Kamp

OPSMGR UR updates are now pushed using Microsoft Update (MU) but be warned, don’t get too lazy !

15 Oct

Hi really short post. I ‘m getting a lot of questions on the new Update Rollup delivery method. Microsoft is now using the long requested windows update aka Microsoft Update for patching most of the system center products. In case of operations manager 2012 you can find them in Microsoft Update under  Windows Updates –> Important updates.

image

I am not going to cover the installation process as described in the MS KB: http://support.microsoft.com/kb/2756127

But what I wanted to point out is that since most of the people will use this MU updating method you still need to import the supplied Managementpack files. The KB documentation is somewhat unclear on this point. You can find the MP files after applying the patches under

%SystemDrive%\Program Files\System Center 2012\Operations Manager\Server\Management Packs for Update Rollups

So i f you didn’t know you will now

happy Scomming.

Audit SCOM SDK Usage Operations

5 May

This weekly blog post will be about the idea and wish I have a long time now: I want to see what a SCOM operator is doing with SCOM. So an Audit trail of the SCOM SDK idea was born.

The Idea:

Its very simple , as a SCOM administrator you have the world for your self. You can create rules , modify overrides , import / delete Management Packs , add users as you wish. If something went wrong they can blame you for it. That’s no problem because you are the the only one that know how to fix this.  But what when you are not the only one with administrator privileges. Who is blaming who ? And if blaming isn’t the issue how do I know What , Who and When something has changed (the 3 audit W’s)  ?

Solution(s):

On the web there are plenty of solutions that try to solve this. Most of them are using SQL triggers to catch the functions the operator is doing and a separate DB to store the generated audit records. This solutions have a very big impact on the SQL engine and I think by applying the triggers to the tables you lose your Microsoft Support Warranty. And what will happen by applying a SCOM service pack …. So this is not the way to go.

I was thinking , since SCOM uses ADAM/AD LDS  for user&operations authorization why not just go this way. Every call to the SCOM SDK (the native scom console, web console,power shell , connectors,ect…) will be checked for authorization . This is done by the Active Directory Lightweight Directory Service (AD LDS), formerly known as Active Directory Application Mode (ADAM). And the nice part of this is that there is a tool that lets you read and edit this authorization store. The tool is called “Authorization Manager”. You can find it as a MMC Snapin. I will show you this later on. For SCOM the authorization catalog can be located on disk as XML file or as a SQL tables. Not going into the details , SCOM 2007 SDK uses a file store and (yes) SCOM 2012 SDK uses a SQL table store. And now the reason we dig into this all: The Authorization layer has a AUDIT trail build in !!!. Simply enable it and setup the correct audit policy and we are getting audit records…

That’s however in theory ….

The real world

Of course I have tried it out before I wrote this post. And yes we are getting audit events and yes it are events for every SDK operation and NO they do not contain parameter information. (95%not) So we can see what operation is executed but can’t see the parameters supplied with the call. For example add user X to role Y. We see the add user role operation but don’t see X and Y in the audit event. Does this make my ideas useless ?  I think not. I found out that there are still some events useable. And hopefully I can get the SCOM product team so far implementing the missing parameter logging.

Some LAB testing

I will now show you how to get the audit events.

Login to the OM2012 MNG server. Open the MMC console. And add the “Authorization Manager” snapin.

image 

The first time you will see this below:

image

Now we are going to make the connection to the OM SDK Authorization Store. Remember for OM2012 this in SQL.  Right click and choose “Open Authorization Store..”

image

Now choose Microsoft SQL. And type in the Store name. I say it very easy but believe me this one cost me hours to find out (maybe because my IQ ;–) ). There is almost no documentation on Authorization manager and SQL connection. And for sure no information where the SCOM store is located. But never mind that’s your luck , because I share it with you guys.

The connection string format I got from http://technet.microsoft.com/en-us/library/cc770467(v=ws.10).aspx and looks like:

mssql://Driver={SQL Server};Server={SCOMSQLSERVER\INSTANCE};/OperationsManager/AzmanStore

Replace SCOMSQLSERVER\INSTANCE to the SQL server and Instance where your SCOM database is on.

Replace OperationsManager with the database name you have used for the operations manager database name while installing SCOM 2012.

For my LAB it will be :

image

Press OK and you are ready.

You will see the store and all it groups. Browse to AzmanStore\Microsoft System Center\Role Definitions and you will see the screen shot below. And I am sure you know it from the operations manager console!

image

Now press right button and select properties –> Definition. And you will see all the SDK operations (tasks) a SCOM administrator role can do. So every time the SDK receives a operations from for example the native console it will first check if this operation may be executed by this user. You can even add or change some operations.  So you can add for example Rule__Get to a normal operator so the operator can now see the rule properties. REMEMBER IF YOU DO YOU MAY LOOSE YOUR MICROSOFT SUPPORT.

image

If you are curios what all the operations availably in SCOM are, Go to the Add… and press Operations. Now you will see the complete list of operations. And this one is huge..

image

So explained all of this you may forget it quickly now. They only thing we have to do is set the auditing flag on the store. Right click on the azmanstore and choose properties.

image

Now go to Auditing and enable the 2 auditing boxes.

image

I just wanted to show you all of this because it is fun core level stuff. But as it seems this 2 boxes are enabled by default. So the next time you can skip this part too. Sorry if I wasted your time

The only real step we have to do is enabling the security auditing policy. So lets do that. First this. The audit records are written to the server eventlog where the SQL server instance is installed. So if you have a 2 box scom installation , one OM , one SQL. You must login to the SQL server to proceed. If you have a 1box scom installation you login to the OM server.

Open the Local Security Policy editor. And enable “Audit object access” for Success and Failure.  Press OK and the events will flow in.

REMEMBER HOWEVER THIS IS TOTALLY SUPPORTED BY MICROSOFT IT WILL GENERATE A HUGE NUMBER OF EVENTS AND MAY HIT THE PERFORMANCE OF YOUR SERVER.

image 

So lets look at the events. Open the windows event viewer and go to the security logs. This will show a huge number of events. Better is to create a custom view by only selecting events with Source  MOMSDK Service Security or Task Category (3).

image

Now if you look at a event you will notice that it has some basic information about the operation and the user plus session it belongs too. The part that is missing for a lot of operations are the parameters belonging to the operation (Data access method) if this was supplied we where done. However I found some operations that give the missing parameter information. For example when you create a new MP you will see this event. Look at the extra info given. 

image

Same for the MP delete

image

I haven’t looked at all the operations but as far I can see now there are not a lot of operations with extended information. Again I hope it will be added soon with a service pack release.

Conclusion:

With the solution described we can see What is done and Who has done it and When its done. The only part missing is some extra information on the What part. So we can really see what this operator is trying to do. But better than having nothing it’s a good start.

To be continued:

I am trying to convince the OM product team to look at this. Meanwhile I have a other idea I will try to work out to solve this.  I will try to post it soon.

Hope you had a good time and again…

HAPPY SCOMMING!

Michel