Doing the magic on the SNMP tables the right way 1/2

5 Jan

The Problem:

Every time I had to make a special management pack that would require  data from SNMP I bumped against the SNMP table format. In my opinion the SNMP table structure isn’t very easy to read and to process in a management pack.  And let me even not talk about the SNMP table indexing with a ‘foreign key’ to a other SNMP table based on the OID value and not the OID number!! This can become a nightmare for a SCOM MP developer.

So one possible solution:

First of all, this solution I am describing below was one of my first versions that where part of the prototype. Meanwhile I have updated this to more advanced/paid/production versions. Due to this I can only share the prototype code with you. This means that I changed some names and the code could be incomplete. But I am sure most of you can use this as first start.

I wanted to get the SNMP table data in a generic way so I could process it in a generic way. For example normally when you want to discover a application component based on SNMP you do a SNMP probe. But you can only generate the new component class and have to fire up a second SNMP probe to fill-in the other wanted properties.   A better way would be to get all the property information in one SNMP probe.  Yes off course you will use a SNMP walk probe for this. But be aware of its data returning. I can be tricky to map the SNMP data result to rows. See picture below:

image

So my solution was to make this simpler by reading the SNMP table walk and map it to a real table in PowerShell. Then I could manipulate this PowerShell table very simple and build up the object discovery properties or use it in monitors / rules.

The module will look like this:

Do the SNMP walk –> convert OID values it to arrays –> map the arrays to row records –> output the row records as property bag data

Very simple approach isn’t it??

So the Datamodule would look like this:

 <DataSourceModuleType ID="DataSource.SNMP2Table" Accessibility="Internal" Batching="false"> 
 <Configuration> 
 <xsd:element minOccurs="1" name="IP" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="CommunityString" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Version" type="xsd:integer" /> 
 <xsd:element minOccurs="1" name="OID" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Walk" type="xsd:boolean" /> 
 <xsd:element minOccurs="1" name="Root_Table_OID" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Fields_To_Show" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Debug" type="xsd:string" /> 
 <xsd:element minOccurs="0" name="IntervalSeconds" type="xsd:integer" /> 
 </Configuration> 
 <OverrideableParameters> 
 <OverrideableParameter ID="Debug" Selector="$Config/Debug$" ParameterType="string" /> 
 <OverrideableParameter ID="IntervalSeconds" Selector="$Config/IntervalSeconds$" ParameterType="string" /> 
 </OverrideableParameters> 
 <ModuleImplementation Isolation="Any"> 
 <Composite> 
 <MemberModules> 
 <DataSource ID="get" TypeID="DataSource.SnmpGet"> 
 <IntervalSeconds>$Config/IntervalSeconds$</IntervalSeconds> 
 <IP>$Config/IP$</IP> 
 <CommunityString>$Config/CommunityString$</CommunityString> 
 <Version>$Config/Version$</Version> 
 <SnmpVarBinds> 
 <SnmpVarBind> 
 <OID>$Config/OID$</OID> 
 <Syntax>1</Syntax> 
 <Value VariantType="8" /> 
 </SnmpVarBind> 
 </SnmpVarBinds> 
 <Walk>$Config/Walk$</Walk> 
 </DataSource> 
 <ProbeAction ID="maptotable" TypeID="Windows!Microsoft.Windows.PowerShellPropertyBagProbe"> 
 <ScriptName>snmp2bag.ps1</ScriptName> 
 <ScriptBody> 
                  ##--------------------------------------
                  ## convert snmp table to propertie bags
                  ##--------------------------------------
                  ## Michel Kamp
                  ## v1.0.2
                  ##--------------------------------------

                  param ([string] $SNMP_RESULTS , [string] $Root_Table_OID , [string] $Fields_To_Show , [string] $Debug)

                  $guid = [guid]::NewGuid()

                  $Debug_File = $env:systemroot + "\temp\" + $guid + "_" + $Root_Table_OID

                  #$Debug_File = "c:\temp\" + $guid + "_" + $Root_Table_OID
 
                  try
                  {
                  If ($Debug  -eq $true) { $SNMP_RESULTS | Out-File -FilePath ($Debug_File + "_RESULTSxml.xml") }

                  [xml] $data = $SNMP_RESULTS
                  $Root_OID = $Root_Table_OID
                  $OID_Fields = $Fields_To_Show.Split(",")

                  If ($Debug -eq $true) { $data.InnerXml | Out-File -FilePath ($Debug_File + "_Dataxml.xml") }

                  $api = New-Object -comObject 'Mom.ScriptAPI'

                  ## fill the array with the SNMP vaules
                  $ValueArray = @{"" = ""}
                  $ValueArray.Clear()
                  foreach ( $key in $OID_Fields)
                  {
                  $ID=$key
                  $ValueArray.Add($ID,($data.DataItem.SnmpVarBinds.SnmpVarBind | Where-Object { $_.OID -like $Root_OID+'.'+$ID+'.*' } ))
                  }


                  # clear the debug output file
                  If ($Debug  -eq $true) { Write-Output "PropertyBag"  |  Out-File -FilePath ($Debug_File + "_outputPropertyBag.xml") }

                  ## create the propertie bags
                  ## Loop the snmp records. First field in $OID_Fields gives the record count
                  for ( $x=0; $x -le ($ValueArray[$OID_Fields[0]].Count -1 ) ; $x++ )
                  {
                  $bag = $api.CreatePropertyBag()

                  foreach ($y in $OID_Fields)
                  {

                  $bag.AddValue("OID"+$y,$ValueArray[$y][$x].OID)
                  $bag.AddValue($y,$ValueArray[$y][$x].Value.InnerXml)
                  If ($Debug  -eq $true)
                  {
                  "OID"+$y + ":" + $ValueArray[$y][$x].OID
                  $ValueArray[$y][$x].Value.InnerXml
                  $ValueArray[$y][$x].Value.InnerXml |  Out-File -Append -FilePath ($Debug_File + "_outputPropertyBag.xml")
                  }
                  }
                  #Return bag values
                  $bag
                  }
                  }
                  # Catch all other exceptions thrown by one of those commands
                  catch {
                  If ($Debug  -eq $true) { $Error | Out-File -FilePath ($Debug_File + "catch.xml") }
                  $Error[0]
                  }
                  ## end
 </ScriptBody> 
 <Parameters> 
 <Parameter> 
 <Name>SNMP_RESULTS</Name> 
 <Value>$Data$</Value> 
 </Parameter> 
 <Parameter> 
 <Name>Root_Table_OID</Name> 
 <Value>$Config/Root_Table_OID$</Value> 
 </Parameter> 
 <Parameter> 
 <Name>Fields_To_Show</Name> 
 <Value>$Config/Fields_To_Show$</Value> 
 </Parameter> 
 <Parameter> 
 <Name>Debug</Name> 
 <Value>$Config/Debug$</Value> 
 </Parameter> 
 </Parameters> 
 <TimeoutSeconds>900</TimeoutSeconds> 
 </ProbeAction> 
 </MemberModules> 
 <Composition> 
 <Node ID="maptotable"> 
 <Node ID="get" /> 
 </Node> 
 </Composition> 
 </Composite> 
 </ModuleImplementation> 
 <OutputType>System!System.PropertyBagData</OutputType> 
 </DataSourceModuleType>

 

The output data will be an Property Bag collection with one SNMP row record / property bag.

So the next part will be mapping it to discovery data. So we get this workflow:

SNMP row records prt bags –> Map to target properties  –> Output target discovery Data 

And again a very straight on approach.

This datamodule will look like this:

 <DataSourceModuleType ID="DataSource.SNMP2Table.DiscoveryData" Accessibility="Internal" Batching="false"> 
 <Configuration> 
 <IncludeSchemaTypes> 
 <SchemaType>System!System.Discovery.MapperSchema</SchemaType> 
 <SchemaType>System!System.ExpressionEvaluatorSchema</SchemaType> 
 </IncludeSchemaTypes> 
 <xsd:element minOccurs="1" name="IP" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="CommunityString" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Version" type="xsd:integer" /> 
 <xsd:element minOccurs="1" name="OID" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Root_Table_OID" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Fields_To_Show" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="InstanceSettings" type="SettingsType" /> 
 <xsd:element minOccurs="1" name="FilterExpression" type="ExpressionType" /> 
 <xsd:element minOccurs="1" name="FilterString" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="ClassId" type="xsd:string" /> 
 <xsd:element minOccurs="1" name="Debug" type="xsd:string" /> 
 <xsd:element minOccurs="0" name="IntervalSeconds" type="xsd:integer" /> 
 </Configuration> 
 <OverrideableParameters> 
 <OverrideableParameter ID="Debug" Selector="$Config/Debug$" ParameterType="string" /> 
 <OverrideableParameter ID="IntervalSeconds" Selector="$Config/IntervalSeconds$" ParameterType="int" /> 
 </OverrideableParameters> 
 <ModuleImplementation Isolation="Any"> 
 <Composite> 
 <MemberModules> 
 <DataSource ID="table" TypeID="DataSource.SNMP2Table"> 
 <IP>$Config/IP$</IP> 
 <CommunityString>$Config/CommunityString$</CommunityString> 
 <Version>$Config/Version$</Version> 
 <OID>$Config/OID$</OID> 
 <Walk>true</Walk> 
 <Root_Table_OID>$Config/Root_Table_OID$</Root_Table_OID> 
 <Fields_To_Show>$Config/Fields_To_Show$</Fields_To_Show> 
 <Debug>$Config/Debug$</Debug> 
 <IntervalSeconds>$Config/IntervalSeconds$</IntervalSeconds> 
 </DataSource> 
 <ConditionDetection ID="mapfiltered" TypeID="System!System.Discovery.FilteredClassSnapshotDataMapper"> 
 <Expression>$Config/FilterExpression$</Expression> 
 <ClassId>$Config/ClassId$</ClassId> 
 <InstanceSettings>$Config/InstanceSettings$</InstanceSettings> 
 </ConditionDetection> 
 </MemberModules> 
 <Composition> 
 <Node ID="mapfiltered"> 
 <Node ID="table" /> 
 </Node> 
 </Composition> 
 </Composite> 
 </ModuleImplementation> 
 <OutputType>System!System.Discovery.Data</OutputType> 
 </DataSourceModuleType>

 

At this point we have all we want to have:

1. module to get the SNMP row record table

2. Module to convert this records to discovery data

What are we missing….. yhea the top discovery rule Knipogende emoticon

This will look like :

Schedule every x minutes –> get the SNMP table as discovery data  -> create the targets with filled in property values

And again the discovery rule will look like below: (remember its prototype code)

 

 <Discovery ID="Discovery.Gateway" Enabled="true" Target="Server" ConfirmDelivery="true" Remotable="true" Priority="Normal"> 
 <Category>Discovery</Category> 
 <DiscoveryTypes> 
 <DiscoveryClass TypeID="Gateway" /> 
 <DiscoveryRelationship TypeID="Server2Gateway" /> 
 </DiscoveryTypes> 
 <DataSource ID="getdata" TypeID="DataSource.SNMP2Table.DiscoveryData"> 
 <IP>$Target/Property[Type="MicrosoftSystemCenterNetworkDeviceLibrary!Microsoft.SystemCenter.NetworkDevice"]/IPAddress$</IP> 
 <CommunityString>$Target/Property[Type="MicrosoftSystemCenterNetworkDeviceLibrary!Microsoft.SystemCenter.NetworkDevice"]/CommunityString$</CommunityString> 
 <Version>2</Version> 
 <OID>1.3.6.1.4.1.6889.2.8.1.104.6.1</OID> 
 <Root_Table_OID>1.3.6.1.4.1.6889.2.8.1.104.6.1</Root_Table_OID> 
 <Fields_To_Show>1,2,3,4,5,6,7,16,17,21,30</Fields_To_Show> 
 <InstanceSettings> 
 <Settings> 
 <Setting> 
 <Name>$MPElement[Name="MicrosoftSystemCenterNetworkDeviceLibrary!Microsoft.SystemCenter.NetworkDevice"]/IPAddress$</Name> 
 <Value>$Target/Property[Type="MicrosoftSystemCenterNetworkDeviceLibrary!Microsoft.SystemCenter.NetworkDevice"]/IPAddress$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Index$</Name> 
 <Value>$Data/Property[@Name='1']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/OID$</Name> 
 <Value>$Data/Property[@Name='OID1']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="System!System.Entity"]/DisplayName$</Name> 
 <Value>$Data/Property[@Name='2']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/IP$</Name> 
 <Value>$Data/Property[@Name='4']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/MAC$</Name> 
 <Value>$Data/Property[@Name='5']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Region$</Name> 
 <Value>$Data/Property[@Name='6']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Location$</Name> 
 <Value>$Data/Property[@Name='7']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Registered$</Name> 
 <Value>$Data/Property[@Name='16']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Type$</Name> 
 <Value>$Data/Property[@Name='17']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/Encrypt$</Name> 
 <Value>$Data/Property[@Name='21']$</Value> 
 </Setting> 
 <Setting> 
 <Name>$MPElement[Name="Gateway"]/RecoveryRule$</Name> 
 <Value>$Data/Property[@Name='30']$</Value> 
 </Setting> 
 </Settings> 
 </InstanceSettings> 
 <FilterExpression> 
 <RegExExpression> 
 <ValueExpression> 
 <XPathQuery>/DataItem/Property[@Name='OID1']</XPathQuery> 
 </ValueExpression> 
 <Operator>MatchesRegularExpression</Operator> 
 <Pattern>[0-9]</Pattern> 
 </RegExExpression> 
 </FilterExpression> 
 <FilterString /> 
 <ClassId>$MPElement[Name="Gateway"]$</ClassId> 
 <Debug>false</Debug> 
 <IntervalSeconds>86400</IntervalSeconds> 
 </DataSource> 
 </Discovery>

Hmm but… Yes the only values you have to specify are:

<OID>1.3.6.1.4.1.6889.2.8.1.41.6.1</OID> : This is the TOP OID of the SNMP walk. Will most of the time be the TOP OID of the SNMP table.
<Root_Table_OID>1.3.6.1.4.1.6889.2.8.1.41.6.1</Root_Table_OID> : This the table you want to convert. Will most of the time be the <OID> value.
<Fields_To_Show>1,2,3,4,5</Fields_To_Show> : This are the SNMP columns you want to get the values of.

You see very simple again. The power is in the <Fields_To_Show> parameter. You specify here the column index numbers of the values you want to use.

image

 

The value of the field number(s) specified will be returned and can be readout with $Data/Property[@Name=’OIDX‘]$ where X is the field number. So OID1 is the g3gatewayNumber field. Use this value for the <InstanceSettings> elements.

At this point you will have a generic way to read out SNMP tables.

So…

This was the end of part 1. Reading out 1 SNMP table is not so hard to do as you have noticed. But what about reading 2 or more SNMP tables and use a ‘foreign key’ to join this SNMP tables based on the OID value (So NOT the OID key)??? Now it is becoming more interesting.. isn’t it ?? Part 2 will be all about this……

 

Happy Scom’ing..

Michel Kamp

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: