Archive | September, 2012

Don’t let the data warehouse write action fool you!

26 Sep

Yes I know. It’s a long time ago I posted. Vacation and most work pressure were and are still the reason. But never less I will share a problem I undergone that looks a small one but can have big impact.

The problem.

You have a workflow that has a PowerShell/vbs script that outputs a property bag with performance data. The performance data contains multiply counters. Now the performance data is going to be written to the OPSDB and DWHDB.  All works okay, you see the performance data counters in the native console. So you say now its okay because the DWH write actions is also writing the same counters to the DWH….  but when you look in the DWH you see that only one counter is stored. But you are sure the workflow outputted multiply counters…. 

Below the performance counters in the native console. All the 4 perf counters are there (yellow) in the ops console

image

Below the DWH.

You see only one rule (yellow) , this was the first in the property bag.

clip_image002

What could be wrong ???

Analyze

The workflow looks like this:

   <Rule ID=”TransferFile.ReadSec” Enabled=”true” Target=”FileTransferClient” ConfirmDelivery=”true” Remotable=”true” Priority=”Normal” DiscardLevel=”100″>
        <Category>Custom</Category>
        <DataSources>
          <DataSource ID=”SMBFileTransfer” TypeID=”FileTransfer”>            <Debug>false</Debug>
            <IntervalSeconds>300</IntervalSeconds>
          </DataSource>
        </DataSources>
         <WriteActions>
          <WriteAction ID=”ToOps” TypeID=”SystemCenter!Microsoft.SystemCenter.CollectPerformanceData” />
          <WriteAction ID=”ToDWH” TypeID=”SCDW!Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData” />
        </WriteActions>      
      </Rule>

1. Frist you check what the property bag output from the datasource SMBFileTransfer  is containing

<Collection><DataItem type=”System.PropertyBagData” time=”2012-09-20T19:55:28.0638791+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><Property Name=”Instance” VariantType=”8″>c:\destionation</Property><Property Name=”Counter” VariantType=”8″>Read Transfer Kbyte Sec</Property><Property Name=”Value” VariantType=”5″>14450.625</Property></DataItem><DataItem type=”System.PropertyBagData” time=”2012-09-20T19:55:28.1079971+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><Property Name=”Instance” VariantType=”8″>c:\destionation</Property><Property Name=”Counter” VariantType=”8″>Read Transfer Total Sec</Property><Property Name=”Value” VariantType=”5″>0.3</Property></DataItem><DataItem type=”System.PropertyBagData” time=”2012-09-20T19:55:28.1079971+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><Property Name=”Instance” VariantType=”8″>c:\destionation</Property><Property Name=”Counter” VariantType=”8″>Write Transfer Kbyte Sec</Property><Property Name=”Value” VariantType=”5″>14450.625</Property></DataItem><DataItem type=”System.PropertyBagData” time=”2012-09-20T19:55:28.1079971+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><Property Name=”Instance” VariantType=”8″>c:\destionation</Property><Property Name=”Counter” VariantType=”8″>Write Transfer Total Sec</Property><Property Name=”Value” VariantType=”5″>0.3</Property></DataItem></Collection>

You see multiply counter values that have to be converted to performance data.

2. Now we check using the WFAnalyzer the converted performance data. See below. It looks okay.

Recieved DataItem <DataItem type=”System.Performance.Data” time=”2012-09-20T19:55:28.1109383+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><TimeSampled>2012-09-20T19:55:28.0638791+02:00</TimeSampled><ObjectName>SMB File Transfer</ObjectName><CounterName>Read Transfer Kbyte Sec</CounterName><InstanceName>c:\destionation</InstanceName><IsNull Type=”Boolean”>false</IsNull><Value>14450.625</Value></DataItem>

Recieved DataItem <DataItem type=”System.Performance.Data” time=”2012-09-20T19:55:28.1109383+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><TimeSampled>2012-09-20T19:55:28.1079971+02:00</TimeSampled><ObjectName>SMB File Transfer</ObjectName><CounterName>Read Transfer Total Sec</CounterName><InstanceName>c:\destionation</InstanceName><IsNull Type=”Boolean”>false</IsNull><Value>0.3</Value></DataItem>

Recieved DataItem <DataItem type=”System.Performance.Data” time=”2012-09-20T19:55:28.1109383+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><TimeSampled>2012-09-20T19:55:28.1079971+02:00</TimeSampled><ObjectName>SMB File Transfer</ObjectName><CounterName>Write Transfer Kbyte Sec</CounterName><InstanceName>c:\destionation</InstanceName><IsNull Type=”Boolean”>false</IsNull><Value>14450.625</Value></DataItem>

Recieved DataItem <DataItem type=”System.Performance.Data” time=”2012-09-20T19:55:28.1109383+02:00″ sourceHealthServiceId=”0F6B7345-4C8E-CFAF-BD7A-454E6C94B77F”><TimeSampled>2012-09-20T19:55:28.1079971+02:00</TimeSampled><ObjectName>SMB File Transfer</ObjectName><CounterName>Write Transfer Total Sec</CounterName><InstanceName>c:\destionation</InstanceName><IsNull Type=”Boolean”>false</IsNull><Value>0.3</Value></DataItem>

3. Next step is to check the write actions. This also looks okay. The “ToDWH “ writeaction should write the data to the DWH.

<WriteActions>

<WriteAction ID=”ToOps” TypeID=”SystemCenter!Microsoft.SystemCenter.CollectPerformanceData” />

<WriteAction ID=”ToDWH” TypeID=”SCDW!Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData” />

</WriteActions>

All looks okay….

Solution

After some mailing with the OM development team the answer was found: Writing multiply counters to the DWH from 1 property bag output is NOT supported! So the DWH write module has a one to one reference map that means only one rule can contain one counter. Be aware no error is reported if this happens..

The only way to solve this is to make 1 rule for every performance counter you want to store in the DWH.  Use a condition detection in the rule for filtering the correct performance counter. See below for a example.

<Rule ID=”TransferFile.ReadSec” Enabled=”true” Target=”FileTransferClient” ConfirmDelivery=”true” Remotable=”true” Priority=”Normal” DiscardLevel=”100″>
<Category>Custom</Category>
<DataSources>
<DataSource ID=”SMBFileTransfer” TypeID=”OPS.SMB.Performance.FileTransfer”> <Debug>false</Debug>
<IntervalSeconds>300</IntervalSeconds>
</DataSource>
</DataSources>
<ConditionDetection ID=”Filter” TypeID=”System!System.ExpressionFilter”>
<Expression>
<SimpleExpression>
<ValueExpression>
<XPathQuery Type=”String”>CounterName</XPathQuery>
</ValueExpression>
<Operator>Equal</Operator>
<ValueExpression>
<Value Type=”String”>Read Transfer Total Sec</Value>
</ValueExpression>
</SimpleExpression>
</Expression>
</ConditionDetection>
<WriteActions>
<WriteAction ID=”ToOps” TypeID=”SystemCenter!Microsoft.SystemCenter.CollectPerformanceData” />
<WriteAction ID=”ToDWH” TypeID=”SCDW!Microsoft.SystemCenter.DataWarehouse.PublishPerformanceData” />
</WriteActions>
</Rule>

THE END

Maybe this will help you. Till next Time.

Happy SCOMMING

Michel Kamp