AppVeyor	Coveralls	Download

Introduction

Statistical analysis

TODO

• Use Update-TypeData
• Display values in Show-Measurement... include minimum and maximum
• Limit displayed range in Show-Measurement
• Check handling of negative values
• More tests
• Get-SlidingAverage
  • Output data similar to Group-Object by adding properties Property, Group and Average
• Filter performance counters from (Get-Counter).CounterSamples
• Progress bar
  • Implement in all cmdlets
  • Support for child bars
• Performance improvements by using ArrayList instead of +=
• Provide predefined buckets to Get-Histogram

Warning

Many cmdlets contained in this modules operate directly on the input objects and add new properties to them. This is a design decision to prevent excessive memory usage caused by copying the input data.

Histograms

Create a histogram from the currently running processes using Get-Histogram. The WorkingSet will be used to express the amount of memory used by the processes. It makes sense to specify the bucket size (-BucketWidth) as well as the number of buckets (-BucketCount).

PS C:\>$Processes=Get-Process
PS C:\>$Histogram=$Processes|Get-Histogram-Property WorkingSet64 -BucketWidth 50mb-BucketCount 10
PS C:\>$Histogram
Index Count
----------17521539425062718191102

The histogram is visualized using Add-Bar. It adds a new property to the input objects containing the bar. Note that the width is predefined to match the current console window.

PS C:\>$Histogram|Add-Bar
Index Count Bar
-------------175####################################################################################################215####################39############42###5062###71#81#91#102###

Statistical analysis

In addition to value distribution, statistics has a lot more to offer. This module overloads the cmdlet Measure-Object and adds many properties like the median and the standard deviation.

PS C:\>$Processes|Measure-Object-Property WorkingSet64
Median : 24731648
Variance : 1,86476251813904E+16
StandardDeviation : 136556307,731977
Percentile10 : 6496256
Percentile25 : 12095488
Percentile75 : 92958720
Percentile90 : 190754816
Confidence95 : 25519460,826598
Count : 110
Average : 76195169,7454546
Sum : 8381468672
Maximum : 875040768
Minimum : 4096
Property : WorkingSet64

Although those values are very interesting, visualizing them using Show-Measurement helps understanding your data set.

PS C:\>$Processes|Measure-Object-Property WorkingSet64 |Show-Measurement---------|---------|---------|---------|---------|---------|---------|---------|---------|---------|
P10
P25
A
c----C
M
P75
P90
---------|---------|---------|---------|---------|---------|---------|---------|---------|---------|
PS C:\>

Simple data

In case you have input data as primitive type (e.g. [int]), use can use ConvertFrom-PrimitiveType to construct a complex (yet still simple) type containing the values. The output can be used for any cmdlet described above.

PS C:\>1..10|ConvertFrom-PrimitiveType
Value
-----12345678910

Analyzing performance counters

Performance counters are retrieved using the builtin cmdlet Get-Counter. Unfortunately, this cmdlet wraps the samples in a separate property. ConvertFrom-PerformanceCounter changes the structure of the input data to make it easier to process. The cmdlet can be used for stream processing in a pipeline.

PS C:\>Get-Counter-Counter '\Processor(_Total)\% Processor Time'-SampleInterval 1-MaxSamples 10|ConvertFrom-PerformanceCounter-Instance _total
Timestamp Value
--------------29.03.201714:29:462,9770372058435529.03.201714:29:473,0609259674754629.03.201714:29:486,4914503919307129.03.201714:29:497,996126354384229.03.201714:29:505,1158450292001829.03.201714:29:514,6398618090314529.03.201714:29:524,5320155948842329.03.201714:29:531,7751766263647329.03.201714:29:5410,118888719775829.03.201714:29:554,1137771992306

The individual values do not tell you enough because they may be peaking once in a while. Get-SlidingAverage calculates an average over a window of the specified size:

PS C:\>Get-Counter-Counter '\Processor(_Total)\% Processor Time'-SampleInterval 1-MaxSamples 10|ConvertFrom-PerformanceCounter-Instance _total |Get-SlidingAverage-Property Value -Size 50,9990774775909330,9544369642022990,5722722271013940,5722722271013940,4173602773440850,797367872038486

Time series

In contrast to reading samples at regular intervals (as seen above for performance counters), you sometime obtain values at irregular intervals. Analyzing the timestamps will tell you as much as the data values. For the sake of this example, let's generate sample data. The timestamp is generated randomly over the last 24 hours:

$now=Get-Date$data=1..10|ForEach-Object {
[pscustomobject]@{
Timestamp=Get-Random-Minimum $now.AddDays(-1).Ticks -Maximum $now.TicksValue=Get-Random-Minimum 0-Maximum 100
}
} |Sort-Object-Property Timestamp

Let's take a closer look at the distance (interarrival time) between samples. By default, the distance is expessed in ticks but you can choose from several units using -Unit.

PS C:\>$data|Get-InterarrivalTime-Property Timestamp
Timestamp Value InterarrivalTicks
-------------------------------6362631941230562563022855416326362633763638448644918224078860863626346924516339221928813185286362635538110590729845658956806362636227950329604268983973888636263691231611392876843657843263626380043540185628109203790464636263902248199552271018127976966362639320755409926929827341440
PS C:\>$data|Get-InterarrivalTime-Property Timestamp -Unit Minutes
Timestamp Value InterarrivalMinutes
---------------------------------6362632601201227524921636263337519778304148636263366516410368494863626350097202790415446362636207092482569519636263635450691840972463626374765879526444763626377355036236898436362638535412913928813

The ticks-based timestamp in the examples above it impossibel to read without any conversion. This module also offers a cmdlet called Expand-DateTime to convert timestamps:

PS C:\>$data|Expand-DateTime-Property Timestamp
Timestamp Value DayOfWeek Year Month Hour WeekOfYear
----------------------------------------------28.03.201718:34:1823 Tuesday 20173181328.03.201719:13:3843 Tuesday 20173191328.03.201720:42:0853 Tuesday 20173201328.03.201721:41:2373 Tuesday 20173211328.03.201722:28:169 Tuesday 20173221328.03.201722:46:4720 Tuesday 20173221328.03.201722:50:0175 Tuesday 20173221329.03.201701:01:2640 Wednesday 2017311329.03.201705:11:4355 Wednesday 2017351329.03.201709:10:3977 Wednesday 20173913

Credits

Build scripts based on PSDeploy by Warren Frame