Pester: Generating Command Interface Tests

I’ve for long thought Interface Testing was the most repetitive yet critical task to let a Module or commands evolve without breaking backward compatibility, and upsetting those who depends on it.

While Unit Testing ensures the code behaves as expected and meets its design and requirements, Interface Testing for commands or functions focuses on the contracts it offers to its consumer. It ensures the contracts (the command prototype or syntax) are consistent over time.

InterfaceTestGenerator.gif

PowerShell being Metadata-driven, it is easy to get command information via Get-Command, and the returned object of type CmdletInfo, FunctionInfo or so on, gives the details of Parameters for each parameter set, or the command list of OutputTypes.

#List of Parameter Sets Name:
(Get-Command Get-Process).ParameterSets.Name

#List of Parameter name for the first ParameterSet
(Get-command Get-Process).ParameterSets[0].Parameters.Name

From those it’s trivial to consistently generate some code to test your Interfaces, in the form you’d like, and here’s an example, with a drawback I’ll explain shortly:


Function Export-StructuralUnitTestFromCommand {
param(
[parameter(Mandatory=$true,ValueFromPipelineByPropertyName = $true,ValueFromPipeline = $true)]
[System.Management.Automation.CommandInfo[]]
$command
)
begin {
$stringBuilder = New-object -TypeName System.Text.StringBuilder
$BuiltInParameters = ([Management.Automation.PSCmdlet]::CommonParameters + [Management.Automation.PSCmdlet]::OptionalCommonParameters)
}
process {
foreach ($cmd in $command)
{
$null = $stringBuilder.Clear()
$cmdName = $cmd.Name
$cmdDefaultParameterSet = $cmd.DefaultParameterSet
$null = $stringBuilder.AppendLine(@"
Describe '$cmdName' {
#getting Command Metadata
`$command = (Get-Command '$cmdName')
It 'has $cmdDefaultParameterSet as Default parameterSet' {
`$command.defaultParameterSet | Should be '$cmdDefaultParameterSet'
}
"@)
$outputTypes = $cmd.OutputType
foreach ($outputType in $outputTypes)
{
$outputTypeName = $outputType.Name
#It 'Output the Type $outputType'
$null = $stringBuilder.AppendLine(@"
It 'contains an outputType of Type $outputTypeName' {
`$command.OutputType.Type -contains [$outputTypeName] | should be `$true
}
"@)
}
$parameterSets = $cmd.ParameterSets
foreach ($ParameterSet in $parameterSets)
{
$ParameterSetName = $ParameterSet.Name
$null = $stringBuilder.AppendLine(@"
Context 'ParameterSetName $ParameterSetName' {
It 'has a parameter Set of Name $ParameterSetName' {
`$command.ParameterSets.Name -contains '$ParameterSetName' | Should be $true
}
`$ParameterSet = `$command.ParameterSets | Where-Object { `$_.'Name' -eq '$ParameterSetName' }
"@)
$parameters = $ParameterSet.Parameters | Where-Object { $_.Name -notin $BuiltInParameters}
foreach ($parameter in $parameters)
{
$ParameterName = $parameter.Name
$TypeName = $parameter.ParameterType.ToString()
$isMandatory = $parameter.isMandatory
$ValueFromPipeline = $parameter.ValueFromPipeline
$ValueFromPipelineByPropertyName = $parameter.ValueFromPipelineByPropertyName
$ValueFromRemainingArguments = $parameter.ValueFromRemainingArguments
$Position = $parameter.Position
$null = $stringBuilder.AppendLine(@"
`$Parameter = `$ParameterSet.Parameters | Where-Object { `$_.'Name' -eq '$ParameterName' }
It 'has compatible parameter $ParameterName' {
`$Parameter | Should Not BeNullOrEmpty
`$Parameter.ParameterType.ToString() | Should be $TypeName
`$Parameter.IsMandatory | Should be `$$([bool]$isMandatory)
`$Parameter.ValueFromPipeline | Should be `$$([bool]$ValueFromPipeline)
`$Parameter.ValueFromPipelineByPropertyName | Should be `$$([bool]$ValueFromPipelineByPropertyName)
`$Parameter.ValueFromRemainingArguments | Should be `$$([bool]$ValueFromRemainingArguments)
`$Parameter.Position | Should be $Position
}
"@)
}
#Closing Context block
$null = $stringBuilder.AppendLine(' }')
}
#Closing Describe Statement
$null = $stringBuilder.AppendLine('}')
Write-Output $stringBuilder.ToString()
}
}
}

view raw

TestGenerator.ps1

hosted with ❤ by GitHub

One way to use the script above:

. .\TestGenerator.ps1
Get-Command Get-Process | Export-StructuralUnitTestFromCommand | `
 Out-File ./Get-Process.tests.ps1 -Force -Encoding utf8
Invoke-Pester .\Get-Process.tests.ps1

The goal of Interface testing, is to ensure backward compatibility of your publicly exposed functions (API, as in Application  Programming Interface). You want a module to offer functionalities in the same way, independently of the evolution of the inner (private) code.

If you follow SemVer for your versioning, when you add functionality to your module and as long as the API is compatible (the test generate as above should pass), you do not increase the Major version: i.e. from 2.1.3345 to 2.2.2138

If you have to make a breaking change (and you have considered creating a new parameter set or a proxy/stub function that provides the same interface that was previously available), then you indicate the breaking change by increasing its Major version: i.e. from 2.2.2138 to 3.0.0

This means that for each release, you should generate the Interface tests, and make sure it runs against the previous release’s ones, and version accordingly.

The main flaw of the sample generator I’ve given above, is that it includes specific Parameter Sets, and name them, although I believe those should be considered internal to the command…

I will revisit this at some point, or feel free to submit a PR!

2 thoughts on “Pester: Generating Command Interface Tests”

Leave a comment