I’ve for long thought Interface Testing was the most repetitive yet critical task to let a Module or commands evolve without breaking backward compatibility, and upsetting those who depends on it.
While Unit Testing ensures the code behaves as expected and meets its design and requirements, Interface Testing for commands or functions focuses on the contracts it offers to its consumer. It ensures the contracts (the command prototype or syntax) are consistent over time.
PowerShell being Metadata-driven, it is easy to get command information via Get-Command, and the returned object of type CmdletInfo, FunctionInfo or so on, gives the details of Parameters for each parameter set, or the command list of OutputTypes.
#List of Parameter Sets Name: (Get-Command Get-Process).ParameterSets.Name #List of Parameter name for the first ParameterSet (Get-command Get-Process).ParameterSets[0].Parameters.Name
From those it’s trivial to consistently generate some code to test your Interfaces, in the form you’d like, and here’s an example, with a drawback I’ll explain shortly:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Function Export-StructuralUnitTestFromCommand { | |
| param( | |
| [parameter(Mandatory=$true,ValueFromPipelineByPropertyName = $true,ValueFromPipeline = $true)] | |
| [System.Management.Automation.CommandInfo[]] | |
| $command | |
| ) | |
| begin { | |
| $stringBuilder = New-object –TypeName System.Text.StringBuilder | |
| $BuiltInParameters = ([Management.Automation.PSCmdlet]::CommonParameters + [Management.Automation.PSCmdlet]::OptionalCommonParameters) | |
| } | |
| process { | |
| foreach ($cmd in $command) | |
| { | |
| $null = $stringBuilder.Clear() | |
| $cmdName = $cmd.Name | |
| $cmdDefaultParameterSet = $cmd.DefaultParameterSet | |
| $null = $stringBuilder.AppendLine(@" | |
| Describe '$cmdName' { | |
| #getting Command Metadata | |
| `$command = (Get-Command '$cmdName') | |
| It 'has $cmdDefaultParameterSet as Default parameterSet' { | |
| `$command.defaultParameterSet | Should be '$cmdDefaultParameterSet' | |
| } | |
| "@) | |
| $outputTypes = $cmd.OutputType | |
| foreach ($outputType in $outputTypes) | |
| { | |
| $outputTypeName = $outputType.Name | |
| #It 'Output the Type $outputType' | |
| $null = $stringBuilder.AppendLine(@" | |
| It 'contains an outputType of Type $outputTypeName' { | |
| `$command.OutputType.Type -contains [$outputTypeName] | should be `$true | |
| } | |
| "@) | |
| } | |
| $parameterSets = $cmd.ParameterSets | |
| foreach ($ParameterSet in $parameterSets) | |
| { | |
| $ParameterSetName = $ParameterSet.Name | |
| $null = $stringBuilder.AppendLine(@" | |
| Context 'ParameterSetName $ParameterSetName' { | |
| It 'has a parameter Set of Name $ParameterSetName' { | |
| `$command.ParameterSets.Name -contains '$ParameterSetName' | Should be $true | |
| } | |
| `$ParameterSet = `$command.ParameterSets | Where-Object { `$_.'Name' -eq '$ParameterSetName' } | |
| "@) | |
| $parameters = $ParameterSet.Parameters | Where-Object { $_.Name -notin $BuiltInParameters} | |
| foreach ($parameter in $parameters) | |
| { | |
| $ParameterName = $parameter.Name | |
| $TypeName = $parameter.ParameterType.ToString() | |
| $isMandatory = $parameter.isMandatory | |
| $ValueFromPipeline = $parameter.ValueFromPipeline | |
| $ValueFromPipelineByPropertyName = $parameter.ValueFromPipelineByPropertyName | |
| $ValueFromRemainingArguments = $parameter.ValueFromRemainingArguments | |
| $Position = $parameter.Position | |
| $null = $stringBuilder.AppendLine(@" | |
| `$Parameter = `$ParameterSet.Parameters | Where-Object { `$_.'Name' -eq '$ParameterName' } | |
| It 'has compatible parameter $ParameterName' { | |
| `$Parameter | Should Not BeNullOrEmpty | |
| `$Parameter.ParameterType.ToString() | Should be $TypeName | |
| `$Parameter.IsMandatory | Should be `$$([bool]$isMandatory) | |
| `$Parameter.ValueFromPipeline | Should be `$$([bool]$ValueFromPipeline) | |
| `$Parameter.ValueFromPipelineByPropertyName | Should be `$$([bool]$ValueFromPipelineByPropertyName) | |
| `$Parameter.ValueFromRemainingArguments | Should be `$$([bool]$ValueFromRemainingArguments) | |
| `$Parameter.Position | Should be $Position | |
| } | |
| "@) | |
| } | |
| #Closing Context block | |
| $null = $stringBuilder.AppendLine(' }') | |
| } | |
| #Closing Describe Statement | |
| $null = $stringBuilder.AppendLine('}') | |
| Write-Output $stringBuilder.ToString() | |
| } | |
| } | |
| } |
One way to use the script above:
. .\TestGenerator.ps1 Get-Command Get-Process | Export-StructuralUnitTestFromCommand | ` Out-File ./Get-Process.tests.ps1 -Force -Encoding utf8 Invoke-Pester .\Get-Process.tests.ps1
The goal of Interface testing, is to ensure backward compatibility of your publicly exposed functions (API, as in Application Programming Interface). You want a module to offer functionalities in the same way, independently of the evolution of the inner (private) code.
If you follow SemVer for your versioning, when you add functionality to your module and as long as the API is compatible (the test generate as above should pass), you do not increase the Major version: i.e. from 2.1.3345 to 2.2.2138
If you have to make a breaking change (and you have considered creating a new parameter set or a proxy/stub function that provides the same interface that was previously available), then you indicate the breaking change by increasing its Major version: i.e. from 2.2.2138 to 3.0.0
This means that for each release, you should generate the Interface tests, and make sure it runs against the previous release’s ones, and version accordingly.
The main flaw of the sample generator I’ve given above, is that it includes specific Parameter Sets, and name them, although I believe those should be considered internal to the command…
I will revisit this at some point, or feel free to submit a PR!
@gaelcolas 
gif is showing very weirdly in chrome – changes colour to green, text not showing clearly.
LikeLike
Yep, I saw that with chrome.. I’m using gifcam to create gifs, and I’m new to this. I will probably try to upgrade and use AVI, it might give better UX.
LikeLike