Service Management Automation: Dealing with failure

I’ve been working some more on SMA the last couple of weeks. If you read my previous blog posts you’ll know that I’m not entirely happy about how SMA lacks a real versioning component, although it is possible to get around that, as I showed.

Working with complex SMA scripts, I’ve also found that debugging is hard. Not as hard as the dreaded PowerShell story in the “regular” Orchestrator product, but almost. It’s funny that the Visual Studio crowd can perform real-time debigging of VMs and websites running in Azure, but us admin types cannot even debug stuff running in a server underneath our desk. Oh well, enough of that.

I’m a firm believer of proper error-handling, even in simple automation scripts. Especially when working with a “partial” black box product such as SMA, we need to be 100% percent sure that we deal with errors in a predictable way. Using PowerShell workflows, that is not always simple. After some reading up on how PowerShell Workflows work (um), I think I’ve come up with a good blueprint. The idea is that each InlineScript returns its state to the overall workflow, using a custom object. That means that you can trigger a second inlinescript if the first one failed, for instance.

One important thing to note is that when your storing the output of an InlineScript in a variable, that script should not output any Write-Output information, as these will screw up the object. Write-Verbose,on the other hand is fine, as these won’t be sent to the returned object in any case. So, here’s my example workflow:

workflow ErrorHandling
{
    $verbosepreference = "Continue"
    #region firstblock
    $FirstBlockResult = inlinescript {
        $verbosepreference = $using:verbosepreference

        $output = [PSCustomObject]@{
        }

        Try {
            $ErrorActionPreference = "Stop"
            #This is the main logic of the script
            [string]$a = "Test"

            #Any variables you need to output from the inlinescript can be put here:
            $output | add-member -MemberType NoteProperty -Name a -Value $a

            #Uncomment the following line to simulate failure
            [int]$b =$a  

            #Don't use write-output, as it will be passed to the result object.
            #Write-Output "Dont do this at home"
        }
        Catch {
            $errorMessage = $Error[0].Exception
        }
        Finally {
            $output | add-member -MemberType NoteProperty -Name error -Value $errorMessage
            $ErrorActionPreference = "Continue"
        }

        #The scriptblock cannot output any "write-output" messages as these will impact the output object ($FirstBlockResult"
        $output
    }
    #endregion

    $a = $FirstBlockResult.a
    write-verbose "Parent script: the value of a is $a"

    if ($FirstBlockResult.error)
    {
        Write-output "Bad stuff happened"
        $FirstBlockResult.Error
        $PerformCleanup = $true
    }

    if ($performcleanup)
    {
        #region failurecatching
        $SecondBlockResult = inlinescript {

            $verbosepreference = $using:verbosepreference
            $ErrorActionPreference = "Stop"

            #The "a" variable is send from the first scriptblock through the parent workflow
            $a  = $using:FirstBlockResult.a
            $verbosepreference = $using:verbosepreference
            Write-verbose "This is the cleanp script"
            write-verbose "This is the second block. the value of a (from the first scriptblock) is $a"
        }
        #endregion
    }
    Else
    {
        Write-verbose "Firstblock Script ran successfully"
        $FirstBlockResult.a
    }

}

What the workflow does:
First, this workflow fails. I’m assigning a string value to an integer variable, just to quickly force an error situation. You can comment out the “$b = $a” line to test it running successfully.

The output of the first InlineScript block is saved to the variable $FirstBlockResult. If this variable has an “error” attribute, I know that the first inlinescript failed, and I can perform a cleanup or whatever, using the second scriptblock. I’m also sending the variable “$a” from the firstscriptblock up to the workflow and then down to the second scriptblock to illustrate how to pass stuff between blocks.

Hopefully this can serve as a boilerplate workflow with error-handling as you venture into the dimly lit hallways of SMA.

PS: Remember, failure is not always bad. If you have a “dispatcher” workflow which calls other “child workflows”, those child workflows should potentially fail if they’re not able to do their business, to let the dispatcher workflow take corrective action or report failure through monitoring or whatever.