A more refined way of getting your runbooks published into SMA using code

Update: If you read this, be sure to read this (finished code for smart publishing)

If you have worked with SMA for more than just preparing for giving a demo (pun very much intended), you have no doubt come to the conclusion that the “copy+paste” way of getting code into SMA is suboptimal. Actually, I’d argue that it works against the whole notion of “Infrastructure as code“.  Despite all this, I really like SMA. It’s like that weird aunt of your, who you love despite all her weirdness. I’m not saying I love SMA, but it sure is a product with really great potential.

Now, I’m luckily not the only one realizing that SMA needs a better story for actually publishing runbooks, and for most folks that story begins with a source control system. There are actually a few posts out there outlining how to use TFS or Visual Studio Online as a source control repo for SMA things. The articles I’ve found share a common flaw:

The idea of a parent-child runbook is not new. However, all of these examples assume that your child runbook is only referenced by one parent, which is of course not the case. In real-life your runbook’s relationship to each other probably look more like this:

There’s no notion of sub- or subsub-runbooks. Each runbook is a piece of code that can be referenced by any other runbook, so the idea of storing runbooks in folders and have the folder level describe the “child-ness” of a given runbook simply doesn’t work in real life.

The reason this parent/child discussion is important in SMA, is that the order which you publish runbooks will determine wether your runbooks will actually run or not. In the example above, if you for example publish Runbook1 before you publish Runbook2, Runbook1 won’t be able to start. It’s a huge bug, and something the SMA team is painfully aware of.

In the meantime, we need to construct the required logic to send runbooks in the right order to SMA. It’s embarrassing, I know.

Anyway, instead of forcing yourself into an arcane structure of folders of parent and child runbooks that will just limit your ability to get anything done, we can tap into PowerShell’s Abstract Syntax Tree to gain an understanding of the relationship between runbooks. This will enable us to scan a folder of runbooks and determine dynamically which order they should be published. In my example above, that would be something like:

This isn’t too hard to solve:
1. List all files in a folder (each file represents a runbook and contains the workflow code for that runbook)
2. Get each runbook’s child runbooks through its AST
3. Make sure the child runbooks are published before publishing the current one
4. In order to avoid publishing a single runbook multiple times, mark it as already processed.
5. Continue on until all files in the folder are processed.

Here’s how to get a list of child runbooks given a path to a file containing the code for a parent runbook:

$ThisWf = get-content $path -Raw
$ThisWfSB = [scriptblock]::Create($ThisWf)
$TokenizedWF = [System.Management.Automation.PSParser]::Tokenize($ThisWfSB,[ref]$null)
$referencedCommands = $TokenizedWF | where {$_.TYpe -eq "Command"}

Note that the $referencedCommands variable will contain all commands referenced, not just child runbooks (such as Write-output and so on). So, wee need to find the commands which represent runbooks, which should also be present in the same directory:

    foreach ($referencedCommand in $referencedCommands)
    {
        $runbookpath = get-childitem -Path $basepath -Recurse:$recurse |where {$_.BaseName -eq $referencedCommand.Content}
        if ($runbookpath)
        {
            Write-Verbose "REFERENCE: $($path.BaseName)--> $($referencedCommand.content)"
            Process-RunbookFile -path $runbookpath.FullName
        }
    }

This piece of code is actually part of a nested function which will iterate on itself until all child runbooks have been imported, and then import itself.

Also note that the current solution does not verify the actual existence of all commands used in a runbook. If one of your runbooks calls a function or workflow called “foo-bar” and the foo-bar.ps1 file isn’t present in the folder structure, we’ll just assume that that function is available via some installed module or whatever.

The last part of the puzzle is to use an arraylist to store information about already processed files, so that the same file (runbook3, for instance) doesn’t get uploaded more than once.

After creating some mock runbooks with the references I’ve outlined in the diagram above, I can test out my logic by running my script (the whole piece of code can be found below):

As you can see, the order of publishing is correct according to my outline diagram.

There is one shortcoming with my script as it sits right now: Everything will have to be re-published on each run, which can slow down a CI process considerably. For instance, if I perform a minor update on the wf3 script, I need to re-publish wf1 and wf5 as well, since these runbooks reference wf3. I’m still thinking about how to do that – keep watching this space!

The code can be found here:
https://gist.github.com/trondhindenes/faeb6c1212f4a3e2457c

Note that the code doesn’t actually implement the publishing to SMA part, which is really the easy part. The code below simply demonstrates a logic you can implement in your own code in order to publish in the right order. So, around the line saying “Write-Verbose PUBLISH” you need to put in your own code for performing the actual publish.

Hopefully this will get you a little bit closer in the quest for a sane method of publishing runbooks in a controlled fashion. You might also be interested in another CI/Version Control/SMA post I wrote a while back.