I’ve just started setting up some continuous deployment for my personal websites. All of the sites are hosted within Azure App Services and the sources are located on either GitHub or BitBucket. By having the source code located on a public accessible repository (be it private or public), it’s rather easy to connect Azure to these locations.

On my day-job I come across a lot of web- and desktop applications which also need continuous integration and deployment steps in order for them to go live. For some of these projects I’ve used Octopus Deploy and currently looking towards Azure Release Management. These are all great systems, but they offer quite a lot of overhead for my personal sites. Currently my, most important, personal sites are so called static websites using MiniBlog (this site) and Hugo (for keto.jan-v.nl). Some of the other websites I have aren’t set up with a continuous deployment path yet.

I don’t really want to set up an Octopus Deploy server or a path in Azure Release Management for these two sites. Lucky for me, the Azure team has come up with some great addition in order to provide some custom deployment steps of your Azure App Service. In order to set this up, you need to enable the automatic deployments via the `Deployment Options` blade in the Azure portal.

image

Normally, when you have set up your site to be deployed every time some change occurs in a specific branch of your repository the Azure App Service deployment system tries to build your site and place the output to the `wwwroot` folder on the file system. Because I don’t need any msbuild steps whatsoever, I need to override this step and create my own, custom, deployment step.

Setting up such a thing is quite easy, you just have to create a `.deployment` file in the root of your repository and specify the build/deployment script which should be executed. This functionality is provided by Kudu, which Azure uses in order to deploy Git repositories to the Azure App Service.

You can specify a custom script in this deployment file, this can either be a ‘normal’ command script (cmd or bat) or a PowerShell script. I have chosen for PowerShell as it offers me a bit more flexibility compared to a normal command script.

The contents of the deployment file aren’t very exciting. For my scenario it looks like the following:

[config]
command = powershell -NoProfile -NoLogo -ExecutionPolicy Unrestricted -Command "& "$pwd\deploy.ps1" 2>&1 | echo"

This will activate the custom deployment step within the Azure App Service as you can see in the following picture (Running custom deployment command…).

image

 

The contents of my PowerShell script, deploy.ps1, aren’t very exciting either. The MiniBlog project is just a normal ASP.NET Website, so I just have to copy the contents from the repository folder to the folder of the website.

robocopy "$Env:DEPLOYMENT_SOURCE\Website" "$Env:DEPLOYMENT_TARGET" /E

You can do some more advanced stuff in your deployment script. For my Hugo website I had to tell the Hugo assembly to build my website. So the contents of this deploy.ps1 script are similar to this.

# 1. Variable substitutions
if ($env:HTTP_HOST -ne "") {
    echo "doing substitutions on $Env:DEPLOYMENT_SOURCE\config.toml"
    gc "$Env:DEPLOYMENT_SOURCE\config.toml" | %{ $_ -replace '%%HTTP_HOST%%', $env:WEBSITE_HOSTNAME } | out-file -encoding ascii "$Env:DEPLOYMENT_SOURCE\config.new.toml"
    mv "$Env:DEPLOYMENT_SOURCE\config.toml" "$Env:DEPLOYMENT_SOURCE\config.old.toml"
    mv "$Env:DEPLOYMENT_SOURCE\config.new.toml" "$Env:DEPLOYMENT_SOURCE\config.toml"
    rm "$Env:DEPLOYMENT_SOURCE\config.old.toml"
} else {
    echo "not doing any substitutions"
}

# 2. Hugo in temporary path
& "$Env:DEPLOYMENT_SOURCE/bin/hugo.exe" -s "$Env:DEPLOYMENT_SOURCE/" -d "$Env:DEPLOYMENT_TARGET/public" --log -v

# 3. Move the web.config to the root
mv "$Env:DEPLOYMENT_SOURCE/web.config" "$Env:DEPLOYMENT_TARGET/public/web.config"

Still not very exciting of course, but it shows a little what can be achieved. I’m not aware of any limitations for these deployment scripts, so anything can be placed inside it. If you need to do something with specific assemblys, like the hugo.exe, you will need to put them in your repository, or some other location which can be accessed by the script.

You can also view the output of your script in the Azure Portal. All data which is outputted by the script (Write-Host, echo, etc.) is shown in this Activity Log.

image

Useful when debugging your script.

If you have any secrets in your web application/site (like connection strings, private keys, passwords, etc.), it might be a good idea to use this custom deployment step to substitute the committed values to the actual values. If these values are stored in the Azure Key Vault, you can just access the key vault and make sure the correct values are placed within your application before it’s deployed.

Using these deployment scripts can help you out when you have some simple scenarios. If your system is a bit more complex or are working in a professional environment, I’d advise to check out one of the more sophisticated deployment systems, like Octopus Deploy or Azure Release Management. These systems offer a quite a bit more options out of the box and it’s easier to manage the steps, security and insights of a deployment.

Next I’ll try to update an Umbraco site of mine to make use of this continuous deployment scenario. This should be rather easy also as it only needs to call msbuild, which is the default action the Azure App Service deployment option invokes.

In my previous post I’ve talked about creating new projects in Octopus Deploy in order to deploy projects to different environments. In this post I’ll explain a bit on how to create Octopus Deploy packages for your Visual Studio projects via Teamcity.

To enable packaging for Octopus you’ll need to include the Octopack NuGet package to the project you are packaging. In my case this will be the Worker project since I’m only working with Microsoft Azure solutions at the moment.

Once this package is successfully installed you will have to modify the project file also to enforce adding files in the Octopus package.

The following line will have to be added to the propertygroup of your build configuration

<OctoPackEnforceAddingFiles>True</OctoPackEnforceAddingFiles>

For reference, a complete propertygroup:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType>pdbonly</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <OctoPackEnforceAddingFiles>True</OctoPackEnforceAddingFiles>
</PropertyGroup>

Your project is now ready to start packing. In my case I had to add a nuspec file also, because the worker project contained an application which has to be deployed to on an Azure Cloud Service.

A nuspec file for Octopus Deploy looks exactly the same as one you would create for NuGet itself. Mine looks like this:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
	<metadata>
		<version>$version$</version>
		<authors>Jan de Vries</authors>
		<owners>Jan de Vries</owners>
		<id>MyCustomer.Azure.MyProduct</id>
		<title>Jan de Vries his cool product</title>
		<requireLicenseAcceptance>false</requireLicenseAcceptance>
		<description>The cool product used by the Jan de Vries software.</description>
		<copyright>Jan de Vries 2015</copyright>
	</metadata>
	<files>
		<!-- Add the files (.cscfg, .csdef) from your Azure CS project to the root of your solution  -->
		<file src="..\MyCustomer.Azure.MyProduct\ServiceDefinition.csdef" />
		<file src="..\MyCustomer.Azure.MyProduct\ServiceConfiguration.Dev.cscfg" />
		<file src="..\MyCustomer.Azure.MyProduct\ServiceConfiguration.Acc.cscfg" />
		<file src="..\MyCustomer.Azure.MyProduct\ServiceConfiguration.Prod.cscfg" />
		<!-- Add the files .wadcfg file to the root to get the diagnostics working  -->
		<file src="..\MyCustomer.Azure.MyProduct\MyCustomer.Azure.MyProduct.WorkerContent\*.wadcfg" />
		<file src="..\MyCustomer.Azure.MyProduct\MyCustomer.Azure.MyProduct.WorkerContent\*.wadcfgx" />
		<!-- Add the service/console application to the package -->
		<file src="..\MyCustomer.Azure.MyProduct.Worker\Application\*.dll" target="Application"/>
		<file src="..\MyCustomer.Azure.MyProduct.Worker\Application\*.exe" target="Application"/>
		<file src="..\MyCustomer.Azure.MyProduct.Worker\Application\*.config" target="Application"/>
	</files>
</package>

Keep in mind, the nuspec file name should be exactly the same as the project name. If you fail to do so, the file will be ignored.

This is all you’ll have to to in Visual Studio. You can of course create your packages via the commandline, but it’s better to let Teamcity handle this.

In order to use Octopus Deploy from within Teamcity you’ll need the teamcity plugin which can be downloaded from the Octopus Deploy download page. After installing the plugin, three new runner type features will be available when creating a new build step.

image

I’m just using the Create release option, since this one is also capable of deploying a release to an environment. We don’t use the Promote release option as we want this to be a conscious, manual, step in the process.

There’s also a new section in the Visual Studio (sln) runner type which enables you to run OctoPack on the solution/projects.

image

If you want to use this for a build step, just enable the checkbox and be sure to set a proper build number in the OctoPack package version box.

On the image below you can check out the settings I’m using to create a new release with Octopus Deploy.

image

As you can see, I’ve added an additional command line argument telling the deployment timeout to be set on 30 minutes.

--deploymenttimeout=00:30:00

This is because the current builds take about 12 to 17 minutes to be deployed to a Cloud Service and the default (configured) Teamcity build timeout is set lower amount. Therefore all deployment builds will be marked as failed if you don’t add this argument.

I’ve also marked the Show deployment progress. This will make sure all Octopus Deploy output will get printed into Teamcity. If something fails, you’ll be able to check it out within Teamcity, assign someone to the failed build and he/she will have all information necessary to fix the failure.

Well, that’s about it on creating a nice starter continuous deployment environment. You can expand this in a way of your liking of course, but these are the basics.

The latest project I was working on didn’t have a continuous integration and continuous deployment environment set up yet. Creating a continuous integration environment was rather easy as we were already using Teamcity and adding a couple of builds isn’t much of a problem. The next logical step would be to start setting up continuous deployment.

I started out by scripting a lot of PowerShell to manage our Azure environment like creating all SQL servers, databases, service busses, cloud services, etc. All of this worked quite well, but managing these scripts was quite cumbersome.

A colleague of mine told me about Octopus Deploy. They had started using this deployment automation system on their project and it sounded like the exact same thing I was doing, just a lot easier!

Setting up the main server and it’s agents (tentacles) was quite easy and doesn’t need much explanation. The pricing of the software isn’t bad either. You can start using it for free and when you need to use more as 10 tentacles or 5 projects, you’ll start paying $700 for the professional license. Spending $700 is still a lot cheaper as paying the hourly rate of a PowerShell professional to create the same functionality.

One of the first things you want to do when starting out with Octopus Deploy is creating your own deployment workflow. Even though this is possible it’s better to navigate around a bit and think about how you want to deploy your software.

The basis of any deployment is having a deployment environment. So, the first thing you need to do is create these environments, like Dev, Test, Acc, Prod and assign a tentacle to each of these.

image

Adding a tentacle to an environment can be done by pressing the Add machine button.

After having created these environments, you can start creating your deployment workflow. The default Octopus experience already provides the most basic steps you might want to use, like running a PowerShell script or deploy a NuGet package somewhere.

image

To make my life easier while deploying to Azure, I’ve created some steps of my own for deploying a Cloud Service, Swapping VIP, checking my current Azure environment and deploying a website using MSDeploy.

image

Creating your own build steps is fairly easy. When creating a new build step you have to override one of the existing steps. For my own Cloud service build step I’ve used the default Deploy to Windows Azurestep and made sure I didn’t had to copy paste the generic fields all the time.

The deployment of a website project was a bit harder compared to deploying a Cloud service. I had already discovered this when deploying the complete environment with just PowerShell, so this wasn’t new for me. The linked article on this (above) describes in-depth on which steps you have to undertake to get this working. For reference I’ll share the script I’ve used in this build step.

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.Web.Deployment")

# A collection of functions that can be used by script steps to determine where packages installed
# by previous steps are located on the filesystem.
 
function Find-InstallLocations {
    $result = @()
    $OctopusParameters.Keys | foreach {
        if ($_.EndsWith('].Output.Package.InstallationDirectoryPath')) {
            $result += $OctopusParameters[$_]
        }
    }
    return $result
}
 
function Find-InstallLocation($stepName) {
    $result = $OctopusParameters.Keys | where {
        $_.Equals("Octopus.Action[$stepName].Output.Package.InstallationDirectoryPath",  [System.StringComparison]::OrdinalIgnoreCase)
    } | select -first 1
 
    if ($result) {
        return $OctopusParameters[$result]
    }
 
    throw "No install location found for step: $stepName"
}
 
function Find-SingleInstallLocation {
    $all = @(Find-InstallLocations)
    if ($all.Length -eq 1) {
        return $all[0]
    }
    if ($all.Length -eq 0) {
        throw "No package steps found"
    }
    throw "Multiple package steps have run; please specify a single step"
}

function Test-LastExit($cmd) {
    if ($LastExitCode -ne 0) {
        Write-Host "##octopus[stderr-error]"
        write-error "$cmd failed with exit code: $LastExitCode"
    }
}

$stepName = $OctopusParameters['WebDeployPackageStepName']

$stepPath = ""
if (-not [string]::IsNullOrEmpty($stepName)) {
    Write-Host "Finding path to package step: $stepName"
    $stepPath = Find-InstallLocation $stepName
} else {
    $stepPath = Find-SingleInstallLocation
}
Write-Host "Package was installed to: $stepPath"

Write-Host "##octopus[stderr-progress]"
 
Write-Host "Publishing Website"

$websiteName = $OctopusParameters['WebsiteName']
$publishUrl = $OctopusParameters['PublishUrl']

$destBaseOptions = new-object Microsoft.Web.Deployment.DeploymentBaseOptions
$destBaseOptions.UserName = $OctopusParameters['Username']
$destBaseOptions.Password = $OctopusParameters['Password']
$destBaseOptions.ComputerName = "https://$publishUrl/msdeploy.axd?site=$websiteName"
$destBaseOptions.AuthenticationType = "Basic"

$syncOptions = new-object Microsoft.Web.Deployment.DeploymentSyncOptions
$syncOptions.WhatIf = $false
$syncOptions.UseChecksum = $true

$enableAppOfflineRule = $OctopusParameters['EnableAppOfflineRule']
if($enableAppOfflineRule -eq $true)
{
    $appOfflineRule = $null
    $availableRules = [Microsoft.Web.Deployment.DeploymentSyncOptions]::GetAvailableRules()
    if (!$availableRules.TryGetValue('AppOffline', [ref]$appOfflineRule))
    {
        throw "Failed to find AppOffline Rule"
    }
    else
    {
        $syncOptions.Rules.Add($appOfflineRule)
        Write-Host "Enabled AppOffline Rule"
    }
}

$deploymentObject = [Microsoft.Web.Deployment.DeploymentManager]::CreateObject("contentPath", $stepPath)

$deploymentObject.SyncTo("contentPath", $websiteName, $destBaseOptions, $syncOptions)

You can probably find it out yourself, but these are the parameters used in the script.

image

After having created your own custom steps it’s time to create a deployment workflow. Create a new Project in Octopus Deploy and head down to the Process tab. Over here you can add all necessary steps for your deployment which might look like this in the end.

image

As you will probably notice, you can configure each step to be executed per environment. In this deployment workflow, we don’t want a manual intervention step before swapping the VIP. We also don’t want to distribute our software across the world, so the East US step is also turned off for development deployments.

The deployment on the Dev environment has ran a couple of times and is up to date with the latest software version. When you are happy with the results, you can choose to upgrade it to a different environment. Depending on the Lifecycle you have configured you can upgrade the deployment to any environment or just to the next stage. We have configured the lifecycle so a package has to be installed on the Development first, then Acceptance and if that environment is proven to be good enough, it will be pushed towards Production. The current status of my testing project looks like this:

image

Zooming in on a specific release, you can see a button to promote the package to the next environment.

image

I hope this helps a bit to see what’s possible with Octopus Deploy. Personally I think it’s a very nice system which really helps gaining insights on your software deployments and works a lot better as scripting your own PowerShell deployments from scratch.

If there’s still something which needs a bit more in-depth explanation or detail, let me know so I can add it in an upcoming post. Keep in mind, I’ve only used Octopus Deploy in an Azure environment, but I’m sure an on-premise installation will work about the same.