OOP 2015 - ALM in the Cloud with Visual Studio Online and Azure

27 January 2015 - Azure, Project Management, Visual Studio

At OOP 2015 conference, I will do a session about Application Lifecycle Management (ALM) in the cloud (link to German abstract). As always, I try to show mostly practical samples instead of boring slides. In this blog post I summarize the talk and reference important resources that I will mention during my session.

Video

The video is available on Channel9. Here it is:

Code

All the code that I use in my session can be found in my GitHub samples repository.

Motivation

Our company software architects was "born in the cloud". We have been running our production infrastructure in Microsoft Azure from the first day on. However, until some months ago, our ALM and test infrastructure was running on-premise. We watched closely what Microsoft did in terms of offering Team Foudation Server (TFS) in the cloud (aka Visual Studio Online or VSO). Moving our ALM system into the cloud seemed like the logical next step in our cloud strategy.

Here are the most important reasons why we had been eagerly waiting for VSO:

  1. Reduce or ideally remove the need for on-premise infrastructure
    We are by far no experts in data center operations. Microsoft's ISO-certified data centers would offer us a much more professional and secure environment for our ALM system.
  2. Reduce maintenance costs
    Keeping the on-premise TFS plus its underlying infrastructure up to date is a time-consuming task. Our job is writing software, not maintaining TFS.
  3. Make us more agile
    Our team grows. Projects are started and stopped every now and then. Our need for infrastructure (e.g. build) is not static, it fluctuates. Using an auto-scaling service in the cloud supports our agile approach of working.

At the end, everything boils down to: focus. We want to focus on building our SaaS offering time cockpit and related software products and remove distractions like maintaining infrastructure.

What is ALM to us?

ALM means different things for different teams. For us, ALM has to cover the following major aspects:

  1. Source Code Control
    This is the most important topic. We need a robust and secure code repository.
  2. Automated Build (including execution of Unit Tests)
    We never hand out software to customers that has been built on developer workstations. Our build processes are fully automated to minimize the risk of human mistakes and guarantee constantly high quality.
  3. Continuous Delivery
    We deploy code to our test and production systems frequently (many times per week). Doing that manually would be too time consuming and error prone. Deployment has to be an integral part of our build and test infrastructure.
  4. Project and Requirements Management
    We work based on Kanban. Our ALM system has to support this agile approach (e.g. backlog management, Kanban board, etc.).
  5. Customer Service
    We need a tool for customer service (trouble ticket management). It has to be integrated with project management so that we can create work items based on tickets e.g. in case of bugs or important feature requests.
  6. Team Communication
    Of course we use email and Skype for team communication. However, both systems have drawbacks. Email is too heavy for short, informal notes and discussions. Skype is great for that but does not offer appropriate history and search. We need a tool integrated with our Customer Service and Project Management system that supports related discussions and notes.
  7. Time Tracking and Financial Project Controlling
    This aspect is very important for us. We need a system for managing billable hours in case of customer projects (e.g. implementation of time cockpit at a customer, consulting projects, etc.) and for tracking our internal development effort (e.g. for managing our internal time budget per sub-project or work item).

What to Consider?

Before you decide whether cloud-based ALM is right for you, you should consider the following questions:

  1. Security
    Going into the cloud does not necessarily mean less security. In our case, cloud means a more secure system. We simply cannot/don't want to run a professional, high-secure, and high-available ALM infrastructure on-premise. Visual Studio Online for instance is available in Europe, has ISO 27001 certification, and includes European Model Clauses into its contracts. However, internal policies or legal restrictions might stop you from putting your ALM systems in the cloud.
  2. Costs
    The cost structure of ALM in the cloud is entirely different from what you might be used to. No single, big, upfront investment. You have to pay based on the number of users and/or your usage (e.g. build hours) instead. You have to invest some time in calculating the costs related to the ALM services you want to use. VSO for instance has detailed pricing information and an online calculator for that.

    If you develop software based on Microsoft technology, you should definitely check out the benefits that you get from Microsoft's Partner Program and its related certification. It can save you lots of money as you could get ALM resources for free.

  3. Internet connection
    You will still have on-premise components (e.g. your development workstations). Therefore, ALM in the cloud means that you have to have a powerful and reliable internet connection. We even moved to a new office location in order to get better internet.
  4. Limited customization
    SaaS solutions like VSO are multi-tenant. A single server (farm) handles lots of different tenants. As a result, many providers limit the degree to which you can tailor the system to your needs. In VSO for instance you still cannot customize e.g. work items, workflows, etc. the way you are used to from your on-premise TFS (Microsoft has informally (in blog comments) said that enabling customization is a goal for Q1 2015)If you use Jira, not all the 3rd party plugins you use might support Atlassian’s Cloud already.  You have to verify that the cloud ALM system of your choice offers all the features you need. You must not assume that the cloud counterpart of the ALM system you use on-premise offers exactly the same feature set.
  5. Migration
    You have to consider a migration project. We took the opportunity of starting a big sub-project (we are switching from WPF/Silverlight to HTML5 for our client). We started development of this new component in VSO from the very beginning. We decided not to migrate TFS history. Instead, we will keep our existing on-premise TFS alive for some time for maintaining legacy code and old change history. However, we have reduced maintenance to an absolute minimum.

Single, Integrated Solution or Best of Breed

On-premise, companies often prefer integrated solutions over a best-of-breed approach. The reasons are maintenance and infrastructure costs. Imagine you want to combine three different systems for requirements management, source code control plus build, and bug tracking. Each of them might use different technology stacks (e.g. .NET, Java, PHP). Maintaining the underlying infrastructure would be costly and time-consuming.

That changes in the cloud entirely. You do not have to care about infrastructure. As long as you tools of choice support appropriate, platform-independent standards for webservices (e.g. REST, OAuth2, OData, webhooks, etc.), integrating them isn't magic.

So best-of-breed is much easier in the cloud than it is on-premise. However, cross-tool integration is never a free lunch.

Our Current Toolset

This is the toolset we currently use for our own ALM:

  • Jira in the Atlassian Cloud for agile project management (Kanban)
  • Zendesk for customer service - it offers a Jira-cloud-integration out of the box
  • Visual Studio Online for source code control (Git), build, and continuous integration
  • Slack for informal team communication - it offers e.g. Zendesk-integration out of the box
  • Time cockpit for time tracking, project controlling, and billing - offers integrations to Jira, Zendesk, and VSO

All of these products run in the cloud and all of them support platform-independent web services. Therefore, it was quite simple to integrate them. The following image shows the integration of the different tools:

Read more about how we build and run our brand new web client for time cockpit in this blog article.

Demo

During my session at OOP I will demonstrate setting up an ALM system with most parts of our current toolset. It will cover:

  • Creation of a new VSO project environment for source code control, project management, and automated build
  • Integration of the VSO project with Microsoft Azure for continuous delivery
  • Integration of Jira with VSO using webhooks and REST webservices

I have created a small sample demonstrating how easy it is to integrate Jira and VSO via webservice. You can use Jira's webhooks to get a message whenever an issue is created. Inside of the webhook you can use TFS' REST API to create e.g. VSO backlog items. You can find the sample code in my GitHub samples repository.

Next Steps

Over time, we plan to replace Jira with VSO because of the following two reasons:

  • Cost reduction
    As Microsoft partners, we do not have to pay (or at least pay less) for VSO. Additionally, VSO offers free accounts for stakeholders who just use VSO for project and requirements management.
  • Customization is coming
    One of the major reasons why we are using Jira is that VSO does not support customizations in the cloud. This will hopefully change in the next few months.