Introducing Shape Up at Houseful

The business-to-business side of Houseful is composed of multiple domain teams who have ownership over particular areas of our core product, Alto. I work on one of these teams and we had a new project which was to build a developer portal to help other companies be able to integrate with our APIs. Typically, our team would have followed a two-week sprint cycle to achieve this quarterly objective. However, for this particular project, we decided to experiment with a new way of working called the Shape Up method.

Rationale

Although two-week sprints generally worked okay, we encountered some common problems that software engineering teams often face, including:

• Of every ten days worked, almost an entire day was dedicated to agile ceremonies
• Sprints are incremental and could leave the team feeling like they're working on an infinite project with no end in sight
• It was difficult to carve out dedicated time to focus on development due to day to day tasks which increased the likelihood of context switching
• When working within a large quarterly boundary, it could be challenging to select the right solution for the problems because it was too big of a timeframe to map out the entire scope of the project

To try and alleviate some of these problems we decided to trial the Shape Up method on an upcoming piece of work. Having worked like this before, I was hoping it would:

• Increase focus time and build momentum
• Reduce meeting time
• Improve team autonomy and collaboration
• Increase our regularity and predictability of shipping value to our customers
• Boost team accountability and motivation

Shape Up

Shape Up is a product management methodology, developed by Basecamp, designed to help teams "feel fast again". There's an entire book, written by Ryan Singer, which explains the method in detail. In this article, we'll explore some of the key concepts and discuss how it worked for us on an actual project.

Some of the high level differences compared to how we'd previously worked were:

• We'd work in a six-week cycle instead of a two-week sprint. Afterwards, we'd schedule a two-week cool down to focus on technical health and any bugs that came in during the cycle.
• The project would be production ready and released within this six-week cycle. The building team would have full responsibility for this.
• We'd use scopes instead of a product backlog.
• Hill charts would be used to monitor progress instead of health checks.
• The only regular meetings would be a daily standup and a product check in once a week.
• We'd address risks and rabbit holes early on in the cycle by doing actual work.

Because the rest of the organisation wasn't working in this way, we weren't able to follow the method religiously. The "pitching" and "betting" process (we'll cover this terminology later in this article) would be adapted but the "shaping" and "building" parts would follow the method directly.

Shaping The Developer Portal

Before we started our build cycle, we shaped the work. Shaping is a process that involves transforming a raw idea into something that is neither too abstract nor too concrete, but instead sits somewhere in the middle.

The raw idea was to build a developer portal to allow external companies to integrate with Alto's APIs. The problem we were trying to solve was that integrating with Alto required our engineering team to integrate with third-parties. Our goal was to shift the process to an automated system that would enable third parties to integrate with us instead. The idea was led by product in collaboration with design and engineering. We first sketched out a high level customer journey:

You can see from the diagram, it's rough and unfinished but it's been thought through and the flow is detailed enough to see how the project could fit together. It's solved in some capacity but is missing detail.

We also needed to call out the boundaries (what not to do). This is important because it should minimise the risk of runaway projects that don't fit the product "appetite".

Appetite

Appetite is product's way of quantifying how much an idea is worth. For most ideas there's a version you could spend six months, six weeks or six days on. Product need to decide, what's this idea worth?

In the instance of our developer portal - it's a new idea. We didn't know if our solution would match the expectation of the customers using it. It wouldn't be worth six months of dedicated development time to establish this. We decided six weeks would be an appropriate amount of time to ship something valuable. We could then collect customer feedback, shape it and future iterations could be based on this feedback.

Set boundaries

For our six-week cycle to work, calling out what not to do was important. Before we started the build cycle, we made the call that:

• We'd focus on solving the problem, not delivering the "perfect" solution (we'd iterate based on customer feedback)
• The process of getting an integration "live" in Alto would not be included in this scope of work - we'd accept this would be a manual process for now
• While the bigger picture would include integrating with the consumer side of Houseful, it would not be considered for this cycle
• Because of an initially low customer base, we'd accept the entire journey might not be automated

Finding the elements

After we'd established a rough solution, we sketched out the customer journey with a bit more detail. Our goal was to identify the key elements of the solution, leaving room for design and additional detail to be added later on. For example our "build an integration" form page:

Risks and rabbit holes

When shaping work, we want to ensure we've called out any risks early on. We want our probability of delivering within our six-week cycle to be as thin tailed as possible:

We should do our best to de-risk the solution by also establishing what the rabbit holes are. We do this by analysing the solution and raising questions such as: "does this require new technical work we've never done?" and "is there a decision we need to settle in advance so it doesn't trip us up?". For the developer portal we established that:

• We'd need a separate authentication database and registration flow so that our developer portal users weren't the same as our Alto users - we hadn't built this before.
• We'd need a public page which wasn't protected by an authentication guard. Our entire codebase was protected so we'd have to refactor the guard implementation.
• We needed to generate Cognito keys for each integration - this wasn't something the team had done before.

To tackle the challenge of building the authentication and public pages, we held a small mobbing session with other engineers. The goal of this session was to write disposable code that would help us gain a better understanding of what needed to be done. There were still unknowns, but we'd de-risked it enough to reduce the probability it would cause a large delay. For the Cognito keys, we were reasonably sure of the solution, it was just new code but there were other engineers who we could work with if we got stuck so we were happy that it wouldn't be a severe risk.

Betting

Traditionally in Shape Up we would typically bring a project to the betting table, where we place bets on the work that we plan to undertake in the next cycle, as well as who would be working on it. This is great because engineers can pitch their own ideas and the building team is composed of only who's needed for the project.

Because we were trying this out for one team and not at an organisation level we'd already established that we were going to be working on the developer portal. We also knew that it would be our domain team who'd be working on it.

We agreed with product we were still making a bet (of engineering time) and we were committing to deliver the developer portal within the cycle. To achieve this there were agreements we had to make:

Time

The first (and most important) was uninterrupted time. We needed dedicated development time for focus and flow. We'd established six weeks of shaped work, if the team got pulled away to work on something else it wouldn't be fair. We needed momentum to progress.

Distractions

The second challenge we faced was handling bugs from our existing products, which are also owned by our team, during the cycle. As mentioned in the book, there is nothing which makes bugs automatically more important than the project we are working on. We'd make a note of the bug and bring it into the cool down period after the six-week cycle. This approach helped us maintain focus and momentum. In the case of a crisis, of course we'd drop everything and help - but crises should be rare.

In our team, we have certain responsibilities which require a small amount of time each day, but cannot be ignored due to their immediate impact on customers. However, we also wanted to avoid getting distracted from our main project. To address this, we decided to assign one engineer to be on "the bench" and handle these tasks. This approach allowed the rest of the team to focus on the project without any distractions. The engineer on the bench would rotate between cycles and usually coincide with time off or holidays. This ensured everyone got a fair chance to handle these responsibilities and reduced the burden on any one person. Any time not used on these tasks would be spent on the project.

Backlog

Like a lot of development teams, we had a centralised backlog of tasks. The problem with backlogs is that it can be challenging to keep them clean. We wanted to avoid the "put it in the backlog and we'll get to it one day" mentality. Instead we'd work off of decentralised lists. The developer portal would have its own project space which would contain our scope list. This is where we'd outline tasks and would be our source of truth for monitoring progress.

Building

After we'd established our bet and how we were going to work, it was time to start the build cycle.

Scopes

We weren't going to add tickets into a backlog so we needed to set up our initial list of what we thought we needed to do. For this process we used "scopes". The idea was to organise the project by structure and not by discipline or person. Each scope should be able to be completed independently. They should be bigger than tasks but smaller than the overall project. For instance, our developer portal might include "Landing page", "Registration flow", "Login flow" etc.

We started by defining high level scopes and then adding associated tasks as a to-do list. We don't class this as planning because we acknowledge we'll have imagined tasks at the beginning and discovered tasks as we build. The scope list and associated tasks will grow over time in the course of doing real work.

At the start of our developer portal build we had ten scopes and thirty-six tasks, by the end we had fifteen scopes and eighty-eight tasks. This is expected, throughout the six-week cycle the scopes got shuffled and tasks got added as we figured out what we needed to do.

We marked our nice-to-haves with a ~. This helped us separate what was a must have and what was not essential for shipping the product and delivering value. We regularly reflected on what we'd categorised as must-have and nice-to-have and weren't afraid to challenge the must-have tasks if newly discovered work had a greater contribution to solving the problem.

Hill charts

Now that we had our initial scope list, we needed a way to track progress. The challenge with using estimates and t-shirt sizes are that they don't always account for unknowns. For example, in our developer portal project, we needed to create a registration flow. While we had a solution in mind, the implementation would be completely new. Attaching a numerical estimate wouldn't be meaningful because we wouldn't be confident until we'd started building. In contrast, a scope such as building the navigation had no interdependencies, and we could reasonably predict what would be involved.

In Shape Up, we use "knowns" and "unknowns" instead of estimates. The focus is shifted from what's done and what's not done to what's unknown and what's solved.

To track our scopes, we use a hill. There's a great example about how they work here. In a nutshell, work is like a hill - there's the uphill phase of figuring out what to do and the downhill phase of execution. Using the example of the two above scopes, our hill chart would look like this:

Our registration flow contains a lot of unknowns, we're not confident we know exactly what we need to do; whereas the navigation scope is largely known. It sits at the top because we know what's involved but we haven't started executing it yet. Once we start developing, it will move down the hill and once it's shipped to production, it will sit at the bottom of the hill because the scope is finished.

This was how are hill chart looked at start of our build cycle:

Building the developer portal

Now that we'd established an initial list of scopes and mapped them to our hill chart. We prioritised getting the unknown scopes to the top of the hill to de-risk them. We also wanted to get one piece done, end-to-end, to help us establish momentum and flow. By the end of the first week, our scope list appeared as follows:

Having gained enough understanding of what was involved in the unknown scopes, it was just a matter of execution. We discovered that one of the tasks from the authentication flow had a lot of work and unknowns involved in it. As it wasn't critical for the initial phase of the project, we decided to split it into a nice-to-have scope and deprioritise it. The problem we were trying to solve could be solved without that scope. We placed it at the bottom of the hill, and if we had the opportunity to revisit it later in the project cycle, we would do so.

Progress

Instead of involving the product in daily tasks, we scheduled a weekly product check-in. During this meeting, we shared demos, raised questions, went over the hill-chart and discussed any scope changes or new tasks we wanted to prioritise as "nice-to-haves". Rather than relying on subjective "how's the project going?" we were able to show real progress and status every week. Here's an overview of how the project progressed from the throughout the cycle:

The result

We'd been continuously shipping to production throughout the cycle and by the end of the final week, we'd finished the project. We'd left a nice-to-have scope and some nice-to-have tasks behind but here's how it looked end-to-end:

How it worked for us

This was our team’s first attempt at working with the Shape Up method. We held a retrospective at the end of the cycle and the feedback was overwhelmingly positive from both the building team and product. We’d successfully shipped a brand new project and delivered significant customer value within a six week period.

Giving the team autonomy over prioritisation of scopes and tasks was very effective. We were able to make trade-offs and come up with different solutions to match the product appetite. We didn’t get stuck on tasks as a result. In our previous way of working - if a ticket was stuck in a swim lane, we’d make sure it moved through the board no matter the effort. This meant we could easily labour over tickets just because we felt like we “had to get them over the line”. In Shape Up, we were focussed on solving the problem and as a team we did a great job at looking at the bigger picture and prioritising what was important to solve it.

Day to day, we had significantly less meetings and more focus time. This helped us build momentum which was essential to shipping on time. We still held pairing and mobbing sessions to solve problems and left with a greater sense of collaboration between different disciplines. One of our engineers noted that he felt more relaxed and less rushed which was a result of focussing on the bigger problem, rather than granular tickets on a board.

Shape Up is undoubtedly a mindset shift from, “how long will it take us to solve the problem?” to “how do we solve the problem in the time that it’s worth?”. This leads to a healthier product and a more engaged development team. We’re now able to collect customer feedback, shape it and iterate.

This isn't to be an exhaustive dive into how Shape Up works, for that I highly recommend that you read the book. It provides more detail than I was able to in this article and there are some ideas I’ve only been able to summarise here. However, I hope that covering some of the concepts in the context of a real project has been useful.