Blog Archives

Shadow Backlog


The team was under pressure to deliver a rewrite of the flagship product after having been handed an agressive delivery date. In spite of the Scrum board and other information radiators in their area, the Scrum Master had to deliver progress reports to the CEO every day. The company president and vice presidents attended sprint reviews and grilled the team whenever a story was not closed or sprint goal not met.

One could easily argue that a fixed scope and fixed delivery date is in no way agile – end of discussion – but this still happens in organizations that use (at least partially) agile practices leaving Scrum Masters to navigate choppy waters.

Of course the team developed some coping mechanisms, the most insidious of which was kicking the can down the road.

Whenever the team encountered unexpected complexity or scenarios when working on a story, they said “this wasn’t in the story that we planned, so let’s call the current story done so that our sprint metrics will look good and handle the extra stuff sometime later.” Everyone except for the Scrum Master was happy – for awhile. Team velocities looked good. Stories were being closed as planned. On the surface it looked like features were being completed. The teams were going to deliver on schedule. The Scrum Master knew deep down that they were either going to deliver late or deliver an application with enormous amounts of technical debt. She also knew that the organization could not handle to truth so she soldiered on.

Then the bubble burst. The collective “oh crap” moment happened when people realized that the system should be production-ready but was not. Some features were incomplete. Bugs were still there from parallel testing that had been performed months ago. There was no room in the time before the expected completion date to resolve all of the open issues. Market launch would be delayed.

The ‘shadow backlog’ existed in the developers’ minds and in the list of open defects that were logged elsewhere from ongoing user testing. There were no product backlog items for them and no sprints in the release plan for addressing them.

– Make sure that all remaining work is visible. Create backlog items for stories, technical debt, and bugs. The backlog consists of Product Backlog Items (PBIs), not just user stories.
– Get user feedback during the sprint to avoid surprises later. Ideally users are integrated into the process via acceptance test driven development (ATDD). At a minimum they have an opportunity to see stories in action and provide feedback in the sprint. Any feedback and new ideas that cannot be incorporated during the sprint should be captured as real backlog items and sized/prioritized.
– Don’t be afraid to keep a story open if it was more difficult or took longer than expected when sprint planning was originally done. Better to take a little heat now rather than have the bubble burst later.
– Avoid ‘bucket stories’ near the end of a release that become a dumping ground for stuff that did not get completed in the sprints. Bucket stories can quickly grow exponentially and delay the release.

The Heartbeat of an Agile Team

What is the one focal point that catches every team member’s attention at the same time every day?  Of course it is their Scrum board. Some teams go through the motions and use their Scrum board because they are told to; for effective teams the board is a way to collaborate, manage work in process, keep track of impediments and know whether or not they are going to meet the sprint goal. Effective teams turn their board into a highly-visible collection of critical information about their effort. It becomes the heartbeat of the team and helps set the cadence for each sprint.

Their board might look something like this:



Sprint Dates and Goal – as a reminder of the goal and the timeline. Seeing the sprint dates can be helpful if users who will be testing attend the standups so that they understand the sprint timing.

Columns for each step in their development process – This can be as simple or as complex as the team wants – it usually evolves as the team learns and tweaks its process to ensure a smooth flow of stories. In the example above, the team experienced issues with too many stories waiting for user testing. They created a column to show which stories are ready for user testing to make it clear to users when something is ready for them to test. During standups, the Scrum Master points out any columns where an excessive number of stories are piling up. Even with Scrum, the team may set limits for how many stories may accumulate in each column before team members swarm to address a bottleneck.

Sprint Calendar – during sprint planning the team members estimate on which days of the sprint they think stories will be Done and also when they will likely reach any other significant milestones in the team’s process such as being ready for user testing. This is a way of performing a sanity check for the sprint plan.  It is also a way for the team to gauge whether or not it is on track during the sprint. When user testing is part of the Definition of Done, it allows the users to plan when they will be performing this testing. Putting team members’ planned vacation days on this calendar avoids surprises in the middle of the sprint. It can also help to put key SME’s or users’ vacation days on the calendar if these could impact the sprint.

Impediments – unresolved impediments. Making these visible is a way to track them and may have the side benefit of prompting outside stakeholders who attend the standup or visit with the team to help the Scrum Master by finding other ways to resolve them.

Additional Stories – these are stories at the top of the Product Backlog that are ready to be brought into a sprint but that are not part of the current Sprint Backlog. These are posted on the board so that they are readily available in case the team reaches the sprint goal early and can complete some additional stories in this sprint. Note that these are not called “Stretch Goals”. That term can be perceived negatively by the team, almost as if they are not pushing hard enough in the first place to meet the sprint goal.

Release Burnup – the bigger picture of what the team is working towards. This help avoid the sense of being in an endless churn of sprints with no end in sight.

For teams using Kanban, many elements on this board still apply but may need slight modification. Note the use of colors throughout the board.  It is amazing how teams will adopt color schemes or symbols that communicate the state of their work, highlight important events, serve as key reminders, etc. One type of information not included on the example above is an area for Technical Debt.  Creating an open area on the board for this invites the team members to note it as soon as it becomes apparent to them.  These notes can then be translated into backlog items during grooming. This encourages open discussion about technical debt and surfaces this information rather than just having it in the team members’ heads.

A note for teams that use online/electronic Scrum or Kanban boards: additional information such as the PTO calendar, Impediments and release burnup can be captured in story cards and placed in a column dedicated to that background information. The sprint calendar can be captured by putting Due Dates on each electronic story card. On a physical or electronic board, the story cards can contain as much information as the team finds helpful. Typically the story number, title, and number of story points are included.


  • The Scrum or Kanban board can be used to communicate much more than just the status of the stories and tasks.
  • The information on this board should reflect each team’s personality and process. It will evolve as the team learns.
  • The Scrum Master can help the team during the standup by directing their attention to potential trouble spots on the board.

Bring the Work to the Teams

WorkToTeamAs new high-priority features surface on a large program it can be tempting to give in to demands from stakeholders to assemble a team of the best people from the existing Scrum teams to build those features. Typically someone will suggest plucking the best people from the various Scrum teams and forming a task force. If anyone tries to talk you into doing this, don’t let them. In agile, restructuring the teams should be done sparingly and carefully. It takes a team of people several sprints (and sometimes months) to learn to work together effectively. Reorganized teams also represent change and the range of human factors that come with change. Large scale agile is about bringing the work to the teams, not assembling groups of people for each package of work that needs to be completed.

Another consideration when someone suggests putting together a task force is that it needs a Product Owner, Scrum Master and the various cross-functional roles such as BAs, QA, etc. to be covered. Gutting existing teams to staff these roles leaves the other teams in disarray. And, of course, we would never even consider sharing people across teams, would we?

The next time someone proposes a task force, take a look at how this work can be done by the teams already in place. Chances are pretty good that the work will be completed faster and better by the established teams.

Evolving Team Structures Through the Product Development Lifecycle

One of the most fascinating management challenges is determining the best structure for Scrum teams based on the stage of the project. This does not mean the major waterfall phases like Analysis, Design, Construction, etc. When developing software for use by customers (especially niche SAAS products) and there is some level of customization or specific features for each client, there are at least a couple of distinct stages in the project:

  1. Develop the base product
  2. Develop client-specific features and convert those clients onto the system as those features are completed

Many agile writings suggest that the way to approach system development is to develop some features and put them in production right away so that clients can start using them immediately. In reality, this does not always work. For example, when an existing system is being replaced, clients cannot be put onto a new system that only has 20% of the functionality of the one being replaced. Of course, this does not preclude some form of testing of that 20% by users within the development organization (e.g. by internal client service reps who are also part of the user base).

Organizing for Base Product Development

Typically the base product development effort focuses on developing the major feature areas. The development teams can be organized along the lines of the features areas as depicted below.

Base Product Team

Organizing for Client Conversions and Onboarding

 As the base product nears completion and the teams start to work on client-specific features, the portfolio or program manager may start noticing that the client code complete date for one of the feature teams above is much later than the other teams because the bulk of the customization for that client falls on that one team. The net result is client on-boarding dates are further out in the future that delay realization of new revenue from the product. This is a clear sign that the team structures need to change. At this stage, the focus shifts to optimizing staff utilization while retaining the benefits of teams building features or feature slices from top to bottom. This is called Focused Balance. Work is focused as much as possible on one or a few teams to gain the efficiencies that come with understanding and ownership of the epic , but when necessary client-specific epics are spread across the teams to deliver them as soon as possible.

Client Onboarding Team

Restructuring the teams

When scheduling and work allocation issues require that multiple teams work on the same epic, the following should be considered when deciding who should be on each team:

  • Knowledge about specific feature areas of the system.
  • Technical skillsets. Rebalancing may be necessary when certain feature areas from the base development phase emphasized a particular technical skillset.
  • Technical leadership – most teams need a technical leader.

Cross-Training and Collaboration

Team members will need to step out of their comfort zones and learn other parts of the system. Product owners, who have become experts in the feature areas that their teams constructed in the initial stages of development, will be overseeing a backlog that contains features and stories that are outside of their current area of expertise. Recognizing and fostering the need for collaboration across the teams is critical to the success of this model. At this stage, the job of the product owner is to connect team members with subject matter experts rather than being the subject matter experts.


  • Team structure may need to change to optimize the schedule for client-specific feature development.
  • Restructuring and rebalancing evenly distributes system knowledge and technical skills across the teams.
  • The Product Owner’s role changes to be less of a subject matter expert for their team and more of a facilitator.