Managing software project teams through transition: Roles & Responsibilities

In this post I talk about a recent engagement with a client around settling on a framework to understand the roles & responsibilities in the development organisation. It was an activity that lasted the tune of just over 12 months to reach a point where everyone impacted could agree with the outcomes. It was really quite a journey with many individuals across the board from senior managers to engineers. The end result was a framework, based on the RACI model, that not only showed the RACI matrix, but also output high-level job descriptions for each key role, that acted as handy input to job-specs that could be tied in with people's objectives, in terms of their personal development plan for performance appraisals management.

The context that triggers this activity is in my view, quite common for any young entity that has just started on the path to software product development (the project team effectively shapes up the future direction & structure for the organisation) - i.e. their first real stab at doing in-house product development. Usually what happens is the project grows & ramps-up as required (building various teams required to get the delivery done), the project, in effect shapes up the future structure of the organisation. I've been in this story a few times already in my career: Work on a next generation concept, build a team, deliver, then re-shape the organisation to focus on this product as the mainstream focus, transforming the project structure into an effective organisational functional model...

In this particular scenario, the situation was around a new unit that was setup to deliver their first major product deployment (Set Top Box (STB) software - completely developed locally in-house), and as part of the experience, the team operated with a fair amount of autonomy for several months (experimenting with agile/scrum) finding their feet, until the business ran out of patience sensing that if things continued as is, the promise made to the business wasn't going to materialise (delayed), and so thus intervened by "management directive", almost pulling the rug from right underneath the team. The project switched focus to a highly-driven delivery mindset, where the team freedom was curtailed to a point that all technical decisions were made through a single entity, called the "Launch Director", who's only focus is to do-what-it-takes to get the product out-to-market, running with complete freedom to cut-and-chop processes (in the guise of efficiency improvements) to get the job done.

In retrospect, this was the right call made by senior leadership - a gap was identified around the lack of a key resource accountable for aligning the technical delivery, a strong driving force was required to reign in the stray streams to get them all pulling in the right direction. Going forward however, we kept the role under "Delivery Owner" to be considered for large-scale new project initiatives.

So the team came eventually out in the end, a little scarred, battle-hardened, with the product delivered to market as an extremely huge success, which would not have been possible had the launch director intervention not been implemented.

Our challenge was how to move forward, with the launch director moving on, leaving this product development team to put the pieces together and continue...

How does the team pick up where it left off? Before the switch to delivery mode, the teams were just entering a rhythm, they knew who the players were, the key project managers, product owners, technical leads, etc. Once the launch director kicked-in, most of those roles faded into the background, people overshadowed, disempowered, meant to just follow the lead, taking direction from management. Coping with this change upset the trajectory the teams were hoping to achieve with their agile/scrum intentions (whilst those sensitive to agile principles can indeed sympathise with the team, it has been my experience that even with best intentions of managers adopting agile teams, there comes a time when a delivery mindset, driven by senior stakeholders, takes priority, delivery must happen!).

So how do you fill the void, dismantle and regroup, redistribute the roles & responsibilities to a point that can sustain the teams going forward in keeping with this mindset of agile/scrum values, and ensure that when the next major product release needs to go-to-market, that they don't fall into the same operational, escalated emergency mode of working: i.e. crisis-delivery mode?? How do you prevent the house of cards from toppling down?

There's no easy answer really, apart from having loads of conversations, being patient with the rebuilding process, a lot of co-ordinating and re-enforcing repeated messaging around respecting boundaries, etc, etc. A lot of feedback loops, discussions do help, but people expect more - eventually one has to reach a point of some certainty, by taking time out to flesh out in writing, the roles / responsibilities / expectations from all.

Suffice to say that this is really a journey through transition - I've taken this particular organisation to a point that enough of a framework is in place to provide the necessary guard rails...

Note: As a consultant I share my work experiences through my writings, whilst respecting my clients, I take time to present generic descriptions that has applicability to general software management experiences. I am by no means passing a value judgement on my client, rather sharing the outcome of the experience that could be helpful to people faced with similar challenges...

The point of this post is to share the learnings of this experience, and not undermine the specifics of the project, this isn't a rant post at all. As a matter-of-fact, the result of establishing this RACI matrix and unpacking the team's roles and responsibilities, has led to the team achieving a major milestone release, without needing the previous management intervention: Between the product, development, test, integration streams, and empowerment of the Release Manager role - this team has come together quite nicely... I've actually been asked to help facilitate this RACI process with other departments within the business unit.

P.S. If you're new to this site, I write about Set Top Box software development topics. Please search through my blog for background reading on the nature & context of digital TV software projects...

Let's start at the beginning...
So we just came off the product's launch to market, people needed to move on, we just couldn't hit the reset button, because the train had left the station. The launch was just the beginning, so whilst in this transition period, we had to still push along with delivery.
The picture at the start looked like this:

What we started with

As you can see, there were essentially two champions in the spotlight. We needed to move to a view that looked something like this:

Desired split (Empower the team!)

So we had a starting point to go forward, what wasn't clear is the delineation of who owns what, so we entered an interim stage, with this picture (the red X indicated who owned the activity):

Third attempt - showing ownership splits

We were definitely headed in the right direction. What was clear was that the matrix was clear enough for the key people involved in delivery activities, but it didn't solve the problem of showing how the wider, cross-functional team participated. It almost sends out the wrong message that the only people involved are the ones with the X-markers. So I decided to introduce the RACI model to the team. This was the version that we approved for mass communication:

RACI matrix showing whole team involvement

Now how did we get there? Time for story telling...
It starts at the top, trying to rationalise a framework that encapsulates as many work-streams as possible (Product Management, Architecture, Development, Integration & Delivery, Project & Program Management), with the challenge of finding an acceptable mix of traditional software development & newer agile philosophies.

In terms of business strategy, the picture takes the following form:

Software Product Development Strategy

This picture is very similar to the one in a previous post where I described the generic product management roadmap model and how it fits in with achieving continuous deliveries to market. Effective product management cannot happen in isolation of systems engineering & architecture, the two go hand-in-hand. The pictures shows some parallel streams of work happening within systems engineering & product management. The short term view is the one where the level of detail and information on the product is the greatest, such that a release or two can be planned. This is usually within the world of software product managers / owners, and the software delivery teams (development, integration & test). The medium term view is less clearer, and is part of the overall product strategic team. The longer term view is the one that's most unclear, and sits in the realm of business strategy. Systems engineering views operate in much the same way. Short-term view, where the information & requirements are fairly detailed, is in the world of a pre-production team. This team is responsible for validating third party components as fit-for-purpose to support the development team (in production mode), working ahead of them to design & qualify APIs, etc. Longer term view has the systems engineering team working on solutions design & possibly future innovations that maybe disruptive to the existing product management model in place.

Product management relies on the technical delivery stream to implement the product features thus delivering to market. Release managers are assigned to work on deliveries. Remember, this is for a team in transition from traditional practices, shaping the path to a more agile-based mindset. We couldn't afford to take the leap into one big team, cross-functional, the team owns everything model - the organisation was very much divided into functional silos, and not ready to change anytime soon, however, the management team are quite ambitious, no one can fault them for not having agility as a principle. With this agility in mind, our goal is to accomplish rapid releases to market, even without having a fully cross-functional team in place:

Template for Release Planning

Whilst this picture does look like waterfall, with staged sequences of first getting the product team to complete scoping the feature, moving on to pre-production team to validate component interfaces, then onwards to the release delivery teams - there is some overlap and parallel work intended such that two releases are being worked on around the same timelines. This is the agility mindset the team would like to achieve. We have to date, somewhat managed this in practice, although with a little difficulty (a lot of the the quality checkpoints are manually-intensive & we are not where we'd like to be with software quality best practices).

I tried to encapsulate the entire product, project, and release philosophy in one picture, touching upon some agile concepts as product backlogs, sprints & releases to market:

Unified team view of release philosophy

The picture is colour coded representing the realm of each work-stream. Release managers are the pivot that binds everything together, managing an integrated delivery plan from all vendors. To do this, release managers are highly dependent to product management for fleshing out the backlogs, providing priorities, etc. Architecture team are always involved across the board to ensure the technical integrity of the product's software architecture is maintained, not only from the STB stack, but also from an end-to-end enterprise level. There are usually four core vendors to a STB software stack: The Application / EPG sitting on top of the device Middleware / Operating system, which sits on top of Device Drivers / STB Manufacturer. Bringing all these pieces together is the job of Systems Integration (SI). Release Managers and SI are one-and-the-same essentially, working closely with the product managers. Vendors shown in the blue blocks are free to adopt a scrum / agile framework. We've shown the Application team as having a backlog for a particular release, that is broken down into n iterations or sprints to deliver Release 2 to market. The approach is to release to market after multiple sprints, fixed-date market releases are generally the status quo for PayTV operators.

Getting the teams to understand the full context of the timeline horizon, I created another visual that showed the relationships & overlapping parallel work-streams:

Zooming into timelines

Next the team needed to appreciate another context around the various types of planning activities that exists, the boundaries within which they should operate.

More pictures and story telling:

Worlds of planning context

A handy table to show what's expected from various planning levels

Every level will have its own unique approach to planning, which is left up to the owners, we avoid being too prescriptive - apart from establishing the high level framework within which teams should operate.

With the core roles outlined, we make sense of the systems engineering disciplines, focusing on the different levels of architecture:

Architects Roles

Digital TV projects often are not restricted to just the world of the set top box, they're increasingly becoming more complicated, moving away from the traditional broadcast engineering systems, to connected services over the internet, video on demand (VOD) and other cloud-based services. As such, when putting a solution together for an end-to-end feature, the architecture & systems engineering teams also need a framework to work against, respecting the boundaries & interfaces, settling on roles, responsibilities, inputs, outputs and deliverables. Click here to have an overview of the various architecture roles for DTV project maps.

Kicking off new large-scale projects with clearly defined high level roles
In the course of this journey, we established the following high level roles as a baseline:

• Delivery Owner (a.k.a Launch Director)
• A Delivery Owner is ultimately accountable for the product delivery. Manages the overall delivery of the product to market according to business expectations, whilst maintaining the technical & architectural solutions are steered in the direction to make this possible. Reports the overall project health to the exec (Steercom), with support from the programme manager. Manages expectations upwards and downwards, including escalations for decision making where decisions fall outside the scope of influence. Otherwise largely empowered to make decisions across all work-streams, whilst respecting individual roles of accountability (does not micro-manage or overstep the boundaries unless forced to do so).

• Work Package Owner (WPO)
• A Work Package Owner (WPO) is ultimately responsible and accountable for delivering on the requirements of the Work Package. Working with an assigned Project Manager (PM), the WPO commits to a delivery plan, communicates all project dependencies to the PM, and uses the PM to escalate risks & issues that will impact the overall delivery of the Programme. This also includes responsibility for resource management, budgeting & quality management.

• Project Manager (PM)
• A Project Manager (PM) assigned to a Work Package effectively provides a service to the WPO, in facilitating & co-ordinating the various activities required to deliver the work package. Working with the WPO, the PM is involved with initation, planning, implementation, controlling & tracking the work package to final delivery. The PM tracks progress of the WP, ensuring the overall Programme Plan is maintained. The PM also manages internally risks associated with the WP, and escalates risks & issues upwards to the Programme Manager as appropriate. The PM leans on the WPO to execute and deliver the WP according to plan. The PM co-ordinates cross WP dependencies, collaborates with other PMs as necessary, as well as interfacing with third-party suppliers and other business units.

• Programme Manager (Prg. Mgr)
• Programme Manager works at the high level, focused on producing a high-level plan (initially) representing the various Work Package streams (a.k.a. workstreams). Working though Work Package PMs, the Programme Manager does not get involved in the detailed planning and tracking activities, but ensures the Work Packages deliver according to plan, maintains a view of the overall Risks & Issues. The Programme Manager manages the timeline and takes responsibility for the overall delivery schedule, working through the Steering Committee & Senior Management Stakeholders. Drives the PMs, provides clarity of overall priorities and business objectives. Involved in decision-making as long as the overall Programme Plan is not compromised, in which case, escalation to Steercom required.

• Quality Assurance Program Manager (QA Prg. Mgr)
• QA Program Manager works at the high level, co-ordinating and driving through the various QA/Test workstreams in producing a consolidated QA plan & test strategy for the Project. The QA Program Manager reports to the overall Programme Manager on the QA activities, and will work through the QA Tech Leads on the overall execution of the test strategy. 

• Steering Committee (Steercom)
• The Steering Committee (Steercom) is a group of key Stakeholders from various parts of the business, who have a primary interest in delivering the project. Responsible for steering the project through various phases, and involved in decision making regarding priority conflicts, business decisions and risks/issues impacting the success of the project. Involved at a very high level, usually kept informed of general project progress through status reports, and often get involved in points of escalation, as required.

• Project Management Office (PMO)
• A Project Management Office (PMO) is a group consisting of Programme & Project Managers, operating as a collective unit, on behalf of a large business unit that have a number of projects & programmes running simultaneously, with competing priorities. The PMO thus facilitates and co-ordinates the various activities required from projects, ensuring overall alignment within the business unit. Awareness of business priorities, the PMO offers guidance in terms of timelines and expectations of delivery. Operating at a high level, with little involvement in driving or managing the detail.

Mapping these roles out has helped in aligning everyone at the start of the project. We use this high-level template for mapping out work package descriptions:

Template for Work Package Cards

Other artefacts - high level role descriptions
A useful side effect from the conversations that resulted in the above framework for roles and responsibilities, resulting in a fair amount of detail that folded nicely as task-level activities for for job specifications:

STB Software Release Manager Job Spec

• Key Delivery Project Manager, owns & drives the Overall Release Plan
• Receives Release Schedule / Backlog from STB Product Manager
• All user stories will have already been prioritised by Product Manager as part of Pre-Release Scoping/Analysis
• Release Manager focuses on closing all remaining open actions, taking over from Product Manager
• Product Manager focuses on scoping the next release (unpacks the roadmap, continuous activity)
• Release Manager drives development, integration & delivery planning/tracking of current release
• Owns the Release Plan (Development, Integration & Delivery)
• Works with SI to prioritize and classify severities (not priorities, priorities come from STB Product Manager)
• Drives defect prioritization process (Infield, Field Trials, xxQA, ATP, SI)
• Drives SI Backlog / Triage volumes
• Communicates status of various aspects of the Release
• Receives instruction from Programme Manager on overall Strategy
• Communicates STB-dependencies to Programme Manager for other workstream for co-ordination
• Drives Vendor Planning & Issue Tracking (Works with vendor Project Managers)
• Co-ordinates all activities with impacted vendors
• Manages the STB-related change request planning & deliveries (Change Request must’ve been approved, and through CR process)
• Owns driving of all Technical issues to closure
• Determines which builds leading up to a Release Candidate are acceptable for downstream SIQA activities
• Manages Feature Stabilisation / Regression tracking with relevant reporting / metrics to STB Product Manager
• Communicates work priorities as agreed by STB Product Manager and published Release Plan
• Acutely-aware of expectations of Release Launch Criteria
• Escalates to STB Product Manager if Release Milestones at risk
• Regularly reviews progress against agreed / committed Defects Backlog with Vendors
• Follows and adheres to General Defect Management Processes
• Works with SI & QA owners to determine strategy
• Collaborates with Manager of Field Trial Strategy / Planning for field trials acceptance of Release Candidates
• Gets product to Release Readiness for Go/No-Go Launch meetings

STB Software Product Manager Job Spec

• Translates the Product Roadmap into sensible Release/GoToMarket milestones (up to 6 months visibility) 
• (Programme Manager Co-ordinates End-to-End dependencies & sequences integration)
• Translates high level business desirables into concrete higher level product use cases / stories (feature backlog)
• Deriving finer-level of detail (user stories per high level use case)
• Pre-Release Scoping & Analysis activities (Unpack Scope to produce Detailed Release/Feature Backlog)
• Hands over Release/Feature Backlog to Release Manager for detailed Development/Integration/Delivery Planning
• Interface to manage expectations from cross business product owners 
• Gives direction where needed on Feature Priorities to Application Product Manager / Release Managers & Vendor Product Owners
• Approves Feature De-scope / Concessions for Major Launches
• Accountable for agreeing on de-scoping features / user stories (including concessions) that pose risk to slipping launch
• Accountable for overall delivery of Roadmap features to Market (current release, next release up to 6 months visibility)
• Not involved with detailed timelines, this is the Release Manager’s job – but provides guidance in terms of milestones and feature priorities
• Manages expectations upwards (with the help of Program & Release Managers)
• Responsible for delivery of High-Level Product/Feature Specifications 
• Responsible for delivering the roadmap as set by the senior management for products
• Main interface to business / senior product managers
• Support In-Field Customer Experience, Call Centre Complaints – feeding back into release backlog
• Owns the Product Release Backlog (New Features AND Improving Stability of existing Features)
• Hands over Release Backlog to Release Manager for Development / Integration / Delivery planning
• Supports & Guides Release Manager in fleshing out Release Timelines
• Defines overall Feature & Defect Priorities as vetted by Business 
• Ensures key people assigned to activities required to flesh out feature use cases (Architects, BAs, POC teams, etc.)
• Responsible for guiding teams on features & defects, to the extent of highlighting the test scenarios for QA teams to consider
• Supports Release Manager in Decision-Making requests
• Primary Point of escalation for Release Manager
• Support the Marketing/Corporate team in GTM activities
• Makes the call on the need for Customer OTA Upgrades / Patches to resolve critical infield issues
• Post-Release Infield Customer Support & Prioritisation
• Provides direction on all Go To Market activities
• Determines which Release Candidates (not weekly builds) are acceptable for downstream QA/Field trial activities
• Based on Release Managers reporting of status of release
• Accountable but Not Critical in Managing of Detailed Delivery Planning Activities
• Depends on Release Manager to manage Release Planning / Execution / Strategy
• Depends on Technical Owners / Architects for resolving technical challenges
• Depends on Programme Manager to take care of cross-workstream dependencies E2E
• Depends on UI Product Manager to manage the UI/UX development backlog
• Own Post-Release Activities
• Ensure information on new features / upgrades / product behaviour / spec changes are communicated to wider business units
• Customer Operations / CSR
• Installers / Field Services
• Training / Marketing / Corporate communications material
• Web Sites, FAQs, Forums, etc.
• Post-Release Infield Customer Support & Prioritisation
• Feedback in-field critical issues to Release Manager for including in next release

STB Software UI/UX Application Product Manager Job Spec

• On-the-ground, day-to-day walking-the-floor, supporting Development Team, BAs, SI, Release Managers
• Performs the roles / tasks required (day-to-day stuff):
• Ultimately accountable for overall UI/UX delivery according to Release Manager’s planning
• Responsible for User Experience (UX) & UI Product Specifications
• Provides guidance to teams on how to realise user stories
• Acts as the interface between Feature Owners resolving questions/issues with feature behaviour, spec, etc.
• Does not get involved with nitty-gritty technical details (architects job) – component architecture resides with component development team 
• Contributes to the UI Release Planning requirements from Release Manager in fleshing out Release Timelines
• Engages with Release & Product Manager on defect priorities w.r.t. BoM deliveries – decisions that are required during the actual sprint activity
• Responsible for all UX/UI Guidance
• Manages the priorities & backlog for decoder analysts / graphics designers to align to the UX/UI planning / release timelines
• Provide clarity of definition-of-done, acceptance criteria, etc. in the context of local UI development team, not Release Plan
• Assist, facilitate & co-ordinate activities required for Pre-Release Planning, supporting the Product Manager
• Works with SI/QA to understand existing Features Stability for possible Reworking / Regression Testing
• Communicates status of UI Component Stability to STB Product & Release Managers

E2E Program Manager Guidelines

• Overall Co-ordination & Guidance of all Work-streams & Planning Direction (Product, Pre-Production, Development, Integration, Delivery, Infield)  Involved
• In lieu of Headend/Backend Release Manager Role, performs the role of Release Manager for Headend/Backend Deliveries, with support from E2E SI & All Business Product Managers
• Closely linked to Product Management team
• Product Management guidance for planning
• Facilitates Backlog Creation
• Requirements Management
• Release Backlog
• End-to-End Architecture tracking
• STB workstream planning guidance, oversight, direction & tracking
• Headend & Backend workstream planning direction & tracking
• Managing vendors – high level planning, direction & relationships
• Managing & Planning E2E Integration points (across the business / enterprise)
• High Level Milestone planning & tracking
• Upwards communication to Exec (at SteerCo for progress against Timelines, Risks, etc.)
• Managing cross-business units interaction
• Strategic consultations on planning / release strategy
• Future projects planning & co-ordination
• Process improvements to deliver efficiencies to help improve overall delivery timelines
• Coaching, Guiding, Mentoring Project/Release/Product Managers/Architects
• Defining Project Charters including Organisational Structures
• High-Level Risk & Issue Management