Agile Project Management with Scrum by Ken Schwaber

Agile Project Management with Scrum (Microsoft Professional) is a well written, easy flowing book that is clearly borne from someone whose had first-hand, real-world experiences of running and managing a variety of Software Projects over a number of years, Ken Schwaber, who is considered the father of Scrum who humbly takes no responsibility for being assigned that title, and so points to much earlier endeavours of many people, pointing to Babatunde's Process Dynamics, Modeling & Control for his first oral presentation of the theory behind Scrum; and sites Degrace's Wicked Problems, Righteous Solutions as the first people to call Scrum Scrum.

Regardless of who is attributed to formalising the Scrum processes, it is a software development methodology that is here to stay, and anyone working in software projects not familiar with Scrum should indeed rethink their strategy. Still, with so many books out there claiming to distil the secrets of Agile/Scrum, finding the right book is indeed challenging -- as with so many books, people focus on the theory without expounding on the practice, the real-world experiences and encounters that as a manager, and especially as someone transitioning from classic project management principles (not necessarily connected to the Watefall process of software development).

This book is definitely different, one that I recommend and fully endorse 100% - so if you're reading this and don't have a copy, get one now!

Why do I like this book so much?

Well, as a developer I'd worked on many projects of different styles, more recently explored with Agile/XP concepts. I've been on training courses, presentations & and participated in practical workshops such as the agile lego game. I've contributed as a developer, lead certain development efforts with a few engineers, then later went on to managing development projects using Waterfall as well as Agile; and more recently came off a massive project with 300+ people, where at the macro-level applied Agile/Scrum principles across the programme, but I didn't have much involvement in the day-to-day planning with individual component teams, where we'd assigned team leaders to act as Scrum Masters. We maintained one huge Product Backlog, not so different to that as Ken describes in Chapter 9 Scaling Projects Using Scrum.  We were developing a product that would deliver to multiple customers, each customer adding unique features contributing to the final feature set, the product itself would service tens of millions of homes -- so we had a strong Product Management group maintaining the heartbeat of the multiple projects using Sprint Time-boxing. We had macro daily planning and status updates, call it Scrum of Scrums.  So I was pretty much focussed on the high level programme and product management, than the low-level activities that a Scrum Master would normally be dealing with, I'd instruct as necessary....

As I was moving to a new job that placed me in a Scrum Master role, I needed to brush up on my Scrum Master knowledge and needed to prepare or translate my heavy Project Management experiences to Scrum principles. This book was certainly written with that purpose in mind. I was looking for real world use cases and reflections of actual projects, something that Ken describes well.

Ken touches on the key concepts of Scrum by using example projects as references, with frank and honest feedback. He includes much of the areas that would trip someone up coming in cold to Agile, and also does not dismiss the value and usefulness of applying the rigour of classic project management dogma (reporting, tracking, metrics, predictions) as old-school, out-with-the-old, in-with-the-new.  In fact, Ken advocates entirely the opposite of that. It certainly takes a while to master Scrum as a Scrum Master, and therefore takes much longer for a company with seasoned managers who know no better, to transition and accept Scrum from day one.  So as important stakeholders in the project and business, these people have a right to be given information in a format that is understandable, and this too, can still be achieved using Scrum.

What really resonated with me was...

Ken has re-affirmed much of my own understanding in that Agile/Scrum is no excuse to cut corners. Scrum is not a short-cut from applying rigour and due diligence.  Way too often, people who are either new to Scrum or have been previously burned by management, fall in love with the concepts of Scrum, especially the tenets of autonomy, freedom and do-what-it-takes attitude to get the job done, immediately put up barriers when someone starts talking about processes...

• "Oh that's waterfall approach, we are young developers, we don't do it like that anymore. We are light on process, and don't need documentation. We don't need to have a process for teaching new engineers, it's a collective, the engineer will be absorbed into the team and learn on the job. We don't have release notes any more, it's automated by Git. We don't need a version numbering system, Git labels are fine. We don't do need pre-planning, we do just enough because it's going to change anyway...."

Principles I connected with...

• The importance of taking time out to produce a Product Backlog early in the project. The argument of not knowing what you're creating because it's unknown is not good enough. Even if you're aiming to brainstorm to produce an initial Proof-of-Concept (POC), that POC is driven by meeting the high level needs of the Product Owner, so it is not an impossible activity to put thoughts onto paper, call it your wishlist, to-do list, whatever - there must be a Product Backlog to start with...how else do you determine the goals, and measure your progress accordingly??  So if you think you're doing Scrum and don't have a Product Backlog, then please go back and reconsider what you're doing...

• Ensuring the Roles/Responsibilities are clearly identified - especially the ownership of the Product Backlog - Is your project big enough to have its own Product Owner, or is this also something the Scrum Master can absorb?? The Product Owner's role is pivotal to setting feature priorities only, but not responsible for driving through scheduling or people management. That is the Scrum Master's job...So spend time to clearly outline the roles and responsibilities...

• Spend enough time Planning - the planning process described by Ken is a useful starting point. Spending a full day on planning and getting the team to uncover the tasks before the Sprint begins is definitely valuable.

• Due diligence must be followed by Scrum Master - measuring progress and reporting on progress, generating metrics and predicting the future are essential aspects to Scrum Management.  Don't be fooled by Scrum being lighter than classic project management. Scrum teams still have a responsibility to meet business objectives, how you go about doing it is the Scrum teams own business, but when asked with business-type questions, then the Scrum team better have enough data to provide professional responses

• Don't misuse Waterfall defence mechanisms - who says you don't do requirements/design/testing - Scrum says you ensure enough is done during the sprint such that at the end of the Sprint you produce a release that is shippable - so a Sprint must support requirements/design/testing/validation/release - same steps as waterfall, but it's constrained to happen within the sprint itself. Of course, one doesn't have to have heavy processes, but it is about applying core engineering principles.

• Self-organising teams are difficult to create, and takes time to master.  As a Scrum Master you have to continuously monitor the team's performance and interactions.  The hard truth is that in the real world, using distributed development or even multicultural teams with mixed permanent/contractor staff, a truly self-organising team is a rare find.

• Common sense - a large part of Scrum is essentially maintaining a common sense and pragmatic approach to things.  One doesn't have to be a certified Scrum Master to manage Scrum projects, but care should be taken in obeying & applying the rules of Scrum, which Ken concludes in Appendix A:
• The ScrumMaster is responsible for ensuring that everyone related to a project, whether chickens or pigs, follows the rules of Scrum.  These rules hold the Scrum process together so that everyone knows how to play. If the rules aren't enforced, people waste time figuring out what to  do.  If the rules are disputed, time is lost while everyone watis for a resolution. These rules have worked in literally thousands of successful projects. If someone wants to change the rules, use the Sprint retrospective meeting as a forum for discussion. Rule changes should originate from the Team, not management.  Rule changes should be entertained if and only if the Scrum Master is convinced that the Team and everyone involved understands how Scrum works in enough depth that they will be skillful and mindful in changing the rules. No rules can be changed until the Scrum Master has determined that this state has been reached.

Watch the man himself here @ GoogleTechTalks:

Managing a Digital TV Programme: Consideration for Project/Organisational Structures...

Following on from my recent post on the role and placement of the "architect" in a DTV systems project; in the same vein I'd like to expound on the role of the project team in the same end-to-end system delivery, touching on overall organisational structure, considering the concepts of:

• End-to-End Product Owner
• Feature Product Owner
• Component Product Owner
• End-to-End Programme Manager
• Headend Programme Manager
• STB Programme Manager
• Headend Component Project Manager
• STB Component Project Manager
• Headend Scrum Master / Tech Lead
• STB Scrum Master / Tech Lead

The collection of some or all these roles will make up your DTV Project team. These roles might each be performed by a person each, or we might have someone performing multiple roles.  Defining these structures up-front at the start of the project is considered Project Management 101, but you'll be surprised there are real world projects operating today where time, effort and preparation hasn't been invested in defining these organisations structures, and are operating in a quasi-chaotic state. I've been on projects where people run like headless chickens, not knowing who's responsible for what, where the boundaries lay in terms of roles and responsibilities, where due diligence isn't practiced through the overall programme: architecture, development, integration, testing & delivery...

The question of whether these roles are managed by a specific organisation unit like your classic Project Office or whether these roles come together naturally through different business units and operate as a virtual project team -- is outside the scope of this discussion.  My aim with this post is to highlight from my experiences what seems to work for DTV projects involving a launch of a new platform like VOD, or replacing old Middleware with new; again with more of a bias towards STB software, although the Headend will not be so dissimilar.  In essence, this can be drilled down to generic component product development at the micro-level, and at the macro-level these are rolled up in the end-to-end system as previously highlighted here for architecture, and the figure below that shows largely how a project team can be structured around the system.

It doesn't really matter about the internal business organisational structure, what matters is that the project has a defined structure and that these roles exist; the roles and responsibilities are well defined and clear, ownership and accountability is understood by all parties, that everyone is connected and on the same page. Of course you'd think this is Product/Project Management 101, that this is a no-brainer -- not so. 

In my experience, companies don't always have this done right, yet you might argue that products are still successfully launched, projects executed and closed, etc. Indeed, but IMHO such projects generally rely on a few individuals that go above and beyond the call of duty, the proactive, driven by a sense of urgency to get things done; filling in the gaps in process & roles -- that companies have grown to depend on.  Whilst this is great and generally seen as heroic efforts, the fire-fighting and critical moments can be saved if a little time is spent up-front by defining & clarifying the project structure.

Again, in the context of DTV projects - the structure depends on the viewpoint.  In general, if you're a broadcaster / PayTV operator (BSkyB, DirecTV, Multichoice, etc), you will have to implement a detailed project team as there is a much wider impact on the business (development, lab, testing, validation, going live, operations, support, customer service) than compared to one of your vendors (NDS, OpenTV, Irdeto, etc). If you're a Professional Services outfit acting as Systems Integrator and ultimately responsible for the delivery of the project, then you might have to mirror the same project structure of your customer.

Let us look at a picture to see where these roles can be placed. Notice this map is exactly the same layout as the architects post:

Figure 1: Overview of DTV System highlighting Project Roles

If you're an Operator focused on content delivery only, then...

You generally hire a System Integrator to manage the design and end-to-end delivery of the system on your behalf. These operators make a conscious decision not to be technical experts and thus rely on preferred partners to manage new development and operations.  The operator would still have someone who's responsible for product ownership, which could be someone from Marketing/Sales. There would also be a Programme or Project manager assigned to manage the project.

This is typical of first-time operators entering the market from cold, there are some professional services companies like NDS, who can manage not only the end-to-end systems development or customisation (generally this is off-the-shelf to enable shortest time to market, followed by product updates post launch) but also able to commission a broadcast centre and operations control from scratch...

So essentially this is an operator that outsource most of the Delivery Project Management - the organisational structure will pretty much be the same as Figure 1.

If you're a mature Operator with an established product and adding new features, but without technical ownership, then...

This scenario again relies on the Operator leaning on preferred partners for providing the technical solution, possibly spanning multiple vendors involved in development of components, system integration, delivery and deployment as well as operational support.  There would be a Product Owner / Manager that defines the high level requirements for the product's feature set. There could also be a Project Office team that takes ownership for the overall co-ordination effort from start to finish, or this could be outsourced to a professional services company to manage the whole project.

Some Operators have separate business entities owning features of the overall value chain, so the primary product spans multiple product owners. It becomes more challenging to manage this distribution of product owners especially when trying to manage an consistent brand and user experience, and coherent presence in the market. But companies are doing this today, it works to an extent, the key being effective communication is pivotal to successful product launches.

Generally though, in this scenario, the Operator has established strong relationships with its vendors, and projects may follow a typical template plan that would be driven by a Project Office team.

You might find the Operator would enlist a Project Board to manage the project's priorities, resources and budgets - that a Product Owner/Manager and Programme Manager reports and takes direction from.

Again, the project organisational breakdown will follow Figure 1.

If you're a mature Operator and responsible for developing and delivering new features, with technical ownership, then...

This can be very complicated and challenging if this is the first time the Operator is responsible for technical ownership. As touched on in my previous post, more and more Operators are taking more ownership in managing DTV projects, around the area of STB & Headend Systems Integration, EPG Development, Interactive Applications development, and some Operators are even developing their own Middleware.

In this scenario, it then becomes fundamental that the organisation starts on the correct footing, ensuring the foundations are cemented correctly, processes and practices firmly put in place as early as possible -- not doing so will definitely result in chaos, which I personally have come to label a "Project Fiasco".

So the roles I'd expect to see an organisational structure clearly defining the Architecture roles, Development, Integration and Testing; as well as a well defined Project Team.

I would expect to see a variant of Figures 1, 2 & 3 for the organisational structure.  There will be interaction with vendors. The relationships, roles & responsibilities must be explicitly communicated....

If you're a Product Vendor that delivers components to Operators, then...

Product Vendors are generally companies that provide software components for STB or Headend components that enable the Operator to function.  As a Product Vendor, the challenge is to internal manage your development organisation to maintain a consistent product line offering features that can be easily customized, tailored or profiled in or out to meet the requirements of your customer, the operator. Typically vendors manage multiple customers. Some customers more important than others - the customer usually generating the most of amount of revenue is assumed the lead customer, if not, the customer that in fact drives forward your product development and innovation.

Vendors typically have a Customer Delivery Manager assigned as an anchor support point, interface to the customer. This Delivery Manager ensures the customer's requirements are accepted by the product development team, and ensures the resulting plans are in accordance with the customer's expectations.  The vendor may choose to use a Product Steering Group that ultimately makes the decisions on customer priorities. Depending on the extent of the products being supplied by the vendor, I would expect to see an organisational structure not so dissimilar to Figure 3.

Brief Definitions

• Product Owner / Product Manager
• Operator - responsible for defining the user experience, high level requirements of the features for the product. For example, Operators looking at VOD will have a Product Owner owning the feature set, along with priorities. The Product Owner takes direction from the Project Board or even the business in understanding the market priorities and urgency of the business w.r.t. launch targets. The Product Owner / Manager will use the services of the architects and project team to get the product specified, designed, implemented and ultimately delivered.  The extent of ownership is around product specification only...The Product Owner/Manager gets actively involved in prioritizing defects leading up to the launch phase of the project, setting priorities, accepting waivers or concessions, etc.

• Vendor - as a vendor, the Product Owner / Manager is different to the role taken on by an Operator.  As a vendor, the Product Owner is concerned with satisfying as many customers as possible, without much deviation or customer-specific features; and will work hard to maintain a consistent offering. The Product Owner generally consists of a Product Team that centrally manage the product backlog. Depending on the size of the projects and demands from customers, this could be one person, or a team of people. If it's a Headend component with specific technical requirements (generally Headend component is a translator - data in, data out component) that implements a protocol, then this could be one person.  However, if this product is a STB Middleware that must sustain multiple customers, and potentially support tens of millions of boxes, then you'd need a separate team managing the product. This Product Management team will own the roadmap for the project, control the work assigned to customer projects, and ensures a coherent release schedule.  The team will also ensure the product specification and requirements are kept-up-to-date supporting the generic product as well as the customer's profile of the product. This Product team will consist of project managers driving through the product roadmap.

• In some cases the term Project Owner / Key Accounts Owner / Customer Delivery Manager is used which basically is the person who's neck is on the chopping block - i.e. the person accountable for the project delivery, who has seniority to make judgement calls to steer the project along in his/her preferred direction, etc - basically the ultimate person in charge, the "X says so" person...

• Programme Manager - this can be generalised both from Operator and Vendor side - and is typically the person responsible for the high level planning of the various work streams that make up the ultimate delivery.  There can be Programme Managers for each system, primarily Headend & STB, as well as managing the overall End-to-End delivery plan.  The Programme Manager works side-by-side with Product Owner in understanding business priorities, but the Programme Manager has autonomy in defining the plan and driving through the execution of that plan.  Programme Managers lean on their respective project managers to take responsibility for the detailed planning that delivers on the high level milestones.

• Project Manager - is responsible for planning, managing and driving through the deliverable of the assigned component/project. Reporting and taking instruction from the Programme Manager, the Project Manager handles the day-to-day issues/planning/tracking, escalating to next level (Programme Manager) as required.

• Scrum Master - can be a Project Manager or Team Leader employing the philosophy of Scrum/Agile to get the component/project team to deliver according to the milestone plan put forward by the Programme Manager and Project Manager. 

What a Product Owner / Manager is not responsible for...

When not acting in the capacity of the Project Owner, i.e. the main person accountable for the success of the delivery, then the Product Owner / Manager should only be focused on managing the Product requirements and Feature set, setting expectations and aligning priorities.  A Product Owner has no influence on the resource structure of the team, the scheduling of the development....that's what the Project Team, including the Programme Manager is there for - the Project Team provides a service to the organisation...Depending on the size/complexity of the deliverable, in DTV projects, they are large enough to warrant a clear separation of boundaries between Project Team & Product Team....Having people stomp on each others toes is not conducive to productivity...

We are an Agile Company, we don't like heavy Project Organisation...

Sure, depends on what kind of company you are. If you're in Application Development writing EPG UIs or Interactive applications, then of course, you can be light on the ground.

If you're in the Middleware business and big enough to be mentioned in the same sentence with NDS, OpenTV, etc -- and you don't have these structures in place, then you're at a competitive disadvantage. I have worked on small and large projects alike, the lack of a clearly defined and sensible Project/Organisation structure does not work...you might reach there eventually through luck, a lot of hard work and unnecessary fire-fighting...

For Operators taking the plunge at taking more technical ownership in your projects, my advice to you is "Do your homework". Spend time up-front investigating, planning and settling on a processes that has been known to work, i.e. proven - by getting real consultants in to advise...By real consultants, I mean not the people who know the theory but have never managed a real life project from Start-to-Finish, not someone who's never worked in the DTV industry before...Invest in the right people up-front, employing or head-hunting candidates who have first-hand experience and proven track records, and pay heed to the advice henceforth...

Typical Project Org Structure for a typical Operator

Below is a simple model of how an Operator might define an Organisational Structure of a DTV project...

Figure 2: A possible Org/Project Structure from the Operator's viewpoint

Typical Org Structure for a Vendor supplying components...

Below is a complicated model of a Vendor who's main business is STB development. Some vendors specialising in components in specific components will own a slice of this structure...

Figure 3: Org/Project Structure from a STB Software Vendor's Viewpoint

PDFs of Diagrams

Figure 1

Figure 2

Figure 3

Your Feedback

Please note my posts are always based on my first-hand experiences from past projects, and is a reflection of reality instead of theory. I am always interested in feedback, please feel free to comment or rate this post!

My first taste of crime in South Africa...

So it's been about six months since we moved back to South Africa and 3 months in our current house, and last weekend we experienced our first encounter with crime. It could've been much worse considering I've not yet sorted home and contents insurance out, so by the Grace of God, we got off easy in the grand scheme of South African crime...

We left home (Johannesburg) in a hurry on Friday afternoon (4th November) to drive up to Pietermaritzbug (6 hour journey) to spend Eid-ul-Adha, the festival of Sacrifice with family. It'd been 15 odd years since I last took part in Qurbani (the slaughtering of a sheep/lamb/cow) and I was going to slaughter an animal for the first time in my life. My kids have never witnessed such a thing before, very difficult to do practise this in UK...so we were all excited.

We had a wonderful Eid, returned home on Monday evening around 5PM. Being tired after the long journey, I didn't quite take notice of all the light entering the dining room and lounge. I'd deactivated the alarm on entry and getting the bags in, when I noticed the strange brightness coming in from the other room. I open the door, and to my amazement saw the dining room door wide open, the glass shattered and the outside gate wide open with the keys left hanging - we'd been burgled!

I quickly look around the house, it seemed the burglars only touched the lounge and dining room, not even the computer room (I wouldn't be typing this if they had) - and they got away with my PlayStation gear and wireless keyboard. The TV unit was shaken up a bit, but apart from that everything seemed in order - how odd, since the computer desk is in the lounge directly opposite the TV, I had my sat nav, photo frame and gadgets lying in plain sight...The only thing I could think of was the alarm had scared them away.

So I called the landlord to report the incident, then my alarm security company, who then alerted the police. I spent all of Sunday evening walking through the scene, testing the alarm, holding the security company to account, even doubting myself whether I actually activated the security system before leaving...Spent the night at my brother's place, next morning the so-called forensics team "fingerprint" experts came and went in like 10 minutes, I had to show them where to lift fingerprints off. Then the inspector came over to have a chat, he was equally baffled and blamed the security company. Spent the rest of Tuesday fighting with the security company and had the door fixed...

This morning the technician from the security company came over to download the event buffer from the alarm system. They were right, I was wrong. Actually the alarm was activated, but there were no reported incidents between Friday and Monday. I then tested some theories out and uncovered some serious loopholes with the system. If you enter the house, and crawl on all fours from the point of entry to the TV room/lounge, the sensors will not pick you up...so that's how they did it.  Now I need to get more sensors installed, more panic buttons and possibly a few more burglar gates and doors...welcome to South Africa!!

What's really scary though is that why hadn't they cleaned me out?? Why take only the PlayStation gear?? Why not come again and again?? The house was available the entire weekend - we have no idea when it could have happened. The neighbours didn't hear or see anything unusual...

Like I said, it could've been worse.  Houses are ransacked whilst people are fast asleep in their beds.  Better to have it happen whilst you're not at home, because you could lose your life...

So on the crime front, UK 1, SA 0.  But such petty crime is also rampant in UK, it could happen to anyone...In Islam, we believe that sometimes these small troubles happen in order to prevent or save you from a bigger calamity, that events are interconnected, and God has His plans and reasoning...We also have a saying to "Trust in God, but tie your camel!". In my rush to beat the traffic on Friday I didn't take the security precautions I usually take of taking all my keys with me, closing all the curtains, and leaving the property reciting verses of protection from the Quran...I also haven't sorted home insurance out...

Here's some pics of the incident:

Point of entry

Note the stone & towel used to break glass, bugger left his hat for show!

TV Area, good thing still got UK plugs

Burglar gate - bent bar

Used my ladder to scope out the bedroom 

TV unit, bottom left had Playstation gear

Architect Roles in Digital TV Projects

The subject of IT architecture is vast and multi-faceted; often open to interpretation by organisations in the same and different businesses alike; especially the definition and expectations of the "architect" role in projects in supporting the technical needs, administration and day-to-day operational management. A number of people attempt to explain what architecture means, from professional institutions like IEEE & TOGAF that provide a framework for describing the various architect roles for IT projects to attempts by wikipedia.

The rationale behind my post is to highlight how the role of architect manifests in companies specialising in digital TV systems, be it on the broadcast systems engineering front, or product developer offering systems and software services. It has been my experience that even with companies involved in the same industry, the use and understanding of the role "architect" varies, and is open to interpretation, often leading to confusion when you're mixed up in customer-vendor service relationships in managing projects. Some companies do attempt at having an organisational structure supporting technical architects, whilst other companies try to hire anyone with a technical background and assign the role "architect". But the aim of this post is NOT to describe the traits and skill-sets of architect VS coder; but to focus on the functional roles that an increasing number of DTV projects are now demanding from architects.

If you're a PayTV operator like BSkyB, DirecTV, Multichoice, etc - you need to think about the level of technical expertise required for your business:

• how much technical systems knowledge do you need to own?
• how much do you depend on external vendors to manage this on your behalf (e.g. the design of your end-to-end system)?
• do you have to produce & own your very own technical specifications for the entire value chain; or just key parts (e.g. EPG Spec, Middleware Spec, UI Spec)?
• or are you comfortable relying on your preferred system partners?
• do you need to have technical people on your side to prevent vendors from pulling the wool over your eyes?
• do you need technical owners for each product/sub-system of your deployment?
• do you want to drive or your suppliers/vendors or be driven (locked-in) by your suppliers solutions?
• do you opt for generic products to configure & customise yourself, or do you impose a custom requirements to serve your purpose, to meet your business needs, according to your environment?

This list goes on & on... There are an increasing number of PayTV operators that are moving away from the traditional vendor-supplier-user model, to taking more ownership and control over their products. Some operators have set-up internal development houses to manage Headend & STB UI/Middleware development; whilst other operators are taking over STB manufacturing in an attempt to control the end-to-end value chain.  In such cases, it therefore becomes even more relevant to have a solid technical product team, connected from end-to-end, with a thorough understanding of complete system, delivery and value chain of products, from high level features, requirements and design to detailed technical operations of the product --- this is not just my opinion, some companies have already embarked on this path (BSkyB, DirecTV, Foxtel).

If you're in the business of providing systems & software services to the above-mentioned operators, the likes of NDS, OpenTV, Irdeto and other Professional Services companies, then you probably have well defined roles for architects, and probably have an organisational structure of an architects group, supporting your many projects, with a group of chief architects as overseers for your product deployments.  If not, then please contact me as I'm interested in learning about what others in the industry are doing here :-)

I've been mulling over writing about DTV architects for a while now, but what really triggered this humble attempt stems from a recent experience on one of the projects I'm managing: I'm sure most of you are familiar with "It's not my problem, I'm an application architect not a systems architect" - you look for a systems architect only to find that one doesn't exist yet!

Anyway, to take this discussion forward - I consider the end-to-end layout of a typical Digital TV system. I'll present the different flavours of architect roles, and conclude laying out the minimum expectation of the roles that must exist to fulfill your typical DTV product deployment, with a bias towards the STB-side than the Headend.

This is NOT a theoretical or academic exercise. I'm describing real world experiences from past projects I've worked on with NDS/BSkyB/DirecTV/Foxtel/OpenTV, although the major influence is from NDS who, in my opinion, do an excellent job at executing complex DTV projects; and have an org structure not so dissimilar to what I'm presenting here.

I am really interested in your comments and feedback, what is the rest of the industry doing in this area?? Feel free to comment directly or drop me an email.

Consider this picture of a traditional end-to-end Digital TV system:

Figure 1: Components of DTV System Value Chain (Download the PDF)

Disclaimer: I'm no architect, this is Version 1, just my understanding and interpretation only. Architects do the best they can under the circumstances....there is a level of pragmatism involved and no one should be under the pretense that the solutions created are perfect, they are what they are i.e. a solution out of a continuum of many possible solutions for the problem which solves a business need at that point in time for a price the organisation can afford under the assumptions of the perceived known knowns and anticipating the future usage for that solution at that time.

A typical end-to-end DTV system is complicated; there's certainly nothing simple to it. At the highest macro level, one could summarise a DTV system as being split fundamentally into three areas (or in architect-speak, subsystems):

• Transport/Signalling: This is basically the physical layer, the profile of the underlying delivery mechanism. It can be abstracted and treated as a separate system as the products that sit on top of this layer still provide the same application behaviour, regardless of medium. I've separated it out because here you'll find physical hardware/infrastructure systems that prepare the data for the transmission across the appropriate delivery mediums (Satellite, Cable, IP, Terrestrial, Mobile)
• Producer: Headend/Backend: Systems that generate, secures & distributes content and information over the delivery medium (satellite, cable, terrestrial, mobile, IP)
• Consumer: Devices - typically the STBs receiving the transmission, decode and presenting the info and a suitable user experience to the subscriber at home

Within each area, the system is broken down further and further down to individual components performing a specific task. In reality, you will find that this end-to-end system is a carefully orchestrated integration of a number of different subsystems; offering a mix of different hardware and software components that are supplied by different companies. This is especially true for the back-end, integrating third party components such as the Conditional Access (CA) system, Billing & Subscriber Management, Scheduling, Information Data Generators, Content Catalogue Servers, Play-out systems, Encoders & Multiplexers.    At the consumer device end, the STB is a system in itself, also broken up into components or subsystems; and also not provided by one single company: Typically the STB consists of an Application Engine / User Interface component (the EPG), Middleware (the TV operating system) and Platform Components (Drivers, Hardware, Kernel). And these core STB components themselves are broken down into sub-components.

So your end-to-end DTV system is complicated & highly component-based. These sub-components, components, sub-systems, systems and ultimately the end-to-end solution must be co-ordinated and controlled -usually all these systems obey strict rules and contracts as defined by Interface Control Documents - Architecture 101...

So where do the different architect roles fit in then?

At the macro level, an end-to-end system could be managed by a team of architects specialising in these broad roles, be it at the Headend or STB domain:

• End-to-End (E2E) Systems Solutions Architect
• This is the top of the hierarchy, the solutions architect has a birds-eye view of the end-to-end system. Generally working directly with the the business (marketing/sales) in understanding the business and market driving factors in creating a solution for a specific need or product. 
• The E2E solutions architect is someone who can work with any technology domain, has a deep appreciation for all the subsystems including third-party external components, and will work with the relevant system architects in defining the high-level solution. This results in producing a grand, high-level design of the system components or building blocks, specifies the requirements and interfaces expected from each subsystem, laying the ground rules for system architects to take forward, adapt as necessary to apply the design in a real-world deployment.  
• The solutions architect is a domain expert, expected to take existing deployments and adapt, expand and enhance as necessary to deliver new product requirements.  This is generally the first stage of the projects design and scoping phase.  
• In some projects the solutions architect goes right into the detail of enhancing the MPEG/DVB signalling specification; and some times even creating a new end-to-end signalling protocol altogether. 
• This is a highly collaborative role, involving a lot of communication back-and-forth with sales/marketing, product development and integrators.  This could be one person, or a team of solutions architects taking responsibility for certain slices of the solution, depending on the scale of the project and problem being solved.
• Some people use the traditional IT term "Enterprise Architect" - but in all the projects I've worked with, the DTV industry tends to use "Solutions Architect". 
• In a nutshell, the solutions architect uses existing subsystems to deliver a solution to a problem and identifying shortcomings/gaps for attention Systems/Product Architects

• End-to-End Product Systems Architect
• Depending on the project and product being delivered, the solutions architect usually takes an existing solution and adapts in to the needs of the new product. In doing so, the solutions architect will touch on many components or subsystems that are impacted by this change, thus highlighting the areas that make up the new product.
• One might therefore assign specific Product Architects to take responsibility and own the components making up the product's value chain. 
• Primarily concerned with interactions between subsystems where Headend and STB is subsystem.
• The Product Architect assumes responsibility for the product solution and works hand-in-hand with the solutions architect in defining the solution that best meets the needs of the business.
• Solutions architects typically get involved at the initiation stage of the project and leave to solve the next challenge - they rarely see a project through to completion.  Product Architects however are expected to stay till the very end, supporting and making all architectural decisions to get the product from development through launch.
• This is still a high level responsibility, product architects work closely together with respective system products architects: Headend & STB.
• The product architect drives through the respective product specifications, covering functional and non-functional requirements (performance, reliability, resilience, etc).
• This role should not be confused with Product Owner or Product Manager; which are more management roles focussed on product features, specifications and high level requirements, which feed the work of the Product Architect.  Depending on the size of the organization and skill-sets available, these roles could be merged.
• Some people tend to refer to Product Architects as Technical Owners.

• System Architect - The system architect spans Headend & STB subsystems:
• STB System Architect / STB Product Architect
• Responsible for the end-to-end stack of the STB covering platform hardware, kernel, driver layer, middleware and EPG - Responsible for the overall STB product, taking ownership for the full product.
• Has a deep understanding of the key areas of the STB to the point of specifying the interfaces for all the above components
• Collaborates with respective EPG & Middleware architects in designing the overall STB solution
• Is responsible for defining the Functional & Non-Functional Requirements of the system: e.g. monitor/stipulate memory usage and own performance
• Helping the product team with requirements, and that includes backlog, supporting the initial planning phases and producing high level estimate
• The gate-keeper for analysing change requests
• Assisting in analysing, reviewing & prioritizing defects

• Headend System Architect / Headend Product Architect
• Whist the STB is a fairly closed system, the Headend on the other hand is an  integration of distributed systems. These systems are all interconnected, the Headend product architect is concerned with the interactions, interoperability and reliability of the integration.
• Works closely with other architects (subsystem/component architects) in defining the product specification. Since the Headend is meant to provide generic services and is a high-risk operation, Headend systems are generally well-thought out in advance, go live months before the STB is ready for deployment. The product architect plays a big part in driving the integration of these systems at the high level.
• Product architects generally own slices of the system, e.g. VOD
• All these slices, i.e. product architects work closely with the Solutions Architect for Headend, sometimes referred to as Chief Architect

• Component Architect - STB
• Application Architect - UI / EPG
• Essentially designs the UI / EPG as a component. Responsible for the internal architecture of this component
• Collaborates with Middleware Architects as necessary e.g. influencing APIs and interfaces
• Works with the STB Product Owner to understand requirements from high-level to detail
• Works closely with user experience specification team to ensure the UI demands are realistic, respecting the boundaries of the architecture of the UI's internal components
• Supports the product team in defect analysis, review and prioritization
• Implement STB system architect's Functional & Non-Functional requirements
• Assist the programme management team in high-level planning and estimates

• Middleware Architect
• Middleware is a system in itself, consisting of multiple components. Without drilling down into too much detail, the Middleware Architect ensures the component satisfies all requirements by providing services to applications using it
• Generally, the STB Architect is in fact the Middleware architect; using Middleware sub-component architects for detail. I worked on a Middleware product that had a centralised STB architect driving a team of component architects, where the Middleware was split into 80+ components...
• Defines APIs both internal and external to the Middleware
• Works with the STB Product Owner to understand requirements from high-level to detail
• Supports the product team in defect analysis, review and prioritization
• Implement STB system architect's Functional & Non-Functional requirements
• Assist the programme management team in high-level planning and estimates

• Platform Architect - Kernel/Drivers
• Responsible for the design and specification of the low-level drivers and kernel interfaces, typically the industry refer to this as Hardware Abstraction Layer (HAL) or Common Driver Interface (CDI)
• In general STB hardware specifications, once a platform launches, undergoes minor updates over time - the interfaces are generally future-proof, and new interfaces try to be backwards compatible, so the Platform Architect is usually required at the start of the project to define the major hardware features and enhancements...

Example scenarios of where this hierarchy is applicable?

Of course, projects don't always have to adopt the complete structure. Real world example projects applicable to this model include: Introducing Hybrid-Satellite/IP features offering Progressive Download (PDL), PushVOD, Broadcast Downlod, PullVOD/IP, Recommendations, Integrated Search, Application Stores, Catch-up, Network PVR, DRM - all these products impact the end-to-end value chain and needs a carefully orchestrated architecture and project execution team.

Why I think projects need to have clearly assigned architect roles?

Projects without a well defined technical team are bound to face difficulty and will most likely fail.

Projects need to have access to a technical team, there must be ownership and accountability for the different areas of the system. More importantly, the product must be connected end-to-end, the value-chain must offer a coherent design solution. Some broadcasters operate separate business units that not well connected. Just like a project team is set-up to manage the project delivery, it is vital to set-up your technical solutions team, assigning people to support the product and project from start to completion.

Take for example, a common project for DTV companies nowadays of changing your STB supplier, be it changing EPG UI or the STB Middleware. For simplicity, assume there are no changes required in the HeadEnd, we're just want to swap out old for new.  At the very least, the project should have the following identified as key roles:

• End-to-End Systems Architect
• STB Architect (doubling as Product Architect)
• EPG UI Architect

At the end of the day, we need to deliver a product. We need to go live and make some money. Having clearly defined roles for architects on your project team will:

• provide clarity, removes confusion over roles, responsibilities & expectations
• highlight gaps in your organisation - for e.g. you might have an EPG architect, but missing a STB architect - this is only going to cause problems down the line for integration.
• provides interfaces and eases communication channels
• promotes a coherent system - without this, chaos looms on the horizon

If you're a broadcaster and lack the skill-set and competencies for these roles, then you should demand this from your vendors. 

I'm an operator, do I really need to go into that level of detail?

Again it depends on how you answered the opening questions, fundamentally how much do you want to be in control?? BSkyB & DirecTV for example, have  STB Middleware & EPG architects; as well as STB & Headend System architects.  BSkyB produce their own Middleware design specification, in addition to requirements. BSkyB are very involved with end-to-end solution design, often taking weeks to review and comment on design proposals. As an operator, BSkyB have a strong technical team who understand the technology implementation of their entire value chain and can influence architectural discussions and decisions from both Headend & STB-side.  It is your product after-all, so why not control it as much as you can? At the macro level, it's down to trusting your suppliers and risk mitigation. Suppose your vendors disappear, do you know enough to support, enhance and maintain your products in their absence??

Useful Reference Material on the Subject

Who needs an architect anyway?

Architecture Definitions

Software Engineering Institute on Architecture definitions

How should an IT Architecture team be organized within a large company?

The Role and Task of the System Architect

The Role of the architect

Difference between coder and architect

The Role of the Agile Architect

Principles of the Agile Architect

Useful book on System Architecting

MIT Coursework on Architecture

MIT Coursework on Role of Architect

Jumbo Universal Remote Control - missed a trick!

This weekend I played tech support to my father-in-law, aged 70 - he was at war with his remote controls. His DSTV remote wasn't working properly, the power button was not doing anything, so he couldn't switch off his TV.  He'd gone out and bought not one, but three jumbo universal remote controls from the local warehouse that sells almost anything, mostly knock-offs made in China.

So he shows me this Jumbo Universal Remote:

At first glance, this remote looked interesting. Very nice, large buttons, good contract. Very nice for the in-laws. Alas, this remote wasn't suitable for my father's-in-law set up.

But here's the really annoying part. The user manual for this remote is a teeny tiny booklet, using a font-size that even I had trouble reading!

I am passionate about accessibility as you'll see from a future post on Talking TVs, a personal project of mine. This lack of regard for the users really annoys me. How can you create a large jumbo remote control and ship a really small user manual with it - I mean what's the point? Are you playing a joke? Are companies so insensitive to the needs of their users??  There is a large and growing user base of people who need accessible products, this market should not be ignored. New products must cater for this group from day one, at the design phase -- accessibility is rarely difficult to add-on post production. Most products that attempt to add accessibility afterwards, and not making it inherently native to the product, fail to deliver...unlike Apple products. Apple, despite their modern and ultra slick products, are surprisingly accessible and being used by many with vision problems (blind/partially sighted)...this is not just my opinion, ask Stevie Wonder for his opinion :-)

At first, I thought the remote must have been a cheap Chinese clone. But no, go to JumboRemoteDotCom and download the user manual and see for yourself!  That's not the only thing - take a minute to think about the users of this remote control, such as my father-in-law: The instructions are virtually impossible to comprehend...sigh.

There are indeed TV companies out there that care about accessibility, and go to lengths spending time and money on R&D to produce decent accessible remote controls. Take BSkyB in the UK for example, who provide accessible remote controls free of charge to their subscribers needing assistance.  More recently in South Africa, DSTV have also produced a large remote control, although not free for subscribers due to a somewhat different business model and market to BSkyB. This company from US also have some interesting products on offer.

DSTV Jumbo Remote:

BSkyB Remote: