Architect Roles in Digital TV Projects

The subject of IT architecture is vast and multi-faceted; often open to interpretation by organisations in the same and different businesses alike; especially the definition and expectations of the "architect" role in projects in supporting the technical needs, administration and day-to-day operational management. A number of people attempt to explain what architecture means, from professional institutions like IEEE & TOGAF that provide a framework for describing the various architect roles for IT projects to attempts by wikipedia.

The rationale behind my post is to highlight how the role of architect manifests in companies specialising in digital TV systems, be it on the broadcast systems engineering front, or product developer offering systems and software services. It has been my experience that even with companies involved in the same industry, the use and understanding of the role "architect" varies, and is open to interpretation, often leading to confusion when you're mixed up in customer-vendor service relationships in managing projects. Some companies do attempt at having an organisational structure supporting technical architects, whilst other companies try to hire anyone with a technical background and assign the role "architect". But the aim of this post is NOT to describe the traits and skill-sets of architect VS coder; but to focus on the functional roles that an increasing number of DTV projects are now demanding from architects.

If you're a PayTV operator like BSkyB, DirecTV, Multichoice, etc - you need to think about the level of technical expertise required for your business:

• how much technical systems knowledge do you need to own?
• how much do you depend on external vendors to manage this on your behalf (e.g. the design of your end-to-end system)?
• do you have to produce & own your very own technical specifications for the entire value chain; or just key parts (e.g. EPG Spec, Middleware Spec, UI Spec)?
• or are you comfortable relying on your preferred system partners?
• do you need to have technical people on your side to prevent vendors from pulling the wool over your eyes?
• do you need technical owners for each product/sub-system of your deployment?
• do you want to drive or your suppliers/vendors or be driven (locked-in) by your suppliers solutions?
• do you opt for generic products to configure & customise yourself, or do you impose a custom requirements to serve your purpose, to meet your business needs, according to your environment?

This list goes on & on... There are an increasing number of PayTV operators that are moving away from the traditional vendor-supplier-user model, to taking more ownership and control over their products. Some operators have set-up internal development houses to manage Headend & STB UI/Middleware development; whilst other operators are taking over STB manufacturing in an attempt to control the end-to-end value chain.  In such cases, it therefore becomes even more relevant to have a solid technical product team, connected from end-to-end, with a thorough understanding of complete system, delivery and value chain of products, from high level features, requirements and design to detailed technical operations of the product --- this is not just my opinion, some companies have already embarked on this path (BSkyB, DirecTV, Foxtel).

If you're in the business of providing systems & software services to the above-mentioned operators, the likes of NDS, OpenTV, Irdeto and other Professional Services companies, then you probably have well defined roles for architects, and probably have an organisational structure of an architects group, supporting your many projects, with a group of chief architects as overseers for your product deployments.  If not, then please contact me as I'm interested in learning about what others in the industry are doing here :-)

I've been mulling over writing about DTV architects for a while now, but what really triggered this humble attempt stems from a recent experience on one of the projects I'm managing: I'm sure most of you are familiar with "It's not my problem, I'm an application architect not a systems architect" - you look for a systems architect only to find that one doesn't exist yet!

Anyway, to take this discussion forward - I consider the end-to-end layout of a typical Digital TV system. I'll present the different flavours of architect roles, and conclude laying out the minimum expectation of the roles that must exist to fulfill your typical DTV product deployment, with a bias towards the STB-side than the Headend.

This is NOT a theoretical or academic exercise. I'm describing real world experiences from past projects I've worked on with NDS/BSkyB/DirecTV/Foxtel/OpenTV, although the major influence is from NDS who, in my opinion, do an excellent job at executing complex DTV projects; and have an org structure not so dissimilar to what I'm presenting here.

I am really interested in your comments and feedback, what is the rest of the industry doing in this area?? Feel free to comment directly or drop me an email.

Consider this picture of a traditional end-to-end Digital TV system:

Figure 1: Components of DTV System Value Chain (Download the PDF)

Disclaimer: I'm no architect, this is Version 1, just my understanding and interpretation only. Architects do the best they can under the circumstances....there is a level of pragmatism involved and no one should be under the pretense that the solutions created are perfect, they are what they are i.e. a solution out of a continuum of many possible solutions for the problem which solves a business need at that point in time for a price the organisation can afford under the assumptions of the perceived known knowns and anticipating the future usage for that solution at that time.

A typical end-to-end DTV system is complicated; there's certainly nothing simple to it. At the highest macro level, one could summarise a DTV system as being split fundamentally into three areas (or in architect-speak, subsystems):

• Transport/Signalling: This is basically the physical layer, the profile of the underlying delivery mechanism. It can be abstracted and treated as a separate system as the products that sit on top of this layer still provide the same application behaviour, regardless of medium. I've separated it out because here you'll find physical hardware/infrastructure systems that prepare the data for the transmission across the appropriate delivery mediums (Satellite, Cable, IP, Terrestrial, Mobile)
• Producer: Headend/Backend: Systems that generate, secures & distributes content and information over the delivery medium (satellite, cable, terrestrial, mobile, IP)
• Consumer: Devices - typically the STBs receiving the transmission, decode and presenting the info and a suitable user experience to the subscriber at home

Within each area, the system is broken down further and further down to individual components performing a specific task. In reality, you will find that this end-to-end system is a carefully orchestrated integration of a number of different subsystems; offering a mix of different hardware and software components that are supplied by different companies. This is especially true for the back-end, integrating third party components such as the Conditional Access (CA) system, Billing & Subscriber Management, Scheduling, Information Data Generators, Content Catalogue Servers, Play-out systems, Encoders & Multiplexers.    At the consumer device end, the STB is a system in itself, also broken up into components or subsystems; and also not provided by one single company: Typically the STB consists of an Application Engine / User Interface component (the EPG), Middleware (the TV operating system) and Platform Components (Drivers, Hardware, Kernel). And these core STB components themselves are broken down into sub-components.

So your end-to-end DTV system is complicated & highly component-based. These sub-components, components, sub-systems, systems and ultimately the end-to-end solution must be co-ordinated and controlled -usually all these systems obey strict rules and contracts as defined by Interface Control Documents - Architecture 101...

So where do the different architect roles fit in then?

At the macro level, an end-to-end system could be managed by a team of architects specialising in these broad roles, be it at the Headend or STB domain:

• End-to-End (E2E) Systems Solutions Architect
• This is the top of the hierarchy, the solutions architect has a birds-eye view of the end-to-end system. Generally working directly with the the business (marketing/sales) in understanding the business and market driving factors in creating a solution for a specific need or product. 
• The E2E solutions architect is someone who can work with any technology domain, has a deep appreciation for all the subsystems including third-party external components, and will work with the relevant system architects in defining the high-level solution. This results in producing a grand, high-level design of the system components or building blocks, specifies the requirements and interfaces expected from each subsystem, laying the ground rules for system architects to take forward, adapt as necessary to apply the design in a real-world deployment.  
• The solutions architect is a domain expert, expected to take existing deployments and adapt, expand and enhance as necessary to deliver new product requirements.  This is generally the first stage of the projects design and scoping phase.  
• In some projects the solutions architect goes right into the detail of enhancing the MPEG/DVB signalling specification; and some times even creating a new end-to-end signalling protocol altogether. 
• This is a highly collaborative role, involving a lot of communication back-and-forth with sales/marketing, product development and integrators.  This could be one person, or a team of solutions architects taking responsibility for certain slices of the solution, depending on the scale of the project and problem being solved.
• Some people use the traditional IT term "Enterprise Architect" - but in all the projects I've worked with, the DTV industry tends to use "Solutions Architect". 
• In a nutshell, the solutions architect uses existing subsystems to deliver a solution to a problem and identifying shortcomings/gaps for attention Systems/Product Architects

• End-to-End Product Systems Architect
• Depending on the project and product being delivered, the solutions architect usually takes an existing solution and adapts in to the needs of the new product. In doing so, the solutions architect will touch on many components or subsystems that are impacted by this change, thus highlighting the areas that make up the new product.
• One might therefore assign specific Product Architects to take responsibility and own the components making up the product's value chain. 
• Primarily concerned with interactions between subsystems where Headend and STB is subsystem.
• The Product Architect assumes responsibility for the product solution and works hand-in-hand with the solutions architect in defining the solution that best meets the needs of the business.
• Solutions architects typically get involved at the initiation stage of the project and leave to solve the next challenge - they rarely see a project through to completion.  Product Architects however are expected to stay till the very end, supporting and making all architectural decisions to get the product from development through launch.
• This is still a high level responsibility, product architects work closely together with respective system products architects: Headend & STB.
• The product architect drives through the respective product specifications, covering functional and non-functional requirements (performance, reliability, resilience, etc).
• This role should not be confused with Product Owner or Product Manager; which are more management roles focussed on product features, specifications and high level requirements, which feed the work of the Product Architect.  Depending on the size of the organization and skill-sets available, these roles could be merged.
• Some people tend to refer to Product Architects as Technical Owners.

• System Architect - The system architect spans Headend & STB subsystems:
• STB System Architect / STB Product Architect
• Responsible for the end-to-end stack of the STB covering platform hardware, kernel, driver layer, middleware and EPG - Responsible for the overall STB product, taking ownership for the full product.
• Has a deep understanding of the key areas of the STB to the point of specifying the interfaces for all the above components
• Collaborates with respective EPG & Middleware architects in designing the overall STB solution
• Is responsible for defining the Functional & Non-Functional Requirements of the system: e.g. monitor/stipulate memory usage and own performance
• Helping the product team with requirements, and that includes backlog, supporting the initial planning phases and producing high level estimate
• The gate-keeper for analysing change requests
• Assisting in analysing, reviewing & prioritizing defects

• Headend System Architect / Headend Product Architect
• Whist the STB is a fairly closed system, the Headend on the other hand is an  integration of distributed systems. These systems are all interconnected, the Headend product architect is concerned with the interactions, interoperability and reliability of the integration.
• Works closely with other architects (subsystem/component architects) in defining the product specification. Since the Headend is meant to provide generic services and is a high-risk operation, Headend systems are generally well-thought out in advance, go live months before the STB is ready for deployment. The product architect plays a big part in driving the integration of these systems at the high level.
• Product architects generally own slices of the system, e.g. VOD
• All these slices, i.e. product architects work closely with the Solutions Architect for Headend, sometimes referred to as Chief Architect

• Component Architect - STB
• Application Architect - UI / EPG
• Essentially designs the UI / EPG as a component. Responsible for the internal architecture of this component
• Collaborates with Middleware Architects as necessary e.g. influencing APIs and interfaces
• Works with the STB Product Owner to understand requirements from high-level to detail
• Works closely with user experience specification team to ensure the UI demands are realistic, respecting the boundaries of the architecture of the UI's internal components
• Supports the product team in defect analysis, review and prioritization
• Implement STB system architect's Functional & Non-Functional requirements
• Assist the programme management team in high-level planning and estimates

• Middleware Architect
• Middleware is a system in itself, consisting of multiple components. Without drilling down into too much detail, the Middleware Architect ensures the component satisfies all requirements by providing services to applications using it
• Generally, the STB Architect is in fact the Middleware architect; using Middleware sub-component architects for detail. I worked on a Middleware product that had a centralised STB architect driving a team of component architects, where the Middleware was split into 80+ components...
• Defines APIs both internal and external to the Middleware
• Works with the STB Product Owner to understand requirements from high-level to detail
• Supports the product team in defect analysis, review and prioritization
• Implement STB system architect's Functional & Non-Functional requirements
• Assist the programme management team in high-level planning and estimates

• Platform Architect - Kernel/Drivers
• Responsible for the design and specification of the low-level drivers and kernel interfaces, typically the industry refer to this as Hardware Abstraction Layer (HAL) or Common Driver Interface (CDI)
• In general STB hardware specifications, once a platform launches, undergoes minor updates over time - the interfaces are generally future-proof, and new interfaces try to be backwards compatible, so the Platform Architect is usually required at the start of the project to define the major hardware features and enhancements...

Example scenarios of where this hierarchy is applicable?

Of course, projects don't always have to adopt the complete structure. Real world example projects applicable to this model include: Introducing Hybrid-Satellite/IP features offering Progressive Download (PDL), PushVOD, Broadcast Downlod, PullVOD/IP, Recommendations, Integrated Search, Application Stores, Catch-up, Network PVR, DRM - all these products impact the end-to-end value chain and needs a carefully orchestrated architecture and project execution team.

Why I think projects need to have clearly assigned architect roles?

Projects without a well defined technical team are bound to face difficulty and will most likely fail.

Projects need to have access to a technical team, there must be ownership and accountability for the different areas of the system. More importantly, the product must be connected end-to-end, the value-chain must offer a coherent design solution. Some broadcasters operate separate business units that not well connected. Just like a project team is set-up to manage the project delivery, it is vital to set-up your technical solutions team, assigning people to support the product and project from start to completion.

Take for example, a common project for DTV companies nowadays of changing your STB supplier, be it changing EPG UI or the STB Middleware. For simplicity, assume there are no changes required in the HeadEnd, we're just want to swap out old for new.  At the very least, the project should have the following identified as key roles:

• End-to-End Systems Architect
• STB Architect (doubling as Product Architect)
• EPG UI Architect

At the end of the day, we need to deliver a product. We need to go live and make some money. Having clearly defined roles for architects on your project team will:

• provide clarity, removes confusion over roles, responsibilities & expectations
• highlight gaps in your organisation - for e.g. you might have an EPG architect, but missing a STB architect - this is only going to cause problems down the line for integration.
• provides interfaces and eases communication channels
• promotes a coherent system - without this, chaos looms on the horizon

If you're a broadcaster and lack the skill-set and competencies for these roles, then you should demand this from your vendors. 

I'm an operator, do I really need to go into that level of detail?

Again it depends on how you answered the opening questions, fundamentally how much do you want to be in control?? BSkyB & DirecTV for example, have  STB Middleware & EPG architects; as well as STB & Headend System architects.  BSkyB produce their own Middleware design specification, in addition to requirements. BSkyB are very involved with end-to-end solution design, often taking weeks to review and comment on design proposals. As an operator, BSkyB have a strong technical team who understand the technology implementation of their entire value chain and can influence architectural discussions and decisions from both Headend & STB-side.  It is your product after-all, so why not control it as much as you can? At the macro level, it's down to trusting your suppliers and risk mitigation. Suppose your vendors disappear, do you know enough to support, enhance and maintain your products in their absence??

Useful Reference Material on the Subject

Who needs an architect anyway?

Architecture Definitions

Software Engineering Institute on Architecture definitions

How should an IT Architecture team be organized within a large company?

The Role and Task of the System Architect

The Role of the architect

Difference between coder and architect

The Role of the Agile Architect

Principles of the Agile Architect

Useful book on System Architecting

MIT Coursework on Architecture

MIT Coursework on Role of Architect