Pragmatic Set-Top-Box QA Testing - Don't just trust Requirements-to-Test Coverage scripts

This post might be a little edgy - I think it's important to understand alternative perspectives around Set-Top-Box (STB) testing, how it is uniquely different to other forms of IT Testing; to understand the dynamics of STB testing that promotes agility and flexibility, whilst simultaneously understanding the risks associated with pragmatic testing.

The Headline:
DON'T BLINDLY TRUST REQUIREMENTS-TO-TEST COVERAGE MAPS

Last year, I wrote quite an in-depth paper on Effective Defect & Quality Management in typical DTV projects. It covered many topics, and touched briefly on the aspect of Project Metrics reporting. This post expands on the subject of QA Metrics tracking, focusing on how this reporting can help change the direction of the project, and instigate changes in focus to the overall QA efforts, particularly around Set-Top-Box QA testing. I advocate the project will change QA focus as stability is achieved with Requirement-to-Test coverage maps, to include more Exploratory, Risk-based & User-based testing.

Worlds of QA Testing in Digital TV Systems Projects

I have previously written about the Digital TV ecosystem with its complexities and challenges in defining the Architecture & Systems Integration spaces. In this post I will share some thoughts around the QA (Quality Assurance) / Testing problem space, as well as generally accepted strategies for solving these problems.

What really triggered this post (which is centred around E2E QA) was a recent conversation with a QA Manager who in passing commented that he's facing a challenge with the "End-to-End QA" (E2E QA) team, in that it takes up to six weeks to complete a full QA cycle. Now some might think this is expected, as E2E QA is the last mile of QA and should take as long as it takes. My response to this is that it depends...

It depends on which phase your project is: is it still in early development / integration testing - where components are being put together for the first time to test out a feature across the value chain? Or is the project well ahead in its iterations having already executed much of the E2E testing? It also depends on the scope and layers of testing the systems and sub-systems before it gets to E2E Testing. Has a version of the system already been promoted to live? If so, what are the deltas between the already live, deployed system compared to the current system under test?

[--- Side note:

This post is one of many to follow that will encompass some of the following topics:

• Worlds of testing / QA
• Is ATP / E2E testing the longest cycle than other system test cycles - or should it be the shortest?
• High level block diagrams of architecture / interfaces / system integration
• Visualisation of technical skills required for various aspects of testing skills required at various areas of the environment: Scale of technical - heat map
• E2E QA is highly technical and should be considered as part of Integration, not a separate QA function
• Which came first: the chicken or the egg? Headend first, then STB, or do both together - big bang?
• How to solve complex systems integration/test in DTV?
• What is end-to-end testing, what areas should you focus on?
• What is required from a Systems ATP?
• What kind of QA team structure is generally accepted practice? How does Agile fit in with this?
• Can E2E Testing be executed using Agile - iterate on features End-to-End?

Side note --- end ---]

Digital TV Summit Day Three...

The summit ended today with a market analysis on the opportunities for PayTV in Africa, covering most of aspects of the end-to-end value chain, from content procurement to delivery and consumption. I found today's session especially interesting and refreshing, a great complement to the book I am currently reading, The Business of Media Distribution: Monetizing Film, TV and Video Content in an Online World, providing an overview into what the business of PayTV actually entails. We learnt from past experiences of people who've worked in several countries in launching PayTV operations in-country, had an overview of a core component of the system (i.e. Content Management System), touched on Next-Generation platforms, and culminated in a demo on a low-cost (sub $100), but advanced Set-Top-Box, exploring the opportunities that await the African market.

My notes cover the following:

• Day 2: Case Study Namibia Spectrum Allocation
• Overview of Africa Markets
• Lessons from Past (failed) African Projects
• Content & Operational Considerations
• Overview of Media Asset Management Systems
• Next Generation Platforms
  
• Overview of Monitoring Systems
  

Digital TV Summit Day Two...

Here's my write up from Day 2 of the SA Digital TV Summit held in Bryanston, Johannesburg. The second day's Agenda went much deeper that the first day, although there was a no-show from the SABC on expanding the topic of Digital Migration. Nevertheless, I learnt quite a bit today as well as refreshing some other topics I don't get to focus on in my day-to-day work. We covered the following areas:

• SA DTT STB Subsidy Rollout Plan
• The Connected TV - Research Feedback
• Insights from Discovery Network
• Update on Namibia's DTT Migration
• Impromptu Discussion on DTT STB Readiness
• Content Delivery Networks Overview

Digital TV Summit Day One...

The inaugural Digital TV Summit of South Africa takes place this week in Bryanston, Johannesburg. I registered almost immediately on receiving the invite back in 2012, since one of the key speakers was to be Collins Khumalo, CEO of Multichoice South Africa. I was looking forward to seeing Collins in action, as my way of assessing how Multichoice contributes to the wider cause of Digital TV in South Africa. Unfortunately, the agenda was changed last minute, some key speakers, including Collins couldn't make it. I was also interested in networking to get an idea of the landscape of this market in South Africa/Africa...

Anyway, I am still attending the conference and will share what I've learnt from the presentations, on this blog - starting with Day One, which gently introduced the conference as being biased towards the topic of Digital Migration / DTT (Digital Terrestrial Transmission) as being one of the biggest event, if not, the biggest event to happen to the TV Broadcast Industry since its inception back in 1974...

Day One covered the following (Click on the links to jump to the topic):

• Digital broadcasting trends in Sub-Saharan Africa
• DTT SwitchOver Discussion
• Looking at Internet Protocol TV (IPTV) and its future in Africa
• Sentech's DTT Infrastructure Readiness Update