QA Manager Job Spec

As a Software & Systems Engineering Management Consultant, I spend a lot of my time helping senior management sort out their department processes, run audits around international benchmarking, teach and coach topics that I'm quite familiar with, help with crafting roles & responsibilities for project & department structures, help set up implementation strategies, and remain on board to see these recommendations & changes through to completion...

Recently, I helped a client rationalise its Quality Assurance (QA) Engineering organisation. I have written about QA/Testing in the past, most of those posts revolve around the experiences from this particular client. You can check out these two resources for my background & knowledge of the Quality Engineering QA/QC disciplines:

• All QA blog posts on this site
• My most read QA white paper 

One of the triggers that drove this client to rethink its QA structure was some work that I did last year, around benchmarking what other organisations do in similar projects, and the outcome from that investigation identified an overload in the QA space, with lots of teams doing similar activities, resulting in duplication. The ratio of QA-to-Developers seemed unnaturally high. I personally knew this from the very beginning, but had to learn the hard way of taking a company through its own journey of self-discovery, that whilst an expert might see flaws and holes on day one, the customer is usually so set in its ways-of-working, that they are blinded by the inefficiencies because the projects do deliver anyway.

Anyway, I helped identify a need for a QA Manager, and helped create a job specification around this - which is what I'd like to share with you, in case you are looking for something similar. In my view, this is still a journey, the QA manager maybe an interim step, as an improvement to embedding some quality disciplines within the organisation, my end goal however, is to lead this client to a Lean Mindset, which takes the best of Agile/Lean processes, and drives quality engineering principles & disciplines right up the value chain, to the source - which is around architecture, design & development...we're not yet ready for that yet, so a QA Manager intervention is as good-a-starting-point-as-any....

P.S. My source of info for drawing up this job spec was the IEEE SWEBOK (I once studied the IEEE CSDP although didn't want to write the exam because I'm currently against any form of certifications, although I'm still a member of IEEE & ACM) as well as my own previous job experience...

 

JOB SPECIFICATION:: QUALITY ASSURANCE MANAGER

ENGINEERING: DECODER QUALITY

 

CONTEXT

[THE COMPANY] has rationalized its various Testing roles in the department and has introduced a new role focused on driving Software Quality Assurance Practice across the business. The reporting department line (CEQA) is overall responsible for applying Software Quality Management (SQM) that applies to all perspectives of decoder software processes, products and resources. SQM defines processes, process owners, and requirements for those processes, measurements of the process and its outputs, and feedback channels.

 

SQM processes consist of many activities. Specifically, with respect to [THE COMPANY], this is broken down into the following areas:

• Software / System Quality Assurance
• Software & Systems Testing (Quality Control / QC)
• Automation Tools & Methods
• Real-world home testing (Field trials)

    

Collectively, the above areas are separate lines reporting to an overall Head: Quality. These SQM streams however are interconnected and share the underlying foundation of quality principles, which all these groups need to adhere.

This role is about a Quality Assurance Manager position, which is based on three pillars:

• Measurement & Metrics
• Standards, Reviews & Audits
• Project-based role as QA Program Manager / QA Director

SOFTWARE QUALITY ASSURANCE (SQA) VISION

SQA processes provide assurance that the software products and processes in the project life cycle conform to their specified requirements by planning, enacting, and performing a set of activities to provide adequate confidence that quality is being built into the software at all levels of the software development lifecycle (SDLC).

The role of SQA with respect to process is to ensure that planned processes are appropriate and later implemented according to plan, and that relevant measurement processes are provided to the organization.

The SQA plan defines the means that will be used to ensure that software developed for a specific product satisfies the user's requirements and is of the highest quality possible within project & business constraints.

SQA takes into account the culture of the development teams, identifies processes, standards and practices, and conventions governing the project, highlighting how they will be checked and monitored to ensure adequacy, compliance, and ultimately the decision that the product is fit-for-purpose from a QA standpoint (checks and balances).

SQA also identifies measures, statistical techniques, quality metrics including procedures for problem reporting and corrective action, techniques & methods including tools, training, reporting & relevant documentation.

SQA also contributes to defining acceptance criteria, as well as reporting and management activities which are critical to software quality.

SQA will also contain a review and audit facility that may involve, but not be limited to the following areas: Management reviews, Technical reviews, Inspections, Walk-throughs & Audits. This can only be implemented once the department has reached a level of maturity and stability of processes (i.e. standards adherence) that have proven to deliver business value, such that it becomes the charter & practice of all projects going forward, and hence warrants a review & audit activity.

WHAT ARE WE LOOKING FOR IN A QUALITY ASSURANCE MANAGER?

• First-hand experience of the full software development life-cycle (SDLC)
  • Fair amount of know-how into software development, integration & testing
• To have delivered at least THREE big projects or products to market
• Demonstrates experience of managing one or more teams often representing multiple disciplines
• Demonstrates ability to co-ordinate, supervise and lead activities, including past experience with managing engineers / teams
• Recognised as an expert in the QA space, with past experience of knowledge-sharing, provided guidance, mentoring & coaching of others
• Someone who is passionate about software quality engineering in the modern world (Note: there is no one-size fits all, our Decoder product development team has embraced agile, and less formal methods – so we are not expecting SQA to bear heavy-weight, traditional quality principles)
  
• Experience with delivering quality products to market in very short timeframes (3-6 months)
• Working knowledge of Test-Driven Development, Continuous Build, Integration & Testing, including Regression testing, Automation tools
• Openness, flexibility & welcoming attitude to new thinking, changes & fluctuations in an erratic, energetic environment.
• We are looking for someone that can bring a little bit of order to chaos, without the dogma
• Demonstrable proof of knowledge in SQA based on achievements from past projects
• Breadth & Depth of SQA knowledge with a willingness to share and teach, train & guide as required
• Experience with setting up, leading an organization / department through changes within QA space, to have led the change, delivered the change, maintained and monitored the change until the change stuck, becoming the new standard for the business.

OVERVIEW OF JOB TASKS / RESPONSIBILITIES

Customers	GM of Engineering Head of QA Development Managers Delivery Managers Programme Managers Release / Project Managers Systems Integration Technical Leads Development Leads STB Manufacturers Middleware Software suppliers CA Providers Product management Field Trial Manager & Leads QC Manager Automation Manager QA Forum QA Steering Group Scrum Masters Product Owners
Requirements Analysis / Specification & Design	Ensures the team have the necessary requirements on time Contributes to QA input criteria into formation & style of Requirements documents Reviews new test methodologies and their impact on requirements and design Anticipates future test requirements and respond accordingly Reviews architecture, interfaces and design – and comments on overall consistency and testability Ensures QA representation in the project / product start-up phase
Test Planning and Design (Strategy, Plan & Test Cases)	Ensures there is a process for test design reviews Monitors and validates that test design phase / test planning is taking place Contributes to new technology / practice adoption (where expertise are relevant) Ensures / Analyses consistency and efficiency of test strategies across the department is maintained across projects & teams Maintains consistency of common test architectures, by actively managing to avoid duplication & fragmentation Reviews test strategy from QC teams Ensures that test strategy is defined and accepted by all parties
Test Development (includes environment, tools & automation)	Ensures the department has adopted use of industry standards or in-house tools Contributes to divisional and company-wide strategy on the use of test management tools Contributes to the definitions, codes of practice, w.r.t. testing process Ensures that test case development and reviews happen consistently, on time & communicated to all impacted parties Assures there is an acceptable test automation strategy and practice in place (via Automation Manager) Ensures automation strategies across multiple projects and teams are consistent, recommends appropriate re-use Acquaints self with industry standards, in-house tools and recommends appropriate uses Contributes to company wide strategy on use of test management tools
Test execution, analysis and reporting	Reviews & Approves changes in test strategies & test execution plans Reviews & Approves test reports from all impacted parties Tests that teams have implemented testing according to the practices set forth (audit / walkthrough testing). Ensures the projects deliver regular, timely reports as necessary Reports overall health of projects, products, teams to senior management Ensures that a product is declared fit-for-purpose subject to applying the various QA checks and balances required Strong influencer in recommending product's suitability for go-live
Integration activities	Helps integrate various teams and disciplines together on a common set of QA principles and practices Works with Integration teams to help define the processes & workflows Maintains oversight that deliverables from SI into downstream QC teams is of an acceptable level Responsible for defining acceptance criteria from all component vendors Holds teams to account for failure in delivering or meeting quality objectives & criteria Ensures that there is an integration plan produced as part of the test planning phase
Quality	Defines and monitors quality metrics and indicators Ensures compliance with quality objectives and goals Acts as a catalyst for change and improvement in performance and quality Implements the department's Quality Charter (help create one if it does not exist). Firm knowledge and grasp of industry best practice in quality principles Proposes adoption of new technologies to improve quality metrics Introduces and drives the adoption of new methodologies Promotes the use of best practices across the division and company-wide Participates in company-wide quality initiatives May perform tests in selected areas in order to keep up with own technical expertise Leads "lessons learnt" sessions on testing activities and escaped defects Proposes methods for implementing improvements to software/code quality
Transverse Relationships	Liaise with key stakeholders to ensure representation of test engineering staff in relevant forums and reviews, including demos to Exec Chairs QA technical meetings Represents area of technology (e.g. STB) outside their domain (e.g. DDM) Is (or on the path to being) recognised as a technical QA specialist company-wide Challenges existing technical concepts and directions Is the ambassador of new technologies and new ways of working Sells new ideas to senior management and customers / vendors Interacts with senior management, peers, engineers (dev / test / integration) & vendors Collaborates with other business units (e.g. DDM) on QA best practices, looking for opportunities for reuse, knowledge-sharing, cross-pollination, etc.
QA Programme Management / QA Director Outputs	Own the QA plan from all QA workstreams, feeding back to overall Programme Manager / Plan – includes CEQA, SIQA, HWQA, DDM, IBS, E2E, Backend, SMC management, Decoder Analysts, UIQA, Security CA Testing, Initial production line QA (is carton and manual correct and are all the accessories in the box, is the right SW build being used, decoder centre handling) Validates that adequate test infrastructure exists across the teams. Drive the creation of an aligned test strategy and charter in conjunction with various QA teams for the product/project including:  What parts of the testing will be outsourced and which will be in-sourced What level of agile methodology to be used in the various test teams for the project Document reporting requirements for the different teams Decide with vendors what level of testing will be done at the vendors and what reporting is required Develop a field trial strategy for the product/project in conjunction with field trial manager Develop a QA communications architecture for the project ­ what meetings are required, frequency and attendance What level of automated testing will be exploited at what stages in the project Develop and enforce agreed quality standards for the different phases of the project. Agree acceptance criteria for the different gates. Review test plans per QA workstream, with the aim of avoiding duplication and promoting re-use Sign-off of Test Plan by Product Owner Ensure that product team provide relevant training to the QA teams Ensure relevant training is made available on any new methodologies/tools to be used for the duration of the program Facilitate cross team resourcing if necessary to make the QA effort more efficient Minimise duplication of testing and ensure adequate test coverage (fill the gaps) Interface with product managers to understand product priorities  Responsible for Defect Metrics & Overall Defect Management Reporting to the Programme Manager & other Stakeholders
Customer Requirements	Assessing customer requirements and ensuring that these are met, communicated and understood internally Obtain customer feedback regarding the quality of software after each new software release into the market.

 

OVERVIEW OF COMPETENCIES

The role will be evaluated on performance w.r.t. general definitions of personal behaviours:

• Result Oriented
  
  • Is directed towards goals and objectives. Strives for effectiveness and excellence. Shows commitment and loyalty to [THE COMPANY].
    
• Reactivity / Proactivity
  
  • The ability to act independently and responsibly out of personal commitment in anticipating future events.
    
• Creativity & Innovation
  
  • The ability to suggest and advance ideas. The ability to challenge and improve existing solutions by thinking outside the box.
    
• Customer Service
  
  • The ability to provide courteous and professional response to customer through attentiveness and empathy.
    
• Flexibility & Adaptability
  
  • The ability to show adapt to new situations and move on effectively from one task to another. The ability to analyse situations through openness to other points of view.
    
• Analysis & Conceptualization
  
  • The ability to analyse information and/or situations and reach conclusions through systematic and methodical thinking. The ability to stand back from an event and focus on the essential in order to establish a direction or priorities.
    
• Rigour
  
  • Works in an accurate and efficient manner while paying close attention to details and complies with defined processes and procedures.
    
• Working with Others
  
  • Ability to work with others to accomplish together a common goal.
    
• Ability to Convince
  
  • The ability to influence and persuade others through well-developed reasoning, and appropriate communication channels and style.
    
• Team Management & Motivation
  
  • Leads and motivates people individually and collectively as a team to achieve their goals. Supports the learning, development and discipline of people through guidance and mentoring.
    
  • Management of staff (Direct reports)
    
  • Effective communication
    
  • Planning and organising
    
   

Core Experience

• Minimum EIGHT (years) traceable record of working in a similar role in the past
  

Education

• University Degree or Higher Diploma (in a technology-related field) desirable
• Software Engineering education preferable

Functional

• Practical understanding of digital TV broadcast and receiver technologies.
• Minimum 8 years experience in a consumer electronic or software quality management environment.
• Set-Top-Box experience very desirable.
• Exposure to software development processes would be beneficial.
• Relevant engineering or IT qualification required.
• Understanding of systems thinking and sub-system integration.
• Experience in using testing tools.
• IT Infrastructure and Networks knowledge would be advantageous.
• Test automation skills would be useful
• Project management skills desirable
• Change Management & Experience in driving process improvements desirable
• Team driven & working through consensus is desirable