We know speed to market matters—now more than ever. Bill Gates said in the nineties that “if you don’t meet customer demand quickly enough, without sacrificing quality, someone else will.”
But software quality also matters, affecting everything from customer loyalty to developer experience. Poor quality causes immediate issues like bugs and crashes, as well as building technical debt and making updates needlessly hard. This can create risks to people’s health and even their lives, as we saw in 2018 when the UK’s NHS suffered a glitch caused by poor software that led to 10,000 patients potentially being given the wrong medication.
So with speed and quality both crucial to software development, what can we do? How can we maintain high quality code while lapping out competitors? This blog will introduce you to Quality Engineering (QE), a practice for integrating quality across the development cycle without sacrificing velocity.
What is Quality Engineering?
Quality Engineering is a systematic approach to building software that embeds quality into every step of software development. It blends production engineering, quality systems and continuous improvement to deeply weave quality into the software development lifecycle (SDLC).
Quality Engineering is not a philosophy or an aspiration: it is a pragmatic approach that makes software quality a goal and a responsibility for everyone in your organization. Where quality control or quality assurance treat quality as a final step before deployment, QE embeds quality considerations from the initial stages of design and planning through to development, deployment, and maintenance. This shift ensures that quality is not just an endpoint but a foundational aspect of the software development process.
Getting quality right means drawing from established methodologies such as:
- Shift-left: Incorporating testing early in the development process to catch defects sooner (more on this below).
- DevOps and DevSecOps: Integrating development, operations, and security to foster collaboration and improve quality and security.
- Continuous Integration and Continuous Deployment (CI/CD): Automating the integration and deployment processes to ensure frequent and reliable code changes to production.
- Software Development Engineer in Test (SDET): Combining development and testing responsibilities to empower certain developers to drive QE while contributing to the codebase.
- Test Driven Development (TDD) and Behaviour Driven Development (BDD): Ensuring development is closely aligned with user requirements and expectations through iterative testing.
It also incorporates tools and tactics, including:
- Automation: Streamlining repetitive tasks to reduce errors and increase efficiency.
- Testing: Systematically identifying potential risks throughout the SDLC.
- AI: Leveraging artificial intelligence to enhance testing and predictive analytics.
Quality Engineering is not defined by individual tests or outcomes: it takes a holistic approach to quality that transforms disparate actions into decentralized standards. Pivoting an organization to Quality Engineering is not easy, but getting it right means reduced costs, improved user experience and increasing the all-important velocity.
Components of Quality Engineering
Before committing to Quality Engineering it is important to have an established definition of software quality backed by clear metrics. This should include performance, reliability, durability, usability, maintainability, speed and other more tailored KPIs. It refers to product quality, the external attributes associated with the final product, as well as quality management throughout the production process.
When it comes to Quality Engineering there are we aspects of software quality to consider, functional quality and structural quality.
Functional Quality
Functional quality refers to how well a software product performs its specified functions under stated conditions. In the context of Quality Engineering, we ensure functional quality through comprehensive testing strategies like Test-Driven Development (TDD) and Behavior-Driven Development (BDD). These approaches involve writing tests before code to define functionality clearly and ensure that the software behaves as expected.
Structural Quality
Structural quality, on the other hand, focuses on the non-functional aspects of the software – characteristics such as architecture, code cleanliness, and system design. Ensuring structural quality with Quality Engineering involves using code reviews, refactoring, and static code analysis tools. These help to identify potential issues in the codebase and architecture early in the development cycle.
Implementing quality engineering means proactively revising every stage of the SDLC to embed quality principles. Some of these principles and characteristics include the following:
- Design for quality: this involves integrating quality standards and user needs into the design process, ensuring that the final product meets the desired quality benchmarks. This foresight minimizes the risk of defects and enhances the product's usability and performance.
- Testing: software testing is simple to understand and hard to implement comprehensively. It involving rigorous evaluation of the product throughout the development cycle. This goes beyond functional testing to include performance, security, and usability testing.
- Process optimization: this focuses on refining the development process itself by identifying bottlenecks, redundancies, or inefficiencies. It is a continuous improvement philosophy that strives to make software development progressively more refined over time.
- Data-driven decision making: taking an evidence-based approach allows teams to make informed decisions that minimize guesswork. Quality engineering allocates resources by analyzing data sources, from response time to test coverage, and from velocity to time to market and net promoter scores.
- Customer Focus: at the core of QE is a strong focus on customers, and efforts to improve quality should be in service of surpassing customer expectations and maximising customer satisfaction. This provides data that drives engineering decisions throughout the company, and involves gathering user feedback, conducting usability testing, and keeping abreast of market trends.
Why is quality engineering important?
Committing to quality engineering is a statement of intent that elevates standards across the organization. When every stakeholder buys into the engineering process it fosters a collective sense of ownership, even pride, in driving process improvement at every level.
The benefits are both immediate, such as clearer expectations and sharper business processes, and sustained, including improved user experience and reduced technical debt.
These benefits include the following:
Cost reductions
By identifying and addressing defects early in the development process, companies can avoid the high costs associated with fixing bugs at later stages or after product release. Early detection through rigorous testing and quality control measures reduces rework and increases efficiency, thereby lowering overall production costs.
Improved UX
Much of quality engineering is built on problem-solving and building engineering processes that can guarantee an improved user experience. By thoroughly testing usability and incorporating user feedback into software engineering, QE helps ensure that the customer voice is heard at every stage of product development. This focus on customer needs and usability leads to better UX.
Faster releases
Quality Engineering streamlines the software production process, speeding up release time without compromising on quality. Because QE emphasizes efficiency through CI/CD, automation, and shift-left testing, it speeds up the time-to-market. This agility allows companies to respond to market demands more effectively and exploit opportunities quickly and effectively.
Improved security/resilience
By incorporating security testing and risk assessment as integral components of the quality assurance process, QE identifies vulnerabilities early, preventing potential breaches and ensuring the product’s reliability in the face of threats. This proactive approach to security increases test coverage and helps build more robust and resilient software that users can trust.
Improved developer experience/culture
By embedding quality improvement throughout the development process, developers can work more efficiently, with fewer disruptions from late-stage defects. Efficient developers are happy developers with low cognitive load and more time spent on deep work. As you measure your engineering culture you should find that improvements in quality correlate to improvements in morale and mood, creating a virtuous cycle.
Quality engineering vs. quality control vs. quality assurance
At this point skeptics may say that quality engineering is just a rebrand of quality control or quality assurance. All are downstream of efforts to streamline manufacturing processes, and overlap with lean principles and statistical analysis. Quality engineering is an evolution of these principles, from discrete steps in an overall process to holistic approach that integrates quality in every step of the SDLC.
Let’s consider each in turn, addressing differences and similarities.
Quality control
Quality Control (QC) refers to the process of identifying defects in finished products before they go to market. QC activities typically occur at the end of the SDLC, where products are tested for compliance with requirements or standards. The primary focus is on defect detection and correction.
While QC is reactive and focused on fixing existing defects, QE is proactive, embedding quality considerations across the SDLC and even before it has begun. It is a subset of quality assurance, focusing specifically on verifying quality.
Quality assurance
Quality Assurance (QA) is a process-oriented approach to preventing defects in products by refining the development and production process . It focuses on ensuring the quality of the processes across the SDLC through processes and methodologies that prevent defects.
Where QA’s focus is on process improvements, QE addresses this and extends it to incorporating tools, automation, and practices like shift-left testing to ensure quality at every stage in the SDLC.
How to measure quality engineering
Like any engineering process, QE uses data to drive project management. It does this through continuous improvement, and best practice engineering metrics are central to this. While true QE involves looking at every part of the SDLC from first principles, here are a few metrics that can kickstart your quality management system.
Defect metrics
This refers to the number of defects found during testing, the severity of defects, and the time it takes to resolve them. High severity defects or a large number of defects indicates issues in the development process that need to be addressed. Monitoring the time to resolve defects also helps in assessing the responsiveness and efficiency of the development team.
Coverage
Coverage metrics, particularly code coverage and test coverage, indicate the extent to which the source code of the application is executed during testing. This helps in identifying untested parts of the codebase, ensuring that critical functionalities are tested and reducing the risk of defects in production. Higher coverage typically implies a lower likelihood of undetected issues, demonstrating a robust product.
Performance and release metrics
These metrics focus on the performance aspects of the application, such as load times, response times, and resource utilization, alongside the frequency, speed, and stability of releases. They are crucial for assessing not only the user experience but also the agility and reliability of the deployment process.
Productivity metrics
Productivity metrics measure the output of the development team, often through indicators like features developed per time unit, commits per day, or story points completed in a sprint. These metrics are valuable for assessing how effectively the team is delivering new functionalities and making progress on the product roadmap.
Efficiency Metrics
Efficiency metrics evaluate how efficiently resources are utilized within the development process. Examples include the ratio of successful builds to total builds or the amount of rework required. High efficiency indicates a streamlined development process, where resources are being used effectively, leading to cost savings and faster time-to-market.
Reliability Metrics
Reliability metrics assess the stability and dependability of a software product over time. This can include uptime, mean time between failures (MTBF), and mean time to recover (MTTR). Reliable software enhances user trust and satisfaction, directly impacting customer retention and brand reputation.
Over time, improvement in these metrics will lead to concrete business outcomes, such as:
- Product Sales Volume: Enhancements in quality and reliability mean better products that command higher sales volumes, as customers prefer dependable products.
- Customer Satisfaction (NPS): Better product design makes for happier customers on the back of improved performance, usability, and reliability.
- Support Request Volume and Type: Higher quality outputs through better processes reduce the number and urgency of support requests, which helps companies to scale while staying lean.
Best practice tips for adopting quality engineering
We have seen the value of speed and quality when building software, so adopting quality engineering sounds like a no-brainer. Yet we also described how this involves analyzing, and sometimes overhauling, every aspect of the SDLC.
Challenges for those adopting QE including communicating and implementing new processes, adjusting culture to prioritize quality at every touchpoint and handling the learning curve that accompanies widespread change.
To help you address these challenges, we recommend the following:
Shift Left
"Shifting left" refers to integrating quality and testing early into the development process, rather than waiting until after a product is completed. Integrating these quality measures earlier in the process helps emphasise quality as a top priority. Embedding these checks and test from the beginning also means that teams can reduce rework and accelerate development cycles, improving the overall quality of the product.
Collaborate with Other Teams
Quality is an output of a highly effective system, and when building software this requires effective collaboration between teams. While some components of software quality are decentralized within teams, others require communication, such as between developers and operations or site reliability engineers and incident response. Quality engineering is a holistic approach, and so requires widespread buy-in and collaboration.
Take a Data-Driven Approach
What gets measured gets managed. Tracking key performance indicators and using them to guide improvements ensures that efforts to enhance quality are focused where they will have the most significant impact. This approach helps teams prioritize issues, allocate resources efficiently, and continuously refine their processes for better results.
Train Employees
Training programs should cover not only the tools and technologies that support QE but also the principles and mindset that underpin it. Make the time to ensure that every team member understands the importance of quality at every stage of development, and has the skills needed to contribute to QE initiatives.
Use an Internal Developer Portal
An internal developer portal acts as a centralized platform for resources, documentation, tools, and communication related to development. It makes for an ideal aid to QE by facilitating knowledge sharing, streamlining access to necessary tools, and enhancing collaboration across teams. Using an internal developer portal can improve the onboarding speed and execution of QE processes. For more insights into the benefits and components of an internal developer portal, consider exploring this article.
How can Cortex help?
To address some of the challenges in quality engineering, consider using an internal developer portal (IDP) like Cortex. Cortex, and other IDPs that offer always-on or live Scorecarding, now make it possible to up-level the production readiness checklist and make it much more dynamic, responsive, and automated. Scorecards and Eng Intelligence can help your team gain insights into quality metrics like code coverage and service uptime to help find initial areas for quality improvement. Schedule a demo today to learn more about how Cortex can unlock quality engineering in your organization.
