How to maximize AI efficiency in QA without compromising cost or quality? Sii Poland announces the second edition of Testing Lab

Artificial intelligence has redefined performance standards in Quality Assurance. The inaugural March edition of our research experiment, Testing Lab – AI Edition, served as a strategic opening, providing the industry with hard data: QA teams backed by LLMs delivered up to twenty times more automated tests than engineers relying on traditional methods. Yet, this unprecedented surge in productivity is merely the first move on the board toward full transformation. Time for the next insights.

Deploying artificial intelligence at scale in the enterprise sector forces organizations to think several moves ahead. Scaling models introduces a new set of challenges: from the non-deterministic nature of algorithms and compliance risks, to exponentially rising infrastructure and token costs. In this landscape, every uncalculated move comes with a business price tag. The upcoming second edition of Testing Lab is our response to these challenges. This initiative will focus on standardizing AI automation in QA, helping tech leaders map out a winning strategy – one that perfectly balances efficiency, cost control, and top-tier software quality.

A data-driven foundation: Key takeaways from the first edition

The March study, involving 20 Sii Poland experts, yielded crucial insights that serve as the starting point for the next phase of the project:

• The scale of the productivity gap: While traditional teams implemented between 5 and 8 tests from scratch (greenfield), teams leveraging coding assistants delivered anywhere from 5 to nearly 200.
• Impact on code quality: An analysis based on 8 engineering criteria revealed that proper prompt structuring and iterative interaction with the model significantly improve the organization and diagnosability of the solution.
• The critical role of competencies: AI acted as a knowledge accelerator. The highest code stability and reproducibility were achieved by highly skilled teams capable of consciously steering the model.
• Identified risks: The experiment exposed the models’ vulnerability to logical errors and getting “stuck” in dead ends, particularly when generating dynamic selectors.

The goal of the second study: From theory to operational optimization

While the first installment of Testing Lab answered the question of “whether AI works in automation,” the current edition tackles the strategic question: how to use AI to maximize results without compromising code quality or inflating costs.

During the upcoming study, Sii Poland’s architects and engineers will stress-test advanced verification environments. The work will center around three core pillars:

• Context Engineering & Cost Management: Optimizing prompt structures and implementing token consumption control mechanisms to reduce the maintenance costs of LLM infrastructure.
• Reproducibility & Standardization: Ensuring predictable test implementations (generating stable results) and creating code based on consistent, predefined rules.
• Operational Efficiency: Scaling and maximizing the output of QA teams through the seamless synergy of human expertise and AI tools.

Our goal is to develop proven guidelines in the form of an Evidence-based AI Playbook for Software Testing, a ready-to-use set of strategic moves and best practices designed to systematize and optimize your approach to software quality.

Make the winning move: Get early access to the Report

Comprehensive technology descriptions, specifications of the developed architectures, and hard optimization data will be published in our second engineering report.

Don’t base your company’s QA strategy on mere market trends. Make decisions grounded in validated research data and stay several moves ahead of the competition.