QA’s role in XP

ThoughtWorks has been following ‘Extreme Programming (XP)’ practices ever since its beginning completing a grand 25 years! As a member of the TW family for close to 9 years now, I can fairly state that the key reason for TW’s consistent success is because the XP values and principles are well ingested in the day to day culture of the people here. Every single role Dev, QA, BA which forms the software development team has applied XP practices over and over in various environments, leading to success every single time.

XP Values and Principles:

To quote ‘some’ of the key differential principles of XP:

• Short development cycles, resulting in early and continuous feedbacks.
• Responding to changing business needs, by delivering business value as needed.
• Incremental planning approach, which evolves throughout the project lifecycle.
• Reliance on automated tests to catch defects early in the cycle.
• Close collaboration within team members

XP also defines values and practices that a software development team shall follow to achieve the above. Communication, feedback, simplicity, courage and respect are the values that enrich the team. Along with these values, practices like unit testing, refactoring, continuous delivery etc helps the team to achieve excellence in software development.

From my personal experience of following XP (and also taking inspiration from other QAs here in TW), here are my thoughts on how a ‘Quality Analyst’ carry the XP values and practices to achieve the one primary goal — A very high Quality Software!

QA in XP:

First of all, a QA in XP understands that ‘Quality is team’s responsibility’ and navigating the team towards a better quality at every stage of software development is their primary role. This is in itself is a huge shift in the mindset.

To better understand this shift,

• A QA pitches in with their domain and technical knowledge in figuring the right scope for the release along with the BA.
• Given a fair share of technical knowledge, they also pitch in solution design along with the devs.
• They write automated tests at various levels (unit, integration, UI etc) while making sure the team receives continuous feedbacks from them.
• They make sure quality CI / CD pipelines are setup which will yield faster feedbacks.

So as we can see there is a holistic approach towards quality throughout the project lifecycle.

Let us now look in detail about the QA practices in each phase of the software development.

Project Discovery / Inception / Planning:

During this phase, QAs along with the development team meets the business to understand their current problems, future directions, competitors, current tech solutions (if any). They assess their current test strategy (if any). With all this understanding the team comes up with a overall plan for execution.

XP is essentially light weight. A QA who is part of this exercise validates whether the releases are planned short enough with the critical / high priority features (sometimes when there are existing high priority defects, making sure that is included in the release) which will help the business efficiently compete with their competitors. They keenly explore the Cross Functional Requirements (CFRs) like Security, Performance, Legal Requirements, Monitoring, Usability, Browser, Device compatibility, Accessibility etc and makes sure they are rightly included in the release plan. They assess if the testing activities associated can be accomplished within the release timelines.

Most importantly QAs formulate a solid test strategy after understanding which features and CFRs means most to the business.

The test strategy typically covers the following:

1. Overall process towards testing. For ex, if the release focuses on responsiveness then they come up with a plan on how to test them, different stages of testing like story, feature, regression and identify most used browsers to formulate an optimised plan.
2. Tools to be used for efficient testing by the entire team.
3. Automated tests coverage to be given at different layers. Typically we try to achieve a concrete test pyramid.
4. A definitive plan for Continuous Deployment (CD). (Smoke tests, Regression tests, Feature Regression tests etc.)
5. A clear plan to test the CFRs and the automated tests coverage for them.
6. Collaboration plan with the existing testing team (if any) and the business for iteration showcases and sign offs to get frequent feedbacks.
7. A plan for sharing new knowledge for distributed QA teams.

Software Development:

This can be further broken down into two sets — the setup phase and the development phase.

I0 — The Setup Phase:

Once the team sets off to development, typically the first iteration i0 is the setup phase. Here the QA does the automation framework setup for UI tests, Integration tests etc and also for CFRs. They get their manual and automation testing environments ready (by themselves or pairing with someone). They have the tests integrated with the CI pipelines. Ideally the CI builds are expected to pass within 10 minutes so that the team takes the feedbacks seriously. The QAs will have to find the right way to fit in the 10 minute bucket even when the test suite grows.

They also share the knowledge to the team about these setup so that they can start looking at the right places for feedbacks. The team is also aware that their check-ins are not going to get deployed unless the build passes. QAs sometimes as well help the team in getting the build to pass.

Development Phase:

Iteration Planning Meeting (IPM):

This meeting is effectively used by the team to understand the business values they will be collectively producing for the current iteration (typically a week or two). The team goes over the precise details and agrees to deliver a set of scope by the end of it.

A QA in this discussion analyses if all possible business flows are covered in the details. Calls out critical regression cycles that is needed because of the current changes. They also raise risks up front if they foresee the current work breaking some of the existing flows. They think through the upcoming features flows and assess if the current flow wouldn’t need rework then.

They look at all the different types of testing activities needed for the iteration and gives a realistic picture of whether the scope can be delivered within the iteration time.

Kick- Offs:

When the devs pick up ‘stories’ for development, BA and QA are called for an initial ‘kick-off’ meeting. This is a flavor of incremental planning. Here the QA discusses all the test cases they have captured in the story card. This is also a place where QAs can figure out any additional scope creep that was not originally planned for the release.

There is also a fair amount of discussion about solution / implementation details using which the QAs prepare their automation tests in advance. For ex, if there is going to be a new service endpoint, they add respective service tests (before the dev work is done) and also call out the unit tests that need to be covered in the kick-off meeting.

Dev-Box Testing:

Once the story development is complete, again the BA and QA are called for dev-box testing. This is a way of making sure frequent and faster feedbacks are given. Also this helps the lifecycle of the fix (if any) navigate much faster as it can be implemented immediately in the dev box when the devs have the fresh context.

Needless to say that the QAs make sure all the Acceptance Criterias (ACs) work and all the test cases which they had prepared earlier works seamlessly. Some amount of integration testing is also carried on here. If there are any defects, the devs will work on it then and there. QAs makes sure that the respective unit tests are added at this stage.

Testing

This is the stage where the QA picks the story for a thorough exploratory and integration testing in the QA environment.

Continuous Integration and Deployment:

Ideally, a QA prepares the automated tests when the story is in development. So once the commit is made, the new tests are expected to pass and the green build gets deployed to the QA environment automatically. That way all the ACs are already tested and the QA in the testing phase can focus on exploratory testing. However this can only be partially achieved due to multiple reasons (for another blog).

In the CI, whenever a commit is made all the unit and integration tests run after which a few UI / functional tests called the “smoke” tests run before the build goes green. Once the build passes, it is deployed to the QA environment. It is only then a QA picks the story for testing.

Automated tests

The QA adds more automated tests at multiple layers either before or after the story development phase. They maintain the test pyramid while doing so. They integrate all of them with the CI. Some may choose to run only a few tests as part of every commit and others run all of the tests for every commit depending on the time taken. Even if only a selected few tests run for every commit, the rest of the tests are run during the night, called the ‘nightly regression’ (Though this is highly discouraged in TW). The results of which are analysed as the first thing in the morning by the QAs. That way, though there is a slight delay, the feedbacks are passed just as fast as possible.

Some may have facilities to run selected few features using feature tags for every commit based on the changes they perform. For example, if they are doing a specific change in shopping cart feature all tests related to shopping cart can be run as smoke tests instead of a predefined set.

Exploratory testing

Lastly the QAs does extensive exploratory testing before they call the work “done”. This is done at every level — story, feature as well as at release level. It is also a common practice here to do a “Bug Bash” where all team members does exploratory testing once a definitive set of feature flows are development complete.

Sign Offs

Once the testing is complete, the QAs takes the iteration results to the business for showcase and gets feedbacks. QAs use this opportunity to gain the confidence of the business folks by narrating all the details as possible and explains the quality aspect of the product.

This cycle of planning, development and testing goes iteratively for every release. Below is one such end to end testing cycle representation made for a mobile software development project.

To conclude, a QA communicates extensively with all the roles — BAs, Devs, Product Owners to keep the feedbacks flowing while giving a lot themselves. They possess the courage to challenge every team member in order to push them towards better quality. They establish the trust with the business folks with their ability to narrate the quality of the product in simple terms! Overall their unique mix of domain, technical and soft skills combined with the XP values and practices brings outs the best software quality!

Let us hear about other TW QAs’ experiences with XP.

Meet us at VodQA for more conversations!