Understanding Application Environments for Good QA

Critical in the software development lifecycle is the presence and maintenance of application environments - the locations where developers do their programming, testers perform their software quality assurance, and members of the wider team can access the latest version of the app for their own needs:

• Developers need a place to code and check their work against the rest of the app.
• QA testers need an environment to test both new features and regress existing functionality.
• Management and other staff need access for reviews, marketing info, and documentation.

An environment includes both the physical hardware and software components to allow the application under development to run properly, as well as the application itself. Environments are typically a self-contained cluster of servers isolated from other environments (or "sandboxed") ensuring the code is in a known state for further work.

Let's look at the typical array of application environments utilized during application development:

• Local
• Development
• QA
• Staging/ Pre-Production
• Production

Local Environment  

Developers create and debug their initial code on their local machines. To facilitate their development and unit testing, developers may use stubs to simulate other pieces of software like a database or web service. The local environment is under the full control of the individual developer because:

• It's easy to quickly update, test, and debug code.
• They can experiment without affecting the work of other developers.

For instance, consider an e-commerce application. The application interacts with web services like a payment gateway to verify a credit card. In a local environment a developer can write the code for the web application against the stubbed credit verification software to facilitate coding as well as unit testing on their local machine.

Development Environment 

Just as the local environment is for the individual developer to program and debug locally, the "development" environment is the shared location for all developers to check their code against all the changes in that release. This is known as integration testing and a common practice nowadays is to automate these tests such that after code is updated to the repository, the application automatically builds and (assuming no errors upon building) the integration tests automatically executed.

One thought to reduce the number of environments is to have the QA engineers use the dev environment for testing. This is a bad idea because:

• QA testing requires a known state of code and a stable set of test data in order to communicate issues effectively to the developers. Since the dev environment changes with every build, it invalidates the testing already performed since the bugs logged may now be fixed or new bugs introduced.

• Shared services (ie. integrations to external services or APIs) are likely still stubbed out in development, whereas in QA they must be connected in order to adequately test the real-life situation.

Continuing with our example of the e-commerce application, the structure of the development environment would be similar to that of the local environment except that the versions of all components would be the latest “tip” version.

QA Environment

Stable versions of the build are then released to the QA environment for thorough functional, integration, regression, and other types of testing. This environment can be further managed by splitting it into two separate environments, QA and Pre-Production, for specific purposes. Note that these terms (QA, Staging, and Pre-prod) are fairly fluid and may be used interchangeably on different teams. 

The bulk of the application testing effort occurs in the QA environment, and the bugs logged refer to issues found in this environment. It is an important step on the path towards the software release readiness, and a separate and controlled environment is critical because:

• • The environment contains a known codebase of the entire app. When a complete test run is performed on a specific build and no high severity issues found, the team has a high assurance that this build is ready to release.

• • The environment is under strict control, so that no code can be modified without going through proper procedure. This adds to the assurance that when released there will be no surprises. 

• Test data can be a lot of work to setup, and often times require the latest data from production which requires time-consuming backups. A stable and separate QA environment decreases the fluctuation of this test data.
• As referred to previously, the QA server has active connections to 3rd party interfaces (eCommerce gateways, APIs, other web services) so that accurate and complete test results can be gathered from every place the application interacts with other systems.  

In our continuing e-commerce application example, the QA team tests it with the web services and the database.

Staging/ Pre-Production Environment

As the application moves towards greater stability, testing is conducted in the Pre-Production environment where the infrastructure is as identical to Production as possible. Having this separate environment is useful for the following reasons:

• • Load and/or stress testing is performed where the application is pushed to its limits. This is done with the help of automation scripts and tools running scenarios with steadily increasing load to the point where the application breaks. Where and when these failures occur expose weaknesses and bottlenecks in the logic, design, and infrastructure of the application.

• • Additional testing like security and user acceptance testing can be done in preprod where it does not get int he way of any work being done by testers in the QA environment.

• Additional functional tests such as exploratory or an automated regression run can uncover missed bugs.

Production Environment

The Production environment is where the users finally get to enjoy the latest released code and enhanced functionality, such as our e-commerce application with web services and databases, and integrations with external services like credit verification.

It goes without saying the production environment must work smoothly and without issues in order to provide a pleasant end-user experience. While successive builds of the application are being tested in dev and QA, the app continues to run unaffected in production.

What's next for application environments?

It should be obvious that having a clear and concise set of environments is critical for an efficient and successful software development lifecycle, from development, through QA, and into production. Getting each environment up and running and the handoff processes from one environment to another is critical to a well-run QA process. For a limited time, OnPath Testing is offering a complimentary QA Audit to examine your processes, and I encourage you to contact me at brian@onpathtesting.com to learn more and ensure your testing is on the right path.