Load Testing Scenarios

SCENARIO #1: DISCOVERY OF CAPACITY 

Discovering the limitations of your web application

In this case, the goal is to subject the application to ever increasing amounts of load in order to determine the point at which response times become unacceptable or the application begins to generate errors, and ultimately if there is a point at which the application fails entirely.

The profile is a diagonal line to and through the number of users anticipated during your highest peak usage period. Because your goal here is discovery of actual capacity rather than confirmation of a particular load level, you want to “overshoot” your anticipated peak by a healthy margin.

In the example above, you may have previously experienced 7,500 concurrent users and be anticipating as many as 10,000 this year. Create a test that ramps gradually to a peak of 15,000 concurrent users. When the test is complete, you will know if tuning or enhancements are required to reliably meet your goal plus a safety margin.

SCENARIO #2: OBSERVATION UNDER STRESS 

Taking your application to a known trouble point to determine what needs tuning.

In this case, the approximate load level where the application begins to degrade or throw errors is known and the goal is to re-create those conditions so a team of application specialists and systems administrators can observe in detail what is going wrong.

In a brief observation test, the goal may be merely to capture information for later study. In an extended observation test, the team may choose to make changes mid-test and observe the impact live.

This form of test may be repeated numerous times or combined with new discovery tests as your teams work through and eliminate bottlenecks in your application logic or configuration.

The profile is a sharp rise to the known load level which is then held during the remainder of the test.

SCENARIO #3: ENDURANCE TEST 

Above-average load for an extended period of time

Endurance testing is designed to hold the application at an above average level of usage for a prolonged period of time. Rather than taking the application to a known trouble area, the goal here is to take it to a point below that but run it long enough to discover if there are not-yet-known symptoms of failure which only appear over time.

In this case, multiple sets of 120-minute load tests are scheduled back-to-back, with an initial fast-ramp to the “above average” peak load and then subsequent 120-minute periods with no ramp-up at all.

SCENARIO #4: CONFIRMATION OF CAPACITY 

Ensuring that your system performs as expected after changes.

Any time your application is modified or the hosting or networking configuration has been updated, it is wise to confirm that your system still performs as expected. In the quality assurance world, this type of test might be referred to as “regression testing” (did I break anything in my application when I made my latest changes?) but we aren’t exhaustively re-testing functionality here. Instead we are confirming that performance criteria (page load times, total time to perform a user scenario, absence of server errors) continue to be met after changes have been performed.

Here the profile is to gradually ramp-up to the previously tested acceptable peak load point and run a test for at least 10 minutes at that level, and then to compare results to previous tests.