Software Testing Methodology

AlexQin

What's System-Level Test
System-level tests consists of batteris of tests that are designed to fully exercise a program as a whole and check that all elements of the integrated system function properly.

Functional System Testing
System tests check that the software functions properly from end-to-end. The components of the system include: A database, Web-enable application software modules, Web servers, Web-enabled application frameworks deploy Web browser software, TCP/IP networking routers, media servers to stream audio and video, and messaging services for email.
A common mistake of test professionals is to believe that they are conducting system tests while they are actually testing a single component of the system. For example, checking that the Web server returns a page is not a system test if the page contains only a static HTML page.
System testing is the process of testing an integrated hardware and software system to verify that the system meets its specified requirements. It verifies proper execution of the entire set of application components including interfaces to other applications. Project teams of developers and test analysts are responsible for ensuring that this level of testing is performed.
System testing checklist include question about:

• Functional completeness of the system or the add-on module
• Runtime behavior on various operating system or different hardware configurantions.
• Installability and configurability on various systems
• Capacity limitation (maximum file size, number of records, maximum number of concurrent users, etc.)
• Behavior in response to problems in the programming environment (system crash, unavailable network, full hard-disk, printer not ready)
• Protection against unauthorized access to data and programs.
  

AlexQin

What's "black-box" (or functional) testing
Black Box Testing is testing without knowledge of the internal workings of the item being tested. The Outside world comes into contact with the test items, --only through the application interface ,,, an internal module interface, or the INPUT/OUTPUT description of a batch process. They check whether interface definitions are adhered to in all situation and whether the product conform to all fixed requirements. Test cases are created based on the task descriptions.
Black Box Testing assumes that the tester does not know anything about the application that is going to be tested. The tester needs to understand what the program should do, and this is achieved through the business requirements and meeting and talking with users.
Funcional tests: This type of tests will evaluate a specific operating condition using inputs and validating results. Functional tests are designed to test boundaries. A combination of correst and incorrect data should be used in this type of test.

AlexQin

What's Scalability and Performance Testing
Scalability and performance testing is the way to understand how the system will handle the load cause by many concurrent users. In a Web environment concurrent use is measured as simply the number of users making requests at the same time.
Performance testing is designed to measure how quickly the program completes a given task. The primary objective is to determine whether the processing speed is acceptable in all parts of the program. If explicit requirements specify program performance, then performance test are often performed as acceptance tests.
As a rule, performance tests are easy to automate. This makes sense above all when you want to make a performance comparison of different system conditions while using the user interface. The capture and automatic replay of user actions during testing eliminates variations in response times.
This type of test should be designed to verify response and excution time. Bottlenecks in a system are generally found during this stage of testing.

AlexQin

What's Stress Testing
Overwhelm the product for performance, reliability, and efficiency assessment; To find the breakpoint when system is failure; to increase load regressively to gather information for finding out maximum concurrent users.

Stress tests force programs to operate under limited resource conditions. The goal is to push the upper functional limits of a program to ensure that it can function correctly and handle error conditions gracefully. Examples of resources that may be artificially manipulated to create stressful conditions include memory, disk space, and network bandwidth. If other memory-oriented tests are also planned, they should be performed here as part of the stress test suite. Stress tests can be automated.
Breakpoint:

the capabilites and weakness of the product:

High volunmes of data

Device connections

Long transation chains

Stress Test Environment:
As you set up your testing environment for a stress test, you need to make sure you can answer the following questions:

• Will my test be able to support all the users and still maintain performance?
• Will my test be able to simulate the number of transactions that pass through in a matter of hours?
• Will my test be able to uncover whether the system will break?
• Will my server crash if the load continues over and over?

The test should be set up so that you can simulate the load; for example:

• If you have a remote Web site you should be able to monitor up to four Web sites or URLs.
• There should be a way to monitor the load intervals.
• The load test should be able to simulate the SSL (Secure Server)
• The test should be able to simulate when a user submits the Form Data (GET method)
• The test should be set up to simulate and authentical the keyword verification.
• The test should be able to simulate up to six email or pager mail addresses and an alert should occur when there is a failure.

It is important to remember when stressing your Web site to give a certain number of users a page to stress test and give them a certain amount of time in which to run the test.
Some of the key data features that can help you measure this type of stress test, determine the load, and uncover bottlenecks in the system are:

• Amount of memory available and used
• The processor time used
• The number of requests per second
• The amount of time it takes ASP pages to be set up.
• Server timing errors.

AlexQin

What's Load Testing
The process of modeling application usage conditions and performing them against the application and system under test, to analyze the application and system and determine capacity, throughout speed, transation handling capabilities, scalabilities and reliability while under under stress.
This tyoe of test is designed to identify possible overloads to the system such as too many users signed on to the system, too many terminals on the network, and network system too slow.
Load testing a simulation of how a browser will respond to intense use by many individuals. The Web sessions can be recorded live and set up so that the test can be run during peak times and also during slow times. The following are two different types of load tests:
Single session - A single session should be set up on browser that will have one or multiple responses. The timeing of the data should be put in a file. After the test, you can set up a separate file for report analysis.
Multiple session - a multiple session should be developed on multiple browsers with one or multiple responses. The multivariate statistical methods may be needed for a complex but general performance model
When performing stress testing, looping transactions back on themselves so that the system stresses itself simulates stress loads and may be useful for finding synchronization problems and timing bugs, Web priority problems, memory bugs, and Windows problems using API. For example, you may want ot simulate an incoming message that is then put out on a looped-back line; this in turn will generate another incoming message. The nyou can use another system of comparable size to create the stress load.
Memory leaks are often found under stress testing. A memory leak occurs when a test leaves allocated memory behind and does not correctly return the memory to the memory allocation scheme. The test seems to run correctly, but after several iteration available memory is reduced until the system fails.

Peak Load and Testing Parameters:
Determining your peak load is important before beginning the assessment of the Web site test. It may mean more than just using user requests per second to stress the system. There should be a combination of determinants such as requests per second , processor time, and memory usage. There is also the consideration of the type of information that is on your Web page from graphics and code processing, such as scripts, to ASP pages. Then it is important to determine what is fast and what is slow for your system. The type of connection can be a critical component here, such as T1 or T3 versus a modem hookup. After you have selected your threshold, you can stress your system to additional limits.
As a tester you need to set up test parameters to make sure you can log the number of users coming into and leaving the test. This should be started in a small way and steadily increased. The test should also begin by selecting a test page that may not have a large amount of graphics and steadily increasing the complexity of the test by increasing the number of graphics and image requests. Keep in mind that images will take up additional bandwidth and resources on the server but do not really have a large impact on the server's processor.
Another important item to remember is that you need to account for the length of time the user will spend surfing each page. As you test, you should set up a log to determine the approximate time spend on each page, whether it is 25 or 30 seconds. It may be recorded that each user spends at least 30 seconds on each page, and that will produce a heightened response for the server. As the request is queued, and this will be analyzed as the test continues.

AlexQin

AlexQin

AlexQin

AlexQin

What's Load/Volume Test

Load/volume tests study how a program handles large amounts of data, excessive calculations, and excessive processing. These tests do not necessarily have to push or exceed upper functional limits. Load/volume tests can, and usually must, be automated.

Focus of Load/Volume Tesing

• Pushing through large amounts of data with extreme processing demands.
• Requesting many processes simulateously.
• Repeating tasks over a long period of time

Load/volume tests, which involve extreme conditions, are normally run after the execution of feature-level tests, which prove that a program functions correctly under normal conditions.

AlexQin

Difference between Load and Strees testing
The idea of stress testing is to find the breaking point in order to find bugs that will make that break potentially harmful. Load testing is merely testing at the highest transaction arrival rate in performance testing to see the resource contention, database locks etc..