Chapter 12.3: Program Testing & Maintenance

9618 AS Computer Science

📚 Learning Objectives

Show understanding of ways of exposing and avoiding faults in programs
Locate and identify different types of errors: syntax, logic, run-time errors
Show understanding of testing methods: dry run, walkthrough, white-box, black-box, integration, alpha, beta, acceptance, stub
Show understanding of the need for a test strategy and test plan
Choose appropriate test data: normal, abnormal, extreme/boundary
Show understanding of the need for continuing maintenance of a system
Understand differences between corrective, perfective, and adaptive maintenance
Analyse an existing program and make amendments to enhance functionality

🌟 Did You Know?

Research has shown that the earlier an error can be found, the cheaper it is to fix it. Testing software before release is essential, but testing throughout development is even more critical. The purpose of testing is to discover errors — program testing can show the presence of bugs, but never to show their absence!

1. Faults and Errors in Programs

Most programs written will contain errors, as programmers are human and do make mistakes. A program fault is something that makes a program not do what it is supposed to do under certain circumstances.

📖 Key Terms

Fault: Something that makes a program not perform as expected
Bug: A fault or error in a program
Debugging: The process of finding and correcting errors in a program
Patch: A small program released to correct an error or provide extra functionality

1.1 Fault Avoidance

Fault avoidance starts with provision of comprehensive and rigorous program specification at the end of analysis phase, followed by use of formal methods:

📝 Fault Prevention Methods

Analysis Phase: Comprehensive and rigorous program specification
Design Stage: Use of structure charts, state-transition diagrams, pseudocode
Coding Stage: Information hiding, encapsulation, exception handling
Testing Stage: Systematic testing to expose faults

⚠️ Important

Testing will show presence of faults to be corrected, but cannot guarantee that programs are fault-free under all circumstances. Faults can appear during the lifetime of a program and may be exposed during live running.

1.2 Why Errors Occur

Software may not perform as expected for a number of reasons:

Source of Error	Description
Programmer mistake	Coding errors made during development
Requirement specification	Specification not drawn up correctly
Design error	Software designer made a design error
User interface	Poorly designed interface causes user mistakes
Hardware failure	Computer hardware experiences failure

2. Syntax Errors

Syntax errors are errors in the grammar of a source program. A syntax error occurs when a program statement does not follow the rules of the programming language, for example, incorrect spelling of a keyword.

📖 What are Syntax Errors?

Errors that break the grammatical rules of a programming language
Program will not run until syntax errors are corrected
Detected during compilation phase
IDEs offer suggestions about what syntax errors are and how to correct them

2.1 Common Syntax Errors

Error Type	Example
Typos and spelling errors	PRINNT instead of PRINT
Missing or extra brackets	IF (x > 5 THEN (missing closing bracket)
Missing quotes	OUTPUT "Hello (missing closing quote)
Misplaced/missing semicolons	In languages that require them
Invalid variable names	Using reserved keywords as variable names
Incorrect use of operators	Using = instead of ← for assignment
Incorrectly nested loops	Mismatched IF/ENDIF, WHILE/ENDWHILE

❌ Pseudocode WITH Syntax Errors:

FUNCTION generate_username(first_name STRING, last_name STRING) RETURNS STRING
    DECLARE username : STRING
    username = SUBSTRING(first_name, 1, 1) + SUBSTRING(last_name, 1, 1)
    RETURN username
ENDFUNCTION

PROCEDURE main()
    OUTPUT "Enter your first name: "
    INPUT first_name
    username ← generate_username[first_name, last_name]  // Wrong brackets!
ENDPROCEDURE

CALL Main  // Wrong capitalization and missing ()

✅ Pseudocode WITHOUT Syntax Errors:

FUNCTION generate_username(first_name : STRING, last_name : STRING) RETURNS STRING
    DECLARE username : STRING
    username ← SUBSTRING(first_name, 1, 1) + SUBSTRING(last_name, 1, 1)
    RETURN username
ENDFUNCTION

PROCEDURE main()
    OUTPUT "Enter your first name: "
    INPUT first_name
    username ← generate_username(first_name, last_name)  // Correct!
ENDPROCEDURE

CALL main()  // Correct!

💡 Exam Tip

Syntax errors are usually detected by the compiler or interpreter before the program runs. If your program won't even start, check for syntax errors first!

3. Logic Errors

Logic errors are errors in the logic of a program, meaning the program does not do what it is supposed to do. The program runs, but produces incorrect output or results. These errors are usually found when the program is being tested.

📖 What are Logic Errors?

Incorrect code that causes program to run but produce wrong results
Program compiles and runs without crashing
Usually found during testing phase
Can be difficult to identify by the person who wrote the program
Use trace tables to help find logic errors

3.1 Common Logic Errors

Error Type	Example	Correct Version
Wrong operator	IF x > 0 OR y > 0	IF x > 0 AND y > 0
Wrong arithmetic	area ← width - length	area ← length * width
Loop one extra time	FOR i = 0 TO 10	FOR i = 0 TO 9
Array index off by one	Accessing index 5 in array[0..4]	Access index 0 to 4 only
Using uninitialized variables	Using count before setting to 0	Initialize: count ← 0
Infinite loops	WHILE x > 0 (x never changes)	Ensure x changes in loop

❌ Pseudocode WITH Logic Errors:

FUNCTION calculate_area(length : REAL, width : REAL) RETURNS REAL
    // Checks for valid dimensions and calculates area
    IF length > 0 OR width > 0 THEN  // Wrong: allows negatives!
        DECLARE area : REAL
        area ← width - length  // Wrong: subtraction instead of multiplication!
        RETURN area
    ELSE
        OUTPUT "Length and width must be positive values."
    ENDIF
ENDFUNCTION

✅ Pseudocode WITHOUT Logic Errors:

FUNCTION calculate_area(length : REAL, width : REAL) RETURNS REAL
    // Checks for valid dimensions and calculates area
    IF length > 0 AND width > 0 THEN  // Correct: both must be positive
        DECLARE area : REAL
        area ← length * width  // Correct: multiplication for area
        RETURN area
    ELSE
        OUTPUT "Length and width must be positive values."
    ENDIF
ENDFUNCTION

📝 How to Find Logic Errors

Use trace tables to dry-run the program step by step
IDEs allow us to single-step through a program
Check values of each variable as they change
Compare expected output with actual output
Test with known input values and verify results

4. Run-Time Errors

Run-time errors happen when a program executes an invalid instruction, encounters an out of bounds error, or attempts to divide by zero. The program may halt unexpectedly or go into an infinite loop.

📖 What are Run-Time Errors?

Errors that cause a program to crash during execution
Program compiles successfully but fails during runtime
May be managed in IDE with suitable error message
If program already released, developer should be informed
A patch may be released to correct the error

4.1 Common Run-Time Errors

Error Type	Example	Prevention
Division by zero	result ← x / 0	Check divisor ≠ 0 before division
Array index out of bounds	Accessing array[10] when array is size 5	Validate index before access
File not found	Opening non-existent file	Check file exists before opening
Memory overflow	Infinite recursion	Set recursion limits
Type mismatch	Adding string to number	Validate data types
Null pointer reference	Accessing uninitialized object	Check for null before access

❌ Pseudocode WITH Run-Time Error:

FUNCTION divide_numbers(a : REAL, b : REAL) RETURNS REAL
    DECLARE result : REAL
    result ← a / b  // Will crash if b = 0!
    RETURN result
ENDFUNCTION

// Calling with b = 0 causes run-time error
answer ← divide_numbers(10, 0)  // CRASH!

✅ Pseudocode WITH Exception Handling:

FUNCTION divide_numbers(a : REAL, b : REAL) RETURNS REAL
    DECLARE result : REAL
    IF b = 0 THEN
        OUTPUT "Error: Cannot divide by zero!"
        RETURN -1  // Return error code
    ELSE
        result ← a / b
        RETURN result
    ENDIF
ENDFUNCTION

// Safe calling
answer ← divide_numbers(10, 0)  // Returns -1 with error message

💡 Exam Tip

The key difference: Syntax errors prevent the program from running, Logic errors allow program to run but produce wrong results, Run-time errors cause program to crash during execution.

5. Testing Methods

Programs need to be tested before they are released. Tests begin from the moment they are written; they should be documented to show that the program is robust and ready for general use.

📖 Key Principle

The purpose of testing is to discover errors. Program testing can be used to show the presence of bugs, but never to show their absence. Testing takes place during the testing stage of the program development life cycle.

5.1 Dry Run / Walkthrough

A dry run (also called a walkthrough) involves manually tracing through the code on paper to predict output and track variables. This is done using a trace table.

📝 How to Dry Run

Write down current contents of all variables
Track conditional values at each step
Use different test data to verify algorithm
Compare expected output with actual output

5.2 Stub Testing

Stub testing is employed during development of modular programs, conducted before all modules have been fully implemented.

📖 Stub Testing Features

A dummy module (stub) is created to simulate or replace an actual module
The stub typically contains an output statement or returns a fixed value
Indicates that the call to the module has been made
Allows testing of incomplete programs

5.3 White-Box Testing

White-box testing tests the internal logic and code structure. The tester knows how the program works and chooses suitable test data that checks every path through the code.

Example: If a program has multiple IF statements, white-box testing ensures every branch (both TRUE and FALSE paths) is tested at least once.

5.4 Black-Box Testing

Black-box testing focuses on functionality of software without considering its internal code structure. Testers do not have access to internal workings.

📖 Black-Box Testing Features

Test cases designed based on specifications and requirements
Tests inputs and expected outputs
No knowledge of how software achieves its results
Primary goal: ensure software functions correctly from user's perspective
Helps identify discrepancies between expected and actual behavior

5. Testing Methods (Continued)

5.5 Integration Testing

Software consists of many modules written by different programmers. Integration testing tests that modules work correctly when joined together into one program.

📖 Integration Testing

Each individual module might have passed all tests
When modules are joined, they must be tested together
Usually done incrementally
Tests interfaces between modules

5.6 Alpha Testing

Alpha testing is performed in-house by software testers before being released to customers. This is the first phase of user acceptance testing.

5.7 Beta Testing

Beta testing is carried out by external users in real-world environments. When software is produced for general sale, there is no specific customer to perform acceptance testing.

📝 Beta Testing Process

After alpha testing, a version is released to limited audience
These beta testers use software in their own environments
The early release version is called beta version
Users feedback any problems to software house
Software house corrects reported faults before final release

5.8 Acceptance Testing

Acceptance testing is the final check to ensure the program meets the original client requirements. For bespoke software, this is performed by the customer.

📖 Acceptance Testing

Bespoke software released to customer for checking
Customer checks it meets requirements and works as expected
Generally part of hand-over process
On successful acceptance testing, customer signs off software

Testing Method	Who Performs It	Purpose
Dry Run	Programmer	Trace through code manually with trace tables
White-box	Developer	Test internal logic, every path through code
Black-box	Tester	Test functionality without seeing code
Integration	Developer	Test modules work together correctly
Alpha	In-house testers	Early testing before customer release
Beta	External users	Real-world testing by selected users
Acceptance	Customer	Final check before sign-off
Stub	Developer	Test incomplete programs with dummy modules

6. Test Data and Test Plans

6.1 Test Strategy and Test Plan

During the design stage of a software project, a suitable testing strategy must be worked out. We need a test plan to ensure testing of software from the very beginning.

📖 Test Plan Considerations

Flow of Control: Does user get appropriate choices? Does chosen option go to correct module?
Validation of Input: Has all data been entered correctly?
Loops and Decisions: Do they perform correctly?
Data Storage: Is data saved into correct files?
Results: Does system produce correct results?
Test Data: Select data that allows us to see if it's handled correctly

6.2 Types of Test Data

Type	Description	Example (Age 12-18)
Normal (Valid)	Typical data values that are valid	Age = 14, 16
Abnormal (Erroneous)	Data values that system should not accept (wrong type)	Age = "H", "@", "twenty"
Extreme	Maximum and minimum values of normal data accepted	Age = 12, 18
Boundary	Values on either side of max/min (largest/smallest unacceptable)	Age = 11, 19

Example Program: A program accepts user age between 12 and 18 inclusive.
Normal: 14, 16 (accepted) | Extreme: 12, 18 (at boundaries, accepted)
Boundary: 11, 19 (just outside, rejected) | Abnormal: "H", "@" (wrong type, rejected)

💡 Exam Tip

Remember: Extreme data is AT the boundary (still valid), while Boundary data is JUST OUTSIDE the boundary (invalid). Both are needed to test the edge cases!

6. Test Data (Continued)

6.3 Sample Test Plan

A program accepts a user's age between 0 and 120 inclusive. Here is a sample test plan:

Test No.	Description	Input	Expected Output	Type
1	Valid age within range	25	"Age accepted"	Normal
2	Input is not a number	"twenty"	"Invalid input"	Abnormal
3	Age above maximum	130	"Age out of range"	Extreme
4	Age at upper boundary	120	"Age accepted"	Boundary
5	Age at lower boundary	0	"Age accepted"	Boundary
6	Age just below lower boundary	-1	"Age out of range"	Boundary
7	Blank input	(blank)	"Please enter your age"	Abnormal

6.4 Selecting Suitable Test Data

A program accepts age between 12 and 18 inclusive:

Type	Test Input	Expected Result
Normal	14	Accepted ✓
Normal	16	Accepted ✓
Extreme	12	Accepted ✓ (minimum valid)
Extreme	18	Accepted ✓ (maximum valid)
Abnormal	H	Rejected ✗ (wrong type)
Abnormal	@	Rejected ✗ (wrong type)
Boundary	11	Rejected ✗ (just below minimum)
Boundary	19	Rejected ✗ (just above maximum)

❌ Common Mistake

Don't confuse extreme data with boundary data! Extreme data is the highest/lowest VALID values (still accepted). Boundary data is just OUTSIDE the valid range (should be rejected). You need to test BOTH!

7. Program Maintenance

Program maintenance is the process of updating or improving a program after it has been delivered to the user. Unlike physical equipment, programs don't wear out, but they may need to be changed due to errors, changing requirements, or new technology.

📖 Why Maintenance is Needed

To fix errors that were missed during testing
To improve performance based on user feedback
To adapt the software to new uses or platforms
To add new features requested by users
To keep up with changing technology and environments

7.1 Types of Maintenance

Type	Purpose	Example
Corrective	Fixes bugs or errors found during real-world use	Fixing a bug that causes crash when special characters entered
Perfective	Improves performance or adds small enhancements	Replacing loading screen with progress bar for better user feedback
Adaptive	Modifies program to support new environments or requirements	Modifying program to work on tablets instead of just desktops

7.2 Corrective Maintenance

Corrective maintenance is used to correct any errors that appear during use. For example, trapping a run-time error that had been missed during testing.

📝 Corrective Maintenance Examples

Fixing a bug that causes program to crash with certain inputs
Correcting calculation errors discovered by users
Patching security vulnerabilities found after release
Fixing logic errors that produce wrong results in specific cases

7.3 Perfective Maintenance

Perfective maintenance is used to improve performance of a program during its use. This includes enhancing functionality and improving user experience.

📝 Perfective Maintenance Examples

Improving the speed of response (optimization)
Adding progress bars for better user feedback
Enhancing user interface based on feedback
Reducing memory usage for better efficiency

7.4 Adaptive Maintenance

Adaptive maintenance is used to alter a program so it can perform any new tasks required by the customer or work in new environments.

📝 Adaptive Maintenance Examples

Working with voice commands as well as keyboard entry
Modifying program to work on tablets instead of just desktop computers
Updating software to work with new operating system version
Adding support for new file formats or hardware

7. Program Maintenance (Continued)

⚠️ Important Points

Programs don't "wear out" like physical equipment, but they need ongoing maintenance
The earlier an error is found, the cheaper it is to fix
Maintenance is part of the program lifecycle that continues after delivery
A patch is a small program released to correct errors or add functionality

🧠 Memory Trick

Corrective = Correct bugs and errors (Fix broken things)
Perfective = Performance improvements (Make things better)
Adaptive = Adapt to new environments (Change for new situations)

7.5 The Role of Patches

A patch is a small program released by developers to run with an existing program to correct an error or provide extra functionality. Patches are an essential part of ongoing maintenance.

📖 When Patches Are Used

Fix security vulnerabilities (corrective)
Add new features (perfective)
Support new hardware or OS versions (adaptive)
Improve performance (perfective)
Fix bugs reported by users (corrective)

8. Exam-Style Questions

1. Explain the difference between a syntax error and a logic error. Give an example of each. [4 marks]

Answer:

Syntax error: An error in the grammar/rules of the programming language that prevents the program from compiling/running
Example: Missing bracket, wrong keyword spelling, missing semicolon
Logic error: An error where the program runs but produces incorrect results due to wrong logic
Example: Using OR instead of AND, subtracting instead of multiplying, infinite loop