This is the first of many revised versions that will be used to describe the testing requirements, and general test plan, in detail, for Java Online Learning Tutor (J.O.L.T). These requirements need to be tested and met for J.O.L.T. to be considered ready to be used by regular users. As the project continues to approach completion, this document will also be edited to reflect any changes, or new requirements that are discovered. These series of documents will provide a record of any changes, as well as whether a test is passing or failing at that point in time. An updated version will be provided at the end of the Detailed Design phase, and a finalized version will be provided at the end of the Acceptance Test phase.

1.2 Introduction

A test plan gives detailed information regarding upcoming testing, including but not limited to, the scope of the testing, scheduling, acceptance criteria, as well as test results. The formats of test plans can vary with the product and organization to which they apply. There are two major elements of a test strategy that will be described in this test plan: Item Pass/Fail Criteria, and approach to testing.

Item Pass/Fail Criteria in this test plan shows which requirements will be tested during which stages of J.O.L.T’s product life. The Item Pass/Fail Criteria is derived from the Requirements Specification document where each requirement or specification will ideally have multiple tests applied, in order to verify the results.

Approach to testing in this test plan describes how the testing of J.O.L.T. will be conducted. Test methods may be determined by standards, regulatory agencies, contractual agreement, or by some other means. This will be discussed at a later time.

This document is the overview of the test plan. The goal of this test plan is to review the various functions and features of J.O.L.T. and test to make sure that each module is functioning properly as well as that each module is functioning correctly in regard to the whole system.

This particular document is incomplete in the individual modules of the test are concerned. Future versions of this document will be complete.

Section 2 Approach to Testing

2.1 Approach to Testing Introduction

Our approach to testing will start with the individual unit tests and proceed outwards, by testing module interaction. Then the modules will be integrated into the whole system, and the system will be tested as a whole. The goal of this is to catch all errors at a low level of complexity, in order to prevent small errors from creating larger ones.

Throughout the testing process our team may have to correct any problems in the systems and proceed to re-test any modules that interact with the module where the correction was made. As our team cannot predict how many of these problems may occur, we cannot predict how many times tests will have to be done. Problems will be discussed with the client in an effort to find a solution that suits the client’s needs.

2.2 Exception Handling To Test

Under certain circumstances J.O.L.T. may encounter an error that could not be predicted to happen. J.O.L.T will try to handle these errors by providing information to the user. Examples of errors that could occur are:

The database could not complete a request for one of various reasons:

A message will appear saying the request could not be completed and to try again. The user should try to submit their request again.

The user tried to submit incorrect java code:

J.O.L.T. will try to find a reason why the code could not run correctly and provide feedback to the user on what went wrong and how to fix it.

2.3 Item Pass/Fail Criteria

The following is a checklist to ensure that individual parts of the system are working properly. This outlines the major requirements that are being tested based on individual modules’ unit tests. These are arranged by user type as they appear in the functional requirements inventory.

2.3.1 Student

-The student user can register with the system

Yes No

-Once completing the registration form the Student receives and email message with a link to activate their account

Yes No

-The Student can Log in to the System Once Registered

Yes No

-The system displays correct error messages when a log in attempt fails

Yes No

-A link to an identity validation page is if password is forgotten.

Yes No

-3 failed login attempts with a given username locks that specific username out of the system

Yes No

-Once Logged in the student can enroll in a course section

Yes No

-Students can view announcements sent to them

Yes No

-Students can delete announcements sent to them

Yes No

-Students can view problem sets assigned to them

Yes No

-Students can view each individual problem within the problem set

Yes No

-Students can view solutions to individual problems, if provided by problem creator.

Yes No

-Students can submit solutions to individual problems within active problem sets in the form of Java™ source code

Yes No

-Student receives immediate, automatic feedback on compile errors, if any

Yes No

-Student receives immediate, automatic feedback on how their solution compares to the test cases

Yes No

-Student can complete problem sets

Yes No

-Student can navigate to individual problems in a problem set without completing them in a specific order

Yes No

-Any incomplete progress made for a problem or problem set is saved for the student

Yes No

-Student can view grades for each assignment in each class they are enrolled in.

Yes No

- Students can view all previously submitted solutions

Yes No

-Student has access to their solutions and grades for all prior classes they were enrolled in.

Yes No

-Student can log out

Yes No

2.3.2 Faculty

-Can log into system.

Yes No

-A failed log in displays an appropriate error message.

Yes No

-A link to an identity validation page is provided if password is forgotten.

Yes No

-3 failed login attempts locks the faculty member out of the system.

Yes No

-Faculty user can create individual problems

Yes No

-Individual problems that are partially completed by faculty are saved to a sandbox area, until they are complete.

Yes No

-Once complete, problem gets transferred to private pool.

Yes No

-Faculty user can create problem sets

Yes No

-Faculty user can import previously created problems to a problem set

Yes No

-Faculty user can import problems from the course pool to a problem set

Yes No

-Faculty user can import problems from the global pool to a problem set

Yes No

-Faculty user can individually create each problem for a problem set

Yes No

-Faculty user can assign problems they create to a category

Yes No

-Faculty user can assign a grading scheme to problem sets

Yes No

-Faculty user can assign a point value to specific problems within the problem set

Yes No

-Faculty user can assign problem sets to the sections they teach

Yes No

-Faculty user can set activation date and time of problem set

Yes No

-Faculty user can set expiration date and time of problem set

Yes No

-Faculty user can submit problems to a Course Pool

Yes No

-Faculty user can search a Course Pool for problems

Yes No

-Faculty user can search the Global Pool for problems

Yes No

- Faculty user can view a grade book for each of the courses

Yes No

-Faculty user can modify grades for all students in each of the courses they are currently teaching

Yes No

-Faculty user can post announcements to students in their courses

Yes No

-Faculty user can view announcements sent to them.

Yes No

-Faculty user can log out

Yes No

2.3.3 Course Coordinator

-Course Coordinator can log into system.

Yes No

-A failed log in displays an appropriate error message.

Yes No

-A link to an identity validation page is provided if password is forgotten.

Yes No

-3 failed login attempts will lead to system lockout.

Yes No

-Course Coordinator can create faculty accounts

Yes No

-Course Coordinator can assign faculty to a section

Yes No

-Course Coordinator can create problems and problem sets for courses they are in charge of

Yes No

-Individual problems that are partially completed by Course Coordinators are saved to a sandbox area, until they are complete.

Yes No

-Once complete, problem gets transferred to private pool.

Yes No

-Course Coordinator can access course tools which will provide statistics on problems and grades for a course

Yes No

-Course Coordinator can create reports over multiple sections of a course involving all problems and problem sets or any subset thereof.

Yes No

-Course Coordinator can manage the course pool for each course they are in charge of

Yes No

-Course Coordinator can add, modify, or delete any problem in their course pool

Yes No

-Course Coordinator can submit problems to the global pool

Yes No

-Course Coordinator can modify grades for all students currently enrolled in a course they currently manage

Yes No

-Course Coordinator can keep track of all grades and any adjustments that are made

Yes No

-Course Coordinator can create announcements for all faculty and students of courses they manage or any subset thereof

Yes No

-Course Coordinator can log out

Yes No

2.3.4 Administrator

-The Administrator can log into system.

Yes No

-A failed log in displays an appropriate error message.

Yes No

-A link to an identity validation page is provided if password is forgotten.

Yes No

-The Administrator can manage all accounts on the system

Yes No

-The Administrator can create course coordinator and faculty accounts

Yes No

-The Administrator can assign courses to course coordinators

Yes No

-The Administrator has all the same abilities as a course coordinator

Yes No

-The Administrator can manage the global pool of problems

Yes No

-Has access to tools for management of all accounts

Yes No

-The Administrator can modify all account information for any user

Yes No

-The Administrator can delete accounts

Yes No

-The Administrator can reset locked accounts

Yes No

-The Administrator can create announcements for all course coordinators, faculty and students, or any subset thereof

Yes No

-The Administrator can log out

Yes No

2.3.5 Java SDK

-Accepts and attempts to compile all Java™ source code submitted by students

Yes No

-Outputs compile errors, when applicable

Yes No

-Creates Java™ Byte Code upon a successful compilation

Yes No

-Executes all successfully compiled Java™ solutions

Yes No

-Monitors students’ submissions while they are running for runtime errors

Yes No

-"Kills" a student’s submission if it takes too long to run (Timeout)

Yes No

-Records the output generated from the students’ submissions

Yes No

2.4 Staffing

Members of the 518 Interactive Development team are responsible for the testing of J.O.L.T, in order to ensure that all of the functional requirements are met.

2.5 Responsibilities

The responsibilities of those who will be testing J.O.L.T are to execute the tests accurately in order to ensure that all of the functional requirements are met.

2.6 Schedule

Testing will begin alongside development which will start on March 10^th 2010 and continue until the Acceptance Test Presentation on April 4^th 2010. Exact members have not been determined yet.

Section 3 Functional Requirements Inventory

3.1 Functional Requirements Inventory Introduction

The following list outlines the required functionality that will be tested as a result of this test plan. All of the tests are designed to test whether one or more of the requirements have been met.

Java Online Learning Tutor will be a web-based application viewable on the major browsers. Browsers included will be Internet Explorer 8, Mozilla Firefox, Safari, and Google Chrome.

All references to Source Code imply Java™ Source Code, made to work with Java™ Version 1.6

3.2 Functional Requirements: Student:

· Will be able to register online with the system

o Will receive email confirmation following registration

· Will be able to log into system.

o A failed log in will display an appropriate error message.

o A link to an identity validation page will be provided if password is forgotten.

· Will be able to enroll into courses they are currently taking

o PIN number provided by instructor required to enroll into course on the system

· Will be able to view announcements sent to them.

· Will be able to view problem sets for each course they are in enrolled in

o Will be able to view each individual problem within the problem set

o Will be able to view hints and solutions to individual problems, if provided by problem creator.

· Will be able to submit solutions to individual problems within active problem sets in the form of Java™ source code

o Code will be compiled by the system online

o Student will receive immediate, automatic feedback on compile errors, if any

o Student will receive immediate, automatic feedback on how their solution compares to the test cases

· Will be able to complete problem sets

o Will be able to navigate to individual problems in a problem set without completing them in a specific order

o Will be able to save any progress made for a problem or problem set

· Will be able to view grades for each class

· Will be able to view all previously submitted solutions

· Will be able to log out

3.3 Functional Requirements: Faculty:

· Will be able to log into the system

· Will be able to create individual problems

· Will be able to create problem sets

o Will be able to import previously created problems to a problem set

o Will be able to import problems from the course pool to a problem set

o Will be able to import problems from the global pool to a problem set

o Will be able to individually create each problem for a problem set

· Will be able to assign problems they create to a category

· Will be able to assign a grading scheme to problem sets

o Will be able to assign a point value to specific problems within the problem set

· Will be able to assign problem sets to the students in the courses they teach

o Will be able to set activation date and time of problem set

o Will be able to set expiration date and time of problem set

· Will be able to submit problems to a Course Pool

· Will be able to search a Course Pool for problems

· Will be able to search the Global Pool for problems

· Will be able to view a grade book for each of the courses

· Will be able to modify grades for all students in each of the courses they are currently teaching

· Will be able to post announcements to students in their courses

· Will be able to view announcements sent to them.

· Will be able to interact with JOLT as a student user

· Will be able to log out

3.4 Functional Requirements: Course Coordinator:

· Will be able to log into the system

· Will be able to create faculty accounts

· Will be able to assign faculty to a course

· Will be able to create content for courses they are in charge of

· Will have access to course tools which will provide statistics on problems and grades for a course

o Will be able to create reports over multiple sections of a course involving all problems and problem sets or any subset thereof.

o Reports may include general statistics such as number of participants, average score, median, low score, and high score.

· Will be able to manage the course pool for each course they are in charge of

o Will be able to add, modify, or delete any problem in their course pool

o Will be able to submit problems to the global pool

· Will be able to modify grades for all students currently enrolled in a course they currently manage

o Will be able to keep track of all grades and any adjustments that are made

· Will be able to create announcement for all faculty and students of courses they manage or any subset thereof

· Will be able to log out

3.5 Functional Requirements: Administrator:

· Will be able log in

· Will be able to manage all accounts on the system

· Will be able to create course coordinator and faculty accounts

o Will be able to assign courses to course coordinators

· Will have the same abilities as a course coordinator

· Will be able to manage the global pool of problems

· Will have access to tools for management of all accounts

o Will be able to modify all account information for any user

o Will be able to delete accounts

o Will be able reset locked accounts

· Will be able to create announcements for all course coordinators, faculty and students, or any subset thereof

· Will be able to log out

3.6 Java SDK:

· Will accept and attempt to compile all Java™ source code submitted by students

o Will output compile errors, when applicable

o Will create Java™ Byte Code upon a successful compilation

· Will execute all successfully compiled Java™ solutions

o Will monitor students’ submissions while they are running for runtime errors

o Will kill a student’s submission if it takes too long to run (Timeout)

‘ Will record the output generated from the students’ submissions

Section 5 Unit Tests

Unit Tests are available in the “Unit Tests” document, which is located on the 518 Interactive website, under the “Documents” link.

4.2 Unit Tests Catalog

Section 5 Deliverables and Approval

5.1 Deliverables

Description and list of the deliverables presented to the client

5.2 Approval

Description of the Approval Process