SPT-P-6101
The Pretest software shall provide the user with the opportunity to exercise the same
degree of detailed control of the pretest activity process that is provided by the
Version 2.0 software.
SPT-P-6102
The number of questions presented to the user by the Pretest software shall be
minimized by the use of default values and values that are specific to the operating
system and compiler family being tested.
SPT-P-6103
The Pretest software shall base the default values for the following upon the operating
system and compiler family being used. If no operating system or compiler family is
identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted
system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or
a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host
system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target
system (or a self-hosted system);
* Command to compile (with no optimization) a program to be run on the target
system (or a self-hosted system);
* Command to compile (with full checking enabled) a program to be run on the
target system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system);
* Command to delete a compilation unit from the current Ada library;
* Command to delete the body (only) of a compilation unit from the current Ada
library; and
* Command to delete the specification (only) of a compilation unit from the current
Ada library.
SPT-P-6104
For commands and values that depend on the operating system and compiler family,
the Pretest software shall make use of default responses applicable to the following
combinations:
Operating System Compiler Family
DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS
SPT-P-6105
The Pretest software shall provide instructions or shall refer to documentation
containing instructions for providing user-defined defaults for the commands and values that are specified for the above (SPT-P-6104) combinations of operating
system and compiler family.
SPT-P-6106
The Pretest software shall ask the user to provide the following information in all
cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross-compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory.
SPT-P-6107
The Pretest software shall assume the indicated values for the following:
* Execution time measurement (EL: Elapsed time)
* Compilation time measurement (EL: Elapsed time)
* Code size measurement technique (L: Label'Address)
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be:
- DOS - .bat
- UNIX - .unx
- VMS - .com
SPT-P-6108
The Pretest software shall assume that the Portable math packages are to be used for
the default setting if the system is an Ada 83 implementation; otherwise, it shall
assume the existence of the math packages required by the Ada 95 standard.
SPT-P-6109
For cross-compilation systems only, the Pretest software shall require the user to
provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target
library.
SPT-P-6110
The Pretest software shall neither ask for nor use the following information that is
requested by the Version 2.0 software:
* Command to execute a program under the debugger;
* Command to capture the status of a completed process;
* Command to export the status of a completing process;
* Command to compile with debugging enabled;
* Command to link a program for execution under the debugger;
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol.
SPT-P-6122
The support.zip file shall be revised to unload the initial files needed for Pretest into a
"working" subdirectory of the "aces" directory.
SPT-P-6123
Pretest shall give the user the opportunity to modify the default as well as the
previously constructed environment file by rerunning Setup.
SPT-P-6124
Setup shall be modified so that the Ada source files required for Pretest will be copied
from the distribution subdirectories to the "source" subdirectory where they can be
accessed for compilation. All other required files shall be copied from the distribution
directories (note SPT-D-8109) to "working". This will be done at the beginning of the
Pretest activity and all files will be renamed at that point if needed.
SPT-P-8107
Setup shall inform the user which (83/95) compiler version is selected in any existing
environment file. It now only informs the user of the selection of cross/self compiler.
SPT-P-8116
Setup shall make a backup copy of any existing environment file before it offers the
user a chance to make changes. It shall also offer the user the chance to revert to the
old version by replacing quit with two choices: (1) generate new file with the new
selections; or (2) exit and save none of the changes; go back to the old version.
SPT-P-8117
The Pretest software unit "zg_incld" (four versions) shall specify the type
CHARACTER for at least one of the literals in the code "for i in 'a' .. 'z' loop".
3.2.2 Utilities (CSC 1.2)
The Utilities CSC is a logical abstraction for the software that provides a variety of
functions to the other CSCs. The utility software consists of utility tests, initialization
software, inclusion software, global type and object declarations, and mathematics
software. The utility tests enable the system to check the system's ability to use a cpu
clock, and measure code size. The initialization software builds baselined timing files
that are used during the building of the operational software. The inclusion software
copies the appropriate timing code into the performance test templates, producing the
actual performance tests. Library units that provide global types and shared objects are
necessary for the compiling of the various CSCs. The mathematics software provides
the software necessary to perform statistical analysis.
The design of this CSC has not been modified since version 2.0 and therefore is not
documented here.
3.2.3 Harness (CSC 1.3)
The HARNESS allows the users to select performance tests individually, by subgroup,
and by group. Harness also allows the user to generate command files to compile, link,
and run the selected tests, and to monitor the results. A new CSU is to be added to the
Harness CSC to allow the user to choose performance tests by identifying performance
issues. The user will be able to choose tests through performance issues and then
modify the individual test selection, or the user may continue to use the HARNESS in
the same manner as was allowed in previous versions of the software.
The use of the HARNESS is optional, but it is expected that all ACES users who run
performance tests will use the HARNESS if their system permits. The basic design of
the Harness software has been modified by adding this new CSU. The design changes
for Harness are detailed in Section 4.3.
Table 3.2.3-1 identifies the requirements allocated to the Harness CSC.
Table 3.2.3-1 Harness Requirements
Requirement ID
Requirement Text
OPS-T-5301
The ACES shall determine the performance characteristics of conversions between
enumeration or numeric (integer, modular, float, fixed, decimal) types and String,
using attributes and instantiations of the generic IO packages in Text_IO.
OPS-T-5302
The ACES shall determine the performance characteristics of the "Succ" and "Pred"
attributes for real types.
OPS-T-5303
The ACES shall determine the performance of a subprogram call when the
subprogram is passed to the calling subprogram via an access-to-subprogram
parameter.
OPS-T-5304
The ACES shall determine the performance characteristics of decimal type operations
(if supported).
OPS-T-5305
The ACES shall determine the performance impact of referencing and assigning to an
object designated by a subprogram's formal access parameter.
OPS-T-5306
The ACES shall determine the performance of assignment of controlled types.
OPS-T-5307
The ACES shall determine the performance of user-defined finalization when exiting
the scope of a controlled object.
OPS-T-5308
The ACES shall determine the time required for a call on a protected operation in the
simplest of cases.
OPS-T-5309
The ACES shall determine the performance characteristics of the requeue statement.
OPS-T-5310
The ACES shall determine the performance characteristics of the delay until
statement.
OPS-T-5311
The ACES shall investigate the performance characteristics of asynchronous transfer
of control in various scenarios.
OPS-T-5312
The ACES shall determine the performance characteristics of references to entities in
child packages.
OPS-T-5313
The ACES shall determine the performance characteristics of the operations in
package Ada.Exceptions.
OPS-T-5314
The ACES shall determine the performance characteristics of the address arithmetic
operations declared in the package System.Storage_Elements.
OPS-T-5315
The ACES shall determine the performance characteristics of the conversion
operations declared in the package System.Address_To_Access_Conversions.
OPS-T-5316
The ACES shall determine the performance characteristics of the storage management
operations declared in the package System.Storage_Pools.
OPS-T-5317
The ACES shall determine the performance characteristics of passing data among
units by using the stream operations given by attributes.
OPS-T-5318
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Fixed.
OPS-T-5319
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Bounded.
OPS-T-5320
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Unbounded.
OPS-T-5321
The ACES shall determine the performance characteristics of the operations declared
in Ada.Numerics.Elementary_Functions and in an instantiation of
Ada.Numerics.Generic_Elementary_Functions, using Float.
OPS-T-5322
The ACES shall determine the performance characteristics of the random value
function declared in instantiations of Ada.Numerics.Discrete_Random.
OPS-T-5323
The ACES shall determine the performance characteristics of the shift and rotate
operations provided by package Ada.Interfaces.
OPS-T-5324
The ACES shall determine the performance characteristics of calls to subprograms
written in C, C++, Fortran, and COBOL, as supported by the specific implementation.
OPS-T-5325
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.
OPS-T-5326
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.Strings.
OPS-T-5327
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.Pointers.
OPS-T-5328
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.COBOL.
OPS-T-5329
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.Fortran.
OPS-T-5401
The ACES shall determine the performance impact of using aliased objects as
subprogram parameters.
OPS-T-5402
The ACES shall determine the performance effects of declaring objects of
unconstrained types (the initial constraint being taken from the initialization
expression).
OPS-T-5403
The ACES shall determine whether a performance cost is incurred when referencing
an extension component of a tagged type.
OPS-T-5404
The ACES shall determine whether a performance cost is incurred in referencing and
updating arrays and records whose components are aliased.
OPS-T-5405
The ACES shall determine the relative performance of dispatching for various kinds of
controlling operands.
OPS-T-5406
The ACES shall determine the performance assignments to an aliased object and
evaluations of an aliased object.
OPS-T-5407
The ACES shall determine the performance impact of using general access types to
reference data at different accessibility levels.
OPS-T-5408
The ACES shall determine the performance characteristics of evaluations of the
Access attribute.
OPS-T-5409
The ACES shall determine the relative performance of elaboration of private type and
non-private type declarations.
OPS-T-5410
The ACES shall determine the performance impact of using discriminants for tasks
and for protected objects.
OPS-T-5411
The ACES shall determine the performance impact of using exception handlers in
accept bodies.
OPS-T-5412
The ACES shall determine the performance characteristics of exception propagation
from protected subprograms and entries of protected objects.
OPS-T-5413
The ACES shall determine the performance impact of the use of representation clauses
that have been added to the language.
OPS-T-5414
The ACES shall determine the performance characteristics of evaluations of the
Bit_Order attribute.
OPS-T-5415
The ACES shall determine the performance impact of using unchecked conversion to
produce valid and invalid values of a type.
OPS-T-5416
The ACES shall determine the performance impact of using the Unchecked_Access
attribute to create access values.
OPS-T-5417
The ACES shall determine the performance characteristics of evaluations of the
Storage_Pool and Storage_Size attributes for access subtypes.
OPS-T-5418
The ACES shall determine the performance characteristics of the operations declared
in package Ada.Characters.Handling.
OPS-T-5419
The ACES shall determine the performance characteristics of Wide_Character
operations.
OPS-T-5420
The ACES shall determine the performance characteristics of Wide_String operations.
OPS-T-5421
The ACES shall determine the performance characteristics of conversions between
enumeration or numeric (integer, modular, float, fixed, decimal) types and
Wide_String, using attributes and instantiations of the generic IO packages in
Wide_Text_IO.
OPS-T-5422
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Fixed.
OPS-T-5423
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Bounded.
OPS-T-5424
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Unbounded.
OPS-T-5425
The ACES shall determine the performance characteristics of the operations declared
in package Streams.Stream_IO. Compare to the performance of equivalent attributes
after applying the Stream function to create a Stream_Access value from the file.
OPS-T-7101
The following tests shall be renamed:
a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01
OPS-T-7102
The following tests shall be renamed:
a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02
OPS-T-7103
The following tests shall be renamed:
a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03
SPT-H-5101
The Harness shall recognize new performance test groups and subgroups, as
determined by new and renamed performance tests.
SPT-H-5102
The Harness shall recognize names of new default main programs that incorporate
new and renamed tests.
SPT-H-5103
Main programs that incorporate new or renamed tests shall not include any previously
existing tests.
SPT-H-5104
The Harness shall display all new test names that are appropriate for the
implementation being tested.
SPT-H-5105
The Harness shall display tests specific to Ada 95 only if the user has identified the
implementation as an Ada 95 implementation.
SPT-H-5106
The Harness shall display group and subgroup names only if the groups and
subgroups contain tests applicable to the language version of the implementation
under test.
SPT-H-6201
The Harness software shall provide the user an option of selecting tests based on the
performance issues of interest to the user.
SPT-H-6202
The Harness shall provide the user with the option of bypassing the performance
issues menu and proceeding directly to the Harness menu.
SPT-H-6203
The Harness shall provide a default set of performance issues, which shall be
preselected when the performance issues menu first appears.
SPT-H-6204
The Harness shall allow the user to select or de-select any of the performance issues.
SPT-H-6205
The Harness shall provide at least the following options on the performance issues
menu:
- select all performance issue selections;
- deselect all performance issue selections;
- request help; and
- quit the performance issues menu (exit to the Groups menu).
SPT-H-6206
If the performance issues menu is not used, the ACES shall work in the same manner
as in ACES, Version 2.0. If a test.sel file exists, Harness shall use that as the initial
test selection. Otherwise, no tests will be selected when the Harness menu is initially
displayed.
SPT-H-7201
The Harness shall provide a mechanism for selecting User-Defined benchmarks in a
fashion similar to the way that tests are currently handled in the ACES.
SPT-H-7202
The Harness shall provide a mechanism for generating command scripts that will
execute User-Defined benchmarks in a fashion similar to the way that command
scripts are currently generated by the Harness.
SPT-H-8201
The Harness software shall indicate, in its error messages in UPDATING, the name of
the file causing the error rather than the name of the file following the file that caused
the error.
3.2.4 Analysis (CSC 1.4)
The Analysis CSC is a logical abstraction for the processes involved in analyzing the
test results generated in the operational software. The Analysis CSC is composed of
the Condense process, Comparative Analysis process and the Single System Analysis
process. The basic design for these components has not been modified since version
2.0 and therefore the design for this CSC is not presented in this document.
Table 3.2.4-1 identified the requirements allocated to the Analysis CSC.
Table 3.2.4-1 Analysis Requirements
Requirement ID
Requirement Text
OPS-T-5301
The ACES shall determine the performance characteristics of conversions between
enumeration or numeric (integer, modular, float, fixed, decimal) types and String,
using attributes and instantiations of the generic IO packages in Text_IO.
OPS-T-5302
The ACES shall determine the performance characteristics of the "Succ" and "Pred"
attributes for real types.
OPS-T-5303
The ACES shall determine the performance of a subprogram call when the
subprogram is passed to the calling subprogram via an access-to-subprogram
parameter.
OPS-T-5304
The ACES shall determine the performance characteristics of decimal type operations
(if supported).
OPS-T-5305
The ACES shall determine the performance impact of referencing and assigning to an
object designated by a subprogram's formal access parameter.
OPS-T-5306
The ACES shall determine the performance of assignment of controlled types.
OPS-T-5307
The ACES shall determine the performance of user-defined finalization when exiting
the scope of a controlled object.
OPS-T-5308
The ACES shall determine the time required for a call on a protected operation in the
simplest of cases.
OPS-T-5309
The ACES shall determine the performance characteristics of the requeue statement.
OPS-T-5310
The ACES shall determine the performance characteristics of the delay until
statement.
OPS-T-5311
The ACES shall investigate the performance characteristics of asynchronous transfer
of control in various scenarios.
OPS-T-5312
The ACES shall determine the performance characteristics of references to entities in
child packages.
OPS-T-5313
The ACES shall determine the performance characteristics of the operations in
package Ada.Exceptions.
OPS-T-5314
The ACES shall determine the performance characteristics of the address arithmetic
operations declared in the package System.Storage_Elements.
OPS-T-5315
The ACES shall determine the performance characteristics of the conversion
operations declared in the package System.Address_To_Access_Conversions.
OPS-T-5316
The ACES shall determine the performance characteristics of the storage management
operations declared in the package System.Storage_Pools.
OPS-T-5317
The ACES shall determine the performance characteristics of passing data among
units by using the stream operations given by attributes.
OPS-T-5318
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Fixed.
OPS-T-5319
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Bounded.
OPS-T-5320
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Unbounded.
OPS-T-5321
The ACES shall determine the performance characteristics of the operations
declared in Ada.Numerics.Elementary_Functions and in an instantiation of
Ada.Numerics.Generic_Elementary_Functions, using Float.
OPS-T-5322
The ACES shall determine the performance characteristics of the random value
function declared in instantiations of Ada.Numerics.Discrete_Random.
OPS-T-5323
The ACES shall determine the performance characteristics of the shift and rotate
operations provided by package Ada.Interfaces.
OPS-T-5324
The ACES shall determine the performance characteristics of calls to subprograms
written in C, C++, Fortran, and COBOL, as supported by the specific implementation.
OPS-T-5325
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.
OPS-T-5326
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.Strings.
OPS-T-5327
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.C.Pointers.
OPS-T-5328
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.COBOL.
OPS-T-5329
The ACES shall determine the performance characteristics of the operations provided
by package Ada.Interfaces.Fortran.
OPS-T-5401
The ACES shall determine the performance impact of using aliased objects as
subprogram parameters.
OPS-T-5402
The ACES shall determine the performance effects of declaring objects of
unconstrained types (the initial constraint being taken from the initialization
expression).
OPS-T-5403
The ACES shall determine whether a performance cost is incurred when referencing
an extension component of a tagged type.
OPS-T-5404
The ACES shall determine whether a performance cost is incurred in referencing and
updating arrays and records whose components are aliased.
OPS-T-5405
The ACES shall determine the relative performance of dispatching for various kinds of
controlling operands.
OPS-T-5406
The ACES shall determine the performance assignments to an aliased object and
evaluations of an aliased object.
OPS-T-5407
The ACES shall determine the performance impact of using general access types to
reference data at different accessibility levels.
OPS-T-5408
The ACES shall determine the performance characteristics of evaluations of the
Access attribute.
OPS-T-5409
The ACES shall determine the relative performance of elaboration of private type and
non-private type declarations.
OPS-T-5410
The ACES shall determine the performance impact of using discriminants for tasks
and for protected objects.
OPS-T-5411
The ACES shall determine the performance impact of using exception handlers in
accept bodies.
OPS-T-5412
The ACES shall determine the performance characteristics of exception propagation
from protected subprograms and entries of protected objects.
OPS-T-5413
The ACES shall determine the performance impact of the use of representation clauses
that have been added to the language.
OPS-T-5414
The ACES shall determine the performance characteristics of evaluations of the
Bit_Order attribute.
OPS-T-5415
The ACES shall determine the performance impact of using unchecked conversion to
produce valid and invalid values of a type.
OPS-T-5416
The ACES shall determine the performance impact of using the Unchecked_Access
attribute to create access values.
OPS-T-5417
The ACES shall determine the performance characteristics of evaluations of the
Storage_Pool and Storage_Size attributes for access subtypes.
OPS-T-5418
The ACES shall determine the performance characteristics of the operations declared
in package Ada.Characters.Handling.
OPS-T-5419
The ACES shall determine the performance characteristics of Wide_Character
operations.
OPS-T-5420
The ACES shall determine the performance characteristics of Wide_String operations.
OPS-T-5421
The ACES shall determine the performance characteristics of conversions between
enumeration or numeric (integer, modular, float, fixed, decimal) types and
Wide_String, using attributes and instantiations of the generic IO packages in
Wide_Text_IO.
OPS-T-5422
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Fixed.
OPS-T-5423
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Bounded.
OPS-T-5424
The ACES shall determine the performance characteristics of the operations declared
in package Strings.Wide_Unbounded.
OPS-T-5425
The ACES shall determine the performance characteristics of the operations declared
in package Streams.Stream_IO. Compare to the performance of equivalent attributes
after applying the Stream function to create a Stream_Access value from the file.
OPS-T-7101
The following tests shall be renamed:
a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01
OPS-T-7102
The following tests shall be renamed:
a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02
OPS-T-7103
The following tests shall be renamed:
a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03
SPT-A-5201
The Analysis software shall recognize the names of new performance test groups,
subgroups, tests, and default main programs.
SPT-A-5202
The Menu program shall allow selection of groups consisting entirely of Ada 95 tests
if and only if at least one selected implementation has been identified as an Ada 95
implementation.
SPT-A-5203
The Comparative Analysis program shall incorporate results from new performance
test groups, subgroups, default main programs, and tests into its analyses and reports.
SPT-A-5204
The Single-System Analysis program shall provide comparisons of sets of new tests
and of sets containing both new and previously existing tests as appropriate.
SPT-A-6301
The Analysis software shall provide default reports which shall be designed to provide
the user with the least amount of information which would also be usable. The first
CA report shall present the System Factors for all test groups. The second shall
present the number of tests successfully implemented.
SPT-A-6302
The default metric for Comparative Analysis reports shall be execution time.
SPT-A-6303
The Analysis software shall provide default reports which shall be designed to provide
the user with the least amount of information which would also be usable. The SSA
report shall present a ratio of the smallest to the largest reported Language Features for
each test comparison.
SPT-A-7301
The ACES shall provide a method to perform Comparative Analysis on User-Defined
benchmarks.
SPT-A-7302
The ACES shall provide a method to perform Single-System analysis on User-Defined
benchmarks.
SPT-A-8301
The Menu software shall allow the user to advance to the next screen without having
to type "n" in addition to the command where multiple selections before going to the
next screen are not possible.
SPT-A-8302
The Analysis software, on the MAIN MENU screen, shall tell the user to select a
command or a tool and a command (separating them with a comma) instead of telling
the user to "select one tool separated by a comma."
SPT-A-8303
The Analysis software shall be designed to exit gracefully if multiple errors come up
after all selections are made instead of having the user resort to a keyboard abortion
command.
SPT-A-8304
The Analysis software shall be designed to give adequate error messages and exit
gracefully if it is missing a file so that the user will know what files are required for it
to function properly. The software shall indicate if the file is missing or if it cannot
locate the name it was given to find and where to make that change.
SPT-A-8305
The Analysis software shall be redesigned so that the GROUPS menu either fits on
one screen or appears on two screens in a way that is easy to use. Presently the title
and "1. Applications" scroll off screen when Ada_95 is an option.
SPT-A-8306
The Analysis software shall be redesigned so that the RUN OR SAVE REQUEST
menu will indicate in all cases either that no reports were chosen (and therefore no
output is to be expected) or what has been chosen and what will be produced.
SPT-A-8307
The Analysis software shall be revised so that when a screen indicates that one of a
group of items must be selected, then the user will really have to make a selection
before going to the next screen.
This occurs on at least the following screens:
- SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS:
- SSA MAIN REPORT OPTIONS
- SSA HLS REPORT OPTIONS
SPT-A-8308
The Analysis software shall present a new menu that is a combination of the present
MAIN and INITIAL menus.
SPT-A-8311
The Analysis software menus shall use language consistent with the actual reports.
This shall include but not be limited to:
- Selection 7 of the SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS,
which indicates on the screen "errors" and on the report "successes".
SPT-D-8309
The Analysis software shall define the help menu for BUILD/MODIFY SYSTEM
NAME FILE (GENERAL INFORMATION) so that the file names explained there are
the same as those that appear in selections and make other corrections as needs so that
the help menus correspond to the screens they serve.
3.2.5 Quick-Look (CSC 1.5)
The Quick-Look CSC provides the users with the ability to quickly setup and run a
subset of the ACES.
The design of the Quick-Look components has not been modified since version 2.0 and
is therefore not presented in this document.
Table 3.2.5-1 identifies the requirements allocated to the Quick-Look CSC.
Table 3.2.5-1 Quick-Look Requirements
Requirement ID
Requirement Text
SPT-Q-6111
The Quick-Look software shall provide the user with the opportunity to exercise the
same degree of detailed control of the Quick-Look activity process that is provided by
the Version 2.0 software.
SPT-Q-6112
The number of questions presented to the user by the Quick-Look software shall be
minimized by the use of default values and values that are specific to the operating
system and compiler family being tested.
SPT-Q-6113
The Quick-Look software shall base the default values for the following upon the
operating system and compiler family being used. If no operating system or compiler
family is identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted
system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or
a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host
system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target
system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system);
and
* Command to delete a compilation unit from the current Ada library.
SPT-Q-6114
For commands and values that depend on the operating system and compiler family,
the Quick-Look software shall make use of default responses applicable to the
following combinations:
Operating System Compiler Family
DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS
SPT-Q-6115
The Quick-Look software shall provide instructions or shall refer to documentation
containing instructions for providing user-defined defaults for the commands and
values that are specified for the above combinations of operating system and compiler
family.
SPT-Q-6116
The Quick-Look software shall ask the user to provide the following information in all
cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross-compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory.
SPT-Q-6117
The Quick-Look software shall assume the indicated values for the following:
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be:
- DOS - .bat
- UNIX - .unx
- VMS - .com
SPT-Q-6118
For cross-compilation systems only, the Quick-Look software shall require the user to
provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target
library.
SPT-Q-6119
The Quick-Look software shall neither ask for nor use the following information that
is requested by the Version 2.0 software:
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol.
SPT-Q-6127
Setup shall be modified so that the Ada source files required for Quick-Look will be
copied from the distribution subdirectories to the "source" subdirectory where they can
be accessed for compilation. All other required files shall be copied from the
distribution directories (note SPT-D-8109) to "ql_work". This will be done at the
beginning of the Quick-Look activity and all files will be renamed at that point if
needed.
SPT-Q-6130
The support.zip file shall be revised to unload the initial files needed for Quick-Look
into a "ql_work" subdirectory of the "aces" directory.
SPT-Q-7101
The Quick-Look facility shall be flexible enough to allow additional User-Defined
benchmark tests to be added.
SPT-Q-7102
The Quick-Look facility shall be able to report on the results of additional User-
Defined benchmark tests.
SPT-Q-8402
The Quick-Look software unit "zg_incld" shall specify the type CHARACTER for at
least one of the literals in the code "for i in 'a' .. 'z' loop".
4.0 DETAILED DESIGN
4.1 Pretest (CSC 1.1)
Figure 4.1-1 illustrates the Pretest architecture.
Figure 4.1-1 Pretest (CSC 1.1) Architecture
4.1.1 Setup (CSC 1.1.1)
The CSUs for this CSC are Identify Environment Data and Generate Scripts.
The design for Setup has not been modified since version 2.0. The only changes made
are to accommodate the new tests for Ada 95 and the use of default choices for users.
4.1.1.1 Identify Environment Data (CSU 1.1.1.1)
The design for this CSU has not been modified since version 2.0. The only changes are
those made to accommodate the use of default choices for users.
4.1.1.2 Generate Scripts (CSU 1.1.1.2)
The design for this CSU has not been modified since version 2.0. The only changes are
those made to accommodate the use of default choices for users.
4.1.2 Scripts (CSC 1.1.2)
One script has been added since ACES Version 2.0. That script is for Pretest Step 0. It
has been added to make the system more automated.
4.1.2.1 Scripts (CSU 1.1.2.1)
Purpose: The script that was added copies all of the necessary files from the
distribution subdirectories automatically into the working directory, with the correct
suffixes.
Requirements: See Section 3.2.1.
Design Constraints: The script is applicable to a specific Ada implementation (a
specific operating system and a specific Ada compiler vendor). Generation of the script
is driven by a non-specific template and by compiler and operating system data
supplied by the user or by default data files.
CHI: None.
Rationale for Other Language: Requires command level interface to the operating
system.
IO Data Elements: None.
Algorithm: Copy Pretest files from the distribution to the working subdirectories.
If file extension is different from system file extension
Change to system file extension
end if
end
Error Handling: None.
Other Units: None.
Invoked By: User.
Calls: None.
Data Structures: None.
Data Files: A new template file shall be designed to perform the copying of the files.
4.2 Utilities (CSC 1.2)
Figure 4.2-1 illustrates the Utilities architecture.
Figure 4.2-1 Utilities (CSC 1.2) Architecture
The design for the Utilities CSC has not been modified since version 2.0.
4.3 Harness (CSC 1.3)
The Harness design has been modified by adding a CSU to allow performance test
selections through the identification of relevant performance issues. The user has the
option of either using performance issues to select tests or selecting tests in the previous
manner by group, subgroup, and test names. The user may also use a combination of
the two methods by identifying performance issues and then modifying the test
selection.
The Select Performance Issues CSU has been added to the Harness CSC and is the only
design change since ACES 2.0.
4.3.1 Select Performance Issues (CSU 1.3.1)
Purpose: Generate a new menu to allow the user to select performance tests by
identifying performance issues. These issues will identify tests that are automatically
selected for processing. After selecting performance issues important to the user's
application, the user will run the Harness menu to modify the test selection and to build
scripts to run the performance tests.
Design Constraints: The user should be able to operate the Harness in the same manner
as previously allowed; the user will still be able to select tests by group, subgroup and
test.
Default performance issues will be assigned when the user first enters the Select
Performance Issues menu.
Help text will be available.
The Select Performance Issue menu will provide options to select all performance
issues and to clear all selections.
CHI: The initial use screen will allow the user to identify how he/she wants to identify
performance tests. The user may identify performance issues important to the user's
application or may enter the harness screen that allows the user to select
groups/subgroups, and individual tests.
Additional menus will be displayed to allow the user to identify performance issues.
The menus will allow the following options:
• default performance issues will initially be selected
• the user can select or de-select any performance issue
• the user can get help information
• the user can select all performance issues
• the user can clear all performance issues
• the user can look at the next or previous screen of performance issues
• the user can quit Harness
• the user can proceed to the Harness main menu
Rationale for other Language: N/A
IO Data Elements:
Input data elements: Test selection data structure.
Output data elements: Test selection data structure.
Quit request.
Algorithm:
Modify zh_hrn15.ada.
Add a call in the first line of the harness procedure (after the begin) to call a new
routine that handles the performance issue selection.
Surround the existing harness code in an if statement (after the begin statement and call
to the new routine).
begin
Select_Performance_Issues (Test_selection_data_structure, -- inout
Quit_Request); -- out
if Not Quit_Request
then
... existing code ...
end if;
-- the basic design of the performance issue selection routine is:
BEGIN -- predefined selection routine
Set Quit_Request = FALSE
Put up Harness_Entry screen.
Get user_selects_harness_tests input.
If user_selects_harness_tests
then
exit procedure
elsif -- select set of tests based on performance issues
loop
read performance_issue and default_PI from zh_prfmi.txt
if performance_issue = blank
then
exit loop
end if
if default_PI
then
set performance_issue selected
end if
end loop
Put up Performance_Issues_Menu.
loop
Get Performance_Issues_Menu_input.
if Performance_Issues_Menu_input = Help
CALL Performance_Issues_Menu_Help.
elsif Performance_Issues_Menu_input = Quit
set Quit_Request = TRUE
exit procedure
elsif Performance_Issues_Menu_input = Next Screen
Put up next Performance_Issue_Menu.
elsif Performance_Issues_Menu_input = Previous Screen
Put up Previous Performance_Issue_Menu.
elsif Performance_Issues_Menu_input = Harness Menu
current_group_name := unassigned
loop
read test_name and performance_issue from zh_pfrmi.txt
if eof then exit loop
if performance issue is selected
then
if group_name (of_test) = current_group_name
if Ada95Compiler
set test selected
if Ada95Test
set test selected
end if
end if
else
read group_name (data_file) into memory
if Ada95Compiler
set test selected
if Ada95Test
set test selected
end if
end if
end if -- group
end if --performance issue selected
end loop
Write test.sel file
exit procedure
elsif Performance_Issues_Menu_input = Clear All
Set All Performance Issues to NOT_SELECTED
elsif Performance_Issues_Menu_input = Select All
Set All Performance Issues to SELECTED
elsif Performance_Issues_Menu_input =
positive or negative performance_issue
modify performance_issues data structure.
put plus (+) or minus (-) next to appropriate
performance_issues when re-displaying menu
else
print input error message
prompt to continue
re-display menu
end if
end loop
else -- Quit
Set Quit_Request = TRUE
exit procedure
end if
END -- performance issue selection routine
Error Handling: The CSU will handle invalid user menu responses and re-display
menu and re-prompt user for new input.
Other Units: None.
Invoked By: Harness main procedure.
Calls: None.
Data Structures: Uses existing Test_selection_data_structure (TestRecord in Package
GlobalPackage) with one additional data element that identifies a test as an Ada83 or an
Ada95 test (Ada95Test : BOOLEAN).
Local Files: An additional input file will be created and will be input to the Harness
software. The structure and data contained in this file is described below:
File Name:
zh_prfmi.txt
text file
Data:
Performance_Issue #1 default_PI
Performance_Issue #2
Performance_Issue #3
Performance_Issue #4 default_PI
...
Performance_Issue_#n-1
Performance_Issue_#n default_PI
blank line
test_name Performance_Issue_#i
test_name Performance_Issue_#j
...
test_name Performance_Issue_#i
test_name Performance_Issue_#k
(end of file)
Test names above will be sorted by group.
4.4 Analysis (CSC 1.4)
Figure 4.4-1 illustrates the Analysis architecture.
Figure 4.4-1 Analysis (CSC 1.4) Architecture
The design for the Analysis has not been modified since version 2.0. The software
components that make up this CSC have not been changed since version 2.0 and their
design is not repeated in this document.
4.5 Quick-Look (CSC 1.5)
Figure 4.5-1 illustrates the Quick-Look architecture.
Figure 4.5-1 Quick-Look (CSC 1.5) Architecture
4.5.1 Setup (CSC 1.5.1)
The CSUs for this CSC are Identify Environment Data and Generate Scripts.
The design for Setup has not been modified since version 2.0.
4.5.1.1 Identify Environment Data (CSU 1.5.1.1)
The design for this CSU has not been modified since version 2.0. The only changes are
those made to accommodate the new tests for Ada 95 and the use of default choices for
users.
4.5.1.2 Generate Scripts (CSU 1.5.1.2)
The design for this CSU has not been modified since version 2.0. The only changes are
those made to accommodate the use of default choices for users.
4.5.2 Scripts (CSC 1.5.2)
The CSUs for this CSC are Control Testing and Control Reporting. A Quick-Look
script controls the entire Quick-Look process by invoking the Control Testing script
and the Control Reporting script. The Control Testing portion of this script directs its
output to the Test Log file, which serves as input to the Control Reporting portion.
4.5.2.1 Control Testing (CSU 1.5.2.1)
The design for Control Testing CSU has not been modified since version 2.0.
4.5.2.2 Control Reporting (CSU 1.5.2.2)
The design for Control Reporting CSU has not been modified since version 2.0.
4.5.3 Report (CSC 1.5.3)
The design for the Report CSC has not been modified since version 2.0. The only CSU
for this CSC is Report which has not been changes since version 2.0 and therefore the
design is not repeated in this document.
5.0 CSCI DATA
There is no new CSCI data for the ACES Version 2.1.
6.0 CSCI DATA FILES
This section identifies the data files associated with the Support Software CSCI 1.0.
6.1 CSCI Data Files
6.1.1 Single System Analysis
There is a new output report file for Single System Analysis.
Purpose: This data file is a report from the Single System Analysis which outputs the
name of the comparison group and the ratio of the fastest to the slowest time
measurement in each comparison. The comparison groups will be divided into tables
by speed advantage. The information will be taken from the existing SSA Multiple
Comparison Reports. Results of comparisons of the data from the fastest to the slowest
test problems in each comparison group will determine in which table it shall be placed.
6.1.2 Comparative Analysis
There is a new output report file for Comparative Analysis.
Purpose: This data file is a report from Comparative Analysis which outputs a table of
the execution time ratios for each group for each system. It will then output the number
of tests of each group successfully completed by each system. It will take this
information from the present System Factors for all groups for all systems and the
Summary of Successes for each group reports.
6.1.3 Harness
There is a new input file to the Harness software.
Purpose: This data file will be read by the Harness software and will identify the
performance issues and the performance tests that are related to each performance issue.
7.0 REQUIREMENTS TRACEABILITY
Table 7.0-1 identifies the requirements for updating the ACES from version 2.0
through 2.1. They are pulled from the Usability Enhancements Document and the
Operational Software Objectives Document.
The requirement IDs were generated using the following technique. The first three
characters indicate whether it concerns support software (SPT) or operational software
(OPS). This is followed by a dash and a single letter which indicates if it refers to
pretest (P), harness (H), quick-look (QL), analysis (A), performance test (T) or
documentation (D).
The Requirement Text column is a narrative of the requirement. The Alloc. column
identifies the CSCs that satisfy the requirement. The Alloc. column will contain
"ACES", "DOC", or CSC identifier(s). When "ACES" is found, the requirement is a
system level requirement and is not specifically allocated to software. When "DOC" is
found, the requirement is allocated to documentation.
The IAD column identifies how the requirement is to be validated. An "I" indicates
that the requirement is satisfied "in accordance with" another requirement. When an "I"
is found it will be followed by a Requirement ID. An "I" means that this requirement is
satisfied when the named requirement is satisfied. An "A" in this column indicates that
the requirement will be satisfied through Analysis. The tester will analyze results,
source code, etc. to ensure that the requirement has been satisfied. A "D" in this
column indicates that the requirement will be satisfied through Demonstration. A test
will be created that will demonstrate that the requirement has been satisfied.
The Fix/Design Change column indicates whether the requirement necessitates a Fix to
the existing software or whether the requirement will cause a change in the design of
the version 2.0 software. When a design change is indicated the reader will find a
description of that change in this document or in the Software Design Document for the
Operational Software.
Table 7.0-1 Requirements Traceability Matrix
Requirement
ID
Requirement Text
Alloc.
IAD
Fix/Design
Change
OPS-D-6202
The Quick-Look and Performance Testing
sections of the Primer shall warn the user that
the Tasking and Delay_Timing tests using
pragma Priority may not behave properly for
Ada 95 implementations not supporting the
Real-Time Annex.
DOC
A
N/A
OPS-D-7201
The Primer shall inform the user that certain
Tasking tests contain references to the
implementation-dependent package "Starlet"
and use the implementation-dependent
"AST_Entry". The Primer shall give
guidance for adapting these tests to
implementations other than VAX-Ada. These
tests have names of the form "tk_in_int_*",
where '*' ranges from 01 to 10. They appear
in the files with names of the form
"tk_in#_.inc", where '#' ranges from 02 to 11.
DOC
A
N/A
OPS-T-5301
The ACES shall determine the performance
characteristics of conversions between
enumeration or numeric (integer, modular,
float, fixed, decimal) types and String, using
attributes and instantiations of the generic IO
packages in Text_IO.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5302
The ACES shall determine the performance
characteristics of the "Succ" and "Pred"
attributes for real types.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5303
The ACES shall determine the performance of
a subprogram call when the subprogram is
passed to the calling subprogram via an
access-to-subprogram parameter.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5304
The ACES shall determine the performance
characteristics of decimal type operations (if
supported).
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5305
The ACES shall determine the performance
impact of referencing and assigning to an
object designated by a subprogram's formal
access parameter.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5306
The ACES shall determine the performance of
assignment of controlled types.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5307
The ACES shall determine the performance of
user-defined finalization when exiting the
scope of a controlled object.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5308
The ACES shall determine the time required
for a call on a protected operation in the
simplest of cases.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5309
The ACES shall determine the performance
characteristics of the requeue statement.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5310
The ACES shall determine the performance
characteristics of the delay until statement.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5311
The ACES shall investigate the performance
characteristics of asynchronous transfer of
control in various scenarios.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5312
The ACES shall determine the performance
characteristics of references to entities in child
packages.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5313
The ACES shall determine the performance
characteristics of the operations in package
Ada.Exceptions.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5314
The ACES shall determine the performance
characteristics of the address arithmetic
operations declared in the package
System.Storage_Elements.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5315
The ACES shall determine the performance
characteristics of the conversion operations
declared in the package
System.Address_To_Access_Conversions.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5316
The ACES shall determine the performance
characteristics of the storage management
operations declared in the package
System.Storage_Pools.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5317
The ACES shall determine the performance
characteristics of passing data among units by
using the stream operations given by
attributes.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5318
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Fixed.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5319
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Bounded.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5320
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Unbounded.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5321
The ACES shall determine the performance
characteristics of the operations declared in
Ada.Numerics.Elementary_Functions and in
an instantiation of
Ada.Numerics.Generic_Elementary_Function
s, using Float.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5322
The ACES shall determine the performance
characteristics of the random value function
declared in instantiations of
Ada.Numerics.Discrete_Random.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5323
The ACES shall determine the performance
characteristics of the shift and rotate
operations provided by package
Ada.Interfaces.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5324
The ACES shall determine the performance
characteristics of calls to subprograms written
in C, C++, Fortran, and COBOL, as supported
by the specific implementation.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5325
The ACES shall determine the performance
characteristics of the operations provided by
package Ada.Interfaces.C.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5326
The ACES shall determine the performance
characteristics of the operations provided by
package Ada.Interfaces.C.Strings.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5327
The ACES shall determine the performance
characteristics of the operations provided by
package Ada.Interfaces.C.Pointers.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5328
The ACES shall determine the performance
characteristics of the operations provided by
package Ada.Interfaces.COBOL.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5329
The ACES shall determine the performance
characteristics of the operations provided by
package Ada.Interfaces.Fortran.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5401
The ACES shall determine the performance
impact of using aliased objects as subprogram
parameters.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5402
The ACES shall determine the performance
effects of declaring objects of unconstrained
types (the initial constraint being taken from
the initialization expression).
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5403
The ACES shall determine whether a
performance cost is incurred when referencing
an extension component of a tagged type.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5404
The ACES shall determine whether a
performance cost is incurred in referencing
and updating arrays and records whose
components are aliased.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5405
The ACES shall determine the relative
performance of dispatching for various kinds
of controlling operands.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5406
The ACES shall determine the performance
assignments to an aliased object and
evaluations of an aliased object.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5407
The ACES shall determine the performance
impact of using general access types to
reference data at different accessibility levels.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5408
The ACES shall determine the performance
characteristics of evaluations of the Access
attribute.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5409
The ACES shall determine the relative
performance of elaboration of private type
and non-private type declarations.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5410
The ACES shall determine the performance
impact of using discriminants for tasks and for
protected objects.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5411
The ACES shall determine the performance
impact of using exception handlers in accept
bodies.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5412
The ACES shall determine the performance
characteristics of exception propagation from
protected subprograms and entries of
protected objects.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5413
The ACES shall determine the performance
impact of the use of representation clauses
that have been added to the language.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5414
The ACES shall determine the performance
characteristics of evaluations of the Bit_Order
attribute.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5415
The ACES shall determine the performance
impact of using unchecked conversion to
produce valid and invalid values of a type.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5416
The ACES shall determine the performance
impact of using the Unchecked_Access
attribute to create access values.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5417
The ACES shall determine the performance
characteristics of evaluations of the
Storage_Pool and Storage_Size attributes for
access subtypes.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5418
The ACES shall determine the performance
characteristics of the operations declared in
package Ada.Characters.Handling.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5419
The ACES shall determine the performance
characteristics of Wide_Character operations.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5420
The ACES shall determine the performance
characteristics of Wide_String operations.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5421
The ACES shall determine the performance
characteristics of conversions between
enumeration or numeric (integer, modular,
float, fixed, decimal) types and Wide_String,
using attributes and instantiations of the
generic IO packages in Wide_Text_IO.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5422
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Wide_Fixed.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5423
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Wide_Bounded.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5424
The ACES shall determine the performance
characteristics of the operations declared in
package Strings.Wide_Unbounded.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-5425
The ACES shall determine the performance
characteristics of the operations declared in
package Streams.Stream_IO. Compare to the
performance of equivalent attributes after
applying the Stream function to create a
Stream_Access value from the file.
CSC
1.3, 1.4,
& 2.1
D
D
OPS-T-6101
Package specifications that do not require
bodies but have them nevertheless shall
include "pragma Elaborate_Body". See the
files "sy_cum??.ada" where ?? is a two-digit
integer ranging from 12 to 22.
CSC 2.1
D
F
OPS-T-6201
The "dt_dp*" series of tests shall issue their
calls to "put_test_name" and "delay_t" from
within the body of a task with the highest
priority. The expression giving the value of
"actual-delay-greater-than-requested" shall
use the ">=" operator. The value of this
variable shall be set to False at the end of each
test.
CSC 2.1
D
F
OPS-T-7101
The following tests shall be renamed:
a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01
CSC
1.3, 1.4,
& 2.1
D
F
OPS-T-7102
The following tests shall be renamed:
a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02
CSC
1.3, 1.4,
& 2.1
D
F
OPS-T-7103
The following tests shall be renamed:
a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03
CSC
1.3, 1.4,
& 2.1
D
F
SPT-A-5201
The Analysis software shall recognize the
names of new performance test groups,
subgroups, tests, and default main programs.
CSC 1.4
A
F
SPT-A-5202
The Menu program shall allow selection of
groups consisting entirely of Ada 95 tests if
and only if at least one selected
implementation has been identified as an Ada
95 implementation.
CSC 1.4
D
F
SPT-A-5203
The Comparative Analysis program shall
incorporate results from new performance test
groups, subgroups, default main programs,
and tests into its analyses and reports.
CSC 1.4
A
F
SPT-A-5204
The Single-System Analysis program shall
provide comparisons of sets of new tests and
of sets containing both new and previously
existing tests as appropriate.
CSC 1.4
D
F
SPT-A-6301
The Analysis software shall provide default
reports which shall be designed to provide the
user with the least amount of information
which would also be usable. The first CA
report shall present the System Factors for all
test groups. The second shall present the
number of tests successfully implemented.
CSC 1.4
D
D
SPT-A-6302
The default metric for Comparative Analysis
reports shall be execution time.
CSC 1.4
D
D
SPT-A-6303
The Analysis software shall provide default
reports which shall be designed to provide the
user with the least amount of information
which would also be usable. The SSA report
shall present a ratio of the smallest to the
largest reported Language Features for each
test comparison.
CSC 1.4
D
D
SPT-A-7301
The ACES shall provide a method to perform
Comparative Analysis on User-Defined
benchmarks.
CSC 1.4
D
F
SPT-A-7302
The ACES shall provide a method to perform
Single-System analysis on User-Defined
benchmarks.
CSC 1.4
D
F
SPT-A-8301
The Menu software shall allow the user to
advance to the next screen without having to
type "n" in addition to the command where
multiple selections before going to the next
screen are not possible.
CSC 1.4
D
D
SPT-A-8302
The Analysis software, on the MAIN MENU
screen, shall tell the user to select a command
or a tool and a command (separating them
with a comma) instead of telling the user to
"select one tool separated by a comma."
CSC 1.4
D
F
SPT-A-8303
The Analysis software shall be designed to
exit gracefully if multiple errors come up after
all selections are made instead of having the
user resort to a keyboard abortion command.
CSC 1.4
D
F
SPT-A-8304
The Analysis software shall be designed to
give adequate error messages and exit
gracefully if it is missing a file so that the user
will know what files are required for it to
function properly. The software shall indicate
if the file is missing or if it cannot locate the
name it was given to find and where to make
that change.
CSC 1.4
A
F
SPT-A-8305
The Analysis software shall be redesigned so
that the GROUPS menu either fits on one
screen or appears on two screens in a way that
is easy to use. Presently the title and "1.
Applications" scroll off screen when Ada_95
is an option.
CSC 1.4
D
D
SPT-A-8306
The Analysis software shall be redesigned so
that the RUN OR SAVE REQUEST menu
will indicate in all cases either that no reports
were chosen (and therefore no output is to be
expected) or what has been chosen and what
will be produced.
CSC 1.4
D
F
SPT-A-8307
The Analysis software shall be revised so that
when a screen indicates that one of a group of
items must be selected, then the user will
really have to make a selection before going
to the next screen.
This occurs on at least the following screens:
- SUMMARY OF ALL GROUPS LEVEL
REPORT OPTIONS:
- SSA MAIN REPORT OPTIONS
- SSA HLS REPORT OPTIONS
CSC 1.4
D
F
SPT-A-8308
The Analysis software shall present a new
menu that is a combination of the present
MAIN and INITIAL menus.
CSC 1.4
D
D
SPT-A-8311
The Analysis software menus shall use
language consistent with the actual reports.
This shall include but not be limited to:
- Selection 7 of the SUMMARY OF ALL
GROUPS LEVEL REPORT OPTIONS,
which indicates on the screen "errors"
and on the report "successes".
CSC 1.4
D
F
SPT-A-8322
The Analysis menus shall be revised as
necessary to be more self explanatory. This
shall include but not be limited to the use of
white space to separate selection areas and the
re-wording of selections to reflect the actual
content of the output.
DOC
D
F
SPT-D-6120
The Primer shall define default options to
help the user who starts at download and ends
with reports. The Primer shall make
accommodations for those users who enter at
an intermediate stage, referencing those files
that are necessary at each point.
DOC
D
D
SPT-D-6121
The Primer shall note that for Pretest, the user
has the option of unzipping only the files
needed to compile and run Setup, Harness and
Analysis. Test files can be added later at the
user's discretion if there is a problem with
space.
DOC
D
F
SPT-D-6125
The Pretest section of the Primer shall be
redesigned so that each step has four parts:
1. PREPARATION: what needs to be done
before executing the step
2. EXECUTION: (if all of the preparation is
complete, user can now run the script)
include comments such as expected time
on a system
3. RESULTS: list of files --- vertical rather
than horizontal. In the case of long lists,
multiple columns can be used.
4. DEFAULT: a special section for those
running in the default mode.
DOC
D
D
SPT-D-6126
The Primer shall indicate that the user need
only unzip the support.zip file, go to the
"working" directory, make a library, compile
and link the Setup files and run Setup. After
that the automated steps can be executed.
DOC
D
D
SPT-D-6128
The Quick-Look section of the Primer shall
be redesigned so that each step has four parts:
1. PREPARATION: what needs to be done
before executing the step
2. EXECUTION: (if all of the preparation is
complete, user can now run the script)
include comments such as expected time
on a system
3. RESULTS: list of files --- vertical rather
than horizontal. In the case of long lists,
multiple columns can be used.
4. DEFAULT: a special section for those
running in the default mode.
DOC
D
D
SPT-D-6129
The Primer shall indicate that the user need
only unzip the support.zip file, go to the
"working" directory, make a library, compile
and link the Quick-Look Setup file and run
the Quick-Look version of Setup. After that
the automated steps can be executed.
DOC
D
D
SPT-D-6207
The Primer, Section 4, shall reflect the
Harness modifications described elsewhere in
this document. In particular, the algorithm
shall be modified to add a description of the
additional activities described in requirement
ID's SPT-H-(6201-6206).
DOC
A
N/A
SPT-D-6208
Section 4.1.1 of the Primer shall describe any
additional prerequisites as a result of the
requirements of requirement ID's SPT-H-
(6201-6206).
DOC
A
N/A
SPT-D-6209
The Primer shall contain one or more sections
(probably numbered 4.1.2, with later sections
renumbered) describing the predefined test
selection process described in requirement
ID's SPT-H-(6201-6206).
DOC
A
N/A
SPT-D-6210
Section 6.1 of the User's Guide shall reflect
the modifications to Harness described in
requirement ID's SPT-H-(6201-6206).
DOC
A
N/A
SPT-D-6211
Section 6.3 of the User's Guide shall be
modified to reflect the modifications to
Harness described in requirement ID's SPT-
H-(6201-6206).
DOC
A
N/A
SPT-D-6212
The User's Guide shall contain a new section
(probably numbered 6.4, with later sections
renumbered) describing the predefined test
selection added to Harness.
DOC
A
N/A
SPT-D-7401
The ACES documentation shall provide a
detailed description of how benchmark tests
are currently constructed.
DOC
D
F
SPT-D-7402
The ACES documentation shall provide a
detailed description of what a user must do in
order to create a user defined benchmark.
DOC
D
F
SPT-D-7403
The ACES documentation shall provide a
detailed description of what the user must do
in order to incorporate a user defined
benchmark into the current ACES structure.
DOC
D
F
SPT-D-7404
The ACES documentation shall provide a
detailed description of what the user must do
in order to capture the results generated by a
user defined benchmark.
DOC
D
F
SPT-D-7405
The ACES documentation shall provide a
detailed description of what the user must do
in order to perform Comparative Analysis
using results from a user defined benchmark.
DOC
D
F
SPT-D-7406
The ACES documentation shall provide a
detailed description of what the user must do
in order to perform Single System Analysis
using results from a user defined benchmark.
DOC
D
F
SPT-D-7407
The ACES documentation shall provide
examples of the steps that the user must
perform when incorporating user defined
benchmarks.
DOC
D
F
SPT-D-8101
The Primer shall indicate to the user at each
step what files must be present before that
step can be executed. This shall be indicated
either directly or in the form of an exact
reference to the required information. The
user shall not be expected to be conversant
with all of the documentation before reaching
the point in question. This shall be consistent
with each possible combination of choices
and shall include but not be limited to Section
3.2.4.a which does not differentiate between
what files need to be in the directory if CPU
rather than EL is selected.
DOC
D
D
SPT-D-8102
The Primer shall present a cross reference
index for ACES Documentation that is
verified and updated.
DOC
D
F
SPT-D-8103
The Primer documentation shall list at the end
of Section 3 all of the referenced files, their
actual file names, and which steps use them.
DOC
D
D
SPT-D-8104
The Primer shall properly and clearly
document the corresponding software. This
will eliminate existing errors and clarify
existing documentation. These shall include
but will not be limited to the following:
- Section 2.1.2: Clarify references to
Harness tests or testing Harness.
- Section 2.2: Incorrect counts of software
and file names when it refers to
"Versions" of ACES and to seven
compressed files.
DOC
D
F
SPT-D-8105
The Primer shall define all Primer-specific
notation. This shall include but is not limited
to the last paragraph of Section 2 which gives
no sample for # sign but uses it in Section
2.1.3 and other places.
DOC
D
F
SPT-D-8106
The User's Guide shall properly and clearly
document the corresponding software to
eliminate existing errors and clarify existing
documentation. These shall include but will
not be limited to the following:
- Section 5.1.4 (first bullet) which refers to
Ada 95 MATCH packages in places
rather than Ada 95 MATH packages.
DOC
D
F
SPT-D-8108
The Primer shall indicate the room needed to
house the uncompressed files as in Section
2.2, paragraph 4 ("The distributed...working
directory.") and elsewhere as needed.
DOC
D
F
SPT-D-8109
The Primer shall indicate the correct
orientation of files. If they are unzipped
according to instructions, an "aces"
subdirectory will be created off the user's
directory and the working, source, support,
tests, assessrs, qlook, and docs subdirectories
will be created off the newly created aces
directory. The individual test subdirectories
will be created off the tests directory and the
individual assessor subdirectories will be
created off the assessrs directory. This is
incorrect in the present Figure 2-1 and in the
text following that figure.
DOC
D
F
SPT-D-8110
The Primer shall indicate at what point the
Setup executable can actually be run.
Sections 3.1, 3.1.1, and 3.1.2 say what to do
before running Setup and what to do once
Setup is run but the user can read on for
several paragraphs before realizing that setup
should have been executed.
DOC
D
F
SPT-D-8111
The Primer shall clarify the directions for
UNIX users. These shall include but not be
limited to:
- Section 3.1.3.2 paragraph 1: The
copying and renaming are now done
automatically;
- Section 3.1.3.2 paragraph 3: Bourne
shell or C shell; and
- Section 3.1.3.2 paragraph 4: Need to
mention that the example given will run
in the back ground.
DOC
D
F
SPT-D-8112
The Primer shall indicate instances in which
there may be a difference in what the user can
expect when running Ada 83 as opposed to
running Ada 95. These shall include but not
be limited to:
- Section 3.2.11.c: If one is running an
Ada 95 compiler, the files specific to Ada
95 will be present as well.
DOC
A
F
SPT-D-8113
The Primer shall indicate which files will be
automatically deleted in each step, and which
can or should be manually deleted. This shall
include but not be limited to Section 3.2.e.
DOC
D
F
SPT-D-8114
The User's Guide shall present a clear and
succinct description of each item covered.
This shall include but not be limited to
Section 5.1.1 paragraphs one and two which
need to be rewritten and combined. The
second paragraph looks as if it should be in
the middle of the first.
DOC
D
F
SPT-D-8115
The Primer shall make better and more
consistent use of CAPITALIZATION,
underlines, italics, and boldface. This shall
include but not be limited to Section 3.2.5.a in
which "source" should be in boldface.
DOC
D
F
SPT-D-8118
The Quick-Look section of the Primer shall
match the Pretest section where appropriate.
Note Primer 8.2.2.2.d.
DOC
D
F
SPT-D-8202
The User's Guide shall be careful of
maintaining adequate references so that they
are complete and accurate. This shall include
but not be limited to:
- The last paragraph of Section 5.4.3 which
refers to Sections 9.1.6 and 6.13 of the
User's Guide as if they are in the
Reader's Guide and gives no indication as
to where in the Reader's Guide one can
find a discussion of the interpretation of
such modified tests.
DOC
A
N/A
SPT-D-8203
The Harness documentation shall list at the
end of each section all default file names,
their original locations, when they are needed,
how they are referenced, when they are
revised by a specific part of the program, and
if the user changes the file name, where else
changes need to be made to be consistent.
DOC
A
N/A
SPT-D-8204
The User's Guide shall warn the user that if
either the "test.sel" or the "test.sum" file is
corrupted, Harness will exit gracefully but the
condition can be cured by deleting the
offending file and running Harness without it.
Harness will happily build another.
References are made in Section 6.2.5 and
Section 6.2.6.
DOC
A
N/A
SPT-D-8205
The User's Guide shall explain "status" in
reference to Harness as referenced in Section
6.4.4, Section 6.4.5, Section 6.5.1 and
elsewhere. If "status" is explained in detail
elsewhere, then that section needs to be
referenced at this point.
DOC
A
N/A
SPT-D-8206
The User's Guide shall clarify whether the
"current subgroup" can be in a different group
than the "current group". This shall include
but not be limited to:
- Section 6.5.12
DOC
A
N/A
SPT-D-8207
The User's Guide shall demonstrate
consistency between its text and its graphics.
This shall include but not be limited to:
- Section 6.5.2 which refers to "in the
groups menu below, the classical group
... number (3)" however, the screen in
Figure 6-4 shows groups 11-17.
DOC
A
N/A
SPT-D-8208
The User's Guide shall fully explain each and
every file it references. This shall include but
not be limited to:
- Section 6.9.3 which displays in Figure 6-
11 the result of the Get_Test_Name_List
(GT) command and prompts the user for
a "Test file name" but does not indicate
what information in what format should
be in that file and whether there is a
default or sample file that the user can
reference.
DOC
A
N/A
SPT-D-8209
The User's Guide shall provide the user with
adequate information on how to modify files.
This shall include but not be limited to text
based data files (e.g., zh_??.txt, za_cosys.txt,
zh_??.lib) that the user is to modify but that
expect a specific number of white space
delimiters between what appear on the screen
to be columns of figures.
DOC
A
N/A
SPT-D-8210
The User's Guide shall present information
about the ACES in a clear, concise, logical
and factual manner. This clarification shall
include but not be limited to:
- Section 6.13.
- The problem referred to in the last
paragraph of Section 6.1 which mentions
an "inconsistency in the Harness".
DOC
A
N/A
SPT-D-8309
The Analysis software shall define the help
menu for BUILD/MODIFY SYSTEM NAME
FILE (GENERAL INFORMATION) so that
the file names explained there are the same as
those that appear in selections and make other
corrections as needs so that the help menus
correspond to the screens they serve.
CSC 1.4
D
F
SPT-D-8310
The Analysis documentation shall list at the
end of each section all default file names,
their original locations, when they are needed,
how they are referenced, when they are
revised by a specific part of the program, and
if the user changes the file name, where else
changes need to be made to be consistent.
DOC
D
F
SPT-D-8312
The Primer shall give clear, concise, and
logical explanations for ACES. This shall
include but not be limited to the following
section:
- Section 3.2.12.c: "introduces this step" --
- for which paragraph and for what
purpose?
- Section 3.2.13.e.3: system1 is an Ada 83.
What if they are testing an Ada 95?
DOC
D
F
SPT-D-8313
The User's Guide shall provide thorough
explanations of all of the possibilities to
which it refers or provide adequate reference
to resources in which the information can be
easily found. This shall include but not be
limited to:
- Section 9.3 par 4 refers to za_anlys.com
which cannot be found.
- Figure 9-27 indicates that the
View/Modify systems selection is
number 5 when it is actually two more
than the number of systems listed.
DOC
D
F
SPT-D-8314
The User's Guide shall fully explain the
importance of data files, how to manually
construct them, what programs need which
ones and will modify them, and where
sample/dummy files can be found. This shall
include but not be limited to:
- Section 9.3.2 tells the user to construct
the file za_cosys.txt and refers the reader
to Section 9.1.4 but both sections fail to
mention the fact that a dummy comes
with the support.zip file and can be easily
modified when running the Analysis
MENU program.
DOC
A
F
SPT-D-8315
The User's Guide shall fully explain how
Condense, CA and SSA work both with the
Menu and separately and what files are
needed for either menu or independent use.
This shall include but not be limited to
clarification of several places in Section 9,
including Section 9.1.1, which refers to
Figure 9-1 and the fact that Condense, CA
and SSA can be run independently of the
Menu but give no more details as to where the
files indicated in Figure 9-1 can be found. It
also needs to cross reference the fact that Step
12 in the Primer will compile this
automatically. It also refers to
"za_anlys.com" which is not part of the test
suite. Much of the information in Section 9
tends to ignore the fact that the Primer and
the menu created in Step 12 of Pretest exist,
as well as the fact that running the files
created by Harness will produce log files that
can be used by Analysis.
DOC
D
F
SPT-D-8316
The Reader's Guide shall clarify its use of the
phrase "classical benchmark tests" by making
appropriate reference to the Classical (Cl_*)
performance tests. It shall further refer to
"benchmark test suite" as the "ACES
performance tests". This shall include, but
not be limited to:
- the Abstract
- Section 2.4.2.4: The paragraph
"Represent classic benchmark ...
available for other languages"
- Section 3.2.8.1
- Section 5, paragraph 1
- Section 5.4.2, paragraph 2
DOC
D
F
SPT-D-8317
The Reader's Guide shall list at the end of
each section all default file names, their
original locations, when they are needed, how
they are referenced, when they are revised by
a specific part of the program, and if the user
changes the file name, where else changes
need to be made to be consistent. This shall
include but not be limited to:
- Section 5.2.2 in which no default names
are given for the No Data, Exceptional
Data, or Multiple Results Reports.
DOC
D
F
SPT-D-8318
The Reader's Guide shall give a listing of all
of the possible Analysis reports and the
implications of the results along with cautions
as to what could cause differences that would
make the report misleading. This shall
include clarification of but not be limited to:
- Section 5.3.2.1.2 (and others) which refer
to Figure X-x, name the report that is
displayed and sometimes give a short
explanation of what each shows. These
should be expanded in their definitions
and separated so that they follow the
reports that they describe.
DOC
D
F
SPT-D-8319
The Reader's Guide shall accurately reflect
changes caused by new versions. This shall
include but not be limited to:
- The third paragraph (Gps:) of Section
5.3.2.2.3 which refers to the number of
defined groups as 17.
DOC
D
F
SPT-D-8320
The Reader's Guide shall explain in
appropriate detail each of the possibilities to
which it refers. This shall include but not be
limited to:
- The fifth paragraph of Section 5.3.2.2.3
(NoData:) which refers to the user
entering the appropriate code but does
not tell what codes are appropriate or
where they can be entered or what will
happen if they are or are not entered.
DOC
D
F
SPT-D-8321
The User's Guide shall adequately explain all
of the adaptations and limitations. This shall
include but not be limited to Section 9.6 and
especially the last limitation ("The Analysis...
batch mode"). Since only 17 groups can now
be displayed on a crt screen, what is meant by
menu, (26) items, (25) systems, and groups.
In addition, references to sections explaining
requirements shall be instituted.
DOC
D
F
SPT-D-8401
The Primer shall correctly, concisely and
adequately explain the Quick-Look section of
the ACES. This shall include but not be
limited to:
- the last sentence of the paragraph
following Section 8.2.2.j which refers to
"compilable Ada --".
DOC
D
F
SPT-D-8501
The Reader's Guide shall present information
in a clear, concise, logical, and factual
manner. This shall include, but not be limited
to:
- Section 3.2.2: This section refers to
pragmas in the first list, but does not
indicate which list is the first one.
- Section 3.2.2: All lists shall be vertical.
If they are too long, multiple columns are
acceptable.
- Section 3.2.4.1.7: The statistics in
paragraph 3 are incorrect. They should
be the opposite, i.e., exchange the words
inner and outer.
- Section 3.2.7.3: "...until the I/O
completes" transitive verb needs an
object.
DOC
D
F
SPT-H-5101
The Harness shall recognize new performance
test groups and subgroups, as determined by
new and renamed performance tests.
CSC 1.3
D
F
SPT-H-5102
The Harness shall recognize names of new
default main programs that incorporate new
and renamed tests.
CSC 1.3
D
F
SPT-H-5103
Main programs that incorporate new or
renamed tests shall not include any previously
existing tests.
CSC 1.3
D
F
SPT-H-5104
The Harness shall display all new test names
that are appropriate for the implementation
being tested.
CSC 1.3
D
F
SPT-H-5105
The Harness shall display tests specific to Ada
95 only if the user has identified the
implementation as an Ada 95 implementation.
CSC 1.3
D
F
SPT-H-5106
The Harness shall display group and subgroup
names only if the groups and subgroups
contain tests applicable to the language
version of the implementation under test.
CSC 1.3
D
F
SPT-H-6201
The Harness software shall provide the user
an option of selecting tests based on the
performance issues of interest to the user.
CSC 1.3
D
D
SPT-H-6202
The Harness shall provide the user with the
option of bypassing the performance issues
menu and proceeding directly to the Harness
menu.
CSC 1.3
D
D
SPT-H-6203
The Harness shall provide a default set of
performance issues, which shall be
preselected when the performance issues
menu first appears.
CSC 1.3
D
D
SPT-H-6204
The Harness shall allow the user to select or
de-select any of the performance issues.
CSC 1.3
D
D
SPT-H-6205
The Harness shall provide at least the
following options on the performance issues
menu:
- select all performance issue selections;
- deselect all performance issue selections;
- request help; and
- quit the performance issues menu (exit to
the Groups menu).
CSC 1.3
D
D
SPT-H-6206
If the performance issues menu is not used,
the ACES shall work in the same manner as in
ACES, Version 2.0. If a test.sel file exists,
Harness shall use that as the initial test
selection. Otherwise, no tests will be selected
when the Harness menu is initially displayed.
CSC 1.3
D
D
SPT-H-7201
The Harness shall provide a mechanism for
selecting User-Defined benchmarks in a
fashion similar to the way that tests are
currently handled in the ACES.
CSC 1.3
D
D
SPT-H-7202
The Harness shall provide a mechanism for
generating command scripts that will execute
User-Defined benchmarks in a fashion similar
to the way that command scripts are currently
generated by the Harness.
CSC 1.3
D
D
SPT-H-8201
The Harness software shall indicate, in its
error messages in UPDATING, the name of
the file causing the error rather than the name
of the file following the file that caused the
error.
CSC 1.3
D
F
SPT-P-6101
The Pretest software shall provide the user
with the opportunity to exercise the same
degree of detailed control of the pretest
activity process that is provided by the
Version 2.0 software.
CSC 1.1
A
F
SPT-P-6102
The number of questions presented to the user
by the Pretest software shall be minimized by
the use of default values and values that are
specific to the operating system and compiler
family being tested.
CSC 1.1
A
D
SPT-P-6103
The Pretest software shall base the default
values for the following upon the operating
system and compiler family being used. If no
operating system or compiler family is
identified, the user shall be required to supply
the values.
* Command to execute a compiled program
on the host system or on a self-hosted
system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script
file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory
paths;
* Symbol used as the terminator of a
directory path;
* Special first line of a script file (allow for
multiple lines);
* Special symbol for the beginning of a
script file line;
* Symbol or word to indicate the beginning
of a comment line;
* Command to echo a string or value to the
output file;
* Delimiter for a string being echoed to the
output file;
* Suffix used to direct echoed string to a
particular file;
* Symbol for the first parameter to a script
file;
* Special form of the first parameter
symbol for use in an echoed string;
* Symbol for the second parameter to a
script file;
* Command to identify the Ada library for
compiling to run on the host system (or a
self-hosted system);
* Suffix expected for an Ada source code
file;
* Command to compile a program (with
default options) to be executed on the
host system (or a self-hosted system);
* Command to compile (with full
optimization) a program to be run on the
target system (or a self-hosted system);
* Command to compile (with no
optimization) a program to be run on the
target system (or a self-hosted system);
* Command to compile (with full checking
enabled) a program to be run on the
target system (or a self-hosted system);
* Command to link a program to run on the
target system (or a self-hosted system);
* Command to delete a compilation unit
from the current Ada library;
* Command to delete the body (only) of a
compilation unit from the current Ada
library; and
* Command to delete the specification
(only) of a compilation unit from the
current Ada library.
CSC 1.1
A
D
SPT-P-6104
For commands and values that depend on the
operating system and compiler family, the
Pretest software shall make use of default
responses applicable to the following
combinations:
Operating System Compiler Family
DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS
CSC 1.1
A
D
SPT-P-6105
The Pretest software shall provide instructions
or shall refer to documentation containing
instructions for providing user-defined
defaults for the commands and values that are
specified for the above (SPT-P-6104)
combinations of operating system and
compiler family.
CSC 1.1
A
F
SPT-P-6106
The Pretest software shall ask the user to
provide the following information in all cases:
* The operating system being used (for the
host system);
* The compiler family being used;
* Whether the system is for cross-
compilation or is self-hosted;
* Whether the system implements Ada 83
or Ada 95; and
* The directory path for the "aces" root
directory.
CSC 1.1
D
D
SPT-P-6107
The Pretest software shall assume the
indicated values for the following:
* Execution time measurement (EL:
Elapsed time)
* Compilation time measurement (EL:
Elapsed time)
* Code size measurement technique
(L: Label'Address)
* Host or self-hosted executable suffix
(.exe)
* Bit size of largest integer type (32)
* Default file name extension for script
files will be:
- DOS - .bat
- UNIX - .unx
- VMS - .com
CSC 1.1
A
D
SPT-P-6108
The Pretest software shall assume that the
Portable math packages are to be used for the
default setting if the system is an Ada 83
implementation; otherwise, it shall assume the
existence of the math packages required by
the Ada 95 standard.
CSC 1.1
A
D
SPT-P-6109
For cross-compilation systems only, the
Pretest software shall require the user to
provide the following commands and values:
* Command to define the Ada library for
target system compilations;
* Command to compile a program to be
run on the target system;
* Command to link a program to be run on
the target system;
* Suffix for an executable file linked to run
on the target system;
* Command to execute a compiled program
on the target system;
* Command to delete the specification
(only) of a unit from the target library;
* Command to delete the body (only) of a
unit from the target library; and
* Command to delete both the specification
and body of a unit from the target library.
CSC 1.1
A
D
SPT-P-6110
The Pretest software shall neither ask for nor
use the following information that is
requested by the Version 2.0 software:
* Command to execute a program under
the debugger;
* Command to capture the status of a
completed process;
* Command to export the status of a
completing process;
* Command to compile with debugging
enabled;
* Command to link a program for
execution under the debugger;
* Method of managing the input and output
files for the Include tool;
* Command for defining a logical symbol;
and
* Operator used in defining a logical
symbol.
CSC 1.1
A
D
SPT-P-6122
The support.zip file shall be revised to unload
the initial files needed for Pretest into a
"working" subdirectory of the "aces"
directory.
CSC 1.1
D
D
SPT-P-6123
Pretest shall give the user the opportunity to
modify the default as well as the previously
constructed environment file by rerunning
Setup.
CSC 1.1
D
D
SPT-P-6124
Setup shall be modified so that the Ada source
files required for Pretest will be copied from
the distribution subdirectories to the "source"
subdirectory where they can be accessed for
compilation. All other required files shall be
copied from the distribution directories (note
SPT-D-8109) to "working". This will be done
at the beginning of the Pretest activity and all
files will be renamed at that point if needed.
CSC 1.1
D
D
SPT-P-8107
Setup shall inform the user which (83/95)
compiler version is selected in any existing
environment file. It now only informs the
user of the selection of cross/self compiler.
CSC 1.1
D
D
SPT-P-8116
Setup shall make a backup copy of any
existing environment file before it offers the
user a chance to make changes. It shall also
offer the user the chance to revert to the old
version by replacing quit with two choices:
(1) generate new file with the new selections;
or (2) exit and save none of the changes; go
back to the old version.
CSC 1.1
D
D
SPT-P-8117
The Pretest software unit "zg_incld" (four
versions) shall specify the type
CHARACTER for at least one of the literals
in the code "for i in 'a' .. 'z' loop".
CSC 1.1
A
D
SPT-Q-6111
The Quick-Look software shall provide the
user with the opportunity to exercise the same
degree of detailed control of the Quick-Look
activity process that is provided by the
Version 2.0 software.
CSC 1.5
A
F
SPT-Q-6112
The number of questions presented to the user
by the Quick-Look software shall be
minimized by the use of default values and
values that are specific to the operating
system and compiler family being tested.
CSC 1.5
A
D
SPT-Q-6113
The Quick-Look software shall base the
default values for the following upon the
operating system and compiler family being
used. If no operating system or compiler
family is identified, the user shall be required
to supply the values.
* Command to execute a compiled program
on the host system or on a self-hosted
system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script
file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory
paths;
* Symbol used as the terminator of a
directory path;
* Special first line of a script file (allow for
multiple lines);
* Special symbol for the beginning of a
script file line;
* Symbol or word to indicate the beginning
of a comment line;
* Command to echo a string or value to the
output file;
* Delimiter for a string being echoed to the
output file;
* Suffix used to direct echoed string to a
particular file;
* Symbol for the first parameter to a script
file;
* Special form of the first parameter
symbol for use in an echoed string;
* Symbol for the second parameter to a
script file;
* Command to identify the Ada library for
compiling to run on the host system (or a
self-hosted system);
* Suffix expected for an Ada source code
file;
* Command to compile a program (with
default options) to be executed on the
host system (or a self-hosted system);
* Command to compile (with full
optimization) a program to be run on the
target system (or a self-hosted system);
* Command to link a program to run on the
target system (or a self-hosted system);
and
* Command to delete a compilation unit
from the current Ada library.
CSC 1.5
A
D
SPT-Q-6114
For commands and values that depend on the
operating system and compiler family, the
Quick-Look software shall make use of
default responses applicable to the following
combinations:
Operating System Compiler Family
DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS
CSC 1.5
A
D
SPT-Q-6115
The Quick-Look software shall provide
instructions or shall refer to documentation
containing instructions for providing user-
defined defaults for the commands and values
that are specified for the above combinations
of operating system and compiler family.
CSC 1.5
A
F
SPT-Q-6116
The Quick-Look software shall ask the user to
provide the following information in all cases:
* The operating system being used (for the
host system);
* The compiler family being used;
* Whether the system is for cross-
compilation or is self-hosted;
* Whether the system implements Ada 83
or Ada 95; and
* The directory path for the "aces" root
directory.
CSC 1.5
D
D
SPT-Q-6117
The Quick-Look software shall assume the
indicated values for the following:
* Host or self-hosted executable suffix
(.exe)
* Bit size of largest integer type (32)
* Default file name extension for script
files will be:
- DOS - .bat
- UNIX - .unx
- VMS - .com
CSC 1.5
A
D
SPT-Q-6118
For cross-compilation systems only, the
Quick-Look software shall require the user to
provide the following commands and values:
* Command to define the Ada library for
target system compilations;
* Command to compile a program to be
run on the target system;
* Command to link a program to be run on
the target system;
* Suffix for an executable file linked to run
on the target system;
* Command to execute a compiled program
on the target system;
* Command to delete the specification
(only) of a unit from the target library;
* Command to delete the body (only) of a
unit from the target library; and
* Command to delete both the specification
and body of a unit from the target library.
CSC 1.5
A
D
SPT-Q-6119
The Quick-Look software shall neither ask for
nor use the following information that is
requested by the Version 2.0 software:
* Method of managing the input and output
files for the Include tool;
* Command for defining a logical symbol;
and
* Operator used in defining a logical
symbol.
CSC 1.5
A
D
SPT-Q-6127
Setup shall be modified so that the Ada source
files required for Quick-Look will be copied
from the distribution subdirectories to the
"source" subdirectory where they can be
accessed for compilation. All other required
files shall be copied from the distribution
directories (note SPT-D-8109) to "ql_work".
This will be done at the beginning of the
Quick-Look activity and all files will be
renamed at that point if needed.
CSC 1.5
D
D
SPT-Q-6130
The support.zip file shall be revised to unload
the initial files needed for Quick-Look into a
"ql_work" subdirectory of the "aces"
directory.
CSC 1.5
D
D
SPT-Q-7101
The Quick-Look facility shall be flexible
enough to allow additional User-Defined
benchmark tests to be added.
CSC 1.5
A
D
SPT-Q-7102
The Quick-Look facility shall be able to
report on the results of additional User-
Defined benchmark tests.
CSC 1.5
SPT-Q-8402
The Quick-Look software unit "zg_incld"
shall specify the type CHARACTER for at
least one of the literals in the code "for i in 'a'
.. 'z' loop".
CSC 1.5
A
D
Table 7.0-2 Requirements Sources
Document Code
Document Title
SPT
Usability Enhancements Document for the ACES Version 2.1
OPS
Operational Software Objectives Document for the ACES Version
2.1
8.0 NOTES
8.1 Abbreviations, Acronyms
ACEC...Ada Compiler Evaluation Capability
ACES...Ada Compiler Evaluation System
CA...Comparative Analysis
CSC...Computer Software Component
CSCI...Computer Software Configuration Item
CSU...Compiler Software Unit
N/A...Not Applicable
PDL...Program Design Language
SDD...Software Design Document
SSA...Single System Analysis
SPS...Software Product Specification