A Proposal for

		       Porting the Common Lisp Test Suite

				      - - -

				  [Proposal #5]

				  Sept. 23, 1988 

			       David M. J. Saslav

				      - - -

Introduction:

	A significant portion of Common LISP functionality is not tested by the
	existing Common LISP regression test suite (found in SYS:VALID;).  This
	suite is run whenever a new system has been compiled.

Proposal:

	Part I:

	Spend one day or so surveying the test suite to determine which
	functionality is without test/validation sequences, attaching
	programmer-time	estimates for each major category of tests not
	currently in existence. (Already underway)

	Part II:

        Write test/validation regression tests for that portion of overall
	CommonLISP functionality not currently covered by the existing Common
	LISP test suite.

[Keith comments:]

If we do not expand and utilize the regression testing suite, the first
comprehensive test of the complete Falcon system will occur when all, or
most, of the modules have been ported.  But the core software can be tested
and verified far in advance, as well as other modules such as Flavors (but
that would be a separate proposal).  Then we can have an early indication
of the scope and nature of newly-introduced problems.  

My experience with the existing suite is favorable, despite the 2 days of
work it took to fix the drivers.  When I first brought it up on the Lambda,
we found over 20 bugs and numerous examples of slightly anomalous, but
acceptable, behaviour (which we documented for future reference).

Granted, this kind of testing is supplemental to the testing that every
developer is responsible for within their own domain.  But the regression
suite should serve as a collecting place for these tests, then they can be
run every time the system changes substantially (i.e. when the system is
recompiled).  That is the meaning of the phrase "regression" -- these tests
are only designed to make sure the state of the software does not
"regress".  It can happen; bug fixing can always introduce new bugs.

In our case, the phrase "regression" has an important meaning:  we must
ensure that the Falcon software is at least as good as, if not better than,
the existing Lambda system.