SCOUG-Programming Mailing List Archives
Return to [ 02 |
January |
2004 ]
>> Next Message >>
Content Type: text/plain
We all want to produce quality software. We want to
produce correct software, a match of the solution set to the
problem set, specifications to requirements. We want to
produce error-free software, free of errors of omission
(missing logic) and commission (incorrect logic). We want
code optimized for space and time: no more, no less than
necessary.
Our primary means for achieving this lies within the testing
phase of QA. By implication QA includes more than testing,
e.g. supporting documentation.
We have two aspects with testing. We have one relative to
detecting errors in execution, thus errors in logic (omission
and commission). We have another relative to correcting
errors in execution, thus errors in logic (omission and
commission). I engage in redundancy here to emphasize that
we have only two types of possible errors related to logic. I
also want to point out that all software which performs
exactly according to the logic of its source is error-free:
software does not create errors, people do.
Now detecting errors requires the creation of test data, data
expected to give correct (or expected) results and data
expected to give incorrect (or unexpected) results. When
correct data gives incorrect results or incorrect data gives
correct results we have a logic error, again either of omission
or commission, again people- not software-based.
Historical wisdom indicates that we cannot exhaustively test
logic because due to time or money constraints we cannot
"afford" to, one, create the test data and, two, to execute
the test even if we could create the test data. To
compensate for this developers attempt to distribute the cost
in time or money. They create artifices like alpha and beta
versions to distribute the load on alpha and beta testers.
Even with thousands of alpha and beta testers, if you believe
the "boasts" of open source and Linux advocates, they still
cannot provide exhaustive testing prior to the release of an
"official" (not alpha or beta) release.
Beyond this we have a common guideline: whatever cost
estimates (time and money) you give for construction, double
it for testing. In effect this triples the cost of producing the
source. Note this guideline accepts the inability to
exhaustively test, detect, and correct source prior to an
"official" release. This inability persists whether you triple,
quadruple, or further multiply the time and money in testing.
The historical wisdom blinds us to the fact that this inability,
which again is people-based, comes directly from, one, the
use of imperative programming languages and, two, from
people-generated test cases, i.e. test data. This historical
wisdom does not take into account our everyday experience
with logic programming, specifically that of SQL.
Before you protest too strongly look again what SQL does as
an example of logic programming or declarative languages in
general. SQL invokes the two-stage proof engine of logic
programming, the completeness proof and the exhaustive
true/false proof. The first assures that the input source
contains enough to contruct the logic for the query (analysis)
and then constructs (design and construction) it. That covers
the first four stages of the SDP: specification (query source),
as well as analysis, design, and construction (completeness
proof). It then automatically executes the exhaustive
true/false proof, the testing.
True, it doesn't automatically generate the test data. No one
reading this, however, should have any doubts that the output
includes "all" true instances of the test data (tables, rows,
and columns) while rejecting "all" false instances. It does
produce error-free code. No one here should disagree that
people through commission or omission produce any logic
errors. If we have an interest in "all" false instances, we
need only "not" the contents of the "WHERE" clause.
SQL uses "clausal" logic dictated by its purposes of finding
true instances of known (or existing) data. Logic
programming supports an alternate form known as "predicate"
logic in which the data rules include the valid range of values
for the variables. As you can have multiple variables the
software generates, i.e. enumerates, all possible combinations
of all possible values of all variables: it automatically
generates all possible "true" value sets. It could just as easily
generate through the same "not" logic we could employ with
SQL all possible "false" value sets.
If we get a "false" result from within a "true" value set, we
have incorrect (or missing) logic. If we get a "true" result
from within a "false" value set, we have incorrect (or missing)
logic. In either case the software itself generates the
"exhaustive" test data.
Unfortunately the actual validation remains a people process.
The amount of either "true" or "false" results could
overwhelm us in the amount of time (which equates
frequently to money) required, thus contributing to the cost
of the software. However, we should lay to rest the historical
wisdom that states the impossibility of performing exhaustive
testing, of generating all possible test cases, test data.
Now a byproduct of this lies in eliminating the need for either
alpha or beta versions of software. You would only need in
either case to distribute the results of execution to somehow
allow their parallel review.
You gain two things from this process. One, you restrict
errors of omission or commission to requirements and
specification, both human writing processes. Once
incorporated as a specification no further mistranslation by
the software can occur. This produces, as in SQL, error-free
code. Two, you have exhaustively tested it with automatic
detection of false instances of true input and true instances of
false input.
The only time-consuming issue remaining lies the validating
the volume of false instances of true input and true instances
of false input. We already have developed techniques of
reducing this: variable on boundary, in boundary, and out of
boundary values.
This should deal effectively with issues regarding error
detection through exhaustive testing. The next part hopefully
will do the same for correction.
=====================================================
To unsubscribe from this list, send an email message
to "steward@scoug.com". In the body of the message,
put the command "unsubscribe scoug-programming".
For problems, contact the list owner at
"rollin@scoug.com".
=====================================================
>> Next Message >>
Return to [ 02 |
January |
2004 ]
The Southern California OS/2 User Group
P.O. Box 26904
Santa Ana, CA 92799-6904, USA
Copyright 2001 the Southern California OS/2 User Group. ALL RIGHTS
RESERVED.
SCOUG, Warp Expo West, and Warpfest are trademarks of the Southern California OS/2 User Group.
OS/2, Workplace Shell, and IBM are registered trademarks of International
Business Machines Corporation.
All other trademarks remain the property of their respective owners.
| 
