SCOUG-Programming Mailing List Archives

<< Previous Message << >> Next Message >>

Date:	Fri, 2 Jan 2004 14:51:48 PST8
From:	"Watson, Dave" <david.watson@gd-ais.com >
Reply-To:	scoug-programming@scoug.com
To:	< "'scoug-programming@scoug.com'" > scoug-programming@scoug.com >
Subject:	SCOUG-Programming: QA equals testing, Part One:Detection

Content Type: text/plain

Lynn said "For the record there are no code bugs, only buggy people..."
Maybe if programmers used better personal hygeine, their code would be
better? Present company excluded, of course...

And: "You can only have two sources of error: (1) the requirements or (2)
their translation into specifications." Nice concept, but not very helpful
in the real world. Having spent a few years writing requirements, I found
that it's pretty hard to define the requirement adequately, up front, and
the waterfall of the development process results in a refinement of the
requirement manifested in the sequential specs. The code becomes a
sulf-fulfilling requirement baseline, no matter what the original spec.
Probably accidental if it actually matches the designer's intentions. And
assuming that the requirement or the translation (or both) are flawed, the
trick becomes actually finding those defects, because it's a slam dunk
certainty you'll have defects both in the requirement and in its
translation. Testing usually tries to verify that the original requirements
spec was satisfied in the delivered code, which is usually a mental
masturbation that only results in justifing the existence of the testing
mafia and making the managers who didn't know any better (most of 'em, in my
opinion) feel like they've managed something. I've been yelled at bunches
of times for code that passed all the tests. Maybe add a corollary that
there are two reasons for delivered error -- (1) error by designer or coder
or tester, or (2) something nobody thought of, e.g., 2-digit dates,
unchecked buffers, etc.

-----Original Message-----
From: pskye@peterskye.com [mailto:pskye@peterskye.com]

... perhaps you can straighten *me* out.

So "validation" is sort of like "take it for a test drive and see how it
handles," yes? You already know the program runs; you just want to see
if the public will think it's a Jag ...
--------------------------

I never realized Peter wasn't straight. Not that it matters -- don't ever
change, we love ya no matter what.

I don't think Lynn was defining validation that precisely (discussing beta
testing, I believe), just describing the use of beta testers as a cheap
supplemntal test. He still seems to hold great faith in the sufficiency of
formal testing, possibly mitigated by the right logic approach and his
1,000,000 years of experience. But my less than 1,000,000 years indicates
that no matter how careful, we'll miss something. Maybe my limits are an
insufficient test.

You know the program runs AS YOU TESTED IT, but you want to know if your
test conditions are realistic as well as adequate. You get results
commensurate with the skill and diligence of the selected testers. The best
use of beta testing is to verify the baseline requirements, which might not
be what real-world users actually want. Doesn't mean it was a specification
error, just that the testers or users refine the rquirement for you. Might
also find some of the wierd interactions that result from software that runs
fine on new years eve 1999 but gives the wrong answer on 1 Jan 2000, for
example. Testing all those possible interactions is the real difficulty in
testing. Complexity results in more possibilities requiring more tests to
validate. So you test enough to show some sort of due diligence so if your
product blows up somehow you'll be able to avoid some of the liability.
Absolutely positively gotta work, you're still not going to test everything.
I read long ago that the Gallileo spacecraft, which completed a successful
mission last September, had about 50K lines of code, and to test every
possible logic path would take about 50 years using mid-80s technology. So
they didn't. Probably tested until they ran out of budget for it, declared
success and colored in the diamond. Turned out about right. Mars Climate
Orbiter had a sudden unspecified stop (aka crash) due to an embarrassing
units error. Whoops. They probably wish they'd tested a little more. Some
vendors just roll it out (apparently) and count on "updates" to correct the
errors. Sort of a spiral development model at end-user expense. Which
probably results in the best code for the lowest cost if you can tolerate
the occasional crash into Mars.

=====================================================

To unsubscribe from this list, send an email message
to "steward@scoug.com". In the body of the message,
put the command "unsubscribe scoug-programming".

For problems, contact the list owner at
"rollin@scoug.com".

=====================================================

Return to [ 02 | January | 2004 ]

SCOUG, Warp Expo West, and Warpfest are trademarks of the Southern California OS/2 User Group. OS/2, Workplace Shell, and IBM are registered trademarks of International Business Machines Corporation. All other trademarks remain the property of their respective owners.