ааааааааааааааааааааааа ааааааааааа ааааааааааа Yuri
Arkhipkin
ааааааааааааааааааааааа ааааааааааааааааааааааа аaryur@yandex.ru
Six Sigma approach to process improvement aims to achieve customerТs satisfaction by reducing faults. Its ultimate performance target is virtually a fault-free processes and products some 3.4 or fewer faulty parts per million (ppm).
A centered six-sigma process has a
normal distribution with mean=target and specifications placed 6 standard
deviations to either side of the mean. At this point, the portions of the
distribution that are beyond the specifications contain 0.002 ppm of the data
(0.001 on each side). Practice has shown that most manufacturing processes
experience a shift, due to drift over time, of 1.5 standard deviations so that
the mean no longer equals target. When this happens in a six-sigma process, a
larger portion of the distribution now extends beyond the specification limits
of 3.4 ppm.
In manufacturing, this shift results
from mechanical wear over time and causes the six-sigma fault rate to become
3.4 ppm. The magnitude of the shift may vary, but empirical evidence indicates
that 1.5 is about average. Within the software and systems sector, it is
reasonable to presume that there are factors that would contribute to such a
shift. Possible examples are declining procedural adherence over time, learning
curve, and constantly changing computer environment. All these factors in
general are due to hazardous variety of time-dependent input data flow,
including semantics of either subject matter data or algorithmТs implementation
particularities. So such a shift in software may be considered as a semantic
shift and it seems there is no average magnitude of 1.5 standard deviations in
software engineering. Each software project has its own values of semantic mean
and semantic shift due to inherent faults of implementation. The software
engineering process just refines these values, thus approaching semantic shift
to comply with semantic mean, aiming to the best fault(failure)-free software
product, reliable enough to achieve customer requirements.а
Software reliability is understood as a probability of failure-free software operation in defined environment for a specified period of time. In general a failure is a deviation of operation results from customer requirements. The deviation is defined by correspondence between algorithmТs specification and its software implementation. Quality of algorithmТs subject matter specification influences software reliability throughout the development process and lifecycle of software product. To achieve continuous improvement of software engineering process, reliability requirements must be defined in an integrated manner for prediction, evaluation, validation, verification, and certification at specification, coding, testing, maintenance, and correction cycles. Thus reliability monitoring needs to be an on line automated process throughout the software development cycle up to the product release.
At the beginning we know nothing in general about algorithm to be implemented, but some ideas concerning input data and results. Consider algorithmТs specification based on input data formal definition. Usefulness of a software reliability model depends on the definition method of input data to be tested for exhaustive fault detection. Input data may be considered as a set of requests (sites), so their number and variety are sufficient for reliability evaluation. Sites are viewed as structural and(or) functional software elements including subject matter data, input variables, decisions, restrictions, memory structures, and the like. All sites are potentially fault inherent and may cause a failure. Software input data set may be viewed as a site set. Site is somewhat like a path. Let the site set structure defines total sitesТ variety number n(t) at different t time of lifecycle and occurrence probability pv(t) (v=1,2,Е,n(t)) of v-th site to be processed, that defines failure probability qv(t)=1-pv(t) while processing this site. Failure probability is greater for the sites that occur more rarely, because it is more difficult to sensitize faults with the help of such exotic sites.
It is mostly improbable that all n(t) sites will yield specifiedа results because it is impossible to implement software without faults. But even if all n(t) sites will yield specified results, this fact is undetectable because in this case we need all n(t) sites to be tested. Practically it is impossible because of great values of number n(t) for almost any software product. Not all sites are to be processed even throughout software lifecycle. To yield specified results at the required reliability level, in practice, it is enough for software product to have only s(t) number of assured fault free sites. The number s(t)а of potential faulty (sensitive) sites, sensitized through testing by time t, defines software test coverage as c(t) = s(t)/n(t) (0<c(t)£1) and is a well known parameter of many software reliability models.
It is natural in general to consider a software to be under operation at given t time moment, if there exists at least some fixed, before stated, minimal part c(t) of operating (fault free) sites ofа total sitesТ variety number n(t).
Consider a software as a system comprised of n(t) sites (elements), operating set of this system consists of states, each including more than s(t) operatingа sites. Consider sites are pair wise independent and uniformly distributed over all possible variety number n(t)=const site set. Software reliability R(t) is evaluated by known equation
R(t) = exp(-l(t)∙t), where l(t) is a software failure intensity measured in faults per site. Failure intensity expression is math proved and implemented in the Agile Software Reliability Monitoring Model (ASRMM).
The ASRMM considers a software as a system, comprised of n(t) sites. Operating set of this system consists of states, each including more than s(t) operatingа sites. Software test process is defined in general, including fault correction and regressive testing, thus refining software lifecycle process as for achieving and supporting reliability requirements according math relations between total sitesТ variety number n(t), test coverage c(t), mean siteТs occurrence probability ps(t), failure intensity l(t), and software reliability R(t). Time t flow in the ASRMM is a natural one and the t-unit is based on every sensitized site rv(v=1,2,Е,n(t)) while testing. Software site set structure is defined during specification as a software sensitive site semantics matrix (SSSSM). Semantics of the siteТs parameter values is defined by software subject matter and algorithmТs implementation particularities. The SSSSM is viewed as the ASRMM data base for generating tests, refining semantic mean and semantic shift of input data flow, thus providing monitoring of the software engineering process.
During software concept definition and input data specification at t=0, before testing, we refine s(0) sensitive sites selected from n(0) sitesТ variety number. Sites are ranged according to their occurrence probabilities pv (0) (v=1,2,Е,n(0)) thus defining initial semantic mean ps(0) of input data flow.
аSix Sigma software (SiSiSo) reliability requirements are evaluated by the ASRMM at the specification cycle by defining minimum needed sensitive sitesТ number s(0)=c(t)∙n(0) for testing to assure achievement ofа two failures per billion sites. Consider SiSiSo reliability requirements may be achieved at failure intensityа l(t)£2.0∙10-9. The table below shows some ASRMM SiSiSo reliability estimations for different sitesТ variety numbers. There are also included test coverage estimations for 4SiSo reliability, when it is required to have not more than 6210 failures per million sites, i.e. l(t)£0.00621. These estimations may be used as initial, preliminary, cost-schedule-reliability measures for software project at the specification cycle.
Software improvement from 4SiSo to SiSiSo, according the ASRMM, is evaluated by test maturity index c6s(t)/c4s(t) that defines extra needed time-cost on testing and if needed on reengineering. Accuracy of reliability evaluations depends not only on accuracy approach to definition of total sitesТ variety number, but on the number of detected faults. It seems that inherent faults define semantic shift of software product.
Test process must assure the achievement of required software test coverage value thus defining the mostly accurate semantic mean compliance with semantic target. Any detected and corrected fault decreases semantic shift. While testing, fault(s) is(are) detected and corrected thus changing initial sensitive sitesТ number s(0) and in general changing the sitesТ variety number n(0), thus refining needed test coverage c(t) to achieve required failure intensity l(t) for any sigma software reliability. Fault flow may vary from minimum, when no faults are detected throughout the test process, to maximum, when every tested site sensitizes fault(s). The ASRMM estimations of minimum needed test coverage at minimum and maximum fault flow are shown in the table. Test process maturity indices cminmin/cminmaxа provide insight into the accuracy range of test definition quality for achieving required software reliability. The less detected faults, the more sites must be sensitized while testing to assure the achievement of required sigma reliability thus to assure the compliance of refined semantic mean with semantic shift of input data flow.
The ASRMM defines MTBF(t)=1/l(t) as a pure time, i. e., the mean number of sites to be processed between failures not more than once. So it is considered that the minimum failure intensity lmin(t) for the given n(t) is l min (t)=1/n(t), thus defining enough sigma test coverage estimations to achieve maximum possible reliability for the given software project. The enough sigma test coverage estimations are also shown in the table below. It seems that Six Sigma reliability may be not enough for large scaled, semantically sophisticated, and critical software projects.
The ASRMM results (see table) give an idea of estimations for needed test coverage changes due to refinements of total software sitesТ variety number and sensitive sitesТ number. These refinements are contributed to the projectа by customer, developers, testers, and the like throughout software engineering process.а
The ASRMM provides continuous evaluations of the achieved reliability level, reflecting changes either in subject matter requirements or software implementation particularities, including changes in the flow of detected faults, thus providing metrics for improvement optimization throughout software engineering process.
The ASRMM reflects real software test process and may be viewed as an online software development tool to be applied for any test phase and strategy including extreme (agile) and exhaustive ones.
Table. Minimum needed 4s, 6s
andа Enough software test coverage at
max and min fault flow throughout test process.
|
Total
software sitesТ variety number n(0) |
Sigma
software requirements |
Minimal
test coverage at min fault flow cminmin |
Minimal
test coverage at max fault flow cminmax |
cminmin/cminmax |
|
1024 |
4s |
0.049 |
0.04101562 |
1.195 |
|
6s |
0.071 |
0.051757813 |
1.365 |
|
|
Enough |
0.0537 |
0.04296875 |
1.252 |
|
|
89362 |
4s |
0.00397 |
0.00367046 |
1.082 |
|
6s |
0.004644032 |
0.003994987 |
1.162 |
|
|
Enough |
0.00430831 |
0.00383832 |
1.122 |
|
|
17343286 |
4s |
0.000252028 |
0.000246147 |
1.024 |
|
6s |
0.000264079 |
0.000252144 |
1.047 |
|
|
Enough |
0.00026206 |
0.00025104 |
1.044 |
|
|
1000000000 |
4s |
0.000032182 |
0.0000319 |
1.00884 |
|
6s |
0.000032745 |
0.000032189 |
1.01727 |
|
|
Enough |
0.00003277 |
0.00003219 |
1.01802 |
|
|
102500700000 |
4s |
0.0000031411 |
0.0000031322 |
1.00284 |
|
6s |
0.0000031584 |
0.000003140 |
1.00585 |
|
|
Enough |
0.00000316286 |
0.00000314316 |
1.00627 |
|
|
1000000000000 |
4s |
0.0000010032 |
0.0000010016 |
1.00159744 |
|
6s |
0.0000010063 |
0.00000100317 |
1.00312011 |
|
|
Enough |
0.00000100744 |
0.00000100372 |
1.00370621 |
