2024年4月3日发(作者：老款科鲁兹)

IEEETRANSACTIONSONSOFTWAREENGINEERING,VOL.37,NO.5,SEPTEMBER/OCTOBER2011649

AnAnalysisandSurveyoftheDevelopment

ofMutationTesting

YueJia,StudentMember,IEEE,andMarkHarman,Member,IEEE

Abstract—MutationTestingisafault-basedsoft

literatureonMutationTestinghascontributedasetofapproaches,tools,developments,perprovidesa

eralsopresentstheresultsofseveraldevelopmenttrendanalyses.

TheseanalysesprovideevidencethatMutationTestingtechniquesandtoolsarereachingastateofmaturityandapplicability,while

thetopicofMutationTestingitselfisthesubjectofincreasinginterest.

IndexTerms—Mutationtesting,survey.

?

1I

NTRODUCTION

UTATION

Testingisafault-basedtestingtechnique

whichprovidesatestingcriterioncalledthe“muta-

tionadequacyscore.”Themutationadequacyscorecanbe

usedtomeasuretheeffectivenessofatestsetintermsofits

abilitytodetectfaults.

ThegeneralprincipleunderlyingMutationTestingwork

isthatthefaultsusedbyMutationTestingrepresentthe

fully

choosingthelocationandtypeofmutant,wecanalso

ultsare

deliberatelyseededintotheoriginalprogrambysimple

syntacticchangestocreateasetoffaultyprogramscalled

mutants,

assessthequalityofagiventestset,thesemutantsare

esultofrunninga

mutantisdifferentfromtheresultofrunningtheoriginal

programforanytestcasesintheinputtestset,theseeded

comeofthe

MutationTestingprocessisthemutationscore,which

ationscore

istheratioofthenumberofdetectedfaultsoverthetotal

numberoftheseededfaults.

ThehistoryofMutationTestingcanbetracedbackto

1971inastudentpaperbyLipton[144].Thebirthofthe

fieldcanalsobeidentifiedinpaperspublishedinthelate

1970sbyDeMilloetal.[66]andHamlet[107].

MutationTestingcanbeusedfortestingsoftwareatthe

unitlevel,theintegrationlevel,

hasbeenappliedtomanyprogramminglanguagesasa

whiteboxunittesttechnique,forexample,Fortranprograms

[3],[36],[40],[131],[145],[181],Adaprograms[29],[192],

Cprograms[6],[56],[97],[213],[214],[237],[239],Java

.TheauthorsarewiththeDepartmentofComputerScience,University

CollegeLondon,MaletPlace,LondonWC1E6BT,UK.

E-mail:{,}@.

Manuscriptreceived21Sept.2009;revised15Feb.2010;accepted1Apr.

2010;publishedonline10June2010.

.

Forinformationonobtainingreprintsofthisarticle,pleasesende-mailto:

tse@,andreferenceIEEECSLogNumberTSE-2009-09-0232.

DigitalObjectIdentifierno.10.1109/TSE.2010.62.

0098-5589/11/$26.00?2011IEEE

M

programs[44],[45],[127],[128],[129],[130],[150],[151],

C#programs[69],[70],[71],[72],[73],SQLcode[43],[212],

[233],[234],andAspectJprograms[12],[13],[17],[90].

MutationTestinghasalsobeenusedforintegrationtesting

[54],[55],[56],[58].BesidesusingMutationTestingatthe

softwareimplementationlevel,ithasalsobeenappliedatthe

designleveltotestthespecificationsormodelsofaprogram.

Forexample,atthedesignlevel,MutationTestinghasbeen

appliedtoFiniteStateMachines[20],[28],[88],[111],

Statecharts[95],[231],[260],EstelleSpecifications[222],

[223],PetriNets[86],Networkprotocols[124],[202],[216],

[238],SecurityPolicies[139],[154],[165],[166],[201],and

WebServices[140],[142],[143],[193],[245],[259].

MutationTestinghasbeenincreasinglyandwidely

as

beenmuchresearchworkonthevariouskindsoftechniques

seekingtoturnMutationTestingintoapracticaltesting

r,thereislittlesurveyworkinthe

stsurveyworkwas

conductedbyDeMillo[62]rksummarized

thebackgroundandresearchachievementsofMutation

Testingatthisearlystageofdevelopmentofthefield.A

surveyreviewofthe(veryspecific)subareaofStrong,Weak,

andFirmmutationtechniqueswaspresentedbyWoodward

[253],[256].AnintroductorychapteronMutationTesting

canbefoundinthebookbyMathur[155]andalsointhe

bookbyAmmannandOffutt[11].Themostrecentsurvey

workwasconductedbyOffuttandUntch[191]

summarizedthehistoryofMutationTestingandprovidean

overviewoftheexistingoptimizationtechniquesforMuta-

r,sincethen,therehavebeenmore

than230newpublicationsonMutationTesting.

Inordertoprovideacompletesurveycoveringallthe

publicationsrelatedtoMutationTestingsincethe1970s,we

constructedaMutationTestingpublicationrepository,

whichincludesmorethan390papersfrom1977to2009

[121].Wealsosearchedformaster’sandPhDthesesthat

havemadeasignificantcontributiontothedevelopmentof

four

,wesearchedtheonline

repositoriesofthemaintechnicalpublishers,including

PublishedbytheIEEEComputerSociety

650IEEETRANSACTIONSONSOFTWAREENGINEERING,VOL.37,NO.5,SEPTEMBER/OCTOBER2011

TABLE1

AListofPhDandMaster’sWorkonMutationTesting

IEEEXPlore,ACMPortal,SpringerOnlineLibrary,Wiley

InterScience,andElsevierOnlineLibrary,collectingpapers

whichhaveeither“mutationtesting,”“mutationanalysis,”

“mutants+testing,”“mutationoperator+testing,”“fault

injection,”and“fault-basedtesting”keywordsintheirtitle

,wewentthroughthereferencesforeach

paperinourrepositorytofindmissingpapersusingthe

way,weperformeda

“transitiveclosure”onTesting

workwhichwasnotconcernedwithsoftware,forexample,

hardware,wasremovedandwealsofilteredoutpapersnot

y,wesentadraftofthispaperto

havemadetherepositorypubliclyavailableat

/pg/jiayue/repository/[121].Overall

growthtrendofallpapersinMutationTestingcanbe

foundinFig.1.

Therestofthepaperisorganizedasfollows:Section2

introducesthefundamentaltheoryofMutationTesting,

includingthehypotheses,theprocess,andtheproblemsof

n3explainsthetechniquesforthe

n4introduces

applicationsofMutationTestingareintroducedinSection5.

Section6summarizestheempiricalexperimentsofthe

n7describesthe

n8discusses

theevidencesfortheincreasingimportanceofMutation

n9discussestheunresolvedproblems,

barriers,

paperconcludesinSection10.

2T

HE

T

HEORYOF

M

UTATION

T

ESTING

Thissectionwillfirstintroducethetwofundamental

discussesthe

generalprocessofMutationTestingandtheproblemsfrom

whichitsuffers.

2.1FundamentalHypotheses

MutationTestingpromisestobeeffectiveinidentifying

adequatetestdatawhichcanbeusedtofindrealfaults[96].

However,thenumberofsuchpotentialfaultsforagiven

programisenormous;itisimpossibletogeneratemutants

ore,traditionalMutation

Testingtargetsonlyasubsetofthesefaults,thosewhichare

closetothecorrectversionoftheprogram,withthehope

eory

isbasedontwohypotheses:theCompetentProgrammer

Hypothesis(CPH)[3],[66]andtheCouplingEffect[66].

1978

[66].Itstatesthatprogrammersarecompetent,which

impliesthattheytendtodevelopprogramsclosetothe

更多推荐

老款,作者