CTWeb User`s manual
Contents
1. L ES Iw pn F ER Figure 13 13 Ifwe agree with this execution configuration we can press again Execute and the tool gives us the set of test cases all the desired pairs De those which have not been excluded are visited at least once if it has been possible those pairs with more weight will have been included more often than those with less furthermore the test suite is ordered according to the sum of the weights of the pairs included in the test case 2 5 Agrammar of the variables file A brief grammar for variables files is the following file sets includes excludes weights oracle testTemplate sets Sets n variableDefinition n variableDefinition variableName t value Mt value includes Includes n combination combination value value value In excludes Excludes An variableName variableName vn pair n pair value value weights Weights In variableName variableName n pair number n oracle Oracle freeText n variableValues n oracleLine n variableValues variableName t value t value ANY An oracleLine simpleOracle conditionOracle otherwiseOracle simpleOracle oracle Xt freeText conditionalOracle conditionalOracle Xt condition Xt freeText otherwiseOracle otherwise Xt freeText testTemplate Test template In freeText In Note that 1 tand n respectively denote a tab and a carriage return 2 The freeText in the Oracle section2. assertTrue manager SE equals GREEN GREEN false manager requestkRed b assertTrue manager SE equals GREEN GREEN false manager requestkRed a assertTrue manager toString equals GREEN GREEN false manager setTime 60 assertTrue manager toString equals YELLOW GREEN false manager setTime 63 assertTrue manager O senumg equals RED YELLOW false manager setTime 66 assertTrue manager toString equals RED RED false manager setTime 83 22 assertTrue manager toString manager requestkRed a assertTrue manager toString manager setTime 86 assertTrue manager SE manager requestkRed a assertTrue manager SE publicvoid test2 equals GREEN RED false equals GREEN RED true equals GREEN GREEN false equals GREEN GREEN false Manager manager new Manager this Semaphore a new Semaphore manager manager setA a assertTrue a toString equals GREEN assertTrue a toString equals GREEN Semaphore b new Semaphore manager manager setB b assert True manager fostring equals GREEN GREEN false manager requestRed b assert True manager Ostramg equals GREEN GREEN false manager requestRed b assertTrue manager SE equals GREEN GREEN false manager setTime 60 a
3. has a polynomial cost and produces a test suite visiting all pairs but whose size is much more small Figure 5 25 test cases in this example AETG polynomial cost A OW polynomial cost 5 Customizable pairwise exponential cost Express Random Wineal cost erbose O Algorithm aetg Results for a maximum of 96 combinationsj 1 KC C 0 2 KC C 4594 O O O o 3 KC C 100 f 4 KC F 273 5 KC K 273 01 0090 24 Another Another 0 i 25 Another Another 100 Computed in 22 milliseconds Pairs visited 100 0 A problem with the file system avoided the creation of a result file in CVS format Press here to download the test case file Algorithm implemented by Macario Polo and Beatriz PErez In order to assist you in the oracle generation you can obtain a decision table by following this link Figure 5 Suppose now that we want to use the test data combinations generated in a set of test cases as those we have written as example c convert C K 0 c convert K C 0 c convert K F 200 For this we can write a template at the text arealabeled Expression to generate test cases Suppose we want that our test cases have this aspect public void test1 Converter c new Converter c convert C K 100 yi Figure 6 To generate test cases like that in Figure 6 we can w
4. pages a good name for the MAGNITUDE variable had been N_MAGNITUDE even though it does not appear in any condition of any conditionalOracle 15 3 The state machine tool State machines have been widely used as models to generate test cases and there exist several coverage criteria to assess the quality of the test suite T 1 State coverage A test suite T satisfies state coverage if each state is covered by one or more test sequences in T 2 Transition A test suite T satisfies this criterion if each transition is traversed by one or more test sequences in T 3 Full predicate For each predicate P on each transition and each test clause ci in P T must include tests that cause each clause ci in P to determine the value of P where ci has both the values true and false A predicate is a boolean expression whose value may determine the triggering of a transition 4 Transition pair For each pair of adjacent transitions Si Sj and Sj Sk T must contain a test that traverses each transition of the pair in sequence Consider for example the state machine in Figure 15 that models the behavior of a supposed banking account Created entry movements size 0 deplosit amount dou ble amount gt 0 balance amount deposit amountdouble amount gt 0 balance amount Positive transfer amount double target Account amount gt 0 amp amp target is ValidNumber gt entry movements An A wit
5. system using symbol TAB method Symbol aliases deposit deposit amount withdrawAndBalanceGreaterThanZero withdraw amount withdrawAndBalanceLessOrEqualThanZero withdraw amount depositAndBalanceGreaterThanZerodeposit amount depositAndBalanceLessOrEqualThanZero deposit amount transferAndBalanceGreaterThanZero transfer amount targetAccount transferAndBalanceLessOrEqualThanZero transfer amount targetAccount States can also be used to the further creation of action oracles The syntax is State TAB expression and the label is State aliases For example State aliases Created Check the account has a balance 0 and has no movements Positive Check the account has a balance gt 0 Negative Check the account has a balance lt 0 Figure 16 As you see there are four sections in the file each highlighted in the figure with boldfaced labels 1 Initial node points to the initial node of the state machine In the example this one is stated called Created 2 With Transitions we represent the transitions in the state machine a The first transition goes from the Zero to the Positive state by means of the a call to the deposit operation The second one corresponds to a deposit call from Positive to Positive c Then the withdraw operation can be called from Positive and may go to two different states to Positive ifthe balance remains gt 0 or to Negative ifthe balance remains lt 0 In this case we represent
6. that the values are restricted to C F and K the oracle expression assertTrue result 4 MAGNITUDE must be added to the test case note also that the values of MAGNITUDE are 0 273 459 4 and 100 Note here that in the conditionalOracle line references to variables values are also preceded by a symbol 4 In the Test template section the tester writes the template used to generate the test cases Note this section finishes with and note also the presence of the keyword ORACLE in test case generation time the tool will substitute this token by the corresponding oracle or oracles A special detail of this section is the use of the first letters of the alphabet to do reference to the first second third sets according to the order they appear in the Sets section Sets SOURCE C F K Another TARGET C F K Another MAGNITUDE 0 273 273 01 459 4 459 41 100 Includes C K 273 01 F K 459 41 Oracle Celsius or Kelvin under absolute zero SOURCE C K MAGNITUDE 273 01 459 41 oracle assertTrue result Integer MIN_VALUE Oracle Transformation between same units SOURCE C F K TARGET C F K MAGNITUDE 0 273 459 4 100 conditionalOracle SOURCE TARGET assertTrue result MAGNITUDE Oracle Transformations FROM invalid units SOURCE Another oracle assertTrue result Integer MIN_VALUE Oracle Transformations TO invalid units TARGET Another oracle assertTrue result Integer MIN_VAL
7. 0 273 01 459 41 or 459 4 is to be made independently of the source unit the result should be oo 2 4 Execution with PROW If we upload the text file in Figure 12 to CTWeb and press Execute the tool shows in the first time the pairs tables corresponding to the three parameters this is it shows the pairs table for MAGNITUDE SOURCE MAGNITUDE TARGET and SOURCE TARGET Since in the Excludes section we have said that for SOURCE TARGET the pairs C KG C P and C O must not be included in any test case these three pairs appear checked Figure 13 The figure also shows the weights assigned to those pairs appearing in the Weights section of the text file Algorithm prow Check below the pairs to be removed and assign weights to pairs 121 pairs in SOURCE TARGET Elements Remove Sel factor Go H BS c F F pa C K H ga CM F SS Go F pa C 1 a C KM F a CM H RS C KG Y ES Cp E RS C O CR F C e ES ome et 1 0 01 P Le S 0 01 P H o o ERES SE ML Y F ba 0 01 0 fF 19 0 01 0 H 0 0 i 2 ML 1 P 0 0 0 C F 0 0 0 C E x ML KM F io lt 0 F H CS 0 F e po 5 j ML ML jr bo 0 K H 0 0 0 K p oo C 0 M e x 0 M z 50 PER MO E ae s
8. CTWeb User s manual May 21 2012 Macario Polo Usaola Beatriz P rez Lamancha Pedro Reales Mateo Alarcos Research Group Institute of Technologies and Information Systems University of Castilla La Mancha Paseo de la Universidad 4 13071 Ciudad Real Spain http alarcos esi uclm es Contact person macario polo uclm es 1 Introduction CTWeb is a web application for generating test cases It includes two tools e A combinatorial tool that gets test cases by applying several combinatorial strategies e A state machine tool that generates test cases from textual specifications of state machines Currently the use of the application is completely open and free although for some functionalities we plan to include a pay per use for companies leaving it free for students and researchers 2 The combinatorial tool Figure 1 shows the main screen of the combinatorial tool on the left side it lists the algorithms implemented by the tool by clicking on the algorithm s name the user gets an explanation of it the tester uses the right hand side to specify the parameters and values of the system or functionality under test Combinatorial testing page Upload feature model file Upload variables file see example of structure Algorithms Data All combinations exponential cost ada ser Ada row DEach choice very low cost lA B JAntirandom exponential cost lt 3 i gt o GE 9Comb li
9. K 0 K C 0 K F Excludes SOURCE TARGET C KG GP GO Weights SOURCE TARGET C F 1 F C 1 KM ML ML KM KG P 1 P KG 1 1 1 Oracle Celsius or Kelvin under absolute zero or negative length or weight MAGNITUDE 300 273 01 459 41 459 4 SOURCE ANY oracle assertTrue result Integer MIN_VALUE Oracle Transformation between same units MAGNITUDE 0 100 SOURCE ANY TARGET ANY conditionalOracle SOURCE TARGET assertTrue result MAGNITUDE Oracle Celsius Kelvin or Fahrenheit under absolute zero or negative length or weight The value is a F temperature just below 02K MAGNITUDE 459 41 SOURCE ANY oracle assertTrue result Integer MIN_VALUE Oracle A Kelvin temperature just under 0 MAGNITUDE 0 01 SOURCE K oracle assertTrue result Integer MIN_VALUE Oracle Conversions from temperatures to other units SOURCE C F K TARGET M Y I KM ML KG P 0 oracle assertTrue result Integer MIN_VALUE 100 KG KG Oracle Conversions from lengths to other units SOURCE M Y I KM ML TARGET C F K KG P 0 oracle assertTrue result Integer MIN_VALUE Oracle Conversions from weights to other units SOURCE KG P 0 TARGET C F K M Y I KM ML oracle assertTrue result Integer MIN_VALUE Oracle CtoK MAGNITUDE 0 100 273 SOURCE C TARGET K oracle assertTrue result MAGNITUDE 273 Oracle From Celsius to Celsius MAGNITUDE 0 100 273 SOURCE C TARGET C oracle a
10. Time time long time 60 a change Yellow Green a Temp Green Red setTime time long time 63 a change b change A Red Yellow setTime time long time 83 a change requestRed s Semaphore s b a requestRed etTime time long time 66 b change requestRed s Semaphore s a requestRed s Semaphore s b a requestRed Figure 20 The system is implemented as a single Java desktop application Figure 21 whose structure is shown in 20 Semaphores lt lt Interface gt gt Color AA JDIAGREEN byte 1 ei YELLOW byte 2 RED byte 3 65 color byte manager Manager Semaphore manager Manager change void requestRed void getColor byte toString java lang String Reg Figure 21 manager Manager la Semaphore Hb Semaphore watch Watch temporalState boolean Manager watch Watch setA a Semaphore void JFSemaphores eo pjp w setBib Semaphore void setTime time long void getTemporalState boolean requestRed semaphore Semaphore void max a long b long long toString java lang String 1 1 manager m nager lt lt Interface gt gt watch IWatch reset void
11. UE Test template public void testTCNUMBER Converter c new Converter double result c convert A B C ORACLE Figure 10 Figure 11shows some of the test cases generated with this text file 1 2 3 4 Test case 1 corresponds to the conditional oracle since itis a conversion from 02 Celsius to Celsius Test case 2 is also a conversion from Celsius to Celsius but the value of MAGNITUDE does not match with the values in the MAGNITUDE values of the conditional oracle This test case is a conversion from 459 41 Celsius degrees which fits with the first Oracle section and therefore its expressions is added The test data of test case number 4 doesn t fit with any oracle the tool adds a comment line explaining this situation The test data in test case 20 fit with two oracles a conversion to invalid units and a conversion under the absolute zero Both oracle expressions are added to the test case although also this situation is added in a comment line ee aetg Results fora maximum of 96 combinations gt for Results fora maximum of 96 combinations maximum of 96 combinations public void test1 Converter c new Converter ouble result c convert C C 0 C C 0 Transformation between same units ssertTrue result 0 public void test2 Converter c new Converter ouble result c convert C C 459 41 C C 459 41 Celsius or Kelvin un
12. can contain variable names with the prefix 3 The freeText in the Test template section may also contain references to the variables but in this case using A B C D etc to do reference to the first second third fourth etc variable 4 There may exist an otherwhise oracle which is an expression that is added to all test cases whose test data do not fit to any other oracle See an example in the next section 2 6 Use of numeric variables in conditional oracles A famous problem in software testing is the determination of the type of a triangle according to three values that represent the lengths of its three sides These values may correspond to an equilateral isosceles or scalene triangle or maybe not to a triangle negative sides sum of two sides greater or equals to the third one As a last text file example the following one can be used to exercise the problem of determining the type of a triangle in this example we have boldfaced the last oracle an otherwise oracle which corresponds to triangles of the scalene type This oracle will be added to all those test cases whose values do not match with any of the other oracles 14 Sets N_I 0 1 2 3 4 5 6 N_J 1 0 1 2 3 4 5 6 N_K 1 0 1 2 3 4 5 6 Oracle EQUILATERAL N_I 1 2 3 4 5 6 N_J 1 2 3 4 5 6 N_K 1 2 3 4 5 6 conditionalOracle N_I N_J amp amp N_J N_K assertTrue result Triangle EQUILATERAL Oracle A line or negative sides
13. ck the account has a balance gt 0 ansfer amount targetAccount Check the account has a balance lt 0 eposit amount V Check the account has a balance gt 0 Figure 19 19 The figure shows in the first time the six paths the tool has generated to go through all transitions then for each path it includes the set of calls required to exercise each transition included in the path as well as the alias of each state 3 3 One more example The following state machine represents a Manager that controls the light flow of two semaphores when there are no pedestrians the manager changes the light of both semaphores a and b sending them the change event every a fixed number of seconds 60 63 66 83 and 86 However a pedestrian may request the red light in any of the semaphores if the semaphore where red is requested is in yellow or red nothing happens if it is in green and the semaphore is a then the managers changes a to yellow either 20 seconds after the request or if less than 20 seconds remain in this time Ifthe red light is requested on b then the request is passed to a Just Created setA s Semaphore s getColor GREEN A established setB s Semaphore s getColor GREEN requestRed s Semaphore s a and watch getTime lt 40 watch putForward 40 setTime time long time 86 watch putForward 40 b change Green Green setTime time long time 86 b change set
14. der absolute zero ssertTrue result Integer MIN_VALUE public void test4 Converter c new Converter F 273 ouble result c convert C F 273 Warning this test case has no oracle assigned a void test20 Converter c new Converter ouble result c convert K Another 273 01 Celsius or Kelvin under absolute zero Ounae 27201 ssertTrue result Integer MIN_VALUE Transformations TO invalid units ssertTrue result Integer MIN_VALUE Warning more than one oracle for this test case Figure 11 2 3 Aless simple example Suppose now a new version of the convert function with the same signature than the previous one but that is now capable of making more types of conversions it may translate temperatures Celsius Fahrenheit and Kelvin C F K lengths Meters Yards Inches Kilometers and Miles M Y I KM ML and weights Kilograms Pounds and Ounces K P 0 For testing this new version of the function we should take into account the appropriate equations for conversions as well as the invalid values for the function s parameters Considering that one cannot convert between different types of units from temperatures to lengths for example the different values of the absolute zero we have seen and that there are no negative lengths or weights 10 the following table shows a set of possible equivalence classes for these parameters Temperatu
15. entation for combinatorial testi Figure 18 Note the list box with the label Select an algorithm depending on the desired coverage criterion the tester will select one of the provided algorithms 18 1 All edges produces a test suite that visits all the transitions in the state machine at least once 2 All pairs produces a test suite that for each state visits all the pairs of input and output transitions at least once 3 The test suite generated by All states visits all the states in the machine at least once 4 Binder generates test cases according to the Binder s algorithm 5 Prime path produces test cases according to the Prime path algorithm If we select for example All transitions and press the button labeled Create test cases CTWeb produces the output shown in Figure 19 Results Path 1 deposit deposit Path 2 deposit withdrawAndBalanceGreaterThanZero Path 3 deposit withdrawAndBalanceLessOrEqualThanZero depositAndBalanceGreaterThanZero Path 4 deposit withdrawAndBalanceLessOrEqualThanZero depositAndBalanceLessOrEqualThanZero Path 5 deposit transferAndBalanceGreaterThanZero Path 6 deposit transferAndBalanceLessOrEqualThanZero depositAndBalanceGreaterThanZero List of test cases May be empty if there are no symbol aliases in the state machine file est case 1 deposit deposit Check the account has a balance 0 and ha
16. f we leave selected the All combinations algorithm and press Execute the tool produces the following results Algorithm allCombinations C C 0 C C 273 C C 273 01 C C 459 4 Results for a maximum of 96 combinations KC C 459 41 C C 100 GEO KC F 273 C F 273 01 C F 459 4 TT 1IKC F 459 41 Mir E dom 0090 e 92 Another Another 273 93 Another Another 273 01 94 Another Another 459 4 Computed in 6 milliseconds Pairs visited 100 0 95 Another Another 459 41 96 Another Another 100 A problem with the file system avoided the creation of a result file in CVS format Press here to download the test case file Algorithm implemented by Macario Polo In order to assist you in the oracle generation you can obtain a decision table by following thi Figure 4 As there are 4 4 and 6 values in the three sets the All combinations algorithms produces 4 4 6 96 test data combinations After the results table we get some information regarding the computation time and the percentage of pairs of data valuesvisited by test cases One of the problems of All combinations is on the one side its high computational cost exponential and on the other side the high number of test cases it produces To deal with this we can use any of the other algorithms provided by CTWeb AETG for example
17. getinitial Miliseconds long S Je getCurrentMiliseconds long putForward parameter long void A i i i i i i Watch initialMiliSeconds long l currentMiliSeconds long manager Manager window Window delay long Watch0 RL E b lt lt Interface gt gt Window setA parameter byte void setBiparameter byte void setTime parameter long void window reset void setManager manager Manager void setWindow window Window void run void updateCurre ntMiliSeconds void getlnitial Miliseconds long getCurrentMiliseconds long putForward delay long void watch Figure 22 The text representation of the state machine is the following This is a small example of a state machine description file Initial node JC Transitions have source state TAB symbol of the alphabet TAB target state Transitions JC setA A established A established setB GG 21 GG requestRedOnA GG GG requestRedOnB GG GG setTime60 YG YG setTime63 RY RY setTime66 RR RR setTime83 GR GR requestRedOnA TGR GR requestRedOnB TGR TGR setTime86 GG GR setTime86 GG Symbols can be mapped to method calls of the system using symbol TAB method Symbol aliases setA Manager manager new Manager this Semaphore a new Semaphore manager manager setA a setB Semaphore b new Semaphore manager manager se
18. hdraw amount double amount gt 0 balance amount amount gt balance balance amount target4balance amount balance gt 0 balance lt 0 ZeroOrNegative entry movements Keposit amount double amount gt 0 balance amount amount balance balance amount target4balance amoun Figure 15 In order to get for example states coverage a possible test case could be create deposit 100 withdraw 200 Obviously states coverage leaves or may leave many uncovered transitions and that s the reason of using stricter coverage criteria 3 1 Description of state machines with text files CTWeb may process state machines described as simple text files The following figure shows a text representation of the state machine in Figure 15 16 This is a small example of a state machine description file Initial node Created Transitions have source state TAB symbol of the alphabet and target state TAB all of them comma separated TAB is a tabulator Transitions Created deposit Positive Positive deposit Positive Positive withdrawAndBalanceGreaterThanZero Positive Positive withdrawAndBalanceLessOrEqualThanZero Negative Negative depositAndBalanceGreaterThanZero Positive Negative depositAndBalanceLessOrEqualThanZero Negative Positive transferAndBalanceGreaterThanZero Positive Positive transferAndBalanceLessOrEqualThanZero Negative Symbols can be mapped to method calls of the
19. inations to test this function 1 First of all as we have three parameters we press the Add set button to add a new set of test data values Then the tool inserts a new column in the right IC Expression to generate test cases Example public void testTCNUMBER Figure 2 2 We continue filling in the row with the test data As we need to add rows we press the Add row button in Figure 2 so many times as we need The screen will look as in Figure 3 Combinatorial testing page Upload feature model file Upload variables file see example of structure Algorithms Data PAll combinations exponential cost add ser Each choice very low cost lA IB JC PAntirandom exponential cost NES ie fo PComb lineal cost E F 273 P Genetic 9 K 2273 01 Costly pairwise exponential cost glAnother Another 459 4 PAETG polynomial cost EAS 459 41 PPROW polynomial cost Dog PCustomizable pairwise exponential cost Expression to generate test cases Example PBacteriologic PRandom lineal cost el z Figure 3 public void testTCNUMBER ClassUnderTest o new ClassUnde o method1 C double result o method2 C B ORACLE A 3 Now can generate the test data combinations by selecting the desired algorithm left side and pressing the Execute button beneath the left side I
20. neal cost Expression to generate test cases Example Genetic public void testTCNUMBER Costly pairwise exponential cost ClassUnderTest o new ClassUnderTest A B ie 5 o method1 C AETG polynomial cost double result o method2 C B A OPROW polynomial cost ORACLE Customizable pairwise exponential cost 2 Bacteriologic Random lineal cost E a Figure 1 2 1 Asimple example Let us suppose we want to test a function that converts temperature measures whose signature is convert sourceUnit String targetUnit String magnitude double double It translates the numeric magnitude passed as third parameter from the source unit to the target unit respectively passed as first and second parameters The conversion functions from Celsius to Kelvin and Fahrenheit are 9 K C 273 F XC 32 Supposing cis an instance of the container class let it be Converter some possible calls to the function under test could be c convert C K 0 c convert K C 0 c convert K F 200 If you remind the minimum possible temperature is the absolute zero which corresponds to 02K 273 C 459 4 F Thus some test data to test this simple function could be those in Table 1 Source units Target units Magnitude C C 0 F F 273 K K 273 01 Another Another 459 4 459 41 100 Table 1 With CTWeb it is very easy to generate data comb
21. ppears Test case generation from state machines From this page you can generate test cases from state machines described as transition tables The tool is capable of getting test cases fulfill Please consider reading a small help about this functionality in this page Upload files Select either a txt file with the transition table of the state machine see an example of a file like this or a um file with the state machine Upload state machine description file No se ha seleccionado ningun archivo Cite this site as Polo M and P rez B 2010 A framework and a web implementation for combinatorial testing White paper of the Alarcos Re http alarcosj esi uclm es CTWeb Figure 17 After uploading the file in Figure 16 the tool shows the transitions table a piece of which is shown in Figure 18 Test case generation from state machines From this page you can generate test cases from state machines described as transition tables The tool is Please consider reading a small help about this functionality in this page Upload files Select either a txt file with the transition table of the state machine see an example of a file like this or a u Upload state machine description file No se ha seleccionado ningun archivo SE A AA Ge Select an algorith Z TE All pairs L All states Binder Prime path Cite this site as Polo Mana Perez B 2010 A framework and a web implem
22. re From C 00 273 273 C 00 From F Equivalence classes 00 459 4 459 4 00 From K 0 IO 00 Table 2 Length From the equivalence classes of Table 2 the tester must propose a set of test data which could be those in Table 3 From C 300 273 01 273 Error value out of range Error value out of range Error value out of range Value Length and weight Type of expected result Error value out of range Error value out of range From F 459 41 459 4 Error value out of range Error value out of range Valid conversion Valid conversion From K 0 01 Error value out of range From C F K 0 Valid conversion 100 Valid conversion Table 3 Figure 12shows a text file for this new version of convert Besides having more values in the variables definition and much more oracles it also has two new sections 1 We can write several Excludes sections Each one starts with the names of a pair of sets and then some lines with pairs of values of these sets that the tester does not desire to include in the test cases In this example we are saying CTWeb that we don t want test cases with conversions from Celsius to Kilograms Pounds or Ounces 11 Sets MAGNITUDE 300 SOURCE C 273 01 273 459 41 459 4 0 01 0 K M Y I KM ML TARGET K M Y I KM ML OCH rm ono Includes 273 C K 459 4 F
23. rite in the afore mentioned test area an expression like this one Expression to generate test cases public void testTCNUMBERO i Converter c new Converte ro c convert A ER SCH Figure 7 Now when CTWeb generates the test data combinations will substitute the TCNUMBER token by the actual index of the combination and the tokens 4 B and C by the values of the first second and third parameters in the current combination In other words the results table will look such as that in Figure 8 note that now the code corresponding to the translation of the combination values has been added into the third column Algorithm aetg Results for a maximum of 96 combinations public void test1 Converter c new Converter 1 C C 0 convert C SE 0 E public void test2 2 kc c 459 41 Converter c new Converter convert C C 459 41 Wi public void test3 Converter c new Converter ZPS convert C C 100 Ta Y Sania tactAl Tf 000 public void test25 nverter c new Converter convert Another Another 100 1 iL 25 Another Another 100 Computed in 44 milliseconds Pairs visited 100 0 A problem with the file system avoided th ation of a result file in CVS format Press here to download the test case file Algorithm implemented by Macario Polo and Beatriz PErez In order to assist you in the oracle generation you can ob
24. s NI ANY NJ ANY N_K ANY conditionalOracle N_I N_J N_K N_I N_K N_J N_J N_K N_I N_I lt 0 N_J lt 0 N_K lt 0 assertTrue result Triangle NO_TRIANGLE Oracle Isosceles NI 1 2 3 4 5 6 NJ 1 2 3 4 5 6 N_K 1 2 3 4 5 6 conditionalOracle N_I N_J amp amp N_I N_K N_I N_K amp amp N_I N_J N_J N_K amp amp N_J N_I assertTrue result Triangle ISOSCELES Oracle Sides do not fit NI 1 2 3 4 5 6 NJ 1 2 3 4 5 6 N_K 1 2 3 4 5 6 conditionalOracle N_I gt N_J N_K N_J gt N_I N_K N_K gt N_I N_J assertTrue result Triangle NO_TRIANGLE Oracle Default oracle otherwise assertTrue result Triangle SCALENE Test template public void testTCNUMBER Triangle t new Triangle t setI A t set B tsetK C t calculateType int result t getType ORACLE Figure 14 As our tool is implemented in Java the conditional expressions of the conditional oracles are processed and evaluated as Java expressions In order to give a suitable processing to conditions that involve numeric variables remember to include the prefix N_ to those numeric variables which will appear in some condition Due to this in the example of the previous figure we called N_I N_Jand N_K to the three variables used In general it is a good idea to name all numeric variables with the prefixN_ In the text filesof the convert function used in the previous
25. s no movements eposit amount Check the account has a balance gt 0 eposit amount Check the account has a balance gt 0 est case 2 deposit withdrawAndBalanceGreaterThanZero Check the account has a balance 0 and has no movements eposit amount V Check the account has a balance gt 0 ithdraw amount Check the account has a balance gt 0 est case 3 deposit withdrawAndBalanceLessOrEqualThanZero depositAndBalanceGreaterThanZero Check the account has a balance 0 and has no movements eposit amount Check the account has a balance gt 0 withdraw amount V Check the account has a balance lt 0 1 Grek te ac 1 Check the account has a balance gt 0 est case 4 deposit withdrawAndBalanceLessOrEqualThanZero depositAndBalanceLessOrEqualThanZero Check the account has a balance 0 and has no movements deposit amount d Check the account has a balance gt 0 withdraw amount V Check the account has a balance lt 0 eposit amount Check the account has a balance lt 0 est case 5 deposit transferAndBalanceGreaterThanZero I Check the account has a balance 0 and has no movements eposit amount Check the account has a balance gt 0 Itransfer amount targetAccount Check the account has a balance gt 0 est case 6 deposit transferAndBalanceLessOrEqualThanZero depositAndBalanceGreaterThanZero d Check the account has a balance 0 and has no movements deposit amount Che
26. ssertTrue manager SE equals YELLOW GREEN false Figure 24 23
27. ssertTrue result MAGNITUDE Oracle Transformations from Km to Miles MAGNITUDE 0 100 SOURCE KM TARGET ML oracle assertTrue result MAGNITUDE 1609 Oracle Transformations from Miles to Km MAGNITUDE 0 100 SOURCE ML TARGET KM oracle assertTrue result MAGNITUDE 1609 Oracle Negative length or weights MAGNITUDE 300 273 01 273 459 41 459 4 0 01 SOURCE M Y I KM ML KG P 0 oracle assertTrue result Integer MIN_VALUE Test template public void testTCNUMBER Converter c new Converter double result c convert B C A ORACLE Figure 12 12 2 The Weights section is used to assign an importance to certain pairs of values As you know pairwise algorithms such as AETG generate test cases until all the pairs of values between any two parameters have been included in at least one test case By assigning weights to pairs the tester expresses that if two different pairs have the same chance of being included in a test case CTWeb should include that with a higher weight By default all pairs have 0 as weight Actually the Excludes and the Weights sections are used only by the PROW algorithm Pairwise with Restrictions Order and Weight which will be described in the next section Note that several oracles use the reserved word ANY what makes reference to any value of the referenced variable The first oracle for example says that always that a conversion of 30
28. st be included in the test case 2 3 a In the first Oracle described as Celsius or Kelvin under absolute zero there appear two values C and K for the SOURCE variable and three values 273 01 and 459 41 for the MAGNITUDE variable This means that this oracle is applicable to all those test cases whose SOURCE variable is C or K and whose MAGNITUDE is 273 01 and 459 41 this is this oracle will be included in all the test cases that try to convert 273 01 C 273 01 K 459 41 C or 459 41 K Moreover the oracle expression for these test cases is that appearing after a tab after the oracle keyword assertTrue result Integer MIN_VALUE The convert function returns when the conversion is erroneous value that is represented as Integer MIN_VALUE Note that in the oracle line references to variables values are preceded by a symbol b The second Oracle is slightly different it involves the three sets SOURCE TARGET and MAGNITUDE but moreover its last line has the keyword conditionalOracle in its last line Conditional oracles have two parameters tab separated the first one is a condition expressed in Java language because this is the programming language in which CTWeb is implemented that when it is true says the tool that the expression included as second parameter must be added to the test case In this example the condition says that when the SOURCE variable is equals to the TARGET note moreover
29. tB b requestRedOnA manager requestRed a requestRedOnB manager requestRed b setTime60 manager setTime 60 setTime63 manager setTime 63 setTime66 manager setTime 66 setTime83 manager setTime 83 setTime86 manager setTime 86 States can also be used to the further creation of action oracles The syntax is State TAB expression and the label is State aliases For example State aliases A established assertTrue a toString equals GREEN GG assertTrue manager toString equals GREEN GREEN false YG assertTrue manager toString equals YELLOW GREEN false RY assertTrue manager toString equals RED YELLOW false RR assertTrue manager toString equals RED RED false GR assertTrue manager toString equals GREEN RED false TGR __assertTrue manager toString equals GREEN RED true Figure 23 If we upload this file and generate a test suite covering All pairs we get a set of test cases that can copy and paste on our IDE Two of these test cases are publicvoid testl Manager manager new Manager this Semaphore a new Semaphore manager manager setA a assertTrue a toString equals GREEN assertTrue a toString equals GREEN Semaphore b new Semaphore manager manager setB b assertTrue manager toString equals GREEN GREEN false manager requestRed a assertTrue manager toString equals GREEN GREEN false manager requestRed a
30. tain a decision table by following this link Figure 8 See also in Figure 8 the link highlighted with a red arrow if you press it a new window with all the test cases generated is open you can copy and paste it to work with it public void test1 Converter c new Converter c convert C C 0 public void test2 Converter c new Converter c convert C C 459 41 public void test3 Converter c new Converter esca a OA MIA A MIA Figure 9 2 2 Uploading the data from a test file Instead of filling in by hand the data area we can upload a variables file that moreover can be enriched with more information to generate the tests The following Figure 10 is a possible text file to test the convert function Note it has several sections 1 In the Sets section we add a line for each variable or set After the variable s name there is a tab and also a tab after each variable value In the Includes section we add those combinations that we want to include always in the test suite writing their values with a comma between each two values To exemplify we have added the test cases C K 273 01 and F K 459 41 Then there are several Oracle sections Each oracle may have a description tab separated from the Oracle keyword In the following lines we specify also tab separated the values of the variables for which the oracle expression which appears in the last line mu
31. these two possibilities with two different transitions i withdrawAndBalanceGreaterThanZero that goes from Positive to Positive ii withdrawAndBalanceLessOrEqualThanZero that from Positive to Negative d The next two transitions correspond to calls to deposit from the Negative state that may put the machine in Positive or Negative e Finally the two calls to transfer from Positive are represented in the last two lines of this section 3 In the Symbol aliases section the tester assigns messages or triggers to the transitions enumerated in the Transitions section For example it is said 17 that deposit used in the transitons Zero deposit Positive actually corresponds to a call to deposit amount that withdrawAndBalanceGreaterThanZero and withdrawAndBalanceLessOrEqualThanZeroare really calls to withdraw amount etc The tester may assign here actual parameters or as in this example just leave the parameter names and assign values later although actually she he may assign any test 4 As each state represents an invariant condition that the system must fulfill with it is in that state the State aliases section is useful to add the test cases the condition that must be checked when the state is reached For example when the machine is in Created it should be tested that has a balance of zero and that it has no movements 3 2 Processing state machines text files In Figure 17 the web form for uploading state machines files a
Download Pdf Manuals
Related Search