Skip to content

test: Set per-test expected duration - #300

Open
abmusse wants to merge 1 commit into
masterfrom
tests-set-expected-duration
Open

test: Set per-test expected duration#300
abmusse wants to merge 1 commit into
masterfrom
tests-set-expected-duration

Conversation

@abmusse

@abmusseabmusse commented Jul 15, 2020

Copy link
Copy Markdown
Member

Calculated the expected test duration by running each test case 5 times and computing the average run time.

Functional Tests

Ran using ssh transport to connect from local machine to remote IBM i machine.

TestTime (ms)Time (ms)Time (ms)Time (ms)Time (ms)Average
CL command tests242617401722167116421840.2
SH command tests331829492827310427252984.6
QSH command tests361133663496302230043299.8
sendToDataQueue149113611434135313431396.4
receiveFromDataQueue144713431377144013331388
clearDataQueue147413731361142814411415.4
getTCPIPAttr227815191484148814211638
getNetInterfaceData193014581403140013561509.4
retrUsrAuth151114891601145915151515
rtrCmdInfo180613461495141713471482.2
retrPgmInfo391113261416132713651869
retrSrvPgmInfo189313351466135413171473
retrUserInfo143113531468132213151377.8
retrUsrAuthToObj135013681378129413401346
addToLibraryList138513341335134213011339.4
getPTFInfo173114781382143914711500.2
getProductInfo131813441329135713291335.4
getInstalledProducts254924732423249224852484.4
prepare & execute381016951697180816822138.4
addQuery & fetch251516881553160515241777
ensure issue #11 was resolved171615521557157915551591.8
tables334517121618199915062036
tablePriv268115491576180715481832.2
columns296515591634161715671868.4
columnPriv372219221575164015212076
procedures217415211544171915061692.8
pColumns262015441577166216131803.2
primaryKeys271831051530156615162087
foreignKeys248028701562159315912019.2
statistics408619961912222917262389.8
specialskipskipskipskipskipskip
rowCountskipskipskipskipskipskip
createUserSpace154314431422144814351458.2
setUserSpaceData135913671334132713641350.2
getUserSpaceData133712891304131613631321.8
deleteUserSpace131313501339131913171327.6
getSysValue133315201366137014181401.4
getSysStatus303214301340136715911752
getSysStatusExt240523992410240424192407.4
getJobStatus135112981332132013431328.8
getJobInfo131712871331129913411315
getDataArea130913221347130613501326.8
addParam137214311416136914111399.8
addReturnskipskipskipskipskipskip

Deprecated functional tests

Ran using HTTP transport to connect from local machine to remote IBM i machine.

TestTime (ms)Time (ms)Time (ms)Time (ms)Time (ms)Average
iCmd()595559680464510561.6
iSh()176217751730170816491724.8
iQsh()329936262973341422973121.8
iDataQueue Constructor111211.2
sendToDataQueue364380358312328348.4
receiveFromDataQueue394404365304313356
clearDataQueue405446318319311359.8
iNetwork Constructor111121.2
getTCPIPAttr600356395328357407.2
getNetInterfaceData356549390373358405.2
iObj Constructor111111
retrUsrAuth323318335330322325.6
rtrCmdInfo319335316316313319.8
retrPgmInfo473326320323345357.4
retrSrvPgmInfo462324307306316343
retrUserInfo525321300320316356.4
retrUsrAuthToObj329353312328297323.8
addToLibraryList324317332307304316.8
iProd Constructor111111
getPTFInfo385330361336329348.2
getProductInfo325337319302306317.8
getInstalledProducts268616071520158215221783.4
prepare & execute1183604576519542684.8
addQuery & fetch470475485449435462.8
ensure issue #11 was resolved1033471470459445575.6
tables537475457456460477
tablePriv497493479521456489.2
columns510529523518513518.6
columnPriv472498483460466475.8
procedures471480464455501474.2
pColumns484491479476493484.6
primaryKeys498467461466450468.4
foreignKeys479466471462458467.2
statistics1259765759728729848
specialskipskipskipskipskipskip
rowCountskipskipskipskipskipskip
iUserSpace Constructor111221.4
createUserSpace338341343378353350.6
setUserSpaceData298322313323324316
getUserSpaceData311334332301301315.8
deleteUserSpace305302337302323313.8
iWork Constructor121111.2
getSysValue328330325325323326.2
getSysStatus534339338313337372.2
getSysStatusExt139714101404142114621418.8
getJobStatus303303319323314312.4
getJobInfo349313311315327323
getDataArea438314307319303336.2
addParam324320331368334335.4
addReturnskipskipskipskipskipskip

Unit Tests

TestTime (ms)Time (ms)Time (ms)Time (ms)Time (ms)Average
SH command tests 1333333
SH command tests 2010000.2
CL command tests 1100000.2
CL command tests 2000000
QSH command tests 1001000.2
QSH command tests 2000000
Connection Constructor 1454454.4
Connection Constructor 2141042
add110110.8
debug000000
getTrasportOptions001000.2
run223232.4
ProgramCall Constructor333333
toXML000010.2
addParam 1101110.8
AddParam 2100010.4
AddParam 3010000.2
AddParam 4000100.2
AddParam 5010000.2
AddParam 6000100.2
AddReturn 1010000.2
AddReturn 2000000

Deprecated Unit Tests

TestTime (ms)Time (ms)Time (ms)Time (ms)Time (ms)Average
iSh function 1444444
iSh function 2110110.8
iCmd function 1000110.4
iCmd function 2011000.4
iQsh function 1001000.2
iQsh function 2100110.6
iConn Constructor 1544454.4
iConn Constructor 2111211.2
add011010.6
debug111111
getTrasportOptions111100.8
run233222.4
iSql constructor333533.4
toXML011100.6
addQuery 1101000.4
addQuery 2001010.4
fetch 1111100.8
fetch 2000000
fetch 3001000.2
fetch 4000000
commit 1010010.4
commit 2000100.2
commit 3001010.4
commit 4000100.2
prepare 1011010.6
prepare 2001000.2
execute 1010010.4
execute 2001000.2
execute 3000010.2
execute 4000000
tables 1011110.8
tables 2000000
tablePriv 1010110.6
tablePriv 2001000.2
columns 1010010.4
columns 2000000
columnPriv 1011000.4
columnPriv 2100000.2
procedures 1010000.2
procedures 2100100.4
pColumns 1001000.2
PColumns 2100000.2
primaryKeys 1000000
primaryKeys 2101000.4
foreignKeys 1000100.2
foreignKeys 2100010.4
statistics 1000100.2
statistics 2001010.4
special 1001000.2
special 2001010.4
count 1000000
count 2000010.2
count 3001000.2
rowCount 1001110.6
rowCount 2000000
free000110.4
describe 1000000
describe 2000010.2
describe 3000000
iPgm Constructor334443.6
toXML211111.2
addParam 1222222
addParam 2111111
addParam 3011110.8
addParam 4010100.4
addParam 5110100.6
addReturn010100.4
xmlToJson Tests 13612131318.5
xmlToJson Tests 222111.5
xmlToJson Tests 311100.75
xmlToJson Tests 444444
xmlToJson Tests 54965516.25

@abmusse
abmusseforce-pushed the tests-set-expected-duration branch 2 times, most recently from adfdb35 to d8b6629CompareJuly 18, 2020 00:50
@abmusseabmusse mentioned this pull request Jul 18, 2020
@abmusse
abmusseforce-pushed the tests-set-expected-duration branch from d8b6629 to d9beb96CompareJuly 20, 2020 15:38
@abmusse
abmusse marked this pull request as ready for review July 20, 2020 16:15
@abmusse
abmusse requested a review from kadlerJuly 20, 2020 16:15
@github-actions

Copy link
Copy Markdown

👋 Hi! This pull request has been marked stale due to inactivity. If no further activity occurs, it will automatically be closed.

@abmusseabmusse added the keep-open Exempts stale action from auto closing the issue/pr. label Aug 20, 2020
@kadlerkadler removed the stale label Oct 6, 2020
@kadlerkadler reopened this Oct 6, 2020
@abmusseabmusse linked an issue Oct 21, 2020 that may be closed by this pull request

@kadlerkadler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

keep-openExempts stale action from auto closing the issue/pr.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set per-test expected duration

2 participants

@abmusse@kadler