Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); 4) Examples · jedazard/PRIMsrc Wiki · GitHub
Skip to content

4) Examples

J-E Dazard, PhD edited this page Nov 11, 2020 · 131 revisions

Below are two examples of codes and results generated by PRIMsrc while carrying out a Survival Bump Hunting (SBH) search in a synthetic and real-world datasets.

First, attach the package PRIMsrc with its dependencies:

 #===================================================
# Attaching the `PRIMsrc` library with its dependencies
#===================================================
library("PRIMsrc")

A simplistic example

Illustration of the Survival Bump Hunting (SBH) search in a synthetic dataset, described in the PRIMsrc package as demo dataset Synthetic.1.

Data

In this demo example, we simulate a dataset with n = 250 observations in a p = 3 dimensional space. Here, realizations of true survival times, denoted by t = (t1,t2,...,tn)T, are generated from the exponential survival model: T ~ Exp(λ), where each individual rate λi can be directly estimated, conditionally on covariates xi = (xi1, xi2,..., xip)T, from an exponential regression function, so that hazards will be proportional to covariates main effects:

ti ~ Exp(λi), for i = 1,...,n

where λi = λ0(t) exp(ηTxi)
and η = (η1, η2,..., ηp)T

 #===================================================
# Demo with a synthetic dataset
# Use help for descriptions
#===================================================
data("Synthetic.1", package="PRIMsrc")
?Synthetic.1
head(Synthetic.1)
 y delta X1 X2 X3
1 0.1475693 1 0.0724409 0.1571447 0.4693207
2 1.0332351 0 0.7698878 0.7521790 0.6536864
3 0.9172719 0 0.3254684 0.9866740 0.1781815
4 0.3887982 0 0.8385150 0.7635480 0.9189738
5 0.8317025 0 0.6424662 0.6666725 0.7454203
6 1.7131669 0 0.2640069 0.9361957 0.5129831

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/synthetic.log", sep=""))

Specification of the PRSP parameters:

 alpha <- 0.01
beta <- 0.10
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this simple example, parameters for running the PRSP algorithm are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion="lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

To run the PRSP algorithm, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Here, the SBH search is carried out without variable pre-selection (vs = FALSE) and with computation of p-values (pv = TRUE).

Other ancillary parameters are: usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), without any parallelization (parallel.vs = FALSE, parallel.rep = TRUE, parallel.pv = TRUE, conf=NULL), verbose option (verbose = FALSE) and a seed (seed = 123).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
synt1 <- sbh(X = Synthetic.1[ , -c(1,2), drop=FALSE],
y = Synthetic.1[ ,1, drop=TRUE],
delta = Synthetic.1[ ,2, drop=TRUE],
B = 30,
K = 5,
vs = FALSE,
cv = TRUE,
cvtype = "combined",
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5), parallel.vs = FALSE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested serial replicated 5-fold cross-validated procedure with 32 replications. Variable screening: FALSE Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE
Decision rule: EXTREMUM Parallelization of computation of PRSP: FALSE No screening of covariates. Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: X1 X2 X3 1 2 3 Directions of directed peeling of used covariates: X1 X2 X3 1 -1 -1 Generating box rules of used covariates ...
Generating box statistics ...
No computation of p-values. Finished!

Results

The sbh() function generates a 'sbh' object called 'synt1' that is used for other functions.

There is a S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(synt1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 30 replications. VARIABLE SCREENING:
Variable screening: FALSE CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED ALGORITHM PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

The 'sbh' object 'synt1' contains a number of values including the cross-validated fitted SBH model (synt1$cvfit), that is gathered in a list with 12 fields:

  • cv.maxsteps: numeric scalar of maximal number of peeling steps over the replicates.
  • cv.nsteps: numeric scalar of optimal number of peeling steps according to the optimization criterion.
  • cv.boxind: logical matrix in TRUE, FALSE of individual observation box membership indicator (columns) for all peeling steps (rows).
  • cv.boxind.size: numeric vector of box sample size for all peeling steps.
  • cv.boxind.support: numeric vector of box support for all peeling steps.
  • cv.rules: data.frame of decision rules on the covariates (columns) for all peeling steps (rows).
  • cv.screened: numeric vector of screened (pre-selected) covariates, indexed in reference to original index.
  • cv.trace: numeric vector of the modal trace values of covariate usage for all peeling steps.
  • cv.sign: numeric vector in {-1,+1} of directions of peeling for all used (selected) covariates.
  • cv.used: numeric vector of covariates used (selected) for peeling, indexed in reference to original index.
  • cv.stats: numeric matrix of box endpoint quantities of interest (columns) for all peeling steps (rows).
  • cv.pval: list with 2 fields of two vectors. The first cv.pval$pval is a numeric vector for log-rank p-values of separation of survival distributions, The second cv.pval$seed is is an integer scalar if parallelization is used, or an integer vector of A values, one for each permutation, if parallelization is not used.

There is also a S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once, including the sought-after cross-validated SBH estimates:

 print(synt1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
synt1$cvfit$cv.maxsteps
[1] 131
 synt1$cvfit$cv.nsteps
[1] 109
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
synt1$cvfit$cv.boxind.size
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 250 245 243 240 237 235 233 230 229 225 224 222 220 218 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 213 211 208 208 202 201 201 196 194 192 188 188 185 183 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 181 179 178 175 172 168 165 160 158 157 154 150 147 145 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 144 142 140 137 134 133 129 126 124 122 122 122 121 121 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 117 116 112 112 111 109 106 106 106 105 103 101 101 99 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 98 95 94 94 93 92 92 90 89 89 89 86 85 84 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 81 79 78 76 75 75 72 70 70 70 69 68 66 64 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 62 62 62 60 59 59 58 56 55 51 50 
 synt1$cvfit$cv.boxind.support
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 1.00 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.92 0.90 0.90 0.89 0.88 0.87 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 0.85 0.84 0.83 0.83 0.81 0.80 0.80 0.78 0.78 0.77 0.75 0.75 0.74 0.73 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 0.72 0.72 0.71 0.70 0.69 0.67 0.66 0.64 0.63 0.63 0.62 0.60 0.59 0.58 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 0.58 0.57 0.56 0.55 0.54 0.53 0.52 0.50 0.50 0.49 0.49 0.49 0.48 0.48 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 0.47 0.46 0.45 0.45 0.44 0.44 0.42 0.42 0.42 0.42 0.41 0.40 0.40 0.40 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 0.39 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.36 0.36 0.36 0.34 0.34 0.34 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 0.32 0.32 0.31 0.30 0.30 0.30 0.29 0.28 0.28 0.28 0.28 0.27 0.26 0.26 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 0.25 0.25 0.25 0.24 0.24 0.24 0.23 0.22 0.22 0.20 0.20 
 synt1$cvfit$cv.boxind[synt1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE TRUE 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE 248 249 250 TRUE FALSE TRUE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
synt1$cvfit$cv.screened
## X1 X2 X3 ## 1 2 3
 synt1$cvfit$cv.used
## X1 X2 X3 ## 1 2 3
 #=====================================================
# Cross-validated covariate traces
#=====================================================
synt1$cvfit$cv.trace
 step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 step10 step11 step12 step13 0 2 2 2 1 1 2 2 2 2 2 2 2 2 step14 step15 step16 step17 step18 step19 step20 step21 step22 step23 step24 step25 step26 step27 1 1 1 1 1 1 1 2 2 2 2 1 1 1 step28 step29 step30 step31 step32 step33 step34 step35 step36 step37 step38 step39 step40 step41 2 2 2 2 1 1 1 1 1 1 1 1 1 1 step42 step43 step44 step45 step46 step47 step48 step49 step50 step51 step52 step53 step54 step55 2 1 2 1 2 2 1 1 2 3 1 2 2 2 step56 step57 step58 step59 step60 step61 step62 step63 step64 step65 step66 step67 step68 step69 2 2 2 2 2 2 1 1 2 2 1 2 1 1 step70 step71 step72 step73 step74 step75 step76 step77 step78 step79 step80 step81 step82 step83 2 2 1 1 2 1 1 2 1 1 1 1 1 1 step84 step85 step86 step87 step88 step89 step90 step91 step92 step93 step94 step95 step96 step97 1 1 1 1 1 1 1 1 1 1 1 1 1 1 step98 step99 step100 step101 step102 step103 step104 step105 step106 step107 step108 1 1 1 1 1 1 1 1 1 1 1 
 #=====================================================
# Cross-validated decision rules
#=====================================================
synt1$cvfit$cv.rules$frame
 X1 X2 X3
step0 X1>=0.00 +/- 0.00 X2<=1.00 +/- 0.00 X3<=1.00 +/- 0.00
step1 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.01 X3<=1.00 +/- 0.00
step2 X1>=0.01 +/- 0.01 X2<=0.98 +/- 0.00 X3<=1.00 +/- 0.00
step3 X1>=0.02 +/- 0.02 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step4 X1>=0.03 +/- 0.01 X2<=0.97 +/- 0.01 X3<=1.00 +/- 0.00
step5 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step6 X1>=0.04 +/- 0.01 X2<=0.96 +/- 0.01 X3<=1.00 +/- 0.00
step7 X1>=0.04 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step8 X1>=0.05 +/- 0.01 X2<=0.94 +/- 0.01 X3<=1.00 +/- 0.00
step9 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step10 X1>=0.05 +/- 0.01 X2<=0.93 +/- 0.01 X3<=1.00 +/- 0.00
step11 X1>=0.05 +/- 0.01 X2<=0.92 +/- 0.01 X3<=1.00 +/- 0.00
step12 X1>=0.05 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.00
step13 X1>=0.06 +/- 0.01 X2<=0.91 +/- 0.01 X3<=1.00 +/- 0.01
step14 X1>=0.06 +/- 0.01 X2<=0.90 +/- 0.01 X3<=0.99 +/- 0.01
step15 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step16 X1>=0.07 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step17 X1>=0.08 +/- 0.01 X2<=0.89 +/- 0.01 X3<=0.99 +/- 0.01
step18 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step19 X1>=0.08 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.99 +/- 0.01
step20 X1>=0.09 +/- 0.01 X2<=0.88 +/- 0.01 X3<=0.98 +/- 0.01
step21 X1>=0.09 +/- 0.01 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step22 X1>=0.09 +/- 0.02 X2<=0.87 +/- 0.01 X3<=0.98 +/- 0.01
step23 X1>=0.09 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step24 X1>=0.10 +/- 0.02 X2<=0.86 +/- 0.01 X3<=0.98 +/- 0.01
step25 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step26 X1>=0.10 +/- 0.02 X2<=0.85 +/- 0.02 X3<=0.98 +/- 0.01
step27 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step28 X1>=0.11 +/- 0.02 X2<=0.84 +/- 0.01 X3<=0.98 +/- 0.01
step29 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step30 X1>=0.12 +/- 0.02 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step31 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step32 X1>=0.12 +/- 0.03 X2<=0.83 +/- 0.01 X3<=0.98 +/- 0.01
step33 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step34 X1>=0.13 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step35 X1>=0.14 +/- 0.03 X2<=0.82 +/- 0.01 X3<=0.98 +/- 0.01
step36 X1>=0.14 +/- 0.03 X2<=0.81 +/- 0.02 X3<=0.97 +/- 0.02
step37 X1>=0.15 +/- 0.03 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step38 X1>=0.16 +/- 0.02 X2<=0.80 +/- 0.02 X3<=0.97 +/- 0.02
step39 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.02 X3<=0.97 +/- 0.02
step40 X1>=0.16 +/- 0.02 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step41 X1>=0.17 +/- 0.03 X2<=0.79 +/- 0.03 X3<=0.97 +/- 0.02
step42 X1>=0.17 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step43 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.97 +/- 0.02
step44 X1>=0.18 +/- 0.03 X2<=0.78 +/- 0.03 X3<=0.96 +/- 0.02
step45 X1>=0.18 +/- 0.03 X2<=0.77 +/- 0.03 X3<=0.96 +/- 0.02
step46 X1>=0.19 +/- 0.03 X2<=0.76 +/- 0.03 X3<=0.96 +/- 0.02
step47 X1>=0.20 +/- 0.04 X2<=0.75 +/- 0.04 X3<=0.96 +/- 0.02
step48 X1>=0.20 +/- 0.03 X2<=0.74 +/- 0.04 X3<=0.95 +/- 0.03
step49 X1>=0.21 +/- 0.04 X2<=0.73 +/- 0.04 X3<=0.95 +/- 0.03
step50 X1>=0.21 +/- 0.04 X2<=0.72 +/- 0.04 X3<=0.95 +/- 0.03
step51 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.94 +/- 0.03
step52 X1>=0.22 +/- 0.03 X2<=0.72 +/- 0.04 X3<=0.93 +/- 0.03
step53 X1>=0.22 +/- 0.03 X2<=0.71 +/- 0.04 X3<=0.93 +/- 0.03
step54 X1>=0.23 +/- 0.03 X2<=0.71 +/- 0.03 X3<=0.93 +/- 0.03
step55 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step56 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.93 +/- 0.03
step57 X1>=0.23 +/- 0.03 X2<=0.70 +/- 0.03 X3<=0.92 +/- 0.03
step58 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step59 X1>=0.24 +/- 0.03 X2<=0.69 +/- 0.03 X3<=0.92 +/- 0.03
step60 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step61 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step62 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.92 +/- 0.03
step63 X1>=0.24 +/- 0.03 X2<=0.68 +/- 0.03 X3<=0.91 +/- 0.03
step64 X1>=0.25 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step65 X1>=0.26 +/- 0.03 X2<=0.67 +/- 0.03 X3<=0.91 +/- 0.02
step66 X1>=0.26 +/- 0.03 X2<=0.66 +/- 0.03 X3<=0.91 +/- 0.02
step67 X1>=0.26 +/- 0.02 X2<=0.66 +/- 0.04 X3<=0.90 +/- 0.02
step68 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step69 X1>=0.27 +/- 0.03 X2<=0.65 +/- 0.04 X3<=0.90 +/- 0.02
step70 X1>=0.27 +/- 0.03 X2<=0.64 +/- 0.04 X3<=0.90 +/- 0.02
step71 X1>=0.27 +/- 0.03 X2<=0.63 +/- 0.04 X3<=0.90 +/- 0.02
step72 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step73 X1>=0.28 +/- 0.04 X2<=0.62 +/- 0.04 X3<=0.90 +/- 0.02
step74 X1>=0.29 +/- 0.03 X2<=0.62 +/- 0.03 X3<=0.90 +/- 0.02
step75 X1>=0.29 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.90 +/- 0.02
step76 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step77 X1>=0.30 +/- 0.04 X2<=0.61 +/- 0.03 X3<=0.89 +/- 0.02
step78 X1>=0.31 +/- 0.04 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step79 X1>=0.32 +/- 0.03 X2<=0.60 +/- 0.03 X3<=0.89 +/- 0.02
step80 X1>=0.32 +/- 0.04 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step81 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step82 X1>=0.33 +/- 0.03 X2<=0.59 +/- 0.03 X3<=0.89 +/- 0.02
step83 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step84 X1>=0.34 +/- 0.03 X2<=0.58 +/- 0.03 X3<=0.89 +/- 0.02
step85 X1>=0.35 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step86 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step87 X1>=0.36 +/- 0.03 X2<=0.57 +/- 0.03 X3<=0.89 +/- 0.02
step88 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step89 X1>=0.37 +/- 0.03 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step90 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step91 X1>=0.37 +/- 0.04 X2<=0.56 +/- 0.03 X3<=0.89 +/- 0.02
step92 X1>=0.38 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step93 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step94 X1>=0.39 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step95 X1>=0.40 +/- 0.04 X2<=0.55 +/- 0.02 X3<=0.89 +/- 0.02
step96 X1>=0.41 +/- 0.03 X2<=0.54 +/- 0.01 X3<=0.89 +/- 0.02
step97 X1>=0.42 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step98 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step99 X1>=0.43 +/- 0.03 X2<=0.54 +/- 0.02 X3<=0.89 +/- 0.02
step100 X1>=0.44 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step101 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step102 X1>=0.45 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step103 X1>=0.46 +/- 0.03 X2<=0.53 +/- 0.02 X3<=0.88 +/- 0.02
step104 X1>=0.46 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step105 X1>=0.47 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step106 X1>=0.48 +/- 0.03 X2<=0.52 +/- 0.02 X3<=0.88 +/- 0.03
step107 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
step108 X1>=0.49 +/- 0.02 X2<=0.52 +/- 0.03 X3<=0.88 +/- 0.03
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
synt1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 250 0.00 0.00 1.00 0.62 0.5 3.00 0.42
step1 0.98 245 0.46 1.30 0.49 0.61 0.5 3.00 0.42
step2 0.97 243 0.64 2.44 0.49 0.60 0.5 3.00 0.41
step3 0.96 240 0.69 3.59 0.48 0.55 0.5 3.00 0.41
step4 0.95 237 0.71 4.58 0.48 0.50 0.5 3.00 0.40
step5 0.94 235 0.73 5.57 0.47 0.47 0.5 3.00 0.40
step6 0.93 233 0.74 6.62 0.47 0.45 0.5 3.00 0.40
step7 0.92 230 0.78 8.17 0.46 0.42 0.5 3.00 0.39
step8 0.92 229 0.81 9.84 0.45 0.40 0.5 3.00 0.39
step9 0.90 225 0.84 11.56 0.45 0.37 0.5 3.00 0.38
step10 0.90 224 0.86 12.69 0.44 0.36 0.5 3.00 0.38
step11 0.89 222 0.89 14.46 0.44 0.33 0.5 3.00 0.37
step12 0.88 220 0.92 16.44 0.43 0.32 0.5 3.00 0.37
step13 0.87 218 0.93 17.64 0.43 0.31 0.5 3.00 0.36
step14 0.85 213 0.95 19.46 0.42 0.30 0.5 3.00 0.36
step15 0.84 211 0.95 20.31 0.42 0.29 0.5 3.00 0.35
step16 0.83 208 0.95 21.40 0.42 0.27 0.5 3.00 0.35
step17 0.83 208 0.96 22.27 0.41 0.25 0.5 3.00 0.34
step18 0.81 202 0.97 23.33 0.41 0.23 0.5 3.00 0.34
step19 0.80 201 0.97 24.24 0.41 0.20 0.5 3.00 0.34
step20 0.80 201 0.97 25.14 0.40 0.19 0.5 3.00 0.33
step21 0.78 196 0.99 26.49 0.40 0.17 0.5 3.00 0.33
step22 0.78 194 1.00 27.78 0.39 0.17 0.5 3.00 0.33
step23 0.77 192 1.01 28.78 0.39 0.16 0.5 3.00 0.32
step24 0.75 188 1.03 30.76 0.39 0.16 0.5 3.00 0.32
step25 0.75 188 1.04 31.73 0.38 0.15 0.5 3.00 0.31
step26 0.74 185 1.04 32.69 0.38 0.15 0.5 3.00 0.31
step27 0.73 183 1.05 33.78 0.38 0.14 0.5 3.00 0.30
step28 0.72 181 1.06 34.64 0.38 0.13 0.5 3.00 0.30
step29 0.72 179 1.07 35.51 0.37 0.12 0.5 3.00 0.30
step30 0.71 178 1.08 36.83 0.37 0.12 0.5 3.00 0.29
step31 0.70 175 1.10 38.25 0.37 0.10 0.5 3.00 0.29
step32 0.69 172 1.11 39.77 0.36 0.09 0.5 3.00 0.28
step33 0.67 168 1.13 41.74 0.36 0.09 0.5 3.00 0.28
step34 0.66 165 1.16 43.71 0.35 0.08 0.5 3.00 0.27
step35 0.64 160 1.19 46.08 0.35 0.08 0.5 3.00 0.26
step36 0.63 158 1.22 49.13 0.34 0.07 0.5 3.00 0.26
step37 0.63 157 1.26 52.42 0.34 0.06 0.5 3.00 0.25
step38 0.62 154 1.30 55.42 0.34 0.05 0.5 3.00 0.24
step39 0.60 150 1.33 58.39 0.33 0.05 0.5 3.00 0.23
step40 0.59 147 1.37 61.31 0.33 0.04 0.5 2.99 0.23
step41 0.58 145 1.39 63.70 0.32 0.04 0.5 2.99 0.22
step42 0.58 144 1.43 66.19 0.32 0.04 0.5 2.99 0.22
step43 0.57 142 1.46 69.23 0.32 0.04 0.5 2.99 0.21
step44 0.56 140 1.51 72.80 0.31 0.03 0.5 2.99 0.20
step45 0.55 137 1.55 76.45 0.31 0.03 0.5 2.99 0.19
step46 0.54 134 1.59 79.38 0.30 0.02 0.5 2.98 0.19
step47 0.53 133 1.64 83.21 0.30 0.02 0.5 2.97 0.18
step48 0.52 129 1.66 84.43 0.30 0.02 0.5 2.96 0.18
step49 0.50 126 1.71 88.77 0.29 0.02 0.5 2.93 0.18
step50 0.50 124 1.77 92.72 0.29 0.02 0.5 2.91 0.17
step51 0.49 122 1.81 95.97 0.29 0.01 0.5 2.89 0.16
step52 0.49 122 1.83 96.81 0.29 0.01 0.5 2.88 0.16
step53 0.49 122 1.85 98.13 0.29 0.01 0.5 2.87 0.16
step54 0.48 121 1.89 101.28 0.29 0.01 0.5 2.82 0.15
step55 0.48 121 1.93 103.87 0.28 0.01 0.5 2.80 0.15
step56 0.47 117 1.96 106.66 0.28 0.01 0.5 2.80 0.15
step57 0.46 116 1.99 108.23 0.28 0.01 0.5 2.77 0.14
step58 0.45 112 2.02 109.75 0.28 0.01 0.5 2.77 0.14
step59 0.45 112 2.04 111.15 0.28 0.01 0.5 2.75 0.14
step60 0.44 111 2.07 112.69 0.28 0.01 0.5 2.75 0.14
step61 0.44 109 2.10 115.10 0.28 0.01 0.5 2.73 0.13
step62 0.42 106 2.14 117.14 0.28 0.01 0.5 2.73 0.13
step63 0.42 106 2.16 118.20 0.28 0.01 0.5 2.73 0.13
step64 0.42 106 2.18 119.07 0.28 0.01 0.5 2.73 0.13
step65 0.42 105 2.22 121.54 0.28 0.01 0.5 2.71 0.13
step66 0.41 103 2.26 123.51 0.28 0.00 0.5 2.71 0.12
step67 0.40 101 2.30 125.95 0.27 0.00 0.5 2.68 0.12
step68 0.40 101 2.32 126.38 0.27 0.00 0.5 2.68 0.12
step69 0.40 99 2.34 127.68 0.27 0.00 0.5 2.67 0.12
step70 0.39 98 2.39 130.23 0.27 0.00 0.5 2.65 0.11
step71 0.38 95 2.42 132.04 0.27 0.00 0.5 2.64 0.11
step72 0.38 94 2.47 134.49 0.27 0.00 0.5 2.63 0.11
step73 0.38 94 2.50 135.93 0.27 0.00 0.5 2.63 0.10
step74 0.37 93 2.54 137.01 0.27 0.00 0.5 2.62 0.10
step75 0.37 92 2.59 139.86 0.27 0.00 0.5 2.60 0.10
step76 0.37 92 2.62 141.57 0.27 0.00 0.5 2.59 0.10
step77 0.36 90 2.66 143.33 0.27 0.00 0.5 2.59 0.09
step78 0.36 89 2.70 144.76 0.28 0.00 0.5 2.59 0.09
step79 0.36 89 2.76 148.41 0.27 0.00 0.5 2.59 0.08
step80 0.36 89 2.81 150.49 0.28 0.00 0.5 2.59 0.08
step81 0.34 86 2.86 152.48 0.28 0.00 0.5 2.57 0.08
step82 0.34 85 2.90 153.91 0.28 0.00 0.5 2.54 0.07
step83 0.34 84 3.00 159.64 0.28 0.00 0.5 2.51 0.07
step84 0.32 81 3.07 162.31 0.28 0.00 0.5 2.50 0.06
step85 0.32 79 3.15 166.45 0.28 0.00 0.5 2.47 0.06
step86 0.31 78 3.22 169.81 0.28 0.00 0.5 2.43 0.05
step87 0.30 76 3.28 171.06 0.28 0.00 0.5 2.38 0.05
step88 0.30 75 3.39 176.15 0.28 0.00 0.5 2.31 0.05
step89 0.30 75 3.49 180.62 0.28 0.00 0.5 2.26 0.04
step90 0.29 72 3.58 185.35 0.28 0.00 0.5 2.12 0.04
step91 0.28 70 3.66 188.02 0.28 0.00 0.5 2.10 0.03
step92 0.28 70 3.74 191.05 0.28 0.00 0.5 1.86 0.03
step93 0.28 70 3.87 196.70 0.28 0.00 0.5 1.67 0.02
step94 0.28 69 3.91 197.15 0.28 0.00 0.5 1.62 0.02
step95 0.27 68 3.99 200.17 0.28 0.00 0.5 1.52 0.02
step96 0.26 66 4.08 201.39 0.29 0.00 0.5 1.43 0.02
step97 0.26 64 4.22 206.84 0.29 0.00 0.5 1.17 0.01
step98 0.25 62 4.29 208.25 0.29 0.00 0.5 1.13 0.01
step99 0.25 62 4.21 210.65 0.30 0.00 0.5 1.07 0.01
step100 0.25 62 4.29 211.90 0.30 0.00 0.5 1.04 0.01
step101 0.24 60 4.27 215.01 0.31 0.00 0.5 0.85 0.01
step102 0.24 59 3.99 212.21 0.32 0.00 0.5 0.81 0.01
step103 0.24 59 4.03 212.45 0.33 0.00 0.5 0.76 0.01
step104 0.23 58 3.12 211.73 0.37 0.00 0.5 0.72 0.01
step105 0.22 56 2.84 213.76 0.38 0.00 0.5 0.66 0.01
step106 0.22 55 2.66 214.82 0.40 0.00 0.5 0.54 0.01
step107 0.20 51 1.72 213.71 0.43 0.00 0.5 0.51 0.01
step108 0.20 50 1.40 215.13 0.45 0.00 0.5 0.49 0.01
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0.00 0.00
step1 0.00 1.03 0.43 1.00 0.00 0.01 0 0.00 0.00
step2 0.01 1.78 0.22 1.56 0.01 0.03 0 0.00 0.00
step3 0.01 1.87 0.17 1.69 0.01 0.06 0 0.00 0.00
step4 0.01 2.08 0.10 1.35 0.01 0.04 0 0.00 0.00
step5 0.01 1.87 0.10 1.49 0.01 0.04 0 0.00 0.00
step6 0.01 1.93 0.10 1.93 0.01 0.04 0 0.00 0.00
step7 0.01 2.00 0.09 1.98 0.01 0.02 0 0.00 0.00
step8 0.01 2.34 0.09 2.52 0.01 0.03 0 0.00 0.01
step9 0.01 2.26 0.08 2.58 0.01 0.03 0 0.00 0.01
step10 0.01 2.43 0.07 2.39 0.01 0.03 0 0.00 0.01
step11 0.01 2.11 0.06 2.40 0.01 0.03 0 0.00 0.01
step12 0.01 2.36 0.07 2.85 0.01 0.02 0 0.00 0.01
step13 0.01 2.35 0.07 3.11 0.01 0.02 0 0.00 0.01
step14 0.01 2.26 0.07 3.10 0.01 0.01 0 0.00 0.01
step15 0.01 2.01 0.06 2.96 0.01 0.01 0 0.00 0.01
step16 0.01 2.22 0.05 2.61 0.01 0.03 0 0.00 0.01
step17 0.01 2.30 0.06 2.95 0.01 0.04 0 0.00 0.01
step18 0.01 2.43 0.06 3.03 0.01 0.05 0 0.00 0.01
step19 0.01 2.80 0.06 3.01 0.01 0.03 0 0.00 0.01
step20 0.01 2.64 0.05 3.00 0.01 0.03 0 0.00 0.01
step21 0.01 2.49 0.05 3.22 0.01 0.01 0 0.00 0.01
step22 0.01 2.62 0.05 2.91 0.01 0.01 0 0.00 0.01
step23 0.01 2.58 0.05 3.19 0.01 0.01 0 0.00 0.01
step24 0.01 2.70 0.05 3.30 0.01 0.01 0 0.00 0.01
step25 0.01 2.18 0.05 3.30 0.01 0.01 0 0.00 0.01
step26 0.01 2.18 0.05 3.79 0.01 0.01 0 0.00 0.01
step27 0.01 2.00 0.06 3.89 0.01 0.01 0 0.00 0.01
step28 0.01 2.27 0.06 3.89 0.01 0.02 0 0.00 0.01
step29 0.01 2.93 0.05 3.50 0.01 0.02 0 0.00 0.01
step30 0.01 3.61 0.06 4.24 0.01 0.02 0 0.00 0.01
step31 0.01 3.59 0.06 4.64 0.01 0.02 0 0.00 0.01
step32 0.02 3.97 0.06 4.28 0.01 0.01 0 0.00 0.01
step33 0.01 3.66 0.06 4.97 0.01 0.01 0 0.00 0.01
step34 0.02 3.85 0.07 5.71 0.01 0.01 0 0.00 0.01
step35 0.02 4.27 0.08 6.27 0.01 0.01 0 0.00 0.01
step36 0.02 4.03 0.09 7.25 0.01 0.01 0 0.00 0.01
step37 0.02 4.18 0.09 7.33 0.01 0.01 0 0.00 0.02
step38 0.01 3.67 0.09 7.77 0.01 0.01 0 0.00 0.02
step39 0.01 2.97 0.08 7.20 0.01 0.01 0 0.00 0.01
step40 0.01 3.18 0.09 7.42 0.01 0.01 0 0.01 0.02
step41 0.01 3.05 0.10 8.25 0.01 0.01 0 0.01 0.02
step42 0.02 3.87 0.11 9.23 0.01 0.00 0 0.01 0.02
step43 0.01 3.52 0.10 8.97 0.01 0.00 0 0.01 0.02
step44 0.01 3.22 0.09 8.35 0.01 0.00 0 0.01 0.02
step45 0.01 2.87 0.09 8.67 0.01 0.01 0 0.03 0.02
step46 0.01 2.98 0.09 8.61 0.01 0.01 0 0.06 0.02
step47 0.01 2.82 0.10 9.25 0.01 0.01 0 0.07 0.02
step48 0.01 3.15 0.11 10.13 0.01 0.01 0 0.07 0.02
step49 0.01 3.39 0.13 11.26 0.01 0.01 0 0.10 0.02
step50 0.01 3.54 0.14 11.78 0.01 0.00 0 0.12 0.02
step51 0.01 3.55 0.12 10.19 0.01 0.00 0 0.13 0.02
step52 0.01 3.11 0.11 9.41 0.01 0.00 0 0.14 0.02
step53 0.01 3.18 0.10 8.76 0.01 0.00 0 0.15 0.02
step54 0.01 3.11 0.09 8.13 0.01 0.00 0 0.16 0.01
step55 0.01 3.08 0.09 8.24 0.01 0.00 0 0.16 0.01
step56 0.01 3.05 0.10 8.62 0.01 0.00 0 0.16 0.01
step57 0.01 2.90 0.10 8.14 0.01 0.00 0 0.17 0.01
step58 0.01 2.92 0.10 8.69 0.01 0.00 0 0.17 0.01
step59 0.01 3.00 0.11 9.32 0.01 0.00 0 0.16 0.01
step60 0.01 2.83 0.11 9.32 0.01 0.00 0 0.16 0.01
step61 0.01 2.94 0.11 9.21 0.01 0.00 0 0.16 0.01
step62 0.01 2.43 0.11 9.40 0.01 0.00 0 0.16 0.01
step63 0.01 2.73 0.11 9.17 0.01 0.00 0 0.16 0.01
step64 0.01 3.21 0.13 9.53 0.01 0.00 0 0.16 0.01
step65 0.01 2.92 0.11 8.77 0.01 0.00 0 0.15 0.01
step66 0.01 3.17 0.12 8.83 0.01 0.00 0 0.15 0.01
step67 0.01 3.07 0.13 9.60 0.01 0.00 0 0.14 0.01
step68 0.01 3.19 0.15 11.40 0.01 0.00 0 0.14 0.01
step69 0.01 3.20 0.16 11.68 0.01 0.00 0 0.14 0.01
step70 0.01 3.12 0.16 12.09 0.01 0.00 0 0.13 0.01
step71 0.01 3.22 0.17 12.27 0.01 0.00 0 0.13 0.01
step72 0.01 2.94 0.17 11.77 0.01 0.00 0 0.12 0.01
step73 0.01 3.21 0.18 12.96 0.01 0.00 0 0.12 0.01
step74 0.01 2.93 0.19 13.08 0.01 0.00 0 0.11 0.01
step75 0.01 2.73 0.18 12.36 0.01 0.00 0 0.07 0.01
step76 0.01 2.83 0.17 11.59 0.01 0.00 0 0.05 0.01
step77 0.01 2.69 0.16 11.02 0.01 0.00 0 0.05 0.01
step78 0.01 2.70 0.16 12.16 0.01 0.00 0 0.05 0.01
step79 0.01 2.59 0.19 13.86 0.01 0.00 0 0.05 0.02
step80 0.01 2.81 0.21 14.97 0.01 0.00 0 0.05 0.02
step81 0.01 2.59 0.21 14.84 0.01 0.00 0 0.11 0.02
step82 0.01 3.00 0.22 16.35 0.01 0.00 0 0.11 0.02
step83 0.01 3.17 0.26 18.24 0.01 0.00 0 0.16 0.02
step84 0.01 3.00 0.28 19.50 0.01 0.00 0 0.18 0.02
step85 0.01 3.16 0.29 19.96 0.01 0.00 0 0.23 0.02
step86 0.01 3.35 0.30 20.69 0.01 0.00 0 0.27 0.02
step87 0.01 3.29 0.33 21.45 0.01 0.00 0 0.29 0.02
step88 0.01 3.23 0.33 22.58 0.01 0.00 0 0.42 0.02
step89 0.01 3.09 0.33 22.96 0.01 0.00 0 0.51 0.02
step90 0.01 2.89 0.34 23.52 0.01 0.00 0 0.63 0.02
step91 0.01 2.85 0.32 23.03 0.01 0.00 0 0.62 0.02
step92 0.01 2.91 0.41 27.85 0.01 0.00 0 0.75 0.02
step93 0.01 2.68 0.39 27.08 0.01 0.00 0 0.76 0.02
step94 0.01 2.73 0.41 27.64 0.01 0.00 0 0.79 0.02
step95 0.01 2.56 0.41 28.23 0.01 0.00 0 0.78 0.02
step96 0.01 3.32 0.40 26.91 0.01 0.00 0 0.77 0.02
step97 0.01 3.46 0.38 24.04 0.01 0.00 0 0.64 0.02
step98 0.01 3.39 0.36 23.34 0.01 0.00 0 0.64 0.02
step99 0.01 3.47 0.88 24.57 0.04 0.00 0 0.62 0.01
step100 0.01 3.44 0.87 21.17 0.04 0.00 0 0.60 0.01
step101 0.01 3.51 1.21 20.39 0.05 0.00 0 0.51 0.01
step102 0.01 3.13 1.62 18.37 0.07 0.00 0 0.47 0.01
step103 0.01 2.83 1.64 18.61 0.07 0.00 0 0.43 0.01
step104 0.01 3.09 2.26 19.43 0.10 0.00 0 0.43 0.01
step105 0.01 3.13 2.37 19.10 0.10 0.00 0 0.39 0.01
step106 0.01 2.86 2.55 20.86 0.10 0.00 0 0.20 0.01
step107 0.01 2.85 2.49 20.93 0.09 0.00 0 0.21 0.01
step108 0.01 3.06 2.37 21.52 0.09 0.00 0 0.21 0.01

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`X1`,`X2`) plane
# Synthetic dataset
#=====================================================
plot(x = synt1,
main = paste("Scatter plot projection of data in (`X1`,`X2`) plane \n (synthetic dataset)", sep=""),
proj = c(1,2), steps = synt1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", synt1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL, horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# Synthetic dataset
#=====================================================
plot_profile(object = synt1,
main = "CV tuning profiles \n (synthetic dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# Synthetic dataset
#=====================================================
plot_traj(object = synt1,
main = "CV peeling trajectories \n (synthetic dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE) 

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# Synthetic dataset
#=====================================================
plot_trace(object = synt1,
main = "CV trace plots \n (synthetic dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = synt1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_synthetic_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (synthetic dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

...

 #=====================================================
# Plot of CV survival probability curves (without p-values)
# Synthetic dataset
#=====================================================
plot_km(object = synt1,
main = "CV survival probability curves \n (clinical dataset - without p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:synt1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_synthetic_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

A real-world example

Illustration of the Survival Bump Hunting (SBH) search in a publicly available clinical dataset. This data is from the Women's Interagency HIV cohort Study (WIHS) described in the PRIMsrc package as demo dataset Real.1.

Data

The dataset consists of a numeric data.frame containing n = 485 complete observations (samples) by rows and p = 4 clinical covariates by columns, not including the censoring indicator and (censored) time-to-event variables.

 #===================================================
# Demo with a real dataset
# Use help for descriptions
#===================================================
data("Real.1", package="PRIMsrc")
?Real.1
head(Real.1)
 y delta Age IDU Race CD4.count
1 0.02 1 48 0 1 6.95
2 0.02 1 35 1 1 2.51
3 0.02 1 28 0 1 0.18
4 0.02 1 46 1 0 4.65
5 0.02 1 31 0 1 0.08
6 0.02 1 45 1 1 2.05

Code

Specification of the cluster parameters for setting-up a parallel backend on a local machine, e.g. a Quad core PC platform running WINDOWS with SOCKET communication:

 if (require("parallel")) {
print("'parallel' is attached correctly \n")
} else {
stop("'parallel' must be attached first \n")
}
## Loading required package: parallel
## [1] "'parallel' is attached correctly \n"
 cpus <- detectCores(logical = TRUE)
conf <- list("spec" = rep("localhost", cpus),
"type" = "SOCKET",
"homo" = TRUE,
"verbose" = TRUE,
"outfile" = paste(getwd(), "/real.log", sep=""))

Specification of the Penalized Partial Likelihood (PPL) parameters for variable pre-selection:

 alpha <- 1
nalpha <- 1
nlambda <- 100
vsarg <- paste("alpha=", alpha, ",nalpha=", nalpha, ",nlambda=", nlambda, sep="")

Specification of the PRSP parameters:

 alpha <- 0.05
beta <- 0.05
peelcriterion <- "lrt"
cvcriterion <- "lrt"
cvarg <- paste("alpha=", alpha, ",beta=", beta, ",peelcriterion=\"", peelcriterion, "\",cvcriterion=\"", cvcriterion, "\"", sep="")

In this instance, parameters are taken as:

  • Peeling quantile (Patience/Greediness trade-off): alpha=0.01
  • Minimal Box support (Stopping Rule): beta=0.10
  • Peeling criterion is Log-Rank Test; peelcriterion="lrt"
  • Cross-validation criterion is Log-Rank Test: cvcriterion = "lrt"

Note that the parameters in cvarg come as a string of characters between double quotes, with comas separated values, and without white spaces. In the above example, it will look like this:

cvarg="alpha=0.01,beta=0.10,peelcriterion=\"lrt\",cvcriterion=\"lrt\""

Here, we use cross-validation with a Replicated Combined Cross-Validation (RCCV) technique (cv = TRUE, cvtype = "combined") that is replicated (B = 30) times with 5-fold cross-validation (K = 5). Also, the SBH search is carried out here with variable pre-selection and default Penalized Partial Likelihood variable pre-selection method (vs = TRUE, vstype = "ppl").

Likewise, parameter vsarg come as a string of characters between double quotes, with comas separated values without white spaces. In the above example, it will look like this:

vsarg="alpha=1,nalpha=1,nlambda=100"

Other ancillary parameters are: computation of log-rank permutation p-values (A = 1000, pv = TRUE), medium conservativeness of variable screening (vscons = 0.5), adequate smoothing (span = 0.35), usage of 2 decimals precision of outputs (decimals = 2), extremum cross-validation rule for tuning/optimizing model parameters (onese = FALSE), median survival probability at which we want to get the endpoint box survival time (probval = 0.5), maximal survival time at which we want to get the endpoint box survival probability(timeval = NULL), with parallelization of replications and computation of p-values (parallel.vs = TRUE, parallel.rep = TRUE, parallel.pv = TRUE, conf=conf), verbose option (``verbose = FALSE) and a seed (seed = 123`).

The code and its output are shown below:

 #=====================================================
# Computation of PRSP object
#=====================================================
real1 <- sbh(X = Real.1[ , -c(1,2), drop=FALSE],
y = Real.1[ ,1, drop=TRUE],
delta = Real.1[ ,2, drop=TRUE],
B = 30,
K = 5,
A = 1000,
vs = TRUE,
cv = TRUE,
vstype = "ppl",
vsarg = vsarg,
cvarg = cvarg,
pv = TRUE,
control = sbh.control(probval = 0.5, vscons = 0.5, span = 0.35), parallel.vs = TRUE,
parallel.rep = TRUE,
parallel.pv = TRUE,
conf = conf,
verbose = FALSE,
seed = 123)
Requested parallel replicated 5-fold cross-validated procedure with 32 replications. Variable screening: TRUE Variable screening technique: PPL Cross-validation: TRUE Cross-validation technique: COMBINED PRSP cross-validation criterion: LRT PRSP Peeling criterion: LRT PRSP Peeling percentile: 1 %
PRSP Minimal box support: 10 %
Computation of p-values: TRUE Decision rule: EXTREMUM Parallelization of computation of variable screening: FALSE Parallelization of computation of PRSP: TRUE Parallelization of computation of p-values: TRUE Screening of informative covariates ... Successfully completed screening of covariates. Covariates screened: Age IDU Race CD4.count 1 2 3 4 Directions of directed peeling of screened covariates: Age IDU Race CD4.count 1 1 -1 -1 Fitting the Survival Bump Hunting model using the PRSP algorithm ... Generating cross-validated profiles of peeling steps and optimal peeling lengths from all replicates ...
Generating box memberships ...
Successfully completed PRSP algorithm. Covariates used: Age CD4.count 1 4 Directions of directed peeling of used covariates: Age CD4.count 1 -1 Generating box rules of used covariates ...
Generating box statistics ...
Computation of p-values ... Finished!

Results

The generated 'sbh' object is real1, that is used for other functions.

S3-method summary function to summarize the main parameters used to generate the 'sbh' object:

 summary(real1)
S3-class object: ' sbh ' Replicated 5-fold cross-validated procedure with 32 replications. VARIABLE SCREENING:
Variable screening: TRUE Variable screening technique: PPL CROSS-VALIDATION:
Cross-validation: TRUE Cross-validation technique: COMBINED PRSP PARAMETERS:
Cross-validation criterion: LRT Peeling criterion: LRT Peeling percentile: 1 %
Minimal box support: 10 %
REPORTING:
Decision rule: EXTREMUM Number of decimals: 2 Computation of p-values: TRUE

S3-method print function to display all the cross-validated estimated values of the 'sbh' object at once:

 print(real1)

Or, they can be accessed one-at-a-time. See examples below:

 #=====================================================
# Cross-validated maximal and optimal numbers of peeling steps
#=====================================================
real1$cvfit$cv.maxsteps
[1] 15
 real1$cvfit$cv.nsteps
step10 10
 #=====================================================
# Cross-validated box size, support and membership indicator
#=====================================================
real1$cvfit$cv.boxind.size
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 485 482 476 470 454 284 92 79 78 74 
 real1$cvfit$cv.boxind.support
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.00 0.99 0.98 0.97 0.94 0.59 0.19 0.16 0.16 0.15 
 real1$cvfit$cv.boxind[real1$cvfit$cv.nsteps,]
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 TRUE TRUE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 FALSE FALSE TRUE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE 481 482 483 484 485 TRUE FALSE FALSE FALSE FALSE 
 #=====================================================
# Cross-validated screened and used covariates
#=====================================================
real1$cvfit$cv.screened
 Age IDU Race CD4.count 1 2 3 4
 real1$cvfit$cv.used
 Age CD4.count 1 4
 #=====================================================
# Cross-validated covariate traces
#=====================================================
real1$cvfit$cv.trace
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 0 1 4 4 1 1 1 4 4 4
 #=====================================================
# Cross-validated decision rules
#=====================================================
real1$cvfit$cv.rules$frame
 Age CD4.count
step0 Age>=19.00 +/- 0.00 CD4.count<=19.33 +/- 0.00
step1 Age>=20.73 +/- 0.69 CD4.count<=19.11 +/- 1.04
step2 Age>=20.80 +/- 0.61 CD4.count<=12.44 +/- 0.15
step3 Age>=21.13 +/- 0.86 CD4.count<=12.36 +/- 0.18
step4 Age>=21.77 +/- 0.73 CD4.count<=12.21 +/- 0.13
step5 Age>=22.60 +/- 0.86 CD4.count<=12.11 +/- 0.34
step6 Age>=23.50 +/- 1.78 CD4.count<=11.87 +/- 0.52
step7 Age>=25.37 +/- 2.57 CD4.count<=11.73 +/- 0.63
step8 Age>=27.13 +/- 2.22 CD4.count<=11.43 +/- 0.74
step9 Age>=28.20 +/- 1.71 CD4.count<=10.86 +/- 0.73
 #=====================================================
# Cross-validated highest-risk box/group statistics
#=====================================================
real1$cvfit$cv.stats
$mean
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 1.00 485 0.00 0.00 1.00 2.20 0.5 10.8 0.17
step1 0.99 482 0.81 3.84 0.50 2.11 0.5 10.8 0.16
step2 0.98 476 0.53 3.47 0.49 2.09 0.5 10.8 0.17
step3 0.97 470 0.45 3.67 0.49 2.06 0.5 10.8 0.18
step4 0.94 454 0.13 1.68 0.49 2.06 0.5 10.8 0.18
step5 0.59 284 0.11 1.45 0.49 2.03 0.5 10.8 0.17
step6 0.19 92 0.15 2.30 0.48 1.91 0.5 10.8 0.16
step7 0.16 79 0.22 3.62 0.48 1.83 0.5 10.8 0.14
step8 0.16 78 0.23 3.75 0.48 1.87 0.5 10.8 0.13
step9 0.15 74 0.26 4.54 0.48 1.83 0.5 10.8 0.12
$sd
Support Size LHR LRT CER EFT EFP MEFT MEFP
step0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00
step1 0.02 9.66 0.25 1.50 0.00 0.03 0 0 0.00
step2 0.03 15.82 0.18 1.50 0.00 0.06 0 0 0.01
step3 0.05 23.96 0.18 2.23 0.00 0.07 0 0 0.01
step4 0.11 51.04 0.13 2.03 0.01 0.14 0 0 0.01
step5 0.14 65.55 0.08 1.54 0.01 0.24 0 0 0.02
step6 0.10 49.49 0.08 2.01 0.01 0.27 0 0 0.02
step7 0.10 48.30 0.07 2.11 0.01 0.31 0 0 0.02
step8 0.11 51.11 0.09 2.48 0.01 0.33 0 0 0.02
step9 0.08 40.27 0.09 2.66 0.01 0.28 0 0 0.02
 #=====================================================
# Cross-validated log-rank permutation p-values
#=====================================================
real1$cvfit$cv.pval
$pval
step0 step1 step2 step3 step4 step5 step6 step7 step8 step9 1.000 0.099 0.070 0.049 0.177 0.183 0.107 0.049 0.042 0.030 $seed
[1] 123

Plots

The following codes will generate plots on the standard output or screen (device=NULL) or in an output file depending on the specified graphical device (device="PS"):

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Scatter plot of the data in (`Age`,`CD4.count`) plane
# WIHS clinical dataset
#=====================================================
plot(x = real1,
main = paste("Scatter plot projection of data in (`Age`,`CD4.count`) plane \n (clinical dataset)", sep=""),
proj = c(1,2), steps = real1$cvfit$cv.nsteps,
pch = 16, cex = 0.5, col = c(1,2),
boxes = TRUE,
col.box = 2, lty.box = 2, lwd.box = 1,
add.caption.box = TRUE, text.caption.box = paste("Step: ", real1$cvfit$cv.nsteps, sep=""),
device = "PS",
file = "Scatter_plot_projection_of_data_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV tuning profiles
# WIHS clinical dataset
#=====================================================
plot_profile(object = real1,
main = "CV tuning profiles \n (clinical dataset)",
pch = 20, col = 1, lty = 1, lwd = 0.5, cex = 0.5,
add.sd = TRUE, add.profiles = TRUE,
add.caption = TRUE, text.caption = c("Mean","Std. Error"),
device = "PS", file = "CV_tuning_profiles_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV peeling trajectories (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_traj(object = real1,
main = "CV peeling trajectories \n (clinical dataset - used covariates only)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
device = "PS", file = "CV_peeling_trajectories_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 7, height = 7)
 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = NULL,
horizontal = FALSE)

 #=====================================================
# Plot of CV trace plots of covariate importance (used covariates only)
# WIHS clinical dataset
#=====================================================
plot_trace(object = real1,
main = "CV trace plots \n (clinical dataset - used covariates only)",
xlab = "Box Mass", ylab = "Covariate Range (centered)",
col = 1, lty = 1, lwd = 0.5, cex = 0.5,
toplot = real1$cvfit$cv.used,
center = TRUE, scale = FALSE,
device = "PS", file = "CV_trace_plots_clinical_dataset", path = getwd(),
horizontal = FALSE, width = 5, height = 5)
 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = NULL,
horizontal = TRUE)

 #=====================================================
# Plot of CV survival probability curves (with p-values)
# WIHS clinical dataset
#=====================================================
plot_km(object = real1,
main = "CV survival probability curves \n (clinical dataset - with p-values)",
xlab = "Time", ylab = "Probability",
steps = 1:real1$cvfit$cv.nsteps,
nr = 3, nc = 5,
col = c(1,2), lty = 1, lwd = 0.5, cex = 0.5,
add.caption = TRUE,
text.caption = c("outbox","inbox"), device = "PS", file = "CV_survival_probability_curves_clinical_dataset", path = getwd(),
horizontal = TRUE, width = 9, height = 7)

Clone this wiki locally