Extended Platform Tests

This repository holds the non-kubernetes, end-to-end tests that need to pass on a running cluster before PRs merge and/or before we ship a release. These tests are based on ginkgo and the github.com/kubernetes/kubernetes e2e test framework.

Prerequisites

• Git installed.
• Golang installed.
• Have the environment variable KUBECONFIG set pointing to your cluster.

New Test Folder

If you create a new folder for your test case, please add the path to the include.go file.

Compile the executable binary

The generated extended-platform-tests binary in the cmd/extended-platform-tests/ folder. If you want to compile the openshift-tests binary, please see the origin.

$ mkdir -p ${GOPATH}/src/github.com/openshift/
$ cd${GOPATH}/src/github.com/openshift/
$ git clone git@github.com:openshift/openshift-tests.git
$ make clean
$ make build

Run ./extended-platform-tests --help to get started.

This command verifies behavior of an OpenShift cluster by running remote tests against the cluster API that exercise functionality. In general these tests may be disruptive or require elevated privileges - see the descriptions of each test suite.Usage: [command]Available Commands: help Help about any command run Run a test suite run-monitor Continuously verify the cluster is functional run-test Run a single test by name run-upgrade Run an upgrade suiteFlags: -h, --help help for this command

How to run

You can filter your test case by using grep. Such as, For example, to filter the OLM test cases, you can run this command:

$ ./bin/extended-platform-tests run all --dry-run|grep "\[Feature:Platform\] OLM should"I0410 15:33:38.465141 7508 test_context.go:419] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready"[Feature:Platform] OLM should Implement packages API server and list packagemanifest info with namespace not NULL [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should [Serial] olm version should contain the source commit id [Suite:openshift/conformance/serial]""[Feature:Platform] OLM should be installed with catalogsources at version v1alpha1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should be installed with clusterserviceversions at version v1alpha1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should be installed with installplans at version v1alpha1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should be installed with operatorgroups at version v1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should be installed with packagemanifests at version v1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should be installed with subscriptions at version v1alpha1 [Suite:openshift/conformance/parallel]""[Feature:Platform] OLM should have imagePullPolicy:IfNotPresent on thier deployments [Suite:openshift/conformance/parallel]"

You can save the above output to a file and run it:

$ ./bin/extended-platform-tests run -f <your file path/name>

Or you can run it directly:

$ ./bin/extended-platform-tests run all --dry-run | grep "\[Feature:Platform\] OLM should"| ./bin/extended-platform-tests run --junit-dir=./ -f -

How to run a specific test case

It searches the test case title by RE(Regular Expression). So you need to specify the title string detailly. For example, to run this test case: "[Serial] olm version should contain the source commit id", you can do it with 2 ways:

• You may filter the list and pass it back to the run command with the --file argument. You may also pipe a list of test names, one per line, on standard input by passing "-f -".

$ ./bin/extended-platform-tests run all --dry-run|grep "\[Serial\] olm version should contain the source commit id"|./bin/extended-platform-tests run --junit-dir=./ -f -

• You can also run it as follows if you know which test suite it belongs to.

$ ./bin/extended-platform-tests run openshift/conformance/serial --run "\[Serial\] olm version should contain the source commit id"

Debug

Sometime, we want to keep the generated namespace for debugging. Just add the Env Var: export DELETE_NAMESPACE=false. These random namespaces will be keep, like below:

...Dec 18 09:39:33.448: INFO: Running AfterSuite actions on all nodesDec 18 09:39:33.448: INFO: Waiting up to 7m0s for all (but 100) nodes to be readyDec 18 09:39:33.511: INFO: Found DeleteNamespace=false, skipping namespace deletion!Dec 18 09:39:33.511: INFO: Running AfterSuite actions on node 1...1 pass, 0 skip (2m50s)[root@preserve-olm-env openshift-tests-private]# oc get nsNAME STATUS AGEdefault Active 4h46me2e-test-olm-a-a92jyymd-lmgj6 Active 4m28se2e-test-olm-a-a92jyymd-pr8hx Active 4m29s...

How to generate bindata

If you have some new YAML files used in your code, you have to generate the bindata first. Run make update to update the bindata. For example, you can see the bindata has been updated after running the make update. As follows:

$ git status	modified: test/extended/testdata/bindata.go	new file: test/extended/testdata/olm/etcd-subscription-manual.yaml