test-remote suite is located in
It is the main test suite for checking the validity of an Opstrace cluster's health and functionality.
We run this test suite regularly as part of CI for each code commit.
Architecturally, this test suite is designed to be executed locally on your machine for testing a remote Opstrace cluster through the Internet.
Most of the individual tests in this suite inspect cluster properties through the data API endpoints, i.e., from the user's (and their tooling's) point of view. Other tests in this suite may inspect implementation details through private interfaces.
WARNING: that this test suite is invasive. That is, never run it against a production cluster.
The recommended way to run this test suite is via
make test-remote executed at the root directory of the Opstrace repository.
This starts the individual test runner components as Docker containers.
make test-remote, the environment in the current shell needs to be configured, as documented below.
First, set cluster name and cloud provider of the to-be-tested cluster:
export OPSTRACE_CLUSTER_NAME=<opstrace-cluster-name>export OPSTRACE_CLOUD_PROVIDER=<aws or gcp>
Next up, make your local
kubectl configuration point to the running cluster:
Now build or rebuild the container images that the test runner needs (can often be omitted, but if in doubt, run this):
Now invoke the test runner with
Follow the log output.
A property of
make test-remote is that it picks up local changes to test code files.
make test-remote can be used when modifying/developing tests locally.
However, note that the test runner container image and periphery container images need to be rebuilt for more certain changes:
For example, if you change
test/test-remote/package.json you need to rebuild the test runner image with
For larger test development tasks, you may want to run test code directly on your machine within an uncontainerized NodeJS runtime environment.
To that end, enter the test-remote NPM package directory and install the
test-remote NPM package into your current NodeJS environment:
cd test/rest-remoteyarn install
To execute the complete test suite, run the following command (with
test/test-remote being the current working directory):
yarn run mocha
To run a specific (set of) test(s), use a Mocha test filter expression. Example:
yarn run mocha --grep 'short'
Further notes for development and debugging:
- Mocha test (suite) teardown is, as of now, not always reliably executed. A SIGINT-aborted test runner might leave resources behind, such as running child processes.
- Containerized applications (e.g., FluentD) and all kubectl processes leave behind their stdout/err data in the host's /tmp directory. If things go wrong, look in the /tmp directory.
- About test isolation: cleanup does not need to be perfect yet, but please write tests so that they can be run arbitrarily often against the same cluster and still succeed, without relying on a previous test run's side effects to be present. That is, whenever a test inserts data, make it so so that the corresponding metrics or logs injected have unique/random properties (in a log stream label name, for instance).