|
| 1 | +:title: Testing Deis |
| 2 | +:description: How to test Deis |
| 3 | + |
| 4 | +.. _testing: |
| 5 | + |
| 6 | +Testing Deis |
| 7 | +============ |
| 8 | + |
| 9 | +Deis is a distributed system with many moving parts, which makes it of paramount |
| 10 | +importance to test every change thoroughly. |
| 11 | + |
| 12 | +Deis is also a set of components that correspond to directories in the source |
| 13 | +code repository. Most components are Docker containers, two are command-line |
| 14 | +clients, and one contains the documentation. Components have source-code level |
| 15 | +`unit tests`_ and black-box type `functional tests`_. `integration tests`_ |
| 16 | +verify the behavior of the components together as a system. |
| 17 | + |
| 18 | +GitHub pull requests for Deis are tested automatically by a Jenkins |
| 19 | +`continuous integration`_ (CI) system at http://ci.deis.io. Contributors should |
| 20 | +run the same tests locally before proposing any changes to the Deis codebase. |
| 21 | + |
| 22 | + |
| 23 | +Set Up the Environment |
| 24 | +---------------------- |
| 25 | + |
| 26 | +To run all tests, you will need: |
| 27 | + |
| 28 | +- Vagrant 1.6.5 or later |
| 29 | +- VirtualBox 4.3 or later |
| 30 | +- Docker 1.3.0 |
| 31 | +- PostgreSQL server |
| 32 | + |
| 33 | +The tests assume that you have Deis' `source code`_ in your ``$GOPATH``: |
| 34 | + |
| 35 | +.. code-block:: console |
| 36 | +
|
| 37 | + $ go get -u -v github.com/deis/deis |
| 38 | + $ cd $GOPATH/src/github.com/deis/deis |
| 39 | +
|
| 40 | +Disable Docker TLS |
| 41 | +^^^^^^^^^^^^^^^^^^ |
| 42 | + |
| 43 | +If you aren't using Linux, your Docker client probably talks to a Docker server |
| 44 | +over a TCP connection, as in boot2docker_. Since Docker 1.3.0, that connection |
| 45 | +is secured with TLS. Deis' tests use Docker client code directly, but don't yet |
| 46 | +incorporate TLS changes. |
| 47 | + |
| 48 | +For now, you must disable Docker TLS to run Deis' functional tests. Here is how |
| 49 | +to revert to plain TCP for boot2docker_ 1.3.0. You will add the line |
| 50 | +``DOCKER_TLS=""`` to line 24 of */etc/init.d/docker* in your boot2docker VM, so |
| 51 | +that section will look like this: |
| 52 | + |
| 53 | +.. code-block:: bash |
| 54 | +
|
| 55 | + start() { |
| 56 | + DOCKER_TLS="" # <- Add this line and save! |
| 57 | + # Not enabling Docker daemon TLS by default. |
| 58 | + if [ "$DOCKER_TLS" != "" ]; then |
| 59 | +
|
| 60 | +Open a shell to the boot2docker VM, make the change, and restart docker: |
| 61 | +
|
| 62 | +.. code-block:: console |
| 63 | +
|
| 64 | + $ boot2docker up && boot2docker ssh |
| 65 | + $ sudo vi /etc/init.d/docker # edit as above, and save |
| 66 | + $ sudo /etc/init.d/docker restart |
| 67 | + $ exit |
| 68 | +
|
| 69 | +Back on your host machine, ensure your ``DOCKER_HOST`` environment variable |
| 70 | +uses port `2375`: |
| 71 | +
|
| 72 | +.. code-block:: console |
| 73 | +
|
| 74 | + $ export DOCKER_HOST=tcp://192.168.59.103:2375 |
| 75 | + $ unset DOCKER_TLS_VERIFY |
| 76 | + $ docker info |
| 77 | + Containers: 34 |
| 78 | + ... |
| 79 | +
|
| 80 | +Start a Docker Registry |
| 81 | +^^^^^^^^^^^^^^^^^^^^^^^ |
| 82 | +
|
| 83 | +Deis' functional tests build Docker images and test them locally. The images are |
| 84 | +then pushed to a `Docker registry`_ so that integration tests can test them as |
| 85 | +binary artifacts--just as a real-world provisioning of Deis pulls images from |
| 86 | +the Docker Hub. |
| 87 | +
|
| 88 | +If you don't have a Docker registry already accessible for your testing or for |
| 89 | +continuous deployment, start one locally: |
| 90 | +
|
| 91 | +.. code-block:: console |
| 92 | +
|
| 93 | + $ make dev-registry |
| 94 | + registry |
| 95 | +
|
| 96 | + To use local boot2docker registry for Deis development: |
| 97 | + export DEV_REGISTRY=192.168.59.103:5000 |
| 98 | +
|
| 99 | +
|
| 100 | +Run the Tests |
| 101 | +------------- |
| 102 | +
|
| 103 | +The unit and functional tests for each component are in their respective |
| 104 | +directories. The integration tests, scripts, and supporting go packages are in |
| 105 | +the ``tests/`` directory in the project root. |
| 106 | +
|
| 107 | +Scripts in the ``tests/bin`` directory are the best place to start. These test |
| 108 | +individual pieces of Deis, then bring up a Vagrant cluster and test all of them |
| 109 | +as a system. They call ``tests/bin/test-setup.sh`` to test for important |
| 110 | +environment variables and will exit with a helpful message if any are missing. |
| 111 | +
|
| 112 | +test-integration.sh |
| 113 | +^^^^^^^^^^^^^^^^^^^ |
| 114 | +
|
| 115 | +- runs documentation tests |
| 116 | +- builds Docker images tagged with ``$BUILD_TAG`` |
| 117 | +- runs unit and functional tests |
| 118 | +- creates a 3-node Vagrant CoreOS cluster |
| 119 | +- pushes the Docker images to a registry |
| 120 | +- provisions the cluster for Deis with the registry images |
| 121 | +- runs all integration tests |
| 122 | +- takes roughly an hour |
| 123 | +
|
| 124 | +.. code-block:: console |
| 125 | +
|
| 126 | + $ ./tests/bin/test-integration.sh |
| 127 | +
|
| 128 | + >>> Preparing test environment <<< |
| 129 | +
|
| 130 | + DEIS_ROOT=/Users/matt/Projects/src/github.com/deis/deis |
| 131 | + DEIS_TEST_APP=example-go |
| 132 | + ... |
| 133 | + >>> Running integration suite <<< |
| 134 | +
|
| 135 | + make -C tests/ test-full |
| 136 | + ... |
| 137 | + >>> Test run complete <<< |
| 138 | +
|
| 139 | +test-smoke.sh |
| 140 | +^^^^^^^^^^^^^ |
| 141 | +
|
| 142 | +- runs documentation tests |
| 143 | +- builds Docker images tagged with ``$BUILD_TAG`` |
| 144 | +- runs unit and functional tests |
| 145 | +- creates a 3-node Vagrant CoreOS cluster |
| 146 | +- pushes the Docker images to a registry |
| 147 | +- provisions the cluster for Deis with the registry images |
| 148 | +- runs a "smoke test" that pushes and scales an app |
| 149 | +- takes roughly 45 minutes |
| 150 | +
|
| 151 | +test-latest.sh |
| 152 | +^^^^^^^^^^^^^^ |
| 153 | +
|
| 154 | +- installs the latest ``deis`` and ``deisctl`` client releases |
| 155 | +- creates a 3-node Vagrant CoreOS cluster |
| 156 | +- provisions the cluster for Deis with latest release images |
| 157 | +- runs a "smoke test" that pushes and scales an app |
| 158 | +- takes roughly 30 minutes |
| 159 | +
|
| 160 | +Run Specific Tests |
| 161 | +^^^^^^^^^^^^^^^^^^ |
| 162 | +
|
| 163 | +Run the tests for a single component this way: |
| 164 | +
|
| 165 | +.. code-block:: console |
| 166 | +
|
| 167 | + $ make -C logger test # unit + functional |
| 168 | + $ make -C controller test-unit |
| 169 | + $ make -C router test-functional |
| 170 | +
|
| 171 | +
|
| 172 | +Customize Test Runs |
| 173 | +------------------- |
| 174 | +
|
| 175 | +The file ``tests/bin/test-setup.sh`` is the best reference to environment |
| 176 | +variables that can affect the tests' behavior. Here are some important ones: |
| 177 | +
|
| 178 | +- ``HOST_IPADDR`` - address on which Docker containers can communicate for the |
| 179 | + functional tests, probably the host's IP or the one assigned to boot2docker_. |
| 180 | +- ``DEIS_TEST_APP`` - name of the `Deis example app`_ to use, which is cloned |
| 181 | + from GitHub (default: ``example-go``) |
| 182 | +- ``DEIS_TEST_AUTH_KEY`` - SSH key used to register with the Deis controller |
| 183 | + (default: ``~/.ssh/deis``) |
| 184 | +- ``DEIS_TEST_SSH_KEY`` - SSH key used to login to the controller machine |
| 185 | + (default: ``~/.vagrant.d/insecure_private_key``) |
| 186 | +- ``DEIS_TEST_DOMAIN`` - the domain to use for testing |
| 187 | + (default: ``local3.deisapp.com``) |
| 188 | +
|
| 189 | +
|
| 190 | +.. _`unit tests`: http://en.wikipedia.org/wiki/Unit_testing |
| 191 | +.. _`functional tests`: http://en.wikipedia.org/wiki/Functional_testing |
| 192 | +.. _`integration tests`: http://en.wikipedia.org/wiki/Integration_testing |
| 193 | +.. _`continuous integration`: http://en.wikipedia.org/wiki/Continuous_integration |
| 194 | +.. _boot2docker: http://boot2docker.io/ |
| 195 | +.. _`source code`: https://github.com/deis/deis |
| 196 | +.. _`Docker registry`: https://github.com/docker/docker-registry |
| 197 | +.. _`Deis example app`: https://github.com/deis?query=example- |
0 commit comments