docs(scheduler): sort sections and update roadmap

mboersma · mboersma · commit 01c4059ea0d3 · 2015-07-29T15:43:18.000-06:00
diff --git a/docs/customizing_deis/choosing-a-scheduler.rst b/docs/customizing_deis/choosing-a-scheduler.rst
@@ -11,14 +11,14 @@ The :ref:`scheduler` creates, starts, stops, and destroys each :ref:`container`
 of your app. For example, a command such as ``deis scale web=3`` tells the
 scheduler to run three containers from the Docker image for your app.
 
-Deis defaults to using the `Fleet Scheduler`_. The Tech previews of the `Swarm Scheduler`_,
-`Mesos`_ and `Kubernetes`_ are available for testing. Keep watching `dynamic metadata fleet PR 1077`_
-for support of runtime changes to metadata.
+Deis defaults to using the :ref:`fleet_scheduler`. Tech previews of schedulers based on
+:ref:`Kubernetes <k8s_scheduler>`, :ref:`Mesos with Marathon <mesos_scheduler>`, and
+:ref:`Swarm <swarm_scheduler>` are available for testing.
 
 .. note::
 
-    If you are using a scheduler other than fleet app containers will not be rescheduled if deis-registry is unavailable.
-    For more information look at `deis-registry issue 3619`.
+    If you are using a scheduler other than fleet, app containers will not be rescheduled if
+    deis-registry is unavailable. For more information, see `deis-registry issue 3619`_.
 
 Settings set by scheduler
 -------------------------
@@ -49,6 +49,7 @@ setting                                   description
                                           "mesos_marathon" or "k8s" (default: "fleet")
 ====================================      ===============================================
 
+.. _fleet_scheduler:
 
 Fleet Scheduler
 ---------------
@@ -71,74 +72,27 @@ To use the Fleet Scheduler backend explicitly, set the controller's
 
     $ deisctl config controller set schedulerModule=fleet
 
-
-Swarm Scheduler
----------------
-
-.. important::
-
-    The Swarm Scheduler is a technology preview and is not recommended for
-    production use.
-
-`swarm`_ is a scheduling backend for Docker:
-
-    Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts
-    into a single, virtual host.
-
-..
-
-    Swarm serves the standard Docker API, so any tool which already communicates
-    with a Docker daemon can use Swarm to transparently scale to multiple hosts...
-
-Deis includes an enhanced version of swarm v0.2.0 with node failover and optimized
-locking on container creation. The Swarm Scheduler uses a `soft affinity`_ filter
-to spread app containers out among available machines.
-
-Swarm requires the Docker Remote API to be available at TCP port 2375. If you are
-upgrading an earlier installation of Deis, please refer to the CoreOS documentation
-to `enable the remote API`_.
-
-.. note::
-
-    **Known Issues**
-
-    - It is not yet possible to change the default affinity filter.
-
-To test the Swarm Scheduler backend, first install and start the swarm components:
-
-.. code-block:: console
-
-    $ deisctl install swarm && deisctl start swarm
-
-Then set the controller's ``schedulerModule`` to "swarm":
-
-.. code-block:: console
-
-    $ deisctl config controller set schedulerModule=swarm
-
-The Swarm Scheduler is now active. Commands such as ``deis destroy`` or
-``deis scale web=9`` will use `swarm`_ to manage app containers.
-
-To monitor Swarm Scheduler operations, watch the logs of the swarm-manager
-component, or spy on Docker events directly on the swarm-manager machine:
+.. _k8s_scheduler:
 
 Kubernetes Scheduler
 --------------------
 
 .. important::
 
-    The Kubernetes Scheduler is a technology preview and is not recommended for
-    production use.
+    The Kubernetes Scheduler is a technology preview and is not recommended for production use.
+    Since it requires overlay networking, Kubernetes can only be enabled on a new cluster.
 
-`Kubernetes`_ orchestration system for docker containers:
+`Kubernetes`_ is an orchestration system for Docker containers:
 
-    Kubernetes provides APIs to manage, deploy and scale docker containers. Kubernetes deploys docker containers as `pods`_,
-    unique entity across a cluster. Containers inside a pod share the same namespaces. More information about `pods`_.
+    Kubernetes (k8s) provides APIs to manage, deploy and scale Docker containers. Kubernetes
+    deploys containers as `pods`_, providing a unique entity across a cluster, but allowing
+    containers within the pod to share a namespace.
 
-Kubernetes requires overlay network so that each pod can get a unique IP across the cluster which is achieved by using `flannel`_.
-Deis repository provides a new user-data file to bootstrap CoreOs machines on the cluster with flannel.
+Kubernetes requires the `flannel`_ overlay network so each pod receives a unique IP address within
+the cluster. Existing Deis clusters cannot simply turn on overlay networking. Instead, provision
+a new cluster to enable `flannel`_.
 
-To test the Kubernetes Scheduler, first install and start the kubernetes components:
+To test the Kubernetes Scheduler, first install and start the Kubernetes components:
 
 .. code-block:: console
 
@@ -150,23 +104,26 @@ Then set the controller's ``schedulerModule`` to "k8s":
 
     $ deisctl config controller set schedulerModule=k8s
 
-The Kubernetes scheduler is now active. Commands such as ``deis destroy`` or
-``deis scale web=9`` will use kubernetes ApiServer to manage app pods.
+The Kubernetes scheduler is now active. Commands such as ``deis destroy`` or ``deis scale web=9``
+will use the Kubernetes ApiServer to manage app pods.
 
-Deis creates a `replication controller`_ to manage pods and a `service`_ which acts as a proxy and routes traffic to the pods associated with the App.
-A new release uses rolling deploy mechanism where in we create and replace the pod with the new release one by one until all the pods are replaced properly or rollback to the older release
-if any error occurs which differs from the existing schedulers where we create all the containers at a time and replace them with the old ones.
+Deis creates a `replication controller`_ to manage pods and a `service`_ which proxies traffic to
+the pods for your app. Unlike other Deis schedulers, new app releases on Kubernetes do a rolling
+deploy: pods with the new release replace old pods one at a time, until all are replaced (or until
+an error forces a rollback to the previous release).
 
 .. note::
 
     **Known Issues**
 
-    - Kubernetes ApiServer is not HA. If ApiServer is rescheduled it will reschedule all the kubernetes units.
-    - Kubernetes does resource based scheduling. Specifying limits will create reservation of the specified resource on the node.
-    - Installing flannel is not backwards compatible as it changes Docker bridge which requires restarting
-      Docker Engine.
-    - Docker Engine is started with arguments ``--iptables=false --ip-masq=false`` for the docker not to create IP masquerading rule and use the one created by flannel.
+    - The flannel overlay network is not backward-compatible with earlier Deis clusters, since it
+      changes Docker networking and requires new units from Deis' user-data file.
+    - The Kubernetes ApiServer is not HA. If the ApiServer is rescheduled, it will reschedule all
+      Kubernetes units.
+    - Kubernetes implements resource-based scheduling. Specifying limits will create a reservation
+      of that resource on the node.
 
+.. _mesos_scheduler:
 
 Mesos with Marathon framework
 -----------------------------
@@ -188,7 +145,7 @@ Mesos with Marathon framework
     service discovery, load balancing and REST API to control your Apps.
 
 Deis uses the Marathon framework to schedule containers. Since Marathon is a framework for long-running
-jobs, Deis uses the `Fleet Scheduler`_ to run batch processing jobs. ``deisctl`` installs a standalone Mesos
+jobs, Deis uses the :ref:`fleet_scheduler` to run batch processing jobs. ``deisctl`` installs a standalone Mesos
 cluster. To install an HA Mesos cluster, follow the directions at `aledbf-mesos`_, and set the etcd key
 ``/deis/scheduler/mesos/marathon`` to any Marathon node IP address. If a request is received by a regular
 Marathon node, it is proxied to the master Marathon node.
@@ -221,6 +178,58 @@ Deis starts Marathon on port 8180. You can manage apps through the Marathon UI,
     - Deis does not yet use Marathon's docker container API to create containers.
     - CPU shares are integers representing the number of CPUs. Memory limits should be specified in MB.
 
+.. _swarm_scheduler:
+
+Swarm Scheduler
+---------------
+
+.. important::
+
+    The Swarm Scheduler is a technology preview and is not recommended for
+    production use.
+
+`swarm`_ is a scheduling backend for Docker:
+
+    Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts
+    into a single, virtual host.
+
+..
+
+    Swarm serves the standard Docker API, so any tool which already communicates
+    with a Docker daemon can use Swarm to transparently scale to multiple hosts...
+
+Deis includes an enhanced version of swarm v0.2.0 with node failover and optimized
+locking on container creation. The Swarm Scheduler uses a `soft affinity`_ filter
+to spread app containers out among available machines.
+
+Swarm requires the Docker Remote API to be available at TCP port 2375. If you are
+upgrading an earlier installation of Deis, please refer to the CoreOS documentation
+to `enable the remote API`_.
+
+.. note::
+
+    **Known Issues**
+
+    - It is not yet possible to change the default affinity filter.
+
+To test the Swarm Scheduler backend, first install and start the swarm components:
+
+.. code-block:: console
+
+    $ deisctl install swarm && deisctl start swarm
+
+Then set the controller's ``schedulerModule`` to "swarm":
+
+.. code-block:: console
+
+    $ deisctl config controller set schedulerModule=swarm
+
+The Swarm Scheduler is now active. Commands such as ``deis destroy`` or
+``deis scale web=9`` will use `swarm`_ to manage app containers.
+
+To monitor Swarm Scheduler operations, watch the logs of the swarm-manager
+component, or spy on Docker events directly on the swarm-manager machine:
+
 
 .. _Kubernetes: http://kubernetes.io/
 .. _Mesos: http://mesos.apache.org
diff --git a/docs/roadmap/roadmap.rst b/docs/roadmap/roadmap.rst
@@ -24,27 +24,29 @@ through the routing mesh.
 
 Scheduling and Orchestration
 ----------------------------
-Today Deis uses `Fleet`_ for scheduling.  Unfortunately, Fleet does not support
-resource-based scheduling, which results in poor cluster utilization at scale.
+Today Deis uses :ref:`Fleet <fleet_scheduler>` for scheduling.  Unfortunately, Fleet does not
+support resource-based scheduling, which results in poor cluster utilization at scale.
 
 Fortunately, Deis is composable and can easily hot-swap orchestration APIs.
 Because the most promising container orchestration solutions are under heavy development,
 the Deis project is focused on releasing "technology previews".
 
-These technology previews will help the community try different orchestration solutions easily,
-report their findings and help guide the future direction of Deis.
+These technology previews help the community try different orchestration solutions easily,
+report their findings and guide the future direction of Deis. :ref:`Kubernetes <k8s_scheduler>`,
+:ref:`Mesos with Marathon <mesos_scheduler>`, and :ref:`Swarm <swarm_scheduler>` will all be
+available for testing in Deis v1.9.
 
- - [X] Swarm preview
- - [ ] `Mesos preview`_
- - [ ] `Kubernetes preview`_
+ - [X] :ref:`Swarm preview <swarm_scheduler>`
+ - [X] :ref:`Mesos preview <mesos_scheduler>`
+ - [X] :ref:`Kubernetes preview <k8s_scheduler>`
 
 Etcd 2
 ------
 A CP database like etcd is central to Deis, which requires a distributed lock service and key/value store.
 As problems with etcd directly impact platform stability, Deis must move to the more stable etcd2.
 
- - [ ] Switch to etcd2
- - [ ] Migration strategy for etcd 0.4.x -> etcd2
+ - [X] Switch to etcd2
+ - [X] Migration strategy for etcd 0.4.x -> etcd2
 
 This feature is tracked as GitHub issue `#3564`_.
 
@@ -106,7 +108,7 @@ Networking v2
 To provide a better container networking experience, Deis must provide an overlay network
 that can facilitate SDN and improved service discovery.
 
- - [ ] Overlay Network
+ - [X] Overlay Network (flannel)
  - [ ] `Internal Service Discovery`_
  - [ ] Migration Strategy
 
@@ -135,14 +137,11 @@ This feature is tracked as GitHub issue `#231`_.
 .. _`CoreUpdate`: https://coreos.com/docs/coreupdate/custom-apps/coreupdate-protocol/
 .. _`Deis Push`: https://github.com/deis/deis/issues/2680
 .. _`Docker Registry v2`: https://github.com/deis/deis/issues/3814
-.. _`Fleet`: https://github.com/coreos/fleet#readme
 .. _`Google Omaha Protocol`: https://code.google.com/p/omaha/wiki/ServerProtocol
 .. _`Interactive Deis Run`: https://github.com/deis/deis/issues/117
 .. _`Internal Service Discovery`: https://github.com/deis/deis/issues/3072
-.. _`Kubernetes preview`: https://github.com/deis/deis/issues/3850
 .. _`like CoreOS`: https://coreos.com/releases/
 .. _`Log Tailing`: https://github.com/deis/deis/issues/465
-.. _`Mesos preview`: https://github.com/deis/deis/issues/3809
 .. _`Service Broker API`: http://docs.cloudfoundry.org/services/api.html
 .. _`TTY Broker component`: https://github.com/deis/deis/issues/3808
 .. _`Update client/agent`: https://github.com/deis/deis/issues/3811
