Skip to content

running-deis-without-ceph.rst

Latest commit

 

History

History
222 lines (165 loc) · 7.07 KB

running-deis-without-ceph.rst

File metadata and controls

222 lines (165 loc) · 7.07 KB
title:Running Deis without Ceph
description:Configuring the cluster to remove Ceph from the control plane.

Running Deis without Ceph

This guide is intended to assist users who are interested in removing the Ceph dependency of the Deis control plane.

Note

This guide was adapted from content graciously provided by Deis community member Arne-Christian Blystad.

Requirements

External services are required to replace the internal store components:

  • S3-compatible blob store (like Amazon S3)
  • PostgreSQL database (like Amazon RDS)
  • Log drain service with syslog log format compatibility (like Papertrail)

Understanding component changes

Either directly or indirectly, all components in the :ref:`control-plane` require Ceph (:ref:`store`). Some components require changes to accommodate the removal of Ceph. The necessary changes are described below.

Logger

The :ref:`logspout` component attaches to Docker containers on each host and listens for log events from platform components and running applications. It ships these to the :ref:`logger` component. By default, the logger writes the logs to a distributed Ceph filesystem. These logs can then be fetched by the :ref:`controller` component via HTTP.

In a Ceph-less clutser, the Logger component should be configured, instead, to use in-memory log storage. Optionally, a drain may also be configured to forward logs to an external log service (such as Papertrail) for longer-term archival.

Database

The :ref:`database` runs PostgreSQL and uses the Ceph S3 API (provided by deis-store-gateway) to store PostgreSQL backups and WAL logs. Should the host running database fail, the database component will fail over to a new host, start up, and replay backups and WAL logs to recover to its previous state.

We will not be using the database component in the Ceph-less cluster, and will instead rely on an external database.

Registry

The :ref:`registry` component is an instance of the offical Docker registry, and is used to store application releases. The registry supports any S3 store, so a Ceph-less cluster will simply reconfigure registry to use another store (typically Amazon S3 itself).

Builder

The :ref:`builder` component is responsible for building applications deployed to Deis via the git push workflow. It pushes to registry to store releases, so it will require no changes.

Store

The :ref:`store` components implement Ceph itself. In a Ceph-less cluster, we will skip the installation and starting of these components.

Deploying the cluster

This guide assumes a typical deployment on AWS by following the :ref:`deis_on_aws` guide.

Deploy an AWS cluster

Follow the :ref:`deis_on_aws` installation documentation through the "Configure DNS" portion.

Configure logger

The :ref:`logger` component should be configured to use in-memory storage. Optionally it may also be configured to drain logs to an external service for longer-term archival.

$ STORAGE_ADAPTER=memory
$ DRAIN=udp://logs.somewhere.com:12345 # Supported protocols are udp and tcp; for backwards compatibility, "syslog" is an alias for udp
$ deisctl config logs set storageAdapterType=${STORAGE_ADAPTER} drain=${DRAIN}

Configure registry

The :ref:`registry` component won't start until it's configured with a store.

S3 store configuration sample:

$ BUCKET=MYS3BUCKET
$ AWS_S3_REGION=some-aws-region #(e.g., us-west-1)
$ deisctl config registry set s3bucket=${BUCKET} \
                              s3region=${AWS_S3_REGION} \
                              s3path=/ \
                              s3encrypt=false \
                              s3secure=false

By default, the registry will try to authenticate to S3 using the instance role. If your cluster is not running on EC2, you can supply hard coded API access and secret key:

$ deisctl config registry set s3accessKey=your-access-key \
                              s3secretKey=your-secret-key

For reference, here's example of a policy you could attach to the role/user used by the registry:

{
  "Statement": [
    {
      "Resource": [
        "arn:aws:s3:::MYBUCKET"
      ],
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Effect": "Allow"
    },
    {
      "Resource": [
        "arn:aws:s3:::MYBUCKET/*"
      ],
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Effect": "Allow"
    }
  ],
  "Version": "2012-10-17"
}

Openstack-swift support requires Swift3 middleware to be installed. Here is a sample configuration:

$ SWIFT_CONTAINER=mycontainer
$ SWIFT_USER=system:root
$ SWIFT_SECRET_KEY=testpass
$ deisctl config registry set bucketName=${SWIFT_CONTAINER}
$ deisctl config store set gateway/accessKey=${SWIFT_USER} \
                           gateway/secretKey=${SWIFT_SECRET_KEY} \
                           gateway/host=10.1.50.1 \
                           gateway/port=8080

Configure database settings

Since we won't be running the :ref:`database`, we need to configure these settings so the controller knows where to connect.

$ HOST=something.rds.amazonaws.com
$ DB_USER=deis
$ DB_PASS=somethingsomething
$ DATABASE=deis
$ deisctl config database set engine=postgresql_psycopg2 \
                              host=${HOST} \
                              port=5432 \
                              name=${DATABASE} \
                              user=${DB_USER} \
                              password=${DB_PASS}

Deploy the platform

The typical :ref:`install_deis_platform` documentation can be followed, with one caveat: since we won't be deploying many of the typical Deis components, we cannot use deisctl install platform or deisctl start platform -- instead, we use deisctl install stateless-platform and deisctl start stateless-platform.

These commands tell deisctl to skip the components that we don't need to use.

Confirm installation

That's it! Deis is now running without Ceph. Issue a deisctl list to confirm that the services are started, and see :ref:`using_deis` to start using the cluster.

Upgrading Deis

When following the :ref:`upgrading-deis` documentation, be sure to use stateless-platform instead of platform.