Segovia:

Table 1-2 and Table 1-3 include example configuration and considerations for both third-party and OpenStack components:

Table 1-2. Third-party component configuration

Component Tuning Availability Scalability

MySQL binlog-format = row

Master/master replication. However, both nodes are not used at the same time. Replication keeps all nodes as close to being up to date as possible (although the asynchronous nature of the replication means a fully consistent state is not possible). Connections to the database only happen through a Pacemaker virtual IP, ensuring that most problems that occur with master-master replication can be avoided.

Not heavily considered. Once load on the MySQL server increases enough that scalability needs to be considered, multiple masters or a master/slave setup can be used.

Component Tuning Availability Scalability Qpid max- connec tions=1000 worker- threads=20 connection- backlog=10, sasl security enabled with SASL-BASIC authentication

Qpid is added as a resource to the Pacemaker software that runs on Controller nodes where Qpid is situated. This ensures only one Qpid instance is running at one time, and the node with the Pacemaker virtual IP will always be the node running Qpid.

Not heavily considered. However, Qpid can be changed to run on all controller nodes for scalability and availability purposes, and removed from Pacemaker.

HAProxy maxconn 3000 HAProxy is a software layer-7 load balancer used to front door all clustered OpenStack API components and do SSL termination. HAProxy can be added as a resource to the Pacemaker software that runs on the Controller nodes where HAProxy is situated. This ensures that only one HAProxy instance is running at one time, and the node with the Pacemaker virtual IP will always be the node running HAProxy.

Not considered. HAProxy has small enough performance overheads that a single instance should scale enough for this level of workload. If extra scalability is needed, keepalived or other Layer-4 load balancing can be introduced to be placed in front of multiple copies of HAProxy. Memcached MAXCONN="8192"

CACHE SIZE="30457"

Memcached is a fast in-memory key-value cache software that is used by OpenStack components for caching data and increasing performance. Memcached runs on all controller nodes, ensuring that should one go down, another instance of Memcached is available.

Not considered. A single instance of Memcached should be able to scale to the desired workloads. If scalability is desired, HAProxy can be placed in front of Memcached (in raw tcp mode) to utilize multiple Memcached instances for scalability. However, this might cause cache consistency issues.

Component Tuning Availability Scalability

Pacemaker Configured to use corosync and cman as a cluster

communication stack/ quorum manager, and as a two-node cluster.

Pacemaker is the clustering software used to ensure the availability of services running on the controller and network nodes:

• Because Pacemaker is cluster software, the software itself handles its own availability, leveraging corosync and cman underneath. • If you use the GlusterFS native client, no virtual IP is needed, since the client knows all about nodes after initial connection and automatically routes around failures on the client side.

• If you use the NFS or SMB adaptor, you will need a virtual IP on which to mount the GlusterFS volumes.

If more nodes need to be made cluster aware, Pacemaker can scale to 64 nodes.

GlusterFS glusterfs performance profile “virt” enabled on all volumes. Volumes are setup in two-node replication.

Glusterfs is a clustered file system that is run on the storage nodes to provide persistent scalable data storage in the environment. Because all connections to gluster use the gluster native mount points, the gluster instances themselves provide availability and failover functionality.

The scalability of GlusterFS storage can be achieved by adding in more storage volumes.

Table 1-3. OpenStack component configuration

Component Node type

Tuning Availability Scalability

Dashboard (horizon)

Controller Configured to use Memcached as a session store, neutron support is enabled,

can_set_mount_point = False

The dashboard is run on all controller nodes, ensuring at least once instance will be available in case of node failure. It also sits behind HAProxy, which detects when the software fails and routes requests around the failing instance.

The dashboard is run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for the dashboard as more nodes are added. Identity

(keystone) Controller Configured to use Memcached forcaching and PKI for tokens. Identity is run on all controllernodes, ensuring at least once instance will be available in case of node failure. Identity also sits behind HAProxy, which detects when the software fails and routes requests around the failing instance.

Identity is run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for Identity as more nodes are added.

Component Node type

Tuning Availability Scalability

Image Service (glance)

Controller /var/lib/glance/images is a GlusterFS native mount to a Gluster volume off the storage layer.

The Image Service is run on all controller nodes, ensuring at least once instance will be available in case of node failure. It also sits behind HAProxy, which detects when the software fails and routes requests around the failing instance.

The Image Service is run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for the Image Service as more nodes are added. Compute

(nova) Controller,Compute Configured to use Qpid,qpid_heartbeat = 10, configured to use Memcached for caching, configured to use libvirt, configured to use neutron.

Configured nova-consoleauth to use Memcached for session management (so that it can have multiple copies and run in a load balancer).

The nova API, scheduler, objectstore, cert, consoleauth, conductor, and vncproxy services are run on all controller nodes, ensuring at least once instance will be available in case of node failure. Compute is also behind HAProxy, which detects when the software fails and routes requests around the failing instance.

Compute’s compute and conductor services, which run on the compute nodes, are only needed to run services on that node, so availability of those services is coupled tightly to the nodes that are available. As long as a compute node is up, it will have the needed services running on top of it.

The nova API, scheduler, objectstore, cert, consoleauth, conductor, and vncproxy services are run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for Compute as more nodes are added. The scalability of services running on the compute nodes (compute, conductor) is achieved linearly by adding in more compute nodes.

Block Storage (cinder)

Controller Configured to use Qpid, qpid_heartbeat = 10, configured to use a Gluster volume from the storage layer as the backend for Block Storage, using the Gluster native client.

Block Storage API, scheduler, and volume services are run on all controller nodes, ensuring at least once instance will be available in case of node failure. Block Storage also sits behind HAProxy, which detects if the software fails and routes requests around the failing instance.

Block Storage API, scheduler and volume services are run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for Block Storage as more nodes are added.

Component Node type

Tuning Availability Scalability

OpenStack Networking (neutron) Controller, Compute, Network

Configured to use QPID, qpid_heartbeat = 10, kernel namespace support enabled, ten ant_network_type = vlan, allow_overlapping_ips = true, tenant_net work_type = vlan, bridge_uplinks = br- ex:em2, bridge_mappings = physnet1:br-ex

The OpenStack Networking service is run on all controller nodes, ensuring at least one instance will be available in case of node failure. It also sits behind HAProxy, which detects if the software fails and routes requests around the failing instance.

OpenStack Networking’s ovs- agent, l3-agent-dhcp- agent, and metadata- agent services run on the network nodes, as lsb resources inside of Pacemaker. This means that in the case of network node failure, services are kept running on another node. Finally, the ovs-agent service is also run on all compute nodes, and in case of compute node failure, the other nodes will continue to function using the copy of the service running on them.

The OpenStack Networking server service is run on all controller nodes, so scalability can be achieved with additional controller nodes. HAProxy allows scalability for OpenStack Networking as more nodes are added. Scalability of services running on the network nodes is not currently supported by OpenStack Networking, so they are not be considered. One copy of the services should be sufficient to handle the workload. Scalability of the ovs- agent running on compute nodes is achieved by adding in more compute nodes as necessary.