CrowdClustering

Background

Atlassian does not officially support clustering with Crowd and probably won't for a long time. I have voted for the feature so time will tell. Some users are running crowd in a cluster which may or may not work for this application.

External Documents

• Clustering is not Supported - Official page with some hints for unsupported clustering
• Main threads concerning clustering crowd
• Token Memory Caching must be disabled
• Tomcat Load Balancer - Howto

Load Balancing

A load balancer is required in front of the Crowd nodes to provide a cohesive interface to the cluster. This also ensures the load is spread out amongst all of the Crowd nodes.

Single point of failure? How can we assure HA on the load balancer itself?

Tomcat

Tomcat provides a load balancing solution but that may not scale well enough for such a large system.

Hardware Balanacer

A Cisco ACE is probably overkill and can be difficult to configure

MySQL Replication/Clustering

There seems to be one main solution that works for people with a few options along the way. Multiple identical instances of Crowd are installed on a number of boxes behind a load balancer. Each instance needs access to the same data for tokens to be accepted across the cluster. If using a replicated set the replication must be quick enough (at least on the token table) to ensure that a user who authenticates against one node has their token replicated to the other nodes before they attempt to use another crowd protected service.

MySQL Administration Tips

Use this cron job to monitor database utilization over time:

• */5 * * * * echo `date`:\ `mysqladmin status` >> /var/log/mysql-status

Shared Database

It is possible for the Crowd nodes to share a single database so the tokens table is always up to date. It would be a good idea to have some provision for failover like a master/master database setup.

How many clients can a single db server handle?

Pros

• Simple to set up
• Low admin overhead

Cons

• Likely performance bottleneck
• Won't scale well

Replication

A single master database asynchronously writes changes to a number of slave databases. All writes occur on the master database, all reads occur on the slave databases.

Pros

• Scales well

Cons

• Complex Administration
• Replication must occur before an authorized client is allow access to a resource

MySQL Clustering

• Overview
• 4 Node Example Setup
  • Servers should be identical
  • Cluster should run on separate subnet
  • Gigabit eth recommended

Pros

• Scales well
• Designed for no single point of failure
• Internally handles node failures
• No replication delays

Cons

• Complex Administration

-- ChristopherPepe - 07 Apr 2009