Queue-Based Load Leveling
Also known as
- Load Leveling
- Message Queuing
Intent
Queue-Based Load Leveling aims to manage the load in a system by using a queue to level the workload between producers and consumers, ensuring that heavy loads are handled smoothly without overwhelming the system.
Explanation
Real-world example
Imagine a popular restaurant with a limited number of kitchen staff (consumers) and a large number of customers placing orders (producers). During peak hours, if all customers were served immediately, the kitchen would be overwhelmed, leading to long wait times and potential mistakes in orders. To manage this, the restaurant implements a queue-based load leveling system using a ticketing machine.
When customers place orders, they receive a ticket number and their order is placed in a queue. The kitchen staff then processes orders one at a time in the order they were received. This ensures that the kitchen can handle the workload at a manageable pace, preventing overload and maintaining service quality. Customers wait comfortably knowing their order is in line and will be handled efficiently, even during the busiest times.
In plain words
Queue-Based Load Leveling is a design pattern that uses a queue to manage and balance the workload between producers and consumers, preventing system overload and ensuring smooth processing.
Wikipedia says
Message Queues are essential components for inter-process communication (IPC) and inter-thread communication, using queues to manage the passing of messages. They help in decoupling producers and consumers, allowing asynchronous processing, which is a key aspect of the Queue-Based Load Leveling pattern.
Programmatic Example
The Queue-Based Load Leveling pattern helps to manage high-volume, sporadic bursts of tasks that can overwhelm a system. It uses a queue as a buffer to hold tasks, decoupling the task generation from task processing. The tasks are then processed at a manageable rate.
First, let's look at the MessageQueue and Message classes. The MessageQueue acts as a buffer, storing messages until they are retrieved by the ServiceExecutor. The Message represents the tasks to be processed.
public class Message {
// Message details
}
public class MessageQueue {
private Queue<Message> queue;
public MessageQueue() {
queue = new LinkedList<>();
}
// Method to add a message to the queue
public void addMessage(Message message) {
queue.add(message);
}
// Method to retrieve a message from the queue
public Message getMessage() {
return queue.poll();
}
}
Next, we have the TaskGenerator class. This class represents the task producers. It generates tasks and submits them to the MessageQueue.
public class TaskGenerator implements Runnable {
private MessageQueue msgQueue;
private int taskCount;
public TaskGenerator(MessageQueue msgQueue, int taskCount) {
this.msgQueue = msgQueue;
this.taskCount = taskCount;
}
@Override
public void run() {
for (int i = 0; i < taskCount; i++) {
Message message = new Message(); // Create a new message
msgQueue.addMessage(message); // Add the message to the queue
}
}
}
The ServiceExecutor class represents the task consumer. It retrieves tasks from the MessageQueue and processes them.
public class ServiceExecutor implements Runnable {
private MessageQueue msgQueue;
public ServiceExecutor(MessageQueue msgQueue) {
this.msgQueue = msgQueue;
}
@Override
public void run() {
while (true) {
Message message = msgQueue.getMessage(); // Retrieve a message from the queue
if (message != null) {
// Process the message
} else {
// No more messages to process
break;
}
}
}
}
Finally, we have the App class which sets up the TaskGenerator and ServiceExecutor threads and submits them to an ExecutorService.
public class App {
public static void main(String[] args) {
var msgQueue = new MessageQueue();
final var taskRunnable1 = new TaskGenerator(msgQueue, 5);
final var taskRunnable2 = new TaskGenerator(msgQueue, 1);
final var taskRunnable3 = new TaskGenerator(msgQueue, 2);
final var srvRunnable = new ServiceExecutor(msgQueue);
ExecutorService executor = Executors.newFixedThreadPool(2);
executor.submit(taskRunnable1);
executor.submit(taskRunnable2);
executor.submit(taskRunnable3);
executor.submit(srvRunnable);
executor.shutdown();
}
}
In this example, the TaskGenerator threads generate tasks at a variable rate and submit them to the MessageQueue. The ServiceExecutor retrieves the tasks from the queue and processes them at its own pace, preventing the system from being overwhelmed by peak loads.
Running the application produces the following console output:
[main] INFO App - Submitting TaskGenerators and ServiceExecutor threads.
[main] INFO App - Initiating shutdown. Executor will shutdown only after all the Threads are completed.
[pool-1-thread-2] INFO TaskGenerator - Message-1 submitted by pool-1-thread-2
[pool-1-thread-1] INFO TaskGenerator - Message-5 submitted by pool-1-thread-1
[pool-1-thread-1] INFO TaskGenerator - Message-4 submitted by pool-1-thread-1
[pool-1-thread-2] INFO TaskGenerator - Message-2 submitted by pool-1-thread-2
[pool-1-thread-1] INFO TaskGenerator - Message-3 submitted by pool-1-thread-1
[pool-1-thread-2] INFO TaskGenerator - Message-1 submitted by pool-1-thread-2
[pool-1-thread-1] INFO TaskGenerator - Message-2 submitted by pool-1-thread-1
[pool-1-thread-2] INFO ServiceExecutor - Message-1 submitted by pool-1-thread-2 is served.
[pool-1-thread-1] INFO TaskGenerator - Message-1 submitted by pool-1-thread-1
[pool-1-thread-2] INFO ServiceExecutor - Message-5 submitted by pool-1-thread-1 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-4 submitted by pool-1-thread-1 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-2 submitted by pool-1-thread-2 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-3 submitted by pool-1-thread-1 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-1 submitted by pool-1-thread-2 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-2 submitted by pool-1-thread-1 is served.
[pool-1-thread-2] INFO ServiceExecutor - Message-1 submitted by pool-1-thread-1 is served.
[pool-1-thread-2] INFO ServiceExecutor - Service Executor: Waiting for Messages to serve ..
[pool-1-thread-2] INFO ServiceExecutor - Service Executor: Waiting for Messages to serve ..
[pool-1-thread-2] INFO ServiceExecutor - Service Executor: Waiting for Messages to serve ..
[pool-1-thread-2] INFO ServiceExecutor - Service Executor: Waiting for Messages to serve ..
[main] INFO App - Executor was shut down and Exiting.
[pool-1-thread-2] ERROR ServiceExecutor - sleep interrupted
Applicability
- When there are variable workloads, and you need to ensure that peak loads do not overwhelm the system
- In distributed systems where tasks are produced at a different rate than they are consumed
- For decoupling producers and consumers in an asynchronous messaging system
Known Uses
- Amazon Web Services (AWS) Simple Queue Service (SQS)
- RabbitMQ
- Java Message Service (JMS) in enterprise Java applications
Consequences
Benefits:
- Decouples the producers and consumers, allowing each to operate at its own pace
- Increases system resilience and fault tolerance by preventing overload conditions
- Enhances scalability by allowing more consumers to be added to handle increased load
Trade-offs:
- Adds complexity to the system architecture
- May introduce latency as messages need to be queued and dequeued
- Requires additional components (queues) to be managed and monitored
Related Patterns
- Asynchronous Messaging: Queue-Based Load Leveling uses asynchronous messaging to decouple producers and consumers
- Circuit Breaker: Often used in conjunction with Queue-Based Load Leveling to prevent system overloads by temporarily halting message processing
- Producer-Consumer: Queue-Based Load Leveling is a specific application of the Producer-Consumer pattern where the queue serves as the intermediary
- Retry: Works with Queue-Based Load Leveling to handle transient failures by retrying failed operations