Building a Producer-Consumer Kernel Module in Linux: A Step-by-Step Guide

Introduction to the Producer-Consumer Problem in Kernel Modules

The producer-consumer problem is a classic synchronization challenge in operating systems. In this tutorial, you'll implement a kernel module in C that spawns multiple processes to produce and consume items in a shared buffer. This mirrors real-world scenarios like a video streaming app where producers (encoders) generate frames and consumers (decoders) process them, or a gaming server handling player actions. By the end, you'll have a working module that passes test cases similar to those in CSE330 Project 1.

Understanding the Assignment Requirements

Your task is to complete producer_consumer.c inside a provided git repository. The module should:

• Accept command-line arguments: number of processes, buffer size, producers, consumers, and lines to display from dmesg.
• Use kernel threads or workqueues to simulate producers and consumers.
• Ensure each produced item is consumed exactly once (no duplicates, no losses).
• Match CPU time reported by ps command.
• Exit cleanly without errors.

Test cases range from simple (1 producer, 0 consumers) to complex (1000 processes, 1 producer, 1 consumer). Bonus test case requires error-free exit.

Setting Up Your Environment

First, download and unzip the repository. Ensure you have a Linux system with kernel headers installed. Run make to build the module. The template provides basic structure; your job is to implement the producer-consumer logic.

Key Kernel APIs You'll Use

• kthread_run – create kernel threads
• wait_queue_head_t – for blocking threads when buffer full/empty
• spinlock_t or mutex – protect shared buffer
• sched_setscheduler – to set CPU affinity if needed

Designing the Producer-Consumer Logic

Think of the buffer as a queue in a fast-food app: orders (items) come in, are placed in a queue, and are processed by kitchen staff. If the queue is full, new orders wait; if empty, staff idle. Your kernel module must handle this with proper synchronization to avoid race conditions.

Buffer Structure

struct buffer {
    int *items;
    int size;
    int in, out;
    spinlock_t lock;
    wait_queue_head_t not_full, not_empty;
};

Producer Thread

int producer(void *arg) {
    struct buffer *buf = (struct buffer *)arg;
    int item = 0;
    while (item < total_items) {
        wait_event_interruptible(buf->not_full, buf->in - buf->out < buf->size);
        spin_lock(&buf->lock);
        // add item to buffer
        buf->items[buf->in % buf->size] = item;
        buf->in++;
        spin_unlock(&buf->lock);
        wake_up(&buf->not_empty);
        item++;
    }
    return 0;
}

Consumer Thread

int consumer(void *arg) {
    struct buffer *buf = (struct buffer *)arg;
    int item;
    while (consumed < total_items) {
        wait_event_interruptible(buf->not_empty, buf->in > buf->out);
        spin_lock(&buf->lock);
        item = buf->items[buf->out % buf->size];
        buf->out++;
        spin_unlock(&buf->lock);
        wake_up(&buf->not_full);
        consumed++;
    }
    return 0;
}

Handling Multiple Producers and Consumers

With multiple producers, each should produce a unique set of items. Use a global atomic counter to assign items. Similarly, consumers must ensure each item is consumed once. The test scripts validate this by counting total produced and consumed items.

Matching CPU Time with ps

To pass the test, your module must report CPU time consistent with ps. Use get_cpu() and put_cpu() or read current->utime and current->stime. You may need to accumulate time in a per-process structure.

Testing Your Module

Run the test script: sudo ./test.sh 10 5 1 0 25. This spawns 10 processes, buffer size 5, 1 producer, 0 consumers. Expected: producer fills buffer (5 items), module exits without error. Check dmesg for output.

Common Pitfalls

• Deadlock: Ensure spinlocks are released before waiting.
• Race condition: Use proper memory barriers (spin_lock ensures ordering).
• Missing wake-ups: Always wake the other side after producing/consuming.

Real-World Trend: AI Chatbot Load Balancing

This kernel module mirrors how AI chatbots like ChatGPT handle requests: incoming messages (producers) are queued, and model inference engines (consumers) process them. Buffer size controls how many requests wait in memory. Understanding this synchronization helps you design scalable systems.

Final Checklist

1. Implement producer and consumer threads.
2. Use wait queues for blocking.
3. Protect buffer with spinlock.
4. Track total items produced/consumed.
5. Ensure module cleanup on exit.
6. Test with all provided test cases.

Conclusion

By completing this kernel module, you've learned essential OS concepts: synchronization, threading, and kernel programming. These skills are directly applicable to backend development for high-traffic apps, game server architecture, and real-time data processing. Good luck with your CSE330 project!