Building a Memory Manager Kernel Module for CSE330 Project 2: Working Set Size and Swapping Explained

Understanding Memory Management in Linux Kernel Modules

Memory management is a critical component of operating systems, and in CSE330 Project 2, you will build a kernel module that monitors memory usage of processes. Specifically, you need to track the Resident Set Size (RSS) and Working Set Size (WSS). This tutorial will guide you through the concepts and implementation without giving away the full solution, so you can learn the underlying principles and complete your assignment successfully.

What Are RSS and WSS?

RSS (Resident Set Size) represents the amount of physical memory currently allocated to a process. It includes code, data, and stack pages that are present in RAM. WSS (Working Set Size) is the subset of virtual pages that are actively used by the process. Tracking WSS helps the OS decide which pages to swap out under memory pressure.

In your project, you will implement a kernel module that reads process memory information from /proc filesystem and computes these metrics. The test cases require that for test case 2, WSS outputs should be 300 MB, 200 MB, and 100 MB, while RSS remains constant at 300 MB. For test case 3 (bonus), you need to detect swapping activity.

Setting Up the Development Environment

Before writing code, ensure you have a Linux system with kernel headers installed. Download the project repository, unzip it, and navigate to the directory. The file memory_manager.c contains TODO markers where you need to insert your implementation. Do not modify any other files.

To compile, use the provided Makefile. Run make to build the kernel module. You may need to run sudo for loading the module.

Step 1: Understanding the Test Script

The test.sh script runs test cases by launching processes with specific memory access patterns. For example, test case 2 creates processes that allocate memory and then access pages in a way that changes the working set. Your module must correctly report the WSS for each process at the moment the test queries it.

Study the test script to understand when and how your module is invoked. The script likely uses procfs or sysfs to communicate with your module. You will need to implement a file in /proc that returns RSS and WSS values.

Implementing the Kernel Module

Your kernel module should:

• Register a proc file entry (e.g., /proc/memory_manager).
• When read, it should output RSS and WSS for a given PID or for all processes.
• Compute WSS by iterating over the process's page table and counting accessed pages (using the accessed bit in page table entries).

To get the process's memory info, use struct task_struct and struct mm_struct. Access the page table via walk_page_range or manually traverse page table entries.

Here is a skeleton of the read function:

static ssize_t proc_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) {    // Get current process or iterate over all processes    // For each process, compute RSS (mm->total_vm - mm->hiwater_vm?)    // Compute WSS by checking accessed bits    // Format output and copy to user    return len;}

Remember to use proper locking (e.g., rcu_read_lock) when accessing process lists.

Computing WSS with Accessed Bits

The Linux kernel tracks whether a page has been accessed via the _PAGE_ACCESSED flag in the page table entry. To compute the working set, you can scan the process's virtual memory areas (VMAs) and for each page, check if the accessed bit is set. Then clear the bit (using ptep_test_and_clear_young) to prepare for the next measurement.

This approach is similar to how the kernel's own working set estimation works. The test cases expect specific WSS values because the test process accesses memory in a controlled pattern.

Testing Your Module

After implementing the TODO parts, run the test script:

./test.sh 2

This should output lines with RSS and WSS values. Ensure that for the three iterations, WSS matches 300 MB, 200 MB, and 100 MB respectively, while RSS stays at 300 MB. If not, debug your WSS calculation.

For the bonus test case 3, run with sudo:

sudo ./test.sh 3

Here, you need to track swap usage. In iteration 1, swap should be 0; in iteration 3, swap should be greater than 0. This requires monitoring /proc/meminfo or using vmstat from within your module. Alternatively, you can read swap usage from /proc/<pid>/status fields like VmSwap.

Common Pitfalls and Tips

One common mistake is not properly clearing the accessed bits, leading to overestimation of WSS. Also, ensure that your module correctly handles multiple PIDs. Use kernel helper functions like for_each_process to iterate processes.

Another tip: test your module with simple programs before running the full test script. Write a small C program that allocates memory and accesses pages, then read your proc file to see if RSS and WSS match expectations.

Relating to Real-World Memory Pressure Scenarios

Memory management is crucial in modern systems, especially with the rise of AI and large language models that require huge amounts of RAM. For example, when running a model like GPT-4 on a local server, the OS must efficiently manage memory between processes. The working set concept helps decide which pages to keep in RAM for optimal performance. Similarly, in cloud computing, hypervisors use similar techniques to overcommit memory.

In gaming, large open-world games like Cyberpunk 2077 benefit from working set tracking to reduce stuttering. The OS can preload pages that the game is likely to access next.

Conclusion

By completing this project, you will gain hands-on experience with kernel programming and memory management. Focus on understanding the page table traversal and accessed bit manipulation. Use the provided test cases to validate your implementation. Good luck!