Malware Analysis Phase I & II: A Practical Guide for CS6035 Students

Understanding Malware Analysis in CS6035

Malware analysis is a critical skill in cybersecurity, and the CS6035 project at Georgia Tech challenges you to apply both static and dynamic techniques. Whether you're analyzing suspicious files for Phase I or extracting hidden flags for Phase II, this guide will help you navigate the process. We'll use the provided Joe Sandbox reports and VM tools to identify behaviors and decode obfuscated payloads.

Phase I: Behavior Identification with Joe Sandbox

In Phase I, you must determine which of five malware samples exhibit each of 20 behaviors. Each behavior is worth 2.5 points (0.5 per sample). The key is to read the Behaviors and Signatures sections of the Joe Sandbox report carefully. For example, if a report shows the malware called CreateFile in a system directory, that indicates a file drop attempt. Remember: attempted counts even if it fails.

Think of this like a fantasy football draft: each behavior is a player stat, and you need to check every sample's report for evidence. Don't rely on reputation—focus on observed actions. Use the VM's pre-installed tools like Wireshark or tcpdump for network behaviors, but the reports usually suffice.

Phase II: Static and Dynamic Analysis for Flags

Phase II requires you to decode payloads and extract flags from the same five samples. You'll use CyberChef, Python scripts, and command-line tools. The FAQ warns about multiple encoding layers—like an onion—so automate with a loop if needed. For network traffic, capture on the loopback interface and follow TCP streams.

A common pitfall is formatting: your GTID must be exactly 9 digits without angle brackets. If you get "No Flag here. Incorrect Phrase," double-check your decoding steps. CyberChef's "Magic" recipe can help, but manual decoding is often necessary.

Step-by-Step Workflow for Phase I

1. Download the VM Early

The VM is over 9 GB, so start downloading immediately. Use the provided credentials: username malware, password MachuPicchu_1450. You have limited sudo access for Wireshark and tcpdump.

2. Access the Joe Sandbox Reports

Each sample has a report in the assignment portal. Focus on the Behavior Summary and Signatures. For example, if a sample attempts to modify the Windows registry, look for RegSetValue calls. Mark the behavior as true if there's evidence.

3. Format Your JSON Submission

Follow this example template:

{
  "behavior01": {
    "malware1": true,
    "malware2": false,
    "malware3": true,
    "malware4": false,
    "malware5": true
  },
  "behavior02": { ... },
  ...
}

Label behaviors as behavior01 through behavior20. No comments or trailing commas. You have 5 attempts, so use them wisely.

Step-by-Step Workflow for Phase II

1. Set Up the VM for Dynamic Analysis

Ensure the VM is configured per instructions. You'll run the malware samples in an isolated environment. Use Wireshark on loopback to capture traffic from the malware's C2 communications.

2. Decode Payloads with CyberChef

Many samples use base64, hex, or XOR encoding. Try CyberChef's From Base64 and XOR Brute Force. If you hit a dead end, write a Python script to automate layers. For instance, a sample might do base64 then gzip then XOR with key 0x42.

3. Extract Flags via HTTP Requests

Some flags require sending a GET request with your GTID. Use curl in the VM:

curl "http://example.com/flag?gtid=123456789"

Ensure no extra characters. If you get 404, double-check the endpoint.

Common Mistakes and How to Avoid Them

• Incomplete JSON: Provide answers for all 20 behaviors and 5 samples. Missing entries cause rejection.
• Misinterpreting "drop": Dropping a file means creating or moving it into a directory, not deleting.
• Skipping decoding layers: Each layer may use a different encoding. Keep peeling until you get a readable flag.
• Ignoring network traffic: Some behaviors (like C2 communication) are only visible in Wireshark.

Trend Connection: Malware Analysis Like AI Training

Just as AI models are trained on labeled data to recognize patterns, you are training your analytical eye to spot malicious behaviors. The Joe Sandbox reports are your training dataset. In 2026, AI-driven malware is on the rise, making static and dynamic analysis even more crucial. Think of this project as building your own threat detection model—one behavior at a time.

Final Tips for Success

• Start early—the VM download and analysis take time.
• Re-read the FAQ; it answers 90% of common issues.
• Use Python to automate repetitive decoding tasks.
• For Phase I, mark false if you're unsure—guessing can lower your score.
• For Phase II, verify your GTID format before submitting.

By following this guide, you'll be well-prepared to tackle both phases. Remember: the goal is to learn, not just to get points. Happy analyzing!