Not All Edges are Equally Robust: Evaluating the Robustness of Ranking-Based Federated Learning

Not All Edges are Equally Robust: Evaluating the Robustness of Ranking-Based Federated Learning

Speakers: Zirui Gong, PhD Student, Griffith University; Yanjun Zhang; Leo Yu Zhang; Zhaoxi Zhang; Yong Xiang; Shirui Pan

Conference: IEEE S&P

YouTube: https://www.youtube.com/watch?v=Rv2h0TWOi3E

Overview

This talk, presented by Zirui Gong from Griffith University, delves into the often-overlooked security vulnerabilities within Federated Learning (FL) frameworks, specifically focusing on a purportedly robust variant known as Fed-Ranking Learning (FRL). The research, a collaborative effort between Griffith University, University of Technology Sydney, and Deakin University, challenges the prevailing assumption of FRL's resilience against client-side attacks. It identifies a critical vulnerability that allows malicious clients to significantly impact the global model despite the framework's inherent defenses.

The core of the presentation introduces a novel attack method, Vulnerable Edge Manipulation (VEM), meticulously designed to exploit these newfound weaknesses. VEM systematically targets and manipulates specific "vulnerable edges" within the neural network's architecture, demonstrating a far greater impact than existing state-of-the-art attacks. This work is crucial for the advancement of secure decentralized machine learning, as it uncovers a fundamental flaw in a framework previously considered highly robust, thereby necessitating a re-evaluation of its security posture and inspiring the development of more resilient FL systems.

The significance of this research lies in its systematic analysis, which not only exposes a critical vulnerability but also provides a concrete, highly effective attack vector. By demonstrating that even robust FL architectures like FRL can be compromised, the paper underscores the continuous need for rigorous security evaluations in decentralized AI. The findings are vital for practitioners and researchers alike, guiding them toward a deeper understanding of FL security and the imperative to design more robust defensive mechanisms against sophisticated adversarial manipulations.

Background

▶ Watch: Introduction to federated learning security challenges (0:00)

Federated Learning (FL) has emerged as a promising decentralized approach to machine learning, enabling multiple clients to collaboratively train a shared model without centralizing their sensitive data. This paradigm effectively addresses data privacy concerns and facilitates model training on diverse, geographically distributed datasets. However, the decentralized nature of FL inherently introduces security challenges, making it susceptible to various client-side attacks, such as data poisoning or model poisoning, where malicious clients can attempt to degrade or manipulate the global model. These vulnerabilities pose a significant hindrance to FL's widespread adoption in real-world applications, particularly in sensitive domains like healthcare or finance.

In response to these security concerns, researchers have proposed more robust FL frameworks. One such framework, which is the focus of this talk, is Fed-Ranking Learning (FRL). FRL distinguishes itself from traditional FL training in two key ways, designed to enhance its robustness. Firstly, instead of exchanging full model updates or gradients, FRL clients communicate edge rankings to the central server. These edge rankings represent a permutation of integers indicating the importance of each neuron (or edge) within the client's local network. This communication format is intended to narrow down the potential space for malicious updates from an infinite range to a finite, discrete set, thereby bounding the potential damage an attacker can inflict within a defined budget.

Secondly, on the server side, FRL employs majority voting to aggregate these client-submitted edge rankings into a global ranking, rather than using traditional aggregation techniques like federated averaging. The intuition behind majority voting is that it prevents individual malicious clients from making significant modifications to the global model, as each client contributes only a single "vote," making it difficult for a small number of adversaries to sway the consensus of a larger group of benign clients. Following the aggregation, the server utilizes the Lottery Ticket Hypothesis, which posits that within a large, randomly initialized neural network, there exists a sub-network (a "winning ticket") that, when trained in isolation, can achieve performance comparable to the original full network. Based on the global ranking, FRL selects the top K edges to form this sub-network, which then serves as the global model. This mechanism further reinforces the belief in FRL's robustness, as only the most important and consensually agreed-upon edges contribute to the final model.

Prior research and experiments, as highlighted by the speaker, had suggested that existing state-of-the-art attacks achieved "very low attack impact" on FRL frameworks, even under varying malicious rates and data heterogeneity (referred to as "90 degree" in the transcript). This perceived resilience led to the question central to this work: Are FRL frameworks truly robust against all forms of client-side attacks, or do hidden vulnerabilities still exist? This talk seeks to answer that question by conducting a systematic analysis of FRL's robustness and exploring its potential weak points.

Key Findings

▶ Watch: Analysis of FedRanking's perceived robustness (3:00)

The central contribution of this research is the definitive demonstration that Fed-Ranking Learning (FRL), despite its design principles aimed at robustness, harbors a critical vulnerability. The key findings can be summarized as follows:

Firstly, this work presents the first systematic analysis of FRL's robustness, moving beyond anecdotal evidence or limited attack scenarios. Through rigorous investigation, the researchers uncovered the existence of specific "vulnerable edges" within the network architecture. These vulnerable edges are defined as those whose importance difference from the selection boundary (i.e., the threshold for being included in the top K edges) is smaller than the maximum damage a malicious client can inflict in a single round. This theoretical insight fundamentally challenges the assumption that FRL's discrete ranking format and majority voting aggregation inherently provide complete protection against adversarial manipulation.

Secondly, based on this critical discovery, the researchers designed and implemented a novel attack method called Vulnerable Edge Manipulation (VEM). VEM is specifically engineered to target and effectively manipulate these identified vulnerable edges. Unlike previous attacks that might attempt to broadly poison model updates or rankings, VEM employs a sophisticated, three-stage process to precisely identify, optimize for, and then exploit these sensitive network components.

Thirdly, extensive experiments conducted across various network architectures and datasets conclusively demonstrated that VEM significantly outperforms existing state-of-the-art attacks. The attack achieved a remarkable 53% attack impact, which is 3.7 times more impactful than other comparable attacks. This substantial performance gap underscores VEM's efficacy and the severity of the vulnerability it exploits. The experiments also included studies on the impact of different percentages of malicious clients and varying degrees of data heterogeneity, consistently showing VEM's superior performance across diverse adversarial settings.

In essence, the key finding is that FRL is not as robust as previously believed. The existence of vulnerable edges, coupled with the effectiveness of the VEM attack, reveals a significant security flaw that could allow adversaries to subtly but effectively degrade or alter the global model's behavior, thereby undermining the integrity and trustworthiness of FRL systems.

Technical Deep Dive

▶ Watch: Overview of the BAM attack framework (5:00)

The Vulnerable Edge Manipulation (VEM) attack is a sophisticated, three-stage framework designed to exploit specific weaknesses within the Fed-Ranking Learning (FRL) architecture. The underlying threat model for VEM assumes an un-targeted local model poisoning attack against the FRL framework. The attacker's objective is to degrade the overall performance or integrity of the global model without targeting specific classes or outputs. It is assumed that the adversary controls up to M out of U clients in each training iteration, designated as malicious clients. Crucially, the attack operates in an update-agnostic setting, meaning the adversary does not possess any information about the updates or rankings contributed by other benign clients.

The VEM framework comprises three distinct stages:

1. Vulnerable Edge Identification

The first critical step involves theoretically proving the existence of and then identifying vulnerable edges within the neural network. To facilitate this analysis, the researchers first reformulate the discrete edge ranking problem into a more tractable, continuous representation. Specifically, the edge ranking, which is a permutation of integers, is encoded into a 2D matrix. This encoding is crucial because it preserves the necessary information while circumventing non-differentiable operations like sorting and indexing, which would otherwise complicate mathematical analysis and optimization.

The intuition behind a vulnerable edge is that its current importance, relative to the selection boundary (the threshold for being included in the top K edges of the global model), is sufficiently close to this boundary such that a malicious client can push it across. More formally, an edge is considered vulnerable if the difference in its importance score and the selection boundary is smaller than the maximum damage an attacker can cause in a single communication round. Based on this intuition, the researchers theoretically proved the existence of such vulnerable edges within the network. Furthermore, they derived both upper and lower bounds for identifying these vulnerable edges within each layer of the neural network, providing a systematic method for their discovery. This theoretical groundwork is fundamental to the attack, as it pinpoints the specific targets for manipulation.

2. Optimization Function Formulation

Once the vulnerable edges are identified, the next stage focuses on formulating an optimization function to manipulate them effectively. The primary objective is to maximize the deviation of the global model's importance for these vulnerable edges after the attack compared to their original values before the attack. This maximal deviation ensures that the adversary's influence is amplified on the most critical, yet susceptible, components of the network.

The challenge here lies in the inherent discreteness of the edge ranking space. Since edge rankings are permutations of integers, the optimization function is not continuous, making direct gradient-based optimization methods inapplicable. To overcome this, the researchers ingeniously utilize the Gumbel-softmax method. Gumbel-softmax is a differentiable approximation of a categorical distribution, which allows for the conversion of a discrete optimization problem into a continuous one. By applying Gumbel-softmax, the discrete 2D matrix representing the edge rankings can be treated as a continuous variable during optimization. This enables the simultaneous updating and optimization of the discrete matrix, allowing the attack to efficiently find the optimal parameters that maximize the desired manipulation of vulnerable edges. The optimization process thus identifies the specific malicious rankings that, when aggregated, will have the greatest impact on the target vulnerable edges.

3. Malicious Update Generation

In the final stage, the optimized parameters obtained from the Gumbel-softmax-enabled optimization function are used to generate the actual malicious updates. These updates are crafted to represent the adversary's contribution to the edge rankings, designed to specifically push the identified vulnerable edges across the selection boundary or significantly alter their perceived importance. When these malicious rankings are submitted by the controlled clients and aggregated by the server's majority voting mechanism, they exert the maximum possible influence on the global model, leading to a substantial degradation of its performance or a shift in its learned features. This three-stage process ensures that the attack is highly targeted, computationally efficient, and maximally impactful despite the FRL framework's robust design principles.

Demo / Proof of Concept

▶ Watch: Stage 1: Identifying vulnerable edges theoretically (6:00)

While the talk does not describe a live demonstration of the Vulnerable Edge Manipulation (VEM) attack, it presents comprehensive experimental results that serve as a robust proof of concept and validation of its efficacy. The researchers conducted extensive experiments to compare VEM against six different state-of-the-art attacks, evaluating their performance under seven distinct defense mechanisms commonly employed in federated learning.

The results unequivocally demonstrated VEM's superior performance, outperforming all other attacks by a significant margin. The attack achieved a remarkable 53% attack impact, which was quantified as being 3.7 times more impactful than the next best existing attack. This metric likely refers to the degradation of the global model's accuracy or some other critical performance indicator, showcasing the severe consequences of VEM.

Furthermore, the study included detailed analyses of the attack's impact under varying conditions. Experiments were conducted to assess the attack impact with different percentages of malicious clients, demonstrating that VEM maintains its effectiveness even with a limited number of adversaries. Another crucial study investigated the attack's performance under different degrees of data heterogeneity (referred to as "90 degree" in the transcript) within the dataset. These results consistently showed that VEM significantly outperforms other attacks across diverse data distribution scenarios, highlighting its robustness and generalizability. These empirical findings provide compelling evidence that VEM successfully identifies and exploits the critical vulnerabilities within Fed-Ranking Learning, validating the theoretical claims and demonstrating the practical threat posed by this novel attack.

Defensive Implications

▶ Watch: Experimental results: BAM significantly outperforms existing attacks (9:00)

The findings presented in this talk carry significant defensive implications for the design and deployment of Fed-Ranking Learning (FRL) and, more broadly, for secure Federated Learning (FL) systems. The most critical implication is that FRL, despite its innovative use of edge rankings and majority voting for aggregation, is demonstrably not as robust as previously believed. The existence of vulnerable edges and the effectiveness of the Vulnerable Edge Manipulation (VEM) attack reveal a fundamental weakness that defenders must acknowledge and address.

For practitioners and researchers working with FRL, this research underscores the urgent need to re-evaluate the security posture of their systems. Relying solely on the discrete nature of ranking-based communication or the consensus mechanism of majority voting is insufficient to guarantee robustness against sophisticated, targeted attacks like VEM. New defense mechanisms are imperative to protect against the precise manipulation of these vulnerable edges.

Potential defensive strategies could involve:

• Enhanced Anomaly Detection: Developing more sensitive anomaly detection algorithms that can identify subtle, yet coordinated, manipulations of edge rankings, particularly those targeting edges near the selection boundary. Traditional defenses might overlook these manipulations if they don't drastically alter the overall ranking but strategically shift a few critical ones.
• Adaptive Aggregation Mechanisms: Exploring alternative aggregation strategies that are less susceptible to targeted attacks on vulnerable edges. While majority voting is robust against random noise, VEM shows it's vulnerable to intelligent, coordinated attacks. This could involve incorporating trust scores for clients, weighting votes based on historical behavior, or using more robust statistical aggregation methods that are less sensitive to outliers in the vicinity of the selection threshold.
• Certified Robustness: The speaker explicitly mentions future work on "certified robustness evaluation method." This is a crucial direction for defense, aiming to provide mathematical guarantees that the FRL model will maintain its performance within certain bounds even under adversarial conditions. This would involve designing FRL variants that are provably resilient to attacks like VEM.
• Dynamic Selection Boundaries: Instead of fixed top-K edge selection, introducing dynamic or adaptive selection boundaries that are harder for an adversary to predict or exploit.
• Obfuscation of Edge Importance: Making it harder for malicious clients to accurately identify vulnerable edges by obfuscating or randomizing the perceived importance of edges, without compromising the overall model utility for benign clients.

Ultimately, the work signals that the security landscape for ranking-based FL is more complex than initially thought. Defenders need to move beyond generic poisoning defenses and develop fine-grained, context-aware countermeasures that specifically target the mechanics of ranking manipulation and the exploitation of critical network components.

Key Takeaways

• Fed-Ranking Learning (FRL) is not fully robust: Despite its design with edge rankings and majority voting, FRL possesses critical vulnerabilities that can be exploited by sophisticated attackers.
• Discovery of "Vulnerable Edges": The research identifies and theoretically proves the existence of "vulnerable edges" – specific neurons whose importance can be subtly manipulated to significantly impact the global model.
• Vulnerable Edge Manipulation (VEM) is highly effective: The VEM attack, designed to exploit these vulnerabilities, achieves a 53% attack impact, making it 3.7 times more effective than existing state-of-the-art attacks.
• Gumbel-softmax is crucial for attacking discrete spaces: The use of the Gumbel-softmax method is key to converting the discrete edge ranking problem into a continuous one, enabling effective optimization and manipulation.
• Urgent need for new FRL defenses: The findings highlight the necessity for developing more advanced, targeted defensive mechanisms to protect FRL systems against precise adversarial manipulations of network components.
• Future work includes targeted attacks and certified robustness: The researchers plan to investigate targeted attacks against FRL and develop certified robustness evaluation methods to build a stronger foundation for secure FL.

About the Speaker(s)

The primary speaker for this presentation is Zirui Gong, a PhD student at Griffith University in Australia. His research focuses on the security aspects of Federated Learning frameworks, particularly in evaluating their robustness against various client-side attacks. This work is a collaborative effort involving other researchers, including Yanjun Zhang, Leo Yu Zhang, Zhaoxi Zhang, Yong Xiang, and Shirui Pan, from Griffith University, University of Technology Sydney, and Deakin University, indicating a strong academic background in machine learning security and distributed systems.