IRRedicator: Pruning IRR with RPKI-Valid BGP Insights

IRRedicator: Pruning IRR with RPKI-Valid BGP Insights

Speakers: Minhyeok Kang

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=AdCCGWtsn2A

Overview

The Internet's routing infrastructure, underpinned by the Border Gateway Protocol (BGP), faces persistent security vulnerabilities due to its foundational design prioritizing stability over security. This talk introduces IRRedicator, a novel machine learning-based approach designed to significantly enhance the reliability of the Internet Routing Registry (IRR) by identifying and eliminating stale or inaccurate Route objects. Presented by Minhyeok Kang, this work addresses a critical challenge: while the Resource Public Key Infrastructure (RPKI) offers strong cryptographic routing security, its adoption remains limited, leaving the more widely deployed but often unreliable IRR as a primary source for BGP validation.

IRRedicator tackles the pervasive problem of outdated entries within the IRR, which can lead to BGP hijacking, route leaks, and misconfigurations. By leveraging a comprehensive longitudinal study of inconsistencies between IRR and RPKI data, coupled with a detailed characterization of BGP announcement patterns, the system employs a LightGBM classification model to predict the validity of Route objects. This research is crucial for network operators and Internet service providers seeking to improve the accuracy of their BGP filtering policies, thereby bolstering the overall security and stability of the global routing system. The findings underscore the continued importance of the IRR and provide a robust, data-driven mechanism to make it a more trustworthy resource in the evolving landscape of routing security.

Background

▶ Watch: Introduction to IRRedicator and BGP security challenges (0:00)

The Border Gateway Protocol (BGP) is the distributed routing protocol that enables global internet connectivity. However, its original design from decades past lacked inherent security mechanisms, making it susceptible to various attacks and misconfigurations. Routers traditionally cannot authenticate the origin of announced IP prefixes, paving the way for malicious actors to announce prefixes they do not legitimately own, leading to traffic diversion or "prefix hijacking." This fundamental flaw has driven the development of two primary data sources aimed at providing reliable mappings between IP prefixes and their authorized Autonomous Systems (ASes): the Internet Routing Registry (IRR) and the Resource Public Key Infrastructure (RPKI).

The IRR, introduced in 1995, is a globally distributed database where network operators publish their routing information, primarily through Route objects. A Route object specifies an IP prefix and the AS authorized to announce it, often using the Routing Policy Specification Language (RPSL) for rich metadata. Operators download these objects to validate and filter BGP announcements. However, IRR's voluntary nature and lack of enforcement mechanisms have led to a significant problem: stale objects. These are outdated entries that no longer reflect current ownership, often because operators fail to update them, and there's no easy mechanism for third parties to remove old data. This unreliability severely hampers IRR's effectiveness.

To overcome IRR's limitations, RPKI was introduced in 2008. RPKI provides a cryptographically verifiable way to bind IP prefixes to their authorized origin ASes using Route Origin Authorization (ROA) objects. A ROA specifies an IP prefix, its maximum length, and the AS authorized to originate it. These objects are digitally signed by Certificate Authorities (CAs) rooted in the Regional Internet Registries (RIRs), ensuring their authenticity. Routers can validate BGP announcements by checking if the origin AS matches a valid ROA. Despite its strong security guarantees, RPKI's adoption has been slow due to operational complexities, the risk of misissued ROAs causing legitimate routes to be filtered, and its inability to protect against route leaks further down the AS path. Deployment disparities across RIRs are also evident; as of March 2023, RIPE NCC had 59.2% IPv4 prefix coverage by ROAs, while AFRINIC only had 23.9%.

Several efforts have attempted to improve IRR quality. IRR pruning with RPKI validation involves removing IRR objects that conflict with RPKI ROAs. RIPE NCC initiated this, and IRRd version 4 implemented support for it. However, this only applies to the subset of IRR objects covered by RPKI, which, as the study shows, is less than half. Another method is age-based filtering, like that used by JPIRR, which removes Route objects not updated for over a year. While this encourages operator maintenance, it can inadvertently remove valid, but infrequently updated, entries. Both methods have limited coverage, leaving a significant portion of IRR objects unaddressed.

To understand the current routing security landscape, a 12-year longitudinal study was conducted, analyzing IRR and RPKI deployment alongside real BGP announcements. As of March 1st, 2023, ROA objects covered approximately 37.15% of the IPv4 address space and 28.46% of ASes. In stark contrast, the composite ALL-IRRs database covered a much larger 74.23% of IPv4 space and 68.80% of ASes. This highlights IRR's significantly broader deployment. Despite RPKI's growth from 15.8% to 37.2% IPv4 coverage between December 2019 and March 2023, it still covers less than 40% of the total IPv4 address space, underscoring IRR's continued importance. Notably, a mid-2022 decrease in IRR's IPv4 coverage by 8.3 percentage points was due to the removal of 6.7K Route objects, 3.4K of which belonged to the DoD Network Information Center and lacked ROA coverage, indicating a significant gap.

Regarding BGP announcement coverage, IRR covered 93.1% of BGP announcements, more than double RPKI's 42.1%. For BGP validation, the fraction of valid announcements against IRR reached 92.7%, and against RPKI, it was 97.4% as of March 1st, 2023. Significant drops in validation percentages were observed, often linked to large-scale misconfigurations or attacks, such as AS37468 (Angola Cables) making over 176K unannounced BGP announcements on July 19th, 2018. In summary, while IRR offers superior coverage, its observed validation drops and prevalence of stale objects mean it cannot be reliably used alone for BGP announcement validation. This background sets the stage for IRRedicator's goal: to leverage RPKI insights to enhance IRR quality where RPKI itself falls short in coverage.

Key Findings

▶ Watch: Detailed overview of IRR, RPKI, and existing solutions (2:00)

The comprehensive longitudinal study and subsequent development of IRRedicator yielded several critical findings and contributions that redefine the approach to Internet routing security:

• Superior IRR Coverage but Persistent Staleness: The study unequivocally confirmed that the Internet Routing Registry (IRR) continues to offer significantly broader coverage than RPKI. As of March 2023, IRR covered 74.2% of the IPv4 address space and 93.1% of BGP announcements, compared to RPKI's 37.2% IPv4 and 42.1% BGP announcement coverage. This highlights IRR's indispensable role in routing despite its known reliability issues.
• Growing Inconsistencies Between IRR and RPKI: A critical observation was the increasing number of inconsistencies between IRR Route objects and RPKI ROAs. While the percentage of inconsistent IP prefixes among overlapping entries decreased from 11.2% to 5.7% (for ALL-IRRs), the absolute number of such inconsistent IP prefixes actually increased by a factor of 2.7 over the measurement period. These inconsistencies affected 5.3% of BGP announcements, with the number of inconsistent BGP announcements increasing from 7.7K to 20.8K, underscoring a worsening problem of stale IRR data conflicting with legitimate RPKI attestations.
• Distinct Characteristics of Stale Route Objects: A thorough characterization revealed clear patterns distinguishing inconsistent (stale) Route objects from consistent ones based on their BGP announcement behavior. Inconsistent Route objects were found to be significantly older (median age 7.9 years vs. 2.9 years for consistent objects), exhibited extremely low activeness in BGP (only 0.6% active in a single day window vs. 29.3% for consistent), showed lower Lifespan and Uptime metrics, and had significantly longer inactive periods. These distinct BGP dynamics form the empirical basis for IRRedicator's detection technique.
• Development of the IRRedicator ML Model: The research successfully developed IRRedicator, a machine learning-based technique using LightGBM augmented with a classification-with-rejection mechanism. This model effectively identifies stale Route objects by leveraging the observed BGP announcement patterns, even for those not covered by RPKI.
• High Accuracy and Generalizability: IRRedicator demonstrated exceptional performance. It achieved an F1-score of 0.988 (precision 0.981, recall 0.996) on RPKI-labeled test sets. Crucially, its generalizability was validated against an independent ground truth dataset of transfer logs, achieving an F1-score of 0.975 (precision 0.952, recall 0.999), confirming its robust ability to identify stale Route objects regardless of RPKI overlap.
• Significant Improvement in IRR Data Quality: When compared to existing filtering methods, IRRedicator (IRR-ML) proved far more effective. It filtered out a remarkable 1.57 million (58.5%) Route objects from the ALL-IRR database. This drastic pruning reduced the percentage of potentially stale objects from 72% to 40% of the total, while still maintaining 93% accuracy in validating BGP announcements and covering 87% of all BGP announcements. Only 0.07% of objects filtered by IRR-ML were actively announced in BGP, compared to 3.12% for IRRd4-filtered objects, indicating superior identification of truly stale data.
• Model Reliability and Explainability: The model demonstrated high reliability, maintaining an AUC score of 0.989 even with a one-year gap between training and testing data. SHAP analysis revealed that Lifespan and Uptime were the most influential features for prediction. Furthermore, the model is resilient against adversarial attacks due to its reliance on historical, multi-vantage point BGP data and existing filtering mechanisms, successfully identifying 66.0% of real-world prefix hijack incidents as invalid.

These findings collectively highlight the critical need for a dynamic, data-driven approach to managing IRR data and establish IRRedicator as a powerful tool for improving the security and trustworthiness of Internet routing.

Technical Deep Dive

▶ Watch: Longitudinal study on IRR and RPKI coverage over time (4:00)

IRRedicator's technical core lies in its sophisticated machine learning pipeline, designed to characterize and classify the validity of IRR Route objects based on their observed BGP announcement patterns. This approach moves beyond simple heuristics or direct RPKI conflict detection to infer staleness even for objects not covered by RPKI.

The process begins with dataset construction, where each Route object $x_i$ is transformed into a feature vector. The target label $y_i$ is assigned based on RPKI validation status: 1 for RPKI-valid, 0 for RPKI-invalid. Route objects not covered by RPKI ROAs are left unlabeled and excluded from the training set, as their ground truth cannot be established solely through RPKI. The feature vector $x_i$ is comprehensive, comprising 312 features categorized into two groups:

1. Window-based Features (260 features): These are derived from a set of 13 key BGP pattern metrics, measured across 20 different monitoring window sizes. The metrics include:

• Lifespan: The duration between the first and last observed BGP announcement for a prefix-origin pair within a window.
• Uptime: The total number of days a BGP announcement for the pair was observed within the window.
• Relative Uptime: The quotient of Uptime and Lifespan, indicating announcement frequency.
• Up: The number of transitions from an inactive to an active state.
• Down: The number of transitions from an active to an inactive state.
• Active Days: Statistics (min, max, average, standard deviation) on consecutive active days.
• Inactive Days: Statistics (min, max, average, standard deviation) on consecutive inactive days.

Measuring these 13 metrics across 20 distinct window sizes (e.g., 1 day, 7 days, 30 days, up to 12 years) yields $13 \times 20 = 260$ features.

1. Statistical Features (52 features): For each of the 13 BGP pattern metrics, statistical measures (minimum, maximum, average, and standard deviation) are computed across the 20 monitoring window sizes. This provides $13 \times 4 = 52$ additional features, capturing the variability and trends of each metric across different observation periods.

A crucial step is dataset filtering to mitigate potential mislabeling in the ground truth. This addresses two primary issues: errors in the MaxLength attribute of ROAs, which could incorrectly mark a valid Route object as invalid, and cases where Route objects had different ASNs from corresponding ROAs but both ASNs belonged to the same ISP. These issues are mitigated by carefully considering MaxLength attributes and establishing AS relationships to correct labels where appropriate.

For model selection, a machine learning approach was chosen over simple heuristic thresholds due due to its superior capabilities in automated threshold selection, generalization to unseen data, and robustness to noise. Ten candidate ML models were evaluated, including Multi-Layer Perceptron, Logistic Regression, SVM, Decision Tree, Random Forest, Bagging, AdaBoost, Gradient Boosting Machine, XGBoost, and LightGBM. Performance was measured using average precision (area under the precision-recall curve). LightGBM emerged as the top performer, achieving an average precision of 0.9998 for ALL-IRR testsets and 0.9996 for RADb testsets, while also demonstrating high training efficiency.

A key design consideration is preventing the incorrect prediction and potential deletion of valid Route objects. To address this, IRRedicator integrates a classification with rejection technique into the LightGBM model. This allows the model to refrain from making a prediction when it lacks sufficient confidence (uncertainty) or when multiple classes appear equally dominant (ambiguity). This is implemented using a custom loss function and specific LightGBM attributes. When a prediction is rejected, it flags the Route object for potential expert human intervention, adding a crucial layer of safety.

The system design and implementation envision IRRedicator being applied by IRR administrators (like RADb and RIRs) through a multi-step validation process for each Route object:

1. Activeness Check: The system first verifies if the prefix-origin pair of the Route object has been observed by any BGP vantage points within the largest monitoring window (e.g., 12 years). If no activity is detected, it's marked as inactive.
2. Rejection Check: If active, the system applies the classification-with-rejection conditions. If the model is uncertain or ambiguous, the Route object is marked as rejected, deferring to human review.
3. Classification: If not rejected, the LightGBM model assigns a predicted value (probability of validity) to the Route object. A higher value indicates a higher probability of being valid.

Integration into existing IRR systems can be achieved by leveraging the flexible structure of RPSL. Similar to how IRRd version 4 introduced the rpki-ov-state attribute for RPKI validation results, IRRedicator's validation result can be stored in a new attribute within Route objects. This allows network operators to define custom policies based on the model's predicted values, such as disregarding or assigning lower priority to origin information if the predicted value falls below a certain threshold.

A potential challenge is detection delay – the time it takes to identify a stale object if a legitimate prefix is transferred but the original Route object isn't deleted. To mitigate this, inspired by work on de-facto ownership, a label-flipping augmentation technique is incorporated. This involves perturbing the features of a valid Route object to simulate a lack of announcements over a specified period (e.g., x days) and then reversing its label to 'invalid'. A "de-facto owner" is defined as an entity that has announced a prefix for at least 11 consecutive days. If a prefix-origin has not been announced for 11 days, it is no longer considered a de-facto owner, and its label is changed to 'invalid'. This allows identification of stale objects if they haven't been announced for about two weeks. However, a limitation exists: legitimate reasons for not announcing IP prefixes for extended periods (e.g., traffic engineering, security concerns) could lead to inadvertent deletion of valid entries, a trade-off requiring careful consideration.

Demo / Proof of Concept

▶ Watch: IRR vs. RPKI: BGP announcement coverage comparison (5:00)

While the talk does not describe a live demonstration of the IRRedicator system in action, the rigorous evaluation and results section serves as a comprehensive proof of concept for the model's efficacy and generalizability. The research meticulously validated IRRedicator's performance against multiple ground truth datasets and compared it with existing IRR filtering approaches, demonstrating its practical viability and significant improvements.

The model's performance was first evaluated using a test set labeled with RPKI validation results. Through Bayesian optimization with 5-fold cross-validation for hyperparameter tuning, IRRedicator achieved remarkably high performance. At a threshold of 0.202, it reached a maximum F1-score of 0.988, with a precision of 0.981 and a recall of 0.996. This indicates the model's excellent balance between identifying valid objects correctly and not missing actual stale objects. Both precision and recall remained consistently above 0.98 across a wide range of thresholds, showcasing its robustness.

To assess model generalizability beyond RPKI-derived labels, an independent ground truth dataset of transfer logs was used. These logs provide information about IP prefix transfers between organizations, allowing for the identification of 25,143 valid and 1,323 stale Route objects. Even with this entirely separate dataset, IRRedicator demonstrated strong performance, achieving a maximum F1-score of 0.975 (precision 0.952, recall 0.999) at a threshold of 0.000003. At the 0.202 threshold used in the RPKI-labeled evaluation, it still achieved an F1-score of 0.971. These results conclusively confirm the model's ability to generalize and accurately identify stale Route objects using data independent of RPKI, proving its real-world applicability.

Furthermore, IRRedicator (referred to as IRR-ML) was compared against existing IRR filtering approaches: (i) the whole IRR (no filtering) and (ii) IRRd4's RPKI validation. The comparison used four criteria: number of remaining Route objects, ratio of active Route objects, ratio of covered BGP announcements, and ratio of valid BGP announcements.

For the ALL-IRR database, IRRd4 filtered out 0.29 million (10.8%) Route objects, showing comparable BGP-related performance to the unfiltered IRR. In contrast, IRRedicator filtered out a significantly larger number: 1.57 million (58.5%) Route objects. While this led to a slight decrease in the ratio of covered BGP announcements, only 0.07% of the Route objects filtered by IRRedicator were actively announced in BGP, compared to 3.12% for IRRd4-filtered objects. This stark difference indicates IRRedicator's superior effectiveness at identifying truly stale or unused Route objects. The average Uptime for IRRedicator-filtered objects was a mere 0.027 over a 14-day window, equivalent to only 0.38 active days.

A similar comparison using the JPIRR database, which employs age-based filtering, further highlighted IRRedicator's advantages. IRRd4 again filtered a small number of objects, performing similarly to the unfiltered JPIRR. However, IRRedicator filtered out 6.5K (25.1%) Route objects from JPIRR. Despite a negligible decrease in covered BGP announcements, the percentage of active Route objects in JPIRR increased by 11.0 percentage points, reaching 45.0%. This demonstrates that IRRedicator significantly improves the quality of IRR databases managed even by existing age-based policies.

These rigorous evaluations provide strong evidence that IRRedicator is not just a theoretical concept but a highly effective and practical solution for pruning stale IRR entries, significantly enhancing the quality and trustworthiness of IRR data.

Defensive Implications

▶ Watch: Leveraging RPKI to identify and address IRR inconsistencies (6:00)

IRRedicator offers profound defensive implications for network operators, Internet service providers, and Regional Internet Registries (RIRs) striving to secure the global routing infrastructure. By providing a robust, data-driven mechanism to prune stale and inaccurate entries from the Internet Routing Registry (IRR), it directly addresses a long-standing vulnerability in BGP security.

1. Enhanced BGP Filtering Accuracy: The primary defensive benefit is the ability to significantly improve the accuracy of BGP filtering. Network operators rely on IRR data to build ingress and egress filters, preventing the propagation of unauthorized routes. By using IRRedicator to identify and remove stale Route objects, operators can ensure their filters are based on up-to-date and legitimate routing information. This reduces the risk of BGP hijacking and route leaks caused by outdated or malicious IRR entries. The model's ability to achieve 93% accuracy in BGP announcement validation while covering 87% of all BGP announcements represents a substantial improvement over unfiltered or conventionally pruned IRR data.

1. Reduced Misconfigurations and Outages: Stale IRR data can lead to unintended filtering of legitimate routes or the acceptance of invalid routes, causing service outages and connectivity issues. IRRedicator's ability to filter out 58.5% of ALL-IRR Route objects, with only 0.07% of these being actively announced, means that a vast amount of "noise" and potential sources of error can be eliminated. This directly contributes to a more stable and resilient routing environment.

1. Proactive Security Posture: IRRedicator empowers RIRs and IRR administrators to take a more proactive stance against routing insecurity. Instead of relying solely on voluntary operator updates, which often lag, they can continuously run the IRRedicator model against their databases. The model's reliability (AUC score 0.989 with a one-year gap) and flexibility with monitoring windows (good performance even with a 1-day window) make it suitable for continuous deployment. Integrating the model's predicted values into new RPSL attributes, similar to rpki-ov-state in IRRd4, allows for dynamic policy enforcement and operator-defined thresholds for trustworthiness.

1. Bridging the RPKI-IRR Gap: While RPKI offers superior cryptographic guarantees, its limited deployment (less than 40% IPv4 coverage) means IRR remains critical for the majority of the Internet. IRRedicator effectively bridges this gap by injecting RPKI-derived insights and BGP-observed ground truth into the broader IRR ecosystem. It ensures that the vast majority of prefixes not yet covered by RPKI can still benefit from enhanced validation and pruning, leveraging observed BGP behavior as a proxy for legitimacy.

1. Informed Policy Development: The detailed characterization of stale objects – their age, activeness, Lifespan, and Uptime – provides valuable insights for policy makers. RIRs and network communities can use this information to develop more effective policies for IRR registration and maintenance, potentially incentivizing operators to keep their records current or automating the removal of clearly inactive entries.

1. Consideration of Trade-offs: Defenders must be aware of the inherent trade-off in the detection delay mitigation technique, where legitimately unannounced prefixes might be flagged as stale. While the classification-with-rejection mechanism helps by flagging uncertain cases for human review, operators need to establish clear policies for handling such instances. This might involve a grace period, manual override procedures, or additional contextual checks before permanent deletion. The goal is to maximize security gains without inadvertently disrupting legitimate network operations.

By adopting IRRedicator, network operators gain a powerful tool to clean their routing data, enhance their BGP filtering, and move closer to a more secure and stable Internet routing environment, complementing the ongoing efforts of RPKI deployment.

Key Takeaways

• IRR's Enduring Importance: Despite RPKI's cryptographic security, the Internet Routing Registry (IRR) remains critical due to its significantly broader coverage (74.2% of IPv4 space, 93.1% of BGP announcements), highlighting the urgent need to address its pervasive staleness problem.
• Escalating Inconsistencies: There is a growing number of conflicting IRR entries when compared to RPKI (5.7% of overlapping prefixes), impacting 5.3% of BGP announcements, demonstrating a worsening problem of unreliable IRR data.
• Predictive Power of BGP Patterns: Stale IRR objects exhibit distinct BGP announcement patterns—they are older, less active, have shorter Lifespans and Uptimes, and longer inactive periods—which can be effectively leveraged to identify them.
• IRRedicator's High Accuracy and Generalizability: The LightGBM-based IRRedicator model, enhanced with a classification-with-rejection mechanism, achieves exceptional accuracy (F1-score of 0.988) and generalizes robustly across different ground truths (F1-score of 0.975 on independent transfer logs).
• Significant Improvement in IRR Quality: IRRedicator drastically prunes stale Route objects, filtering out 58.5% of ALL-IRR entries. This improves BGP announcement validation accuracy to 93% while still covering 87% of all announcements, making IRR data significantly more trustworthy.
• Practical Defensive Tool: IRRedicator offers network operators and RIRs a powerful, data-driven tool to enhance BGP filtering, reduce misconfigurations, and proactively secure routing infrastructure by providing a reliable method to identify and manage outdated IRR information.

About the Speaker(s)

The talk "IRRedicator: Pruning IRR with RPKI-Valid BGP Insights" was presented by Minhyeok Kang. Based on the provided transcript and metadata, Minhyeok Kang is the researcher who presented this work at the NDSS Symposium. No further biographical details about their title, affiliation, or specific background are available within the provided materials.