What would you ask a crystal ball for AWS IAM?

What Would You Ask a Crystal Ball for AWS IAM?

Speakers: Nick Siow

Conference: fwd:cloudsec North America 2025

YouTube: https://www.youtube.com/watch?v=nkLNlvXZ8CM

Overview

Nick Siow from Netflix's cloud security team presented the journey from a failed enterprise metrics initiative to the creation of Yams, a newly open-sourced IAM simulation engine designed to answer the questions that existing tools cannot handle at massive scale. Born from the frustrating realization that Netflix's AWS environment — with over 100,000 IAM roles and equally numerous resources — was simply too large and complex for existing IAM analysis tools, Yams provides a Go-based library, server, and CLI that can load an entire enterprise IAM environment, simulate access decisions across billions of permission combinations, and deliver answers in seconds rather than hours. The tool was open-sourced the day of the talk.

Background

▶ Watch: The IAM edge case: a principal with no policies but resource policy trust sti... (3:30)

Netflix is one of the oldest and largest AWS customers, with infrastructure spanning over a decade of architectural decisions. This creates what Siow described as "archaeological layers" of IAM configurations — role types from different eras, migrations through multiple IAM features, and patterns that no longer match their original intent. The environment has every conceivable access pattern, from legacy roles older than many practitioners' careers to bleeding-edge configurations using the latest IAM features.

The project originated not in cloud security but in an enterprise security metrics program that needed simple quantitative answers to questions like "how many principals have access to this resource?" In most domains, this is straightforward — who can access a Google Doc is answered by looking at the access control list. In AWS IAM, answering the same question requires evaluating inline policies, attached policies, resource policies, permission boundaries, Resource Control Policies (RCPs), Service Control Policies (SCPs), default root trust, and edge cases like explicitly trusted principals in resource policies having access regardless of their own empty policy set.

Existing tools fell short at Netflix's scale. AWS Policy Simulator was inaccurate and clunky. Access Analyzer was growing but prohibitively expensive. PMapper and IAMSpy were excellent tools but could not handle Netflix's environment — PMapper ran for 8 hours attempting to build an access graph before running out of memory. The IAM policy data for the environment, serialized to JSON, measured in multiple gigabytes.

Key Findings

▶ Watch: 50-200 billion simulations needed just for SQS access metrics (10:00)

There is a capability gap between IAM questions and tooling answers. The tools that existed before Yams could handle thousands of simulations but fell apart at millions or billions. For Netflix's environment, calculating basic access metrics for just SQS (100,000+ roles times a handful of actions times 100,000+ queues) required 50 to 200 billion simulations.

The "who" and "why" matter more than the "how many." The first iteration of Siow's metrics tool could produce a number — how many principals can access a resource — but that number was not actionable without knowing which principals and why they had access. Debugging requires understanding which policy layer (identity policy, resource policy, SCP, permission boundary) is producing the allow or deny decision.

IAM is more like a programming language than an access control list. The more advanced IAM features an organization uses — ABAC, condition keys, cross-account trust, multi-resource API calls — the less the authorization model resembles a static list and the more it resembles a program that must be executed to understand.

Two new metrics for IAM governance: Exposure and Efficiency. Exposure measures how many principals can access a resource. Efficiency divides the number of principals that actually need access (from CloudTrail data events or instrumented SDK telemetry) by the number that can access it. An efficiency of 100% means access is perfectly right-sized; 1% means you are 99x over-provisioned. Both metrics translate intuitively into OKRs and executive reporting.

Technical Deep Dive

▶ Watch: Live CLI demo: loading environment in 10 seconds, wildcard simulation (20:00)

Yams is written in Go for performance and operates as a library, REST API server, and CLI. It ingests data from AWS Config (periodically refreshed from S3 or local files) and loads the full IAM environment — principals, resources, policies, accounts, SCPs, RCPs — in approximately 10 seconds, compared to the hours required by previous tools.

The CLI wraps the REST API and supports several query modes. Single simulation takes a principal, action, and resource triplet and returns allow/deny with the reasoning chain. Wildcard simulation allows omitting any one of the three parameters: omit the principal to find all principals that can perform an action on a resource; omit the resource to find all resources a principal can access with a given action; omit the action to find all actions a principal can perform on a resource.

The explain flag provides a human-readable explanation of why an access decision was made — for example, "Red Role has access except for an explicit deny in the resource policy on this bucket." The trace flag walks through the entire evaluation chain: resource policies, SCPs, identity policies, permission boundaries, showing how each layer contributed to the final decision.

Overlays enable hypothetical analysis. Users can redefine any aspect of their environment — modify an SCP, change a tag, add a permission to a role — and simulate the impact without making actual changes. This addresses the critical use case of "what happens if I deploy this SCP change?" which has historically been a source of production incidents at Netflix.

The tool supports full IAM feature coverage including ABAC (Attribute-Based Access Control) with tag conditions, organization-based conditions, cross-account trust, permission boundaries, and all policy types. Resource policies with explicit principal trust (the edge case where a principal with no identity policies can still have access via resource policy trust) are correctly modeled.

For Netflix's metrics program, Yams enables calculation of Exposure (principals with access to a resource) and Efficiency (principals that need access divided by principals that have access). Since Netflix cannot afford CloudTrail Data Events at their scale, they instrument SDKs directly to collect access telemetry — a technique previously presented at re:Invent 2022.

Demo / Proof of Concept

▶ Watch: Explain flag reveals explicit deny from ABAC resource policy (24:00)

Siow demonstrated the Yams CLI against a real (personal) AWS test environment. The demo showed server startup with S3-based data sources loading in approximately 10 seconds, inventory queries with case-insensitive search, principal inspection with permission boundary resolution, resource policy inspection, single-point simulation with allow/deny results, wildcard simulation finding all principals with access to a bucket, the explain flag revealing an explicit deny from a resource policy with ABAC conditions, the trace flag walking through the full evaluation chain, and overlay-based hypothetical simulation adding S3:PutObjectAcl to a role and seeing it reflected in results.

Defensive Implications

▶ Watch: Exposure and Efficiency metrics for IAM governance and OKRs (33:00)

Yams directly addresses several critical defender needs. For IAM troubleshooting, on-call engineers can quickly determine why an access decision is being made without manually navigating multiple policy layers. For posture checks, organizations can define security invariants ("no principal outside this account should access this resource") and programmatically verify them. For boundary analysis, teams can measure how much IAM access crosses environment boundaries (test-to-prod, account-to-account). For change impact analysis, the overlay feature allows previewing SCP, RCP, or policy changes before deployment.

Organizations should evaluate Yams as a complement to or replacement for existing IAM analysis tools, particularly if they are large AWS Config customers with complex IAM configurations that overwhelm PMapper, IAMSpy, or Access Analyzer's cost model.

Key Takeaways

• Yams is a new open-source IAM simulation engine written in Go, released the day of this talk, capable of loading and simulating massive AWS environments.
• Existing IAM tools break at scale: PMapper, IAMSpy, and Access Analyzer either run out of memory, take too long, or cost too much for very large environments (100K+ roles).
• IAM is a programming language, not an ACL: Advanced features make static analysis insufficient; simulation is required to understand actual access.
• Exposure and Efficiency are new IAM governance metrics: They translate complex IAM posture into intuitive numbers that work for executive reporting and OKRs.
• Overlays enable safe change previewing: Simulate the impact of SCP, policy, or tag changes without touching the real environment.
• AWS should provide this capability natively: Three separate talks at this conference addressed the same IAM understanding gap, which only the first party can solve with 100% accuracy.

About the Speaker(s)

Nick Siow is on the cloud security team at Netflix, having previously worked at Apple in a similar role. His GitHub is publicly available, and he describes his favorite AWS services as IAM, S3, and SQS. Yams began as a 20% side project in 2023 and evolved through multiple iterations (iam-metrics-next, metrics-v2) before being open-sourced. He noted the irony of presenting Netflix-specific tooling at the same conference where another speaker said "You Are Not Netflix."