Verify end-to-end SMS delivery after TCR approval

[moveson]

After TCR approves the 10DLC campaign (#1893), verify that SMS notifications are delivered end-to-end:

• Subscribe to SMS notifications for a participant in an active event
• Confirm SNS subscription is created and confirmed (ARN populated)
• Trigger a progress notification (e.g., add a split time in live entry)
• Verify SMS is received on the subscribed phone number
• Reply STOP — verify opt-out is handled by the carrier
• Reply START — verify re-opt-in works

Once verified, close #1216.

Tracked by #1216.

[moveson]

End-to-end SMS delivery verified ✅ (2026-04-28)

A real ProgressNotifier publish from the production OST app reached the subscriber's phone via SNS topic fan-out → SMS delivery. The path is confirmed working end-to-end.

Acceptance status

• Subscribe to SMS notifications for a participant in an active event
• Confirm SNS subscription is created and confirmed (ARN populated) — verified via aws sns list-subscriptions-by-topic on follow-2026-testrock-100-mark-oveson
• Trigger a progress notification — fired the test event from prod
• Verify SMS is received on the subscribed phone number
• Reply STOP — verify opt-out is handled by the carrier (not yet exercised)
• Reply START — verify re-opt-in works (not yet exercised)

The STOP/START checks are independently testable; AWS handles them automatically via the Default opt-out list (SelfManagedOptOutsEnabled: false). They are functionally orthogonal to the publish path that this issue gates. Recommend leaving them as a quick spot-check before closing #1216.

What was actually wrong (and why this took an evening to debug)

After PR #1936 landed the wiring (passing AWS.MM.SMS.OriginationNumber as a message attribute on every ProgressNotifier publish), SNS deliveries to phone subscribers were silently failing with the misleading carrier-level error:

"The provided number does not exist or does not belong to the account"

All the obvious causes were ruled out one at a time:

Hypothesis	Verdict
Credential aws.sms_origination_number not set in prod	False — verified via prod console
Wiring not deployed	False — 76f4da6 includes commit 79b2c36a3
Number not active / not in same region as SNS	False — +17626898865 is ACTIVE in us-west-2, same as SNS publishes
SMS sandbox still on	False — IsInSandbox: false in us-west-2
DefaultSenderID="OST" interfering	False — cleared, no change
Number not in a Pinpoint Pool	False — created pool, no change
SNS doesn't see the number	False — aws sns list-origination-numbers returns it as Active

Actual root cause: AWS introduced resource policies on Pinpoint SMS Voice v2 origination identities (rolled out in 2024). PSV2-acquired phone numbers default to having no resource policy, and SNS — even within the same AWS account — is not implicitly allowed to use them. The "does not exist" error is just AWS's misleading way of saying "you didn't grant SNS permission."

Fix: A single resource policy attached to the phone number:

```json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "AllowSnsToSendSms",
"Effect": "Allow",
"Principal": {"Service": "sns.amazonaws.com"},
"Action": "sms-voice:SendTextMessage",
"Resource": "arn:aws:sms-voice:us-west-2:186555151487:phone-number/phone-2b852695526a4f299aa8232ca53acd2e",
"Condition": {"StringEquals": {"aws:SourceAccount": "186555151487"}}
}]
}
```

Attached via `aws pinpoint-sms-voice-v2 put-resource-policy --region us-west-2 --resource-arn --policy file://policy.json`. (Note the CLI flag is `--policy`, not `--resource-policy` despite the API name.)

Reference: AWS re:Post — "The provided number does not exist or does not belong to the account" error

Other AWS account changes made during the debugging session (now in place)

• SNS SMS delivery status logging enabled (us-west-2): IAM role `SNSSMSDeliveryStatusLogging` created; SNS configured with `DeliveryStatusIAMRole` + `DeliveryStatusSuccessSamplingRate=100`. A scheduled Claude Code routine (trig_016ExpaDfkfPE3cr4miUA2H6) will open an issue on 2026-05-13 to dial the success-sampling rate down to 5% (failure logging stays at 100%).
• DeletionProtectionEnabled=true on phone number phone-2b852695526a4f299aa8232ca53acd2e — provisioning this number took 7 months and 7 TCR submissions.
• DefaultSenderID="OST" cleared from SNS account attributes — was a no-op for US destinations, but ruled out as a confounding factor.
• Pinpoint Pool pool-4370e8841b10453986b9be912c96c9df created during debugging — turned out NOT to be required (the resource policy was the actual fix). Harmless to leave; can be deleted via delete-pool if you prefer minimal resources.

Related

• SMS is not working #1216 (umbrella, can be closed once STOP/START spot-check passes)
• Remove admin-only gate on SMS subscription buttons #1892 (remove admin-only gate — now genuinely unblocked)
• Track AWS SMS opt-outs in app to avoid silently-dropped Notifications #1947 (track opt-outs in app — opened during this debugging session)
• PR Wire registered 10DLC origination number into SMS publishes #1936 (wiring — necessary but not sufficient on its own)

[moveson]

Final verification: STOP/START opt-out loop confirmed (2026-04-28)

After the keyword auto-responses on the pool were updated to match the TCR v7 attestation (HELP/STOP/START), the full inbound-keyword loop was tested end-to-end:

• Texted STOP → received the OST stop message → number +13038806481 appeared in the Default opt-out list (`describe-opted-out-numbers`) with EndUserOptedOut: true
• Fired a test event with the user opted out → email arrived as expected; no SMS arrived (AWS dropped at opt-out check); a corresponding entry appeared in sns/us-west-2/186555151487/DirectPublishToPhoneNumber/Failure
• Texted START → received the OST opt-in confirmation → number was removed from the opt-out list (`describe-opted-out-numbers` now returns ResourceNotFoundException)
• Fired one more test event → SMS arrived on the subscriber's phone

All six acceptance items now check out. Recommend closing this issue. The remaining open items in the SMS umbrella (#1216) are #1892 (remove admin-only gate so non-admin users can subscribe) and #1947 (track opt-outs in app to avoid silent-drop waste).

Important configuration notes captured during verification

The keyword auto-responses are configured on the pool pool-4370e8841b10453986b9be912c96c9df, not on the phone number directly. This is because once a phone number is associated with a pool, AWS rejects `put-keyword` on the phone number with `PHONE_NUMBER_ASSOCIATED_TO_POOL`. The pool is therefore load-bearing — do not delete it without first relocating the keyword config. This contradicts the earlier "pool can be deleted" note from a few comments up.

Configured keyword text matches TCR v7 attestation verbatim:

Keyword	Action	Message
HELP	AUTOMATIC_RESPONSE	OpenSplitTime: For help with SMS notifications, visit https://opensplittime.org/sms_info or email mark@opensplittime.org. Reply STOP to cancel messages.
STOP	OPT_OUT	OpenSplitTime: You have been unsubscribed from all SMS text message updates. Reply START to resubscribe. For help, visit https://opensplittime.org/sms_info
START	OPT_IN	OpenSplitTime: You have opted in to receive SMS text message updates from OpenSplitTime. Reply STOP to cancel. Reply HELP for help. Msg & data rates may apply.