Disclaimer
Reported to the affected company, one of the main banks in Chile, and since mitigated. The screenshots are real captures with hostnames, credentials, addresses and personal data blurred out. Identifiers in the text are placeholders. Validation was read only.
Introduction
On 30 September we pulled a production MySQL password out of a 500 page on a bank’s rewards site. The host it named didn’t resolve, so the credential looked dead on arrival. It wasn’t. The part of an RDS hostname shared by every instance in the same account and Region led us to a sibling: same password, still working, full admin over 428,068 identity records.
The leak
A rewards site belonging to one of the main banks in Chile had been returning a Laravel exception on every page for months. Not a maintenance notice: a full stack trace, printed because the database connection was failing. Laravel prints the arguments of every frame on the stack, and the frame that opens the database connection takes the connection array as its argument.
Host, schema, username, password. To anyone who asked.
The /api/* routes leaked a second schema through the same error. So we copied the credential and
went to connect, and the hostname did not resolve.
$ dig +short db-prd-mysql-5-pub.<account>.us-east-1.rds.amazonaws.com
(no answer)
Nothing came back. The application was failing the same way with getaddrinfo failed, which is why
every route returned 500: the server could not resolve its own database either.
The obvious reading is that somebody retired this database months ago, nobody noticed the site was dead, and the password belongs to a machine that no longer exists. We nearly accepted it.
What stopped us is that NXDOMAIN is not a security state. It is the absence of a record, and a deleted instance and a renamed one look identical from outside. DNS told us the name was gone. The password lives in the MySQL user table, not in DNS, so it had said nothing about that at all.
Sibling enumeration
Split an RDS endpoint at the dots and only the first two parts matter:
db-prd-mysql-5-pub . abcdefghijkl . us-east-1.rds.amazonaws.com
└ instance id └ fixed per account and Region
(abcdefghijkl is the placeholder AWS uses in its own documentation. The real one is twelve
lowercase alphanumerics and is redacted everywhere in this post.)
The first label is the instance identifier, typed by a human at create time. The middle label is the one that matters. AWS documents it plainly: it is generated per account and Region, and every instance in that Region shares it. Then the line that mattered here:
If you rename your DB instance, the endpoint is different but the fixed identifier is the same.
So a leaked endpoint is not one hostname. It is a namespace holding every database that account runs in that Region, and the only unknown is an identifier capped at 63 characters of letters, digits and hyphens, unique per account per Region. In practice people make it descriptive, because the console shows that name to whoever is on call at 3am.
db-prd-mysql-5-pub reads as inventory: environment, engine, index, accessibility flag. A 5
implies a 4 and a 6. So we held everything constant except the number:
for n in $(seq 1 12); do
host="db-prd-mysql-${n}-pub.<account>.us-east-1.rds.amazonaws.com"
printf '%-56s' "$host"; dig +short "$host"
done
Eleven returned nothing. The one ending in 8 came back with an address.
Resolving at all put it in the same account and Region. Resolving to a routable address said more:
a private instance answers inside RFC1918 space, so a public one was reachable from outside, which
the -pub had been advertising all along. And the whole search runs against AWS nameservers, so it
sends nothing to the bank.
What we didn’t know yet was whether the sibling held the same data or accepted the leaked credential.
Getting in
The first question people ask is whether the database sat behind the web server’s firewall. It did not, and we had already checked, because the web host was the obvious place to look for reuse.
The application server answers on 443 and closes everything else, so had the database lived behind that box the credential would have been worthless. But a managed database is not behind the application. RDS is a separate host, with its own address and its own security group. This one was public, so that firewall was never in the path:
┌─ 443 ─→ the rewards site leaks the credential
anyone, no auth ───┤
└─ 3306 ─→ the RDS sibling accepts it
Which is the whole reason walking the namespace was worth it: what we wanted was never reachable through the application, only beside it.
Then we tried the leaked credential against the sibling. It accepted the password and held the same application data.
Two lines of that output set the severity. USER() shows the address we connected from, which is
already bad. CURRENT_USER() shows MySQL matched the account at @%, so the grant is not pinned
to the application server, or a subnet, or a VPC. It accepts connections from anywhere, and we were standing
proof of it.
The grants were the second surprise. For an application whose job is reading a rewards table, the
account held privileges across *.* including CREATE USER and GRANT OPTION. We did not create
one, but a role that can delegate itself means rotating the leaked password alone would not
necessarily end an intrusion that had already used it.
Then we counted the signups table twice, a few minutes apart. It went from 428,068 to 428,069.
Not a forgotten snapshot, then: a live database still taking writes behind a storefront that had been erroring at every visitor for months. The website was dead. The data underneath it never was.
Across the three schemas: registrations holding national ID, passport, date of birth, email, phone and address; payment and card authorisation data; purchase history; stored customer session tokens; and partner OAuth secrets.
We read five registration rows to confirm the exposure, then stopped. Three of the five were not customers but staff, on company email domains, sitting in the same table as the public.
Timeline
Five days from report to fix, on something needing a credential rotation, a security group change and a grant rewrite. Good turnaround for a bank.
Takeaway
NXDOMAIN is not a fix. The hostname is the most disposable part of a cloud endpoint, while the account identifier wrapped around it survives renames and turns one leaked string into a namespace you can walk without touching the target. Before writing a secret off as stale, enumerate the siblings of the thing it names.
Fix order: rotate the credential, make the instance private, cut the grant to what the app actually uses, turn off debug output.
Until the next one, stay curious, stay ethical.