# The control plane is the only thing here worth indexing.
#
# User content lives on the content domain, and the Router Worker sends X-Robots-Tag: noindex with every
# response there (invariant I8). That is deliberate: a spammer's page must not inherit this domain's
# standing, and it is enforced by a header rather than by a file a crawler may ignore.

User-agent: *
Allow: /

# Nothing here is a page, and crawling it only wastes a budget that should go to the ones in the sitemap.
Disallow: /api/

# The account pages, which say nothing at all to a crawler: signed out they are a sign-in prompt, and
# signed in they are one person's own figures. Also marked noindex in the page itself (routes/meta.ts),
# because a disallowed path can still be indexed from a link elsewhere without ever being fetched.
Disallow: /account

# The command-line consent screen. Its own rule rather than a prefix of the one above, because it is not
# an account page: it is only ever reached from a URL a local process just printed, and its query string
# is a live sign-in challenge. A crawler following one would find a consent screen for a sign-in nobody
# started.
Disallow: /cli-auth

# The device approval screen, for the same reason: it is reached from a code printed in a terminal, and it
# takes a credential decision.
Disallow: /device

# The problem report page, opened from links a terminal, a chat or a crashed page printed. Its query string
# is one failure's code and trace id, and there is nothing on it worth a search result.
Disallow: /report

# Where an invitation link lands, and the sharpest case of the same argument: the query string is a live
# invitation to join somebody else's account, and a crawler following one out of an email would spend it.
Disallow: /invite

# The two pages the account emails link to — confirming an address, choosing a new password. Their query
# strings are one-use tokens, and a crawler following one out of a mailbox would be spending somebody's
# link. `/signin` and `/signup` are not here: they are pages people search for.
Disallow: /verify
Disallow: /reset

# Where the deletion mail's "Keep my account" link lands. Its query string is a one-use token too.
Disallow: /cancel-deletion

# Where a mail's unsubscribe link lands when somebody opens it. Its query string names an address, and a
# crawler pressing the button there would unsubscribe a person who never asked.
Disallow: /unsubscribe

# The code step of a sign-in for somebody with two-factor on. It is meaningless without the short-lived
# cookie the step before it set, so a crawler would only ever find "that took a while".
Disallow: /auth/2fa

# The three questions somebody is asked once after their first sign-in. Signed out it redirects, and
# signed in it is about one person — there is nothing on it a search result could usefully lead to.
Disallow: /welcome

# The not-found screen. Nothing links here and no visitor arrives on this path — the router renders it
# under whatever wrong path was typed, so the URL stays what the person can see and correct. The rule
# exists so the one path that could reach it directly is not worth a crawl either.
Disallow: /404

# Two sitemaps, because this host serves two builds. This one is the app's routes, written beside the
# router and held to it by a test. The second is emitted by the documentation build — one URL per prose
# page plus one per API reference tag — and it cannot live in the file above: that one is asserted to
# contain the app's routes and nothing else, which is what stops it drifting from the router.
#
# Multiple Sitemap lines are part of the format rather than a trick, and both are absolute because these
# lines are read independently of where the file was fetched from.
Sitemap: https://dropto.run/sitemap.xml
Sitemap: https://dropto.run/docs/sitemap.xml