When our sign-in links lost their magic

Much to our surprise, and contrary to what the logs showed, our customer was not able to sign in to a recently launched web portal. The authentication flow is implemented using “magic links”. Users enter their email, receive a message that includes an individualized, one-time link that they can exchange for a valid session in their browser. Except in this case the link did not work.

We went through logs and built test setups, trying to reproduce and fix this issue. The somewhat unsatisfying outcome is: we never found the root cause of why the link failed. We did, however, fix the issue. In this blog post we are going to share the implementation that fixed magic links being invalid for our customer.

Let’s look at the steps involved for a user to sign in using magic links:

1. On the website, enter their email address
2. Open inbox and find the mail sent by the server
3. In the email click the link and be signed in

How magic links flows work

To make the experience of switching between the website and the email inbox as seamless as possible, we want the link from the email to “magically” sign users into the website. To make this work we have to bend the rules of how HTTP and browsers work.

Bending the Rules of HTTP Link to this headline

There are a couple of different ways a browser can request and send data to a server. The most common way is a GET request. These requests usually read data from the server. POST requests are commonly used to write data to the server. Both methods can send data to the server, but there are differences in how the data is handled and what it should be used for.

Data sent via GET requests is:

• limited in size (theoretically)
• visible to the browser and its history
• can theoretically be bookmarked
• supposed to be idempotent (meaning you can send it to the server as often as you want and it won’t update data on the server)

Data sent via POST requests is:

• unlimited in size (theoretically)
• isn’t stored in browser history (and is not included in the URL)
• cannot be bookmarked
• may alter state on the server (so it can create or update documents and do so as often as the request is sent)

There’s another distinction between the two: POST requests cannot be triggered by an ordinary link. To send a post request to a server using the browser you would typically use a form element and confirm the data with a submit button.

During the verification of a sign-in token the token is “consumed”. Every token is valid for a set period of time and for one verification only. Sending the token to the server for verification is altering data. This should usually be done using a POST request, which cannot be triggered by a link. Using a POST request would break the otherwise very seamless user flow where, by clicking on a link, the user gets signed into the website or platform.

For this reason many authentication libraries use a GET request to send the sign-in token to the server. The server then processes the request and either signs in the user or redirects to the log-in form – which does not adhere to the usual rule of thumb.

What Went Wrong? Link to this headline

We don’t know exactly. According to our logs what was happening is this:

1. user requested a sign-in link
2. server sent out the sign-in link via email
3. after a couple of seconds the link would be opened and a session would be issued
4. and then, after some more time, the link would be opened again. At this point the token would be invalid as it had been used before and the sign-in would fail.

The unexpected part was the two requests trying to log in. We understand the second magic link opening was triggered by the user. But before they reached the portal the link had been invalidated by the first request which, apparently, was not initiated by the user.

Presumably some sort of system was automatically visiting the links that were being sent via email to our users. We never found out which system this was, so we can only speculate that it was some kind of link preview tool (or a security system like an antivirus or phishing protection software?).

How magic links were being consumed by an automated service before our users could open them

Balancing Usability, Security & Standard Compliance Link to this headline

The question at hand was: how can we prevent automated systems (bots) from consuming the one-time tokens while keeping the flow quick and elegant for our users?

A very obvious first measure was to implement the token verification as a POST request. This would fix the issue at hand as most (non-malicious) bot would refrain from sending POST requests to our service. Instead of using the built-in API route of the authentication library we were using, we implemented a small page that would come with a (pre-filled) HTML form element holding the token and a submit button labeled “Click to Sign In”.

<form method="POST">
  <input type="hidden" name="token" value={token} />
  <button type="submit">Click to Sign In</button>
</form>

In our test setup this actually worked: triggering previews and fetching the “magic link” content from the command line would not invalidate the token.
It did however break the “one-click” (or one-tap) workflow we had before.

How a newly introduced, interstitial page decides whether to automatically submit the magic link token

Letting the browser trigger an automated submission of the form would restore the previous experience and keep some bots from invalidating the token. Full link previews like the ones built into default iOS and macOS Mail clients, however, would still happily trigger the form submission and log the user into the “preview webview”. A subsequent click on the link would fail. The preview webview typically does not share its session or any other data with the corresponding mobile and desktop Safari (or any other browser).

There appears to be no reliable way to distinguish preview webview windows from regular windows using the headers that get sent along with the request. We decided to allow the one-click experience for only the browser that initiated the magic link flow. What we ended up doing is:

1. user requests a sign-in link and the server responds with a short-lived cookie
2. server sends out the sign-in link via email
3. bot opens the sign-in link but doesn’t have the cookie set, form won’t be submitted
4. user opens the sign-in link in their browser, does have the cookie set, form will submit automatically
5. server clears the temporary cookie, replacing it with a proper session cookie
6. user is authenticated

The automation does not work when the user opens the sign-in link from a different browser than the one they used to request the sign-in link. In this case they can still click the button manually, making the one-click flow a two-click flow, which seems like a fair trade-off.

We implemented a couple more guardrails. Individually they don’t reliably prevent automated token invalidation, but taken together they might make such cases unlikely.