Ryfujk

In the OAuth2 authentication process refresh tokens should be used only once. When the refresh_token is used it will return a new access_token and a new refresh_token.

This is also in the RFC6819 spec:

5.2.2.3. Refresh Token Rotation

Refresh token rotation is intended to automatically detect and
prevent attempts to use the same refresh token in parallel from
different apps/devices. This happens if a token gets stolen from the
client and is subsequently used by both the attacker and the
legitimate client. The basic idea is to change the refresh token
value with every refresh request in order to detect attempts to
obtain access tokens using old refresh tokens. Since the
authorization server cannot determine whether the attacker or the
legitimate client is trying to access, in case of such an access
attempt the valid refresh token and the access authorization
associated with it are both revoked.

The OAuth specification supports this measure in that the token's
response allows the authorization server to return a new refresh
token even for requests with grant type "refresh_token".

Note: This measure may cause problems in clustered environments,
since usage of the currently valid refresh token must be ensured. In
such an environment, other measures might be more appropriate.

This also allows the authentication server to recognize that a refresh_token was compromised, since it should only be used once. If a new renew request with the same refresh_token comes in the authentication server knows there is something fishy going on.

I wonder what is the proper way for the server to deal with such a scenario though? My guess would be that at least all the access_tokens for that particular client should be invalidated directly.

How do OAuth2 servers usually deal with multiple requests using the same refresh_token?

Access tokens invalidation

You cannot invalidate all access tokens for particular client_id. The client_id is usually bound to one application, but this app is used by more users. And even one user may use same app from different devices. The refresh token creates a kind of session - it must by unique for a particular app, user and device. Furthermore, a client may call refresh token with a narrower scope and in this case you do not want to invalidate the old access token with wider scope - the client may still use it.

With my limited experience, the OAuth servers do not invalidate the access tokens on the refresh token call. Access tokens are short lived, they just expire in time.

Refresh token multiple requests

See RFC6819 section 6: The authorization server MAY issue a new refresh token ... The authorization server MAY revoke the old refresh token after issuing a new refresh token to the client.
The spec. allow some freedom, so the implementations vary. The very secure implementation is to issue a new refresh token and invalidate the old one each time. But this makes troubles with concurrent calls (e.g. a multi threaded apps). So some servers has just one long lasting refresh tokens (simple implementation, less secure). Others keeps valid the old refresh token a short time (e.g. 2 min) after the new refresh token is issued.