Difficulty select should mirror local user's selected beatmap. That is, require the user to select the host's "suggestion" to be able to override difficulty. Only perform the override if local selected beatmap matches the beatmap being loaded.

Make it obvious what the lobby selected difficulty is set to, both so the user knows an override is occurring, and so the user can undo the override easily.