User agent format when connecting to s2s API about nxapi HOT 12 CLOSED

Hi! Nice to talk together! I have plans to develop imink for Splatoon 3. The information you put together is great.

And I have joined your Discord server.

from nxapi.

That's really weird. When did this start happening? I did update the user agent format in the last version, it should look like this now (from my server logs, the commit information shouldn't be included for release builds):

nxapi/1.2.0 (git 0e63020 main; node 16.14.2; darwin 19.6.0) nxapi-app (Chromium 102.0.5005.61; Electron 19.0.1)

The nxapi-app bit will be different from the command line or if anyone's using this as a library.

I'm using a development version but I'll check if it's working properly in the release version (maybe I broke something with Rollup?).

from nxapi.

@samuelthomas2774 The earliest instance of it was 2022-07-06 07:27:19 UTC, so just very recently. Recent logs show a mix of users making requests I'd estimate 1/3 of the time with nxapi/0.2 and 2/3 of the time with the nxapi/weird.numbers user agent. I'm guessing "1.2.0" isn't getting treated as a string or something and as a result is messing up??

I'm happy to provide you with a more full (sanitized, no IPs or info on other clients) log from the past ~24h if you want.

(Also an aside but jeez, you have a lot of users, just saying lol… doesn't look like it's just a couple haywire clients as has sometimes happened before with other projects using the API… now s2s users keep getting the "too many requests" error so I had to increase the # of global requests allowed per minute from 4 to 10 just now.)

from nxapi.

Just tested the release build of the app straight from GitHub, and also with some debug logs and it all seems to be sending properly formed user agents...

That should be sent from 78.32.231.153/2001:470:6d29:51:5472:173:9b2a:564e/2001:470:6d29:57:346c:b76d:9f81:3d1b (screenshot timestamp was 15:09:14 UTC).

Are you seeing any other versions reported? 0.2.0 is quite old now (and I didn't release that until I added an update check), plus there should be quite a few other versions between 0.2.0 and 1.2.0. I'll check older versions as well just in case it's any of those. I don't know why this would only be appearing today though, even if it is to do with the new format in 1.2.0, as that was released Saturday.

Sorry about this, those user agents are obviously very wrong so please do block them if it's causing issues.

from nxapi.

Yep, I see a line from 2001:470:6d29:57:346c:b76d:9f81:3d1b getting logged – says it's at 2022-07-06 15:08:18 so maybe my server is a little behind or something, lol, whatever, close enough.

I'm seeing pretty much only 0.2 and then all the weird lots-of-numbers ones in recent logs. Your request mentioned above is the only one made in the past hour since I opened this issue using the new parenthesis format.

Looks like it is actually only three users over and over sending the 0.2 user agent strings, with multiple requests every minute – I'll go ahead and block those IPs server-side since clearly something's going haywire there and they're not aware of what their client is doing.

FWIW there are older logs, e.g. from June where I see 1.1.0 and 0.3.0 (and 0.3… which is different?) showing up, and a couple 1.0.0s.

I don't have a ton of time to look into this now as I'm on vacation this week but I can confirm it's not on my end – the logging works just by using PHP to pull $_SERVER['HTTP_USER_AGENT'] and that wouldn't be screwing with the provided string.

from nxapi.

The earliest instance of it was 2022-07-06 07:27:19 UTC, so just very recently.

My bad, @samuelthomas2774, I was wrong about this. The earliest instance looks like it was:

"xxx.xxx.xxx.xxx", "2022-07-05 06:55:43", "nxapi/0.7.972239190788646", 200, "OK"

They started immediately after that – like, various different ones from multiple IPs, all starting that exact minute.

from nxapi.

Also an aside but jeez, you have a lot of users, just saying lol…

I'm not sure I do... GitHub doesn't show download stats for release asset downloads, but according to the repository traffic stats (from I think the last two weeks?) there's only been 5 unique visitors to the releases page, so if these are legitimate nxapi users they didn't download the app from here. npm download stats (cli-only/library use) are a bit higher but I think that includes lots of bots (e.g. for malware/vulnerability scanning?). I'd like to have my own analytics for actual app use but I also don't really want to be collecting anything, plus that doesn't really work as nxapi is at it's core just a library for the API stuff (the CLI and Electron app are just built on top of it).

Of course ideally I wouldn't be sending you/NexusMine anything as well, then this wouldn't ever be a problem. I'm still trying to figure out how gen_audio_h/gen_audio_h2 works (as in, looking at the code and pretending I might know what any of it means lol). I'm thinking it would be great to have a WebAssembly binary that (somehow?) calls the function, then we're not really sharing how it works and could still tie it to an API to stop Nintendo complaining, and prevent sending invalid tokens. (See #10.)

FWIW there are older logs, e.g. from June where I see 1.1.0 and 0.3.0 (and 0.3… which is different?) showing up, and a couple 1.0.0s.

0.3 on it's own isn't right... The version number comes from the package.json version field so should always be exactly one of these: https://www.npmjs.com/package/nxapi?activeTab=versions. (And pre-1.0.0 there's no Electron app so they should all be from npm.) The other versions sound right though.

The number of downloads I can see certainly doesn't match up with the request volume you're getting, so I have no idea where those requests are coming from.

Sorry this is taking up some of your holiday. If you want to send me some logs I'll have a look and see if I can figure out what this is, but comparing my download stats with the number of users I can see from just the partial IP addresses in the screenshot (plus having no idea how these user agents could be generated) I'm not sure these actually are nxapi users (but then I don't know why they'd be fake either). It might be worth having a look at the IP addresses, as e.g. they shouldn't be Tor or other TCP/HTTP proxy exits because nxapi uses node-fetch which by default can't use a proxy (not that I wouldn't want to support Tor users, just it would be suspicious as it shouldn't actually be possible now).

Edit: Those numbers do look like they're (in the higher range of) valid safe numbers in JavaScript, so even if not nxapi, I'd say these requests are probably coming from a Node.js program. Math.random() * 10000?

Edit: Just so you know, other than misusing nxapi as a library, the only things that should cause nxapi to request more tokens than normal are: launching web services in the Electron app (as those tokens can't really be cached regardless of how long they're valid for), the nxapi nso webservicetoken command, and the /webservice/:id/token endpoint of the API proxy server. They're all actions that the user must manually perform though. Everything else should cache tokens properly, mostly attempting to match Nintendo, so they are requested at most once every two hours, up to 3 times (once for coral auth, SplatNet 2 and NookLink). Since v1.2.0 nxapi will print a warning message with a stack trace if it's used as a library and a user agent string isn't set, which is why I changed the format.

from nxapi.

I've tested some older versions (with added debug logs) and they're all sending correct user agents.

nxapi/1.2.0 (node 16.14.2; win32 10.0.19043) nxapi-app (Chromium 102.0.5005.61; Electron 19.0.1)

nxapi/1.1.0

nxapi/0.3.0

nxapi/1.2.0 (node 16.15.0; win32 10.0.19043) nxapi-cli

They should show up in your logs between 15:15-15:33 UTC from 78.32.231.153/2001:470:6d29:51:f108:ec21:54a3:404b/2001:470:6d29:57:f108:ec21:54a3:404b (there should be 5 requests, I forgot to edit one version to log the user agent).

I'm very sure now there's no way those user agents could actually be generated by nxapi. Even if used as a library, the user agent will always start with nxapi/1.2.0 (...) (or end with nxapi/version before 1.2.0), dependents can only append to it (or nxapi prints a warning if they don't).

Looks like it is actually only three users over and over sending the 0.2 user agent strings

I was going to test 0.2.0 but tested 0.3.0 instead because it changes the X-ProductVersion header from 2.0.0 to 2.1.1. 2.0.0 is now being rejected by Nintendo, so those clients will never work now. nxapi itself doesn't retry anything automatically (except once after the token expires) so I don't know why they keep sending requests.

Edit: I am aware of one project, NSO-FriendStatus, that depends on nxapi (via the CLI). Possibly that could cause logins to be attempted every minute if it fails?

from nxapi.

Hi @frozenpandaman, are you still getting these randomised requests to your API? Still really confused by this 😕.

I've been working on some changes (haven't pushed anything yet) to try and prevent excessive API requests even if nxapi is used by someone in an automated script.

from nxapi.

Hi @frozenpandaman, is this still an issue?

Since v1.3.0 nxapi will refuse to authenticate (and thus possibly contact your API) more than 4 times per hour (e12bb36), and will also append unidentified-script to the user agent if it detects it's not running in a terminal (no TTY). Also nxapi now contacts my server to check certain settings (mainly version numbers to report to Nintendo, although there's also the ability for me to remotely disable using certain APIs including yours if necessary); from that I can see (in the last day) nxapi was used by at least 12 users, including myself. I'm happy to share these logs with you (addresses redacted) if you'd like to check any if any requests correlate with my logs. As these changes were made in v1.3.0 feel free to block all older versions on your end if you'd like.

Also just a few other updates:

• I've created a discussion with a few ideas for getting the f parameter generation working locally. I don't know if any of those could work as reverse engineering native code is way outside of my knowledge but I'm still trying anyway. #10
• #11, I haven't updated this since I mentioned you there a few days ago but mentioning it here anyway.
• Also I created a Discord server for this project + Nintendo's smart device apps in general, feel free to join if you want to discuss anything. I've also set up automatic notifications for updates to Nintendo's apps which you might find helpful. https://discord.com/invite/4D82rFkXRv

from nxapi.

@samuelthomas2774 Sorry for the late reply! Thanks so much for all your help here. I just checked the logs and the version numbers seem to now be working correctly. I'm seeing recent requests from versions 1.3.0 and 0.2 which is fine by me.

Thanks for the other updates as well, will be keeping my eye on that other discussion & I joined the server!

from nxapi.

Thanks for checking, it's good to hear that whatever was causing this is resolved now. 0.2 shouldn't be sent from nxapi but as long as it's not causing any issues...

I've seen that you've just switched to @JoneWang's imink API in splatnet2statink; I'll switch the default in nxapi to imink as well then, as the imink API seems to be more stable than flapg. I'll leave the option to use s2s+flapg until you shut down your API. (@JoneWang, my last comment applies to you as well - feel free to join my Discord server, and you might be interested in #10, and #11 if you're planning to add support for Splatoon 3 to imink.)

Edit: @JoneWang, I just saw your tweet - this looks amazing, I'd love to hear more about how this is set up!

from nxapi.