Giter Club home page Giter Club logo

Comments (118)

lqhuang avatar lqhuang commented on June 26, 2024 10

Hi, guys. Firstly, thanks for your discussions @Mygod @madeye .

Iโ€™m not a expert on TCP/IP protocol. I'm interested in computer science, but I major in physics and write codes for science researches. Maybe, I could provide some views from a pure user rather than a developer.

When I heard @madeye want to deprecate obfuscation in the next release of shadowsocks-libev, I forked a 2.6.0 version branch into into my repositories immediately. I definitely understand why you think it's a dirty hack for given ISPs and breaks KISS principle. But as a user, I don't like the design of seperating obfuscation into simple-obfs. I'm a zealot of minimalist lifestyle and a practitioner for KISS principle, which means I will choose to enable TCP-BBR algorithm to speed up connections instead of choosing kcptun. Using kcptun will involve another thing, so it's not elegant for me.

The same problem is happen to obfuscation. But obfuscation is a little different with kcptun protocol,

On the server side, when the obfuscation is enabled, it still can handle normal protocol without obfuscating. So,

| Client-Obfs |   Server-Obfs  |  Working |
| No          |   Yes          |  Yes     |

If there is no necessary, clients don't need to make too many changes. For now, I'm glad to try this new feature just like enable UDP reply, Fast-open and One-time authentication. Yeah, I like obfuscation as a option --obfs <http|tls> much more than a new project called simple-obfs.

Imaging someone wants to use both kcptun and simple-obfs for multi-user configuration now, I think it's a terrible, trivial and dirty work to configure during different tools. It is not KISS at all.

Hence, please could you reconsider carefully whether to remove this new feature in the next release of shadowsocks-libev? Or just keep a fundanmental and recommended implementation of obfuscation in shadowsocks-libev, but also provide plugin supports to another obfuscations (like obfs4proxy). A much better solution also is welcome.

Anyway, I think your discussions are meaningful! It's a good try for further developments of shadowsocks protocol. Just keep it and make community of shadowsocks better.

These are my humble opinions. And I appreciate your time to read these. Thanks for your efforts.


PS: the following is not a technical content, just a little personal opinion. I'm sorry for that I'm too lazy to open a new email and send to @madeye.

I have some foreign friends who live and work in China. I volunteer to provide shadowsocks service to them for free. Some of them wanted to donate to me and I rufused all the time. Of course, I always got lots of thanks or became respected. "For the freedom of internet" someone said. They don't know who is the hero in the background. But, I know. I think all these honors should belong to you(@madeye) and @clowwindy. What's more, to everyone(shadowsocks.org members and others) who contributes to shadowsocks protocol and makes it better. That's how open source works!

Finally, Happy New Year! ๐ŸŽ‰

Cheers!

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024 2

@lqhuang Wow! It's really the longest feedback we have received in this issue tracker.

Thanks for your feedback and suggestion! I think one misunderstanding here is that we would remove simple-obfs from shadowsocks. Instead, we will keep providing simple-obfs as a plugin service of shadowsocks.

In other words, for the end users, nothing will change in the next release, although the command line may change to --plugin simple-obfs --plugin-args "--obfs http --obfs-host www.example.com".

The biggest advantage from this design is that we can have more and more plugin services designed for shadowsocks without modifying the upstream, e.g. shadowsocks-libev.

In the multi-user scenario, if you're using shadowsocks-manager, everything will work as usual. The server will automatically start the plugin service defined in the config or the command line. However, if you're with any third-party panel, changes may be required.

At last, KCPTUN, simple-obfs and all the future plugins are all optional. If you haven't experienced serious QoS, please don't enable them. Especially, as simple-obfs is actually a "dirty hack for given ISPs", it may introduce additional characteristics to shadowsocks protocol.

Thanks again for your feedback. Please keep watching our projects and send us more feedbacks!

from shadowsocks-org.

hellofwy avatar hellofwy commented on June 26, 2024 1

Since JSON have been used as config format, why not just use JSON to exchange config info? It's more readable.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024 1

@riobard The motivation of URL here is actually for sharing server info with mobile devices. As both Android and iOS supports "intents" with URL schema, a user can configure the app by clicking the URL or scan a QR code.

For example, with shadowsocks-android installed, any shadowsocks QR code scanned by any QR scanner will let system send a intent to shadowsocks and get the server configured automatically.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Should client enable the extension that's been configured when importing?

I prefer disabled by default, even the config is available, especially for the experimental obfuscating.

Should client export the extension configuration even if it's not enabled?

I think if the extension is configured, we may also export the settings.

from shadowsocks-org.

lqhuang avatar lqhuang commented on June 26, 2024

@madeye Thanks for your illustration! It's easier to understand now! Pherhaps I was misleaded by the manual page of simple-obfs. I admit that --obfs <http|tls> as a option is a simplest and most lazy way to play obfuscating tricks. I tried obfuscation and it seems to improve connections under some tests.

Thanks again for your replay. You're doing a great work:)

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

What about encoding a plugin's settings into one BASE64 string? Then the URL looks like ss://passwd:method-auth@ip:port/?plugin=BASE64_STRING. It will help to simplify the URL parsing.

To follow #28, we should remove kcp_port or plugin_port in the url.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Hmm. The executable name should be the same even in a cross platform scenario? The developer of plugins should take care of this, make sure no conflict between different plugins.

For platform specific settings, users may have to modify them after importing, e.g. interface name to bind. I think shadowsocks clients are only required to pass the arguments to the plugin, never parse them.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Should plugin provide configuration user interface? In that case plugin would need to do the argument parsing.

For example, I may want to use /data/data/com.github.shadowsocks.plugin.kcptun/lib/libkcptun.so as executable path. What about this?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

I prefer no specific UI for different plugins.

IMO, a shadowsocks-android plugin package should provide an interface to copy binaries from its data directory to shadowsocks-android's. In addition, the name of the executable should keep consistent on all the platforms. This name can also be passed from plugin package to shadowsocks-android.

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024

Want to pitch about the Matrix URIs scheme for the configuration, e.g.:

                                plugin  key=val   key=val
                                  |        |         |
                                ------ --------- ---------
   ss://passwd:method@host:port/kcptun;mode=fast;crypt=aes/obfs;host=foobar.io
   \_/  \___________/ \_______/ \________________________/ \_________________/
    |        |            |                 |                       |
 protocol  authority    remote            kcptun                simple-obfs

Which makes grouping more easily.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@em0minate We don't support multiple plugins though. Also:

...as matric URI parsing is not a feature of the web. This is just something which could have been.

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024
  • It's a proposal for configuration URI spec leaves room for flexibility, not limited by the client side implementation tho.
  • Matrix URLs doesn't makes into web standard we currently are, but it's still being valid URI form, only requires few parsing around URI's pathname, which more organized than plain query parameters.
    Angular 2 for one, its router mechanism build on top of matrix parameters. Also JAX-RS has good support too.

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024

Quick demo to illustrate the parsing:

const { parse } = require('url');



const uri = 'ss://passwd:method@hostname:7070/kcptun;mode=fast;crypt=aes/obfs;host=foobar.io';


const {      protocol, auth, hostname, port, pathname } = parse(uri);
console.info(protocol, auth, hostname, port);

const plugins = pathname
    .split('/')
    .filter(Boolean)
    .map(s => s.replace(';', '?').replace(/;/g, '&'))
    .map(s => parse(s, true))
    .map(({pathname, query}) => ({[pathname]: query}))
;

const options = Object.assign({}, ...plugins);
console.info(options);

play this around on runkit

from shadowsocks-org.

qiuyuzhou avatar qiuyuzhou commented on June 26, 2024

Hello,

I have added kcptun to ShadowsocksX-ng recently.Add kcptun configures to the qrcode like

ss://aes-256-cfb:[email protected]:4000?Remark=kcptun%20test&OTA=false&Kcptun=true&mode=normal&key=secrit&crypt=aes-128&datashard=10&parityshard=3&nocomp=false&mtu=1350

The codes has been push to develop branch on github, but has not released a new version.

Any suggestions?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

The porposal of @em0minate looks quite interesting. What do you think? @Mygod

@qiuyuzhou Thanks for the implementation, please wait for @Mygod's final decision.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

What if plugin uses non-standard CLI though? This would imply a change to SIP003.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod Could you elaborate more on "non-standard CLI"? Is there any example?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@madeye Maybe like:
some-plugin [file1] [file2] [file3]...?

Actually there's no constraint that one must use getopt-style command line options. One could even use inline bash scripts.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod OK, we should add a restriction in SIP003.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

And even if you do that, every client from now on will be forced to parse CLI to generate QR code. And what about illegal characters? (URL encode/decode?)

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod Hmm.. You're right.

What about this #27 (comment)? We encode the CLI in BASE64?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Although I have to say this is actually a more cross-platform solution. For example, options in Windows are by convention prepended by '/' while in getopt they are prepended by one or two -s. I suggest to pass PT-like configuration to client but that would require us to bomb SIP003.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Personally I prefer to URL/base64 encoding CLI if we're not bombing SIP003.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Unless a plugin is implemented as native Win32 application, I don't think we would have issues in CLI.

In addition, to support cross-platform, a SIP003 plugin should support POSIX getopt() instead of Win32 getopt().

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod Agreed.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Then we need to put those constraints in SIP003 as well.

Also do you think URL encode or base64 encode is better? I prefer to URL encode since there usually doesn't seem to be sensitive information going there (given the examples of kcptun and simple-http).

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod URL encode should be better, although we made a mistake last year to use BASE64 encode the whole URL. ๐Ÿ˜ณ

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@madeye It's okay since it's harder to extract password from the url. ๐Ÿ˜›

Another thing we need to consider about compatibility is how command line in Windows and bash in *nix handles special characters, e.g. --arg="argument 1"\ and' 2' is equivalent to --arg "argument 1 and 2". But I suspect the first one would work under Windows.

Also the client will still need to parse getopt themselves if they are going to provide an user interface. For example, see the platform for shadowsocks-android plugin: shadowsocks/shadowsocks-android@03a78f6

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@madeye What do you think?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod I think Windows can also handle --arg "argument 1 and 2"? And what about letting shadowsocks-windows do the tricks instead?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@wongsyrone Okay. What do you think?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod If the argument follows --arg1 "arg1 value" --arg2 "arv2 value" pattern, I think there shouldn't be any problem on Windows

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

No I meant the special characters handling.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod I think we can handle this in C#. If not, we can even put a bash from msys into shadowsocks-windows.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

We can certainly do that but don't you think there are going to be too many hacks/workarounds if we keep going down this path?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Hmm... So maybe #27 (comment) is better?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

We could use it if it were better documented. For example, how do we handle special characters in matrix URIs?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

What about encoding each plugin in the URL matrix?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Also matrix URL brings another issue for compatibility since we are currently using non web safe base64 encoding, which means [A-Za-z0-9+/] is the alphabet for base64.

I'm looking for better alternatives for now.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

The problem with matrix URIs is that it's not RFC3986-compatible (in plain English: matrix URIs are not URIs).

If the ss URI can be reconstructed, we should make it RFC3986-compatible:

SS-URI = "ss://" userinfo "@" hostname ":" port [ "/" ] [ "?" query ] [ "#" fragment ]
userinfo = websafe-base64-encode-utf8(method [ "-auth" ] ":" password)

The last / should be appended if query or fragment is present. Example: ss://[email protected]:8888/?plugin=url-encoded-plugin-argument-value&unsupported-arguments=should-be-ignored#Dummy+profile+name. This kind of URIs can be parsed by standard libraries provided by most languages.

For plugin argument, we can use the similar format as TOR_PT_SERVER_TRANSPORT_OPTIONS, which have the format like simple-obfs;obfs=http;obfs-host=www.baidu.com where

Colons, semicolons, equal signs and backslashes MUST be
escaped with a backslash.

What do you think?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod I'm OK with this. Let's add TOR_PT_SERVER_TRANSPORT_OPTIONS as SS_PLUGIN_OPTIONS in SIP003.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

And should we use simple-obfs to refer to the plugin instead?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

I prefer obfs-local as the plugin name, as obfs-local and obfs-server would accept different options.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Okay.

from shadowsocks-org.

fortuna avatar fortuna commented on June 26, 2024

Here is a pass-by suggestion: when you start thinking about configs, you should take into account multi-level nesting. The Matrix scheme allows for one level of nesting, but you may need other levels.

In the past I realized that a better solution is a nested structure such as JSON instead. However, simply URL-encoding JSON leads to long and terrible URLs. But you can come up with alternative encodings, such as JSURL. That library is javascript, but the code is small and could be reproduced.

Anyway, just my 2 cents. Sorry for parachuting into this conversation where I have no involvement. I've been interested in contributing to Shadowsocks and am taking a look around.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Plugins should avoid nesting and use format like a.b=1;a.c=2;d.e=3 for now. I don't think it will be encoded into ridiculously big URIs for now.

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024

I don't see Matrix URI is incompatible to URI, as code snippet showed on above, is exactly utilized the standard URL parsing functionality, in addition with extra parsing of pathname from:

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                                    href                                     โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ protocol โ”‚โ”‚   auth    โ”‚      host       โ”‚           path            โ”‚ hash  โ”‚
โ”‚          โ”‚โ”‚           โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค       โ”‚
โ”‚          โ”‚โ”‚           โ”‚ hostname โ”‚ port โ”‚ pathname โ”‚     search     โ”‚       โ”‚
โ”‚          โ”‚โ”‚           โ”‚          โ”‚      โ”‚          โ”œโ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค       โ”‚
โ”‚          โ”‚โ”‚           โ”‚          โ”‚      โ”‚          โ”‚ โ”‚    query     โ”‚       โ”‚
"  http:   // user:pass @ host.com : 8080   /p/a/t/h  ?  query=string   #hash "
โ”‚          โ”‚โ”‚           โ”‚          โ”‚      โ”‚          โ”‚ โ”‚              โ”‚       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
(all spaces in the "" line should be ignored -- they are purely for formatting)

On encoding issue, you could have two protocols as:

ss://passwd:method@host:port/kcptun;mode=fast;crypt=aes/obfs;host=foobar.io
ss-base64://cGFzc3dkOm1ldGhvZEBob3N0OnBvcnQva2NwdHVuO21vZGU9ZmFzdDtjcnlwdD1hZXMvb2Jmcztob3N0PWZvb2Jhci5pbw==

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@em0minate Our current approach is based on Matrix URI. I just checked again and we could use that syntax but that one doesn't make as much sense as putting it into query. Keep in mind SIP003 doesn't support multiple plugins.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@Mygod What's the plan to support both password and key in the websafe-base64 encoded userinfo field? Also, do we need a new scheme prefix because now we have three variants of SS URL?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Because in URL there is a field specifically for user info which is perfect for method and password. Using base64 prevents illegal characters in URL since we want the URL to be valid.

You are welcome to give compatibility suggestions here. I didn't think a lot about that.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

Thanks for the explanation! Now I see how we ended up here. I have the following suggestions:

  1. We should deprecate the strange "matrix" URI that is ss://[base64-encoded] form. It's not legal URI and it doesn't make any sense, as discussed in #50.
  2. We should not base64 encode the userinfo field either. RFC 3986 section 2 already has a standard way (percent-encoding) to represent special characters and this is universally supported.

It's still open question where and how to include password/key in the URL. RFC 3986 section 3.2.1 says password in plaintext is deprecated due to obvious security reasons.

Additionally, we should keep it in mind that it's possible that future versions of Shadowsocks might be able to support multiple users on the same port. If we use the username of userinfo to specify cipher name, we'll lose the ability to support multiuser in the future. Maybe we should leave the userinfo empty for now and instead put the cipher method in query string (e.g. /?cipher=aes-128-gcm)?

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

By the way RFC 3986 section 7.6 is a real concern. URL userinfo is illy-designed and a social engineering nightmare.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

To keep the density of the password or key in the URL, I think BASE64-URL is still the best choice.

To support multiple-user, if we follow the current approach that encoding both cipher and password in BASE64, we should be safe.

And surprisingly, with BASE64 encoded user info, we can even avoid semantic attacks...

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@madeye Can you explain a little bit about how the base64-encoded userinfo work with both password and key? I didn't find any detail above.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@riobard Still not defined... Currently, we only support password here.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@madeye I thought we only need base64 to encode random keys. If we only support passwords, wouldn't it be easier to just follow the original URL userinfo format?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

I thought we only need base64 to encode random keys.

It'd be doable.

If we only support passwords, wouldn't it be easier to just follow the original URL userinfo format?

Right. We just encode this part for future change.

I'm thinking of pattern like BASE64(cipher:password:<password>) and BASE64(cipher:key:<key>). Any suggestion?

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

How about this? For traditional password-based deployment and backward compatibility

ss://cipher:password@host:port/?extension_parameters

For new key-based deployment

ss://[username@]host:port/?cipher=aes-128-gcm&key=base64:[base64-encoded-key]

The [username@] is optional and will be used only when future versions support multiuser.

We can distinguish between the two because the new format does not have userinfo field.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Currently we are using userinfo to identify if a URL is a legacy pattern. https://github.com/shadowsocks/shadowsocks-android/blob/master/mobile/src/main/scala/com/github/shadowsocks/utils/Parser.scala#L36

Move cipher and key out of the userinfo would make things much more complicated.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024
  1. Encoding method hides unnecessary details that normal user doesn't need to know.
  2. I thought embedding key into URLs is not supported.
  3. I also thought multiuser isn't going to be supported.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Also, method-key combination is used as authentication information. I don't see the point of moving it to query parameters.

For matrix URIs, I have only seen it used in OpenWRT web front-end and nowhere else. And it doesn't make symatic sense either.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

Hmm, this looks more complicated than I originally thought. Let me think a bit more about the issue.

from shadowsocks-org.

fortuna avatar fortuna commented on June 26, 2024

+1 for using JSON (#27 (comment))

It's more readable, and can also be used for versioning:

{
  "v1": {
    ..fields for v1 configs...
  }
}

Parsing becomes very easy:

config = JSON.parse(input);
if ("v2" in config) {
  parseV2(config.v2);
} else if ("v1" in config) {
  parseV1(config.v1);
} else {
  throw "config version not supported";
}

To make the serialization concise, it's possible to use binary formats such as CBOR or BSON and base64 encode after that to make it url-safe.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@fortuna Yes, JSON strings should be much easier to parse. However, If we encode everything in a URL, we also lose all the semantic information.

Anyway, I think encoded JSON in a URL looks doable, if we don't really care about the semantic things here.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

I'm not sure. But if we really have to embed JSON into the URL, I'd suggest that we use Data URIs, something like

ss://host:port/?config=data:application/json;base64,[base64-encoded JSON]

It would also allow people to use other format e.g. YAML by

ss://host:port/?config=data:text/x-yaml;base64,[base64-encoded YAML]

But then the question becomes what would be an acceptable JSON format as there are multiple implementations with different features.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

Anyway the major concern of base64 in the URL is that it's not possible for a human to manually edit it (like change cipher/password) without using an external tool. Nor is it feasible to spot misspellings, which happen quite often.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@Mygod when you just setup the server and want to try if everything works?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@Mygod In that case it's only copy-n-paste? Then URLs are not really necessaryโ€ฆ they can just copy and paste JSON strings.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod Right, And most of time, URLs are for QR code only.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@riobard Yes, for server and PC clients, JSON configs work much better.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

OK, I need to clarify my understanding so we are on the same page:

  1. URLs are for human to read and edit, so they must be semantically correct and easy to change and spot typing errors.
  2. For users to easily setup mobile clients, QR codes are better because users can just take a photo. We don't need to worry about semantics. In this case, QR code containg base64-encoded JSON might be a better solution.

Is it correct?

from shadowsocks-org.

hellofwy avatar hellofwy commented on June 26, 2024

I think the initial design to use base64 has two purpose:

  1. The password can't been seen from the first sight.
  2. Since QR is used to exchange info, so put ss:// before the base64 part, make it more recognizable.

So it's just look like URL, but it's not URL.

The initial implement is simple, but now we have so many implements and the plugins and AEAD ciphers.
I think we should redesign the configuration format used for sharing.

If use JSON๏ผŒwe can extends it like this to add implement specific configurations:

{
    ...,
    "implement": "[email protected]",
    "implement-config": {
        ...
    }
}

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@hellofwy Maybe we should study the config format for major implementations first and see if there's any common pattern.

from shadowsocks-org.

hellofwy avatar hellofwy commented on June 26, 2024

And I think users should not edit the URLs or JSONs directly, since not everyone is family with the format.

But readability is more helpful.

The format is just use as exchange format, clients should convert it to it's local format. If they can't recognized the implement specific configurations, they should just prompt errors and show the JSON to users. Servers just provide someway to generate the configrations.

We can just focus on the standard exchange format.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

@madeye I see. So the "ss://" prefix is for that use case, and it does not matter if the URL is actually semantically correct, right?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@riobard Exactly.

from shadowsocks-org.

fortuna avatar fortuna commented on June 26, 2024

Re: #27 (comment)
@hellofwy , you can avoid a type + value field in json by making the field for the value be named after the type:

{
  "shadowsocks-libev": {
    ....shadowsocks-libev config...
  }
}

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024
  1. URI/URL is used for clicking;
  2. Most common type of data encoded in QR codes is URI/URLs;
  3. Using data URI scheme isn't acceptable because Android can't redirect this kind of URI to our app;
  4. Embedding data URI into ss URI is even uglier.

If there isn't any more argument, I think we can finalize this design.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

Right. I think we need to finalize this issue now. Shadowsocks-android has been blocked by this SIP for a while.

Since we cannot come up with a better approach than @Mygod's, let's follow this SIP for now.

If there is any further enhancement to the URL, we may take a new schema like ss2://, and then we won't worry about the backward compatibility.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

If there's no obviously better plan, it's better to stick with the original non-base64-encoded SS URL (ss://cipher:password@host:port) for now.

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024

As I've said:

ss://...
ss-base64://...

And if you really prefer JSON and base64 encoding, just pick JWT, then you could have:

ss-jwt://...

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@riobard One should never use ss URLs as the main configuration. They weren't and won't be designed for this.

@em0minate Yep. Feel free to add more URL ideas that we are NOT going to implement.

@madeye I don't see anything better. So let's keep using "ss://" { (base64-encoded) cipher ":" password } "@" host ":" port. Plugin and other future features are passed (URL-encoded) as URI query and name/comment (also URL-encoded) is passed as URI fragment.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

Why not? Multiple tools already do it that way, e.g. cow, shadowproxy.

Since we're not supporting keys in the URL anyway, I don't see why the original non-base64-encoded format would not work.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024
  1. Plain text password is not a recommended practice and has been deprecated (by RFC) IIRC;
  2. Base64 encoding these prevents user that doesn't know a lot about the details of the protocol changing them;
  3. Again URIs are mainly used for sharing, not configuring.

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024
  1. Agreed. But base64-encoding the userinfo does not solve this problem.
  2. I don't see why this is necessary.
  3. Please explain why.

Overall, I think if there's no obvious benefit, let's just keep using the original format since it's been adopted by other tools already.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024
  1. It doesn't. There's no real solution to this problem either.
  2. It's not. It's good to have.
  3. URIs isn't semantically clear nor compact nor easy to memorize for new users.

Another benefit is this representation is more compact if passwords have illegal characters in it in which case using URL encoded strings is less convenient. Also this format is already used in shadowsocks-android.

I agree that there's no significant benefit for using this one but there's no significant drawback either. And the old-fashioned plain text SS urls are never used and/or supported in most widespread clients. Why prioritize other projects over our own?

from shadowsocks-org.

riobard avatar riobard commented on June 26, 2024

Well, I was just hoping to avoid one more variant of URLsโ€ฆ Now we have three :|

from shadowsocks-org.

em0minate avatar em0minate commented on June 26, 2024

Yep. Feel free to add more URL ideas that we are NOT going to implement.

Oh, so it's not a proposal were one should bother to having discussion at the first place?

IMO, having rough formed configuration is BAD than to have it at all, especially when pulling off a spec.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@em0minate We're definitely not going to support 3 kinds of URLs, or infinite possibilities of URL variants. Thank you.

@madeye What do you think which one we should use?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod Let's keep using your purposed one in shadowsocks-android, since only shadowsocks-android supports SIP003 plugins for now.

The legacy URL should still be supported, as most of the known clients support it.

BTW, I only know two kinds of URLs. What's the third one?

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

@madeye He's talking about everything-plain-text URL used by other projects that supports Shadowsocks protocol.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod OK, that should not be a problem for us. At least all the official desktop and mobile clients support the legacy BASE64 URL.

from shadowsocks-org.

Mygod avatar Mygod commented on June 26, 2024

Should we also keep support for generating old format if the user wishes to use it?

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod I think it should be useful today.

from shadowsocks-org.

madeye avatar madeye commented on June 26, 2024

@Mygod What about generating legacy URL for configs without plugins? It makes sense and won't break backward compatibility.

from shadowsocks-org.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.