socketsupply / socket Goto Github PK

Part of the changes by @jwerle in this commit is incorrect.

The loadFileRequest: method doesn't work with http://10.0.0.80:5173/

The app will crash with:

http://10.0.0.80:5173/ is not a file URL

Tested on iOS 16

We have to ./bin/clean.sh every time we change runtime-preload.hh because the symbols are inlined into the objects that use them.

I have a couple questions here.

1. Suppose I have a SQLite database and a multi-window application. SQLite needs to be running in a single process and shared amongst the windows. I see system.send and system.receive but I'm not sure how to specify a single process just for SQLite.
2. Is it possible to manage windows more granularly? Suppose I want to open a separate "help" window. Ideally, I could launch a window with some args to the renderer or something.

I watched the talk show at NodeConf EU 2022 and loved the idea so I was here to try it out but was unable to get it set up on my windows machine.

I followed the instruction but the compilation was unsuccessful. I think i am short on some packages so can you please add some detail instruction and requirements to get it setup?.

not ok - the build tool failed to compile.

Thinking more about keeping all Socket APIs in window.parent, I think that this approach has a few cons:

• it's not safe to add our own methods and properties to windows.parent, because DOM spec could potentially add methods or properties with the same name
• keeping these properties in windows.parent could lead to a misconception that they are part of existing specs

I propose to polyfill only methods and properties that exist in the DOM spec and doesn't work in a web view. All other methods and properties should be moved to another safe namespace.

Synopsis

Theoretically, we can simplify our code by using POST method (and possibly RPC?) instead of (or additionally to) GET encoding/decoding URI. This way request body could be of any MIME type like application/json, text/plain, etc. This might be easier, faster and less error prone then using GET and URI parameters. Also, this can help with automatic generation of .d.ts for the Socket API client code or custom JS interfaces for the Socket Runtime.

Example

Lets take the window.setTitle IPC message for example. Currently the Socket IPC is a GET request which looks like this:

ipc://window.setTitle?index=1&window=0&value=Hello%20World!

Note: Hello%20World! is an encoded "Hello World!" string

Here we have to write the code in main.cc by hands, which could be error-prone, especially if there is more parameters with different types and there are multiple parameters which should use decodeURIComponent

Alternative solution is a POST message with a body with a certain content-type:

Request: ipc://window.setTitle
Method: POST
Content-Type: application/json

Body:

{
  "index": 1,
  "window": 0,
  "value": "Hello World!"
}

This is much easier to parse, but has a disadvantage: we need to have a JSON parser in the Socket Runtime.
This is an open question on how we can parse body, but one alternative is using INI parser we already use for the config. In that case request will look like this:

Request: ipc://window.setTitle
Method: POST
Content-Type: text/plain

Body:

index = 1
window = 0
value = "Hello World!"

Typing

We can define a schema to automatically generate TypeScript definitions and possibly C++ code as well. For example we have such schema.json file (could be INI, JSON Schema, etc., I use custom schema in this example):

{
  "window.setTitle": {
    "request": {
      "description": "Sets the window title"
      "properties": {
        "window" : {
          "description": "The window index"
          "optional": true,
          "default": 0,
          "type": "number"
        },
        "value": {
          "description": "The title to be set"
          "optional": false,
          "type": "string"
        }
      }
    }
  }
}

Same way the response could be specified here. This schema can help to generate types for the Socket API, which will include all the methods to use in the window__ipc.postMessage:

type Method =
  | 'window.getStatus'
  | 'window.getTitle'
  ...

and the inferred parameters for all the Method types via TypeScript's infer, so all of window.__ipc.postMessage, socket.ipc.postMessage (or socket.ipc.send, etc), socket.runtime.setTitle and their results are statically typed, have autocomplition and documentation.
This schema could generate the parts of main.cc and/or bridge.cc as well (before the push, w/ the header that this file is autogenerated)

Open questions (besides those I've mentioned earlier)

• How useful would it be to allow binary MIME types like application/octet-stream, image/gif, audio/mpeg, video/mpeg, etc.?
• Can we use custom content-type to add support for https://github.com/socketsupply/ltp in future?

There are two options for releasing on npm:

1. Include all the static libraries for all supported platforms in the npm module.
2. Post-install script to download the artifacts directly from someplace like S3.
3. Publish npm modules for each supported architecture and use a post-install script to npm i the module for the specific architecture.

Another consideration is ensuring developers have a comprehensive list of what they need. It's very possible that they npm install ssc just to have building an application fail since they don't have everything necessary.

At the moment when we make changes in this repo, we need to update io accordingly. socket-examples doesn't gaurantee that everything works fine. We need to use io to test socket and vice versa. Both should be in sync. If we make this change, we only need one PR in this repo to make sure all tests are passing. After that we can update socket-examples to use most recent versions of socket and io.

All other platforms use a static build of libuv when building an application using ssc. Windows still compiles everything from scratch.

Add support in Windows for building applications using static libraries instead of building from scratch.

Discuss the hypothetical native API implementation. Possible features a native API will support are:

• Register custom methods that are called when ipc:// is intercepted.
• Metrics gathering
• Event notifications (like when a new window is open)

Other considerations are if it's possible to lessen the cross-platform pain of native development; how/if allowing native code to run in iOS/Andriod is possible; or how do we allow native extensions to be loaded (e.g. they compile a shared library, and we add a JS API that directs where to load it from).

I don't see web app listed. It's the one thing missing.

Log all output to a known log file. Necessary for two reasons:

1. Tests in Windows can't read normally from stdout so the log file must be checked for success.
2. Useful for future bug reports so we can request the log file from the user.

Main question is, where is the log file supposed to go for each platform? I'm most concerned about iOS and Android.

See branch ipc-cleanup in ssc-test

Running npm test, I see

    • 'name' value is required in socket.ini +0ms

But the socket.ini file does have a name.

Current defaults are here:

Any advice on how to download and install app updates on desktop?

This commit broke the Windows CI 00e2e6d

https://github.com/socketsupply/socket/actions/runs/4063245908/jobs/6995257824

cc @jwerle @mribbons

Should have documentation about the file ssc.config — what fields are required, etc.

• WKWebKit (macOS): https://developer.apple.com/documentation/webkit/wkprocesspool
• WebKit2GTK (Linux): https://webkitgtk.org/reference/webkit2gtk/stable/enum.ProcessModel.html

Using this feature it would be possible to use native Web APIs like postMessage and MessageChannel for communication instead of communication via Socket Runtime IPC.

Edge on Windows doesn't need this

What OS are you using (uname -a, or Windows version)?

$ uname -a
Darwin Julians-MacBook-Pro.fritz.box 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64 x86_64

macOS 12.6

What programming language are you using (C/C++/Go/Rust)?

$ gcc --version
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ clang --version
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ go version
go version go1.18.4 darwin/amd64
$ rustc --version
zsh: command not found: rustc

What did you expect to see and what you saw instead?

$ cd desktop-node-react
$ ssc compile -r .
• warning! $CXX env var not set, assuming defaults +0ms
• preparing build for mac +9ms
• package prepared +198ms
• ran user build command +138ms
• compiled native binary +30027ms
• Creating Window#0 +2ms
• Creating Window#1 +166ms
• Showing Window#0 (seq=0) +44ms

Full output: https://gist.github.com/juliangruber/96d4d8dab602a2393e53de45c98b295e

Currently, I see {{title}} on my Android device for an app I am testing. It isn't defined in socket.ini so this makes sense. Ideally, it should fall back to a default value (name) or be a required field. name currently cannot have white spaces, while title is a friendly name that can include white space and be seen by humans. Which SHOULD require both IMO, or issue a warning that we're falling back to name.

cc @chicoxyzzy @heapwolf

1. ssc build creates a <builddir>/<os>/<name>-dev.app
2. ssc build --prod removes <builddir>/<os>/<name>-dev.app and creates a <builddir>/<os>/<name>.app (without -dev suffix)
3. ssc build removes <builddir>/<os>/<name>.app and creates a <builddir>/<os>/<name>-dev.app

That's a bug. We should have both.

I found this solution. It's in Rust, but it has comments. We can try it tauri-apps/wry@2eba8c9

Currently, socket exports window.__args and window.__ipc. If these are going to be considered public then we should document them.

Ref: #14

We need to remove automatic process.open when we create a new window and call it manually (for the main window in most cases).

Required changes:

• remove process.open IPC call from the window runtime preload
• add io.backendProcess.isOpen(): boolean and related method in Socket Runtime
• add io.backendProcess.open(force?: boolean): void and update related method in Socket Runtime. force will kill old process and open new one
• add io.backendProcess.kill(): void

We also need to improve windows management from io

• add io.runtime.navigate({ window, value }: { window: number, value: string }): void

Following the docs using a Mac M1 with svelte or react boilerplate I get an empty white screen when running npm start. Vanilla boilerplate works fine.

When inspecting the console (within the white screen) react boilerplate logs:
Not allowed to load local resource: "/path/appname/dist/mac/appname.app/Contents/Resources/index.js"

With svelte I also get an error, but different:
TypeError: 'text/css' is not a valid JavaScript MIME type.

All I do is

1. xcode-select --install (or install Xcode app).
2. curl -s -o- https://sockets.sh/sh | bash -s
3. npx create-socket-app svelte/react
4. npm start

I have tried

1. Reinstalling CommandLineTools and replacing it with Xcode app
2. On two different M1 machines with different OS versions etc. Same issue.

The ipc:// API needs to be fully documented. Especially since Windows requires using http:// instead of ipc://.

The socket-api repo does exist, but it serves as a complete porcelain API. Not just simple examples of how to use the plumbing IPC commands.

Note: After discussion, it was decided that the IPC XHR API would be kept private for the time being. Making it an officially supported API this early would bring a lot of pain in the future. This should still be documented, but the priority has dropped because of this.

Verify the minimal possible requirements for performing a build in Windows. Will start with a clean system and create a checklist of what's necessary so it can be documented online.

When will TypeScript definitions be added for the global APIs?

Do a comparison of ssc with https://github.com/neutralinojs/neutralinojs to see the difference in approach, functionality, etc.

Ref: https://github.com/socketsupply/qa/issues/66

I see this error when testing an application on Android:

VM4:61 Uncaught (in promise) TypeError: Could not determine UserMessageHandler.postMessage in Window
    at Object.value [as postMessage] (<anonymous>:61:13)
    at console.<computed> [as error] (index.js:56115:25)
    at index.js:52122:15

This is thrown in an "else" situation of this global VM script:

Object.defineProperties(window.__ipc, {                                   
  postMessage: {                                                          
    value: (...args) => {                                                 
      if (window?.webkit?.messageHandlers?.external?.postMessage) {       
        return window.webkit.messageHandlers.external.postMessage(...args)
      }                                                                   
      if (window?.chrome?.webview?.postMessage) {                         
        return window.chrome.webview.postMessage(...args)                 
      }                                                                   
      throw new TypeError(                                                
        'Could not determine UserMessageHandler.postMessage in Window'    
      )                                                                   
    },                                                                    
    enumerable: true                                                      
  }                                                                       
}) 

The Android implementation makes postMessage available on window.external, so this should be the fix:

    if (window?.external?.postMessage) {                         
        return window.external.postMessage(...args)                 
      }

As part of resolving android build for windows, I resolved an issue where libuv src wasn't copied to SOCKET_HOME in order to be available for Android NDK build.
This resulted in a silent drop out of ssc.exe during npm build.
I resolved the issue:
00e2e6d

The change has been reverted as part of a separate libuv issue, this single line can be undone without affecting windows build:
abade8b#diff-e9a96fb537f64b60e955f6d260d0d3c697cb45050752cf4d834833486b4d8530L88

-   Copy-Item -Path "$WORKING_BUILD_PATH\libuv\src" -Destination "$ASSET_PATH\uv\src" -Recurse -Force

Do I need to manually fix these every time I run ssc build --platform=ios or is there a setting in ssc.config that handles it?

edit: I'm using this as a workaround.

commit d6631891d2e0e305c7b4d29a5ee0b0c1945e0799
Author: Alec Larson <[email protected]>
Date:   Fri Dec 9 13:42:47 2022 -0500

    fix: use CFBundleIcons in iOS plist

diff --git a/src/cli/templates.hh b/src/cli/templates.hh
index faabbbe..89ff587 100644
--- a/src/cli/templates.hh
+++ b/src/cli/templates.hh
@@ -895,8 +895,20 @@ constexpr auto gXCodePlist = R"XML(<?xml version="1.0" encoding="UTF-8"?>
 <dict>
   <key>CFBundleIdentifier</key>
   <string>{{bundle_identifier}}</string>
-  <key>CFBundleIconFile</key>
-  <string>ui/icon.png</string>
+  <key>XCAssetsFolderPath</key>
+  <string>ui/Assets.xcassets</string>
+  <key>CFBundleIcons</key>
+  <dict>
+    <key>CFBundlePrimaryIcon</key>
+    <dict>
+      <key>CFBundleIconFiles</key>
+      <array>
+        <string>AppIcon</string>
+      </array>
+      <key>UIPrerenderedIcon</key>
+      <false/>
+    </dict>
+  </dict>
   <key>NSAppTransportSecurity</key>
   <dict>
     <key>NSAllowsArbitraryLoads</key>

Then I'm copying my Assets.xcassets folder into the dist/<platform>/ui folder in my build script.

When building with .\bin\install.ps1 -debug then running the tests no output is printed to the console. It's necessary to inspect the window to see the test results.

The current state of the code is functional, but slowing down development on adding new features. We'll need some dedicated time to do some cleanup and code restructuring. The primary goals are to:

1. Prevent issues like 78e2e2e from coming up that are easy to miss but difficult to find.
2. Make it easier for community contributors to help out.
3. Generalize how things are done.

This issue will track and discuss specifics.

This is a regression caused by d62d366

The #if DEBUG macro cannot be moved out of init.cc, as that's the only file that isn't pre-compiled by bin/install.sh.

I ran ssc init without realizing it would put files in the current directory (and ask for a project name like many other init commands do).

Running it in a folder with contents should ask the user before creating files.

We can add versioning to the config, so when people update the version of ssc and run it, we can know the version of config.ini is not compatible with the version of ssc and throw a readable error and maybe even provide an update script.

I think we can add an ungrouped field like this:

[config]
revision: 20220130

[build]
# ...

[meta]
# ...

Then,

• if ssc wants a newer version of config, we print something like Current version of config is outdated and not compatible with your version of 'ssc'
• if ssc version is too old, we print Current version of config is newer than installed 'ssc' can use. Please update your version of 'ssc'.

Automatic updates

We can add something like ssc config update to automatically update the config version

"Migrations" should provide the change for every field in such manner

// from non-versioned to 20220130

// [] build -> [build] script; identity
// [] env -> [build] env; identity
// [] executable -> x
// [] flags -> [build] flags; identity
// [] headless -> [build] headless; identity
// [] name -> [build] name; identity
// [] input -> [build] input; identity
// [] output -> [build] output; identity
// [] bundle_identifier -> [meta] bundle_identifier; identity
// [] copyright -> [meta] copyright; identity
// [] description -> [meta] description; identity
// [] file_limit -> [meta] file_limit; identity
// [] lang -> [meta] lang; identity
// [] maintainer -> [meta] maintainer; identity
// [] title -> [meta] title; identity
// [] version -> [meta] version; identity

Left value is a current value which we should move or delete
Right value is a new field and optional mapping function when we need to change a value (the mapping function isstd::identity by default)

This way we can create a chain of transformations for each config field for any old version to the latest version

Comments above reflect the recent changes made in #124

Currently we use https://github.com/socketsupply/ssc-test to add test script to the document's <head>

We can implement feature in the Socket Runtime, so developers don't have to add an extra dependancy. Also test files don't need to be in the production bundle. That should be implemented as a condition in the build script.

ssc build --test=test.js should:

• copy test files to the <builddir>/<os>/<name>-dev.app (that's the task for a build script, nothing to do here)
• inject the test.js script on ssc build -r --test=test.js or ssc run --test=test.js
• ...or warn when there's no test.js file in the resources directory

One thing I'm not sure is whether should a value for the --test option be mandatory (currently it's not)

Things like the native open file dialog would be great.

For some reason when building with ccache the binary doesn't end up in the build folder.

install.ps1 has been reverted as per:
abade8b

The now reverted change which removed $LIBUV_BUILD_TYPE from the Copy-Item commands was due to the fact that I observed libuv building directly to $WORKING_BUILD_PATH\libuv\build on two separate Windows systems.

I have done some investigation and can't see how the cmake --config switch does anything other than govern which config is used for build, it does not influence the output folder.

https://cmake.org/cmake/help/latest/manual/cmake.1.html:

--config <cfg>
For multi-configuration tools, choose configuration <cfg>.

Testing has been performed against libuv 1.44.2 (tag) and 1.x (branch).

cmake --version
cmake version 3.13.4

clang --version
clang version 15.0.1
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\Llvm\x64\bin

Note that with the now reverted code, install.ps1 operates as follows on my workstation:

ok - built libuv
Copy-Item : Cannot find path 'D:\code\socket\socket\build\libuv\build\Release\uv_a.lib' because it does not exist.
At D:\code\socket\socket\bin\install.ps1:49 char:3
+   Copy-Item "$WORKING_BUILD_PATH\libuv\build\$LIBUV_BUILD_TYPE\uv_a.l ...
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (D:\code\socket\...elease\uv_a.lib:String) [Copy-Item], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.CopyItemCommand

@trevnorris Can your versions and we can start comparing from there...

Some other requirements for having a stable nightly build:

• Guidelines on how dev branches land in master to prevent breaking changes.
• The website will need a place to host the nightly build.
• Will need an official list of supported platforms and architectures.
• Check that all tests pass before doing the nightly build.

Example: https://github.com/socketsupply/serverless-studio/actions/runs/3274841554/jobs/5388902283

• compiled native binary +51939ms
[20](https://github.com/socketsupply/serverless-studio/actions/runs/3274841554/jobs/5388902283#step:9:21)
• 'ssc compile' is deprecated, use 'ssc build' instead +0ms
[21](https://github.com/socketsupply/serverless-studio/actions/runs/3274841554/jobs/5388902283#step:9:22)
• executable not found: /home/runner/work/serverless-studio/serverless-studio/./dist/linux/serverless-studio-dev_v0.3.71-_amd64/opt/Serverless-Studio-dev/serverless-studio-dev +0ms
[22](https://github.com/socketsupply/serverless-studio/actions/runs/3274841554/jobs/5388902283#step:9:23)
Error: Process completed with exit code 1.

Log: https://github.com/socketsupply/socket/actions/runs/4090560259/jobs/7057647538

This error started to appear after a443134, but I don't see anything wrong there

Instead of building a binary using an application's files, bundle them and pass that to ssc to be run. The a few advantages of doing it this way are:

1. Building apps takes far less time
2. Users are not required to install build tools (especially painful to do on Windows)
3. Native addon development becomes possible by allowing the user to build a shared library then having the app include that using a JS API.

add ssc update command to cli to update the ssc command

@heapwolf started work implementing CoreWebView2CustomSchemeRegistration for custom schemas in e77bbc5, but was removed in 35467ec to get around compilation errors.

Investigate bringing back CoreWebView2CustomSchemeRegistration so Windows can use ipc:// instead of http:// to intercept requests.

Example implementation: https://github.com/MicrosoftEdge/WebView2Samples/blob/main/SampleApps/WebView2APISample/AppWindow.cpp#LL1208C28-L1208C71

I see in your documentation about security "NEVER try to build a browser. NEVER evaluate arbitrary code.".

But is there a way to securely display a webview that loads an external website? My use case is currently just for displaying a checkout process for purchasing the app. But I can also imagine wanting to display website previews, etc.

Thanks!