• Set up storybook in paranext-core
• Get a basic button story working

Make the extension logger better by either doing something similar to VSCode's Telemetry or make some system that prefixes logs with the extension name or something along those lines.

Depends on #50

We should use different tsconfig files with different path aliases on @process or something to allow us to split synchronous code between different modules. This would help to clean up WebSockets and allow us to get a cryptographically secure nonce more easily.

This was previously an issue about NormalModuleReplacementPlugin, but TJ changed it when he made a demo branch for similar work #190 main...fd04d8a most of the original info is the same, so it is preserved below. It would also be useful to add the tslint rule mentioned in 190 https://typescript-eslint.io/rules/no-restricted-imports

We should also investigate if you can exclude the process folder in the other path aliases like @extension-host to avoid duplicity and confusion.

--- Original ---

Currently, we are using the IgnorePlugin to split code between processes. This leads to messy dynamic imports in Factory files like NetworkConnectorFactory.ts. We should instead use NormalModuleReplacementPlugin, which will clean up imports and make it possible to split out code more easily. Also need to stop using conditional require in papi.ts.

Hopefully, this change will allow us to throw errors if webpack discovers that a process is using a file outside of its appropriate reach instead of silently ignoring the import like we are doing now. Also investigate if eslint allows you to show importing some regex as an error so you can show errors for importing in the wrong folders at write-time.

We also need to stop using globalThis for variables we need that differ depending on the process. Let's use NormalModuleReplacementPlugin instead. We can make an abstract class in the shared folder that gets extended in each process that needs it with the appropriate implementations and overrides to tweak the object as needed, then create an instance of that class and export it. Like in NetworkService for example.

Maybe we can make aliases @process for this specific process type and @processGroup for shared code between the process types like node and client. However, that does not solve the problem with extension host getting code from both node and client folders.

Also probably should make a small replacement to allow using crypto in node process and in browser without having to require crypto on the node side. See Util.ts newNonce

Provide a way for WebViews to access assets like javascript files, images, and such. They may need a function that gives them contextually appropriate URIs. VSCode example

Depends on #44, #50

Need to decide on if we will continue supporting UnsubscriberAsync or if we just want to give up on that complexity and just let people unsubscribe only after awaiting promises.

After that, need to add a Disposable class and interface so that we can dispose of things automatically through the PAPI instead of having everything keep track of a bunch of unsubscribers manually.

Investigate https://github.com/sillsdev/cra-craco-electron to determine if we want to restart our app on it.

• Cost from switching - how long will it take to switch now vs a year from now?
  • Investigate differences from CRA and see if we can adapt ERB
• Benefits from switching
• Drawbacks from switching

check x-platform builds through GHA

When doing some testing with the C# client after implementing events, an exception occurred that crashed the process:

13:34:33.952 > [dotnet data provider] stderr: System.ArgumentException: Unexpected message type: event
   at Paranext.DataProvider.Data.MessageConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in C:\Users\tj_co\source\repos\paranext-core\c-sharp\Data\MessageConverter.cs:line 34
   at System.Text.Json.Serialization.Converters.CastingConverter`2.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Paranext.DataProvider.Web.PapiClient.ReceiveMessage[TReturn]() in C:\Users\tj_co\source\repos\paranext-core\c-sharp\Web\PapiClient.cs:line 220
   at Paranext.DataProvider.Web.PapiClient.RegisterRequest(Enum`1 requestToHandle, Func`2 doStuff) in C:\Users\tj_co\source\repos\paranext-core\c-sharp\Web\PapiClient.cs:line 104
   at Paranext.DataProvider.Program.Main() in C:\Users\tj_co\source\repos\paranext-core\c-sharp\Program.cs:line 26
   at Paranext.DataProvider.Program.<Main>()

I believe the C# client is expecting a response message from the server immediately following its request message. When something that isn't a response comes in, bad things happen (in this case, an event came in which the C# client doesn't currently know how to handle. So the exception doesn't look like the problem I'm describing. But handling events in C# is in #28). However, I don't think it will be guaranteed in the future that a response will occur immediately after a request. As such, I propose we associate a request Id to a Task that request id represents so that, when a response comes in for that request Id, the Task can be resolved appropriately. Otherwise, we can always handle messages appropriately

• Get MUI
• Create a few basic components that wrap MUI components and use our own props (the MUI component props and such should not be exposed to the papi. It should be an implementation detail
  • Button
  • Input field
  • Switch (on/off)
  • Dropdown select (can't tell if there is one in mui. Maybe get from here if not https://react-select.com/home )
  • Slider
  • Others you think will be useful

Depends on #39

Our production version of storybook (static page that shows the components and their abilities) isn't working. If we ever need to make it work (it may be a convenient way to document and demonstrate to extension developers what components they can use), we need to fix the production webpack configuration in .storybook/main.ts.

The cost to fix the storybook publication build was too expensive so we decided to create a separate repo that would have all of the components and run storybook. This turned into a research task to understand what options are available for Figma and Storybook integration. Below is the link to the research results:

Figma and Storybook Integration Research

To test the production storybook, follow the steps here: https://storybook.js.org/docs/react/sharing/publish-storybook

Depends on #39

Add a way to get the object that extensions expose from their activate function. Also need information about what extensions are installed, active, etc.

VSCode extensions reference

Depends on: architecture decision

This would mean getting rid of returning unsubscribers from activate. Let's just add a contextual unsubscribers array like VSCode does.

Depends on: #50

Need to make a way for extensions to access papi in a controlled way. They need to be force-restricted to accessing only their own project folder and their own user folder and such. Maybe we can make a proxy for each extension that restricts certain functionality based on "attributes" attached to the properties of the services on the papi or something. We also need to consider how to do this for web views as well.

Make a note to extension developers that they can use this in place of fs.

It would be wise to investigate VSCode's context object they pass into the activate function and get an idea how that might be helpful to understand for our design.

Depends on #30, #56

As an extension developer, I want to be able to interact with APIs exposed on the C# side that interact with the platform in meaningful ways like getting the current scripture reference.

We need to make general requests from the C# side. For example, we will need to tell the PAPI which events to send us from the network assuming we end up doing that in #22 as planned (the actual work of registering for events would be in #58, not here). Or maybe we need to ask the frontend to give us the current scripture reference. Or something along these lines. Assuming we believe this will be a need, let's implement this.

Looks like we can adapt the code around here to make a new Request method.

As an architect I want to have a rough plan of how Paranext can be subdivided into different repositories.

Develop a plan for how to subdivide Paranext into distributed but related github repos. Setup any additional repos that we know we need now.

Figure out a good way to generate type code for different data types that are carried across the PAPI interface and used in C# and in JS

Empower extensions to use JSX, TypeScript, and npm imports by making a good extension template with webpack, babel, etc.

https://www.npmjs.com/package/rc-dock
https://github.com/tjcouch-sil/paranext-poc/tree/main/react-electron-poc

• Choose a framework that is extensible and allows theming
• Ability to add functionality to the docking framework

Determine how we can support the following:

• Tabs
• Draggable panels
• Drop panels by location of mouse similar to PT9
• Autohide
• Contextual area dropzones
• Buttons in tabs
• Buttons in tab groups
• Floating Windows
• Ability to move dockable floating window to a separate window

Any features that we would need to write a lot of custom code for we should create issues to handle in the future.

We should probably be using secure websockets wss

We want to swap to using Gitflow branching strategy on all of our repositories.

https://nvie.com/posts/a-successful-git-branching-model/

• Fix installer icon

• Add portable package to windows

  • In package.json
  • 

• Consider the fixes in 4 must-know tips for building cross platform Electron apps.

• Create full release 0.0.1

Depends on #6

Load up extension file in the extension host and let it run papi stuff

Ideas:

• Need file service
• In debug mode, watch unzipped extension files and reload as appropriate
• Webpack/babel

Enable objects that the PAPI controls for the extensions. We need to be able to handle things like webviews, resources, etc.

Depends on #30

As an extension developer, I want to be able to go somewhere not in the code to find all the features of the API so I can easily navigate around and get a detailed view of what I can do.

We need to find a way to generate an external document describing PAPI and possibly hook it up to our CICD process.

Looks like TypeDoc does what we want:
https://github.com/TypeStrong/typedoc
Examples:

• https://typedoc.org/example/
• https://ticlo.github.io/rc-dock/

There are others in case we want to try multiple.

Create a Scripture Reference Selector component (I think there is an Open Components Ecosystem one going around... Or you can start from the one from the POC. Also see Scripture Utils. Or make your own from scratch!)

In the past, Ian made this design.

Depends on #39

Make webviews good. Make them persist, deserializeable, have api that controls the webview and the webview panel, etc.

Depends on #45, #31

As a developer, I want to develop on a base of code projects including the relevant technologies for the software so I can develop the software.

• Combine ERB and Edge
• Add C# solution that integrates well with the Node stuff

Let's investigate if Electronite is a good fit for us.

• Is it maintained well?
• Is having Graphite support useful and worth it?
• Can it fit into our ecosystem?
• Is there an electronite builder tool?

https://github.com/unfoldingWord/electronite

We should think of a nice way to credit VSCode for all the inspiration and wisdom they have given us. Maybe add a shoutout to them in our readme.

Very open-ended.

• Need to make Uri into a class instead of just a string. Need to make it serializable and deserializable across the PAPI. Maybe adjust JSON.stringify and JSON.parse functions to allow this
• That way, we can change the util functions for Uris into static functions on the Uri class
• Also probably should change the name for the %appdata% folder from 'app' to 'user' or something to be less confusing

Create an internet access service on the papi, and shim out internet access so extension developers can't access the internet outside our avenues. Probably pretty much just perfectly mirror the fetch api (maybe even just pass the args straight to it).

Depends on #30

Closely related to #75, but not quite dependent on it.

As an architect I need to provide an example extension that does something useful.

Develop an extension that appears in the Paranext "Hello World" example.

Consider using Chris Klapp's Scripture viewer code to display Scripture text in a read only view.

Depends on #44, #59

Create a table component! Here are some React table components from which to choose (or choose a different one!):

https://adazzle.github.io/react-data-grid/#/common-features
https://react-table-library.com/?path=/docs/getting-started-installation--page
https://react-data-table-component.netlify.app/?path=/story/getting-started-intro--page
https://tanstack.com/table/v8
https://table-react-component.vercel.app/demo/animation
https://komarovalexander.github.io/ka-table/#/overview
https://primereact.org/datatable/
(https://mui.com/material-ui/react-table/#sorting-amp-selecting)

UX wants the table to have at least these features:

• Editable contents
• Multi-select(?)
• Sorting
• Filtering
• RTL support

Depends on #39

Add typescript member ordering rules.

https://typescript-eslint.io/rules/member-ordering/

See https://github.com/paranext/paranext/wiki/Code-Style-Guide#class-declaration-order

As a developer, I want to develop on a base of code projects including the relevant technologies for the software so I can develop the software.

• Communicate between the Electron Main process and the renderer process through WebSocket
• Spawn extension process manager from Electron main process
• Create C# project as a websocket client
• Communicate between extension process manager and C# project

We probably need the ability to listen for and to send events on the PAPI from C#. (If we want to wait until we actually need it, that's fine.)

Depends on #22

We need tests for C#, Node main process, Node renderer process, React, integration

The main process's web server, renderer process client, extension host client, and C# client all start at different times and don't wait on each other well. We need to fix this problem. Probably this task should mainly be about retrying WebSocket connections when they fail until they succeed. Hopefully, the activation event issue #51 may solve problems relating to actual commands not sending properly and such, but maybe this task can solve remaining issues if there are any after that point.

When you run npm run package, you get an installer at paranext-core\release\build that has a generic installer icon. Let's fix it to use our icon.

EDIT: Changed to just uploading icon files. Actual installer updating will happen in #14

Sometimes, when you resize tab groups just the right amount, the ... button that shows up when there are too many tabs to display flickers on and off the screen. This is probably some small bug in rc-dock that has to do with not testing the widths correctly according to how we're doing things. We should use the patch package on rc-dock.

We need to add a logging service so we stop using console.whatever all the time. Apparently we already have some kind of electron-specific logging library. Let's use it!

Some discussion on the topic: https://github.com/paranext/paranext-core/pull/24/files#r1103914506

One important place is in ServerNetworkConnector.onClientConnect

Currently, events on the WebSocket go around to all clients whether or not they need to listen to those WebSocket messages.

Make events more efficient by adding event registration. Each process that wants events needs to tell main that it wants those events.

We can do this by adding a routeEvent function in NetworkService similar to its routeRequest that tells the ServerNetworkConnector which clients to send event messages to. Then we can also add a new serverRequestHandlers entry in NetworkService that registers and unregisters processes from listening to events.

Split out from and depends on #22

Create a way to provide data from the PAPI.

• Data providers pull data from some data source (in whatever data shape) and create their own data in their extension
• PAPI-provided registration and unregistration functions and hooks to ingest the data
• Ingestors need to be able to specify selectors that limit the amount of data being provided to them
• Need setters to change the data
• PAPI should provide events that update the data on the frontend when it updates on the backend

Depends on #45

We need to be able to add a few command-line arguments when running Paranext:

• Log level
• Debug mode
• Don't run the extension host (dev environment only...?)
• Don't run the C# client (dev environment only...?)
• Don't run the renderer (headless)

Change launch.json configuration for .NET Core Launch so it runs an npm script like npm run start:data. That way, people can run the code with or without VSCode all from npm similar to how the Electron: Main configuration just runs npm run start. Make sure VSCode debugging is preserved.

We need a way to close Paranext well, which closes all websocket connections and processes cleanly.

A couple of Node closing libraries:
https://www.npmjs.com/package/node-cleanup - very old but seemingly quite flexible/powerful. Seems like it may run async closing code on process.exit(), but it seems unclear
https://www.npmjs.com/package/exit-hook - newer, maintained library that has an asynchronous closing code api but does not successfully await async closing code when using process.exit(). It has a gracefulExit() function to use instead

Related discussions with details about Node processes, WebSocket connections, and ideas for closing them well:
#36 (comment)
#36 (comment)

See #115

https://www.electronjs.org/docs/latest/tutorial/security#7-define-a-content-security-policy

We need to be able to wait on extensions to load when we run commands or do other dependent things on those extensions. Design from #180:

Every request polls a few times to see if the request becomes available at some point. Easy dependency resolution (unfortunately, this slows down failures on unavailable requests, but whatever. It also does not solve circular request dependencies, but who cares)

Depends on #30, #22

Create Events and Event Handler API for distribution over WebSocket