gogins / cloud-5 Goto Github PK

• Prove whether or not fresh build of Strudel has the piano roll working. It works.
• If not, debug it. If so, diff a fresh checkout of strudel with a fresh checkout of cloud-music/strudel. No difference.
• If no diff, diff the fresh checkout of strudel with cloud-music/strudel after patching. Nothing unexpected after pre-build patch.
• Build both and run. _Both work at first, but clearing caches and reloading a cloud-music piece makes its pianoroll go away.
• Then diff. Takes forever, but this is interesting -- why different dates?

michaelgogins@Michaels-MBP ~/temp % find . -name "canvas.node" -ls
73342649      976 -rwxr-xr-x    3 michaelgogins    staff              496744 Sep  7 19:55 ./strudel/node_modules/.pnpm/[email protected]/node_modules/canvas/build/Release/canvas.node
74410627      976 -rwxr-xr-x    1 michaelgogins    staff              496984 Sep 13 10:43 ./cloud-music/strudel/website/node_modules/.pnpm/[email protected]/node_modules/canvas/build/Release/canvas.node
74449643      976 -rwxr-xr-x    1 michaelgogins    staff              496840 Sep 13 10:46 ./cloud-music/strudel/node_modules/.pnpm/[email protected]/node_modules/canvas/build/Release/canvas.node

• If there's nothing obvious, try git bisect in cloud-music/strudel. Didn't seem to do anything, but when I was done, it was working again.
• If that's no help, debug it.

CsoundAC is designed to generate scores recursively outside of time, that is, transformations can move both forwards and backwards in time. The score must therefore be generated before it can be rendered.

This enhancement will enable CsoundAC to generate scores during rendering, normally in real time, thus enabling the composition of "always-on" or indefinitely playing pieces.

CsoundAC needs to support both terminating but repeating processes or cycles (as in Tidal Cycles and Strudel), and non-terminating processes (as used in my current cloud-music pieces). Ideally such cycles can be nested. Non-terminating processes can be implemented as cycles with an infinite duration.

I do not need any sort of live coding environment. Therefore, the new part of CsoundAC that will schedule and functionally compose cycles can probably be much simpler than the corresponding parts of Strudel. In fact, I should take care to make the scheduler as simple as possible.

In addition, the scheduler does not need (I think) to manage control channels, but only notes, thus the granularity of scheduling is the cycle duration. This can of course change from cycle to cycle. And there is a complicating case, where an indefinite process lasts much longer than any cycle. This case can be handled easily with a finite cycle that repeats an indefinite number of times, and whose generating process is not interrupted by the end of its cycle.

The SimpleWebServer app from the Play store will indeed run a local Web server than can also be accessed by Chrome running on the Chromebook. The Strudel REPL in cloud-5 runs just fine in this environment. Currently csound_loader.js flunks on Chromebooks because operating_system is unknown, but this is probably fixable.

https://github.com/google/swissgl

This might simplify running shader code in cloud-5.

This is what I have been missing. See https://web.dev/articles/custom-elements-best-practices.

<cloud5-score-generator>
<cloud5-visuals-to-notes>
<cloud5-notes-to-visuals>
<cloud5-strudel>
<cloud5-log>
<cloud5-about>
<cloud5-controls>

and finally:

<cloud5-infrastructure>

This makes the most sense if custom elements can properly extend other custom elements. In that case, there can be a JavaScript library that defines the base custom elements, and then the user will extend those base custom elements. I'm not doing this.

Alternatively, and perhaps more simply, the custom elements can simply accept hooks. That's what I'm doing.

To dos:

• The controls for the piano roll are not working. Actually they are, but they need to be faster and smoother and have a way to re-center the piano roll. I simply made the trackball move faster. I am not implementing re-centering.
• Strudel should be a full overlay. Not doing at this time.
• The log overlay is not working.
• Provide getters and setters for properties that are overlays or that have back references.
• There are differences in styling between Chrome and NW.js that should not be.
• The piano roll is not resizing. Fixed by binding this to onResize.
• Add functions to create dat.gui menu to <csound5-piece>.
• Remove all shadow root code, it makes things more complicated than necessary, and is not needed for efficiency or encapsulation in this particular application.
• Show About overlay on startup. Not doing, the shader is what should show by default.
• Get shader working with https://github.com/google/swissgl.
• Research how to properly implement overlays/tabs.
• Research how Strudel REPLs can be instantiated. Too hard to figure out, so I just stuck in the existing Strudel REPL as IFrame (for now).
• Better names for user contributions (addons).
• Port my old shader code to a new <cloud5-shadertoy> custom element for adapting shaders from ShaderToy.
• Get audio visualization working.
• Get visual music working.
• Use arrow functions (i.e. lambdas), not closures or .bind(this), for all callbacks, etc.
• Move all hardcoded styling into cloud-5.css.
• Update the About overlays in all tutorial pieces.
• Refactor tutorial pieces.
• Hide buttons in the piece that are not used. Done by hiding all buttons for optional components by default, then those optional custom elements that are actually used make their buttons visible.
• Get cloud-5.js into the automatically generated documentation. I can do that, but it works very poorly, doesn't pick up class properties, doesn't combine different files well, etc. So I'm putting this off.
• Move the paper into the cloud-5 repository, and link to the pdf from the README. I'm only including the .pdf.

https://github.com/dbkaplun/driftless/blob/master/README.md

I think that Strudel uses this general idea.

• Introducing myself.
• A piece of interactive visual music based on cloud-5.
• A demonstration of installation and usage (download, unzip, run, perform).
• A piece of fixed medium music, one of the good ones if I can't come up with something new.
• A demonstration of algorithmic composition, diving into the interactive piece.
• Questions.

Comment out pieces that don't bear repeated listening.

• No popup on Cloud Music No. 4.
• Fix Cloud Music No. 6 -- change csoundn to csoundm.
• Fix Cloud Music No. 7 -- change csoundn to csoundm.
• Fix Cloud Music No. 8 -- change csoundn to csoundm.
• Maybe 6, 7, and 8 are too similar, if so remove one or two. Removed 7.
• Make dat.gui parameters behavior more usable, see below.
• Update Cloud Music No. 1 with new UI.
• Update Cloud Music No. 9 with new UI.
• Update csound-ac (a dependency of csound-wasm).
• Update csound-wasm.
• Update Strudel.
• Update gogins.github.io
• Share on Discord.

Currently the CsoundAC patterns work on "note" values in all Patterns, but the question arises: should they only work the Note Pattern, i.e. when onTrigger is called? In that case, would the functions still compose?

The current implementation works, but is not always intuitive. That's why notes on the piano roll (which are only triggered notes) sometimes start jumping around when CsoundAC Patterns are applied.

I think I have to create a whole new set of functions to test this. Or maybe both sets should be implemented.

• Clean up the pianoroll note buffer so it doesn't just become a memory leak.
• If possible, get the buffer to work also with non-generative Patterns. I got it to work either for regular Patterns, or else for generative Patterns, but not for both at the same time. Good enough for now.
• Put in a patch to make this work.
• Improve diagnostics, especially the sync diagnostics.
• Refactor StatefulPatterns into its own module that does not need CsoundAC or Csound.
• Clean up csoundac.mjs and improve comments.
• Profile and, if possible, improve performance. There still seems to be a buildup of haps in the buffer. Maybe I should not try to make this usable for non-generative Patterns, or maybe I just need to filter the haps some more.

• Write diagnostic messages that enable proving or disproving that StatefulPatterns is working.
• Try a new callback similar to triggers. I don't see how to do that; they put in superdough, but it's still wired into WebAudio.
• Reduce patches of Strudel to a minimum. I think I have done that. It's still a hack. I need to talk to @felixroos.

I still have not got the stateful patterns correct. In http://localhost:8000/csoundac_example_03_acCT.html if the output is piano the music sounds correct, but the pianoroll is flattened and jumping. If the output is csoundn, the pianoroll shows up, but it only shows the haps from Logistic, and that is what csoundn plays, ignoring all the chord stuff that sounded like it was working with piano.

There are several versions of this utility... too many. The problem is here:

/**
 * There is an issue on Android in that csound may be undefined when the page is 
 * first rendered, and be defined only when the user plays the piece.
 */
var load_csound = async function(csound_message_callback_) {
    let operating_system = get_operating_system();
    if (operating_system === "Android" && typeof csound === 'undefined') {
        csound_message_callback("Operating system is Android, but Csound is not yet defined.\n");
        // On Android, Csound uses only stdout for messages; this becomes 
        // console.log, so we assign our "csound message callback" to 
        // console.log.
        return;
    }

I've made a variant, csoundloader.js, that omits this case, and should just proceed to load the WebAssembly build of Csound.

The objective is to build a zip file that contains the complete cloud-5 directory, and release that in this repository.

• Renamedist to cloud-5.
• Add a make zip script to package.json.
• Test and proofread all example pieces, especially the About panes.
• Improve the piano roll displays. Active notes should be more luminous, bars should be thicker.
• Replace the Strudel favicon with my own. This will be 16, 32, and 48 pixels square. I created the icons and the manifest but it doesn't work!
• Release! as beta... full release will come when patches are none or few.
• Notify various people.

This might apply to ChordPatterns, ScalePatterns, and PitvPatterns.

The current chord would be assigned to a prior chord before undergoing a transformation.

Then, in the *Patterns class, there could be e.g. an option to derive a voiced chord as the closest voiceleading from the prior chord. For example, in addition to ChordPatterns.acT there would be a ChordPatterns.acTL. Or, there could be a flag to control this.

It might be that all the stateful patterns classes would have both a voiced and an unvoiced chord (and a voiced and an unvoiced prior chord) to facilitate this.

This would complete the project of getting the most important parts of the CsoundAC chord spaces and scales code working in the Strudel context.

Refactor and improve the cloud-music repository so that it can serve as my primary computer music playpen. Pros:

• Completely platform-independent.
• No external dependencies.
• Download the repository and you can get right to work.
• I can write music both for the cloud and for fixed media in the same environment.

Cons:

• Not quite as fast as native Csound.
• No native plugins, especially native vst3 plugins.
• csound-wasm requires maintenance.
• Limited to real-time performance.

Questions:

1. Can csound-wasm render soundfiles to local storage? This is actually the most important question. Currently, because Strudel uses real-time WebAudio, it's not possible to render a piece that uses Strudel to a soundfile. Possibly there is a workaround by getting the real-time audio to also go to a soundfile. No, this does not look easy. It can be done on the server side using the fs module, but cloud-music is completely on the browser side. Possibly it can be done using the Emscripten filesystem APIs and an IndexedDB. I will look at the canonical Csound's WASM build for clues. Looks complex, don't want to write it or maintain it. Should be able to use a screen recorder extension for the browser.
2. Can csound-wasm open native shared libraries? No, and there are no plans for that, although it would be possible along the lines of JNI or CFFI. That would of course open security holes in the sandbox.

Tasks:

• Make cloud-music into an npm module. Actually, I'm not making a new module, but I am adopting the npm system for adding dependencies and building with vite.
• Complete the integration of CsoundAC into Strudel; this must be done without complicating the maintenance of Strudel. Done using a Git submodule for Strudel and patching it with scripts.
• Make a fresh clone of cloud-music and then try to get it up and running, fixing problems as they arise.
• Implement some sort of render to file facility for cloud-music pieces.
• Add an npm package script to deploy dist as docs to GitHub.

• https://github.com/DISTRHO/Cardinal is the repository.
• https://github.com/DISTRHO/Cardinal/tree/main/src/emscripten is just a demo, no documentation or build configuration.

It seems import("./MyModule.mjs") works if "./" coincides with the Web server root; otherwise, not. For my GitHub pages, the Web server root is "gogins.github.io/". This is a serious problem.

This would make using a cloud-music repository as an actual computer music studio much more attractive.

Standard minimalist techniques of musical composition can be factored into classes or functions. I have done some of this before. Maybe I can use what I did, maybe I can add to it, or I may need to re-implement this.

• Cell generators.
• Sample from collections of cells.
• Convolve cells.
• Augment and diminish the duration of cells.
• Cells containing scales, chord, or neo-Riemannian operations should apply them to all sounding notes.
• Cells could be of PITV sequences.

piano is in sync with csoundn when different outputs are stacked, but piano does not show up in pianoroll. I think this is caused by how I color the notes, they sound correct.

• 200 word bio.
• Program notes.
• The piece itself - interactive visual music using cloud-5.

I'm using the full kit here, pieces in HTML5 with other languages embedded, possibly including Strudel. The objective is to make all new pieces using a recent older piece as a template, regardless of the target platform, i.e. browser vs. NW.js, with as many resources as make sense already embedded in the piece.

I will create several templates by updating the best pieces, all in the cloud-5/public directory:

• Poustinia-v5e.html, generates a score using CsoundAC and both plays and renders it using native plugins, with a piano roll score. Runs only in NW.js.
• cloud_music_no_1.html, generates an animated visual using GLSL, samples it, and plays it using pure HTML. Runs both in browsers and in NW.js.
  • Add dat.gui controls for all instruments in the piece.
  • Fix levels.
  • Fix typography in About overlay.
• cloud_music_no_9.html, runs a Strudel patch that can be live coded, with an accompanying GLSL music visualization. Runs both in browsers and in NW.js.
  • Add dat.gui controls for all instruments in the piece.
  • Make the music visualization at least noticeable.
• Clean up typography on About overlay.
• Put in some way of re-arranging instrument definitions without editing the Csound orchestra. Not worth the effort.
• Put in folding hints to make editing easier.
• See if there's a better way to embed Strudel. Doesn't seem easy.

I shouldn't go crazy here.

I think the problem is that Haps from generative patterns that go to a default output are flat and jumping.

When this is working I may put it in a pull request to Strudel.

The idea is to simplify writing custom Patterns with arbitrary callables that deal only with primitive JavaScript types.

Inputs and outputs should include controls as well as pitch. Time should be available in the interface as linear time from the beginning of the performance, linear time from the beginning of the cycle, and time as a fraction of the cycle.

The underlying motivation is to use cloud-music and therefore Strudel as a platform for compositions using CsoundAC and /or that depend upon computational irreducibiity.

The licenses used in cloud-music may may or may not be compatible.

1. Csound, CsoundAC, and csound-wasm are LGPLv2.1 or later.
2. My pieces are Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
3. Felix Roos' patches are CC BY-NC-SA 4.0.
4. Strudel code and documentation are GNU Affero General Public License v3.0.

My patches and Roos' patches are compatible.

Useful with Chords, Scales, and PITV. Should take a Chord as a stateful parameter, and as a Pattern, emit a stack of the notes comprising the chord.

Everything is peachy keen and I have solved some problems that I thought would stop my show.

However, I don't always see my pieces on gogins.github.io unless I manually clear the browser cache and/or application memory from the developer console. This applies to the REPL iframe as well.

I need to:

• Completely clear the browser cache on first accessing gogins.github.io.
• Clear whatever it takes to see each piece when requested instead of the Strudel REPL.
• Clear whatever it takes to see the embedded REPL when it first is requested.

Shader program uniforms are not defined...

Iain Duncan has persuasive examples of using s7 in Max here.

Duncan's WASM repository is here.

This is research on resources to stimulate thinking for new pieces.

Put diagnostic messages in cloud-5 that will prove or disprove the correctness of scheduling in Strudel-based pieces. These will be marked sync so that the console can be filtered to show only these messages. Each trigger will increment and print a counter. Some of this already exists, but they need to be put in for all triggers, and to be in a uniform format.

For reasons yet unknown, acPVVV does not seem affected by add, so I added a patternifiable bass parameter and then it works.

Recent updates (CsoundAC? csound-wasm? Strudel?) have broken cloud-5.

I'm trying to see if one of my patches to cloud-5 is the problem:

michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/webaudio/webaudio.mjs ../cloud-5.save/strudel/packages/webaudio/webaudio.mjs 
// Duplicates of declaration/initialization here:
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/core/cyclist.mjs ../cloud-5.save/strudel/packages/core/cyclist.mjs 
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/core/pianoroll.mjs ../cloud-5.save/strudel/packages/core/pianoroll.mjs 
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/core/pattern.mjs ../cloud-5.save/strudel/packages/core/pattern.mjs
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/website/astro.config.mjs ../cloud-5.save/strudel/website/astro.config.mjs 
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/superdough/sampler.mjs ../cloud-5.save/strudel/packages/superdough/sampler.mjs 
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff strudel/packages/csound/index.mjs ../cloud-5.save/strudel/packages/csound/index.mjs 
michaelgogins@Michaels-MacBook-Pro ~/cloud-5 % code --diff cloud-5/strudel_repl.html ../cloud-5.save/cloud-5/strudel_repl.html 

I fixed the patch to cyclist.mjs, does that help?

Bring Strudel back into cloud-music, but only as a Web resource, not as a submodule or a branch of Strudel. If possible. And it should be loaded from the Web server filesystem, not from a remote URL.

• Implement stateful Patterns.
• Template strings don't work in the REPL, replace them.
• Get my csoundn Pattern to work from user code, or else patch csound/index.mjs with my code. The csoundnpattern is now registered in csoundac.mjs.
• Fix up csoundac.mjs to use stateful Patterns. Added eOP and O for octavewise revoicing.
• Fix sloppy timing. I think I have done this.
• Clean up diagnostics.
• Clean up comments.
• Create tests for all new Patterns. These can be Strudel patches that move through a sequence of Patterns.
• Clean up documentation.
• Log an issue with Strudel about Csound not working, with a pull request for a fix if possible. No need, Csound is working again.
• Log an issue with Strudel about the non-dominant trigger in csound with a minimal working example. This has been fixed by Roos.
• Present the StatefulPatterns class with a minimal working example in the discussion about state. Not doing this, as I can't actually get it to work.
• Log an issue with Strudel about pianoroll not working with this stuff with a minimal working example. Felix Roos explained this, I don't think this is easily solvable. It arises from the unpredictability of mixing the stateless chain of queries with stateful objects.

I checked out the Mercury Playground, it does not seem as capable as either Strudel or cloud-5.

The total-serialism code itself contains much duplication of what already exists elsewhere, but it looks well-written.

In particular "algorithmic methods" has stuff that I don't have and that looks useful.

God save me from re-inventing every wheel that rolls along.

The objective is to have a directory containing all of cloud-music that can simply be run from a thumb drive (or copied somewhere) and run in place, similar to a Flatpak installation in Linux. The package would be virtually the same as the current cloud-music dist directory, but with different pieces and an index.html page.

On a thumb drive, the computer music studio is pocketable and can be plugged into anything from a phone to a desktop workstation. Work becomes completely portable.

The actual plan now: rename cloud-music to cloud-5, take out all pieces that are not examples, and do all new compositions in gogins.github.io (the online cloud-music).

• Come up with a really good name for the thing. For now, "cloud-5."
• Clean up all examples to work with current libraries. There's still a problem with stateful generators and pianoroll, but that's a separate issue.
• Ensure each piece will always start with decent parameters.
• Create a new index.html that doesn't interfere with the cloud-music Web site, and is about how to use the thing.
• Verify that it runs on personal computers. Works on macOS, and on Ubuntu 24 (on Chrome only, not on Firefox).
• Verify that it runs on Android. Theminimal.http and cloud_music_no_1.http examples play, but `message.html1 does not, and there is an issue with the Csound loader on Android.
• Ditto for iPhone? No way to test this for now.
• Verify that it runs on Norns. Not doing, using Norns requires using a computer to code for and run Norns, there's no reason not to just use the computer to run cloud-5.

• Simple Node, duration from Score.
• Node with duration from Score, plus delay and extend.
• Node with sequence of generators.
• Node with duration from Node ("nesting").
• Stack.
• Stack with phasing.
• Logistic (uses state in Node to persist values of map).
• Simple transformation of Logistic. Seems to be a problem with modulations, but not with transformations or scheduling.
• Recursive Nodes with durations from Nodes.
• Import MIDI file, if possible. Needs much work. Maybe do later. For now, MIDI files can be included in HTML or JavaScript files in some encoded form.

Further testing will be done by trying to write a real composition.

• Reduce patches of Strudel code to the absolute minimum. That already seems to be the case.
• Determine if any of my patches are no longer necessary (run the Logistic example and Cloud Music No. 9). That does not seem to be the case.
• PR: API for embedded Strudel, as a separate module.
• PR: FIx multiple file separators.
• PR? StatefulPatterns. Not doing.

Switch to using the regular or "canonical" build of Csound for WebAssembly from the Csound repository. This can be done by just loading the file from a server, or from the Web server's public HTML filesystem (which is what I need, for stability).

See https://kunstmusik.github.io/icsc2022-csound-web/ and https://vlazzarini.github.io/vanilla/. The latter is more useful as it assumes less.

To load as an ECMAScript module:

<script type="module">
import csoundbrowser from 'https://cdn.jsdelivr.net/npm/@csound/[email protected]/+esm'
</script>

I don't know where the bug is. The "i" statements that go to Csound.InputMessage are correct, and they produce the intended result. But, when the invoked instrument instance runs, it only prints 0 for p1 and therefore nstrstr returns an empty string.

I suspect the problem is caused by routing a string to p6 with Strudel controls encoded as text. I will back that out and see if that helps. Yes, that fixed it so that's the cause.

Get this thing off the ground:

• Get csound-wasm working.
• Install csound-wasm.
• Do a proof of concept for generating an always-playing score, and using a shader to visualize the music.
• This in turn requires, for always-on pieces, some means of synchronizing score generation in segments or loops. It will not do to generate an infinitely long score in one go.
• Do a proof of concept for using a shader to generate a score.
• This in turn requires integrating chord spaces and scales while generating scores from sampled visuals.
• Provide examples with basic user controls, including at least audio level and reverb.
• Enable saving controls to local storage, restoring them from local storage, and restoring them from defaults. Also, copy to and from the system clipboard.
• Refactor code to preserve encapsulation. This can be done, I think, simply by using different <script> elements and commenting them. Actually, not doing this due to the mixture of languages (HTML, JavaScript, WebGL/GLSL, Csound. I simply put in a comment.
• Improve GitHub pages.
• Fill out user controls for Nos. 1 and 2.
• Fill out help, credits, and resources for Nos. 1 and 2.
• Change No. 1 to use tempo.
• Use only local resources from the server root.
• See if there is anything to do about the reverb slap in No. 1.
• Decent default parameters. They need to take effect the first time a piece is loaded.
• No. 2 sometimes hangs after a while. Can't reproduce in debugger.

Not sure if this is possible, but it sure would be great to have.

Add a pnpm run deploy option in package.json that will:

• Leave all .md and .html files in the cloud-music root directory (gogins.github.io). This allows adding pieces and notes to cloud-music that do not come from cloud-5.
• Remove the _astro directory and contents from cloud-music.
• Ditto for the assets directory.
• Recursively copy cloud-5/cloud-5/* to the cloud-music root directory.
• Copy 'cloud-5/cloud-5/index.htmltocloud-5.html` in cloud-music.
• Copy the CLOUD-MUSIC.md file in cloud-music to README.md.

• This facility should provide a button on the cloud-music menu that will start recording from fout.
• When the user clicks on that button again, the file recorded by fout will be automatically downloaded to the user's computer, and recording to the fout file will pause.
• There should be a short fadein and fadeout. Not doing at this time.
• The cloud-music piece will continue performing whether the file is downloaded or not.

With repeated starts and stops, the output soundfile will grow, but will not usually be as long as the actual performance.

See:

• Victor Lazzarini's tutorial.
• The code for that tutorial.
• The wasm/browser source code relevant to the above.
• A StackOverflow example.

I'm not sure I need all this. After all the soundfile already exists in MEMFS and I know its filepath.