atsushieno / augene-ng Goto Github PK

After compiling project into *.tracktionedit file, what often happens to me is that I check the results on Tracktion Waveform, try some other audio plugin instruments, and set up exactly the same thing on another AudioPluginHost to generate *.filtergraph file as a new input to augene. It is waste of time. If I could directly import back audio plugin settings from *.tracktionedit, then I could just keep manipulating the plugin UIs on Waveform, save the changes, and re-compile.

Having source and destination on the same file is worrisome, but it's not critical if (1) we store *.tacktionedit files on VCS (git etc.), and/or (2) we generate intermediate backup files when we interpret those plugin settings either from *.filtergraph or *.tracktionedit.

(based on atsushieno/augene#8)

On augene composition workflow, it has to load the entire tracktionedit file every time, and loading a song with sfz using sfizz can take quite long time. Even a single track with UI Standard Guitar along with a few effectors takes 3-4 seconds to load at augene-player. This cannot be waitable for even 10 tracks.

We would need certain shortcuts to avoid full reloading, like hot reloading audio plugin settings parts and compiling MIDI parts respectively. To achieve that however, the entire compilation tool should migrate from JVM augene-editor to C++ augene-player (which is not going to be just a player anymore then).

In the older .NET version it was simply an annoying idea, but now that augene-project is Kotlin MPP, it is possible that we would be able to build it for native. Current blocker is though google/ksp (used by kotracktive-project) that breaks at Kotlin Native builds (issue #5), so it is somewhat future task to achieve.

So far everything in this project assumes that the music source is MML. However it is not mandatory. We do have some tailored sysex and/or meta events that Midi2TracktionEditConverter treats as direct operations to manipulate .tracktionedit, but they can be also done in any SMF generators.

One thing to note though is that we will be switching to MIDI 2.0 UMPs at some stage and the target MML syntaxes (macros) will be of default-macro2.mml. Then SMF importer might become unusable or less useful.

Since we are the only one who can do it, why not!?

• we need audio plugins installed at certain location. Maybe studiorack helps it?
• we need augene-player run headlessly, to achieve (1) audio plugin lookup and cache as JUCE config settings, (2) "export" plugin helper MMLs as part of config files, and (3) "render" tracktionedit files into wav.
• extend #include processor to make it to load files from the config directory too.
• ffmpeg or whatever to (a) remove silence tail, and (b) convert to mp3 and/or flac.

If we enable nativeMain etc. in kotractive/build.gradle and run ./gradlew build, it results in weird failure at nativeTest, particularly at EditModelReaderTest.readTemplate(). The test failure log does not really tell anything (K/N does not give us a stacktrace), and due to #10 we cannot really debug the test on IDEA. I got lost.

I actually went beyond and inserted some debug println()s around, and figured out that PropertyInfo constructor does not receive valid propertyMetaType parameter. They were often passed from typeBoolean, typeInt, typeString etc. (defined in EditModelMetadata.kt). I suspect that those vals are not appropriately initialized on (and only on) Kotlin/Native. The constructor parameter is non-null, and those vals are NOT null, but what is actually passed to PropertyInfo() IS null.

It is most likely a bug in Kotlin/Native runtime, but we may be able to workaround the issue by changing the code structures (place those referent vals elsewhere e.g. inside some object).

We need "Exprort" feature on augene-player. At least to achieve CI builds of MP3 files from project sources.

Compose for Desktop supports platform installable package (.msi/.dmg/.deb/.rpm) as package Gradle target, but for now it is impossible. It is due to some complicated situation... package Gradle task works only with JDK 16 or later, so we can run it IF we use JDK 16. However Android build fails if JDK 16 is used. It seems an upstream JetBrains issue.

(inherited from atsushieno/augene#9)

This time I go forward and drafting specification on how we achieve support for automation tracks.

In *.tracktionedit files, an automation track is represented as an <AUTOMATIONCURVE> element in the target <PLUGIN> element within the <TRACK> element, and the <TRACK> element at the same time contains a corresponding <AUTOMATIONTRACK> element that specifies currentAutoParamPluginID attribute that indicates <PLUGIN>'s id and currentAutoParamTag attribute that indicates the <AUTOMATIONTRACK>'s paramID (which is also the actual parameter ID of the plugin).

The actual AUTOMATIONCURVE looks like:

      <AUTOMATIONCURVE paramID="0">
        <POINT t="1.25" v="0.304347813129425" c="0.0"/>
        <POINT t="1.5" v="0.717391312122345" c="0.0"/>
        <POINT t="5.5" v="0.239130437374115" c="0.0"/>
        <POINT t="6.25" v="0.739130437374115" c="0.0"/>
        <POINT t="8.75" v="0.260869562625885" c="0.0"/>
      </AUTOMATIONCURVE>

... and AUTONATIONTRACK looks like:

    <AUTOMATIONTRACK id="1004" colour="ffff0000" currentAutoParamPluginID="1136" currentAutoParamTag="0">
      <MACROPARAMETERS id="1010"/>
      <MODIFIERS/>
    </AUTOMATIONTRACK>

Since we human beings don't recognize plugin parameter IDs, we'll need some comprehensive mapping from each parameter name to the parameter ID. It is not available on AudioPluginHost, so we will have to create some mapping generator tool that can then be consumed by AugeneProject and AugeneModel in augene-project.

The most probable candidate MIDI messages that are mapped to automation parameters are NRPNs. But since any MIDI messages can be also consumed by the target MIDI plugin, occupying NRPNs is not a very safe idea. It should be at least an optional feature, and desirably there should be other supported mappings e.g. SysEx messages with certain manufacturer IDs.

Currently when we reference audio graphs using INSTRUMENTNAME, we have to specify ALL the relevant plugins within the *.filtergraph file i.e. it specifies both an instrument and effectors. That means, ALL the tracks that reference the audio graph by INSTRUMENTNAME will share the identical set of effectors as well. For example, sfzz-ui-metal-gtx.filtergraph currently specifies kpp distruction effector, which would be very specific for songs that want to use it. We will end up creating *.filtergraphs for each, only to differentiate effector settings.

That's not ideal. We should be able to indicate the instrument and effects by any combinations to not bloat the filtergraphs.

Idea: use ';' to split instruments into multiple IDs and combine them for each track.

When native build is enabled in both kotractive-project (considering that #5 is gone), native build still fails due to:

/sources/ktmidi/augene-ng/kotractive-project/kotractive/build/generated/ksp/nativeMain/kotlin/dev/atsushieno/kotractive/MetaTypeDataTypes.kt:10: error: undefined reference to 'kfun:dev.atsushieno.kotractive.DataTypes#<get-dataType>(){}dev.atsushieno.kotractive.DataType'

Build failure repro on native-issue-2 branch: https://github.com/atsushieno/augene-ng/runs/3775952797?check_suite_focus=true

Kotlin Slack community had no solution to this. https://kotlinlang.slack.com/archives/C013BA8EQSE/p1631002492073100

Currently in our main branch we use mugene-ng to generate MidiMusic i.e. MIDI 1.0 music data to further feed MidiToTracktionEditConverter. This ends up the input bound to various limitations such as 16 channels and various per-note features (controllers, pitch 7.9, per-note management etc.).

We are going to improve it. In midi2 branch we compile MMLs to Midi2Music. It's just started and not sure when it's verified to work, but it is a first step towards bigger improvements.

(copied from atsushieno/augene#1)

Right now, when we use this tool to define audio plugin graphs for each track, those plugins are recorded as bare filtergraph generated by AudioPluginHost. When we switch environment across platforms, those plugins become different (they would be AU on Mac, and they would be VST3 on Windows and/or Linux, or LV2 on Linux). Since my secondary purpose of this project is to support AAP, it has to be also portable to Android too.

Since they don't share the same set of plugins, it is song author's responsibility to use the set of common audio plugins. But even if they use only such plugins, those filtergraph files from AudioPluginHost is platform dependent.

Maybe there should be some kind of mapping aliases, or at least per-platform filtergraphs. But the fundamental problem lying around here is, what is an "identical" plugin? Some possible usage scenarios:

• I use Collective on Linux, Mac, and Windows. They are all VST2, and portable across those desktop. Are they "identical" ? They should be.
• I use different versions of Collective on those platforms. Are they identical? They probably still should be.
• I use ADLplug AU on Mac, and ADLplug LV2 on Linux. Are they identical? Argurable.
  • Their behavior would be almost identical. But they have different unique IDs.
  • Also, they wouldn't share the same "state" binary blob.
  • Furthermore, there is no assurance on whether they share the same parameter set or not. At least JUCE on VST3 often generates extraneous parameter set for MIDI CC dispatching (like 2050+ extra).
• What if I use ADLplug on Linux as LV2 somewhere, and ADLplug via lv2vst i.e. as VST2 elsewhere?

Can we cover some or all of them?

Some relevant web resources (will be updated):

• https://www.kvraudio.com/forum/viewtopic.php?t=285627
• https://www.kvraudio.com/forum/viewtopic.php?t=397199

On plugin state portability:

For LV2 plugins, there is LV2_State_Map_Path feature that should be provided by the host of which a plugin can make use to convert absolute and abstract file paths. https://lv2plug.in/doc/html/group__state.html#structLV2__State__Map__Path

It is used by sfizz for example, so that SFZ files are stored in abstract form, not immediate path.

The native-issue branch exposes the issue with native build support:

https://github.com/atsushieno/augene-ng/actions/runs/1198125545

e: Could not find "/home/runner/work/augene-ng/augene-ng/kotractive-project/kotractive/build/generated/ksp/nativeMain/classes" in [/home/runner/work/augene-ng/augene-ng/kotractive-project, /home/runner/.konan/klib, /home/runner/.konan/kotlin-native-prebuilt-linux-x86_64-1.5.30/klib/common, /home/runner/.konan/kotlin-native-prebuilt-linux-x86_64-1.5.30/klib/platform/linux_x64]

it's known at Kotlin Slack https://kotlinlang.slack.com/archives/C013BA8EQSE/p1630246449011600

(partly imported from atsushieno/augene#2 and atsushieno/augene#3)

It is possible to support arbitrary audio plugin player backend. Right now it has kotracktive Midi2TracktionEdit and tracktion_engine as the only format, but it is only the converter and sequencer engine player format.

There are some other open song formats such as bitwig/dawproject which might be useful too (Bitwig has no OSS player ability though). dawproject uses some format called vstpresetas a plugin reference.

(Unlike when I created augene, JUCE6 and Waveform11 support VST3 plugins, so there is no gap between those two nowadays.)

Currently we don't import them, but it should be doable.

It has been known when I had been stuck at #5 but while that issue is gone it's still there:

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':commonizeNativeDistribution'.
	at org.gradle.api.internal.tasks.properties.DefaultTaskProperties.resolve(DefaultTaskProperties.java:83)
	at org.gradle.execution.plan.LocalTaskNode.resolveMutations(LocalTaskNode.java:200)
	at org.gradle.execution.plan.DefaultExecutionPlan.getResolvedMutationInfo(DefaultExecutionPlan.java:665)
	at org.gradle.execution.plan.DefaultExecutionPlan.selectNext(DefaultExecutionPlan.java:577)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$executeNextNode$1(DefaultPlanExecutor.java:166)
	at org.gradle.internal.resources.DefaultResourceLockCoordinationService.withStateLock(DefaultResourceLockCoordinationService.java:45)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:155)
	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
Caused by: java.lang.ClassCastException: class org.jetbrains.kotlin.gradle.dsl.KotlinJvmProjectExtension_Decorated cannot be cast to class org.jetbrains.kotlin.gradle.dsl.KotlinProjectExtension (org.jetbrains.kotlin.gradle.dsl.KotlinJvmProjectExtension_Decorated is in unnamed module of loader 
...
(snip)

To workaround the problem, disable native build again (disable native targets in kotractive/build.gradle.kts).

While Tracktion does not support per-note controller in MIDI 2.0 in general, it supports MPE. It is stored in *.tracktionedit like:

        <NOTE p="39" b="9.999999999999998" l="5.000000000000002" v="100" c="0">
          <PITCHBEND b="0.0" v="-2.500001907348633"/>
          <PITCHBEND b="1.0" v="-2.500001907348633"/>
          <PITCHBEND b="4.0" v="-16.83333206176758"/>
          <PITCHBEND b="3.0" v="-18.33333206176758"/>
          <PITCHBEND b="2.0" v="-20.33333587646484"/>
        </NOTE>

Currently there is no dedicated support for per-note pitchbend (6nh) in mugene-ng default-macro2.mml, but once it is supported we should be able to convert them into <PITCHBEND> elements like above.

kotractive-project often fails incremental builds. There should be some remedy according to https://github.com/google/ksp/tree/main/docs