When working on serialization of some packets I tried to improve the interface to use some more generic methods (will post another pull request) I have stumbled upon the code for UShort (de)serialization.

For some reason these use usual Int as the input for the method instead of UShort which exists in the Kotlin stdlib. Is there any reasoning why the code does not follow the other examples? Types like UInt, ULong and UChar all use already existing types and that seems to be a weird exception.

In an attempt to dive into the federated learning infrastructure of one of my students, I noticed that the following error is continuously being spammed in the console when running the IPv8 demo application:

nl.tudelft.ipv8.exception.PacketDecodingException: Incoming packet has an invalid public key
	at nl.tudelft.ipv8.messaging.Packet.getAuthPayload(Packet.kt:81)
	at nl.tudelft.ipv8.messaging.Packet.getAuthPayload(Packet.kt:34)
	at nl.tudelft.ipv8.Community.onIntroductionResponsePacket$ipv8(Community.kt:336)
	at nl.tudelft.ipv8.Community$4.invoke(Community.kt:50)
	at nl.tudelft.ipv8.Community$4.invoke(Community.kt:23)
	at nl.tudelft.ipv8.Community.onPacket(Community.kt:153)
	at nl.tudelft.ipv8.messaging.Endpoint.notifyListeners(Endpoint.kt:28)
	at nl.tudelft.ipv8.messaging.EndpointAggregator.onPacket(EndpointAggregator.kt:75)
	at nl.tudelft.ipv8.messaging.Endpoint.notifyListeners(Endpoint.kt:28)
	at nl.tudelft.ipv8.messaging.udp.UdpEndpoint.handleReceivedPacket$ipv8(UdpEndpoint.kt:188)
	at nl.tudelft.ipv8.messaging.udp.UdpEndpoint$bindSocket$1.invokeSuspend(UdpEndpoint.kt:164)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.IllegalArgumentException: Bin prefix 3040301006072aefbfbd48efbfbd does not match LibNaCLPK:
	at nl.tudelft.ipv8.keyvault.LibNaClPK$Companion.fromBin(LibNaClPK.kt:59)
	at nl.tudelft.ipv8.keyvault.JavaCryptoProvider.keyFromPublicBin(JavaCryptoProvider.kt:14)
	at nl.tudelft.ipv8.messaging.Packet.getAuthPayload(Packet.kt:79)
	... 16 more

I converted the binary prefix to hex myself for readability. It seems that this is the old prefix we used for M2Crypto support. I suggest to ignore errors related to an invalid prefixes and ignore the message in its entirety.

We've encountered an issue with kotlin-ipv8. When using the most recent version of the kotlin-ipv8 repo in the trustchain-superapp, the following errors are introduced in the tests in kotlin-ipv8:

> Task :ipv8:compileTestKotlin FAILED
e: /Users/runner/runners/2.169.0/work/trustchain-superapp/trustchain-superapp/kotlin-ipv8/ipv8/src/test/java/nl/tudelft/ipv8/BaseCommunityTest.kt: (21, 2): Unresolved reference: OptIn
e: /Users/runner/runners/2.169.0/work/trustchain-superapp/trustchain-superapp/kotlin-ipv8/ipv8/src/test/java/nl/tudelft/ipv8/BaseCommunityTest.kt: (21, 8): This class can only be used as an annotation or as an argument to @UseExperimental
e: /Users/runner/runners/2.169.0/work/trustchain-superapp/trustchain-superapp/kotlin-ipv8/ipv8/src/test/java/nl/tudelft/ipv8/attestation/trustchain/TrustChainCrawlerTest.kt: (19, 2): Unresolved reference: OptIn
e: /Users/runner/runners/2.169.0/work/trustchain-superapp/trustchain-superapp/kotlin-ipv8/ipv8/src/test/java/nl/tudelft/ipv8/attestation/trustchain/TrustChainCrawlerTest.kt: (19, 8): This class can only be used as an annotation or as an argument to @UseExperimental

I saw that in the trustchain-superapp:master and trustchain-superapp:dao older commits of kotlin-ipv8 are used, this explains why the error did not show up before on GitHub. I am trying to figure out at the moment which commit exactly of kotlin-ipv8 has introduced the error.

I discovered that also blocks from other apps, peerchat in my case, are stored in the database.
Not aware that this might happen I tried to unpack the transaction, which results in a crash.

What I had to do is to check block.type before unpacking.
What I can also do is to change the serviceId for the TrustChainCommunity, but as we are also open source, this serviceId is also addressable by other apps. What I fear is that someone can use DOS attacks to make the db explode (no disk space left).

Am I missing something?
I would rather add an encrypted API key in the blocks, so that the app can check if this block comes from the same app and only store those blocks.

We are working on a social media app based on ideas from Scuttlebut and we are using IPv8 as a base.
We are also minting coins and paying them out like universal basic income.
The TrustChainCommunity looks suitable for our coins and payments.

The documentation lacks of such a scenario, so its not clear for me how to secure the transactions in my app.
Its said that you should send a proposal and wait for an agreement.
So far so good, but I don't understand when the transaction is complete. When can both sides say "now the transaction is ready"?
Why shall I broadcast the blocks?
Why shall I crawl the trustchain?
How to identify that a proposal/agreement is coming from the same or an authorized app?

Normally I would say that for the receiver the transaction is ready when he gets the proposal. But what if the agreement never reaches the payer? Then the payer has only got a half block without agreement. Can he not double spend the coins or shall half blocks count as spent?

hi thanks for your very useful and perfect project, i am working on this project but can't find where that trust scores calculation. please help me. what module or section do that?

What are the steps towards removing the bootstrap peers that are currently hardcoded in this project? Does it require an implementation of a distributed hash table? I can understand it will be much harder to find peers without bootstrap nodes/trackers, but is it possible with the current functionalities? I guess that there needs to be more functionalities and testing for peers to do broadcasting of their identities to find other peers.

I would like to use your project in another open source project, but because of the lack of a license I might ran into legal issues.
For this reason a post about my app on reddit in an open source forum has already been deleted.
My project neither had a license.

May you please provide an open source license for kotlin-ipv8.

To be able to use kotlin-ipv8 I had to change Community.kt where all bootstrap servers are listed (remove one node which causes errors and to add my own tracker node) .
Also I had to add exception handling to pragmatically stop my app from crashing.

Thank you

The Trustchain functionality does not work at all when running from the command line. The tests do, however, pass. This is specifically due to this snippet in Community.kt:

scope = CoroutineScope(Dispatchers.Main + job)

The Main dispatcher is meant to interact with UI objects and as such would like an Android, JavaFX, or Swing dependency. This is of course undesirable when running from command line. When running from command line, the application crashes when functionality using the Main dispatcher is used, for instance when signing agreement blocks. The demo-jvm application is too rudimentary to pick up on this. The unit tests also do not find this because during testing, the Main dispatcher is set to be a TestCoroutineDispatcher (see BaseCommunityTest.kt) which does work without a GUI. Additionally, it is considered bad practice to hardcode your dispatchers.

How to reproduce

Replace startIpv8() in Application.kt with the following and run two instances of the application:

private fun startIpv8() {
    val myKey = JavaCryptoProvider.generateKey()
    val myPeer = Peer(myKey)
    val udpEndpoint = UdpEndpoint(8090, InetAddress.getByName("0.0.0.0"))
    val endpoint = EndpointAggregator(udpEndpoint, null)

    val config = IPv8Configuration(overlays = listOf(
        createDiscoveryCommunity(),
        createTrustChainCommunity(),
        createDemoCommunity()
    ), walkerInterval = 1.0)

    val ipv8 = IPv8(endpoint, config, myPeer)
    ipv8.start()

    val trustCom = ipv8.getOverlay<TrustChainCommunity>()!!
    val demoCom = ipv8.getOverlay<DemoCommunity>()!!

    trustCom.registerBlockSigner("trust_issue_block", object : BlockSigner {
        override fun onSignatureRequest(block: TrustChainBlock) {
            logger.info("Received an issue block.")
            trustCom.createAgreementBlock(block, mapOf<Any?, Any?>())
        }
    })

    scope.launch {
        while (true) {
            if (demoCom.getPeers().isNotEmpty()) {
                trustCom.createProposalBlock(
                    "trust_issue_block",
                    mapOf("message" to "hi"),
                    demoCom.getPeers()[0].publicKey.keyToBin()
                )
            }

            for ((_, overlay) in ipv8.overlays) {
                printPeersInfo(overlay)
            }
            logger.info("===")
            delay(5000)
        }
    }

    while (ipv8.isStarted()) {
        Thread.sleep(1000)
    }
}

The Overlay states an demo-android module as an example of how to configure an Android app module, initialize IPv8, and interact with the overlays. Where can I find it? it seems to be missing.
Im trying to run the demo network on an S7/S8 phone and its missing a built configuration.

Hi,
I'm working on a trustchain implementation for a Covid app, I'm able to send / receive my own messages in string format. However sometimes the app crashes with the following stacktrace at seemingly random times.
I'm I working on an older version or am I doing something wrong in my message generator.
Full project on github

Stacktrace

    java.lang.IllegalArgumentException: Bin prefix �-�����p��� does not match LibNaCLPK:
        at nl.tudelft.ipv8.keyvault.LibNaClPK$Companion.fromBin(LibNaClPK.kt:51)
        at nl.tudelft.ipv8.android.keyvault.AndroidCryptoProvider.keyFromPublicBin(AndroidCryptoProvider.kt:15)
        at nl.tudelft.ipv8.android.messaging.bluetooth.GattClientManager$IPv8GattCallback$initialize$2.onDataReceived(GattClientManager.kt:67)
        at no.nordicsemi.android.ble.ReadRequest.lambda$notifyValueChanged$0(ReadRequest.java:240)
        at no.nordicsemi.android.ble.-$$Lambda$ReadRequest$406mAhxV10eMfmbBI4l07-ac_9c.run(Unknown Source:6)
        at android.os.Handler.handleCallback(Handler.java:789)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6944)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)

Message

class MyMessage(val message: String) : Serializable {
    override fun serialize(): ByteArray {
        return message.toByteArray()
    }

    companion object Deserializer : Deserializable<MyMessage> {
        override fun deserialize(buffer: ByteArray, offset: Int): Pair<MyMessage, Int> {
            return Pair(MyMessage(buffer.toString(Charsets.UTF_8)), buffer.size)
        }
    }
}

send/reciever functions

    fun broadcastHelloMessage(peer : Peer, message: String) {
        val packet = serializePacket(MESSAGE_ID_HELLO, MyMessage(message))
        send(peer.address, packet)
    }
    /**
     * Receiving a packet from peer over network
     */
    private fun onHelloMessage(packet: Packet) {
        val (peer: Peer, payload: MyMessage) = packet.getAuthPayload(MyMessage.Deserializer)
        scope.launch{

            _message.value = "Recieved message '${payload.message}' from peer ${peer.mid}"
        }
    }

After PR #67 , logging functionality of command line applications (e.g. demo-jvm) stopped working. The console returns:

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
[...]

Presumably, this has to do with outdated dependencies. I will post an update shortly.

We are using the apply plugin syntax in the Gradle build files:

apply plugin: "application"
apply plugin: "kotlin"
apply plugin: "kotlinx-serialization"

This syntax, however, is not recommended anymore, also see here. Instead, we could use the plugins directive like:

plugins {
    id 'org.jetbrains.kotlin.<...>' version '1.5.21'
}

See for example my recent PR: #49

I see some tests in the source directory though. I guess they should run?

It would be great if this repository could also support iOS, which should be possible using multiplatform programming. Yesterday, I spent a few hours looking into this. It seems that the Kotlin runtime is compiled to native C code and thus can be executed using an Objective-C/Swift compatibility layer.

Some of my findings:

• Proper multiplatform programming requires deep understanding of the Gradle build system since different platforms require different conditions. For example, one cannot simply import the BouncyCastle library on iOS since this library uses hardware features only found on Android.
• Multiplatform programming stil feels very experimental. I had to work around quite a few bugs in order to get even a basic example up and running. The iOS simulator also does not properly start and I have to launch it from Xcode instead.
• The startup of Kotlin on iOS seems to take a few seconds.
• If we want to run IPv8 on iOS, we have to rethink some of the IPv8 interactions with libraries. Luckily, it seems that there is iOS compatibility for the sqldelight library. However, it seems that we are unable to import from the java standard library which means we cannot re-use the UDPEndpoint logic on the iOS side. We probably need iOS-specific logic for that. Also, the lazysodium library advertises support for iOS but I could not find much about that.

In summary, I think adding iOS support to this repository is doable but very challenging.

If I understand correctly, the service ID generated in TrackerCommunity class uses a sha1 hash of a private key. Should this be upgraded to a more secure hashing algorithm (sha2?)

    override val serviceId: String = sha1(defaultCryptoProvider.generateKey().keyToBin()).toHex()

The node IPv4Address("131.180.27.161", 6427) stated in Community.kt sends bad INTRODUCTION_REQUESTS
I get warnings: Unable to decode authenticated payload

Is it more save to only use our own tracker(s) nodes?

Another issue I found relates to the serialization of the Long values.

In the code seen below we can see an explicit cast to Int before putting the data in the buffer, thus the function downcasts to Int in serialization and upcasts back to Long when deserializing, causing confusion.

I suggest conforming to the naming and just using Long in transit and create a separate explicit Int serialization.

When a device registers an ENETUNREACH IO error, kotlin-ipv8 can crash, due to the UdpEndpoint.kt code.
Stack trace:

Fatal Exception: java.io.IOException: sendto failed: ENETUNREACH (Network is unreachable)
       at libcore.io.IoBridge.maybeThrowAfterSendto(IoBridge.java:576)
       at libcore.io.IoBridge.sendto(IoBridge.java:544)
       at java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:125)
       at java.net.DatagramSocket.send(DatagramSocket.java:723)
       at nl.tudelft.ipv8.messaging.udp.UdpEndpoint$send$2.invokeSuspend(UdpEndpoint.kt:75) <-------------------------
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:561)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:727)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:667)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:655)

When one currently wants to use this repository, there is no other option than to add this project as submodule, as done by the superapp. However, Gradle does not really support this kind of architecture and support for it is whacky. The "proper" way to re-use the code in this repository is by building a .jar file and including it in the project (the same with depending on Python modules). Therefore, it might be a good idea to build this file when merging a PR.