Giter Club home page Giter Club logo

rlp_03's Introduction

Java RELP Server Library (rlp_03)

rlp_03 implements RELP server in Java

License

AGPLv3 with additional permissions granted in the license.

Features

Current

  • Fast (~100 000 transactions in a second per thread)

  • Secure (TLS)

Usage

Setting dependencies

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <dependencies>
        <!-- this library -->
        <dependency>
            <groupId>com.teragrep</groupId>
            <artifactId>rlp_03</artifactId>
            <version>${see_latest_at_github}</version>
        </dependency>
    </dependencies>
</project>
Note
See rlp_01 for relp client
Note
See rlo_06 for syslog decoding

Examples

Dependencies for examples

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <dependencies>
        <!-- this library -->
        <dependency>
            <groupId>com.teragrep</groupId>
            <artifactId>rlp_03</artifactId>
            <version>${see_latest_at_github}</version>
        </dependency>
        <!-- rlp_01 for relp-client -->
        <dependency>
            <groupId>com.teragrep</groupId>
            <artifactId>rlp_01</artifactId>
            <version>${see_latest_at_github}</version>
        </dependency>
    </dependencies>
</project>

Server with shared handler for all connections. See ExampleRelpClient.java for client.

package com.teragrep.rlp_03.readme;

import com.teragrep.rlp_03.eventloop.EventLoop;
import com.teragrep.rlp_03.eventloop.EventLoopFactory;
import com.teragrep.rlp_03.frame.delegate.DefaultFrameDelegate;
import com.teragrep.rlp_03.frame.delegate.FrameContext;
import com.teragrep.rlp_03.frame.delegate.FrameDelegate;
import com.teragrep.rlp_03.server.ServerFactory;
import com.teragrep.rlp_03.channel.socket.PlainFactory;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.io.UncheckedIOException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.function.Consumer;
import java.util.function.Supplier;

/**
 * For use cases in the README.adoc
 */
public class ReadmeTest {

    @Test
    public void testServerSetup() {
        int listenPort = 10601;
        int threads = 1; // processing threads shared across the connections
        ExecutorService executorService = Executors.newFixedThreadPool(threads);

        /*
         * System.out.println is used to print the frame payload
         */
        Consumer<FrameContext> syslogConsumer = new Consumer<FrameContext>() {

            // NOTE: synchronized because frameDelegateSupplier returns this instance for all the parallel connections
            @Override
            public synchronized void accept(FrameContext frameContext) {
                System.out.println(frameContext.relpFrame().payload().toString());
            }
        };

        /*
         * DefaultFrameDelegate accepts Consumer<FrameContext> for processing syslog frames
         */
        DefaultFrameDelegate frameDelegate = new DefaultFrameDelegate(syslogConsumer);

        /*
         * Same instance of the frameDelegate is shared with every connection
         */
        Supplier<FrameDelegate> frameDelegateSupplier = new Supplier<FrameDelegate>() {

            @Override
            public FrameDelegate get() {
                System.out.println("Providing frameDelegate for a connection");
                return frameDelegate;
            }
        };

        /*
         * EventLoop is used to notice any events from the connections
         */
        EventLoopFactory eventLoopFactory = new EventLoopFactory();
        EventLoop eventLoop;
        try {
            eventLoop = eventLoopFactory.create();
        }
        catch (IOException e) {
            throw new RuntimeException(e);
        }

        Thread eventLoopThread = new Thread(eventLoop);
        /*
         * eventLoopThread must run, otherwise nothing will be processed
         */
        eventLoopThread.start();

        /*
         * ServerFactory is used to create server instances
         */
        ServerFactory serverFactory = new ServerFactory(
                eventLoop,
                executorService,
                new PlainFactory(),
                frameDelegateSupplier
        );

        try {
            serverFactory.create(listenPort);
            System.out.println("server started at port <" + listenPort + ">");
        }
        catch (IOException ioException) {
            throw new UncheckedIOException(ioException);
        }

        /*
         * Send Hello, World! via rlp_01
         */
        new ExampleRelpClient(listenPort).send("Hello, World!");

        /*
         * Stop eventLoop
         */
        eventLoop.stop();

        /*
         * Wait for stop to complete
         */
        try {
            eventLoopThread.join();
        }
        catch (InterruptedException interruptedException) {
            throw new RuntimeException(interruptedException);
        }
        System.out.println("server stopped at port <" + listenPort + ">");

        /*
         * Close the frameDelegate
         */
        try {
            frameDelegate.close();
        }
        catch (Exception e) {
            throw new RuntimeException(e);
        }
        executorService.shutdown();
    }
}

If a separate handler is required for each connection which doesn’t need to be a thread-safe, create a new FrameDelegate in the Supplier<FrameDelegate>

        Supplier<FrameDelegate> frameDelegateSupplier = () -> {
            System.out.println("Providing frameDelegate for a connection");
            return new DefaultFrameDelegate(frameContext -> System.out.println(frameContext.relpFrame().payload().toString()));
        };

If a deferred handler is required for command processing, pass custom RelpEvent implementation to DefaultFrameDelegate via the Map<String, RelpEvent> constructor. See ReadmeDeferredTest.java for an example.

Contributing

You can involve yourself with our project by opening an issue or submitting a pull request.

Contribution requirements:

  1. All changes must be accompanied by a new or changed test. If you think testing is not required in your pull request, include a sufficient explanation as why you think so.

  2. Security checks must pass

  3. Pull requests must align with the principles and values of extreme programming.

  4. Pull requests must follow the principles of Object Thinking and Elegant Objects (EO).

Read more in our Contributing Guideline.

Contributor License Agreement

Contributors must sign Teragrep Contributor License Agreement before a pull request is accepted to organization’s repositories.

You need to submit the CLA only once. After submitting the CLA you can contribute to all Teragrep’s repositories.

rlp_03's People

Contributors

eemhu avatar kortemik avatar ronja-ui avatar strongestnumber9 avatar

Stargazers

 avatar

Watchers

 avatar  avatar

rlp_03's Issues

double closing of selector and serverSocketChannel

Following code should be migrated

try {
serverSocketChannel.close();
}
catch (IOException ioException) {
LOGGER.warn("IOException while closing serverSocketChannel", ioException);
}
finally {
try {
selector.close();
}
catch (IOException ioException) {
LOGGER.warn("IOException while closing selector", ioException);
}
}

to

public void close() throws IOException {
serverSocketChannel.close();
selector.close();
}

because try-with-resources here closes it

try (SocketPoll socketPoll = new SocketPoll(executorService,socketFactory, selector, serverSocketChannel, frameDelegateSupplier)) {

Readme has an error

Description

        /*
         * DefaultFrameDelegate accepts Consumer<FrameContext> for processing syslog frames
         */
        DefaultFrameDelegate frameDelegate = new DefaultFrameDelegate(syslogConsumer);

        /*
         * Same instance of the frameDelegate is shared with every connection
         */
        Supplier<FrameDelegate> frameDelegateSupplier = new Supplier<FrameDelegate>() {
            @Override
            public FrameDelegate get() {
                System.out.println("Providing frameDelegate for a connection");
                return frameDelegate;
            }
        };

Should be following

        /*
         * New instance of the frameDelegate is provided for every connection
         */
        Supplier<FrameDelegate> frameDelegateSupplier = new Supplier<FrameDelegate>() {
            @Override
            public FrameDelegate get() {
                System.out.println("Providing frameDelegate for a connection");
                return new DefaultFrameDelegate(syslogConsumer);
            }
        };

because DefaultFrameDelegate does transaction id (txn) tracking and will reject furthe connections. please fix ReadmeTest, DeferredReadmeTest and the README.adoc

at com.teragrep.rlp_03.RelpServerSocket.processRead(RelpServerSocket.java:98) Maximum lock count exceeded

Exception in thread "Thread-1" java.lang.Error: Maximum lock count exceeded
        at java.util.concurrent.locks.ReentrantLock$Sync.nonfairTryAcquire(ReentrantLock.java:141)
        at java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:365)
        at com.teragrep.rlp_03.RelpServerSocket.processRead(RelpServerSocket.java:98)
        at com.teragrep.rlp_03.SocketProcessor.grandSelector(SocketProcessor.java:201)
        at com.teragrep.rlp_03.SocketProcessor.lambda$run$0(SocketProcessor.java:124)
        at java.lang.Thread.run(Thread.java:750)

Coverity: Resource leak on an exceptional path In com.​teragrep.​rlp_03.​SocketPoll.​processAccept

Describe the bug

The system resource will not be reclaimed and reused, reducing the future availability of the resource.

In com.​teragrep.​rlp_03.​SocketPoll.​processAccept(java.​nio.​channels.​ServerSocketChannel, java.​nio.​channels.​SelectionKey): Leak of a system resource on an exception path (probably error handling) (CWE-404)

Coverity says

 98    private void processAccept(ServerSocketChannel serverSocketChannel, SelectionKey selectionKey) throws IOException {
      1. Condition selectionKey.isAcceptable(), taking true branch.
 99        if (selectionKey.isAcceptable()) {
100            // create the client socket for a newly received connection
      2. alloc_fn: A new resource is returned from allocation method accept.
      3. var_assign: Assigning: clientSocketChannel = resource returned from serverSocketChannel.accept().
101            SocketChannel clientSocketChannel = serverSocketChannel.accept();
102
      4. Condition com.teragrep.rlp_03.SocketPoll.LOGGER.isDebugEnabled(), taking true branch.
103            if (LOGGER.isDebugEnabled()) {
      5. Throwing java.io.IOException (or subclass) from call to getLocalAddress; exiting method with uncaught exception.
     
CID 436388: (#1 of 1): Resource leak on an exceptional path (RESOURCE_LEAK)
6. leaked_resource: Variable clientSocketChannel going out of scope leaks the resource it refers to.
104                LOGGER.debug("ServerSocket <{}> accepting ClientSocket <{}> ", serverSocketChannel.getLocalAddress(), clientSocketChannel.getRemoteAddress());
105            }

Expected behavior

How to reproduce

Screenshots

Software version

private void processAccept(ServerSocketChannel serverSocketChannel, SelectionKey selectionKey) throws IOException {
if (selectionKey.isAcceptable()) {
// create the client socket for a newly received connection
SocketChannel clientSocketChannel = serverSocketChannel.accept();
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("ServerSocket <{}> accepting ClientSocket <{}> ", serverSocketChannel.getLocalAddress(), clientSocketChannel.getRemoteAddress());
}
// tls/plain wrapper
Socket socket = socketFactory.create(clientSocketChannel);

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

Coverity: Resource leak on an exceptional path In com.​teragrep.​rlp_03.​ServerFactory.​create()

Describe the bug

The system resource will not be reclaimed and reused, reducing the future availability of the resource.

In com.​teragrep.​rlp_03.​ServerFactory.​create(): Leak of a system resource on an exception path (probably error handling) (CWE-404)

Coverity says

64    public Server create() throws IOException {
65        config.validate();
66
67        SocketFactory socketFactory;
     1. Condition tlsConfig.useTls, taking true branch.
68        if (tlsConfig.useTls) {
69            socketFactory = new TLSFactory(tlsConfig.getSslContext(), tlsConfig.getSslEngineFunction());
     2. Falling through to end of if statement.
70        }
71        else {
72            socketFactory = new PlainFactory();
73        }
74
     3. alloc_fn: A new resource is returned from allocation method open.
     4. var_assign: Assigning: selector = resource returned from java.nio.channels.Selector.open().
75        Selector selector = Selector.open();
     5. Throwing java.io.IOException (or subclass) from call to open; exiting method with uncaught exception.
    
CID 436389: (#1 of 1): Resource leak on an exceptional path (RESOURCE_LEAK)
6. leaked_resource: Variable selector going out of scope leaks the resource it refers to.
76        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
77        serverSocketChannel.socket().setReuseAddress(true);
78        serverSocketChannel.bind(listenSocketAddress);
79        serverSocketChannel.configureBlocking(false);
80        serverSocketChannel.register(selector, OP_ACCEPT);
81
82        return new Server(executorService, frameProcessorSupplier, serverSocketChannel, socketFactory, selector);
83    }

Expected behavior

How to reproduce

Screenshots

Software version

public Server create() throws IOException {
config.validate();
SocketFactory socketFactory;
if (tlsConfig.useTls) {
socketFactory = new TLSFactory(tlsConfig.getSslContext(), tlsConfig.getSslEngineFunction());
}
else {
socketFactory = new PlainFactory();
}
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.socket().setReuseAddress(true);
serverSocketChannel.bind(listenSocketAddress);
serverSocketChannel.configureBlocking(false);
serverSocketChannel.register(selector, OP_ACCEPT);
return new Server(executorService, frameProcessorSupplier, serverSocketChannel, socketFactory, selector);
}
}

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

build failure

Description
downgrade spotless to

diff --git a/pom.xml b/pom.xml
index a91c349..0a1a135 100644
--- a/pom.xml
+++ b/pom.xml
@@ -118,7 +118,7 @@
       <plugin>
         <groupId>com.diffplug.spotless</groupId>
         <artifactId>spotless-maven-plugin</artifactId>
-        <version>2.43.0</version>
+        <version>2.30.0</version>
         <configuration>
           <java>
             <eclipse>

Coverity: Lock not released on all paths In com.​teragrep.​rlp_03.​Status.​complete()

Describe the bug

In com.​teragrep.​rlp_03.​Status.​complete(): This method acquires a JSR-166 (java.util.concurrent) lock, but does not release it on all exception paths out of the method. (From SpotBugs description)

Coverity says

22    void complete() {
CID 436390: (#1 of 1): UL: Lock not released on all paths (FB.UL_UNRELEASED_LOCK_EXCEPTION_PATH)
1. defect: com.teragrep.rlp_03.Status.complete() does not release lock on all exception paths.
23        lock.lock();
24        done.set(true);
25        pending.signal();
26        lock.unlock();
27    }

Expected behavior

How to reproduce

Screenshots

Software version

void complete() {
lock.lock();
done.set(true);
pending.signal();
lock.unlock();
}

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

Infinite loop in runServerTest when relp connections are not gracefully terminated

Testing tool with rlp_01 and multiple threads sending 1 event batches in busy loop -> kill the program and this is what server gets stuck in:

java.net.SocketException: Connection reset
        at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
        at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:411)
        at com.teragrep.rlp_03.RelpServerPlainSocket.read(RelpServerPlainSocket.java:164)
        at com.teragrep.rlp_03.MessageReader.readRequest(MessageReader.java:109)
        at com.teragrep.rlp_03.RelpServerPlainSocket.processRead(RelpServerPlainSocket.java:102)
        at com.teragrep.rlp_03.SocketProcessor.processReadWriteClose(SocketProcessor.java:421)
        at com.teragrep.rlp_03.SocketProcessor.runMTMessageSelector(SocketProcessor.java:381)
        at com.teragrep.rlp_03.SocketProcessor.lambda$runMultiThreaded$0(SocketProcessor.java:235)
        at java.base/java.lang.Thread.run(Thread.java:833)
java.net.SocketException: Connection reset
        at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
        at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:411)
        at com.teragrep.rlp_03.RelpServerPlainSocket.read(RelpServerPlainSocket.java:164)
        at com.teragrep.rlp_03.MessageReader.readRequest(MessageReader.java:109)
        at com.teragrep.rlp_03.RelpServerPlainSocket.processRead(RelpServerPlainSocket.java:102)
        at com.teragrep.rlp_03.SocketProcessor.processReadWriteClose(SocketProcessor.java:421)
        at com.teragrep.rlp_03.SocketProcessor.runMTMessageSelector(SocketProcessor.java:381)
        at com.teragrep.rlp_03.SocketProcessor.lambda$runMultiThreaded$0(SocketProcessor.java:235)
        at java.base/java.lang.Thread.run(Thread.java:833)

truncation support

Description
currently rlp_03 accepts records of any size, however there may be cases where limiting the maximum payload size is preferred. These may be such as clients sending accidentally (or deliberately) large relp frames and these reserve quite much memory on the processing side.

Use case or motivation behind the feature request
Support configurable limit for payload truncation. RelpParser would store up-to-the-limit amount of data and then process the rest but not store. Truncated frames should be marked as such and the total size before truncation should be made available.

Related issues

Additional context

Coverity: Lock not released on all paths In com.​teragrep.​rlp_03.​context.​RelpReadImpl.​run()

Describe the bug

In com.​teragrep.​rlp_03.​context.​RelpReadImpl.​run(): This method acquires a JSR-166 (java.util.concurrent) lock, but does not release it on all exception paths out of the method. (From SpotBugs description)

Coverity says:

 54    @Override
 55    public void run() {
 56        try {
 57            LOGGER.debug("task entry!");
     
CID 436387: (#1 of 1): UL: Lock not released on all paths (FB.UL_UNRELEASED_LOCK_EXCEPTION_PATH)
1. defect: com.teragrep.rlp_03.context.RelpReadImpl.run() does not release lock on all exception paths.
 58            lock.lock();
 59            if (LOGGER.isDebugEnabled()) {
 60                LOGGER.debug("task lock! with activeBuffers.size() <{}>", activeBuffers.size());
 61            }
 62

Expected behavior

How to reproduce

Screenshots

Software version

@Override
public void run() {
try {
LOGGER.debug("task entry!");
lock.lock();
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("task lock! with activeBuffers.size() <{}>", activeBuffers.size());
}

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

Remove or implement commented out code in CommandFunction.java

Description

Clean up code where necessary

Use case or motivation behind the feature request

Commented out code is burden from code maintenance point of view

Related issues

Additional context

// private final Set<String> enabledCommands;
public CommandFunction() {
/*
this.enabledCommands = new HashSet<>();
this.enabledCommands.add("open");
this.enabledCommands.add("close");
this.enabledCommands.add("abort");
this.enabledCommands.add("serverclose");
this.enabledCommands.add("syslog");
this.enabledCommands.add("rsp");
int maximumLength = 0;
for (String command : enabledCommands) {
if (command.length() > maximumLength) {
maximumLength = command.length();
}
}
maximumCommandLength = maximumLength + 1; // space
*/
}

Coverity: CID 454007: Exceptional resource leaks (RESOURCE_LEAK) in ClientFactory.java

Describe the bug

New defect(s) Reported-by: Coverity Scan
Showing 1 of 1 defect(s)


** CID 454007:  Exceptional resource leaks  (RESOURCE_LEAK)
/src/main/java/com/teragrep/rlp_03/client/ClientFactory.java: 76 in com.teragrep.rlp_03.client.ClientFactory.open([java.net](http://java.net/).InetSocketAddress)()


________________________________________________________________________________________________________
*** CID 454007:  Exceptional resource leaks  (RESOURCE_LEAK)
/src/main/java/com/teragrep/rlp_03/client/ClientFactory.java: 76 in com.teragrep.rlp_03.client.ClientFactory.open([java.net](http://java.net/).InetSocketAddress)()
70                 throws IOException, InterruptedException, ExecutionException {
71             // this is for returning ready connection
72             CompletableFuture<EstablishedContext> readyContextFuture = new CompletableFuture<>();
73             Consumer<EstablishedContext> establishedContextConsumer = readyContextFuture::complete;
74     
75             ClientDelegate clientDelegate = new ClientDelegate();
>>>     CID 454007:  Exceptional resource leaks  (RESOURCE_LEAK)
>>>     Variable "clientDelegate" going out of scope leaks the resource it refers to.
76             ConnectContext connectContext = connectContextFactory
77                     .create(inetSocketAddress, clientDelegate, establishedContextConsumer);
78             LOGGER.debug("registering to eventLoop <{}>", eventLoop);
79             eventLoop.register(connectContext);
80             LOGGER.debug("registered to eventLoop <{}>", eventLoop);
81             EstablishedContext establishedContext = readyContextFuture.get();
82             LOGGER.debug("returning establishedContext <{}>", establishedContext);
83             return clientDelegate.create(establishedContext);
84         }

Additional context

Reported by Coverity

write javadoc

Description
currently none exists, but it would be pretty much nice that some did!

client closing a socket uncleanly causes the server to loop

Following loops forever

relpServerSocket.read> exit with totalBytesRead: -1
messageReader.readRequest> exit with readBuffer: java.nio.DirectByteBuffer[pos=0 lim=262186 cap=262186]
messageReader.readRequest> entry with parser: com.teragrep.rlp_01.RelpParser@5aa6b1df and parser state: TXN
relpServerSocket.read> entry 
...
relpServerSocket.read> exit with totalBytesRead: -1
messageReader.readRequest> exit with readBuffer: java.nio.DirectByteBuffer[pos=0 lim=262186 cap=262186]
messageReader.readRequest> entry with parser: com.

Coming from:
https://github.com/teragrep/rlp_03/blob/main/src/main/java/com/teragrep/rlp_03/RelpServerSocket.java#L167

Needs a fix on both 1.0-fixes and on main.

update minimum required jdk to 11

Describe the bug
bytebuffer causes problems while mixing pre-java 9, build fails because spotless requires java 11

Expected behavior
both should work

How to reproduce
use java 8 compiled classes with java 11

Screenshots

Software version
47efd37

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context
suggesting upgrade to jdk 11

limits in command and transaction id are not accurate

Describe the bug
rlp_03 allows 10 numbers for id and maximum command length:

jshell> Integer.MAX_VALUE
$1 ==> 2147483647

jshell> String a = "2147483647";
a ==> "2147483647"

jshell> a.length()
$5 ==> 10
private static final int maximumIdNumbers  = String.valueOf(Integer.MAX_VALUE).length() + 1; // space
private static final int maximumLengthNumbers = String.valueOf(Integer.MAX_VALUE).length() + 1; // space

same goes for maximumCommandLength, it is defined to 12 however specs say otherwise

private static final int maximumCommandLength = 12; // serverclose + space

Expected behavior
relp specs say:
TXNR = NUMBER ; NUMBER = 1 * 9DIGIT
DATALEN = NUMBER ; NUMBER = 1 * 9DIGIT
COMMAND = 1 * 32ALPHA

How to reproduce
see code

Screenshots

Software version
982e819

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

server replies with wrong transaction id in serverclose

Describe the bug
server replies with wrong transaction id in serverclose

3 serverclose 0

Expected behavior
server should reply with transaction id 0 in serverclose:

0 serverclose 0

How to reproduce
send close frame from client to server, check returned response.

Screenshots

3 close 0

is replied with

txn=3, command=rsp, payloadLength=0, payload=, endOfTransfer=
txn=3, command=serverclose, payloadLength=0, payload=, endOfTransfer=

Software version
3.1.0

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

Cleanup code from commented out logger calls

Description

Get rid of these

$ grep -Ri "//.*LOGGER"
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:                //LOGGER.info("complete <{}> after resume", complete);
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:            //LOGGER.debug("activeBuffers.isEmpty() <{}>", activeBuffers.isEmpty());
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:                //LOGGER.debug("while activeBuffers.isEmpty() <{}>", activeBuffers.isEmpty());
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:                //LOGGER.debug("frame complete");
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:        //LOGGER.debug("innerLoop rv <{}>", rv);
src/main/java/com/teragrep/rlp_03/context/RelpReadImpl.java:            //LOGGER.debug("socket need to read more bytes");
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:    //private static final Logger LOGGER = LoggerFactory.getLogger(FragmentImpl.class);
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:        // LOGGER.info("accept input<{}> with bufferSliceList.size() <{}>", input, bufferSliceList.size());
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:            //LOGGER.info("isComplete.get() <{}>", isComplete.get());
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:        //LOGGER.info("called toBytes");
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:        // LOGGER.info("concatenating from bufferSliceList.size <{}>", bufferSliceList.size());
src/main/java/com/teragrep/rlp_03/context/frame/fragment/FragmentImpl.java:        //LOGGER.info("BYTES! parseRule <{}> returning bytes <{}>", parseRule, new String(bytes, StandardCharsets.UTF_8));
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:    //private static final Logger LOGGER = LoggerFactory.getLogger(RelpFrameImpl.class);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:        //LOGGER.info("submit for input <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:            //LOGGER.info("thisBuffer <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                //LOGGER.info("accepting into TXN thisBuffer <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                //LOGGER.info("accepting into COMMAND thisBuffer <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                //LOGGER.info("accepting into PAYLOAD LENGTH thisBuffer <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                    //LOGGER.info("creating PayloadFunction with payloadSize <{}>", payloadSize);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                //LOGGER.info("accepting into PAYLOAD thisBuffer <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:                //LOGGER.info("accepting into endOfTransfer");
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:            //LOGGER.info("after read input <{}>", input);
src/main/java/com/teragrep/rlp_03/context/frame/RelpFrameImpl.java:        //LOGGER.info("returning rv <{}>", rv);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:    // private static final Logger LOGGER = LoggerFactory.getLogger(PayloadFunction.class);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:            // LOGGER.info("adding whole buffer byteCount.get() <{}> input.limit() <{}>", byteCount.get(), input.limit());
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:            // LOGGER.info("total byte count after adding whole buffer <{}>", byteCount.get());
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:            // LOGGER.info("adding partial buffer byteCount.get() <{}> input.limit() <{}>", byteCount.get(), input.limit());
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:            // LOGGER.info("created bufferSlice <{}>", bufferSlice);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadFunction.java:        // LOGGER.info("return <{}> because byteCount.get() <{}> payloadLength <{}>", byteCount.get() == payloadLength, byteCount.get(), payloadLength);
src/main/java/com/teragrep/rlp_03/context/frame/function/TransactionFunction.java:    //private static final Logger LOGGER = LoggerFactory.getLogger(TransactionFunction.class);
src/main/java/com/teragrep/rlp_03/context/frame/function/TransactionFunction.java:            //LOGGER.info("read byte b <{}>", new String(new byte[]{b}, StandardCharsets.UTF_8));
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadLengthFunction.java:    //private static final Logger LOGGER = LoggerFactory.getLogger(PayloadLengthFunction.class);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadLengthFunction.java:            //LOGGER.info("input <{}>, b <{}>, bufferSliceList <{}>", input, new String(new byte[]{b}, StandardCharsets.UTF_8), bufferSliceList);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadLengthFunction.java:        //LOGGER.info("returning <{}> with bufferSliceList <{}>", rv, bufferSliceList);
src/main/java/com/teragrep/rlp_03/context/frame/function/PayloadLengthFunction.java:            //LOGGER.info("payloadLength oversize <{}>", currentLength, illegalArgumentException);
src/main/java/com/teragrep/rlp_03/context/frame/function/EndOfTransferFunction.java:    //private static final Logger LOGGER = LoggerFactory.getLogger(EndOfTransferFunction.class);
src/main/java/com/teragrep/rlp_03/context/frame/function/EndOfTransferFunction.java:        // LOGGER.info("apply with input <{}> bufferSliceList.size() <{}>", input, bufferSliceList.size());
src/main/java/com/teragrep/rlp_03/context/frame/function/EndOfTransferFunction.java:            // LOGGER.info("read byte b <{}>", new String(new byte[]{b}, StandardCharsets.UTF_8));
src/main/java/com/teragrep/rlp_03/context/buffer/BufferLeasePool.java:        //LOGGER.info("internalOffer <{}>", queue.size());
src/main/java/com/teragrep/rlp_03/context/buffer/BufferLeaseImpl.java:                // LOGGER.info("released bufferLease id <{}>, refs <{}>", bufferLease.id(), bufferLease.refs());
src/test/java/com/teragrep/rlp_03/context/frame/RelpFrameTest.java:            // LOGGER.info("contentIter <{}>", contentIter);

Use case or motivation behind the feature request

Either these logger calls are useful and should be used, or they are not useful and should be removed. Leaving them in causes confusion from code maintenance point of view as it is not exactly obvious whether something is relevant or even correct.

Related issues

#68

Additional context

Memory leak in new connection handler

Opening new relp sessions will eventually cause OutOfMemoryError on RELP-AC thread

Exception in thread "RELP-AC" java.lang.OutOfMemoryError: Cannot reserve 262186 bytes of direct buffer memory (allocated: 15778868117, limit: 15778971648)
        at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
        at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:121)
        at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
        at com.teragrep.rlp_03.MessageReader.<init>(MessageReader.java:84)
        at com.teragrep.rlp_03.RelpServerPlainSocket.<init>(RelpServerPlainSocket.java:88)
        at com.teragrep.rlp_03.SocketProcessor.processAccept(SocketProcessor.java:331)
        at com.teragrep.rlp_03.SocketProcessor.runMTAcceptSelector(SocketProcessor.java:297)
        at com.teragrep.rlp_03.SocketProcessor.lambda$runMultiThreaded$1(SocketProcessor.java:255)
        at java.base/java.lang.Thread.run(Thread.java:833)

Achieved by running this in multiple threads:

while True:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((hostname, port))
    sock.send(b"1 open 63 relp_version=0\nrelp_software=relp-checker,1.0.0\ncommands=syslog\n2 close 0\n")
    data = sock.recv(1024)

Culprit is most likely

this.readBuffer = ByteBuffer.allocateDirect(MAX_HEADER_CAPACITY + 1024*256);
never being cleared

Clean and enhance tests that are not properly implemented

Description

Implement proper tests where necessary and fail instead of printing exceptions

Use case or motivation behind the feature request

Tests can be broken and not detected properly. There should be no cases in automatic tests that either throws or needs to be tried and caught

Related issues

Additional context

Failure not properly detected

} catch (InterruptedException e) {
e.printStackTrace();
}
try {
testSendBatch();
testSendMessage();
} catch (IOException | TimeoutException | IllegalStateException e) {
e.printStackTrace();
}

} catch (UnrecoverableKeyException | CertificateException | KeyStoreException | IOException | KeyManagementException e) {
e.printStackTrace();
}
} catch (IOException ioException) {
ioException.printStackTrace();
}

@Test
public void testOpenAndCloseSession() throws IOException, TimeoutException {
long count = 10000;
while (count > 0) {
RelpConnection relpSession = new RelpConnection();
relpSession.connect(hostname, port);
relpSession.disconnect();
count--;
}
}

$ grep -Ri throws src/test/ | grep -v 'Assertions'
src/test/java/com/teragrep/rlp_03/ConnectionStormTest.java:    public void init() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/ConnectionStormTest.java:    public void testOpenAndCloseSession() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/context/channel/SocketFake.java:    public long read(ByteBuffer[] dsts) throws IOException {
src/test/java/com/teragrep/rlp_03/context/channel/SocketFake.java:    public long write(ByteBuffer[] dsts) throws IOException {
src/test/java/com/teragrep/rlp_03/context/channel/SocketFake.java:    public void close() throws IOException {
src/test/java/com/teragrep/rlp_03/ManualTest.java:    public void runServerTest() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/ManualTest.java:    public void runServerTlsTest() throws IOException, InterruptedException, GeneralSecurityException {
src/test/java/com/teragrep/rlp_03/ManualPerformanceTest.java:    public void runServerTest() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/ManualPerformanceTest.java:        public void close() throws Exception {
src/test/java/com/teragrep/rlp_03/CloseRelpFrameServerRXConsumerTest.java:        public void close() throws Exception {
src/test/java/com/teragrep/rlp_03/CloseRelpFrameServerRXConsumerTest.java:    private void init() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/CloseRelpFrameServerRXConsumerTest.java:    private void cleanup() throws InterruptedException {
src/test/java/com/teragrep/rlp_03/CloseRelpFrameServerRXConsumerTest.java:    public void testSendMessage() throws IOException, TimeoutException, InterruptedException {
src/test/java/com/teragrep/rlp_03/MultiClientTest.java: public void testMultiClient() throws InterruptedException, IllegalStateException {
src/test/java/com/teragrep/rlp_03/MultiClientTest.java:    public void init() throws IOException, InterruptedException {
src/test/java/com/teragrep/rlp_03/MultiClientTest.java:    public void cleanup() throws InterruptedException {
src/test/java/com/teragrep/rlp_03/MultiClientTest.java:    private void testSendMessage() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/MultiClientTest.java:    private void testSendBatch() throws IllegalStateException, IOException,
src/test/java/com/teragrep/rlp_03/ServerShutdownTest.java:    public void testServerShutdownSingleThread() throws IOException, InterruptedException {
src/test/java/com/teragrep/rlp_03/ServerShutdownTest.java:    public void testServerShutdownMultiThread() throws IOException, InterruptedException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void init() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void testSendMessage() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void testSendSmallMessage() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void testOpenAndCloseSession() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void testSessionCloseTwice() throws IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void clientTestOpenSendClose() throws IllegalStateException, IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void clientTestSendTwo() throws IllegalStateException, IOException, TimeoutException, InterruptedException {
src/test/java/com/teragrep/rlp_03/SendMessageTest.java:    public void testSendBatch() throws IllegalStateException, IOException, TimeoutException {
src/test/java/com/teragrep/rlp_03/CloseByteConsumerTest.java:        public void close() throws Exception {
src/test/java/com/teragrep/rlp_03/CloseByteConsumerTest.java:    public void init() throws IOException, InterruptedException {
src/test/java/com/teragrep/rlp_03/CloseByteConsumerTest.java:    public void cleanup() throws InterruptedException {
src/test/java/com/teragrep/rlp_03/CloseByteConsumerTest.java:    public void testSendMessage() throws IOException, TimeoutException, InterruptedException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/CustomTrustManager.java:            throws CertificateException {
src/test/java/com/teragrep/rlp_03/tls/SSLContextWithCustomTrustAndKeyManagerHelper.java:    public static SSLContext getSslContext() throws NoSuchAlgorithmException {
src/test/java/com/teragrep/rlp_03/TlsClientTest.java:        ) throws GeneralSecurityException, IOException {
src/test/java/com/teragrep/rlp_03/TlsClientTest.java:    public void init() throws IOException, GeneralSecurityException, InterruptedException {
src/test/java/com/teragrep/rlp_03/TlsClientTest.java:    public void cleanup() throws InterruptedException {
src/test/java/com/teragrep/rlp_03/TlsClientTest.java:    public void testTlsClient() throws IOException, TimeoutException, GeneralSecurityException {
src/test/java/com/teragrep/rlp_03/TearDownTest.java:    public void init() throws InterruptedException, IOException {
src/test/java/com/teragrep/rlp_03/TearDownTest.java:    public void cleanup() throws InterruptedException {
src/test/java/com/teragrep/rlp_03/TearDownTest.java:    public void testDirtyClosureAndReopen() throws IOException,

Inconsistencies in LoggerFactory.getLogger calls

Describe the bug

src/main/java/com/teragrep/rlp_03/context/buffer/BufferLeaseImpl.java: private static final Logger LOGGER = LoggerFactory.getLogger(BufferLease.class); -> BufferLease.class instead of BufferLeaseImpl.class like in other *Impl classes

src/test/java/com/teragrep/rlp_03/ConnectionStormTest.java: private static final Logger LOGGER = LoggerFactory.getLogger(SendMessageTest.class); -> Wrong logger class

src/test/java/com/teragrep/rlp_03/ManualPerformanceTest.java: private static final Logger LOGGER = LoggerFactory.getLogger(ManualTest.class); -> Wrong logger class

src/test/java/com/teragrep/rlp_03/ManualPerformanceTest.java: private static final Logger LOGGER = LoggerFactory.getLogger(FrameConsumer.class); -> Wrong logger class Subclasses, are as expected

src/test/java/com/teragrep/rlp_03/ManualPerformanceTest.java: private static final Logger LOGGER = LoggerFactory.getLogger(Reporter.class); -> Wrong logger class Subclasses, are as expected

Expected behavior

Loggers are consistent

How to reproduce

grep -Ri "LoggerFactory.getLogger" src/

Screenshots

Software version

Desktop (please complete the following information if relevant):

  • OS:
  • Browser:
  • Version:

Additional context

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.