Giter Club home page Giter Club logo

one-nio's Introduction

one-nio Javadocs

one-nio

one-nio is a library for building high performance Java servers. It features OS capabilities and JDK internal APIs essential for making your high load applications get maximum of the underlying system.

one-nio highlights

  • Optimized native socket library.
  • API for managing gigabytes of RAM beyond Java Heap.
  • Fast and compact serialization mechanism.
  • RPC client/server technology which is an order of magnitude faster than Java RMI.
  • Java accessors for perf events and eBPF.

More info on the wiki page

https://github.com/odnoklassniki/one-nio/wiki

one-nio's People

Contributors

1ou avatar apangin avatar avrecko avatar dependabot[bot] avatar dormidon avatar genuss avatar incubos avatar jarviscraft avatar mmityushkin avatar svladykin avatar svmironov avatar vlad-bondarenko avatar ymakarov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

one-nio's Issues

undefined symbol: OPENSSL_init_ssl

Hi!
I built my app with one-nio 1.2.0 and it doesn't start. In logs I can see:
/usr/bin/java: symbol lookup error: /tmp/libonenio.97307c0a.so: undefined symbol: OPENSSL_init_ssl
Both servers: where I built and where I tryed to start, have openssl version 1.0.2k-fips
What I should do?

java.time bug

    byte[] res;
    try (PersistStream stream = new PersistStream()) {
      Instant now = Instant.now();
      stream.writeObject(now);
      stream.writeObject(now);
      res = stream.toByteArray();
    }
    try (DeserializeStream des = new DeserializeStream(res)) {
      System.out.println(des.readObject());
      System.out.println(des.readObject());
    }

Output:

2022-07-13T06:21:56.517Z
null

java.util.Locale support

Do you plan to support java.util.Locale serialization? Or should I use my own Serializer<Locale>?

one-nio version >=1.6.1 doesn't work with glibc 2.31

I have stock Debian 11 with kernel 5.10.162, compiled with glibc 2.31.
All versions of one-nio prior to 1.6.1 worked on my setup.
But since 1.6.1 it requires at least glibc 2.34 (ldd libonenio.so says "version `GLIBC_2.34' not found").
Please consider rebuild project with old version of glibc.

json deserialization numbers to string

I have a field defined as String id. Most of the time it is a number but sometimes it is a String. The received json is sometimes formatted as number "id":123 and sometimes as string "id":"foo". Serialization fails as it expects a String not a number.

How do I handle this case in code? I'd like the number to be coverted to String. Do I use a custom @SerializeWith?

Can't acquire a socket at the first time after the peer restarted or the remote socket closed

The RpcClient can't acquire a socket at the first time after the peer restarted, because the IOException is thrown when the client invokes the remote method, and the RpcClient only catch SocketException instead of IOException in one.nio.rpc.RpcClient:

    private byte[] invokeRaw(Object request, int timeout) throws Exception {
        byte[] buffer = serialize(request);

        Socket socket = borrowObject();
        try {
            try {
                // Issue: stale connection(eg. the remote has been restored after the peer closed)
                //and IOException occurs but can't be catched
                sendRequest(socket, buffer, timeout);
            } catch (SocketException e) {
                // Stale connection? Retry on a fresh socket
                destroyObject(socket);
                socket = createObject();
                sendRequest(socket, buffer, timeout);
            }

But the sendRequest() method can throw IOException after the peer socket closed:

   private void sendRequest(Socket socket, byte[] buffer, int timeout) throws IOException {
        if (timeout != 0) socket.setTimeout(timeout);
        socket.writeFully(buffer, 0, buffer.length);
        socket.readFully(buffer, 0, 4);
    }

This issue also exists in one.nio.http.HttpClient.invoke() method:

    public Response invoke(Request request) throws InterruptedException, PoolException, IOException, HttpException {
        int method = request.getMethod();
        byte[] rawRequest = request.toBytes();
        ResponseReader responseReader;

        Socket socket = borrowObject();
        boolean keepAlive = false;
        try {
            try {
                // Issue: stale connection(eg. the remote has been restored after the peer closed) 
                //and IOException occurs but can't be catched
                socket.writeFully(rawRequest, 0, rawRequest.length);
                responseReader = new ResponseReader(socket, BUFFER_SIZE);
            } catch (SocketException e) {
                // Stale connection? Retry on a fresh socket
                destroyObject(socket);
                socket = createObject();
                socket.writeFully(rawRequest, 0, rawRequest.length);
                responseReader = new ResponseReader(socket, BUFFER_SIZE);
            }

BigInteger problem

Serialization BigInteger with different versions of Java produces different result

    byte[] res;
    try (PersistStream stream = new PersistStream()) {
      stream.writeObject(new BigInteger("123"));
      res = stream.toByteArray();
    }
    System.out.println(res.length + " bytes, hash=" + Arrays.hashCode(res));

Java 8 output:
141 bytes, hash=-1637736666
Java 11 and 17 output:
169 bytes, hash=1514769574

If BigInteger serialized with Java 8 and deserialized with Java 11

    try (DeserializeStream des = new DeserializeStream(res)) {
      System.out.println(des.readObject());
    }

Output:

Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitCount
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.bitLength
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.lowestSetBit
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNum
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitCountPlusOne
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.bitLengthPlusOne
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.lowestSetBitPlusTwo
Jul 15, 2022 4:14:09 PM one.nio.serial.Repository logFieldMismatch
WARNING: [1b5f8d93540f6312] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
123

If vice versa
Output:

Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitCountPlusOne
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.bitLengthPlusOne
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.lowestSetBitPlusTwo
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Stream field is missed locally: int java.math.BigInteger.firstNonzeroIntNumPlusTwo
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitCount
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.bitLength
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.lowestSetBit
Jul 15, 2022 4:14:59 PM one.nio.serial.Repository logFieldMismatch
WARNING: [4b1f962d58472cfa] Local field is missed in stream: int java.math.BigInteger.firstNonzeroIntNum
123

Вэб-сокеты

Добрый день!

Планируется ли поддержка вэб-сокетов, как сервера так и клиента с возможностью реконнекта при сетевых проблемах?

Register fields types for loading type

For more convinient to use the serial package will be nise when library generate code for some type to generate code for all types of fields of this class.

Example:
I have class

class MyDto {
        private BigDecimal decimal;
    }

and 2 JVMs between which will be data exchange. (First - serialize, second - deserialize)

Even if I preregister this class on both JVM like

Repository.preload(MyDto.class);

second jvm will not be able to deserialize a data because there is no a serializer for BigDecimal.
I know that we can request a schema but it's no needed here we have one already.

Tutorial/Docs in-memory off-heap

Please make a tutorial and docs: how to create in-memory (off-heap) objects, how to allocate, how to serialize/deserialize (if need, i can't understand) this to byte array (located off-heap), how to change it fields.
For example now (in current system) I have class
User
{
long Id;
List[ItemA] ItemsA; // class ItemA { .. }
List[ItemB] ItemsB; // class ItemB { .. }
// and many more field
}

// and global collection
static HashMap[long, User] AllUsers;

// .. dynamically created users, dynamically filled ItemsA, B, ..

as I understand, I need use SharedMemoryMap.java or subclasses, but can't understend how.
and how to allocate new User, it's new ItemA, how to put, get to User.ItemsA collection.

is it enough to replace my static HashMap[long, User] AllUsers into SharedMemoryMap[long, User] or not?


should I see https://github.com/odnoklassniki/shared-memory-cache ? can't understand where is the full cache/allocate code.

Selector.wakeup missing

The java.nio.channels.Selector has a wakeup method so that externally the thread waiting on the selector can be notified. E.g. it can be used to signal the io thread to start sending some payload over the network.

The one.nio.net.Selector doesn't have a wakeup. This means that one-nio can't be used as a replacement for networking code that relies on the wakeup.

Pool checker unhappy: borrowObject() and returnObject() in Pool only operate the first object

In the one.nio.pool.Pool, the object allocation and deallocation only occurs in the head of the pool. This way or strategy leads to other objects can't be checked in a pool heartbeat checker of application:

 public final T borrowObject() throws PoolException, InterruptedException {
        synchronized (this) {
            for (long timeLimit = 0; ; ) {
                // First try to get an idle object from the queue
                T object = pollFirst();
                if (object != null) {
                    return object;
                }
    public final void returnObject(T object) {
        synchronized (this) {
            if (!closed) {
                if (waitingThreads > 0) notify();
                addFirst(object); // Using addLast() for pool checker friendly
                return;
            }
        }
        destroyObject(object);
    }

Suggest that Using addLast() instead of addFirst() in Pool.returnObject(), and no performance loss in LinkedList.

@JsonName illegal field "name"

In last rev:351d9d3490023c1c87d6d41ba1d5c03448ed055b change field in @JsonName, "name" to "value", but tests(JsonTest) not changed

Java 22 - error serialization and deserialization BigDecimal

Example:
var byt = one.nio.serial.Serializer.persist(BigDecimal.ZERO);
Error:

Exception in thread "main" java.lang.VerifyError: Bad invokespecial instruction: current class isn't assignable to reference class.
Exception Details:
  Location:
    sun/reflect/Delegate0_BigDecimal.write(Ljava/lang/Object;Lone/nio/serial/DataStream;)V @9: invokespecial
  Reason:
    Error exists in the bytecode
  Bytecode:
    0000000: 2bc0 0015 4c2b b200 1bb7 001f 2c2b b400
    0000010: 23b6 0031 2c2b b400 34b6 0038 b1       

	at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3572)
	at java.base/java.lang.Class.getConstructor0(Class.java:3777)
	at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2955)
	at one.nio.serial.gen.DelegateGenerator.instantiate(DelegateGenerator.java:108)
	at one.nio.serial.GeneratedSerializer.<init>(GeneratedSerializer.java:57)
	at one.nio.serial.Repository.generateFor(Repository.java:379)
	at one.nio.serial.Repository.get(Repository.java:182)
	at one.nio.serial.PersistStream.writeObject(PersistStream.java:52)
	at one.nio.serial.Serializer.persist(Serializer.java:140)

Version one-nio: 1.7.1

JDK16 SelectableJavaSocket illegal reflective access

Just want to mention that in JDK 16, illegal-access=deny is the default, this means that getPollMethodHandle will return null.

It is possible to just use illegal-access=permit but would be nicer not to need to add a custom JVM parameter.

Serialization of java.time.* classes doesn't work out of the box

At the moment the read method of for example LocalDate is generated to something like this

public final Object read(DataStream var1) throws IOException, ClassNotFoundException {
    LocalDate var2;
    var1.register(var2 = new LocalDate);
    JavaInternals.unsafe.putInt(var2, 12L, var1.readInt());
    JavaInternals.unsafe.putShort(var2, 16L, var1.readShort());
    JavaInternals.unsafe.putShort(var2, 18L, var1.readShort());
    var2.readObject(NullObjectInputStream.INSTANCE);
    return var2;
}

But the LocalDate#readObject will throw InvalidObjectException because it uses serialization proxy java.time.Ser. The simple solution is to exclude calling of this method from generated code like this:

Repository.setOptions(LocalDate.class, Repository.SKIP_READ_OBJECT);
// ... same for all java.time.* classes

However it could be made in a general way if we check for presence of writeReplace method and exclude readObject automatically.
Do you have plans for this? I suppose I can make a PR, it looks like a simple condition in DelegateGenerator#generateRead.

reactive-streams

does one-nio framework has plans to support reactive-streams?

Http paramers parse error

Hi!

Create web app and use code:
@path("/organization")
public Response organization(Request request, HttpSession session) throws IOException {
Response response;
try {
if (request.getMethod() == Request.METHOD_GET) {
String name = request.getParameter("name");
String legalName = request.getParameter("legal_name");
String region = request.getParameter("region");
String locality = request.getParameter("locality");
String address = request.getParameter("address");

Method request.getParameter(String s) return wrong value.

Thanks.

Socket multicast support

I find the Socket is missing join/leave multicast support. Any plans to add it? Why was this left out? Would it be fine if this gets included?

I made a quick try without native support here 65bc2aa. Would the API changed be acceptable?

Exception in serialize java.sql.Time and java.util.Date

This test is failed:

package one.nio.serial;

/*imports {...}  */

public class SerializationTest extends TestCase {

    /*  others test-cases {...} */

    static class SqlDatetime implements Serializable {
        private final java.sql.Date date;
        private final java.sql.Time time;

        SqlDatetime(java.sql.Date date, java.sql.Time time) {
            this.date = date;
            this.time = time;
        }

        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (o == null || getClass() != o.getClass()) return false;
            SqlDatetime sqlDatetime = (SqlDatetime) o;
            return Objects.equals(date, sqlDatetime.date) &&
                    Objects.equals(time, sqlDatetime.time);
        }
    }

    public void testSqlDatetime() throws IOException, ClassNotFoundException {
        checkSerialize(new SqlDatetime(new java.sql.Date(System.currentTimeMillis()),
                                       new java.sql.Time(System.currentTimeMillis())));
    }

}

with an exception:

java.io.IOException: writeObject() is not fully supported. See implementation notes.

	at one.nio.serial.gen.NullObjectOutputStream.unsupported(NullObjectOutputStream.java:159)
	at one.nio.serial.gen.NullObjectOutputStream.writeLong(NullObjectOutputStream.java:130)
	at java.util.Date.writeObject(Date.java:1323)
	at sun.reflect.Delegate1_Date.calcSize(Unknown Source)
	at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:115)
	at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:122)
	at sun.reflect.Delegate0_SqlDatetime.calcSize(Unknown Source)
	at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:115)
	at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:122)
	at one.nio.serial.SerializationTest.clone(SerializationTest.java:38)
	at one.nio.serial.SerializationTest.checkSerialize(SerializationTest.java:84)
	at one.nio.serial.SerializationTest.testSqlDatetime(SerializationTest.java:391)

That's because one-nio generates Serializer like this (decompile to source):

public final class Delegate6_Date extends MagicAccessorImpl implements Delegate {
    public Delegate6_Date() {
    }

    public final void calcSize(Object var1, CalcSizeStream var2) throws IOException {
        ((Date)var1).writeObject(NullObjectOutputStream.INSTANCE);
    }

    public final void write(Object var1, DataStream var2) throws IOException {
        ((Date)var1).writeObject(NullObjectOutputStream.INSTANCE);
    }

    public final Object read(DataStream var1) throws IOException, ClassNotFoundException {
        java.sql.Date var2;
        var1.register(var2 = new java.sql.Date);
        var2.readObject(NullObjectInputStream.INSTANCE);
        return var2;
    }
}

and NullObjectOutputStream throws this exception.

Not starting server

I am doing a course project in the project's boundaries "Technopolis" (https://github.com/kilinochi/2019-highload-dht) and I have a problem with server - he is not running.

My main class

` public final class Server {
private static final int PORT = 8080
private Server() {
// Not instantiable
}

public static void main(String[] args) throws IOException {
    // Temporary storage in the file system
    final File data = Files.createTempDirectory();

    // Start the service
    final DAO dao = DAOFactory.create(data);
    final Service service =
            ServiceFactory.create(
                    PORT,
                    dao);
    service.start();
    Runtime.getRuntime().addShutdownHook(
            new Thread(() -> {
                service.stop();
                try {
                    dao.close();
                } catch (IOException e) {
                    throw new RuntimeException("Can't close dao", e);
                }
            }));
}

}`

and RestService class, which implement Service

`public final class RestService extends HttpServer implements Service {

private static final Logger logger = LoggerFactory.getLogger(RestService.class);
private static final String API = "/v0/entity";

private final DAO dao;

public RestService(final int port, @NotNull final DAO dao) throws IOException {
    super(getConfig(port));
    logger.info("Server is running on port " + port);
    this.dao = dao;
}

@Override
public void start() {

}

@Override
public void stop() {

}

@Path(API)
public Response entity (@Param("id") final String id, final Request request) {

    if(id == null || id.isEmpty()) {
        return new Response(Response.BAD_REQUEST, "Key not found".getBytes(Charsets.UTF_8));
    }

    final ByteBuffer key = ByteBuffer.wrap(id.getBytes(Charsets.UTF_8));

    logger.info("Request " + request.getMethod() + " with param: " + id);

    try {
        switch (request.getMethod()) {
            case Request.METHOD_GET:
                final ByteBuffer value = dao.get(key);
                final ByteBuffer duplicate = value.duplicate();
                byte [] body = new byte[duplicate.remaining()];
                duplicate.get(body);
                return new Response(Response.OK, body);
            case Request.METHOD_PUT:
                dao.upsert(key, ByteBuffer.wrap(request.getBody()));
                return new Response(Response.CREATED, Response.EMPTY);
            case Request.METHOD_DELETE:
                dao.remove(key);
            default:
                return new Response(Response.METHOD_NOT_ALLOWED, Response.EMPTY);
        }
    }
    catch (IOException e) {
        return  new Response(Response.INTERNAL_ERROR, Response.EMPTY);
    }
    catch (NoSuchElementException e) {
        return new Response(Response.NOT_FOUND, "Key not found".getBytes(Charsets.UTF_8));
    }
}

private static HttpServerConfig getConfig(final int port) {
    if(port <= 1024 || port >= 65535) {
        throw new IllegalArgumentException("Invalid port");
    }
    AcceptorConfig acceptorConfig = new AcceptorConfig();
    acceptorConfig.port = port;
    HttpServerConfig config = new HttpServerConfig();
    config.acceptors = new AcceptorConfig[]{acceptorConfig};
    return config;
}

} `

So, out all of this
Снимок экрана от 2019-09-29 20-01-54

And server not running...

Warning for unused classes

Hi!

After running my app, output to log:
мар 31, 2018 5:35:31 PM one.nio.serial.Repository addOptionalBootstrap
WARNING: Missing optional bootstrap class: one.app.remote.reflect.MethodId
мар 31, 2018 5:35:31 PM one.nio.serial.Repository addOptionalBootstrap
WARNING: Missing optional bootstrap class: one.app.remote.comp.RemoteMethodCallRequest

I see source code and found row on one.nio.serial.Repository:
addOptionalBootstrap("one.app.remote.reflect.MethodId");
addOptionalBootstrap("one.app.remote.comp.RemoteMethodCallRequest");

This rows is garbage?

Thanks.

HttpClient extra whitespace break Transfer-Encoding chunked header

As in the title. We noticed that if the server returns some extra whitespace for transfer-encoding header this condition isn't met.

  if (method != Request.METHOD_HEAD && mayHaveBody(response.getStatus())) {
                if ("chunked".equalsIgnoreCase(response.getHeader("Transfer-Encoding: "))) {
                    response.setBody(readChunkedBody());
                } else { 

BigDecimal with new Java does not work any more

Deserialization BigDecimal with Java 1.8.0_351 does not work any more:

java.io.IOException: readObject() is not fully supported. See implementation notes.
	at one.nio.serial.gen.NullObjectInputStream.unsupported(NullObjectInputStream.java:170)
	at one.nio.serial.gen.NullObjectInputStream.readFields(NullObjectInputStream.java:50)
	at java.math.BigDecimal.readObject(BigDecimal.java:3798)
	at sun.reflect.Delegate14_BigDecimal.read(Unknown Source)
	at one.nio.serial.GeneratedSerializer.read(GeneratedSerializer.java:127)
	at one.nio.serial.DeserializeStream.readObject(DeserializeStream.java:75)
...

Http path

Добрый день!

Как реализовать в Router обработку http-запроса, в виде /service/1, /service/567565 (шаблон /service/{id})?
Есть аннотация @path, нужно ее использовать с указанием шаблона адреса?

Unable to connect to https://api.kucoin.com due to javax.net.ssl.SSLException: error:0A000410:SSL routines::sslv3 alert handshake failure

Hi folks. I am not sure if this is the right place to raise this but I thought I would create an issue none the less. Please feel free to move it to the appropriate place or close it out if this is not an issue.
I was attempting to use the HttpClient provided by one nio but I was unable to get it to connect to https://api.kucoin.com. The client works with other servers that I have tried with but for this one server, it fails. I tried with both one-nio-1.5.0.jar and the latest one-nio-1.6.1.jar. For 1.5.0, I used openssl1.1 and for 1.6.1, I used openssl3.0. I cloned the repo and built it myself and ensured that the libonenio.so object exists in the jar and then compiled it with my client code shown below.

import one.nio.http.HttpClient;
import one.nio.http.HttpException;
import one.nio.net.ConnectionString;
import one.nio.pool.PoolException;

import java.io.IOException;

public class MyHttpClient {

    public HttpClient client;

    public MyHttpClient(String hostPort) {
        this.client = new HttpClient(new ConnectionString(hostPort));
    }

    public void syncGet(String path) {
        try {
            System.out.println(this.client.get(path));
        } catch (InterruptedException | PoolException | IOException | HttpException e) {
            throw new RuntimeException(e);
        }
    }
}

However, I always ran into the following error when attempting to connect to the server:

Testing Kucoin connection
Exception in thread "main" java.lang.RuntimeException: one.nio.pool.PoolException: SocketPool[api.kucoin.com:443] createObject failed: javax.net.ssl.SSLException: error:0A000410:SSL routines::sslv3 alert handshake failure
	at util.http.MyHttpClient.syncGet(MyHttpClient.java:24)

From the error message, it looks like the issue might be with sslv3. It's possible that the server here doesn't support sslv3 but my question is why is one.nio/openssl using SSLV3 to connect to the server? I would have thought that it would use TLSv1.2. I tried to forcefully disable sslv3 in but was still unsuccessful. I modified ssl.c as shown below.
https://wiki.openssl.org/index.php/List_of_SSL_OP_Flags

src/one/nio/net/native/ssl.c

JNIEXPORT jlong JNICALL
Java_one_nio_net_NativeSslContext_ctxNew(JNIEnv* env, jclass cls) {
    AppData* appData = create_app_data();
    if (appData == NULL) {
        throw_by_name(env, "javax/net/ssl/SSLException", "Cannot allocate SSL app data");
        return 0;
    }

    SSL_CTX* ctx = SSL_CTX_new(TLS_method());
    if (ctx == NULL) {
        free_app_data(appData);
        throw_ssl_exception(env);
        return 0;
    }

    SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY | SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);
    fprintf(stderr, "\nSetting options to disable compression and ssl v3\n");
    unsigned long currOptions = SSL_CTX_get_options(ctx);
    fprintf(stderr, "\nCurrent options(pre set): %d\n", currOptions);
    SSL_CTX_clear_options(ctx, currOptions); // Disable TLSv1.3 compat mode.

    currOptions = SSL_CTX_get_options(ctx);
    fprintf(stderr, "\nCurrent options(post clear): %d\n", currOptions);
    currOptions |= SSL_OP_BIT(17); // Disable compression.
    currOptions |= SSL_OP_BIT(25); // Disable SSL V3.
    currOptions |= SSL_OP_BIT(29); // Disable TLSv1.3
    SSL_CTX_set_options(ctx, currOptions);
    fprintf(stderr, "\nCurrent options(post set): %d\n", SSL_CTX_get_options(ctx));
    fprintf(stderr, "\nSuccessfully set options to disable compression and ssl v3\n");


    // Disable read-ahead until premature socket close is fixed
    // SSL_CTX_set_read_ahead(ctx, 1);
    SSL_CTX_set_info_callback(ctx, ssl_info_callback);
    SSL_CTX_set_app_data(ctx, appData);

    setup_dh_params(ctx);
    setup_ecdh_params(ctx);

    return (jlong)(intptr_t)ctx;
}

The output looks as follows:

Testing Kucoin connection

Setting options to disable compression and ssl v3

Current options(pre set): 1179648

Current options(post clear): 0

Current options(post set): 570556416

Successfully set options to disable compression and ssl v3
Exception in thread "main" java.lang.RuntimeException: one.nio.pool.PoolException: SocketPool[api.kucoin.com:443] createObject failed: javax.net.ssl.SSLException: error:0A000410:SSL routines::sslv3 alert handshake failure
	at util.http.MyHttpClient.syncGet(MyHttpClient.java:24)

570556416 === 0b100010000000100000000000000000 (17th, 25th and 29th bit are set)

[ubuntu@ip-10-20-82-90 ~]$ openssl s_client -connect api.kucoin.com:443 -tls1_2
CONNECTED(00000003)
depth=1 C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = kucoin.com
verify return:1

....

No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2746 bytes and written 296 bytes
Verification error: unable to get local issuer certificate
---
New, TLSv1.2, Cipher is ECDHE-ECDSA-CHACHA20-POLY1305
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-CHACHA20-POLY1305

I would be really grateful if someone can help me out with this. Please do let me know if this is a known issue which has workarounds. I went through the issues posted previously but was unable to find any that matched this.

ClassNotFoundException is not serializable for Adoptium jdk 17

When RpcSession fails with deserialization of incoming request, in particular, when some method or class is missing on server side, it throws ClassNotFoundException and tries to RpcSession#writeResponse with this exception. But RpcSession#writeResponse fails too, because ClassNotFoundException cannot be serialized.

It leads to the following exception on server side:

16:23:18,095|W|[NIO Selector #2] Local type not found: EchoService$Message
16:23:18,117|E|[NIO Selector #2] Cannot process session from 127.0.0.1
java.io.IOException: writeObject() is not fully supported. See implementation notes.
	at one.nio.serial.gen.NullObjectOutputStream.unsupported(NullObjectOutputStream.java:159)
	at one.nio.serial.gen.NullObjectOutputStream.putFields(NullObjectOutputStream.java:50)
	at java.base/java.lang.ClassNotFoundException.writeObject(ClassNotFoundException.java:144)
	at sun.reflect.Delegate0_ClassNotFoundException.calcSize(Unknown Source)
	at one.nio.serial.GeneratedSerializer.calcSize(GeneratedSerializer.java:116)
	at one.nio.serial.CalcSizeStream.writeObject(CalcSizeStream.java:129)
	at one.nio.rpc.RpcSession.writeResponse(RpcSession.java:196)
	at one.nio.rpc.RpcSession.handleDeserializationException(RpcSession.java:250)
	at one.nio.rpc.RpcSession.processRead(RpcSession.java:108)
	at one.nio.net.Session.process(Session.java:222)
	at one.nio.server.SelectorThread.run(SelectorThread.java:70)

  • Both client and server runs on openjdk version "17.0.9" 2023-10-17
  • Checked on one-nio 1.6.1 and 1.7.1

Little get started for serializer

Is there any tutorial 'hot to use serializer'?

I have serveral questions:

  1. Did I right understand that a work do through
  • one.nio.serial.Serializer#serialize
  • one.nio.serial.Serializer#deserialize
    Is it the best (for perfomance) way?
  1. What is the difference between persist and serialize?
  2. You told that there is a feature to request a schema. I didn't find it =(
  3. Can I add a custom Serializer for my type?

Thanks!

UndeclaredThrowableException thrown when RpcClient service as a bean in spring boot application

When I use RpcClient proxy service as a spring bean in spring boot, the java.lang.reflect.UndeclaredThrowableException is thrown in application bootstrap. A demo stack trace:

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
        at com.sun.proxy.$Proxy0.hashCode(Unknown Source)
        at java.util.HashMap.hash(HashMap.java:339)
        at java.util.HashMap.put(HashMap.java:612)
        at EchoClient.main(EchoClient.java:57)
Caused by: one.nio.pool.PoolException: SocketPool[localhost:9000] createObject failed
        at one.nio.pool.SocketPool.createObject(SocketPool.java:145)
        at one.nio.pool.SocketPool.createObject(SocketPool.java:24)
        at one.nio.pool.Pool.borrowObject(Pool.java:93)
        at one.nio.rpc.RpcClient.invokeRaw(RpcClient.java:100)
        at one.nio.rpc.RpcClient.invoke(RpcClient.java:43)
        at one.nio.rpc.RpcClient.invoke(RpcClient.java:73)
        ... 4 more
Caused by: java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:118)
        at one.nio.net.JavaSocket.connect(JavaSocket.java:63)
        at one.nio.net.Socket.connect(Socket.java:76)
        at one.nio.pool.SocketPool.createObject(SocketPool.java:135)
        ... 9 more

The demo likes this:

    public static void main(String[] args) throws Exception {
        Map<EchoService, EchoService> services = new HashMap<>();
		
        ConnectionString conn = new ConnectionString(args[0]);
        EchoService client = (EchoService) Proxy.newProxyInstance(
                EchoClient.class.getClassLoader(),
                new Class[] {EchoService.class},
                new RpcClient(conn));
		
        services.put(client, client); //  UndeclaredThrowableException thrown here
        // ... emitted

java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync

Привет!

Получаю ошибку при попытке десериализовать класс у которого есть поле типа ReentrantLock. Следующий код

DeserializeStream stream = new DeserializeStream( bytes );
return ( T )stream.readObject();

выбрасывает exception java.lang.IllegalAccessError: java/util/concurrent/locks/ReentrantLock$Sync

SharedMaps: how use correctly

Good afternoon, comrades, tell me about the use of distributed cards.
I have two test app instances that declare a shared map across 1 and the same file.

When writing data from the first application to the SharedMemoryStringMap and receiving, I see the recorded data.

I'm launching a second application that tries to read data using a key that was written by the first application in SharedMemoryStringMap. As a result, I get a zero result.

The second application can see the data in the file only after the first application calls the map.close () method.

Tell me what can I do wrong?

Current use case:

JVM 1:

        final String file = "./data.dat";

        final SharedMemoryStringMap<String> map = new SharedMemoryStringMap<>(65535, file, 4 * M);
        map.setSerializer(String.class);

        map.put("12345", "test-shared-str");

        String s = map.get("12345");
        System.out.println(s);

       // start daemon thread ...

JVM 2:

        final String file = "./data.dat";

        final SharedMemoryStringMap<String> map = new SharedMemoryStringMap<>(65535, file, 4 * M);
        map.setSerializer(String.class);

        String s = map.get("12345");  <-- null value
        System.out.println(s);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.