Giter Club home page Giter Club logo

wrends's Introduction

Wren:DS logo

Wren:DS

License Gitter

Wren:DS is an LDAPv3 compliant directory service, which has been developed for the Java platform, providing a high performance, highly available, and secure store for the identities managed by your organization. Its easy installation process, combined with the power of the Java platform makes Wren:DS the simplest, fastest directory to deploy and manage.

Wren:DS is one of the projects in the Wren Security Suite, a community initiative that adopted open‐source projects formerly developed by ForgeRock, which has its own roots in Sun Microsystems’ products.

Contributions

Contributing Guide Contributors Pull Requests Last commit

Getting the Wren:DS

You can get Wren:DS application in couple of ways:

Download binary release

The easiest way to get the Wren:DS is to download the latest binary release.

Build the source code

In order to build the project from the command line follow these steps:

Prepare your Environment

Following software is needed to build the project:

Software Required Version
OpenJDK 8 and above
Git 2.0 and above
Maven 3.0 and above

Build the source code

All project dependencies are hosted in JFrog repository and managed by Maven, so to build the project simply execute Maven package goal.

$ cd $GIT_REPOSITORIES/wrends
$ mvn clean package

Built binary can be found in ${GIT_REPOSITORIES}/wrends/opendj-server-legacy/target/package/wrends-${VERSION}.zip.

Docker image

You can also run Wren:DS in a Docker container. Official Wren:DS Docker images can be found here.

Documentation

Project documentation can be found in our documentation platform (docs.wrensecurity.org).

Acknowledgments

Large portions of the source code are based on the open-source projects previously released by:

  • Sun Microsystems
  • ForgeRock

We'd like to thank them for supporting the idea of open-source software.

Disclaimer

Please note that the acknowledged parties are not affiliated with this project. Their trade names, product names and trademarks should not be used to refer to the Wren Security products, as it might be considered an unfair commercial practice.

Wren Security is open source and always will be.

wrends's People

Contributors

aldaris avatar austingene avatar brunolavitforgerock avatar chrisridd avatar csovant avatar fabiop-fr avatar forgerocker avatar fyrbach avatar gaetan-fr avatar gary-williams38 avatar glennongit avatar imanja0 avatar jamesphillpotts-fr avatar jnrouvignac avatar karelmaxa avatar kortanul avatar ludomp avatar markcraig avatar matthew-swift avatar minnoroth avatar mkofahl avatar ncapponi avatar nicolasc-fr avatar patrickdiligentforgerock avatar pavelhoral avatar pvarga88 avatar robobik avatar sberthol avatar violette avatar ylecaillez avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wrends's Issues

Wren:DS in Docker fails to start with existing instance data

Summary

Wren:DS in Docker (release 5.0.0) attempts to initialize every time, even when instance dir is already populated, which results in error on subsequent starts.

Steps To Reproduce ("Repro Steps")

  1. Run Wren:DS with empty volume for instance data: docker run --rm -v wrends-data:/opt/wrends/instance wrensecurity/wrends:5.0.0, wait for it to initialize.
  2. Stop Wren:DS (Ctrl+C)
  3. Run the same command again (re-using the now populated volume)

Alternatively, instead of using volumes, create a container, start it, stop it and start it again.

Expected Result (Behavior You Expected to See)

Wren:DS start with existing instance data which has been created during the first start.

Actual Result (Behavior You Saw)

After following repro steps above, Wren:DS fails to start with the following output:

$ docker run --rm -v wrends-data:/opt/wrends/instance wrensecurity/wrends:5.0.0
First start...
Server Already Configured
setup command-line can only be used with servers that have not yet been
configured. The current server:
- Contains data
- Has already been configured

Screenshots
N/A

Environment

  • Product version (include GIT commit ID if using a dev build): Image from Docker Hub, version 5.0.0.
  • Operating system: N/A (Docker)
  • JRE version: N/A (Docker)
  • Was instance upgraded from a previous version? If so, what version were you using before? N/A

Additional Notes

The problem lies in unset variable INSTANCE_DIR here:

if [ ! -d $INSTANCE_DIR/config ]; then
.

Unable to build on Windows Adoptium / Temurin (error configuring jdk.xml.xpathExprGrpLimit)

Getting the following exception when building the codebase:

Caused by: java.lang.IllegalArgumentException: TransformerFactory does not recognise attribute 'jdk.xml.xpathExprGrpLimit'.
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.setAttribute (TransformerFactoryImpl.java:541)
    at org.forgerock.opendj.maven.GenerateConfigMojo.initializeStylesheets (GenerateConfigMojo.java:400)
    at org.forgerock.opendj.maven.GenerateConfigMojo.execute (GenerateConfigMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
$ mvn -version
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: N:\apps\apache-maven-3.8.1
Java version: 17.0.1, vendor: Eclipse Adoptium, runtime: C:\Program Files\Eclipse Adoptium\jdk-17.0.1.12-hotspot
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

This is connected with #41. Nut sure why and how that setting does not work. Might need someone else to confirm this is a real issue and not just some inconsistency in my Windows environment.

Simplify connection handler names

Summary

Connection handlers have too long and descriptive name with Connection Handler suffix. Connection handler names are used when referencing the handler so this makes all commands unnecessarily more complex.

Connection Handler       : Type : enabled : listen-port : use-ssl
-------------------------:------:---------:-------------:--------
HTTP Connection Handler  : http : false   : 8080        : false
JMX Connection Handler   : jmx  : false   : 1689        : false
LDAP Connection Handler  : ldap : true    : 1389        : false
LDAPS Connection Handler : ldap : true    : 1636        : true
LDIF Connection Handler  : ldif : false   : -           : -

Solution You'd Like to See

I would like to remove Connection Handler suffix and leave only the protocol name as the handler name.

Workarounds/Alternatives

None.

Additional Notes

None.

After uninstalling this service, all created user data cannot be cleared

Hello, I would like to ask where the user data created by this service is currently stored. After I uninstalled it, I found that the user is still not deleted. After I reinstalled, the same user cannot be created successfully, and the prompt already exists and cannot be added. , how to deal with this problem?

Unable to build with precommit profile

Unit tests in opendj-server-legacy module are only executed with precommit maven profile. We don't have this profile in our CI pipeline which is probably mistake.

We need to fix unit tests so that they are passing and add this profile to CI pipeline.

Rest2LDAP requests hang indefinitely if an async request thread encounters a RuntimeException

Summary

If a background request thread encounters a RuntimeException (NullPointerException, IllegalStateException, etc), that thread silently dies off, leaving the main request thread blocking idefinitely on an Object.wait() call.

Affected Version

master (4.0.0-SNAPSHOT)

Steps to Reproduce

One possible way to reproduce this was mentioned in my post on the FR forums:
https://forum.forgerock.com/topic/possible-bug-in-crest-when-using-clientnaming/#post-17428

A slightly easier way is to simply modify the read() method of JsonConstantPropertyMapper to throw a NullPointerException inside the promsise, as follows:

    Promise<JsonValue, ResourceException> read(final Context context, final Resource resource,
                                               final JsonPointer path, final Entry entry) {
        final Promise<JsonValue, ResourceException> resultPromise =
            newResultPromise(new JsonValue(null));

        return resultPromise.then(
          new Function<JsonValue, JsonValue, ResourceException>() {
              @Override
              public JsonValue apply(JsonValue jsonValue) throws ResourceException {
                  throw new NullPointerException("Test");
              }
          });
    }

Then run the BasicRequestsTest.

Expected Results

A request that unexpectedly encounters a RuntimeException while assembling a response returns an appropriate message along with a 500 Internal Server error.

Actual Results

The request hangs indefinitely. Multiple subsequent requests that each result in a runtime exception eventually cause the application server to become unresponsive because there are no more threads available in the thread pool. The only remedy is to restart the application server.

The main thread stack looks something like this:

	  at java.lang.Object.wait(Object.java:-1)
	  at java.lang.Object.wait(Object.java:502)
	  at org.forgerock.util.promise.PromiseImpl.await(PromiseImpl.java:618)
	  at org.forgerock.util.promise.PromiseImpl.getOrThrow(PromiseImpl.java:144)
	  at org.forgerock.json.resource.AbstractAsynchronousConnection.query(AbstractAsynchronousConnection.java:80)
	  at org.forgerock.json.resource.AbstractAsynchronousConnection.query(AbstractAsynchronousConnection.java:89)
	  at org.forgerock.opendj.rest2ldap.BasicRequestsTest.testQueryAll

The cause of this behavior appears to be that the asynchronous promise-based method Rest2LDAP uses to obtain results has no top-level exception handler on the result threads.

As a potential way to mitigate this issue, it may make sense to modify org.forgerock.json.resource.AbstractAsynchronousConnection.query() to use a sensible timeout (5 mins?). Note that this code changed substantially between json-resource-21.0.0-alpha-6 (which is what Wren has in source control) and json-resource-21.0.0-alpha-17 (which is only available in Wren's FR archive).

[3.5.1] Get DS 3.5.x checked-in, merged, and buildable

we have a CDDL copy of OpenDJ 3.5.x.

we should:

  1. create a sustaining/3.5 branch off of the forgerock/3.0.0 tag
  2. check-in our copy of 3.5.x on that branch.
  3. update the branch with build changes from sustaining/3.0 so that it can build.

PBKDF2-HMAC-SHA256 and the PBKDF-HMAC-SHA512 password encoding schemes

Summary

It would be good to have PBKDF2-HMAC-SHA256 and the PBKDF-HMAC-SHA512 password encoding schemes.

Solution You'd Like to See

Add these types will allow stronger password encryption & security. This seems to have been done a year ago on openidentity opendj open source project, so if the licences are compatible, it might just mean importing it here.

Workarounds/Alternatives

With openidentity opendj this works, provided you manually add the schema to schema/02-config.ldif, then these password types are available and function. The OID's need to be allocated uniquely as part of the build. Likely the DS schema && IODs got blatted / lost between major releases last year, the version just after openidentity/opendj which is described in PR #228 over there. ( the below dummy 9999* OIDs need to be made real )

This is the server schema missing on the openidentity opendj binaries :

$ diff config/schema/02-config.ldif config/schema/02-config.ldif.dist
5903,5914d5902
< objectClasses: ( 1.3.6.1.4.1.36733.2.1.2.99998
< NAME 'ds-cfg-pbkdf2-hmac-sha256-password-storage-scheme'
< SUP ds-cfg-pbkdf2-password-storage-scheme
< STRUCTURAL
< MAY ds-cfg-pbkdf2-iterations
< X-ORIGIN 'OpenDJ Directory Server' )
< objectClasses: ( 1.3.6.1.4.1.36733.2.1.2.99999
< NAME 'ds-cfg-pbkdf2-hmac-sha512-password-storage-scheme'
< SUP ds-cfg-pbkdf2-password-storage-scheme
< STRUCTURAL
< MAY ds-cfg-pbkdf2-iterations
< X-ORIGIN 'OpenDJ Directory Server' )

Additional Notes

This PR describes this change from the open identity project : OpenIdentityPlatform/OpenDJ#228

deprecate and remove Persistit Data Backend (PDB)

ForgeRock introduced the Persistit Data Backend (PDB) in OpenDJ 3.0.0 and deprecated it in DS 5 (i.e. OpenDJ 4.0.0).

There are multiple reasons why they abandoned it:

  • ForgeRock had to fork the project because Akiban was acquired and no longer maintains it.
  • Even after patches from FR, it has remaining issues that can cause deadlock in production,
  • Unit tests for Persistit are painfully slow and fail if you actually run them.
  • PDB is no longer needed because JE licensing is now compatible with OEM deployments of OpenDJ / Wren:DS. We still need to upgrade the JE backend – see #1.

OpenIDM Account Change Notification Handler Does Not Package Properly on Windows

Affected Versions

  • 3.5.1

Summary

If you build the Wren:DS project on Windows, the artifacts that get produced for the "OpenIDM Account Change Notification Handler" do not contain the correct files. This leads to duplicate JAR files inside the opendj-openidm-account-change-notification-handler-3.5.1.zip file, and a missing extensions manifest file inside the opendj-openidm-account-change-notification-handler-3.5.1.jar file.

Environment

  • Windows 10 - Version 1709 (OS Build 16299.309)
  • Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T14:49:05-05:00)

Steps to Reproduce

  1. Build Wren:DS 3.5.1 from source on a Windows 10 machine.
  2. Using the ZIP file under opendj-openidm-account-change-notification-handler/target, attempt to follow plug-in installation instructions from ForgeRock.

Note that step 5 in FR's documentation is incorrect; the command should be:

cd /path/to/opendj/bin
./ldapmodify \
 --port 389 \
 --bindDN "cn=Directory Manager" \
 --bindPassword "password" \
 --defaultAdd \
 --filename \
 ../config/openidm-accountchange-plugin-sample-config

Expected Results

  • The plug-in can be installed successfully (even with the default, sample config).

Actual Results

  • The ZIP file contains two JAR files under lib/extensions (opendj-openidm-account-change-notification-handler-3.5.1.jar and opendj-openidm-account-change-notification-handler-3.5.1-shaded.jar), when it should only contain one.
  • The JAR inside the ZIP file is missing the appropriate manifest file (org.forgerock.opendj.config.AbstractManagedObjectDefinition).
  • Attempting to use the JAR file without that manifest leads to the following error message when trying to install the plug-in:

The Directory Server is unwilling to add configuration entry cn=OpenIDM Notification Handler,cn=Account Status Notification Handlers,cn=config because one of the add listeners registered with the parent entry cn=Account Status Notification Handlers,cn=config rejected this change with the message: An error occurred while trying to initialize an instance of class org.forgerock.openidm.accountchange.OpenidmAccountStatusNotificationHandler as an account status notification handler as defined in configuration entry cn=OpenIDM Notification Handler,cn=Account Status Notification Handlers,cn=config: ClassCastException: org.forgerock.opendj.server.config.meta.AccountStatusNotificationHandlerCfgDefn$AccountStatusNotificationHandlerCfgServerImpl cannot be cast to org.forgerock.openidm.accountchange.server.OpenidmAccountStatusNotificationHandlerCfg (OpenidmAccountStatusNotificationHandler.java:221 AccountStatusNotificationHandlerConfigManager.java:360 AccountStatusNotificationHandlerConfigManager.java:225 AccountStatusNotificationHandlerConfigManager.java:48 ServerManagedObjectAddListenerAdaptor.java:66 ConfigAddListenerAdaptor.java:221 ConfigurationHandler.java:463 ConfigurationBackend.java:404 LocalBackendAddOperation.java:460 LocalBackendAddOperation.java:161 LocalBackendWorkflowElement.java:736 LocalBackendWorkflowElement.java:1051 LocalBackendWorkflowElement.java:894 AddOperationBasis.java:506 TraditionalWorkerThread.java:148)

The error results from Wren:DS not bootstrapping the configuration classes while loading the extension, which in turn means that the classes are not loaded by the time it needs to provide them to the extension. The extension tries to typecast the configuration to match the expected configuration type, but fails because Wren:DS is using only the generic configuration definition instead of the one for the plug-in.

Additional Information

The issue appears to be platform-specific. Packaging this on Ubuntu Linux 16.04 with Apache Maven 3.3.9 worked fine.

`testGetFreePort` fails in WSL on Windows

Summary

For some reason, on Java 8 in Ubuntu 18.04 running in WSL, the DS 3.x tests are failing for testGetFreePort. It looks like the JRE or Windows are allowing Java to bind two different server sockets to the same port (port sharing), which violates the test's setup expectations.

Steps To Reproduce ("Repro Steps")

  1. Checkout sustaining/3.0.
  2. Build the project with mvn clean install.

Expected Result (Behavior You Expected to See)

The build should succeed with no test failures.

Actual Result (Behavior You Saw)

The build fails with this error:

Full failure log
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
Configuring TestNG with: TestNG652Configurator

Test environment:

Java version: 1.8.0_191
Java vendor: Oracle Corporation
JVM name: OpenJDK 64-Bit Server VM
JVM version: 25.191-b12
JVM vendor: Oracle Corporation
JVM info: mixed mode
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
OS: Linux 4.4.0-17763-Microsoft amd64
Processors: 4
Max memory: 2837446656
Total memory: 192937984

How to read the progressive status info:

Test duration status: {Total min:sec. Since last status sec.}
Test count status: {# test classes # test methods # test method invocations # test failures}.
TestClass (the class that just completed)

{ 0:00 ( 0s)} { 0c 0m 0i 0f} : starting
{ 0:00 ( 0s)} { 1c 6m 6i 0f} : CertificateTestCase
{ 0:00 ( 0s)} { 2c 9m 9i 0f} : ProductInformationTest
{ 0:00 ( 0s)} { 3c 13m 21i 0f} : SetupCliTestCase
{ 0:00 ( 0s)} { 4c 16m 24i 0f} : DataConfigurationTestCase

             T E S T   F A I L U R E ! ! !

Failed Test: org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase#testGetFreePort
Failure Cause: java.lang.AssertionError: actual value:<61728> should not be equal to:<61728>
org.fest.assertions.Fail.failure(Fail.java:228)
org.fest.assertions.Fail.fail(Fail.java:218)
org.fest.assertions.Fail.failIfEqual(Fail.java:53)
org.fest.assertions.GenericAssert.isNotEqualTo(GenericAssert.java:228)
org.fest.assertions.IntAssert.isNotEqualTo(IntAssert.java:71)
org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase.testGetFreePort(ListenerSettingsTestCase.java:80)


{ 0:00 ( 0s)} { 5c 20m 28i 1f} : ListenerSettingsTestCase
{ 0:00 ( 0s)} { 6c 38m 46i 1f} : ModelTestCase
{ 0:00 ( 0s)} { 7c 43m 51i 1f} : RuntimeOptionsTestCase
The following unit tests failed:
org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase#testGetFreePort

Include the ant option '-Dtest.failures=true' to rerun only the failed tests.

Wrote full test report to:
/mnt/c/Users/guypa/Documents/WrenProjects/wrends/opendj-server/target/surefire-reports/results.txt
Test classes run interleaved: 0
Final amount of memory in use: 3.0 MB
Final number of threads: 1

Tests run: 51, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.897 sec <<< FAILURE! - in TestSuite
testGetFreePort(org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase) Time elapsed: 0.06 sec <<< FAILURE!
java.lang.AssertionError: actual value:<61728> should not be equal to:<61728>
at org.fest.assertions.Fail.failure(Fail.java:228)
at org.fest.assertions.Fail.fail(Fail.java:218)
at org.fest.assertions.Fail.failIfEqual(Fail.java:53)
at org.fest.assertions.GenericAssert.isNotEqualTo(GenericAssert.java:228)
at org.fest.assertions.IntAssert.isNotEqualTo(IntAssert.java:71)
at org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase.testGetFreePort(ListenerSettingsTestCase.java:80)

Results :

Failed tests:
ListenerSettingsTestCase.testGetFreePort:80 actual value:<61728> should not be equal to:<61728>

Tests run: 51, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.884 s
[INFO] Finished at: 2019-04-18T14:51:48Z
[INFO] Final Memory: 27M/252M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project opendj-server: There are test failures.
[ERROR]
[ERROR] Please refer to /mnt/c/Users/guypa/Documents/WrenProjects/wrends/opendj-server/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Environment

  • Product version (include GIT commit ID if using a dev build): 3.0.0 @ 3a60e66
  • Operating system:
    Ubuntu 18.04.1 LTS
    Linux 4.4.0-17763-Microsoft #379-Microsoft Wed Mar 06 19:16:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
    
  • JRE version:
    openjdk version "1.8.0_191"
    OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
    OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
    
  • Was instance upgraded from a previous version? If so, what version were you using before? No

Additional Notes

Looking into this issue revealed a separate issue with the way that the DS 3.x SDK looks for a free port. A separate issue for that is pending.

Rest2LDAP CREST API descriptors ignore read-only collection sub-resources

Summary

If a collection sub-resource defined in the Rest2LDAP endpoints JSON file is marked "isReadOnly": true, that endpoint does not appear at all in the API descriptor returned by CREST.

Affected Version

master (4.0.0-SNAPSHOT)

Steps to Reproduce

  1. Alter the example-v1.json file, and add the following under resourceTypes/example-v1/subResources (ensure you don't remove the existing users and groups sub-resource definitions):
                "all-users": {
                    "type": "collection",
                    "dnTemplate": "ou=people,dc=example,dc=com",
                    "resource": "frapi:opendj:rest2ldap:user:1.0",
                    "namingStrategy": {
                        "type": "clientDnNaming",
                        "dnAttribute": "uid"
                    },
                    "isReadOnly": true

2, Alter the Rest2LdapJsonConfiguratorTest, adding the line System.out.println(prettyPrint(api)); right after the parseJson(prettyPrint(api)); line.
3. Run the test.
4. Observe the console output from the test.

Expected results

The output includes the new all-users sub-resource, even though it's read-only. Something like the attached expected.json.

Actual results

The output omits all read-only collection sub-resources. See the attached actual.json.

Attachments

actual vs. expected JSON.zip

Persistent search result entry does not contain controls

When doing persistent search via LDAPConnectionHandler2 the result entry does not contain EntryChangeNotification control. This causes issues in Wren:AM that uses that information in SMS component.

I have already debugged what is happening and the issue is that LDAPClientConnection2 is mapping search result entry to response using method that leads to calling incorrect overloaded constructor.

Quite obvious fix is to add overloaded method to Responses class. However I would like to verify that it is the correct way and also possibly investigate if there is a place where we can add unit test to test this behaviour.

Debian Standard Packages Can't Use Doc Generated Ref from Same Reactor

Affected Version

Wren:DS 3.5.1

Repro Steps

  1. From a checked-out copy of Wren:DS, use mvn versions:set -DnewVersion=X.Y.Z inside the opendj-bom module to bump the version of the project to one that does not exist in JFrog.
  2. Run mvn clean install -Ppackages -Pforgerock-release -Dignore-artifact-sigs from the root of the Wren:DS repo.

Expected

All projects build successfully.

Actual

The "Wren:DS Debian Standard Package" module fails because it's looking for "Wren:DS Doc Generated References" in JFrog instead of using what was built in the same maven reactor / build.

Docker run crashes on OSX

Running recommended command docker run --rm --name wrends-test -p 1389:1389 -p 1636:1636 wrensecurity/wrends:latest causes docker to crash.

opendj-setup-5163692219923348077.log

On the other hand, the docker image runs smoothly when i build it first and then run it.
docker build . -t wrends-arm:latest
docker run --rm --name wrends-test -p 1389:1389 -p 1636:1636 wrends-arm:latest

Remove the ForgeRock Commercial License

Summary

I would like Wren:DS distributions not to contain any reference to ForgeRock's commercial license.

Solution You'd Like to See

  • The TXT files that contain the terms of the FR commercial license are no longer included in JAR packages for Wren:DS.
  • The FR commercial license is no longer displayed on the first screen of the interactive installer. In its place, we should be displaying the CDDL instead.

Workarounds/Alternatives

Users currently have to ignore the commercial license during install, since it doesn't apply to them. This causes confusion.

GeneralizedTime incorrectly handles years greater than 9999

Summary

The GeneralizedTime component inconsistently formats / parses the date when the year of the date is greater than 9999. During serialization, the year is serialized as is (e.g. 292278994), but the parsing process expects the year to be in YYYY format.

Steps To Reproduce ("Repro Steps")

@Test
public void testGeneralizedTimeMaxYear() {
    String serialized = GeneralizedTime.valueOf(Long.MAX_VALUE).toString();
    GeneralizedTime parsed = GeneralizedTime.valueOf(serialized);
}

The preceding test ends with the following exception:

org.forgerock.i18n.LocalizedIllegalArgumentException: The provided value "2922789940817071255.807Z" is not a valid generalized time value because "78" is not a valid month specification
	at org.forgerock.opendj.ldap.GeneralizedTime.valueOf(GeneralizedTime.java:221)

Expected Result (Behavior You Expected to See)

The behaviour of the GeneralizedTime component will be consistent if the year is greater than 9999.

Unable to maven install

[WARNING] [Retry 1 of 10] Waiting 750 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 2 of 10] Waiting 1500 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 3 of 10] Waiting 2250 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 4 of 10] Waiting 3000 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 5 of 10] Waiting 3750 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 6 of 10] Waiting 4500 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 7 of 10] Waiting 5250 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 8 of 10] Waiting 6000 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 9 of 10] Waiting 6750 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway
[WARNING] [Retry 10 of 10] Waiting 7500 milliseconds before retrying key request from hkps.pool.sks-keyservers.net/192.146.137.98 after last request failed: HTTP/1.1 502 Bad Gateway

Investigate possibility of using RocksDB as backend

Wren:DS has a plugable storage backend. Currently it primarily uses Berkeley DB Java edition as storage backend. At some point it also supported PDE however that didn't workout well so support was marked deprecated. PDE support was added due to ForgeRock's concerns about the licensing of Berkeley DB. However soon after PDE support was released the licensing of Berkeley DB was changed to a license which no longer posed a problem for ForgeRock.

While Berkeley DB was long the go to embedded storage backend new challengers have emerged. One of these is RocksDB. RocksDB was / is developed by Facebook and is based on research by Google (LevelDB).

RocksDB is supposedly, very, very fast. I think it would be interesting to do some research to see if A) RocksDB could be used as a backend for Wren:DS and B) If so, how much performance gain would that give us.

maven-jar-plugin fails on opendj-server-legacy after updating to wrensec-parent 3.x

Summary

After upgrading Wren:DS to use wrensec-parent-3.0.0-SNAPSHOT, the build fails to complete successfully when building opendj-server-legacy.

Affects Version

4.0.0-SNAPSHOT (at 513505e)

Environment

  • Ubuntu 16.04.5 LTS
  • Apache Maven 3.3.9
  • Java 1.8.0_181-b13

Steps

Within the wrends folder checked out to master, run the following command:

mvn clean install -DskipTests -Denforcer.skip=true -DignoreArtifactSigs -DskipTests

Expected

The build succeeds without error.

Actual

The build fails during the maven-jar-plugin execution of the opendj-server-legacy module:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.1.0:jar (build-bootstrap-client-jar) on project opendj-server-legacy: You have to use a classifier to attach supplemental artifacts to the project instead of replacing them. -> [Help 1]

Root Cause

The project builds many JARs with different names, but without any classifiers. In the 3.x version of the maven-jar-plugin, classifiers are now mandatory. But, adding them will cause the JARs to be renamed.

Artifact filenames / ZIP files end up with colons in the filename

Summary

Artifacts produced by Wren:DS end up with colons in their filenames, which is not an allowed character in Windows. This causes several issues, including build failure on Windows platforms.

Environment

Windows 10, JDK 8

Steps to Reproduce

  1. Checkout the Wren:DS project at the 3.0.0 tag.
  2. Run mvn clean install.

Expected Results

The build succeeds.

Actual Results

The build fails with an error about "negative time". This is actually because the JAR files don't get written to disk properly because their filenames contain characters not allowed in Windows.

Additional Notes

  • This is like caused by the colon in the product name (i.e. Wren:DS).
  • This may not be an issue in 3.5+ since in that version, there are new, separate Maven properties to control the lowercase product name that's used for archives.
  • Even on Linux systems, this issue causes the names of files inside the Wren:DS zip file not to be correct when extracted.

Calling DS status returns exit code 6

When calling for status of DS i'm getting an exit code 6 as a return.

./status -ns

Server Run Status: Started
Open Connections: 0
Host Name: <hostname>
Administrative Users: cn=Directory Manager
Installation Path:
<installation path>/wrends/opendj-server-legacy/target/package/wrends
Version: Wren:DS Server 4.0.0-RC2-SNAPSHOT
Java Version: <not available> (*)
Administration Connector: Port 4444 (LDAPS)

Address:Port: --
Protocol: LDIF
State: Disabled

Address:Port: 0.0.0.0:636
Protocol: LDAPS
State: Disabled

Address:Port: 0.0.0.0:1389
Protocol: LDAP
State: Enabled

Address:Port: 0.0.0.0:1689
Protocol: JMX
State: Disabled

Address:Port: 0.0.0.0:8080
Protocol: HTTP
State: Disabled

Base DN:     dc=example,dc=com
Backend ID:  userRoot
Entries:     <not available> (*)
Replication:

Get exit code in bash:
echo $?
6

top-level build fails on `opendj-dsml-servlet`

It's not possible to build all of OpenDJ successfully in one pass. the opendj-dsml-servlet project will fail because it can't find packages, but re-building just that package succeeds.

something is causing the classpath to be stale during the build.

Steps

  1. checkout Wren:DS from sustaining/3.0.0.
  2. run mvn clean install.

Expected

  • the build succeeds

Currently

  • the build fails because it cannot find several packages of generated source code:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project opendj-dsml-servlet: Compilation failure: Compilation failure:
[ERROR] opendj-dsml-servlet/src/main/java/org/opends/dsml/protocol/DSMLAddOperation.java:[37,40] package org.opends.server.protocols.ldap does not exist
  • afterwards, building with mvn clean install -pl opendj-dsml-servlet succeeds.

HTTP endpoints do not return full JSON response

Summary

HTTP endpoints do not return full JSON response.

$ curl -u admin:password http://localhost:8080/admin
{"_rev":"0000000054c141ca"
$ curl -u admin:password http://localhost:8080/api
{"_rev":"0000000054c141ca"

Steps To Reproduce ("Repro Steps")

  1. Enable HTTP Connector Handler
  2. Send GET requests to HTTP endpoints

Expected Result (Behavior You Expected to See)

Full readable JSON response.

Actual Result (Behavior You Saw)

Incomplete JSON response.

Environment

  • Product version: 4.0.0-RC2
  • JRE version: openjdk 17.0.6

DS starts in read-only mode when another replica is unreachable

Summary

When in multi-master replication mode a single server is unreachable (socket connection has to timeout) it will cause replication server to not be able to accept any connection due to data server's own socket timeout. This happens during the first DS-RS handshake phase and is accompanied with the following error in the log:

category=SYNC severity=ERROR msgID=178 msg=Directory server 15265 was attempting to connect to replication server 25602 but has disconnected in handshake phase

Steps To Reproduce ("Repro Steps")

  1. Download attached docker-compose.yml
  2. Start both server instances with docker-compose up -d
  3. Configure replication:
docker exec -it wrends-test1 \
        dsreplication enable --adminUID admin --adminPassword password --trustAll --no-prompt --baseDN dc=example,dc=com \
        --host1 wrends-test1 --port1 4444 --bindDN1 "cn=Directory Manager" \
        --bindPassword1 password --replicationPort1 8989 \
        --host2 wrends-test2 --port2 4444 --bindDN2 "cn=Directory Manager" \
        --bindPassword2 password --replicationPort2 8989
docker exec -it wrends-test1 \
        dsreplication initialize-all --adminUID admin --adminPassword password --trustAll --no-prompt \
        --baseDN dc=example,dc=com --hostname wrends-test1 --port 4444
  1. Shutdown both servers with docker-compose stop
  2. Start only the first server with docker-compose up -d wrends-test1
  3. Try to perform modification with docker exec -it wrends-test1 ldapdelete -h localhost -p 1389
docker exec -it wrends-test1 ldapdelete -h localhost -p 1389 "uid=user.1,ou=People,dc=example,dc=com"

Expected Result (Behavior You Expected to See)

Server deletes the requested LDAP entry.

Actual Result (Behavior You Saw)

The following error is returned:

Processing DELETE request for uid=user.1,ou=People,dc=example,dc=com
The LDAP delete request failed: 53 (Unwilling to Perform)
Additional Information:  The Replication is configured for suffix
dc=example,dc=com but was not able to connect to any Replication Server

Additional Notes

I have spent several hours trying to debug this. The underlying issue is that replication server's connection listener is trying to contact all other replication servers when accepting new connection. As the other server does not exist, this attempt timeouts with the same timeout value as is the data server's timeout for the connection handshake.

I am not sure why we need to wait for the replication domain to actually contact all servers. Simply increasing handshake timeout wouldn't work if we have multiple servers in the domain.

Creating this issue to actually track potential discussion regarding solving this problem.

`opendj-doc-generated-ref` plug-in fails in `opendj-server-legacy` on Windows machines with JDK8

Summary

It's currently not possible to build master on Windows machines, even if all dependencies are satisfied and PGP verification is being skipped, because the opendj-doc-generated-ref plug-in fails to generate documentation.

Affects Version

4.0.0-SNAPSHOT (at 4678646)

Environment

  • Windows 10.0.17134 Build 17134
  • Apache Maven 3.3.9
  • Java 1.8.0_172-b11

Steps

Within the wrends folder checked out to master, run the following command:

mvn clean install -DskipTests -Denforcer.skip=true -DignoreArtifactSigs -DskipTests

Expected

The build succeeds without any failures.

Actual

The build fails during the opendj-doc-generated-ref plug-in execution:

[INFO]
[INFO] --- opendj-doc-maven-plugin:4.0.0-SNAPSHOT:generate-config-ref (generate-configuration-reference-doc) @ opendj-doc-generated-ref ---
[INFO]
[INFO] --- opendj-doc-maven-plugin:4.0.0-SNAPSHOT:generate-global-acis-table (generate-global-acis-table-and-ref-for-doc) @ opendj-doc-generated-ref ---
[INFO]
[INFO] --- opendj-doc-maven-plugin:4.0.0-SNAPSHOT:generate-schema-ref (generate-global-acis-table-and-ref-for-doc) @ opendj-doc-generated-ref ---
[INFO]
[INFO] --- opendj-doc-maven-plugin:4.0.0-SNAPSHOT:generate-xml-messages-doc (generate-log-reference-doc) @ opendj-doc-generated-ref ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Wren:DS Doc Helper Maven Plugin .................... SUCCESS [ 14.641 s]
[INFO] Wren:DS Doc Generated References ................... FAILURE [ 35.501 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 56.519 s
[INFO] Finished at: 2018-08-23T00:59:02-04:00
[INFO] Final Memory: 46M/556M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.forgerock.opendj:opendj-doc-maven-plugin:4.0.0-SNAPSHOT:generate-xml-messages-doc (generate-log-reference-doc) on project opendj-doc-generated-ref: NullPointerException: entry -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :opendj-doc-generated-ref

Root Cause

It looks like the issue is that the plug-in is using Paths.get() within loadPropertiesFromJar() to assemble the path that's passed-in to the JAR file. On Windows machines, this is problematic because the method generates a path with backslashes instead of forward slashes, and the ZIP spec requires forward slashes.

Bug: X-Enum Syntaxes Crash Control Panel During Schema Parsing

Affected Version: 3.0

Summary

The Wren:DS / OpenDJ Control Panel application does not appear to properly handle custom X-ENUM syntaxes; when they are defined in the server, the Control Panel erroneously flags the schema as being incomplete and will not provide a listing of the directory or the schema.

Steps to Reproduce

  1. Install Wren:DS, being sure to create at least a root entry for the DC.
  2. Create a custom X-ENUM syntax (for example: https://docs.oracle.com/cd/E19476-01/821-0509/enumeration-syntax-extension.html). You can create the syntax through either the ldapmodify CLI, or by editing 99-user.ldif.
  3. Define an attribute type that uses the new X-ENUM syntax (reference the SunDS article mentioned earlier). You can create the syntax through either the ldapmodify CLI, or by editing 99-user.ldif (place it AFTER the X-ENUM syntax definition).
  4. Restart Wren:DS.
  5. Attempt to connect to the server with Wren:DS / OpenDJ Control Panel.

Expected Results

  • Control Panel connects to the server without error.
  • If you browse the schema using the Control Panel application, you will find the OID of the X-ENUM syntax under "Attribute Syntaxes" in the "Manage Schema" window.
    image

Actual Results

  • Control Panel connects but displays errors.
  • The error displayed is:

Error Reading Configuration
The definition for the attribute type with OID 1.3.6.1.4.1.32473.5 declared that it should have a syntax with OID 1.3.6.1.4.1.32473.4. No such syntax is configured for use in the Directory Server

image

Additional Info

Anecdotally, it seems like this might be fixed in the 4.0 branch, according to https://bugster.forgerock.org/jira/browse/OPENDJ-3252. However, simply cherry-picking aaf3145 (the commit cited in the comments for that ticket) on to the 3.0 SDK does NOT solve the issue for 3.x. There was likely a larger re-factor of this code that addressed it.

Searching for Ports 1,000 at a Time can Result in "Invalid port exception"

Summary

Wren:DS SDK defines a PORT_INCREMENT value of 1000, which means that if a port it wants to use is in-use, it skips 1,000 ports at a time until it can find the desired port. This can lead to failures during test runs if the port number the OS hands to the test is close to the maximum port number (e.g. max is 65535 and starting port number is 64536).

At run-time, presumably this issue is less severe because Wren:DS has no preference for starting port number, so it has a larger pool of addresses to search 1,000 at a time. This defect only appears when the starting port # is within a few thousand port numbers of the RFC-defined max port #.

Steps To Reproduce ("Repro Steps")

NOTE: These steps do not reliably reproduce the issue because the issue depends on what ports are available on the host machine.

  1. Checkout Wren:DS @ 9e5a6fa.
  2. Build the project with mvn clean install.

Expected Result (Behavior You Expected to See)

The build should succeed with no test failures.

Actual Result (Behavior You Saw)

Sporadically, tests may fail with an IllegalArgumentException and message of "Invalid port.".

Environment

  • Product version (include GIT commit ID if using a dev build): 3.0.0 @ 9e5a6fa
  • Operating system:
    Ubuntu 18.04.1 LTS
    Linux 4.4.0-17763-Microsoft #379-Microsoft Wed Mar 06 19:16:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
    
  • JRE version:
    openjdk version "1.8.0_191"
    OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
    OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
    
  • Was instance upgraded from a previous version? If so, what version were you using before?
    No.

Sub-tree flattening and filtering for Rest2LDAP collection sub-resources

This one comes from an internal feature request we're working on for Rest2LDAP.

Summary

As a developer querying a deeply nested LDAP directory of users, I would like it if nested entries could be retrieved and filtered through the Rest2LDAP interface like flat collections so that I don't have to pull the full list of entries just to get information about a single entry.

Component Affected

Rest2LDAP (OpenDJ 3.5+)

Conditions of Acceptance

The Rest2LDAP module within OpenDJ is modified as follows:

  • A new option is added to the Rest2LDAP configuration file to allow collection sub-resources to support sub-tree flattening, such that nested LDAP entries are included as though they were all at the same flat level in the directory.
  • A new option is added to the Rest2LDAP configuration file to allow collection sub-resources to support an optional "base" search filter that can be used to restrict what entries get returned through the sub-resource. This search filter supplements any filters provided in the request.
  • Validation is added to ensure that the new sub-tree flattening option is only being used if the sub-resource is marked as read-only in the config (without this, there is no way to unambiguously map attempts to create new entries to a specific DN). Any attempt to configure a writable sub-resource with sub-tree flattening turned on must result in an error.
  • Sub-resources must be modified to behave as follows:
    • When the sub-resource has the new subtree flattening option toggled on: it should return all LDAP entries and sub-entries under the base DN that match both search filters provided on the query string as well as the search filter specified in the configuration for the sub-resource (i.e. it expands the search scope to SearchScope.SUBORDINATES).
    • When the sub-resource has the new subtree flattening option toggled off or has not specified whether the option is toggled on: it should return only entries that exist directly under the base DN, excluding sub-entries (i.e. the search scope is SearchScope.SINGLE_LEVEL), and then apply both search filters provided on the query string as well as the search filter specified in the configuration for the sub-resource (if any).

Workarounds / Other Approaches

  • Set-up a singleton resource that's mapped to a virtual static group that queries the whole sub-tree and selects the desired users from everywhere under the desired base DN. By rendering the members of the virtual static group, you can access all of the users everywhere in the subtree instead of just at the base level. Major drawback: Huge performance hit, since trying to access a single record requires pulling down the whole list of users from the server.
  • Don't use subtrees -- put all users under a single flat portion of the directory (not ideal for most use cases).
  • Don't use Rest2LDAP -- query LDAP directly from the application (not ideal because it requires all consumers to understand LDAP).

import-ldif crashes on OutOfMemoryError

Summary

import-ldif crashes with OutOfMemoryError on Java 17.

Steps To Reproduce ("Repro Steps")

  1. Run setup with sample data creation:

    ./wrends/setup \
      --cli \
      --backendType je \
      --baseDN dc=example,dc=com \
       --ldapPort 1389 \
      --adminConnectorPort 4444 \
      --rootUserDN cn=Directory\ Manager \
      --rootUserPassword password \
      --no-prompt \
      --noPropertiesFile \
      --doNotStart \
      --sampleData 200

Expected Result (Behavior You Expected to See)

import-ldif successfully imports data without errors.

Actual Result (Behavior You Saw)

The import stucks on message:

Importing Automatically-Generated Data (200 Entries) ........

The log in /tmp/opendj-setup-*.log contains following error:

[24/11/2022:11:56:07 +0100] category=QUICKSETUP seq=72 severity=INFO msg=import-ldif arg list: [/tmp/wrends/bin/import-ldif, --offline, -n, userRoot, -A, /tmp/opendj-install17722320096080648957.template, -s, 0, -F, --noPropertiesFile]
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=73 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=0 severity=INFO msg=Installation Directory:  /tmp/wrends
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=74 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=1 severity=INFO msg=Instance Directory:      /tmp/wrends
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=75 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=2 severity=INFO msg=JVM Information: 17.0.5+8-Ubuntu-2ubuntu1 by Private Build, 64-bit architecture, 3856662528 bytes heap size
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=76 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=3 severity=INFO msg=JVM Host: foobar, running Linux 5.19.0-23-generic amd64, 15426191360 bytes physical memory size, number of processors available 16
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=77 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=4 severity=INFO msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=78 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=5 severity=FINE msg=JE backend 'userRoot' does not specify the number of cleaner threads: defaulting to 16 threads
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=79 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=6 severity=FINE msg=JE backend 'userRoot' does not specify the number of lock tables: defaulting to 37
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=80 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=24 severity=INFO msg=Wren:DS Server 4.0.0-M4-SNAPSHOT starting import (build 20221124115033, Rd7998ffe96597b13df5e2d4af125455f05fbf0da)
[24/11/2022:11:56:12 +0100] category=QUICKSETUP seq=81 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:12 +0100] category=BACKEND seq=25 severity=INFO msg=The amount of free memory available to the import task is 3470996275 bytes. The number of phase one buffers required is 640 buffers
[24/11/2022:11:56:12 +0100] category=QUICKSETUP seq=82 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:12 +0100] category=BACKEND seq=26 severity=INFO msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 4194304 bytes
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=84 severity=WARNING msg=import-ldif error log: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=85 severity=WARNING msg=import-ldif error log:      at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:64)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=86 severity=WARNING msg=import-ldif error log:      at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:363)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=87 severity=WARNING msg=import-ldif error log:      at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool.<init>(OnDiskMergeImporter.java:2890)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=88 severity=WARNING msg=import-ldif error log:      at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.newHeapBufferPool(OnDiskMergeImporter.java:466)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=89 severity=WARNING msg=import-ldif error log:      at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.newBufferPool(OnDiskMergeImporter.java:401)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=90 severity=WARNING msg=import-ldif error log:      at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.importLDIF(OnDiskMergeImporter.java:210)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=91 severity=WARNING msg=import-ldif error log:      at org.opends.server.backends.pluggable.BackendImpl.importLDIF(BackendImpl.java:660)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=92 severity=WARNING msg=import-ldif error log:      at org.opends.server.tools.ImportLDIF.processLocal(ImportLDIF.java:853)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=93 severity=WARNING msg=import-ldif error log:      at org.opends.server.tools.tasks.TaskTool.process(TaskTool.java:246)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=94 severity=WARNING msg=import-ldif error log:      at org.opends.server.tools.ImportLDIF.process(ImportLDIF.java:243)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=95 severity=WARNING msg=import-ldif error log:      at org.opends.server.tools.ImportLDIF.mainImportLDIF(ImportLDIF.java:113)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=96 severity=WARNING msg=import-ldif error log:      at org.opends.server.tools.ImportLDIF.main(ImportLDIF.java:88)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=83 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:14 +0100] category=BACKEND seq=27 severity=INFO msg=Import LDIF environment close took 0 seconds
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=97 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:14 +0100] category=BACKEND seq=28 severity=INFO msg=Flushing data to disk

Environment

  • Product version (include GIT commit ID if using a dev build): master (d7998ff)
  • Operating system: Kubuntu 22.10, Linux 5.19.0-23-generic
  • JRE version: openjdk 17.0.5 2022-10-18
  • Was instance upgraded from a previous version? If so, what version were you using before? No
  • CPU: AMD Ryzen 7 PRO 4750U (16 threads)
  • Memory: 14GB

Additional Notes

  • I did not notice this problem on Java 8 neither 11.
  • Manual invocation of import-ldif with -Xmx256m works.
  • It works on some other computers with different CPU and memory size.

building Wren:DS fails without direct DNS access

When building Wren:DS in a network without public DNS available, maven fails while trying to verify the artifact signatures:

[DEBUG]   (f) project = MavenProject: org.forgerock.opendj:opendj-server-parent:3.0.0 @ /home/olbohlen/git/wren/wrends/pom.xml
[DEBUG]   (f) scope = test
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@704b2127
[DEBUG]   (f) verifyPomFiles = true
[DEBUG] -- end configuration --
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.JarResourceLoader.
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.FileResourceLoader.
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.ThreadContextClasspathResourceLoader.
[DEBUG] URLResourceLoader: Exception when looking for 'http://wrensecurity.org/trustedkeys.properties
java.net.UnknownHostException: wrensecurity.org
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)

as a work-around, mvn -Dignore-artifact-sigs clean install works.

‼️ Evaluate sustaining/3.5.x for potential important fixes

From gitter chat:

The only difference between sustaining/3.5.x and the current master are the following commits:

ad669ec93a Upgrade Wren:DS to `wrensec-parent` (2.2.0)
7765a3cb9a Remove Unused Wren Deploy Values
7dd5225a57 Fix Acct. Change Notification Package Name
b210895b0e Fix file names to conform to class names in them.
47b06874c8 Fix path to Wren:DS JAR file for Doc Generation
bef165bab8 Add note about OPENDJ-3377 as OpenIG Workaround
8bef7b5dcb OPENDJ-3377 Switch OpenDJ dependency from openig-toolkit to chf-oauth2
55da63bff3 Update 3.5.1 for WrenSec repos & info
91c3b0ba77 Add Official OpenDJ 3.5.1 Sources from Backstage

I think we can go through those and apply them if necessary.

The following will be evaluated (the main idea behind this is that if there is a critical patch, it has unit tests as well):

git show 91c3b0ba77 -I'^ *([/\*]|@Override)' -- **/src/test/** > changes.diff

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.