realm / realm-java Goto Github PK

Realm is a mobile database: a replacement for SQLite & ORMs

License: Apache License 2.0

Java 69.54% C++ 7.83% Groovy 0.06% HTML 0.03% CMake 0.11% Shell 0.44% Kotlin 21.77% JavaScript 0.12% Dockerfile 0.06% Python 0.05%

java android database mobile realtime-database mobile-database nosql-database

realm-java's Introduction

Realm is a mobile database that runs directly inside phones, tablets or wearables. This repository holds the source code for the Java version of Realm, which currently runs only on Android.

Realm Kotlin

The Realm Kotlin SDK is now GA and can be used for both Android and Kotlin Multiplatform. While we are still adding features, please consider using Realm Kotlin for any new project, and let us know if you miss anything there!

Features

Mobile-first: Realm is the first database built from the ground up to run directly inside phones, tablets, and wearables.
Simple: Data is directly exposed as objects and queryable by code, removing the need for ORM's riddled with performance & maintenance issues. Plus, we've worked hard to keep our API down to very few classes: most of our users pick it up intuitively, getting simple apps up & running in minutes.
Modern: Realm supports easy thread-safety, relationships & encryption.
Fast: Realm is faster than even raw SQLite on common operations while maintaining an extremely rich feature set.
Device Sync: Makes it simple to keep data in sync across users, devices, and your backend in real-time. Get started for free with a template application and create the cloud backend.

Getting Started

Please see the detailed instructions in our docs to add Realm to your project.

Documentation

Documentation for Realm can be found at mongodb.com/docs/atlas/device-sdks/sdk/java/. The API reference is located at mongodb.com/docs/atlas/device-sdks/sdk/java/api/.

Getting Help

Got a question?: Look for previous questions on the #realm tag — or ask a new question. We actively monitor & answer questions on StackOverflow! You can also check out our Community Forum where general questions about how to do something can be discussed.
Think you found a bug? Open an issue. If possible, include the version of Realm, a full log, the Realm file, and a project that shows the issue.
Have a feature request? Open an issue. Tell us what the feature should do, and why you want the feature.

Using Snapshots

If you want to test recent bugfixes or features that have not been packaged in an official release yet, you can use a -SNAPSHOT release of the current development version of Realm via Gradle, available on Sonatype OSS

buildscript {
    repositories {
        mavenCentral()
        google()
        maven {
            url 'https://oss.sonatype.org/content/repositories/snapshots/'
        }
        jcenter()
    }
    dependencies {
        classpath "io.realm:realm-gradle-plugin:<version>-SNAPSHOT"
    }
}

allprojects {
    repositories {
        mavenCentral()
        google()
        maven {
            url 'https://oss.sonatype.org/content/repositories/snapshots/'
        }
        jcenter()
    }
}

See version.txt for the latest version number.

Building Realm

In case you don't want to use the precompiled version, you can build Realm yourself from source.

Prerequisites

Download the JDK 8 from Oracle and install it.
The latest stable version of Android Studio. Currently 4.1.1.
Download & install the Android SDK Build-Tools 29.0.3, Android Pie (API 29) (for example through Android Studio’s Android SDK Manager).
Install CMake version 3.18.4 and build Ninja.
Install the NDK (Side-by-side) 21.0.6113669 from the SDK Manager in Android Studio. Remember to check ☑ Show package details in the manager to display all available versions.
Add the Android home environment variable to your profile:
```
export ANDROID_HOME=~/Library/Android/sdk
```
If you are launching Android Studio from the macOS Finder, you should also run the following command:
```
launchctl setenv ANDROID_HOME "$ANDROID_HOME"
```
If you'd like to specify the location in which to store the archives of Realm Core, define the REALM_CORE_DOWNLOAD_DIR environment variable. It enables caching core release artifacts.
```
export REALM_CORE_DOWNLOAD_DIR=~/.realmCore
```
macOS users must also run the following command for Android Studio to see this environment variable.
```
launchctl setenv REALM_CORE_DOWNLOAD_DIR "$REALM_CORE_DOWNLOAD_DIR"
```

It would be a good idea to add all of the symbol definitions (and their accompanying launchctl commands, if you are using macOS) to your ~/.profile (or ~/.zprofile if the login shell is zsh)

If you develop Realm Java with Android Studio, we recommend you to exclude some directories from indexing target by executing following steps on Android Studio. It really speeds up indexing phase after the build.
- Under /realm/realm-library/, select build, .cxx and distribution folders in Project view.
- Press Command + Shift + A to open Find action dialog. If you are not using default keymap nor using macOS, you can find your shortcut key in Keymap preference by searching Find action.
- Search Excluded (not Exclude) action and select it. Selected folder icons should become orange (in default theme).
- Restart Android Studio.

Download sources

You can download the source code of Realm Java by using git. Since realm-java has git submodules, use --recursive when cloning the repository.

git clone [email protected]:realm/realm-java.git --recursive

git clone https://github.com/realm/realm-java.git --recursive

Build

Once you have completed all the pre-requisites building Realm is done with a simple command.

./gradlew assemble

That command will generate:

a jar file for the Realm Gradle plugin
an aar file for the Realm library
a jar file for the annotations
a jar file for the annotations processor

The full build may take an hour or more, to complete.

Building from source

It is possible to build Realm Java with the submodule version of Realm Core. This is done by providing the following parameter when building: -PbuildCore=true.

./gradlew assembleBase -PbuildCore=true

You can turn off interprocedural optimizations with the following parameter: -PenableLTO=false.

./gradlew assembleBase -PenableLTO=false`

Note: Building the Base variant would always build realm-core.

Note: Interprocedural optimizations are enabled by default.

Note: If you want to build from source inside Android Studio, you need to update the Gradle parameters by going into the Realm projects settings Settings > Build, Execution, Deployment > Compiler > Command-line options and add -PbuildCore=true or -PenableLTO=false to it. Alternatively you can add it into your gradle.properties:

buildCore=true
enableLTO=false

Note: If building on OSX you might like to prevent Gatekeeper to block all NDK executables by disabling it: sudo spctl --master-disable. Remember to enable it afterwards: sudo spctl --master-enable

Other Commands

./gradlew tasks will show all the available tasks
./gradlew javadoc will generate the Javadocs
./gradlew monkeyExamples will run the monkey tests on all the examples
./gradlew installRealmJava will install the Realm library and plugin to mavenLocal()
./gradlew clean -PdontCleanJniFiles will remove all generated files except for JNI related files. This reduces recompilation time a lot.
./gradlew connectedUnitTests -PbuildTargetABIs=$(adb shell getprop ro.product.cpu.abi) will build JNI files only for the ABI which corresponds to the connected device. These tests require a running Object Server (see below)

Generating the Javadoc using the command above may generate warnings. The Javadoc is generated despite the warnings.

Upgrading Gradle Wrappers

All gradle projects in this repository have wrapper task to generate Gradle Wrappers. Those tasks refer to gradle property defined in /dependencies.list to determine Gradle Version of generating wrappers. We have a script ./tools/update_gradle_wrapper.sh to automate these steps. When you update Gradle Wrappers, please obey the following steps.

Edit gradle property in defined in /dependencies.list to new Gradle Wrapper version.
Execute /tools/update_gradle_wrapper.sh.

Gotchas

The repository is organized into six Gradle projects:

realm: it contains the actual library (including the JNI layer) and the annotations processor.
realm-annotations: it contains the annotations defined by Realm.
realm-transformer: it contains the bytecode transformer.
gradle-plugin: it contains the Gradle plugin.
examples: it contains the example projects. This project directly depends on gradle-plugin which adds a dependency to the artifacts produced by realm.
The root folder is another Gradle project. All it does is orchestrate the other jobs.

This means that ./gradlew clean and ./gradlew cleanExamples will fail if assembleExamples has not been executed first. Note that IntelliJ does not support multiple projects in the same window so each of the six Gradle projects must be imported as a separate IntelliJ project.

Since the repository contains several completely independent Gradle projects, several independent builds are run to assemble it. Seeing a line like: :realm:realm-library:compileBaseDebugAndroidTestSources UP-TO-DATE in the build log does not imply that you can run ./gradlew :realm:realm-library:compileBaseDebugAndroidTestSources.

Examples

The ./examples folder contains many example projects showing how Realm can be used. If this is the first time you checkout or pull a new version of this repository to try the examples, you must call ./gradlew installRealmJava from the top-level directory first. Otherwise, the examples will not compile as they depend on all Realm artifacts being installed in mavenLocal().

Standalone examples can be downloaded from website.

Running Tests on a Device

To run these tests, you must have a device connected to the build computer, and the adb command must be in your PATH

Connect an Android device and verify that the command adb devices shows a connected device:
```
adb devices
List of devices attached
004c03eb5615429f device
```

Run instrumentation tests:

cd realm
./gradlew connectedBaseDebugAndroidTest

These tests may take as much as half an hour to complete.

Running Tests Using The Realm Object Server

Tests in realm/realm-library/src/syncIntegrationTest require a running testing server to work. A docker image can be built from tools/sync_test_server/Dockerfile to run the test server. tools/sync_test_server/start_server.sh will build the docker image automatically.

To run a testing server locally:

Install docker and run it.
Run tools/sync_test_server/start_server.sh:
```
cd tools/sync_test_server
./start_server.sh
```
This command will not complete until the server has stopped.

Run instrumentation tests

In a new terminal window, run:

cd realm
./gradlew connectedObjectServerDebugAndroidTest

Note that if using VirtualBox (Genymotion), the network needs to be bridged for the tests to work. This is done in VirtualBox > Network. Set "Adapter 2" to "Bridged Adapter".

These tests may take as much as half an hour to complete.

Contributing

See CONTRIBUTING.md for more details!

This project adheres to the MongoDB Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

The directory realm/config/studio contains lint and style files recommended for project code. Import them from Android Studio with Android Studio > Preferences... > Code Style > Manage... > Import, or Android Studio > Preferences... > Inspections > Manage... > Import. Once imported select the style/lint in the drop-down to the left of the Manage... button.

License

Realm Java is published under the Apache 2.0 license.

Realm Core is also published under the Apache 2.0 license and is available here.

Feedback

If you use Realm and are happy with it, all we ask is that you, please consider sending out a tweet mentioning @realm to share your thoughts!

And if you don't like it, please let us know what you would like improved, so we can fix it!

realm-java's People

Contributors

Stargazers

Watchers

Forkers

cmelchior baboleevan alexguoq marudx pellucide hassanabidpk vijayrawatsan harichidipotu panicha cmai hoadn romantic011 michaelluk eeearl robertschou akozeletsky leequer b2b2244424 djyde verachadw schiffe namjae cking24343 donglongwen-daren jokit shenmeng fqyb1986 flyouting sxhao radityak lucamtudor amaniak noobycoder haikuowuya cyan-at xingrz teflonlondon god7 robertojrojas livotov quickpoint halyna nitiwari-dev codingwithus famersbs gnagel webuildstuff yodamaster team55 ghe0106 d-buchanan01 idelgado dotgc jirawuts kidnox djun100 22goodmen ludmiloff yoshyosh dzhng zhourihu5 ariellev thorbenprimke shiva10162 nazeebodan brianmcmichael hkmopen shiratsu kerasking gavioto gokussjx jbaginski stephen1706 raskilas njovy rvganesh sarvex ok3141 biddyweb sethjback denismirchandani hansangyoon dugin13 shadow7 dy4km zerobitzero superyfwy luiguibalarezo hanhailong markmillerkeene d1m0nh banji180 darineraka alin-rosu ajrulez jamesguo ncit javigon yzp531 msoftware

realm-java's Issues

Design and implement rowset column aggregation and manipulation [2-3h]

Design and implement the aggregation and manipulation operations for all the table-or-view-specific column types, e.g. getAll(), setAll(), etc.

In Loadlibrary we should try both tightdb, tightdb32, tightdb64 and otherwise come with a helpful message.

FindUnique() - I don't quite get the idea with the method when you get exceptions when it doesn't "match".

For now I removed it in the example, since this one we include to customers.

Make the code generation for .example by located in .example.generated

not in *.generated as now

Change Code generation tool from Jannocesor to "Manual"

Putting generated code in my.application.generated is problematic

When code is generated 'on the fly' during compilation, it is not possible to write something like this in the application code:

import my.application.generated.*;

Because there is nothing inside a package of that name initially. If you try, you will get a compile time error.

This effectively forces you to use fully qualified names when referring to any of the generated classes:

my.application.generated.MyTable = new my.application.generated.MyTable();

Way too painful!

Changing the generator such that it no longer appends ".generated" to the package name works very well in my tests, so why don't we simply do that?

Here is my test code:

package my.application;

import com.tightdb.*;
import com.tightdb.lib.Table;

public class Test {

    @Table
    class hilbert {
        String fido;
    }

    @Table
    class banach {
        String type;
        String number;
    }

    public static void main(String[] args)
    {
        SharedGroup db = new SharedGroup("/tmp/test.tightdb");
        try {
            WriteTransaction transact = db.beginWrite();
            try {
                BanachTable banach = new BanachTable(transact);
                banach.add("John", "Doe");
                transact.commit();
            }
            catch (Throwable e) {
                transact.rollback();
                throw e;
            }
        }
        finally {
            db.close();
        }
    }
}

Optimize code generation for add()

In add() you might as well call insert() instead of duplicating the generated code:

public Phone add(String type, String number) {
try {
long position = size();

        insertString(0, position, type);
        insertString(1, position, number);
        insertDone();

        return cursor(position);
    } catch (Exception e) {
        throw addRowException(e);
    }

}

public Phone insert(long position, String type, String number) {
    try {
        insertString(0, position, type);
        insertString(1, position, number);
        insertDone();

        return cursor(position);
    } catch (Exception e) {
        throw insertRowException(e);
    }

Improve type-safety of table and view columns [1-2h]

Introduce table-or-view-specific column for all column types, to provide column operations for a list of records - aggregation, projection, etc. and disable cursor-specific operations. For example, the get() operation should return a list of values, instead only one.

Let us hide the "lib" in com.tightdb.lib.Table

I don't much like the "lib" part of

import com.tightdb.lib.Table;

First of all, the word "lib" does not really say anything that would help the customer understand why Table would be located there rather than in "com.tightdb". Secondly, I believe it is a very small change to eliminate it, since Table is the only class that the customer is likely to ever have to refer to in that package.

If we moved the Table annotation class to com.tightdb, the customer could simply write

import com.tightdb.*;

For now he needs at least

import com.tightdb.*;
import com.tightdb.lib.Table;

Another improvement (though less attractive in my opinion) would be to move Table to "com.tightdb.annotations", it seems that convention is fairly widespread:

import com.tightdb.*;
import com.tightdb.annotations.Table;

Implement code generation for the other columns types [2-3h]

implement code generation for the other columns types (boolean, date, binary, mixed)

Migrate to Core JNI API v1.1 [0.5-1h]

Migrate (table declaration etc.) to v1.1 of the Core JNI API:

hand-written example,
code generator.

Warn/Error if any unsupported types are used in a @Table class.

Warn/Error if any unsupported types are used in a @table class.

Why a separate step to transform code genrator templates?

Apparently, there is a separate step in the build process that transforms the templates of the code generator from individual files into strings inside a Java class.

Why did we do that?

In my opinion, it serves only to complicate the build process.

Why don't we simply include the template files inside the JAR file? One would then have to use getResource() or getResourceAsStream() on Class or ClassLoader.

Nested table "Phone" doesn't work

Should our code generator be part of tightdb.jar?

In my opinion, the code generator should not be part of tightdb.jar, because it is only needed during compilation of the customers Java code. When the customer deploys the code, they still need to include tightdb.jar, but there is no longer any need for the generator.

I suggest we produce two JARs:
tightdb.jar
tightdb-codegen.jar

Anonymous variadic macros are not part of C++03

tightdb_java2/tightdb_jni/src/util.hpp defines an anonymous variadic macro. Since they are not part of C++03, we should not use them.

Note: Anonymous variadic macros were introduced in C99 as well as C++11.

Implement code generation for column list retrieval [0.5-1h]

Implement code generation for column list retrieval.

Code generation: Can you automatically refresh project after code generation

Version warning from annotation processor

I get a warning when the annotation processor runs because it declares that it does not understand Java source format beyond version 6.

In com/tightdb/generator/CodeGenProcessor.java:

@SupportedSourceVersion(SourceVersion.RELEASE_6)

should probably be changed to

@SupportedSourceVersion(SourceVersion.RELEASE_7)

Of course, we can only make this change if our processor actually understands version 7 syntax properly. If not, we must update the processor.

Query "2b" doesn't work

In Example (that I have included in tightdb/java2) there are a few things that doesn't work. They are mentioned in comments.

Replace the improvised with real types from v1.1 [0.5-1h]

Replace the improvised column types with the real types from Core JNI API v1.1.

Improve type-safety of query columns [1-2h]

Improve the type-safety of the query mechanism, by introducing query-only columns for all the column types. Such columns will allow query construction, but not aggregation or projection, so the following statement won't be possible:

persons.firstName.is("X").or().salary.set(1234);

Integrate JNI changes from tightdb/Java

Integrate JNI changes from tightdb/Java.
All index parameters are now changed from int to long
Multiple new methods are available. So they should propagate to the highlevel interface

With a cursor, obtaining the row number (with getPosition() ), fails. Returns 0 even for last row.

Complete the library for nested tables [3-5h]

Complete the design and implement the library for nested tables.

Implement code generation for nested tables [2-3h]

Implement code generation for nested tables (basic operations).

Abstract query methods: count, average, sum, max, min is missing.

FieldSorter in code generator fails

When the code generator runs during compilation from command line, the invocation of FieldSorter from CodeGenProcessor.processAnnotatedElements() fails with messages like the following:

warning: The file doesn't exist: /tmp/temp.pEPW/my.java
Note: Scanning source path '/tmp/temp.pEPW' for table spec 'hilbert'
error: Table spec retrieval failed!
error: Field sorting failed, couldn't find table spec: hilbert

This causes the generator to abort.

When I comment out the invocation, everything works perfectly (assuming a few other reported issues are fixed as suggested).

The following comment suggests that the FieldSorter is a temporary "work around", so maybe we should fix the core problem and get rid of it.

// sort the fields, due to unresolved bug in Eclipse APT
fieldSorter.sortFields(fields, model, sourcesPath);

Make the Code generation less "verbose" if possible and easy (since we later change codegeneration method.)

Implement table and view iterator [1-2h]

Implement table and view iterator.

Generated tables: Remove access to some methods

Methods like insertXXX(), insertDone() etc should now be accessible by the generated Tables in highlevel API since the generated insert() should be used instead.
There are likely other methods visible that should be hidden as well.

Code generator constructs invalid names for source files.

AbstractAnnotationProcessor.writeToSourceFile(), joins a package name with a class name using a "/" rather than a ".". This causes the generator to produce an invalid name for the generated source file, which in turn causes the generator to abort.

Implement the view and query library [8-12h]

Implement the view and query library for advanced operations (search & aggregation).

Change "@NestedTable" to "@Table"

Implement the code generation for view and query [3-5h]

Implement the view and query library for advanced operations (search & aggregation).

Add support for new TableViewBase methods in highlevel API.

findFirstBoolean
findFirstDate
findAllBoolean
findAllDate

getMixedType
getSubtableSize
clearSubTable

addLong (also add to TableBase)
getSortedView (only TableBase)

Complete the library for the other columns types [3-4h]

Complete the design and implement the library for the other columns types (boolean, date, binary, mixed)

Write JavaDocs for the public API classes

Write JavaDocs for the public API classes.

Automated Testcases with TestNG - and codecoverage tool

Code generation should generate the code under the project instead of com.tightdb.generated

What was it with the JDK6 dependency? How can we also support JDK7?

Requiring lowercase class names when declaring tables

Currently, classes that are used to declare tables must have uncapitalized names. This is because the code generator generates a cursor class with the same name except that in this case it is capitalized.

I don't like having to write classes with lower case names, and I fear that a lot of Java programmers are going to hate us for it.

I suggest that we either:

Rename the cursor class from Employee to EmployeeCursor or EmployeeRow, and thereby allowing for the declaration class to be renamed from 'employee' to Employee.
Rename the declaration class from 'employee' to EmployeeRow, or EmployeeDecl, or EmployeeSpec.

In the latter case, the code generator would then strip off the suffix. It might even be made to recognize and strip off any of the three mentioned suffices. Since the declaration table is not actually used for anything, currently, the prettyness of its name should not matter too much.

On the other hand, the declaration class could be made to have some useful purposes, for example, we might want to allow something like the following (this assumes that the declaration class is renamed to 'Employee' and the cursor class is renamed to 'EmployeeRow'):

public void insertEmployees(Set<Employee> employees)
{
    for (Employee employee: employees) {
        employeeTable.add(employee);
    }
}

and

public Set<Employee> extractEmployees()
{
    Set<Employee> employees;
    for (EmployeeRow row: employeeTable) {
        employees.add(row.getEmployee());
    }
    return employees;
}

Serialization using Groups

In TutorialExample, I added example of how to serialize. That doens't look impressive as I had to use the lowlevel interface because we don't have a templated Table class. Maybe we could add a method in Table to return a factory table or something like that so that Group works better. Or maybe some of the new methods Anirban has added will help.

There is also an unresolved method call to JNI that I can't understand when I just create a Group.
Maybe it will be resolved after point 1) above.