Giter Club home page Giter Club logo

5balls / coleitra Goto Github PK

View Code? Open in Web Editor NEW
0.0 1.0 0.0 1.09 MB

Cross platform vocable and grammar trainer using spaced repetition algorithm - unfinished rewrite of existing and working (but due to license reasons unpublished) app / program

Home Page: https://coleitra.org

License: GNU Affero General Public License v3.0

CMake 0.54% CWeb 23.26% C++ 64.28% QML 7.63% QMake 0.40% MATLAB 0.97% C 2.02% Dockerfile 0.61% Vim Script 0.28%
vocabulary-learning vocable-trainer grammar android app desktop vocable spaced-repetition spaced-repetition-algorithm flahscards

coleitra's Introduction

coleitra

Attention

Unfinished software! This software is not in a usable state yet. Please wait for the first release!

coleitra is an open source (AGPL-3.0-only) vocable and grammar trainer using spaced repetition algorithms. It's intended usage is on a mobile phone with an android operating system but it can be compiled for the desktop as well. This is the source code repository of the program, for more information about the program itself check the coleitra webpage.

In principle the used toolkit Qt also compiles for iOS but I can't support this operating system right now.

This document is written in the markup language called "reStructuredText". This language is used in the python programming language for documentation, you can use the Docutils text processing system to create a nicely formatted version of this document but it should be readable in plain text as it is.

Install

The program is written following the programming paradigm of literate programming introduced by the computer scientist and mathematician Donald Knuth.

Instructions assume you have a linux shell available and are familiar with using it (If not it is easy to learn though - search for bash tutorial and there should be plenty available to get you started.). It should be possible to compile this program on other operating systems as well but I currently only use linux so I can't provide any help there.

The documentation, the program source code and some other necessary files are contained in the doc directory in the format for the "nuweb" program and you need the nuweb program to get both documentation and the binary of the program. The nuweb program is not to be confused with the noweb program which is also used for literate programming.

Requirements

nuweb

Unfortunately nuweb does not seem to be included in major linux distributions. Download the latest release from the nuweb webpage and follow the instructions in the README file (They probably tell you to run make nuweb on your shell to build an executable file called "nuweb").

pdflatex

Install some variant of pdflatex from your operating system. You probaby need to install some packages which have names starting with "texlive".

Qt5

Install either the development packages for Qt5 (these are usually different from the "regular" library packages) or compile them yourself from source. Unfortunately I can't recommend using the installer from Qt itself as it requires registering with a seperate account and I strongly disagree with this decision of the Qt team to forcefully collect user data.

You should follow the instructions of Qt - either on their webpage or in the source tarball, but for personal reference, this are the commands I used last for compiling the libraries for cross compiling for android (you need the android SDK and NDK for cross compiling the android app, see below):

./configure -xplatform android-clang \
--disable-rpath \
-nomake tests -nomake examples \
-android-ndk ~/src/foreign/android-sdk/ndk-bundle \
-android-sdk ~/src/foreign/android-sdk \
-no-warnings-are-errors \
-openssl-runtime \
-optimize-size \
-I ~/src/foreign/openssl-1.1.1i/include \
-prefix ~/src/foreign/qt5-android-install-20201222
make
su
make install
exit

Choose the open source license and accept the LGPLv3 offer. It may take quite some time to compile Qt as it is a large library (expect several hours of compile time depending on your setup) to speed up the process you can make use of multiple cores by adding -j4 to the make commnd (in the case of four cores for example). If compiling the desktop version on linux the -xcb switch seems to be needed or at least the required dependencies to be able to add this option, maybe it is automatically compiled when the dependencies are fullfilled.

It might help to pass also the -ltcg flag to configure to enable link time optimization and make the resulting binary smaller but I could not make it work yet.

Qt5 Debian package installation

Tested on debian version 11.2 (bullseye); might not be complete:

For compiling:

apt-get install qtbase5-dev qtdeclarative5-dev libqt5svg5-dev
apt-get install qml-module-qtquick2 qml-module-qtquick-controls qml-module-qtquick-controls2

Android SDK and NDK

You don't need Android Studio to compile coleitra. Download just the commandlinetools package (it is usually a bit hidden on googles webpage, you might need to scroll down quite bit), at the time of this writing the file was called commandlinetools-linux-6858069_latest.zip located at this place but that may change.

mkdir ~/src/foreign/android-sdk
mkdir ~/src/foreign/android-sdk/cmdline-tools
unzip commandlinetools-linux-6858069_latest.zip
mv cmdline-tools ~/src/foreign/android-sdk/cmdline-tools/tools
export PATH=$PATH:~/src/foreign/android-sdk/cmdline-tools/tools/bin
export ANDROID_SDK_ROOT=~/src/foreign/android-sdk
sdkmanager ndk-bundle
sdkmanager "platform-tools" "platforms;android-28"

You have to agree to googles license agreement to continue. Directory structure seems to have changed, but this seems to work for the current version.

OpenSSL

Qt5 needs to be configured with OpenSSL which is needed for https requests. Download the last stable version from the OpenSSL webpage, at the time of this writing this is version 1.1.1.. Follow the instructions to compile it for android, in my case this is written in

export ANDROID_NDK_HOME=~/src/foreign/android-sdk/ndk-bundle
export PATH=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin:$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin:$PATH
cd openssl-1.1.1h
./Configure android-arm -D__ANDROID_API__=21
make SHLIB_VERSION_NUMBER= SHLIB_EXT=_1_1.so build_libs

The extension of the libraries needs to be changed from standard naming because android does not seem to like libraries which don't end on .so, so libssl.so.1.1 is not working while libssl_1_1.so is. make install will not work with this extension but this is fine we don't need it.

cmake

Install the cmake package from your operating system.

LAPACK

Install a lapack library package from your operating system, on debian one possible package is named liblapack-dev.

f2c

Install the f2c package from your operating system, on debian the package name is "f2c". (This might not be necessary. It may be needed by the original LAPACK version which was written in fortran.)

nlohmann JSON

Install json parsing library from Niels Lohmann per source from https://github.com/nlohmann/json or as package your the distribution (Debian package is available).

JSON schema validator

Install the JSON schema validator library from Patrick Boettcher per source from https://github.com/pboettch/json-schema-validator or via package manager (I think there is no debian package yet) and install it somewhere where cmake can find it.

coleitra

Compile documentation and create coleitra source code

Run the following code in your shell (pdflatex needs to be run twice as well as nuweb):

cd doc
nuweb -lr coleitra.w
pdflatex coleitra.tex
makeindex coleitra.idx
pdflatex coleitra.tex
nuweb -lr coleitra.w
cd ..

Compile desktop version of coleitra

Run the following code in your shell (the command line tools git and tr are expected to be available):

cd build/x64
cmake ../../src
make

If you have compiled Qt5 at a nonstandard location or in addition to your system libraries (which is not a problem) you have to pass the correct path cmake, using CMAKE_PREFIX_PATH, for example:

cd build/x64
rm -r *
export CMAKE_PREFIX_PATH=/home/flo/src/foreign/qt5-install-20201127
cmake ../../src
make

Compile android version of coleitra

This requires a local installation of the android ndk and sdk. You can download those seperate from the android studio which you don't need for compiling coleitra.

cd build/android
rm -r *
export ANDROID_SDK=~/src/foreign/android-sdk
export ANDROID_NDK=~/src/foreign/android-sdk/ndk-bundle
export JAVA_HOME=/usr/lib/jvm/default-java
cmake -DANDROID_PLATFORM=21 \
-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DCMAKE_PREFIX_PATH=~/src/foreign/qt5-android-install-20201222/ \
../../src
cp ~/src/foreign/qt5-android-install-20201222/jar/QtAndroidNetwork.jar coleitra-armeabi-v7a/libs
make

That the jar file is not copied seems to be a bug in recent Qt versions, there is probably a more elegant way to do this. You might not need to set CMAKE_PREFIX_PATH and CMAKE_FIND_ROOT_PATH_MODE_PACKAGE if you have installed the Qt5 libraries for cross compiling for android system wide. Also this might download quite some android stuff on the first run. Subsequent runs should be faster.

coleitra's People

Contributors

5balls avatar

Watchers

avatar

coleitra's Issues

Implement current (or similar) algorithm

  • Document old algorithm
  • Implement the algorithm used in old coleitra. This should get us started for a working version.
    There should be a real function interface so we can change between algorithms easily and on the fly. (Postponed for later milestone).

Implement delayed grammar lookup

There should be an option to delay the grammar lookup to later. This will simplify input of vocables.

  • Refactor context dependent code to be context independent
  • Clean up lookup code
  • Implement schedule cache option

[Bug]: Testreport

coleitra version

unreleased

Operating System

Debian 11

Git commit

8cd4f7f

Description

I need to test this form!

Steps to reproduce

  1. Got to the issue page
  2. Choose the right template
  3. Fill everything out

Perceived severity by user

Low

Additional information

This forms are a useful feature.

Code of Conduct

Try windows cross compile

Would be nice if it works. Not strictly necessary right now I guess

  • Figure out if mingw is enough (maybe clang can do it as well?)
  • See, if we can use precompiled Qt lib for windows
  • See, if we can use some stupid installer (Nullsoft?). Is there something nicer for windows 11?

Make grammarprovider configurable

Implement all stuff needed to just use json files to configure / extend different language support in grammarprovider

  • Make reasonable data structure
  • Load json files found into such list of data structures
  • Implement calling function mechanism for this data structure

Fix database schema

There are some weaknesses in the old coleitra of the database schema / the implementation of the database handling:

  • Foreign keys are probably not used throughout
  • Languages are not stored as a separate table but strings are used
  • Only the time when a vocable leaves the buffer is stored (i.e. when it is known), but it should be stored, when a vocable is displayed and when it is not known as well
  • There should be an easy way to delete all rows depending on a foreign key postponed for later
  • There is still some bug in the old code, where the program wont start with an empty / new database - this should be covered in a unit test Unit tests should be a separate issue
  • Access to the database should be done without multiple database objects, if necessary there should be a singleton
  • Make sure it is possible to store SM information with the vocable (i.e. degree of knowing) even if not used right now in any algorithm

Implement support for configuration of grammarprovider

Basic JSON / JSON schema support is now included and we have a first schema. Now we can implement a generic approach to work with configuration files.

  • Create datastructure for json data
  • Check if file for language is available in getGrammarInfoForWord and load if not already loaded
  • Write generic function for parsing wiktionary pages if language file is available
  • Convert static functions into configuration files for languages
  • Remove static remains of language specific parsing

Add lua interfaces

Add lua again (should be fast as it works just fine on old coleitra).

But than also think about how to elegantly replace compiled C code at runtime with LUA scripts where possible and safe. This requires exposure of the necessary functions.

Implement reasonable vocable editor

There should be a better vocable editor than in old coleitra to make it usable for people other than myself.

  • Find more clever way to access database singleton (could be QObject parent of edit class)
  • Create good edit element for entering text
  • Create expert mode which allows entering everything
  • Create simple mode for easy entering
  • Modify expert mode for search / editing capability
  • Modify simple mode for search / editing capability

On further thought I think search / editing capabilities don't belong in those modes. Database mode is better for this and fine (should be complete though)

Implement grammar provider

Might put this in a seperate repo for license reasons but this should be supported on both ends, so it needs to be here as well.

  • Implement parser for wiktionary tables (is ok license wise because we don't use the LUA code)
  • Implement grammar provider based on this parser
  • Implement android service which provides grammar information from wiktionary (This convoluted arrangement is necessary for using the LUA scripts from wiktionary which are under an incompatible license).
  • Implement android service which provides grammar information from other code (Find this open source code again)

Android services are something for much later. I consider the base functionality of the grammar provider working good enough for version 0.1 (the whole grammarprovider was originally planned for a later release 0.2, but we needed to implement this earlier).

Fix android compile

Android compile is probably broken right now due to new libraries. Have not done any tests to compile it on my new laptop.

  • Install android SDK
  • Install android NDK
  • Compile OpenSSL for android (try to get it right the first time)
  • Compile android variant of Qt libraries (try to get it right the first time)
  • Compile nlohmann json library for android
  • Compile json schema parser library for android
  • Try to compile LAPACK for android
  • Compile levmar for android
  • Build APK package
  • Document android compile in new Install chapter in coleitra.w

Implement GUI for editing grammarprovider json file

We need an easy way to extend the grammarprovider

  • Write Qt TableView (C++) for json data structure for language
  • Write QML Element for showing this as editable table
  • Write interfacing functions for QML to show it from context with filled out table from editing mode (to easily switch back and forth between this mode and editing mode)

[Bug]: Grammarprovider destructor crashes program

coleitra version

unreleased

Operating System

Debian 11

Git commit

No response

Description

Bug does occur always when quitting program from simple edit mode.

Steps to reproduce

  1. Goto Simple Edit mode
  2. Quit program
  3. Observe segmentation fault

Perceived severity by user

No response

Additional information

Order of destructors is not enforced right now and m_grammar_provider is not available anymore when the destructor is called. Do we even have to disconnect this signals or should we care?

Code of Conduct

Implement reasonable training window

It should be improved over the old coleitra regarding the following points:

  • Text should be scaled to fill available space as large as possible (with necessary line breaks)
  • Multiple meanings should be handled correctly (it should be possible therefore to divide space further and display multiple vocables in the divided spaces - a minimum size where a scrollarea is implemented should be postponed for a later version though)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.