Giter Club home page Giter Club logo

kiwix-js's Introduction

Kiwix

Kiwix is an offline reader for Web content. It's especially thought to make Wikipedia available offline. This is done by reading the content of the project stored in a file format ZIM, a high compressed open format with additional meta-data.

COMPILATION INSTRUCTIONS

Take care, the paragraphs are about the target platforms. If you want to build Kiwix for Android on a GNU/Linux system, you should follow the instructions of the "Android" paragraph.

GNU/Linux

Install pre-requisties in your distro, eg, in Debian based:

sudo apt-get install zip pkg-config aptitude
sudo apt-get install libmagic-dev
sudo aptitude install libtool automake

Static (Probably what you need to do if you don't know)

Kiwix uses shared libraries only. A static build of Kiwix is a build which packages the dependencies. Command line tools (indexer, server, etc) are compiled statically.

Run automake

./autogen.sh

Run autoconf

./configure --enable-compileall --enable-staticbins --disable-android;

Download all dependencies

cd src/dependencies;
make;
cd ../..;

Reconfigure with dependencies (Gecko version)

./configure --enable-compileall --enable-staticbins --disable-android;

Compile

make;

Creates tarbal suitable for distribution (no deps)

make diststatic
  • If you want to be able to run directly from your dev repository copy the "xulrunner" directory in the distributation tarball to the "kiwix" sub-directory of your developement tree.

  • Don't try to install with "make install", if you want to have kiwix install on your system just copy the directory in a /usr/local/bin directory.

Dynamic

GNU/Linux build uses shared libraries by default. You will need the following dependencies (also -dev packages) to compile Kiwix for Linux (this list can vary a little bit depending of the GNU/Linux distribution):

  • g++
  • gcc
  • autoconf
  • automake
  • libtool
  • pkg-config
  • liblzma
  • libicu
  • libmicrohttpd
  • zlib
  • libcrypto++
  • cmake
  • wget
  • aria2
  • libuuid
  • libssl
  • libzim
  • libpugixml
  • libctpp2
  • xulrunner

Debian is the only distribution providing natively all these packages. Ubuntu provides most of them, excluding xulrunner and zimlib. You will have to download and install them separatly or run configure --with-static-dep=SELF. Then run the following commands:

Run automake

./autogen.sh;

Configure the compilation check --help for options. Most dependencies accept --with-dep=XX and with-static-dep=XX.a. Static version of libraries are used for building static binaries (server, indexer, etc) Use --with-dep=SELF or --with-static-dep=SELF to trigger fetch and build for the dependency.

./configure;

Compile

make;

Optionaly install on the filesystem

sudo make install;

Mac OSX Universal

WARNING: To build the Mac OS version you will have to install proprietary software which are free of charge. You will also need to build on an Apple Mac computer.

Configure Macports

Install the following tools and libraries

sudo port install autogen +universal pkgconfig +universal wget +universal gmake +universal coreutils +universal libidl +universal autoconf213 +universal icu +universal;

NOTE: the following commands seem to work better on OS X Yosemite, apparently 'universal' is less relevant and various packages are no longer available as universal options.

sudo port  install autogen pkgconfig  wget  gmake  coreutils  libidl  autoconf213  icu;
sudo port -v install aclocal automake libtool autoconf cmake imagemagick
./autogen.sh alt && ./configure --enable-compileall;
cd src/dependencies && make;
make clean;
./autogen.sh alt && ./configure --enable-compileall --enable-compileall --disable-dependency-tracking --with-target-arch=i386;
make;
make distmac;
make clean;
./autogen.sh alt && ./configure --enable-compileall --enable-compileall --disable-dependency-tracking --with-target-arch=x86_64;
make;
make distmac;
make universal;
Debugging tips:
  • components file type can be either Mach-O dynamically linked shared library or Mach-O bundle but above instructions will create dynamic libraries.
  • Shared Object (.so files on Linux) have .dylib extension on OSX.
  • use dtruss to inspect program execution like strace.
  • dyldinfo -lazy_bind | dyldinfo -bind components/zimAccessor.dylib |grep zim
  • nm -gm components/zimAccessor.dylib | nm -u | nm -g
  • otool -L libzim.dylib
  • install_name_tool -change @executable_path/../libicuuc.dylib libicuuc.dylib kiwix-serve
  • lldb

Android

Look at android/README

Windows

  • Install Windows XP SP2+

  • Install Visual Studio Express 2010

  • Install 7-zip

  • Install MozillaBuild 1.6

  • Install ActivePerl

  • Install Ruby

  • Install NSIS 2.46

  • Install nsis_locate

  • Install nsis_uac

  • Replace installed UAC.dll by new one.

  • Install all software in default locations.

  • Change your Windows PATH environment variable:

    • 7zip
    • NSIS
    • ruby
    • Perl
    • mozilla-build\msys\bin\
  • Get shell from c:\mozilla-build\start-msvc10.bat

mkdir -p /c/slave/windows-32b
git clone git://git.code.sf.net/p/kiwix/kiwix kiwix
cd kiwix
./autogen.sh alt
./configure --disable-indexer --enable-jar
make win
make windist
make wininstaller

Contact

Email: [email protected] or [email protected]

Jabber: [email protected]

IRC: #kiwix on irc.freenode.net

You can use IRC web interface on http://chat.kiwix.org/

More... http://www.kiwix.org/wiki/Communication

LEGAL & DISCLAIMER

Read 'COPYING' file

kiwix-js's People

Contributors

amitpanwar789 avatar ankur12-1610 avatar aritrocoder avatar ashiscs avatar cibersheep avatar d3v-d avatar dan-niles avatar dependabot[bot] avatar dheerajdlalwani avatar ernesst avatar greeshmanth1909 avatar hrishi-raj avatar ismaelvega avatar jaifroid avatar julianharty avatar kelson42 avatar krinkle avatar maneet-k avatar mossroy avatar peter-x avatar popolechien avatar ram690514 avatar rgaudin avatar rishabhg71 avatar sharun-s avatar soloturn avatar taniamalhotra avatar try-catch-stack avatar vasusharma7 avatar ykabusalah avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kiwix-js's Issues

Improve the look and behavior of the search results

The results should be better separated, and be highlighted when the cursor is above (on desktop)
On desktop, using the arrow keys should let the user move through the results, as a kind of "auto-suggest" feature

Usage of the search field is not completely clear at first use

It seems not visible enough. Maybe simply outlining it?
If we simply put the focus on it, it will be naturally outlined byt bootstrap, and the keyboard would also appear

Or maybe center this search field in the page?
Some users were confused by the search icon on the right of this field, and thought they had to click on it before searching

Optimize bzip2 decoding, if possible

Uncompressing an article is very slow on a device : usually around 5s on a Geeksphone Peak.

But I think this will be hard to improve.
Maybe another javascript implementation : https://github.com/kirilloid/bzip2-js or https://github.com/skeggse/node-bzip/blob/master/node-bzip/index.js
The bzip2 compression format seems to support multi-core decoding. On the Peak device, it would be useful (dual-core CPU), but I think that all these javascript implementations do not support multi-core decoding
A better way to optimize this would be to have a b2g API like https://wiki.mozilla.org/WebAPI/ArchiveAPI. But that does not seem to be planned by Mozilla for now...

Add back/forward buttons

Ideally, it would be the standard buttons of the browser.
Like the ones that appear at the bottom of web pages in Firefox inside Firefox OS

If not possible, it's easy to add buttons with javascript that does that

Avoid scanning the sdcard at every startup

Currently, the sdcard is scanned at every startup of the application, and the user has to choose which archive he wants to use.
This should be done only once : the selected archive should be stored on the device, so that the sdcard is not scanned the next time.
The user should also have a way to change the archive stored in its user preferences, in the "Configure section"

Implement "articles nearby" feature (geolocation of articles)

I would geolocate the device, and display a list of articles nearby.

The algorithm to search articles of an archive based on their location seems a bit complicated to implement, but it works on the Qt version of Evopedia
On this Qt version, the articles are also displayed on an openstreetmap map. But the issue is how to have the tiles without Internet Access?
For now, I won't try to locate articles on a map, just display a list of articles nearby

Switch back to Zepto.js, instead of jQuery?

jQuery adds the support for Internet Explorer (9+), including (I suppose) the one on Windows Mobile.

Zepto supports all the other major browsers/devices, and is a bit faster and lighter.
Zepto is less tested with other libraries like Bootstrap. It also needs to be compiled with modules selector and data, which are not in the default version of Zepto.

It seems possible to install both, and choose jQuery only when using IE. But it's not in a require.js way (see doc of Zepto and below)

<script>
document.write('<script src=' +
('__proto__' in {} ? 'zepto' : 'jquery') +
'.js><\/script>')
</script>

Use webworker for bzip2 decoding

As this step takes a long time (~5s on a device for an article), it should not block the browser.
The user feedback would be better (spinner icon turning instead of being stuck)

Fix links between articles when special characters are used

The current implementation uses the same search algorithm as the prefix search. It simply takes the first result.
While it works in most cases, it can lead to a wrong article when there are similar titles that are the same when normalized (with different accents for example).
For example, make a search with "ascii" in the French dump, and display the first article "Ascii". At the end of paragraph "Table des 128 caractères ASCII", there are links to the article of each letter. If you click on the letter A, it should lead you to the article named "A_(lettre)". But it leads you to the article named "Å (lettre)", with an accent on A.

Some links that use an anchor give encoding errors

Example : in the French dump, display the article "Lampassé". There is a link in the second paragraph with name "Léopard". The URL is "Lion_(h%C3%A9raldique)#L.C3.A9opard_et_L.C3.A9opard_lionn.C3.A9"
When you click on it, it gives an error "malformed URI sequence" in utf8.js

I suppose it comes from the strange encoding after the anchor
It works properly in the original application

Make error handling code cleaner and safer

The error handling code is not very clean.
We sometimes test on the error message, which is very bad practice
We sometimes put the error message as the article content
We also call alert() from the algorithm, which should not have a dependency on the GUI.

I'm not proud of all this : it's not clean and not bullet-proof

Add a way to easily download Evopedia archives

It's a bit complicated for the end-user to download an archive separately (with a bittorrent client), then to put it on the sdcard of its phone. It might be a blocker for some.

An idea would be to put a bittorrent client inside Evopedia (as in the original application). The archive could be downloaded directly from it.
Maybe using http://btappjs.com/ or a similar library.

In any case, downloading all these GBytes will take numerous hours, and should be done through an unlimited internet connection (i.e. usually WiFi instead of 3G/4G)

When using hyperlinks in an article, you sometimes end up on another one, with a slightly different name

It's caused by the current implementation of the readArticle function : it re-uses the prefix search, and takes the first result.
When the article can not be found, or if there are several with the same name (after noramlizing), you can end up on the wrong one.

The readArticle function should be re-implemented to lead only to the exact same name. And display an error if it is not found.

On the "small" dump, this behavior can be very easily seen, as most hyperlinks lead to articles that do not exist in the dump.

The article "Mi" cannot be displayed on French dump

On the French dump (2013-02-16), when searching for the "Mi" article (with an "i" without accents and in small case), the article can not be displayed : nothing appears on the screen, and no error message

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.