• Route definitions
• Request handlers
• Response data transformations
• Utilize the lang component (in particular, the db connection)

Depends on #23

This epic is composed of the following:

• Add language component - #8
• Add language and name generator support to language component - #18
• Update all library calls to use the db instead of reading from the filesystem - #23
• Create a REST API for the language app - #22
• Update CLI to use the new REST service #16
• Add CLJS CLI to circumvent long JVM startup times #25

Using RedsSeach, see:

• https://github.com/RedisLabs/JRediSearch
• Doesn't look like there's a Clojure client ...

Depends upon #26

• Come up with a name (use the planned feature in #42)
• Create a new namespace for this world's languages
• Start adding languages (see #41)
• Add corpora for new languages (see #39)
• Update the CLI to support generating content for this world's languages (see #44)

Syntagmata* and Markov* (and common-impl) require access to the running system state right now; unfortunately, when they are created, only a partial system state is known. A new abstraction is needed such that with every function call, the latest running system is passed.

This may mean reverting the current API to its pre-component form, and then creating a component-aware, wrapper API for it.

We can use the same ClojureScript/Node.js setup here we used for UDP client support in hexagram/dice (but instead use cljs-http instead of a UDP client, of course).

See https://github.com/rm-hull/markov-chains

Tasks:

• Create component
  • Add markov and syntagmata generators to :lang map
  • Add db connection to :lang map (using the plugin)
• Define API for accessing data via db (instead of filesystem reads)
  • BLOCKER: Re-work the plumbing around generator implementations #28
  • Generating assembled content
  • Generating random content
  • Generating names
  • Reading stats
• Add appropriate LangDB schema keys to redis plugin
• Update the lein aliases to use the new language component
• Update CLI tools to use the new language component

... and other sources.

Now that Markov chains have landed, let's use them ...

Corpora names and wordlists need to be updated for many of the languages that will be use for the Northern tribes (see issue #41).

This will require language corpora for the following:

• Lakota
• Inuit
• Finnish
• Dwarvish
• Old Norse
• Elf
• Mongolian
• Tibetan
• Maori
• Hawai'ian

And it will require name corpora for the following:

• Amerindian
• Inuit
• Finnish
• Dwarf
• Old Norse
• Quenya
• Mongolian
• Tibetan
• Maori
• Hawai'ian

Component creation and ingest:

• Create component namespace
• Setup docker for RedisSearch
• Create RedisSearch connection function(s)
• Define a plain text, human readable format
• Create functions that convert plain text dictionary files to the format(s) needed by RedisSearch for ingest
• Upon startup, the dictionary component should read all dictionary files, index them, and load them into RedisSearch

Post-ingest, programmatic access:

• Create defrecord for language, word, part of speech, definition
• Map db entries to record instances
• Create component-based API for querying db (as well as updates)
• Add support for full-text search - #26
• Add an API call for searching dictionary entries - #27

For part of speech, standardize on what is used in OpenNLP (see https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html)

Depends upon #7 and #8

Things to name:

• Create a language group for this area: Óðantu
• Define languages for each major area
• The glacial area
• The tribes of the glacial area
• The arboreal forests between the plains and the glaciers
• The tribes of the arboreal forests
• The rivers in the arboreal forest and plains areas
• The plains between the coastal areas and the arboreal forests
• The tribes of the plains
• The western coast area
• The tribes of the western coast
• The western ocean
• The southern coast area
• The tribes of the southern coast
• The southern ocean
• The eastern hills
• The tribes of the eastern hills
• The eastern bay
• The tribes of the eastern bay
• The islands to the south
• The tribes of the islands

If Markov chain data is generated and saved into an ETS table, can that be exported? And imported? (e.g., during system startup?)

Generate words, word lists, sentences, paragraphs, and names.

In particular, for text analysis & generation

Given data structure containing languages, each with words of the same meaning, and the desired percentage contribution to the output, create a new word from the combinations.

Blocks: #41

Start small, e.g., with room/object descriptions ...

Using data from http://www.geonames.org/

Tasks:

• Add the ability to parse user-provided text - #32
• Support command synonyms - #34
  • Identify command synonyms
  • Standardize phrasing internally
• Devise data structure (AST) for the parsed results of user-supplied commands - #37

• Add post-tokenizer, pre-parser phase - this will allow us to rewrite (internally) things that aren't tagged well by OpenNLP (e.g., "pick up" -> "get" or "take")

Right now they are just concatenated strings. This will require some changes to the internal operations ... might need to convert some of the regex operations to set-theoretic operations....

• Update CLI-based system to start the HTTPD service
• Add http client lib dep
• Add formatting renderers for JSON results

This should make use of a hexagram30 general purpose HTTP component that can be used by all apps.

The CLI will use the spelling ilunao and initially just the 8 Norther tribes will be supported (cultures and languages in the other parts of the world haven't been identified yet).

And add the random component to hexagram30/language's supported components

May have to switch docker image to redislabs/redisearch.

See:

• https://github.com/RedisLabsModules/RediSearch
• https://hub.docker.com/r/redislabs/redisearch/
• https://oss.redislabs.com/redisearch/

We'll need this for db support

This is an interesting problem to solve, and one where I suspect the first iteration will be quite naïve. I suspect:

1. The verb will be the root node of the tree
2. Objects in the world will support protocols (a la Java interfaces), and thus will be able to say whether they support the given action indicated by the verb
3. Deeper structure will only be revealed through experimentation

Needs to be done for both languages and names.
This should be at the corpora level.

Maybe refactor names into languages ...