bartfeenstra / betty Goto Github PK

On any page, citations are numbered, with the numbers being specific to that page (the first citation being 1, but that same citation may be the fifteenth on another page). However, as the following example shows, when citations are rendered for a specific claim, they are not sorted by number:

 Birth of Lankester, David Marinus
May 5, 1847 in Vlissingen [27] [28] [15] [29]

Clean private people without non-private descendants. They do not provide any linkage between non-private people.

Now plugins can declare before or after which other plugins they come, we can simplify configuration files by making plugin configuration a dictionary, keyed by plugin name, and with each plugin's configuration as values. The current list was introduced to ensure order, but we do not use that anyway.

Add canonical <link> elements to every page for specific resources to prevent duplicates in search engines.

If we want nginx to support HTTP OPTIONS request, add the following:

if ($request_method = OPTIONS) {
        add_header Allow "OPTIONS, GET";
        return 200;
}

maps.js now uses ancestry.json.j2 to get the necessary places data to render maps. Instead, we should add this data to the HTML pages using RDFa and read that from within maps.js.

Dates such as 1829-10-00 should be parsed with a year and a month, but without a day.

Persons now have shortcut methods for their births and deaths, and we use those in our templates. Instead, we should have shortcuts for start-of-life (birth, baptism, other?) and end-of-life (death, burial, other?) events. We should probably also move these shortcuts from our models to the templating layer.

Allow the nginx plugin to use an alternative www_directory_path so the root directive will no longer have to be hackily replaced after site generation if the site is deployed to another system.

Similarly, allow an explicit configuration setting to deploy the Betty nginx configuration as an upstream backend behind a proxy. This means disabling HTTPS.

Add a functional test for the nginx configuration. We can use a Docker container to run nginx in a platform-independent way.

To give credit where credit is due, we must introduce references. Gramps supports citations, sources, and repositories, and I am wondering if we can group those in a generic references data type, where we allow a hierachy between references. One references can then be a volume in an archive, which itself is a references within a larger organization or archives.

To get an idea of what we must support, possible references types, or properties of possible references types:

• People (real-life or virtual, with contact information)
• Online organizations
• Physical buildings (should we use Place references?)

Ancestry trees should not render missing names at all, but render them as "null" now. Instead, maybe add rendering functions for sortable (Doe, John or ..., Jane) and isolated (John Doe or Jane) for use in different contexts.

Maps' zoom controls are visible through the expanded responsive navigation.

Use HTTP 301 instead of 307 redirects when handling requests for unspecified locales in nginx.conf.j2.

The CLI currently acts as one command. If we instead make this a subcommand, such as betty generate ..., we open up the possibility to add new subcommands in the future without having to break BC then.

Change the 'latest' test run to be on Node.js 12, and add a separate test run for Node.js 10.

Test the produced sitemap using the official XSD.

Change the 'latest' test run to be on Python 3.8, and add a separate test run for Python 3.7.

Test on Node.js 13, which came out last month.

The search results sometimes show incorrectly translated instructions.

The templating system has two entity sort filters: locale_sort and sort_places. Make sure these names are consistent with one another.

Currently events are strings, and event parameters are zero or more values passed on when invoking the dispatcher.

Instead, if we turn events into classes extending a new Event class, we can introduce new functionality without breaking BC, such as propagation control, and additional parameters.

Move generation to plugins, and let plugins expose generators. A genrator can output public data and ahould then define a content define the formats (content types, etc) they can generate. By default we ship with a plugin that generates text/html, but we can later add others, such as application/ld+json , or even one for nginx configuration.

Cache Wikipedia responses based on the HTTP Response Cache headers.This reduces traffic increases build performance.

Some files have insufficient read permissions locally, and those are kept when rsyncing them to the server, causing them to be inaccessible to the public. The deploy script must set these permissions itself.

Ancestry trees are now rendered for single persons as well, which is useless. Hide them in that case.

Move the footer to its own template, so when sites want to override it, they won't have to chase changes to the base template all the time.

We now privatize people unless we can determine they or at least one of their descendants have been dead for at least 125 years.

This means we still privatize people who have long since been dead.

Additionally, also check if people's ancestors have been dead for enough time, and if so, do not privatize them. Example: If someone's parents have been dead for N125 years, or their grandparents for 2N250 years, or their great-grandparents for 3N*125 years(, etcetera), do not private that person.

Find a good value for N. In practice, if someone's parents have been dead for 125 years, they themselves have probably been dead for 75-100 years already as well.

Site-level

These may require us to provide Betty with a site configuration file containing the metadata to render.

• Authorship (#279)
• License
• robots.txt (#31)
• Sitemap (#31)

Resources

• <time> for Date objects with year, month, and day components (#69)
• FOAF for people (#85)
• <address> for places (#70)
• Purl for places
• JSON-LD (#256)

Facebook

• OG (#86)
• https://developers.facebook.com/docs/sharing/webmasters

Twitter

• Cards: #1194
• https://developer.twitter.com/en/docs/tweets/optimize-with-cards/guides/getting-started.html

#25 shows we want features that require we know more about the resources (pages) we render:

• Created time
• Last modified time (rendered time?)
• Media (images, are other types relevant?)
• Authors

We can create a resource object when starting the rendering process of a new page, give templates read/write access, and destroy it once the process ends.

Can we keep a record of all rendered resources, so static assets such as the sitemap can inspect them?

Make the navigation responsive

Implement Jinja2 expression statements

Log events, and display them in the CLI.

Add a mode (development/production), to bet set in the config file, and perhaps in the CLI. We should also pass this on when invoking node.js modules. Tools like webpack can choose (not) to optimize their builds based on this setting.

To make place pages more relevant, include summaries from Wikipedia (this can apply to people and events as well, but in practice we haven't come across anyone or any event important enough to be on Wikipedia yet).

Proposed solution

Use the basic query API. Example.

Betty now has one main UrlGenerator class. Turn this type into an interface, and create three implementations:

• PathUrlGenerator to replace UrlGenerator._generate_for_path.
• IdentifiableUrlGenerator to replace UrlGenerator._generator_for_identifiable.
• 'DelegatingUrlGeneratorto replace the delegating functionaly ofUrlGenerator.generate`

Pages without titles result in HTML documents having titles consisting of a separator and the site title, i.e. - Jane's ancestry. We should remove the separator if there's no page title, such as Jane's ancestry.

Reference people from events. Verify if this is a zero-or-more or a one-or-more relationship. Also see if we can and want to limit the maximum number of people for certain event types. Primary/inevitable life events such as birth and death apply to a single person only, while other events are between people, such as marriages.

• Can be enabled and configured in the configuration file
• Respond to events

Currently the only way to add additional JS is to write a plugin that depends on the Js plugin, and that includes its own JS entry point. This requires a third-party dependency that may not be available, and involves considerable complexity for something that should be really simple.

Can we instead come up with a solution that lets plugins simply expose their JS files?

Definition of done

• Allow Betty to be configured to use one or more locales.
• Consider whether a default locale should be used.
• Allow user-facing strings to be translated.
• Allow dates to be formatted according the configured locale. Keep in mind while in practice Betty has only been used with Gregorian dates, date parts (days, months, years) may be missing, and in the future support for different calendars may be required.
• Consider whether it is a good idea to avoid all-numeric date formats (YYYY-MM-DD, DD-MM-YYYY, or the American MM-DD-YYYY) altogether to avoid confusion.
• For non-clean URLs, add the language to the URL.
• For clean URLs, allow content negotiation by the web server.
• For clean URLs, allow users to specifically request a particular language without using content negotiation as well.
• Update the generated nginx configuration to reflect this functionality.
• Update the <html> tag.
• Ensure JavaScript uses the <html> tag to read the document's language, and displays strings in the correct one.
• Allow plugins and sites to provide extra translations located in their resources directories.
• Extend the sitemap with localized URLs.
• Localize betty.webmanifest.

With a generic URL generator, calling code will not have to know what kind of resource they are dealing with, and what paths to use. An example of what this lets us do, is to list all references (which can come from different types of resources) on citation pages.
This should also ensure clean URLs have trailing slashes.

The maps and trees plugins each individually serialize the entire ancestry to JSON, which is then included in the compiled JS file, taking up unneeded space.

In order for users and plugins to override parts of the content that are generated, introduce a file management solution that supports the following primary features:

• When looking for a file, allow multiple names to be suggested, ranging from more specific to less specific, such as person-P0001.html.j2 and person.html.j2. This allows overrides for specific content, and fallbacks for general usage.
• When looking for a file, or when copying entire trees, allow multiple source directories to be suggested, ranging from more specific to less specific. This allows plugins and end users to override resources such as images and templates, and to provide resources that are more specific than what the core package provides (see the previous primary feature)

We currently rely on the jsonschema module for global configuration. However, most configuration comes from plugins, for which no validation exists.

Instead, introduce a generic ConfigurationError that can be raised, and replace the current JSON Schema-based validation with Python-based checks.

Currently the only way to add additional CSS is to write a plugin that depends on the Js plugin, and that includes its own JS entry point which in turn imports its own CSS file(s). This requires a third-party dependency that may not be available, and involves considerable complexity for something that should be really simple.

Can we instead come up with a solution that lets plugins simply expose their CSS files, and where we merge it into the core package's betty.css?

Gramps allows events to be assigned to families, while these events are actually for specific individuals (marriages, divorces, etc). As a consequence, marriages are now rendered between all family members, for example.

Find a way to determine which individual people family events are for, and implement that in _parse_family().

The HTTP 403 error page says "Page not found" instead of "Access denied".

To prevent expensive operations from slowing down rendering, allow configuration files to specify a caching directory. This can be used for image manpulations and #144, for instance.