gklyne / annalist Goto Github PK

When adding a new field from the entity edit view, a view description view is displayed. But this in turn will typically require that a new field type is created. This is a bit awkward, and not obvious.

Options:

1. 'Add field' goes straight to the 'New field' view, and on completion adds the defined field type to the view. This might require some reworking of the internal sequencing logic (e.g., continue to a "view" that adds the field to the view then redirects to the entity view) The downside of this is maybe when an existing view would do.
2. Make the field URI part of the view definition in the view description (possibly defaulting to value in the field description). This would make it easier to re-use existing field type definitions, but may result in less obvious displays (e.g. lacking clues from the field type name).

Needs thinking about.

Renaming a record type with existing data causes the data for they type to be orphaned; i.e. inaccessible.

E.g.

1. Create type t
2. Create data record(s) for type t
3. List all data - new data shows up in list
4. Rename type t to t1
5. List all data again: no sign of new data

The presumed problem is that the RecordTypeData directory is not renamed when the type is renamed. Manual renaming of the data means that the data is once again visible. But the data records will still carry the original type name.

• Think about use of CURIES in data (e.g. for types, fields, etc.) Need to store prefix info with collection. Think about base URI designation at the same time, as these both seem to involve JSON-LD contexts.
• JSON-LD @contexts support
• Alternative RDF formats support (WONTFIX)

Running the test suite on Windows gives inconsistent results.

The cause seems to be erratic behaviour of the Windows file system when deleting and/or copying directory trees. Delete operations fail with permission denied or directory not empty errors.

I've tried to isolate the problem, but have so far failed to find a fix.

Supporting Windows is not a priority for me (but it may be for users), so I'm not scheduling this as a must-have for the initial release.

• based on generic entity edit view, but using different render field options
• update URI dispatching
• include default view

Think about handling of identifier renaming (re-write data, record equivalence, or ...). (See also "data storage".)

• when renaming through edit form, update URI if it corresponds to previous ID??
• need to think about maintaining a list of corresponding URIs?
• Use server-internal revectoring? What about delete multiple?

Currently there's some inconsistency about view handling which may affect this: confirm views are rendered directly (as this allows form parameters to be provided directly), but other form action links are handled by redirection. Try to be more consistent about this? Create a more general pattern for handling continuation forms? Note: redirect means that different view GET function signatures, with values provided in the request URI, are handled in generic fashion. POST views are more easily handled directly with form parameters as supplied dictionary

• Login continuation should be current page, not always profile
• annalist-manager createsite - allow site description to be provided
• annalist-manager runserver - allow IP-addr:port to be provided
• Home page: options to edit collection id, label and site description
• Improve authorization failure messages cf. AnnalistGenericView.authorize, .form_action_auth, displayinfo.check_authorization, entityedit.save_entity, etc.
  • added scope parameter to replace HTTP method. Is this enough?

E.g.

1. at Annalist log in screen, enter user-guest for username, and select Google for the login provider
2. complete the login
3. click on Home menu item

Browser displays "Server error"

But the same sequence using user_guest for the username is fine.

The login view should check the username is a valid_id before proceeding. Also, the permissions access should be more resilient to ill-formed user ids

There is a possible "Define view+list" bug in the software that needs investigating: when clicking "Define view+list" while defining a new type in the type definition view, a message saying the new type already exists may be displayed.

If this happens, , click Cancel then select the new type in the "List types" display and click Edit. Then click the Define view+list button.

(I'm not sure if this Is now fixed.)

If a type is deleted, its corresponding RecordTypeData directory is not removed, which results in warning messages to the Annalist server log.

If there are no entities of the deleted type, the deletion should also delete the RecordTypeData entity and directory.

If data for the type still exists, then the delete operation should be refused.

This list should probably be reviewed and broken out into sub-issues

• note that field selector field enumerations use the currently saved value of the target record type: dynamic changes in the form won't cause immediate updates to the selector drop-downs. Maybe this should be enhanced using Javascript and hiding values rather than suppressing them? (low priority)
• cross-field default values; e.g. field type defines default for value type via field description
• add "new field" logic to entity edit POST handler
• where possible, migrate methods from editentitybase to subclasses
• review logic - ideally, form handlers will access data from form, then hand off for processing

Create facility to built repeat field and group structure for existing simple field

• Currently it gets tedious creating view forms with repeated fields; need to figure a way to streamline this.
• See also discussion below of introducing "tasks" - this would be an early candidate for that.
• Need to think how the interface would work. Option to add "task" button to any form?

These are the notes on introducing "tasks" from the TODO file:
Introduce notion of "Task", based on form, but linked to "script" action.

• Create a "wizard-like" (or one-form) interface for creating type+list+view set.
  • test by creating contacts/supplies listy for CruisingLog
• Create a "wizard-like" (or one-form) interface for creating field+field-group set.
  • needs to create (a) individual fields in group, (b) field group and (c) field referring to group.
• Procedure for creating type + view definition + list definition + field definitions from a simple overview description
• Procedure for creating enumeration type from simple description of options
• Procedure to migrate textual type annotations to enumeration types
• Simplify repetitive data entry; e.g.
  • Use-case: create bibliographic author details from just full name entered
  • derived field (possibly hidden) with a rule to guide its creation from other fields in a view
  • default value from other field (possibly from a derived field)
  • initial value/identifier templates (e.g. create ID from current date)
    • NOTE: default and initial values behave differently
  • "view source" record editing (of JSON), with post-entry syntax checking.

It looks as if the problem is related to lack of an annal:uri field. I was trying to copy a view record without a URI. This line should specify a default value:

  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/annalist/views/entityedit.py", line 307, in form_render
    entityvals.pop(ANNAL.CURIE.uri)
KeyError: 'annal:uri'

Do a code-wide search for dictionary .pop functions and check they specify default values where needed (at one point, I thought the default just defaulted to None).

From the server log:

INFO 2014-11-21 16:59:16,383 views.entityedit.get:  coll_id test, type_id _view, entity_id Data_view, view_id View_view, action copy
ERROR 2014-11-21 16:59:16,464 Internal Server Error: /annalist/c/test/v/View_view/_view/Data_view/!copy
Traceback (most recent call last):
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/annalist/views/entityedit.py", line 127, in get
    return self.form_render(viewinfo, entity, add_field, continuation_url)
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/annalist/views/entityedit.py", line 307, in form_render
    entityvals.pop(ANNAL.CURIE.uri)
KeyError: 'annal:uri'
ERROR 2014-11-21 16:59:16,464 Internal Server Error: /annalist/c/test/v/View_view/_view/Data_view/!copy
Traceback (most recent call last):
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/annalist/views/entityedit.py", line 127, in get
    return self.form_render(viewinfo, entity, add_field, continuation_url)
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/annalist/views/entityedit.py", line 307, in form_render
    entityvals.pop(ANNAL.CURIE.uri)
KeyError: 'annal:uri'

This lists needs breaking down into minor usability fixes, technical code improvements, and significant new features.

Presentaton:

• ?Align type ID values used in local URI construction with type URIs/CURIEs
• Identifier display: try to find label instead of CURIE display; augment sitedata accordingly?

Technical debt:

• Clean up code for site data creation, rather than using a hacked test case
• Rename src/analist_site to src/annalist_root.
• update to Django 1.7
• re-work field rendering to use Django 1.7 capabilities (use code rather than templates? simplify context generation?)
• Convert literal CURIES to namespace references
• entity object should carry URI only. Other fields (host, path, etc. should be generated as required. Suggest use an internal value that allows x.uri.path, .host, etc. as required,) See also issue #32.
• rationalize/simplify fields and methods in site/collection model classes - there appears to be some duplication
• move util.py to utils package, and rename?
• look into using named tuples instead of dictionaries for rendering
  • cf. http://stackoverflow.com/questions/1336791/
• Investigate Python 3 compatibility
  • try running test cases with -3 option (cf. https://wiki.python.org/moin/Python2orPython3)
  • See also: https://docs.python.org/3/howto/pyporting.html

Request URL:http://localhost:8000/annalist/login_done/?state=P5L-vLeiKCJcLSn5-OpK1zoxNDgxMDA2NTA2&code=4/4nz4vfkKdY-gqB0xpJAOXzV4XaymX3rcJzQwOquA_1c&authuser=0&hd=zymr.com&session_state=a06cbf8e7e7256806e37b5bce6ede1ce254f8f65..39ae&prompt=consent
Request Method:GET
Status Code:500 INTERNAL SERVER ERROR
Remote Address:127.0.0.1:8000
Response Headers
view source
Content-Type:text/html
Date:Tue, 06 Dec 2016 06:42:30 GMT
Server:WSGIServer/0.1 Python/2.7.10
Vary:Cookie
X-Frame-Options:SAMEORIGIN
Request Headers
view source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Cookie:csrftoken=U9fQzrrP2dFBfKtDjkSStFgWdHkKjcRd; sessionid=y7fk75jl8bf4fy4b844403f1jr4z7dzf
Host:localhost:8000
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36
Query String Parameters
view source
view URL encoded
state:P5L-vLeiKCJcLSn5-OpK1zoxNDgxMDA2NTA2
code:4/4nz4vfkKdY-gqB0xpJAOXzV4XaymX3rcJzQwOquA_1c
authuser:0
hd:zymr.com
session_state:a06cbf8e7e7256806e37b5bce6ede1ce254f8f65..39ae
prompt:consent

Created entry with default type, tried to save, error with id, fixed and saved again. Then edit and change type. Saved OK, but entry with default type not removed.

Exact steps to reproduce not yet clear - needs investigation

Possibly related to #22 ?

E.g. in CruisingLog collection, place name links in daily log link to to daily log record, not place record.

Consider refactoring form generator around Idiom/Formlet work; cf.

• http://groups.inf.ed.ac.uk/links/papers/formlets-essence.pdf
• http://homepages.inf.ed.ac.uk/wadler/papers/formlets/formlets.pdf
• http://homepages.inf.ed.ac.uk/slindley/papers/dbwiki-sigmod2011.pdf

• CSS style association with entity types, so (e.g.) different types can be colour-coded.

• Missing resource error reporting in:
  • annalist/views/collection.py (missing collection)
  • annalist/views/genericedit.py (missing collection, type, view description, entity)
  • annalist/views/genericlist.py (missing collection, type, list description)
  • annalist/views/recordtype.py (missing type)

1. Create data record
2. Enter label value
3. Save
4. Edit record
5. Reset previously saved label to blank
6. Save record
7. View record again

The originally-entered label is shown instead of the blank just entered. Also, it shows in the list display.

• Align type ID values used in local URI construction with type URIs/CURIEs? See #4
• Identifier display: try to find label instead of CURIE display; augment sitedata accordingly?
• entity list view: add javascript for selection classes (hide checkbox and highlight row when clicked)

The heading displayed before a list of entities is useless clutter. It would be better to display the label from the list description.

Also, the main page heading is arguably useless clutter, though it is taken from the site description metadata. Maybe it would be more useful if taken from the collection label?

From Kingsley Idehen: https://lists.w3.org/Archives/Public/public-lod/2015Feb/0116.html
[email protected]

Kingsley requests at least one of:

1. LDP
2. WebDAV
3. SPARQL Graph Protocol
4. SPARQL 1.1 Insert, Update, Delete.

My comment: WebDAV(ish) was on the original roadmap - see #32. The intent has been to use vanilla HTTP as far as possible (GET, PUT, POST, etc.) and then use WebDAV PROPFIND(?) to enumerate directory contents. The more complex stuff isn't needed.

Kingsley also mentions not to worry about access control, but leave that to the backend. But it woud be the Annalist server, not the browser, that acesses the backend data so there would need to be some way to convey whatever authentication/authosization tokens are needed. My rough plan was to use an OAUTH2/OIDC enabled backend that should be a small extension from the current OIDC authentication logic already used by Annalist, but the details still need to be worked out.

• extend field edit form to include additional fields used.
• extend view edit form to include additional fields used in sitedata
• add more 'annal:field_entity_type' constraints for fields that are intended to be used only with specific entity types (e.g. fields, views, etc.)

• Blob and file upload support: images, spreadsheets, ...
• Field type to link to uploaded file

I am running from the docker inside Ubuntu:

# annalist-manager createadminuser
INFO:annalist.apps:Annalist version 0.1.22 (development configuration)
INFO:annalist.apps:SETTINGS_MODULE:  annalist_site.settings.devel
INFO:annalist.apps:BASE_DATA_DIR:    /usr/local/lib/python2.7/dist-packages/annalist_root/devel
INFO:annalist.apps:CONFIG_BASE:      /annalist_site/.annalist/
INFO:annalist.apps:DJANGO_ROOT:      /usr/local/lib/python2.7/dist-packages/django
INFO:annalist.apps:SITE_CONFIG_DIR:  /usr/local/lib/python2.7/dist-packages/annalist_root/annalist_site
INFO:annalist.apps:SITE_SRC_ROOT:    /usr/local/lib/python2.7/dist-packages/annalist_root
INFO:annalist.apps:STATICFILES_DIRS: /usr/local/lib/python2.7/dist-packages/annalist_root/annalist/static/
INFO:annalist.apps:DB PATH:          /usr/local/lib/python2.7/dist-packages/annalist_root/db.sqlite3
INFO:annalist.apps:ALLOWED_HOSTS:    
INFO:annalist.apps:LOGGING_FILE:     None (output to console)
Admin user name:        admin
Traceback (most recent call last):
  File "/usr/local/bin/annalist-manager", line 9, in <module>
    load_entry_point('Annalist==0.1.22', 'console_scripts', 'annalist-manager')()
  File "/usr/local/lib/python2.7/dist-packages/annalist_root/annalist_manager/am_main.py", line 180, in runMain
    return runCommand(userhome, userconfig, sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/annalist_root/annalist_manager/am_main.py", line 169, in runCommand
    status   = run(userhome, userconfig, options, progname)
  File "/usr/local/lib/python2.7/dist-packages/annalist_root/annalist_manager/am_main.py", line 120, in run
    return am_createadminuser(annroot, userhome, options)
  File "/usr/local/lib/python2.7/dist-packages/annalist_root/annalist_manager/am_createuser.py", line 177, in am_createadminuser
    if User.objects.filter(username=user_name):
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 145, in __nonzero__
    self._fetch_all()
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 966, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 265, in iterator
    for row in compiler.results_iter():
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 700, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 485, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: auth_user
root@3fce21e26d9d:/# 

Steps to reproduce:

Modify Default_view in new collection. It now shows up in the default list of all entities.

Click on it, change it's Id and click 'Save'

Message displayed is "Record view in collection does not exist"

Expected behaviour is that record is renamed.

But, if item checkbox is selected and edit button used, rename works OK.

• Enumeration type for annal:value_type value types (text, longtext, etc...); use in fields display
  • This won't work immediately as value types MAY be arbitrary identifiers; i.e. not limited to internal values. (NOT NEEDED following re-work in release 0.1.20)
  • Possible candidate for new render type? (NOT NEEDED following re-work in release 0.1.20)
  • Consider possibly allowing user to add new type URIs, and then using enumeration field? (WONTFIX: different issue)
  • How to handle selection vs rendering? May need different enumeration option. (WONTFIX: different issue?)

The grid view will be an alternative form of list display that places each record in a cell, which can then be arranged horizontally and vertically to form a grid. It would be used when it is not practical to gert a useful summary of ann entity in a single tabulated row. This could be used for things like thumbnail galleries with associated metadata.

(Needs planning)

• WONTDO: Support alternate displays for different subtypes (to support BibJSON). Have determined this will be covered by repeating groups added as needed.
• New render types: number, date, ...
• Implement "add repeating field" option to view edit (and entity view?)
• Support repeated field group (to support RecordView and BibJSON)
• Options for scoping enumerations (e.g. fields by record_type); select entries with field matching value from containing form? The goal here is to prevent (say) the Bib_* field entries swamping new view definition options.

Hi Graham,

what is status of Annalist? I was pointed to the project by @stain.

I wonder if you can see some parallels with your approach and the LDT specification we have developed: https://atomgraph.github.io/Linked-Data-Templates/

We have also developed a platform based on the LDT technolgoy.

• Define "User" record type, view and list (keyed by email address for portability?)
• Define permission record type, linked to user - not exactly: the user record is the p[ermissions record
• Implement full authorization structure based on logged-in user permissions (currently just requires authentication for any changes) using stored user/permission data per collection. Ha ha - see below
• Update "no_login" test cases accordingly.

Later:

• Introduce user groups?

This list should be reviewed broken out into separate issues, or moved to other issues.

• view collections doesn't use entered label - problem with entry vocab? (Fixed)
• move entity I/O logic in util module to entity module (keep it all together)
• abstract definition of field_context - currently defined implicitly in views.entityeditbase (overtaken by redesign)

The plan for Annalist has always been to separate the backend entity storage from the front end user interface, thereby allowing hosting of data separately from the Annalist service. Currently Annalist uses a file-based back-end which is not separately deployable.

Entity access has been isolated in a single module, so the effort required to replace file access with HTTP access should be modest once the design details have been worked out.

Notes:

• replace/augment direct file access with HTTP access
• Note that it should be possible to take an Annalist site directory and dump it onto any regular HTTP server to publish the raw data. the Annalist web site should still be able to work with that.
• think about storage of identifier URIs (e.g. record type URIs.) Should they default to be:

1. relative to self
2. relative to base of site
3. relative to host
4. absolute
5. relative to base of collection

Currently, it's (3) or (4), but I think I favour (2) (or (5)?). The intent is that the URI field can be fixed by explicitly entering an absolute URI, but until then they are allocated per site. The expectation is that if data are moved, it will be as complete collections to ensure they are accompanied by their associated metadata. This is easiest with (5), but (2) may be easier to implement.

• review URI structure and relation to file system layout
• need to address data access authorization (OpenID?), resource enumeration (WebDAV?), other issues
• Note that entityroot.set_values currently favours a copy of the URL from the stored/internal data. Currently this is created with a full URI (including hostname), but need not be.
• need to address problem of getting HOST part of site URI when initializing a collection; can this logic be shifted to request code instead of __init__?

When a new entity is created, the label and comment fields are populated with useless junk descriptions. This was helpful in early testing, but is now just clutter. Also, the placeholder logic since added makes this less needed to provide clues to what a field should contain.

Also, new field types default to Default_field, which isn't really helpful. Probably better to leave blank so it's easier to spot that a value should be provided.

• Login link to include option to redirect back to failed page, with form fields populated (i.e. don't lose values entered)

Tasks:

• Fix annalist-manager to create providers directory - maybe as part of logic to help facilitate configuration of providers?
• fix oauth2 app to be more resilient to non-existence of the providers directory

Details:

With a newly installed Annalist server (using --personal config), the front page displays OK, but clicking the Login link gives a 500 server error response (an un-pretty one at that).

The server log shows

OSError: [Errno 2] No such file or directory: '/home/annalist/.annalist/providers/'
redirect_uri: /annalist/login/?continuation=http%3A//annalist.net%3A8000/annalist/profile/&message=&scope=openid%20profile%20email
Internal Server Error: /annalist/login/
Traceback (most recent call last):
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/annalist/anenv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/oauth2/views.py", line 228, in get
    collect_client_secrets()
  File "/home/annalist/anenv/lib/python2.7/site-packages/annalist_root/oauth2/views.py", line 66, in collect_client_secrets
    clientsecrets_files   = os.listdir(clientsecrets_dirname)
OSError: [Errno 2] No such file or directory: '/home/annalist/.annalist/providers/'

The problem here is that the providers directory does not exist.

Also, the software itself should be more resilient against this failure.

Review and tidy
Identify duplicated test cases; eliminate duplication
Identify over-complex test cases, split down into simpler test cases

• review use of "@id" fields - use local URL or fixed URI?
  • see also issue #32
• review TODOs
• review URI for delete type/view/list confirmation
• review field names used in sitedata
• review record list description form (create data and configure URIs)
• review record view description form (create data and configure URIs)
• review field description form (create data and configure URIs)
• Review field descriptions in sitedata: type values seem to be inconsistent??? (e.g. Type vs Entity_type?). Need to start some proper documentation of the form data descriptions.
• review URI design: can we revert to original design (without /c/, /d/, etc.)?
  • In particular, think about flat file deployment to standard web server: would like URI patterns to be consistent with directories.
  • see also issue #32
• can all permission/scope labels be moved from code to data? E.g. use field(s) in view to determine permissions needed to view/edit. This would pave the way to finer-grained permissions if required.
  • currently, linking permissions to types seems to be a reasonable approach, and is how permissions for internal data operations are handled

If default list description for a collection is deleted, collection cannot display. Should revert to default default?

1. Suppose default list for a collection is "test_list"
2. Rename list "test_list" to "test_list1"
3. Error message displays with no data listing.
4. Rename "test_list1" back to "test_list".
5. Data list displays OK again.

• move invocation of authentication to the immediate response handler code?
• refactor list description access out of context handling code (avoid multiple reads)
• refactor code from entityeditbase into more specific views where possible
• rename what is left of entityeditbase -> entityviewbase, or move to generic module
• use proper indexing to accelerate search. (ElasticSearch or Jena?)

Security, robustness, deployability:

• Use "collectstatic" for non-test, non-development deployments, even when using Django static file serving.
• Create configuration for shared deployment behind (say) Apache server
• Look into using github as IDP (they support OAUTH2, but not obviously OIDC)
• Use ORCID as IDP when they support OIDC (cf. http://support.orcid.org/forums/175591-orcid-ideas-forum/suggestions/6478669-provide-authentication-with-openid-connect)
• Script OIDC registration for annalist-manager (cf. http://tools.ietf.org/html/rfc7591)

• Support external link fields, with link browsing where possible

The 'Select' label on view description forms, used to label the drop-drown box for selecting field type, is unhelpful and possibly confusing. Suggest use 'Field type' or similar.

• Support import types/views/lists/fields/etc. from another collection