Currently we have eslint warning in the following files:

Specifying includes for a results node populate ought to increase overall performance by leaving everything else untouched and reducing response sizes.

However, if you're in this unlucky situation, the facts are quite otherwise: collection A has property K, which for most records is undefined, but when it is present you want a $lookup to collection B. In this case (most A records have no value at K), if collection B is at all large (and even if collection A is tiny) your query performance will drop through the basement -- while working I saw adding includes to a populate take a query from a few millisecond to 5-10 seconds where collection A had 16 records and collection B had about 35k.

For some reason at https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/results.js#L20-L39 the aggregation stages that you get if you do specify includes perform far worse than those you get if you don't.

Need to figure out why and prevent it -- there's no reason that projecting properties of the looked-up records should affect performance so badly. By the mongo manual and the code it looks like the queries should perform equivalently, but with the second form somehow mongo knows not to do the lookup when the key isn't even set or it uses an index while the first doesn't

🕙time to make it happen

Right now checked values in a facet don't get any special treatment, and so don't show up at the top of the facet list unless the list is long enough to reach the position where they'd be if they weren't checked. This is a regression from something.

Need to add to https://github.com/smartprocure/contexture-mongo/blob/master/src/example-types/facet.js#L17 roughly like this:

  { $addFields: { isSelected: <is in node.values> } },
  { $sort: { isSelected: -1, count: 1 } } 

to produce a sort by checked-ness first

To allow a blank search to be blank.

An alternative is to add a new type.

Doing this on a results context doesn't currently work when schemaName !== propName:

populate: {
  propName: {
    schema: schemaName,
    localField: 'stuff',
    foreignField: '_id',  
    unwind: true
  }
}

Basically just an alias for the current text - intended to cleanly separate out single text inputs from "multitext" tags inputs

Required for futil-js/contexture-react#117

looks like the component itself is doing $addToSet when it ought to be $sum, and the test is unfinished/inaccurate given those difficulties.

Somehow the new-ish date histogram code got checked in with failing tests. Need to uncomment the file and fix the tests. The code will be commented out by #64

My judgment as to what the cardinality prop on a dateHistogram was intended to count was mistaken on #68 -- the prop should hold a count of unique values. Need to update the code and test to implement that properly.

this pattern on a results node

{ 
  type: 'results'
  ...
  include: [
    ...
    'prop1'
  ],
  populate: {
    prop1: {
      localField: ...
      foreignField: ...
      include: propsWithoutProp2
    },
    prop3: {
      localField: 'prop1.prop2', 
      ...
    }
  }
}

will result in a hard-to-debug long-running query when the foreign collection $lookup for prop1.prop2 tries to find a match for undefined.

Better to detect this situation in the tree populates and throw and error before running

This should be in the agg pipeline:

{$sort: { count: -1 }}

We also should not do the regex filter after limiting by size

Add examples to the docs to make it obvious.

Contexture-mongo will incorrectly remove the 0 value from options, by treating 0 value as a falsey value.

contexture-elasticsearrch introduced dateRangeFacet
https://github.com/smartprocure/contexture-elasticsearch/blob/master/src/example-types/dateRangeFacet.js

It should be ported here as well.

• Eslint
• CircleCI config
• Duti
• Prettier formatting

We have it in contexture-elasticsearch. The main feature is that false would also match against null, undefined, and missing.

To exclude soft deleted records, right now users often do this:

{
  key: 'isDeleted',
  field: 'isDeleted',
  type: 'facet',
  mode: 'exclude',
  values: [true],
}

which could become more semantic like this:

{
  key: 'isDeleted',
  field: 'isDeleted',
  type: 'bool',
  value: false,
}

or even

{
  key: 'isDeleted',
  field: 'isDeleted',
  type: 'bool',
}

Currently, the example types use context to refer to the node. This is confusing, because nodes have another property called context.

In facet, we need to add preserveNullAndEmptyArrays to the $unwind: https://stackoverflow.com/a/37408256

We'd also need to add something similar to missing from the ES terms aggregation to give it a better value: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_missing_value_13

We'll likely need to do something similar in contexture-elasticsearch.

one should be able to build a regex for a mongo search using the same util this repo does. move the function to a util folder

like "populate" for mongo facet values -- support props for collection, foreignField and some fields to _.pick off the foreign record (to keep the response small), and implement a $lookup, $unwind, $project to pass the picked foreign props along on the facet value objects for use in (for example) contexture-react's Facet type display function futil-js/contexture-react#390

Right now, the readme of contexture-mongo and also contexture have a reference to contexture-mongo/types, however this doesn't work at the moment, the real path right now is: contexture-mongo/src/types.