See: https://docs.rs/elastic_types/0.5.2/elastic_types/#a-complete-example

We should add a json example of what the Article type could end up looking like in json, so something like:

{
    "id": 1,
    "title": "An article",
    "content": "Some prose for this article. It's split into sentences and has words for searching.",
    "timstamp": 1435935302478,
    "geoip": {
        "ip": "10.0.0.1",
        "loc": [ -71.34, 41.12 ]
    }
}

That way we know roughly what the sample is building. It doesn't need to be any more involved than that I think.

See: https://docs.rs/elastic_types/0.5.2/elastic_types/#map-your-types

To keep things consistent, the sample here should have a Deserialize derive on it.

One of the main features of this library is to make it easy to serialise types with special formats, like date and geo_point. We should have a simple sample in the readme that shows how you can serialise a type with a date value.

Original issue: elastic-rs/elastic#128

There are a number of errors in the docs (#9, #10), as well as issues talking about types that don't exist (IndexTemplateMapper) and inconsistent treatment of Elasticsearch type names.

They need to be combed through carefully and these issues need to be fixed.

Instead of having to derive FieldType<M, F> directly, we should add a sub trait (could be an alias in the future). So something like:

trait KeywordFieldType<M> {}
impl<T: KeywordFieldType<M>, M: KeywordMapping> FieldType<M, KeywordFormat> for T {}

Will need to make sure this passes trait coherence though.

There are a bunch of warnings in the doc tests that will become errors in the future. These should all be fixed up. The doc tests are pretty messy anyways.

The crates.io link in date macros is wrong. It's linking to json_str.

Update to the latest serde. While I'm at it I'll make sure everything is going to work for the custom derive on stable when the latest version is released in a few days.

Looks like we can remove the proc_macro feature attribute now, so I'll give this a go in elastic_types.

This is a new datatype introduced in 5.2 for value ranges. See here.

Looks like we could represent it as:

struct DateRange<F> {
    gte: Date<DefaultDateMapping<F>>,
    lte: Date<DateRangeMapping<F>>,
}

Some things to think about:

• Support for other bounds; gt or lt. Anything that can be represented by gt or lt can be done by gte and lte so not a big priority.

Ranges also support number types so they'll need to be added.

This library needs to be updated to use the new TokenStream API. See: https://doc.rust-lang.org/book/compiler-plugins.html

Original issue: elastic-rs/elastic#5

Also depends on: #8

It's not just a matter of adding the names, because dates need to be encoded/decoded using the format too, so that needs to be supported either by the macro or using chrono directly.

The derive paths to the crate aren't right for elastic_types_derive when used with elastic. I'm not sure yet how to fix this. I guess our options are:

• Add a feature that changes the path in elastic_types_derive
• Change the base to elastic, meaning it won't work with elastic_types anymore
• Remove elastic_types from elastic so it needs to be imported separately
• Create a new crate elastic_derive that wraps elastic_types_derive but provides a different crate root
• Expect the needed bits to be in scope when deriving

These are all pretty hacky solutions and I'm not a fan of any of them. I'll think about how to work around this.

An Ip datatype can either be an Ipv4 or Ipv6 in Elasticsearch 5.x

See: https://www.elastic.co/guide/en/elasticsearch/reference/master/ip.html

We should probably just implement the same mapping types for Ipv6 (might need to rename the Ip wrapper type), so we require the user know what kind of Ip they want to use.

Rather than always allocating a new String, see if there's an improvement in passing around a delayed format from Format and using serdes collect_str method to write it.

There are some benches for writing and parsing dates so the impact will be easy to measure. It's worth seeing if serde_json overrides the collect_str method, but I'd be surprised if it didn't.

We should just be able to rely on the Deref trait for updating a mutable wrapper value like Boolean<M>.

We can add tests for these.

Previous Thread: elastic-rs/elastic#147

Required for elastic-rs/elastic#5

The current date format macro isn't that great. It unnecessarily allocates strings, doesn't support most valid formats and isn't easy to change.

It needs to be rewritten, we can keep the generic parse functions, but should use a simple FSM to parse the date format. So we can do something a bit like:

match current_char {
    ES_DAY_LOWER => parse_es_lower_days(),
    ES_DAY_UPPER => parse_es_upper_days(),
    ES_MONTH => parse_es_months(),
    ES_ESCAPE => parse_es_escaped(),
    ...
    CHRONO => parse_chrono(),
    c if !c.is_alpha() => parse_literal(),
    _ => Err("invalid format")
}

Where parse_chrono:

match current_char {
    CHRONO_DAY => parse_chrono_days(),
    ...
}

And parse_es_lower_days:

let days = take_while1(|c| c == ES_LOWER_DAYS);
match days.len() {
    1 => Err("invalid format"),
    2 => Item::Days,
    3 => Item::OrdinalDays
}

This should be easier to support.

Right now the field mod is hidden, because I wasn't sure when a user might care about it.

Turns out there is an important case where you'd use it; implementing a field type for your own types. Take this example:

enum Prefix { A, B, C }

impl FieldType<PrefixMapping, KeywordFormat> for Prefix {}

struct PrefixMapping;

impl KeywordMapping for PrefixMapping {}

This will map a Prefix enum as a keyword in Elasticsearch, so a user doesn't need to map everything as strings.

When deriving ElasticType without mapping specified, you'll get one created for you called {DocType}Mapping.

Currently this mapping is public, even if the type you're deriving on isn't. We should either:

• Make this mapping type unreachable if it's not defined by the caller (I prefer this one, but is a bigger change)
• Use the privacy of the document type on the mapping type.

I personally like the first option and would like to see how that would work out.

Create a new wrapper type for object mappings that looks something like:

struct ElasticTypeMappingWrapper<M>(M);

impl<M> Serialize for ElasticTypeMappingWrapper<M> {
    fn serialize(s) {
        M::serialize_type(s)
    }
}

That way we don't need to build expensive serde_json::Value types when constructing mappings.

I'm thinking we should rename the associated ser_type fn to ser_field, and then put ser_type on ObjectMapping so it'll return this wrapped structure instead.

Right now, we have a feature gate on elastic_types_derive that switches the crate base to derive from.

This isn't a great solution. It's weird and will produce errors if it appears multiple times with different features. I'm not sure of a perfect solution, it kind of questions the whole premise of bundling the library.

One approach could be to move the derive logic into another crate: elastic_codegen and then using that in both elastic_types_derive and a new crate elastic_derive. That behaviour is still a bit surprising, and I'm not actually sure what custom derives do if multiple of the same name are in scope.

An alternative is to force the user to depend on elastic_types themselves. But then that makes it feel like a second-class citizen.

I'll do some investigation on the following:

• What happens when two derives use the same name
• What does the API look like if we exclude elastic_types
• What does the API look like if we exclude everything

I feel like there may be some time in the future where a crate can contain its own derives. If this ever becomes a thing then we could make use of that.

Make sure the names for date formats don't have typos in tests.

Now that serde 1.0 is out we should upgrade to the stable API.

There are a few new crates for working with Macros 1.1, which we should be looking to support for codegen on stable as soon as it's a thing:

• quote
• syn

This means ripping out the current procedural macro implementation and replacing it. I'm not sure right now how much work this is going to be, but we don't actually do a lot of work in our codegen anymore so it should be ok.

The API at the moment is suffering from multiple-personalities.

The goal of this project needs to be restated and the whole surface aligned to it. What we want is:

Staticly defined type mapping and serialisation.

So all tools should work towards that goal. For a user that means:

• Defining field mappings
• Defining document types composed of fields with mappings
• Defining document mappings optionally

All tools should be aligned with making this as simple and efficient as possible.

Previous Thread: https://github.com/KodrAus/elasticsearch-rs/issues/126

See: https://docs.rs/elastic_types/0.5.2/elastic_types/string/index.html#examples

The mapping details links are deadzies.

See the macro used in the string mod. We should use a common macro to derive all the PartialEq and Derive traits needed to make the wrapper types useful.

Wrapping the derived code in a mod doesn't work when in a function scope. We should do what serde does instead and put things in a const.

We have inconsistent links in our docs to Elasticsearch. They should all be using master because we're targeting the alpha 0.5 branch.

Previous Thread: https://github.com/KodrAus/elasticsearch-rs/issues/29

Looks like the 0.3.1 release of chrono has been yanked so we're going to need to bump up to 0.4.0.

It also looks like there are some other changes in that release so we'll have to bump the minor version of elastic_types too.

It's a bit weird that deriving ElasticType implements a trait called DocumentType (it might also add a mapping type and derive traits for that).

This should be aligned with the trait bound you'd use to accept a document type.

The options are:

Rename DocumentType to ElasticType:

#[derive(ElasticType)]
struct MyData {
    id: i32,
    title: String
}

fn with_doc<TDocument: ElasticType>(doc: TDocument) {

}

Rename ElasticType to DocumentType:

#[derive(DocumentType)]
struct MyData {
    id: i32,
    title: String
}

fn with_doc<TDocument: DocumentType>(doc: TDocument) {

}

Rename both to ElasticDocument:

#[derive(ElasticDocument)]
struct MyData {
    id: i32,
    title: String
}

fn with_doc<TDocument: ElasticDocument>(doc: TDocument) {

}

I don't have a strong leaning yet. I'll play with it in some actual code and see which one works out best.

Once Macros 1.1 is on stable, we can move the elastic_types_macros dependency to dev-dependencies.

The docs seem to be having trouble uploading and this is tanking the build

I'm getting some errors in elastic for dynamic linking that only seem to show up when building elastic_types without nightly.

I'm hoping this will go away once elastic_types_derive and serde_derive are available on stable and can be moved into dev_dependencies. That way the nightly feature should only be tweaking the way elastic_date_macros works.

Use macros 1.1 to derive date formats instead of an unstable plugin. Formats need to be applied to structs anyway so it makes sense and then eill be supported efficiently on stable.

This is an overarching issue to rework the custom derive macros for types and dates which includes the following:

• Move types functionality into a new crate: elastic_types_codegen #54
• Create a new crate elastic_derive that supports codegen using the alternative crate root #54
• Move date functionality into elastic_types_codegen #38
• Create new date parser as a custom derive that only supports Elasticsearch formats #8

The ElasticFieldMappingWrapper<M> should deref to M so a caller doesn't need to worry about its type.

Right now we need FieldType in scope to make this work. This can be worked around by calling a function to get the mapping where the function just takes a FieldType and calls mapping() on it.

We can now whitelist attributes and don't need to write the input stream to the output.

See: serde-rs/serde#614

This crate should use error_chain for mapping serialisation and sate parse errors.

The link to elastic_types in the docs for elastic_types_macros is broken. It probably should link to the GotHub repo.

Add some examples, basic ones can use cargo's examples feature.

Any that require extra dependencies, like elastic_reqwest can be separate crates in the same directory.

The docs have a bunch of errors and inconsistencies after the recent API churn. These need to be fixed up before releasing 0.9.0

One of the benefits of wrapping the custom derive in a const is that we can re-import the appropriate elastic_types/elastic crate and give it an alias even if the user has imported it under a different name.

This should be a simple matter of splitting the crate_root parameter into krate and path, so we can:

const _IMPL_BLAH: () = {
    extern crate $krate as _derive;

    impl _derive::$path::DocumentType for Blah { }
};

That way we can let consumers reuse the Document type for something more meaningful. I'm not sure why Field needs to be public, maybe there's a reason for it but consumers shouldn't need to interact with it directly. We've got the field_ser function for that.

This would change the basic mapping serialisation to:

let mapping = serde_json::to_string(&DocumentIndexMapping::from(MyType::mapping())).unwrap();

I think DocumentIndexMapping is a better name because it describes what it's going to produce better (the mapping for a type in an index, rather than the mapping for a field on another type). It's a bit longer though.

Original issue: elastic-rs/elastic#129