To here: https://github.com/elastic-rs/elastic
This repo will hang around for historical reasons for a while, but will eventually disappear.
To here: https://github.com/elastic-rs/elastic
This repo will hang around for historical reasons for a while, but will eventually disappear.
See: https://docs.rs/elastic_types/0.5.2/elastic_types/#a-complete-example
We should add a json example of what the Article
type could end up looking like in json, so something like:
{
"id": 1,
"title": "An article",
"content": "Some prose for this article. It's split into sentences and has words for searching.",
"timstamp": 1435935302478,
"geoip": {
"ip": "10.0.0.1",
"loc": [ -71.34, 41.12 ]
}
}
That way we know roughly what the sample is building. It doesn't need to be any more involved than that I think.
See: https://docs.rs/elastic_types/0.5.2/elastic_types/#map-your-types
To keep things consistent, the sample here should have a Deserialize
derive on it.
One of the main features of this library is to make it easy to serialise types with special formats, like date
and geo_point
. We should have a simple sample in the readme that shows how you can serialise a type with a date
value.
Original issue: elastic-rs/elastic#128
Instead of having to derive FieldType<M, F>
directly, we should add a sub trait (could be an alias in the future). So something like:
trait KeywordFieldType<M> {}
impl<T: KeywordFieldType<M>, M: KeywordMapping> FieldType<M, KeywordFormat> for T {}
Will need to make sure this passes trait coherence though.
There are a bunch of warnings in the doc tests that will become errors in the future. These should all be fixed up. The doc tests are pretty messy anyways.
The crates.io
link in date macros is wrong. It's linking to json_str
.
Update to the latest serde
. While I'm at it I'll make sure everything is going to work for the custom derive on stable
when the latest version is released in a few days.
Looks like we can remove the proc_macro
feature attribute now, so I'll give this a go in elastic_types
.
This is a new datatype introduced in 5.2
for value ranges. See here.
Looks like we could represent it as:
struct DateRange<F> {
gte: Date<DefaultDateMapping<F>>,
lte: Date<DateRangeMapping<F>>,
}
Some things to think about:
gt
or lt
. Anything that can be represented by gt
or lt
can be done by gte
and lte
so not a big priority.Ranges also support number types so they'll need to be added.
This library needs to be updated to use the new TokenStream API. See: https://doc.rust-lang.org/book/compiler-plugins.html
Original issue: elastic-rs/elastic#5
Also depends on: #8
It's not just a matter of adding the names, because dates need to be encoded/decoded using the format too, so that needs to be supported either by the macro or using chrono
directly.
The derive paths to the crate aren't right for elastic_types_derive
when used with elastic
. I'm not sure yet how to fix this. I guess our options are:
elastic_types_derive
elastic
, meaning it won't work with elastic_types
anymoreelastic_types
from elastic
so it needs to be imported separatelyelastic_derive
that wraps elastic_types_derive
but provides a different crate rootThese are all pretty hacky solutions and I'm not a fan of any of them. I'll think about how to work around this.
An Ip datatype can either be an Ipv4 or Ipv6 in Elasticsearch 5.x
See: https://www.elastic.co/guide/en/elasticsearch/reference/master/ip.html
We should probably just implement the same mapping types for Ipv6 (might need to rename the Ip wrapper type), so we require the user know what kind of Ip they want to use.
Rather than always allocating a new String
, see if there's an improvement in passing around a delayed format from Format
and using serde
s collect_str
method to write it.
There are some benches for writing and parsing dates so the impact will be easy to measure. It's worth seeing if serde_json
overrides the collect_str
method, but I'd be surprised if it didn't.
We should just be able to rely on the Deref
trait for updating a mutable wrapper value like Boolean<M>
.
We can add tests for these.
Previous Thread: elastic-rs/elastic#147
Required for elastic-rs/elastic#5
The current date format macro isn't that great. It unnecessarily allocates strings, doesn't support most valid formats and isn't easy to change.
It needs to be rewritten, we can keep the generic parse
functions, but should use a simple FSM to parse the date format. So we can do something a bit like:
match current_char {
ES_DAY_LOWER => parse_es_lower_days(),
ES_DAY_UPPER => parse_es_upper_days(),
ES_MONTH => parse_es_months(),
ES_ESCAPE => parse_es_escaped(),
...
CHRONO => parse_chrono(),
c if !c.is_alpha() => parse_literal(),
_ => Err("invalid format")
}
Where parse_chrono
:
match current_char {
CHRONO_DAY => parse_chrono_days(),
...
}
And parse_es_lower_days
:
let days = take_while1(|c| c == ES_LOWER_DAYS);
match days.len() {
1 => Err("invalid format"),
2 => Item::Days,
3 => Item::OrdinalDays
}
This should be easier to support.
Right now the field
mod is hidden, because I wasn't sure when a user might care about it.
Turns out there is an important case where you'd use it; implementing a field type for your own types. Take this example:
enum Prefix { A, B, C }
impl FieldType<PrefixMapping, KeywordFormat> for Prefix {}
struct PrefixMapping;
impl KeywordMapping for PrefixMapping {}
This will map a Prefix
enum as a keyword in Elasticsearch, so a user doesn't need to map everything as strings.
When deriving ElasticType
without mapping specified, you'll get one created for you called {DocType}Mapping
.
Currently this mapping is public, even if the type you're deriving on isn't. We should either:
I personally like the first option and would like to see how that would work out.
Create a new wrapper type for object mappings that looks something like:
struct ElasticTypeMappingWrapper<M>(M);
impl<M> Serialize for ElasticTypeMappingWrapper<M> {
fn serialize(s) {
M::serialize_type(s)
}
}
That way we don't need to build expensive serde_json::Value
types when constructing mappings.
I'm thinking we should rename the associated ser_type
fn to ser_field
, and then put ser_type
on ObjectMapping
so it'll return this wrapped structure instead.
Right now, we have a feature gate on elastic_types_derive
that switches the crate base to derive from.
This isn't a great solution. It's weird and will produce errors if it appears multiple times with different features. I'm not sure of a perfect solution, it kind of questions the whole premise of bundling the library.
One approach could be to move the derive logic into another crate: elastic_codegen
and then using that in both elastic_types_derive
and a new crate elastic_derive
. That behaviour is still a bit surprising, and I'm not actually sure what custom derives do if multiple of the same name are in scope.
An alternative is to force the user to depend on elastic_types
themselves. But then that makes it feel like a second-class citizen.
I'll do some investigation on the following:
elastic_types
I feel like there may be some time in the future where a crate can contain its own derives. If this ever becomes a thing then we could make use of that.
Make sure the names for date formats don't have typos in tests.
Now that serde
1.0 is out we should upgrade to the stable API.
There are a few new crates for working with Macros 1.1, which we should be looking to support for codegen on stable as soon as it's a thing:
This means ripping out the current procedural macro implementation and replacing it. I'm not sure right now how much work this is going to be, but we don't actually do a lot of work in our codegen anymore so it should be ok.
The API at the moment is suffering from multiple-personalities.
The goal of this project needs to be restated and the whole surface aligned to it. What we want is:
Staticly defined type mapping and serialisation.
So all tools should work towards that goal. For a user that means:
All tools should be aligned with making this as simple and efficient as possible.
Previous Thread: https://github.com/KodrAus/elasticsearch-rs/issues/126
See: https://docs.rs/elastic_types/0.5.2/elastic_types/string/index.html#examples
The mapping details links are deadzies.
See the macro used in the string
mod. We should use a common macro to derive all the PartialEq
and Derive
traits needed to make the wrapper types useful.
Wrapping the derived code in a mod
doesn't work when in a function scope. We should do what serde
does instead and put things in a const
.
We have inconsistent links in our docs to Elasticsearch. They should all be using master
because we're targeting the alpha 0.5
branch.
Previous Thread: https://github.com/KodrAus/elasticsearch-rs/issues/29
Looks like the 0.3.1
release of chrono
has been yanked so we're going to need to bump up to 0.4.0
.
It also looks like there are some other changes in that release so we'll have to bump the minor version of elastic_types
too.
It's a bit weird that deriving ElasticType
implements a trait called DocumentType
(it might also add a mapping type and derive traits for that).
This should be aligned with the trait bound you'd use to accept a document type.
The options are:
DocumentType
to ElasticType
:#[derive(ElasticType)]
struct MyData {
id: i32,
title: String
}
fn with_doc<TDocument: ElasticType>(doc: TDocument) {
}
ElasticType
to DocumentType
:#[derive(DocumentType)]
struct MyData {
id: i32,
title: String
}
fn with_doc<TDocument: DocumentType>(doc: TDocument) {
}
ElasticDocument
:#[derive(ElasticDocument)]
struct MyData {
id: i32,
title: String
}
fn with_doc<TDocument: ElasticDocument>(doc: TDocument) {
}
I don't have a strong leaning yet. I'll play with it in some actual code and see which one works out best.
Once Macros 1.1 is on stable
, we can move the elastic_types_macros
dependency to dev-dependencies
.
The docs seem to be having trouble uploading and this is tanking the build
I'm getting some errors in elastic
for dynamic linking that only seem to show up when building elastic_types
without nightly
.
I'm hoping this will go away once elastic_types_derive
and serde_derive
are available on stable
and can be moved into dev_dependencies
. That way the nightly
feature should only be tweaking the way elastic_date_macros
works.
Use macros 1.1 to derive date formats instead of an unstable plugin. Formats need to be applied to structs anyway so it makes sense and then eill be supported efficiently on stable
.
This is an overarching issue to rework the custom derive macros for types
and dates
which includes the following:
types
functionality into a new crate: elastic_types_codegen
#54elastic_derive
that supports codegen using the alternative crate root #54date
functionality into elastic_types_codegen
#38date
parser as a custom derive that only supports Elasticsearch formats #8The ElasticFieldMappingWrapper<M>
should deref to M
so a caller doesn't need to worry about its type.
Right now we need FieldType
in scope to make this work. This can be worked around by calling a function to get the mapping where the function just takes a FieldType
and calls mapping()
on it.
We can now whitelist attributes and don't need to write the input stream to the output.
See: serde-rs/serde#614
This crate should use error_chain
for mapping serialisation and sate parse errors.
The link to elastic_types
in the docs for elastic_types_macros
is broken. It probably should link to the GotHub repo.
Add some examples, basic ones can use cargo's examples feature.
Any that require extra dependencies, like elastic_reqwest
can be separate crates in the same directory.
The docs have a bunch of errors and inconsistencies after the recent API churn. These need to be fixed up before releasing 0.9.0
One of the benefits of wrapping the custom derive in a const
is that we can re-import the appropriate elastic_types
/elastic
crate and give it an alias even if the user has imported it under a different name.
This should be a simple matter of splitting the crate_root
parameter into krate
and path
, so we can:
const _IMPL_BLAH: () = {
extern crate $krate as _derive;
impl _derive::$path::DocumentType for Blah { }
};
That way we can let consumers reuse the Document
type for something more meaningful. I'm not sure why Field
needs to be public, maybe there's a reason for it but consumers shouldn't need to interact with it directly. We've got the field_ser
function for that.
This would change the basic mapping serialisation to:
let mapping = serde_json::to_string(&DocumentIndexMapping::from(MyType::mapping())).unwrap();
I think DocumentIndexMapping
is a better name because it describes what it's going to produce better (the mapping for a type in an index, rather than the mapping for a field on another type). It's a bit longer though.
Original issue: elastic-rs/elastic#129
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.