rhblind / drf-haystack Goto Github PK
View Code? Open in Web Editor NEWHaystack for Django REST Framework
License: MIT License
Haystack for Django REST Framework
License: MIT License
I think it will be easier to use this library if there was an api reference like how DRF has.
thank you for this library it made my life easier
I'm using Django 1.9, with Haystack 2.4.1 (also tried with latest copy).
If I use Haystack directly (i.e. SearchQuerySet().models(Page)) - I can query Elasticsearch.
However, when I try to query it using my HaystackViewSet, I get the following error/traceback:
http://dpaste.com/2CD8B0K
# Viewset
class PageSearchViewSet(HaystackViewSet):
"""
Elasticsearch/Haystack viewset for readonly search/list view.
"""
index_models = [Page, ]
serializer_class = PageSearchSerializer
# Serializer
class PageSearchSerializer(HaystackSerializer):
class Meta:
index_classes = [PageIndex, ]
# search_indexes.py
from haystack import indexes
from .models import Page
class PageIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
def get_model(self):
return Page
def index_queryset(self, using=None):
return self.get_model().objects.all()
Traceback:
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
149. response = self.process_exception_by_middleware(e, request)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
147. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/viewsets.py" in view
87. return self.dispatch(request, *args, **kwargs)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
466. response = self.handle_exception(exc)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
463. response = handler(request, *args, **kwargs)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/mixins.py" in list
48. return Response(serializer.data)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/serializers.py" in data
674. ret = super(ListSerializer, self).data
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/serializers.py" in data
239. self._data = self.to_representation(self.instance)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/rest_framework/serializers.py" in to_representation
614. self.child.to_representation(item) for item in iterable
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/drf_haystack/serializers.py" in to_representation
180. current_index = self._get_index_class_name(type(instance.searchindex))
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/haystack/models.py" in _get_searchindex
72. return connections['default'].get_unified_index().get_index(self.model)
File "/home/dave/.virtualenvs/vestedyeti/local/lib/python2.7/site-packages/haystack/utils/loading.py" in get_index
308. raise NotHandled('The model %s is not registered' % model_klass)
Exception Type: NotHandled at /page/search/
Exception Value: The model None is not registered
When applying Term Boost to a SearchQuerySet
, other results are discarded.
This seems to me like a bug in django-haystack.
See django-haystack/django-haystack#1235 for details.
Hey there, first of all thanks for the work on this. Theres definitely a need for this library and I'm looking forward to working with it :)
I'd like to contribute toward the library, but I'm finding the output from the tests quite confusing and hard to follow. Even when removing the --verbose
flag the output is incredibly verbose, seemingly dumping the contents of the elasticsearch index. Is there a way to stop this behaviour?
The HaystackViewSet overrides the permission_classes variable so that the permission mechanism defined in the settings is bypassed.
HaystackViewSet should not override the permissions globally defined.
We should support Highlighting
Howdy,
Thanks for making this library, made things a lot easier for me. I went through the docs and couldn't find a way to use filter_or from haystack.
For example, I have a date field called date and a boolean field called repeat. I would like to make a search where the date field matches a specific date OR repeat is true.
I tried "search/?date=2016-03-02;repeat_daily=true" but that seems to be doing an AND filter instead.
Any suggestions?
Cheers,
Kelvin
The HaystackFacetSerializer should also include a hyperlink which can be used to narrow on the selected text
using field_options
or overridden field_options
from query string.
Ideally, it should be able to perform drill-down on multiple "selected_facets" (See official Haystack docs).
I'm new to both haystack and and drf-haystack, so pardon me if I'm all wrong because my brain is fried after a long day.
I'm following the examples from the docs about ignore_fields
. But I can't get it to work as expected. I'm basically doing the same as in the example, but fields in ignore_fields
are still serialized and transfered to the client.
While looking at the source, I think I may have a guess why it is so:
As I understand the logic there, ignore_fields
is only checked if exclude
evaluates to True
. I don't have an exclude
(and neither does the example in the docs), thus my ignore_fields
is, alas, ignored.
Do you agree this is what is happening? I'd happily provide a PR for this if you like, but I'd like you to confirm this bug before I do the work. As this is probably a quick fix, you may also want to just do it yourself. Anyway, let me know!
PS: The old logic seems to make more sense to me.
Hi guys,
Great project :)
I am however in a bit of a pickle. Ive setup ElasticSearch, Django Haystack and drf-haystack in my project and it mostly seems to work great. Ive tried a lot of things and I am completely stumped so I am hoping someone here might have an idea whats going on :)
But when I execute this query (submission is a datetime):
?submission__lt=2016-04-01
I get 0 results and the following error in my PyCharm console:
RequestError: TransportError(400, u'SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[AQ0uPhhWQlyk026CpVP89w][haystack][0]: SearchParseException[[haystack][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"filtered": {"filter": {"terms": {"django_ct": ["model.modelitem", "model.modelrepository", "model.modelfamily"]}}, "query": {"query_string": {"fuzzy_max_expansions": 50, "auto_generate_phrase_queries": true, "default_operator": "AND", "analyze_wildcard": true, "query": "(submission:({* TO \\"2016\\\\-04\\\\-01\\"}) AND submission:({* TO \\"2016\\\\\\\\\\\\-04\\\\\\\\\\\\-01\\"}))", "default_field": "text", "fuzzy_min_sim": 0.5}}}}, "from": 0, "highlight": {"fields": {"text": {"store": "yes"}}}}]]]; nested: ElasticsearchParseException[failed to parse date field [2016\\-04\\-01], tried both date format [dateOptionalTime], and timestamp number]; nested: IllegalArgumentException[Invalid format: "2016\\-04\\-01" is malformed at "\\-04\\-01"]; }]')
But when I execute
?submission__gt=2016-04-01
It returns the expected results/documents and no error is displayed
Setup:
OS: Ubuntu 15.10
Python: 2.7.10
Elasticsearch: 1.6.2
From requirements.txt:
Django: 1.9.1
Haystack: 2.5.dev0 (Else it doesnt work with Django 1.9)
drf-haystack==1.5.6
elasticsearch==1.9.0
I’ve tried downgrading Django to 1.8 and Haystack to 2.4 but I had the same error. I’ve tried a few other combinations of versions but there is always something that breaks :) Usually the same error or facet search didnt work.
I ran the same query using the python shell (using just haystack):
>>> lt_results = SearchQuerySet().filter(submission__lt=datetime.date(2016, 4, 1))
>>> lt_results.count()
6
Which led me to believe this has something to do with drf-haystack :) Any help would be greatly appreciated, I am completely stumped on how to fix this.
The faceted search works for all fields, apart from the submission field. When I click the narrow_url (that has a count of 6) and execute that query, it returns 0 results. Maybe that is related somehow?
Well here is my code if that helps:
urls.py:
router = routers.DefaultRouter()
router.register("model/search",ModelItemFacetViewSet, base_name="model-search")
router.register("^search/facets/$", ModelItemFacetViewSet, base_name="model-search-facet")
urlpatterns =(
url(r"", include(router.urls)),
)
views.py:
class ModelItemFacetViewSet(HaystackViewSet):
index_classes = [ModelItemIndex, ModelFamilyIndex, ModelRepositoryIndex]
# This will be used to filter and serialize regular queries as well
# as the results if the `facet_serializer_class` has the
# `serialize_objects = True` set.
serializer_class = ModelItemSerializer
filter_backends = [HaystackHighlightFilter, HaystackAutocompleteFilter]
# This will be used to filter and serialize faceted results
facet_serializer_class = ModelItemFacetSerializer # See example above!
facet_filter_backends = [HaystackFacetFilter] # This is the default facet filter, and
# can be left out.
search_indexes.py:
class SpecimenIndex(indexes.SearchIndex):
text = indexes.CharField(document=True, use_template=True)
name = indexes.CharField(model_attr='name')
submission = indexes.DateTimeField(model_attr='submission', faceted=True)
type = indexes.CharField(model_attr='type', null=True)
abbreviation = indexes.CharField(model_attr='abbreviation', null=True)
class Meta:
abstract = True
def get_model(self):
return Specimen
def index_queryset(self, using=None):
return self.get_model().objects.all()
class ReportableIndex(SpecimenIndex):
class Meta:
abstract = True
def get_model(self):
return Reportable
def index_queryset(self, using=None):
return self.get_model().objects.all()
class ReviewableIndex(ReportableIndex):
id = indexes.CharField(model_attr='id')
life_cycle_phase = indexes.CharField(model_attr='life_cycle_phase', null=True)
model_category = indexes.CharField(model_attr='model_category', faceted=True)
file_size = indexes.CharField(model_attr='generated_information__file_size', faceted=True)
file_format = indexes.CharField(model_attr='generated_information__file_format', faceted=True)
number_of_downloads = indexes.CharField(model_attr='generated_information__number_of_downloads', faceted=True)
number_of_models = indexes.CharField(model_attr='classification_information__number_of_models', faceted=True)
modeling_language = indexes.CharField(model_attr='classification_information__modeling_language', faceted=True)
tool_vendor = indexes.CharField(model_attr='classification_information__generating_tool__vendor', null=True, faceted=True)
tool_product = indexes.CharField(model_attr='classification_information__generating_tool__product', null=True, faceted=True)
tool_version = indexes.CharField(model_attr='classification_information__generating_tool__version', null=True, faceted=True)
quality = indexes.CharField(model_attr='assessment_information__quality')
completeness = indexes.CharField(model_attr='assessment_information__completeness')
level_of_model = indexes.CharField(model_attr='assessment_information__level_of_model')
uuid = indexes.CharField(model_attr='uuid', null=True)
license = indexes.CharField(model_attr='license__name', null=True, faceted=True)
class Meta:
abstract = True
def get_model(self):
return Reviewable
def index_queryset(self, using=None):
return self.get_model().objects.all()
class GroupableIndex(ReviewableIndex):
parent = indexes.CharField(model_attr='parent', null=True)
class Meta:
abstract = True
def get_model(self):
return Groupable
def index_queryset(self, using=None):
return self.get_model().objects.all()
class ModelItemIndex(GroupableIndex, indexes.Indexable):
def get_model(self):
return ModelItem
def index_queryset(self, using=None):
return self.get_model().objects.all()
class ModelFamilyIndex(GroupableIndex, indexes.Indexable):
def get_model(self):
return ModelFamily
def index_queryset(self, using=None):
return self.get_model().objects.all()
class ModelRepositoryIndex(ReviewableIndex, indexes.Indexable):
def get_model(self):
return ModelRepository
def index_queryset(self, using=None):
return self.get_model().objects.all()
All template _text.txt files look like this (model item, model family and model repository) :
{{object.name}}
{{object.id}}
{{object.submission}}
When I am opening next page, I receive 404 error.
Django==1.7.11
django-haystack==2.4.1
elasticsearch==1.9.0
drf-haystack==1.5.6
Python 2.7.3
Elasticsearch 1.1.2
Example code:
class SearchSerializer(HaystackSerializer):
class Meta:
index_classes = [SomeIndex1, SomeIndex2]
field_aliases = {
"query": "text"
}
class SearchViewSet(HaystackViewSet):
serializer_class = SearchSerializer
filter_backends = [HaystackHighlightFilter]
Default pagination_class PageNumberPagination
As a solution I discovered that need to add exclude = ('page', 'per_page', )
into SearchSerializer.
Please, fix it or if it normal add to the docs.
We should support faceting.
This should probably be implemented as a FacetingFilter
, which picks up and filters on faceting queries like narrowing.
https://django-haystack.readthedocs.org/en/latest/faceting.html#faceting
I'm trying to use the __in
field lookup, but I can't figure out the syntax for the value of it.
An example: If I query /path/to/search/?color=blue
, I get all blue objects. If I query /path/to/search/?color=red
, I get all red objects. But how do I get objects that are either blue or red?
I tried /path/to/search/?color__in=x
, where x is:
red,blue
red|blue
[red,blue]
['red','blue']
But without any luck so far. Can anyone help me?
python2.7/site-packages/drf_haystack/serializers.py", line 19, in
from rest_framework.compat import OrderedDict
ImportError: cannot import name OrderedDict
i have this view
class AppSearchView(HaystackViewSet, ProtectedResourceView):
pagination_class = LimitOffsetPagination
serializer_class = AppindexSerializer
max_limit = 50
index_models = [App]
and i am querying using this url but the result is always empty but if i remove offset and limit from the url i get the results
127.0.0.1:8001/v1/App/search/?offset=&limit=20
{
"count": 0,
"next": null,
"previous": null,
"results": []
}
Commit 257cfaa includes a breaking change. get_facet_objects_serializer
expects its kwargs
dict to include an entry with the key "object"
; however, the call from HaystackFacetSerializer.get_objects()
does not include such a value.
Often, I don't need a REST ViewSet and the associated routing. Then I prefer normal REST views.
Currently I use an implementation similar to this one:
from rest_framework.mixins import ListModelMixin
from drf_haystack.generics import HaystackGenericAPIView
class SearchView(ListModelMixin, HaystackGenericAPIView):
"""
A generic view to be used for rendering the result list while searching.
"""
serializer_class = LocationSerializer
def get(self, request, *args, **kwargs):
return self.list(request, *args, **kwargs)
in urls.py:
urlpatterns = (
...
url(r'^search/', SearchView.as_view()),
...
)
This maybe could be added to the docs, in case others have similar needs.
I've created two other indexes which are working fine. I recently created a search index for my User
model, with a UserSearchViewSet
and UserSearchSerializer
. Here's my code:
User model:
class User(AbstractBaseUser, PermissionsMixin):
email = models.EmailField(max_length=255, unique=True)
name = models.CharField(max_length=255, blank=True)
token = models.CharField(max_length=200, default=None, blank=True, null=True)
email_token = models.CharField(max_length=200, default=None, blank=True, null=True)
new_email = models.EmailField( default=None, blank=True, null=True)
is_staff = models.BooleanField(default=False)
is_active = models.BooleanField(efault=True)
notes = models.TextField(lank=True, null=True)
network = models.ForeignKey(AdNetwork, related_name='users', blank=True, null=True)
date_joined = models.DateTimeField(default=timezone.now)
date_created = models.DateTimeField(auto_now_add=True, null=True, blank=True)
last_updated = models.DateTimeField(auto_now=True)
User search index:
class UserIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
network_id = indexes.CharField(model_attr='network_id')
def get_model(self):
return User
def index_queryset(self, using=None):
return self.get_model().objects.all()
User search ViewSet:
class UserSearchViewSet(HaystackViewSet):
serializer_class = serializers.UserSearchSerializer
index_models = [models.User]
def get_queryset(self):
user_network_id = str(self.request.user.network.id) # filter by user network id
return super().get_queryset().filter(network_id=user_network_id)
User search Serializer:
class UserSearchSerializer(serializers.ModelSerializer, HaystackSerializerMixin):
class Meta:
model = User
search_fields = ('text',)
So, here's the problem. Running ./manage.py rebuild_index
works great, and I can see that the User models were indexed. However, going to the url users/search
, I'm greeted with this error:
ValueError at /users/search/
invalid literal for int() with base 10: 'user.user.5'
Which seems to be occurring at rest_framework/fields.py
line 837 in to_representation
My User search index and endpoint is quite literally a carbon copy of the other two I've created. DRF Haystack seems to be treating my User model different. Any ideas? I've been spinning my wheels for hours in the debugger with no luck.
Duh!
I am using solr as search backend but when i import HaystackViewSet it raises exception:
haystack.exceptions.MissingDependency: The 'elasticsearch' backend requires the installation of 'elasticsearch'. Please refer to the documentation.
Installing elasticsearch from pip fixes the problem, but why install an extra dependency that i dont need
Thanks a lot for this great library
Correct me if I'm wrong, but using the fields
attribute HaystackSerializer
subclasses should be optional, correct? That is, if I had an index like so:
class MyIndex(indexes.SearchIndex):
text = indexes.CharField(document=True, use_template=True)
field1 = indexes.CharField(model_attr='field1')
field2 = indexes.BooleanField(model_attr='field2')
field3 = indexes.BooleanField(model_attr='field3')
def get_model(self):
return MyModel
I should be able to create the following serializers that would be equivalent:
class MySearchResultSerializer(HaystackSerializer):
class Meta:
index_classes = [MyIndex]
fields = [
"text", "field1", "field3"
]
class MySearchResultSerializer(HaystackSerializer):
class Meta:
index_classes = [MyIndex]
exclude = [
"field2"
]
If that is true, I believe there is a problem in serializers.py:117:
if field_name not in fields or field_name in exclude or field_name in ignore_fields:
continue
In the case of the second serializer, I would expect that the text
, field1
, and field3
fields were still included. However, they will not be because they will not be found in fields
(which is []
in this case) and the loop will skip.
I will fix it because I have that and some other changes to better support multiple indexes coming, but I just wanted to make sure that I wasn't missing something.
In HaystackFilter.build_filter
there is a check to “Skip if the parameter is not listed in the serializer's fields
or if it's in the exclude
list.”
In my current project, this causes some trouble. My SearchIndex class has the usual field text = indexes.CharField(document=True, use_template=True)
and a bunch of other fields. This text
field holds the index used for searching, while the other fields only contain data to be rendered when displaying the result list. Now, it doesn't make much sense to serialize this text
field and send it to the client, because the result list has no usage for it. Moreover, this field is quite large and thus increases the Ajax payload needlessly.
A workaround would be to add a method to_representation
to my SearchSerializer class and remove that text
field from the result, but then it has already been serialized.
Therefore I'd like to know if there is any special reason for this check, and/or if You/I shall remove this check or add a kind of "ignore" field for this purpose to the serializer class?
BTW, many thanks for this great library.
Hi, thanks for releasing your library. However, I have been having problem with faceting for some days now. I have followed all your steps and also different version combination of drf haystack and elasticsearch, but I don't get any result for facet and no error. So to be sure it is not my code,I tried it with your mockapp and this is what I get when I search http://localhost:8000/search1/?firstname=john/ .
{
"count": 3,
"next": null,
"previous": null,
"results": [
{
"firstname": "John",
"birthdate": "1952-07-26",
"text": "John McLaughlin\n",
"lastname": "McLaughlin",
"full_name": "John McLaughlin",
"letters": [
"J",
"o",
"h",
"n"
]
},
{
"firstname": "John",
"birthdate": "1986-08-05",
"text": "John McClane\n",
"lastname": "McClane",
"full_name": "John McClane",
"letters": [
"J",
"o",
"h",
"n"
]
},
{
"firstname": "John",
"birthdate": "1963-12-18",
"text": "John Baker\n",
"lastname": "Baker",
"full_name": "John Baker",
"letters": [
"J",
I am guessing it might be elasticsearch version. These is what I currently have in my virtual environment after different trials.
Django==1.8
django-filter==0.13.0
django-haystack==2.4.0
django-markdown==0.8.4
djangorestframework==3.3.3
drf-haystack==1.5.6
elasticsearch==1.7.0
And local system elasticsearch version is 2.3. And I understand facet has been deprecated starting from elasticsearch v.1, does it mean I have to install ES v 0.90?
Please, I would appreciate your help on this. Thanks.
It is not needed if one is not using elasticsearch
BooleanFiled raises TypeError on initialization.
The narrow_url
value is actually a path and query string. Update the facet serializer to return a complete URL so that clients don't have to create the URL.
Documentation lacks some info regarding that, I am missing the whole picture. I would have liked to have facets in the result set, for instance on the same level with 'next', 'count', 'results', 'previous' add 'facets'. Or, otherwise, how could I see all the facets for the given page?
Is it possible to apply a filter on the results of the search specific for each request?
I would need something like this:
class ApplesView(ListModelMixin, HaystackGenericAPIView):
index_models = [Apple]
serializer_class = AppleSearchSerializer
def get(self, request, *args, **kwargs):
queryset=self.list(request, *args, **kwargs).filter(eater=request.user)
return self.list(request, *args, **kwargs)
In this way I would display only the results which concern the creator of the request.
Thank you for the great project.
We should have a more consistent way of performing type conversions for filtered parameter values.
Ref #45
I'm trying to get search results of images ordered by the time they were uploaded. But when I apply filters as usually with django rest framework I loose the ability to search. This is expected behaviour? Is there a way around this?
Code
class ImageModelGeneralSerializer(HaystackSerializer):
class Meta:
index_classes = [ImageModelGeneralIndex]
fields = ['id', 'uploaded_at', 'created_by','image', 'caption', 'photographer']
class ImageSearchView(HaystackViewSet):
index_models = [ImageModelGeneral]
serializer_class = ImageModelGeneralSerializer
filter_backends = (OrderingFilter,)
ordering_fields = ('uploaded_at')
ordering = ('-uploaded_at')
Tests fail when using newest elasticsearch library with
AttributeError: 'Elasticsearch' object has no attribute 'delete_by_query'
How to prohibit user to query over a particular field in search index but allow it to be rendered in the serialized result.
search_indexes.py:
class DatasetIndex(indexes.SearchIndex, indexes.Indexable):
highlight = indexes.BooleanField(indexed=False, model_attr='highlight')
bool_test = indexes.BooleanField(indexed=False)
def prepare_bool_test(self, obj):
return False
JSON Response:
{
"bool_test": true,
"highlight": true,
}
I am trying to serialize boolean value from HayStack Search result. It works fine for other fields except the boolean fields. The boolean fields are always returned as true.
The code assumes you are using the PageNumberPagination because it tries to get the page_query_param
We however have the paginator changed to use LimitOffsetPagination
Which uses the limit_query_param and offset_queryparam
The same would go if you use another paginator.
A generic solution is not possible I think, but maybe extracting the code that reads the page_query_param into a method would allow me to override that one method to fix it for us without the need to copy the whole get_narrow_url method.
I do see geopy as a requirement, but it isn't ideal if someone wants to use this without it.
Trying to run drf-haystack makes references to gis.geos which in many cases is not desired. Many Linux distributions do not come with libgeos so a gis error is generated when the user may not be doing anything with gis. Perhaps there is a way this could be imported without requiring libgeos. If not this should be stated in docs.
File "/usr/local/lib/python3.4/site-packages/drf_haystack/filters.py", line 12, in <module>
from haystack.utils.geo import D, Point
File "/usr/local/lib/python3.4/site-packages/haystack/utils/geo.py", line 2, in <module>
from django.contrib.gis.geos import Point
ImportError: cannot import name 'Point'
May be is not drf-haystack issue, but i want to ask.
I want just to filter QuerySet with "?q=bla bla bla" params (using elastixsearch via haystack anyway).
I see in code https://github.com/inonit/drf-haystack/blob/master/drf_haystack/serializers.py#L37 and
# NOTE: Make sure you don't confuse these with model attributes. These
# fields belong to the search index!
that we have mapping haystack_fields in serializer and etc. May be it is good for something, but redundancy in my case.
I just need - search by query q param and return filtered QuerySet.
More over, i use django-filter backend to filter queryset http://www.django-rest-framework.org/api-guide/filtering/#djangofilterbackend and i want to combine filter result with haystack results (for one Model).
May me i must use somthing like this (not tested!):
http://django-haystack.readthedocs.org/en/latest/searchqueryset_api.html#load-all-queryset
drf view:
def queryset_gen(search_qs):
for item in search_qs:
yield item.object
def get(self, request, format=None):
# foo from GET q
sqs = RelatedSearchQuerySet().filter(content='foo').load_all()
# For the Entry model, we want to include related models directly associated
# with the Entry to save on DB queries.
sqs = sqs.load_all_queryset(Entry, self.queryset.select_related(depth=1))
self.queryset = queryset_gen(sqs)
return self.list(request, *args, **kwargs)
or it is wrong way and i can make it better with drf-haystack ?
As you can read in the title, this is not a bug, but i need a representation of my data without launching update_index.
Is it possible?
If so (i hope that), The documentation is incomplete
class LocationSerializer(HaystackSerializer):
class Meta:
# The `index_classes` attribute is a list of which search indexes
# we want to include in the search.
index_classes = [LocationIndex]
# The `fields` contains all the fields we want to include.
# NOTE: Make sure you don't confuse these with model attributes. These
# fields belong to the search index!
fields = [
"text", "address", "city", "zip_code", "autocomplete"
]
Raises error when setting rest framework global permissions in settings.py
Hi,
I am trying to implement django-haystack and drf-haystack in such a way that it work with django-autocomplete-light. I have this code..
from drf_haystack.serializers import HaystackSerializer
from rest_framework.mixins import ListModelMixin
from drf_haystack.generics import HaystackGenericAPIView
from drf_haystack.filters import HaystackAutocompleteFilter
from users.search_indexes import LocationIndex
from users.models import Location
class AutocompleteSerializer(HaystackSerializer):
class Meta:
index_classes = [LocationIndex]
fields = ["text", "state", "autocomplete_city"]
ignore_fields = ["text", "state"]
class LocationAutocompleteView(ListModelMixin, HaystackGenericAPIView):
index_models = [Location]
serializer_class = AutocompleteSerializer
filter_backends = [HaystackAutocompleteFilter]
def get(self, request, *args, **kwargs):
return self.list(request, *args, **kwargs)
I want the output from the view above in the format like this..
{
"pagination":
{
"more": true
},
"results":
[
{
"text": "Bokaro Steel City",
"id": "94"
},
{
"text": "Rajahmundry",
"id": "95"
},
{
"text": "Ballari",
"id": "96"
},
{
"text": "Latur",
"id": "99"
},
{
"text": "Bilaspur",
"id": "109"
},
{
"text": "Tumkur",
"id": "116"
}
]
}
My code above returns all the results in this format... also results are not being paginated..
[
{
"autocomplete_city": "Hyderabad"
},
{
"autocomplete_city": "Chennai"
},
{
"autocomplete_city": "Pune"
},
{
"autocomplete_city": "Surat"
},
{
"autocomplete_city": "Nagpur"
},
{
"autocomplete_city": "Patna"
},
Any help is highly appreciated.
Regards
Add support for customizable query facets.
Given the index and serializer below, I would expect the transcript_languages
field to be rendered as a JSON array (of strings, in this case). However, an array string (e.g. "['Afrikaans', 'Arabic – United Arab Emirates', 'Arabic – Bahrain']"
) is returned.
The serializer should be updated to render MultiValueField
data as a JSON array.
class CourseRunIndex(indexes.SearchIndex, indexes.Indexable):
transcript_languages = indexes.MultiValueField(faceted=True)
def prepare_transcript_languages(self, obj):
# language.name is a string
return [language.name for language in obj.transcript_languages.all()]
class CourseRunSearchSerializer(HaystackSerializer):
class Meta:
index_classes = [CourseRunIndex]
fields = ('transcript_languages', 'text',)
field_aliases = {
'q': 'text',
}
ignore_fields = ('text',)
Actual output
{
"dates": {},
"fields": {
"transcript_languages": [
{
"text": "Arabic – United Arab Emirates",
"count": 1,
"narrow_url": "/api/v1/search/course_runs/facets/?selected_facets=transcript_languages_exact%3AArabic+%E2%80%93+United+Arab+Emirates&q=afrikaans"
},
{
"text": "Arabic – Bahrain",
"count": 1,
"narrow_url": "/api/v1/search/course_runs/facets/?selected_facets=transcript_languages_exact%3AArabic+%E2%80%93+Bahrain&q=afrikaans"
},
{
"text": "Afrikaans",
"count": 1,
"narrow_url": "/api/v1/search/course_runs/facets/?selected_facets=transcript_languages_exact%3AAfrikaans&q=afrikaans"
}
]
},
"queries": {},
"objects": {
"count": 1,
"next": null,
"previous": null,
"results": [
{
"transcript_languages": "['Afrikaans', 'Arabic – United Arab Emirates', 'Arabic – Bahrain']"
}
]
}
}
HaystackFilter should be able to construct queries like:
SearchQuerySet().filter(groups__in=[g.id for g in user.groups.all()])
We should support term boost for queries.
https://django-haystack.readthedocs.org/en/latest/boost.html#term-boost
The detail view returns only the first object from a queryset. If used with an index with multiple indexes, there (most likely) might be multiple objects with the same id.
Right now it's hardcoded in the function which classes those are. (It's great that they are no longer nested).
Would be nice if you can override them on the HaystackFacetSerializer class.
Quick untested example:
class HaystackFacetSerializer(six.with_metaclass(HaystackSerializerMeta, serializers.Serializer)):
"""
The ``HaystackFacetSerializer`` is used to serialize the ``facet_counts()``
dictionary results on a ``SearchQuerySet`` instance.
"""
_abstract = True
serialize_objects = False
paginate_by_param = None
facet_dict_field_class = FacetDictField
facet_list_field_class = FacetListField
facet_field_serializer_class = FacetFieldSerializer
def get_fields(self):
"""
This returns a dictionary containing the top most fields,
``dates``, ``fields`` and ``queries``.
"""
field_mapping = OrderedDict()
for field, data in self.instance.items():
field_mapping.update(
{field: self.facet_dict_field_class(
child=self.facet_list_field_class(child=self.facet_field_serializer_class(data)), required=False)}
)
if self.serialize_objects is True:
field_mapping["objects"] = serializers.SerializerMethodField()
return field_mapping
....
That would allow me to add more information to the facets for display on the client side.
We should support "more like this" for backends which supports it.
We should support multiple indexes.
HaystackGenericAPIView().get_object()
should mimic the behaviour from GenericAPIView().get_object()
which use django.shortcuts.get_object_or_404()
.
We should also raise Http404 if no results are given.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.