chris104957 / django-carrot Goto Github PK

This would be more useful for debugging

need to think about how this impacts on the size of the query sets being returned

In django2, the queryset returned from the API returns items that have recently been deleted.

Need to make the following changes:

• Update the queryset filters in the API so that only objects with an ID get returned
• Update the HTML template to only display items with an ID
• Update the tests to ensure that after deleting all failed tasks, the Get return call returns nothing

Version found: 0.1.1

NOT NULL constraint failed: carrot_scheduledtask.queue

This happens as the form doesn't seem to validating the queue field properly

This is probably down to the number of logging handlers. Requires further investigation

This happens even when save seems to complete successfully

This workaround is required in order to avoid a 500 error on the carrot montior

    TEMPLATES = [
        ...
        'OPTIONS': {
            ...
            'builtins': [
                ...
                'carrot.templatetags.filters'
            ]
        }
    ]

need to fix this so that it works by default and this workaround isn't required

• Instead of showing all tasks in all status in one view, create a view for each type
• This will mean we can also display far more (50?) tasks in each list, instead of just 10

Memory usage seems to be through the roof, which is the cause

We currently have two readmes - one in RST format for PyPI and one MD for GitHub. Need to find a way of having just one

Can I somehow spawn the carrot process from within Django? Currently I'm running the django app in a container, and it would be convenient to just have to launch one process. If not, would you consider PRs for fixing something like this.

In order to maintain Carrot's light config approach, we should remove the bash-based daemon and implement a Django-admin command to start/stop the Carrot service, e.g.

python manage.py carrot_daemon start
python manage.py carrot_daemon stop

This will further simplify the config process

dependent on #38

Something like this:
DEBUG: grey
INFO: green
WARNING: yellow
ERROR/CRITICAL: red

Also needs to support filtering - e.g. users can select the logging level that they want to be visible

Sometimes, an ObjectNotFound error occurs during get_task(), which results in orphaned tasks in the queue

Add a 'run now' button to the scheduled task view which publishes the task immediately - would be useful at times

unpkg.com is far too unreliable

using Vuetify

1. On requeueing a failed task, the consume process fails due to an error parsing the keyword arguments. Traceback

Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/carrot/consumer.py", line 203, in consume
args, kwargs = self.serializer.serialize_arguments(body.decode())
File "/usr/local/lib/python3.5/site-packages/carrot/objects.py", line 194, in serialize_arguments
parsed_args = [ast.literal_eval(arg.strip()) for arg in args]
File "/usr/local/lib/python3.5/site-packages/carrot/objects.py", line 194, in <listcomp>
parsed_args = [ast.literal_eval(arg.strip()) for arg in args]
File "/usr/local/lib/python3.5/ast.py", line 46, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/local/lib/python3.5/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
(
^
SyntaxError: unexpected EOF while parsing

The brackets surrounding the tuple seem to have gotten written to the MessageLog.task_args field. This happens even when the original task has now positional arguments

2. Positional arguments are not currently being displayed in Carrot monitor

following the steps in the readme doesn't actually work. needs bug fix/doc update

Problems:

1. not providing a CARROT variable in the settings causes an attribute error
2. with an empty CARROT attribute, e.g. {}, the consumer starts but shuts down immediately
3. setting the default_broker to amqp://guest:guest@localhost:5672 causes an index error, as it expects a vhost name
4. setting the default broker to amqp://guest:guest@localhost:5672/test raises pika.exceptions.ProbableAccessDeniedError: (530, 'NOT_ALLOWED - vhost default not found')

Task scheduler seems to be creating tasks incorrectly. On creating a Scheduled task with no positional or keyword arguments, when the task is published, the value is as follows:

[0, 'default', '', 'default', {}]

this seems to be the priority, queue, exchange, routing key and keyword arguments parameters. Only the very last item in the above list should be getting passed to the function

Need to make the following changes:

• Implement a reactive UI using VueJS and implement as an SPA
• Allow scrolling through pages of results
• Some tasks may have very long task parameters/error messages. Need to crop these at a certain point to preserve the layout
• In case of task failure, need to save any task log info to the message log, as this facilitates debugging
• Need to refresh the message log list every 5 seconds
• Need to add django_rest_framework to the dependencies
• Remove legacy views/filters/form
• Ensure that user defined auth decorators are applied to API

When a task fails, the only thing visible in the monitor is the traceback. It would be useful to have the full log up until the moment of failure in order to make debugging easier

this is related to the fix for #43

returns the following error:

{
    "active": [
        "This field may not be null."
    ]
}

Version number currently needs to be tagged and edited both in setup.py and conf.py for the docs. this could be simplified further

On stopping or restarting the service, the existing consumers remain attached to the rabbitMQ queue. This causes all sorts of issues

I installed djang-carrot package, added it to installed apps and ran python manage.py makemigrations, but the migrations ended up inside the virtual environment... Why is that?

$ python manage.py makemigrations carrot
Migrations for 'carrot':
venv/lib/python3.6/site-packages/carrot/migrations/0001_initial.py
- Create model MessageLog
- Create model ScheduledTask

The problem is, as the virtual environment is not checked in to source controll I cannot check in the migration, which normally how things are done in Django. What's the thought here? What I'm I doing wrong?

I'm trying to run carrot + django + postgres + rabbitmq in a docker-compose, but carrot just bails out successfully with exit code 0... My docker-compose.yml file looks like this

version: '2'

services:
  rabbit:
    image: 'rabbitmq:3.7-alpine'
    expose:
      - "5672"

  postgres:
    image: 'postgres:9.6'
    volumes:
      - postgres_data:/var/lib/postgresql/data

  django:
    restart: always
    build: ./
    depends_on:
      - postgres
      - rabbit
    expose:
      - "8000"
    ports:
      - "0.0.0.0:8000:8000"

  carrot:
    build: ./
    depends_on:
      - django
      - rabbit
    command:
      bash -c "sleep 8" && python manage.py carrot --traceback

volumes:
  postgres_data:

The container does not seem to produce any output whatsoever. There is no file in /var/log/carrot.log either. Please advice.

I fixed up another simpler django-leek yesterday, but today I saw you had fixed some stuff with the migrations, etc, so I thought I'd give carrot another try as I like the monitor and so on. However, I cannot seem to get it running at all.

Happens in the completed tab. Not always reproducible - for example, searching for test works fine but searching for hello returns an error:

django.db.utils.OperationalError: too many SQL variables

This also means that the message never gets consumed - it just sits in the 'ready' state indefinitely. Presumably it would also be re-consumed on restart of the carrot service

Traceback:

found 0 scheduled tasks to run
Successfully started scheduler
consumer-1 2017-10-13 07:15:09,821:: Started consumer consumer-1
Successfully started 1 consumers for queue default
All queues consumer sets started successfully. Full logs are at /var/log/carrot.log
0 0
Shutdown requested

Eg. clicking Completed then Scheduled very quickly causes the scheduled task list to show a list of completed tasks.

The easy fix for this is to clear the task list on changing the tab before attempting to fetch anything

Need to be able to call a hook on task failure so that users can be notified of failures. For example, something using IFTTT could look like this:

def failure_callback(task, exception, pk):
    import requests
    data = {
        'Value1': task,
        'Value2': exception,
        'Value3': pk,
    }
    requests.post('https://maker.ifttt.com/trigger/my_ifttt_trigger/with/key/my-ifttt-key',  data)

Various formatting issues in the traceback

the v-list may not be the appropriate component for this. Need to find an alternative

Findings:

Readme.md5

• Include usage example (of call command)
• This URL is broken:
  The full documentation is available at readthedocs.io
• As is this one:
  Please refer to Contributing to Carrot
• All the .. code-block:: bash sections are visible (MD doesn't support this). Need to maintain separate MD/RST versions of this readme as PyPi uses RST

quick-start.rst

• In the Configuring your Django project section, the list numbers are all "1"
• Fix the format of the link in the Contributing section

settings.rst

• Need to add a note about brokers with no virtual hosts to the The default broker section
• Improperly formatted bullet point list in the Queue configuration section
• In the same section, need to update the section regarding the concurrency param as it's not clear - no mention that this only affects the service
• the consumer_class config option has not been described in this page - this should be added in

Misc

• Need to add an examples section. This can be expanded later but for now, we should include an example of how to create and apply a custom consumer class
• No mention of logging anywhere in the docs!

http://django-carrot.readthedocs.io/en/latest/index.html>

There is a ">" ;)