Giter Club home page Giter Club logo

ddld's Introduction

DDLD

A centralized download server

RESTful API

GET /api/v1/nodes

  • pattern (string)

    search nodes by the given pattern

POST /api/v1/nodes

sync ACD cache database

DELETE /api/v1/nodes/{id}

move the node to trash

GET /api/v1/cache

  • nodes[] (string)

    compare nodes

POST /api/v1/cache

abort pending downloads

  • (optional) acd_paths[] (string)

    pull files from the given paths

PUT /api/v1/cache/{id}

download the given node

ddld's People

Contributors

legnaleurc avatar

Watchers

 avatar  avatar  avatar

ddld's Issues

Exception in queue processing

2016-07-29 08:11:00,772|ERROR___|Exception in callback functools.partial(<function wrap.<locals>.null_wrapper at 0x6fc33300>, <tornado.concurrent.Future object at 0x6fc31a30>)
Traceback (most recent call last):
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 1017, in run
    yielded = self.gen.send(value)
StopIteration: 3509102690

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/ioloop.py", line 600, in _run_callback
    ret = callback()
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/ioloop.py", line 615, in <lambda>
    self.add_future(ret, lambda f: f.result())
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/concurrent.py", line 232, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 1014, in run
    yielded = self.gen.throw(*exc_info)
  File "<string>", line 6, in _wrap_awaitable
  File "/home/pi/local/src/acddl/acddl/worker.py", line 48, in do
    rv = await future
  File "<string>", line 3, in __await__
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 1008, in run
    value = future.result()
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/concurrent.py", line 232, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/home/pi/local/src/acddl/acddl/worker.py", line 83, in _process
    rv = await rv
  File "/home/pi/local/src/acddl/acddl/controller.py", line 276, in _download
    if await self._need_recycle(node):
  File "/home/pi/local/src/acddl/acddl/controller.py", line 205, in _need_recycle
    required_space = await self._get_node_size(node)
  File "/home/pi/local/src/acddl/acddl/controller.py", line 227, in _get_node_size
    children = await tg.multi(children)
  File "<string>", line 3, in __await__
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 1008, in run
    value = future.result()
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/concurrent.py", line 232, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 789, in callback
    result_list.append(f.result())
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/concurrent.py", line 232, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 3, in raise_exc_info
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/gen.py", line 282, in wrapper
    yielded = next(result)
  File "<string>", line 6, in _wrap_awaitable
  File "/home/pi/local/src/acddl/acddl/controller.py", line 225, in _get_node_size
    children = await self._context.db.get_children(node)
  File "/home/pi/local/src/acddl/acddl/controller.py", line 515, in get_children
    folders, files = await self._worker.do(functools.partial(self._acd_db.list_children, node.id))
  File "/home/pi/local/src/acddl/acddl/worker.py", line 44, in do
    await self._queue.put(task)
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/queues.py", line 163, in put
    self.put_nowait(item)
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/queues.py", line 186, in put_nowait
    self.__put_internal(item)
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/queues.py", line 264, in __put_internal
    self._put(item)
  File "/home/pi/.pyenv/versions/py35/lib/python3.5/site-packages/tornado/queues.py", line 322, in _put
    heapq.heappush(self._queue, item)
RuntimeError: list changed size during iteration

Fix pulling logic

If the file is too old, just check integrity if possible, don't download it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.