The ingestify from pysport

ingestify's People

Contributors

Stargazers

Watchers

ingestify's Issues

StatsbombGithub invalid json response

StatsbombGithub can return:

Make sure responses is checked, and possible also if response contains valid json


<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>503 between bytes timeout</title>
  </head>
  <body>
    <h1>Error 503 between bytes timeout</h1>
    <p>between bytes timeout</p>
    <h3>Error 54113</h3>
    <p>Details: cache-ams21069-AMS 1691565538 962269883</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

Process updates on windows not working

On Windows we use the spawn start method for multiprocessing Pool. This somehow breaks the logger used in run_task

ChatGPT:

import logging
import logging.handlers
import multiprocessing
import os

def worker_process(q, i):
    qh = logging.handlers.QueueHandler(q)
    root = logging.getLogger()
    root.addHandler(qh)
    root.setLevel(logging.INFO)

    logging.info(f'Hello from worker {i}')

def logger_thread(q):
    while True:
        try:
            record = q.get()
            if record is None:  # We send this as a sentinel to tell the listener to quit.
                break
            logger = logging.getLogger(record.name)
            logger.handle(record)  # No level or filter logic applied - just do it!
        except Exception:
            import sys, traceback
            print('Problem:', file=sys.stderr)
            traceback.print_exc(file=sys.stderr)

def main():
    q = multiprocessing.Queue()

    log_thread = multiprocessing.Process(target=logger_thread, args=(q,))
    log_thread.start()

    workers = []
    for i in range(5):
        wp = multiprocessing.Process(target=worker_process, args=(q, i))
        workers.append(wp)
        wp.start()

    for wp in workers:
        wp.join()

    # Tell the logging server to shut down
    q.put(None)
    log_thread.join()

if __name__ == '__main__':
    main()

Recommend Projects