The sia.ch-scrapper from lesh1k

sia.ch-scrapper's People

Contributors

Watchers

sia.ch-scrapper's Issues

Members total and pages total is not correct when running on all 15K entries

Pages/members parsed gives a correct number, while total is different and wrong, by far, about 3K.
For pages it is even null.

Excerpt from metrics.json:

{
    "created_on": "2016-07-13T21:55:56.254Z",
    "number_of_CPUs": 4,
    "number_of_workers": 8,
    "pages": {
        "time": {
            "total": 4095460,
            "min": 5904,
            "max": 18246,
            "avg": "NaN"
        },
        "total": null,
        "parsed": 318
    },
    "members": {
        "time": {
            "total": 20525104,
            "min": 153,
            "max": 3760,
            "avg": "1602.90"
        },
        "total": 12805,
        "parsed": 15804
    }
}, {
    "created_on": "2016-07-13T23:40:37.253Z",
    "number_of_CPUs": 4,
    "number_of_workers": 4,
    "pages": {
        "time": {
            "total": 4641999,
            "min": 5256,
            "max": 18895,
            "avg": "14597.48"
        },
        "total": null,
        "parsed": 318
    },
    "members": {
        "time": {
            "total": 14800604,
            "min": 162,
            "max": 2769,
            "avg": "936.51"
        },
        "total": 12805,
        "parsed": 15804
    }

Recommend Projects