Giter Club home page Giter Club logo

ikoa-web's Introduction

ikoa-web

ikoa-web is a third-party app which helps you using iKOA more easily.

Features:

  • The docker image(186MB in total) is based on i386/alpine.
  • Auto upload to google drive after download finished.
  • Show the download and upload progress in web page real-time.
  • Batch downloading.
  • Task Queue
  • Naming IDs with a tag.
  • Forcing https.
  • Comprehensive download statistics data(csv format).
  • Customizing the admin account for logining in the web page.

Update:

  • You can choose to whether download monthly videos only(find the option in deploy page).
  • Show the waiting time if you download too fast.
  • Check whether the ID belongs to monthly video before download.
  • Choose the rclone upload strategy based on the serialCode quota.
  • Quit the download task if the serialCode quota is zero.
  • Add support for iKOA version-1.5.2
  • Add support for iKOA version-1.6.1

Prerequisites:

  1. You have google team drive(shared drive).
  2. You can generate the google drive SA(service account) by yourself.
  3. You have valid serialCode.

How to deploy:

Click the button below to deploy to heroku using your own heroku account.
Deploy

Tips:

  1. Be careful filling in the "herokuApp" field when deploy the app, otherwise your app can't work properly.
  2. The format of "SA_JSON" field should look like the first line below(delete any spaces between each line,but don't add any line break between each line,then paste it into one line):
    {"type": "service_account","project_id": "xxxxx","private_key_id":"xxxxx",......}  
    {"type": "service_account",\n"project_id": "xxxxx",\n"private_key_id":"xxxxx",......}  x
  3. You can ssh into the server from command line: heroku ps:exec -a yourAppName. See more details
  4. You can update the iKOA version in the folder called fanza(fork the repo first, and replace the iKOA with a newer version,change the repository field in app.json, then deploy your own repo to heroku).
  5. The app will be restarted automatically once every 24 hours continuous running due to heroku's policy. See more details
  6. the format of num ID:"ABC-123" or "ABC-123,abc-124,ABC-125 and more"(comma separated,case insensitive)
  7. the format of cid ID:"abc00123" or "abc00123,abc00124,abc00125 and more"(comma separated,case insensitive)
  8. the format of mgs ID:"259LUXU-1200" or "259LUXU-1200,259LUXU-1201,259LUXU-1202 and more"(comma separated,case insensitive)
  9. If you submit many IDs one time, you can put a tag on these IDs, so they can be downloaded under the same folder.(the tag length should be no more than 10 characters, be free to use chinese or japanese name or any other language)
  10. In theory, ikoa-web can upload no more than 1.5TB data to your google team drive per day.

FAQ:

  • Q: How can I change the config var after deployment?
    A: You can change the config var in the settings page of the dashboard. See more details
  • Q: Why does rclone failed to upload files to google team drive?
    A: First, You need to config "TEAM_DRIVE_ID" "RCLONE_DESTINATION", "LOG_PATH", "SA_JSON_1", "SA_JSON_2" properly.
        And then check whether the two SA has been added into your team drive as a member with write permission at least(contributor or content manager).
  • Q: What does "codenotenough" mean in the csv file?
    A: It means you need to get a valid "SERIAL_CODE".
  • Q: Can ikoa-web bypass google drive's 750GB per day upload limit?
    A: Yes, This is why you need config two SA.
  • Q: How should I do if I can't config the SA_JSON field correctly?
    A: Check whether the one line json string matches exactly 2374 characters.
  • Q: Why does the app shutdown suddenly before all tasks finished sometimes?
    A: Please create an issue and paste the log.

License

ikoa-web is released under the MIT License

ikoa-web's People

Contributors

mason1901 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

ikoa-web's Issues

upload to onedrive or other?

I think you are using rclone to upload , so the files could be uploaded to other cloud drive like onedrive.
can you give some suggest to do it ? [sorry for my poor English]
大佬你应该用的是rclone那么理论上也应该可以上传到别的网盘
请问能简单指点下吗?

VER 1.6

Can update ver 1.6?
thanks

Please update ikoa version to 1.5.2

I've tried to fork the repo and replace the iKOA with a newer version but after deployed the app, it just refused to download. Could you update the version in your repo please.

How can I use docker to deploy in VPS Server

Because Heroku immediately suspends when a deploy is done, I want to use VPS local environment to run, but when I finish the docker build and it's a success, the next step is confusing me, how to run use "docker run"?

App creation failed

图像 1

Create app
App creation failed. Please ensure you have valid values in the above form fields.

Build app Hide build log
There was an issue building your app. This can mean your app.json's project is not a valid Heroku application. Please ensure your app is deployable to Heroku and try again.

=== Fetching app code
=== Building web (Dockerfile)
Sending build context to Docker daemon 27.84MBStep 1/10 : FROM i386/alpine:edge
edge: Pulling from i386/alpine
89c83633045a: Pulling fs layer
89c83633045a: Verifying Checksum
89c83633045a: Download complete
89c83633045a: Pull complete
Digest: sha256:4716d67546215299bf023fd80cc9d7e67f4bdc006a360727fd0b0b44512c45db
Status: Downloaded newer image for i386/alpine:edge
---> 451b8982a039
Step 2/10 : ENV GLIB_VERSION=2.26-6 GLIB_ARCH=i686 PYTHONUNBUFFERED=1
---> Running in e108d4c294ca
Removing intermediate container e108d4c294ca
---> 02f74d009491
Step 3/10 : ADD ./config/ld.so.conf ./tmp/ld.so.conf
---> 95a8525259a5
Step 4/10 : ADD ./webapp/requirements.txt /tmp/requirements.txt
---> d001d27a9423
Step 5/10 : ADD ./config/heroku-exec.sh /etc/profile.d/
---> 80332ccd9315
Step 6/10 : RUN apk update && apk add --no-cache tar procps findutils grep tzdata python3 py3-pip bash curl rclone zlib libgcc ts openssh && apk add --no-cache --virtual .build-dependencies python3-dev libevent-dev gcc musl-dev && pip3 install --no-cache-dir -q -r /tmp/requirements.txt && apk del .build-dependencies && if [[ ! -f /usr/bin/python ]]; then ln -s /usr/bin/python3 /usr/bin/python; fi && mkdir -p glibc-${GLIBC_VERSION} /usr/glibc && ln -s /bin/bash /usr/bin/bash && curl -sL http://mirrors.aggregate.org/archlinux/core/os/${GLIB_ARCH}/glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz -o glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz && tar xf glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz -C glibc-${GLIBC_VERSION} && mv tmp/ld.so.conf /etc/ld.so.conf && cp -a glibc-${GLIBC_VERSION}/usr /usr/glibc/ && glibc-${GLIBC_VERSION}/usr/bin/ldconfig /usr/glibc/usr /usr/glibc/usr/lib && ln -s /usr/glibc/usr/lib/ld-linux.so.2 /lib/ld-linux.so.2 && rm -Rf glibc-${GLIBC_VERSION} glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz && cp /lib/libc.musl-x86.so.1 /usr/lib && cp /lib/libz.so.1 /usr/lib && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone && apk del tar tzdata && rm /bin/sh && ln -s /bin/bash /bin/sh
---> Running in 1eb4fcf29c52
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86/APKINDEX.tar.gz
v20200917-1784-g03591b17aa [https://dl-cdn.alpinelinux.org/alpine/edge/main]
v20200917-1787-g5add7c22af [https://dl-cdn.alpinelinux.org/alpine/edge/community]
OK: 12919 distinct packages available
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86/APKINDEX.tar.gz
(1/60) Installing ncurses-terminfo-base (6.2_p20201003-r1)
(2/60) Installing ncurses-libs (6.2_p20201003-r1)
(3/60) Installing readline (8.0.4-r0)
(4/60) Installing bash (5.0.18-r0)
Executing bash-5.0.18-r0.post-install
(5/60) Installing ca-certificates (20191127-r4)
(6/60) Installing brotli-libs (1.0.9-r1)
(7/60) Installing nghttp2-libs (1.41.0-r0)
(8/60) Installing libcurl (7.72.0-r0)
(9/60) Installing curl (7.72.0-r0)
(10/60) Installing findutils (4.7.0-r0)
(11/60) Installing pcre (8.44-r0)
(12/60) Installing grep (3.5-r0)
(13/60) Installing libgcc (10.2.0-r5)
(14/60) Installing openssh-keygen (8.4_p1-r0)
(15/60) Installing libedit (20191231.3.1-r1)
(16/60) Installing openssh-client (8.4_p1-r0)
(17/60) Installing openssh-sftp-server (8.4_p1-r0)
(18/60) Installing openssh-server-common (8.4_p1-r0)
(19/60) Installing openssh-server (8.4_p1-r0)
(20/60) Installing openssh (8.4_p1-r0)
(21/60) Installing libintl (0.20.2-r0)
(22/60) Installing libproc (3.3.16-r0)
(23/60) Installing procps (3.3.16-r0)
(24/60) Installing libbz2 (1.0.8-r1)
(25/60) Installing expat (2.2.9-r1)
(26/60) Installing libffi (3.3-r2)
(27/60) Installing gdbm (1.18.1-r0)
(28/60) Installing xz-libs (5.2.5-r0)
(29/60) Installing sqlite-libs (3.33.0-r0)
(30/60) Installing python3 (3.8.6-r0)
(31/60) Installing py3-appdirs (1.4.4-r1)
(32/60) Installing py3-ordered-set (4.0.2-r0)
(33/60) Installing py3-parsing (2.4.7-r1)
(34/60) Installing py3-six (1.15.0-r0)
(35/60) Installing py3-packaging (20.4-r0)
(36/60) Installing py3-setuptools (50.3.0-r0)
(37/60) Installing py3-chardet (3.0.4-r4)
(38/60) Installing py3-idna (2.10-r0)
(39/60) Installing py3-urllib3 (1.25.10-r0)
(40/60) Installing py3-requests (2.24.0-r2)
(41/60) Installing py3-msgpack (1.0.0-r0)
(42/60) Installing py3-lockfile (0.12.2-r3)
(43/60) Installing py3-cachecontrol (0.12.6-r0)
(44/60) Installing py3-colorama (0.4.3-r0)
(45/60) Installing py3-distlib (0.3.1-r1)
(46/60) Installing py3-distro (1.5.0-r1)
(47/60) Installing py3-webencodings (0.5.1-r3)
(48/60) Installing py3-html5lib (1.1-r0)
(49/60) Installing py3-pytoml (0.1.21-r0)
(50/60) Installing py3-pep517 (0.8.2-r0)
(51/60) Installing py3-progress (1.5-r0)
(52/60) Installing py3-toml (0.10.1-r0)
(53/60) Installing py3-retrying (1.3.3-r0)
(54/60) Installing py3-contextlib2 (0.6.0-r0)
(55/60) Installing py3-pip (20.1.1-r0)
(56/60) Installing rclone (1.53.0-r0)
(57/60) Installing libacl (2.2.53-r0)
(58/60) Installing tar (1.32-r1)
(59/60) Installing ts (1.0-r1)
(60/60) Installing tzdata (2020a-r0)
Executing busybox-1.32.0-r3.trigger
Executing ca-certificates-20191127-r4.trigger
OK: 121 MiB in 74 packages
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86/APKINDEX.tar.gz
(1/17) Upgrading musl (1.2.1-r1 -> 1.2.1-r2)
(2/17) Installing pkgconf (1.7.3-r0)
(3/17) Installing python3-dev (3.8.6-r0)
(4/17) Installing libevent (2.1.12-r1)
(5/17) Installing libevent-dev (2.1.12-r1)
(6/17) Installing libstdc++ (10.2.0-r5)
(7/17) Installing binutils (2.35.1-r0)
(8/17) Installing gmp (6.2.0-r0)
(9/17) Installing isl (0.18-r0)
(10/17) Installing libgomp (10.2.0-r5)
(11/17) Installing libatomic (10.2.0-r5)
(12/17) Installing libgphobos (10.2.0-r5)
(13/17) Installing mpfr4 (4.1.0-r0)
(14/17) Installing mpc1 (1.2.0-r0)
(15/17) Installing gcc (10.2.0-r5)
(16/17) Installing musl-dev (1.2.1-r2)
(17/17) Installing .build-dependencies (20201005.084628)
Executing busybox-1.32.0-r3.trigger
OK: 270 MiB in 90 packages
(1/16) Purging .build-dependencies (20201005.084628)
(2/16) Purging python3-dev (3.8.6-r0)
(3/16) Purging libevent-dev (2.1.12-r1)
(4/16) Purging libevent (2.1.12-r1)
(5/16) Purging gcc (10.2.0-r5)
(6/16) Purging binutils (2.35.1-r0)
(7/16) Purging libatomic (10.2.0-r5)
(8/16) Purging libgomp (10.2.0-r5)
(9/16) Purging libgphobos (10.2.0-r5)
(10/16) Purging musl-dev (1.2.1-r2)
(11/16) Purging pkgconf (1.7.3-r0)
(12/16) Purging libstdc++ (10.2.0-r5)
(13/16) Purging isl (0.18-r0)
(14/16) Purging mpc1 (1.2.0-r0)
(15/16) Purging mpfr4 (4.1.0-r0)
(16/16) Purging gmp (6.2.0-r0)
Executing busybox-1.32.0-r3.trigger
OK: 121 MiB in 74 packages
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
The command '/bin/sh -c apk update && apk add --no-cache tar procps findutils grep tzdata python3 py3-pip bash curl rclone zlib libgcc ts openssh && apk add --no-cache --virtual .build-dependencies python3-dev libevent-dev gcc musl-dev && pip3 install --no-cache-dir -q -r /tmp/requirements.txt && apk del .build-dependencies && if [[ ! -f /usr/bin/python ]]; then ln -s /usr/bin/python3 /usr/bin/python; fi && mkdir -p glibc-${GLIBC_VERSION} /usr/glibc && ln -s /bin/bash /usr/bin/bash && curl -sL http://mirrors.aggregate.org/archlinux/core/os/${GLIB_ARCH}/glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz -o glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz && tar xf glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz -C glibc-${GLIBC_VERSION} && mv tmp/ld.so.conf /etc/ld.so.conf && cp -a glibc-${GLIBC_VERSION}/usr /usr/glibc/ && glibc-${GLIBC_VERSION}/usr/bin/ldconfig /usr/glibc/usr /usr/glibc/usr/lib && ln -s /usr/glibc/usr/lib/ld-linux.so.2 /lib/ld-linux.so.2 && rm -Rf glibc-${GLIBC_VERSION} glibc-${GLIB_VERSION}-${GLIB_ARCH}.pkg.tar.xz && cp /lib/libc.musl-x86.so.1 /usr/lib && cp /lib/libz.so.1 /usr/lib && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone && apk del tar tzdata && rm /bin/sh && ln -s /bin/bash /bin/sh' returned a non-zero code: 2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.