Giter Club home page Giter Club logo

attachment-downloader's Introduction

Gmail Bulk Attachment Downloader

Currently gmail is not providing to download attachments from different different mails with in single click. Using this program you can do it now.

You need credentials.json before start this utility. You can get it from https://developers.google.com/gmail/api/quickstart/nodejs by enable GMAIL API. Save file credentials.json in the root folder of the project

Clone Project

git clone https://github.com/munir131/attachment-downloader

Install dependencies

npm i

Run program in interactive mode

node index.js

Run program in non-interactive mode

node index.js --label LABEL_NAME

Contributors

Thanks to all the people who already contributed!

Made with contrib.rocks.

attachment-downloader's People

Contributors

dependabot[bot] avatar k-funk avatar lpirkwieser avatar mirkoperillo avatar munir131 avatar thesnook avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

attachment-downloader's Issues

Multiple Labels

Is it possible or easy to add an option to select multiple labels and create folders under ./files?
We have a few hundred labels to go through one by one and create a folder for each and then download the files to be sorted.

It only downloads the first attachment of each email.

I tried both "from:" and label approach, the result is same.

If the email has multiple attachments (in my case, they are all images), only the first one will be downloaded.

Also, when I'm using the "from:" filter, it's even worse: I found at least one email wasn't downloaded at all (it's exactly the 51th one out of 51 emails that isn't downloaded, not sure if relevant.)

Use sqlite for manage files

Initial plan is to organise it more better.

  • Use database to manage downloaded files.
  • Avoid duplicate download of same file

Feature Request: filename/filetype filter

Use case: I only want to save pdfs. Right now, I'm running cd files && rm -- ^*.pdf in zsh after running this script, in order to remove the unwanted files.

This could instead be filtered during the query using filename:pdf as per the docs.

There is, however, one problem with only modifying the query: if an email has more than one attachment, both will still get downloaded. To make this function as expected, there would also need to be logic in function fetchAndSaveAttachment() to filter out unwanted files.

not all attachments are downloaded

hello, firstly I'd like to say nice project!

I tried using it and it found the label I was looking for, it found all the emails under the label but it didn't find any attachments

I traced the problem to index.js, line 189 in the function pluckAllAttachments

      if (!p.body || !p.body.attachmentId) {
        return undefined;
      }

It seems not all messages that have attachments, have an attachmentId. Sometimes they are found in the same object but in the data key. See gmail api https://developers.google.com/gmail/api/reference/rest/v1/users.messages.attachments

it seems the messages that are causing the problem have nested parts which contain the relevant attachmentId

image

The messages in question were forwarded with an iPad FWIW

EDIT: removed something I was wrong about, added what the actual problem was 😅

Optimization: `has:attachment`

Since the goal is to only get emails with attachments, the query can be structured with has:attachment, as per these docs.

The result shouldn't be any different, but it will probably have significant performance improvement to include that.

Hitting rate limits

I get this error with any search with more than a few emails:

GaxiosError: Too many concurrent requests for user
at Gaxios.<anonymous> (/attachment-downloader/node_modules/gaxios/build/src/gaxios.js:72:27)
at Generator.next (<anonymous>)
at fulfilled (/attachment-downloader/node_modules/gaxios/build/src/gaxios.js:16:58)
at processTicksAndRejections (internal/process/task_queues.js:89:5) {

It'll be great if this library could throttle itself to stay within Gmail's API limits!

Didn't work, validation of google account failed

Enter the code from that page here: 4/twF41gClfys6SrAwY7WFACfLEFMsSH1JnIWZMrJW5tEbQkxMISh_Nj4
Error retrieving access token { FetchError: request to https://oauth2.googleapis.com/token failed, reason: connect ETIMEDOUT 172.217.160.74:443
    at ClientRequest.<anonymous> (E:\迅雷下载\gmail_bulk\attachment-downloader\node_modules\node-fetch\lib\index.js:1444:11)
    at ClientRequest.emit (events.js:189:13)
    at TLSSocket.socketErrorListener (_http_client.js:392:9)
    at TLSSocket.emit (events.js:189:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  message:
   'request to https://oauth2.googleapis.com/token failed, reason: connect ETIMEDOUT 172.217.160.74:443',
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT',
  config:
   { method: 'POST',
     url: 'https://oauth2.googleapis.com/token',
     data:
      'code=4%2FtwF41gClfys6SrAwY7WFACfLEFMsSH1JnIWZMrJW5tEbQkxMISh_Nj4&client_id=280431205846-gtd3mva42iv230vhas4do6hsdj5f0jbv.apps.googleusercontent.com&client_secret=X1tQT4lWJsKYAKjLCiTzF1A4&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&grant_type=authorization_code&code_verifier=',
     headers:
      { 'Content-Type': 'application/x-www-form-urlencoded',
        'User-Agent': 'google-api-nodejs-client/3.1.0',
        Accept: 'application/json' },
     params: [Object: null prototype] {},
     paramsSerializer: [Function: paramsSerializer],
     body:
      'code=4%2FtwF41gClfys6SrAwY7WFACfLEFMsSH1JnIWZMrJW5tEbQkxMISh_Nj4&client_id=280431205846-gtd3mva42iv230vhas4do6hsdj5f0jbv.apps.googleusercontent.com&client_secret=X1tQT4lWJsKYAKjLCiTzF1A4&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&grant_type=authorization_code&code_verifier=',
     validateStatus: [Function: validateStatus],
     responseType: 'json' }

UI?

Planning to use UI based application for end user. So, no need to code and enhance UX. Currently, I plan to use Electron as we can get two benefits

  1. Cross platform
  2. Portable

Broken unless gmail has more than 500 emails + missing last page

My gmail currently has ~300 emails. The files directory was empty even after the script said Done.

I modified return getAllMails(auth, 500) to return getAllMails(auth, 100), and was able to get files.

I see two problems with the code:

1

I think that this is causing the bug.
https://github.com/munir131/attachment-downloader/blob/master/index.js#L285-L292
If there's only one page, messageIds will be an empty array. It also looks like the last page is being skipped even when a user has over 500 emails, since messageIds.concat( is only being called when there's a nextPageToken.

Perhaps

      if (response.data) {
        messageIds = messageIds.concat(response.data.messages)
        if (response.data.nextPageToken) {
           spinner.text = "Reading page: " + ++pageCounter
           resolve(getAllMails(auth, maxResults, response.data.nextPageToken))
         }
      }

      spinner.text = "All pages are read"
      resolve(messageIds)

function getListOfMailIdByLabel(...) looks like it may not have the same issue, since message ids are being added to the array in both conditions.


When I tried my above code, there were more attachments, which confirms that the current code is missing a page.

2

maxResults isn't being passed down to the recursive function, but instead, 500 is being used for all subsequent pages after the initial.
https://github.com/munir131/attachment-downloader/blob/master/index.js#L288
Should be resolve(getAllMails(auth, maxResults, response.data.nextPageToken))

BUG: 'length' of undefined

I get the following error:

 node index.js --label TODL
TypeError: Cannot read property 'length' of undefined
    at /home/micha/src/attachment-downloader/index.js:186:34
    at arrayMap (/home/micha/src/attachment-downloader/node_modules/lodash/lodash.js:639:23)
    at Function.map (/home/micha/src/attachment-downloader/node_modules/lodash/lodash.js:9554:14)
    at pluckAttachment (/home/micha/src/attachment-downloader/index.js:180:22)
    at /home/micha/src/attachment-downloader/index.js:52:30

This fixed the error, but I don't know if it's the "correct" fix:

diff --git a/index.js b/index.js
index 418aeb0..4985763 100644
--- a/index.js
+++ b/index.js
@@ -178,7 +178,7 @@ function saveFile(fileName, content) {
 
 function pluckAttachment(mails) {
   return _.compact(_.map(mails, (m) => {
-    if (!m.data) {
+    if (!m.data || !m.data.payload.parts) {
       return undefined;
     }
     const attachment = {

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.