Giter Club home page Giter Club logo

gp-extract-data's Introduction

Google Play Developer Reporting API ETL Playground

This project aims to deepen the understanding of information capture from the Google Play Developer Reporting Api.

Here are some objectives that will be met during the course of development.

  • Create a functional Google Api authentication method;
  • Create a replicable standard contract model for obtaining reports;
  • Generate JSON files from query responses;
  • Transform JSON files into normalized information files;
  • Implement a repetition loop to increment reports with a size greater than one thousand objects;
  • Use environments parameters to period to get raw data;
  • Create a playground MVP based on pandas;
  • Build example datasets from the local folder;-
  • Create an example dashboard;
  • Use environment variables instead of a JSON credentials file;
  • Move params to environment file;
  • Create folder raw_data for the different apps;
  • Create folder datasets for the different apps;
  • Create default methods to log;
  • Merge data frames;
  • Refactor code;
  • Discovery about the return of the Error Report query;
  • Refactor structure responsible for controlling reports;
  • Fix raw_data folders name;
  • Create exception when page failed;
  • Implement retry to ssl timeout error or verify if pageToken is resilient;
  • Fix period information for reports;
  • Create methods to transform/load by metrics type and reports type;
  • Verify existant filters. Ex.: If dataset don't have a date value;
  • Create friendly documentation;
  • Get massive data;
  • Fix without null dates on merge datasets on Playground;
  • Create workload for ingest data on BigQuery;
  • Create module to increment data;
  • Fix datetime from logs;
  • Recognize fresh data;
  • Disable ErrorCount until you understand its real use
  • Separate reports dictionary in json file
  • Add userPerceivedAnrRate metric to AnrRateMetricSet

gp-extract-data's People

Contributors

maquiavelli avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.