Giter Club home page Giter Club logo

mara-metabase's Introduction

Mara Metabase

Build Status PyPI - License PyPI version Slack Status

Automating the setup and configuration of Metabase instances. Comes with

  • Functions for adding users, databases & settings

  • Syncing of field descriptions and metrics defined with Mara Schema to Metabase

  • Syncing of users and permissions to Metabase

  • A Makefile for running Metabase locally

 

Installation

To use the library directly, use pip:

pip install mara-metabase

or

pip install -e git+https://github.com/mara/mara-metabase.git

For an example of an integration into a flask application, have a look at the mara example project 1.

 

Running Metabase

From within a project, include [https://github.com/mara/mara-metabase/tree/master/.scripts/metabase.mk] in your project Makefile (as for example in https://github.com/mara/mara-example-project-1/blob/master/Makefile).

Running make setup-metabase will download a Metabase jar file, run database migrations, add an admin user & create a database connection to the data warehouse. Then, running make run-metabase will start Metabase on port 3000:

Metabase

You can log in with the default username [email protected] and the password 123abc (configurable via mara_metabase/config.py, please change in production).

For running Metabase in production, please have a look at https://www.metabase.com/docs/latest/operations-guide/installing-metabase.html.

 

Features

Programatically setting up Metabase

The file mara_metabase/setup.py contains functions for configuring a Metabase instance by directly writing to its metadata database (rather than manually configuring an instance through the UI). These functions are particularly useful when multiple Metabase instances are maintained (e.g. one for each testing / staging environment).

There is a flask cli command for running the setup function: flask mara_metabase.setup.

Danger: this will remove any other configured database connections from metabase!

 

Metadata sync

If you have a data warehouse schema defined in Mara Schema, then you can automatically sync field descriptions and metric definitions with the update_metadata function in mara_metabase/metadata.py.

Here's an example of an automatically synced definition for the metric average order value:

Metabae metadata sync

This function assumes that all reporting tables were created using the sql-generation features of Mara Schema.

The schema sync can be triggered manually with flask mara_metabase.update-metadata.

Have a look at https://github.com/mara/mara-example-project-1/blob/master/app/pipelines/update_frontends/__init__.py for how to integrate schema sync into a data pipeline.

 

Syncing of users, groups & permissions from Mara ACL

Once you add the Metabase ACL resource in mara_metabase/views.py to your project, you can easily control which users can query which data set:

Metabase ACL

The sync must be explicitly enabled in your setup code to not interfere with a metabase instance which has its own user management:

# e.g. in app/ui/__init__.py
import mara_metabase.acl
mara_metabase.acl.enable_automatic_sync_of_users_and_permissions_to_metabase()

Danger: Enabling this feature will overwrite any existing users, groups & permissions in Metabase

After enabling, all users in Mara ACL will be synced to Metabase together with their respective groups on "Save" of the permissions. Permissions in Metabase can be given for all or for individual data sets.

In this example, users from the "Management" group can query all data sets, and users from "Marketing" only "Customers" and "Leads" (with the exception of Thomas who can also query "Order items" and "Sellers").

This is the same data in the Metabase admin UI:

Metabase permissions

 

The easiest way to try out Mara Metabase is to run the mara example project 1.

mara-metabase's People

Contributors

jankatins avatar leo-schick avatar martin-loetzsch avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

danielseider

mara-metabase's Issues

Optional settings for fields

Some things would in my opinion be useful:

  • Currently only predefined metrics are available but not the underlying columns for these metrics. Sometimes it would be nice to also use metrics in groupers and so on: it would be nice if there could be a way to not hide a column which is the base for a measure.
  • Metabase has a field type, so it would be nice if this could be set based on what is in mara-schema. E.g. if mara-schema would gain a Type.QUANTITY mara/mara-schema#8 it would be nice if this could be mapped to the metabase field type 'Quantity' so these columns are not automatically set to 'Category' in some cases.

As mara-schema should probably not be specific to the viz layer after it, maybe all attributes/measures in mara-schema could get an optional label: Optional[dict] = None so that this could be passed through? E.g. labels={'mara-metabase':{'hide': False, 'field_type': 'Quantity'}}?

Add config to enable/disable user sync to metabase

It was really surprising to me to see that this is quietly enabled by just setting up (=importing) mara_metabase. It would be nice if this could be disabled by default and only enabled by a config.

This just destroyed our metabase roles on prod. Our metabase is Not managed by the mara UI and after clicking the save on the UI after giving someone access all configured mara UI users were reset :-(

Display name for compose metrics is set to formula instead of name

Currently a composed metric gets the formula as display name but simple metrics default to the name as defined in the mara-schema:

https://github.com/mara/mara-metabase/blob/master/mara_metabase/metadata.py#L69

     'definition': {'source-table': table['id'],
                                         'aggregation': [
                                             metric_definition(_metric, table)
                                             if isinstance(_metric, SimpleMetric)
                                             else ['aggregation-options',
                                                   metric_definition(_metric, table),
                                                   {'display-name': _metric.display_formula()}
                                                   ]
                                         ]},

This means if you explore the dataset in metabase and summarize, you will have names as in the data dictionary for simply metrics but the formulas for composed metrics in parts of the graph. E.g. here the headline has the names, but the legend has the formula

image

Is that intended? If yes: would it be ok to make this configureable (without me vendoring /patching the whole metadata sync function?)

Add a button which reruns sync

We do not have easy access to the commandline, so it would be nice to get a button somewhere to run the sync without triggering a pipeline run or so.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.