7

Unlocking Your Data With Metabase and AWS Fargate

 2 years ago
source link: https://www.pulumi.com/blog/unlocking-your-data-with-metabase-and-aws/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Unlocking Your Data With Metabase and AWS Fargate

Posted on Tuesday, Aug 9, 2022

I love data. I mean, I really love data. Data gives you the ability to understand the world around you and, to a certain degree, project what the future could look like. At Pulumi we use data every day to help make smarter product and business decisions.

Though one hurdle we encountered was not only the sheer volume of data we have but also the large disparity of systems storing that data. Like many companies before us, we chose to build a data warehouse, specifically AWS Redshift, to store all of our data.

After grinding through the work to consolidate our data into a single source of truth, we felt like we had conquered the world. That feeling was fleeting as we quickly realized we needed a scalable way to save, abstract, and collaborate with our data across the entire organization.

Using Metabase to unlock our data

As an early stage start-up at the time, we needed a low-cost solution (free) which limited our available options. One advantage we did have at the time was that as a cloud native start-up we were comfortable with the upfront investment and operational overhead of self-hosting the solution. With all of our parameters set, we eventually settled on using Metabase .

What is Metabase?

Metabase is a Business Intelligence Tool that enables you to visualize and collaborate using data from a variety of databases. You do not need to understand the underlying database’s query language, SQL for example, because Metabase provides a UI to interact with your data. You can however drop into a query editor at any moment to perform more advanced queries.

Metabase provides you with two options to run Metabase in production , Open Source (self-managed) and Enterprise (managed). The main difference between the two is that with Open Source you have to manage the infrastructure yourself, whereas with Enterprise, Metabase will manage the infrastructure for you for a fee.

Deploying Metabase to production

Once we settled on our business intelligence tool it was time to wire it up and start living that sweet data-driven lifestyle. To run the service yourself, Metabase provides you with two options: a Docker Image or a .JAR. We were already running our SaaS service on ECS, so naturally, we opted to go with the Docker approach. In addition to running the Docker Image, we also needed to provision a database so that Metabase could store the relevant data it needs to run the service.

To accomplish the task of provisioning all the required resources, the team wrote a Pulumi Component to encapsulate all the complexity and provide a simple interface for provisioning and updating the service. Unfortunately, this was before we had developed Pulumi Packages and the Component was only available within the program itself.

The Metabase Package

That brings us to today, and we are happy to deliver an open source Pulumi Package for running Metabase on AWS ECS. With Pulumi’s Metabase Package, you can quickly get started with Metabase without having to worry about the underlying infrastructure.

In the below sections we will look at example configurations of the Metabase resource, all the way from a bare bones configuration to a more complex configuration with a custom VPC, subnets, and domain.

Default

By default, the Package accepts zero arguments and will deploy your resources into your default VPC, on public subnets, and without a custom domain name.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {});

export const url = metabaseService.dnsName;

Custom VPC

If you would like to run the service in a specific VPC, then you can provide a VPC ID as an argument. The example below will provision your resources in your defined VPC, on public subnets, and without a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
    vpcId: "vpc-123",
});

export const url = metabaseService.dnsName;

Custom Networking

If you would like to run the Load Balancer, Database, or ECS Service on specific subnets you can provide the IDs of those subnets. The subnets you provide need to be a part of the VPC you are deploying into. The example below will provision your resources in your defined VPC (or default if you do not provide a VPC ID), on your specified subnets, and without a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
    vpcId: "vpc-123",
    networking: {
        ecsSubnetIds: [
            "subnet-123",
            "subnet-456",
        ],
        dbSubnetIds: [
            "subnet-789",
            "subnet-abc",
        ],
        lbSubnetIds: [
            "subnet-def",
            "subnet-ghi",
        ],
    },
});

export const url = metabaseService.dnsName;

Custom Domain

If you would like to have your service run behind a custom domain, you can specify the hosted zone and domain name as arguments. The example below will provision your resources in your default VPC, on public subnets, with a custom domain name.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
    domain: {
        hostedZoneName: "example.com",
        domainName: "metabase.example.com",
    },
});

export const url = metabaseService.dnsName;

All Together Now

You can provide all the arguments at once if you would like more control over your infrastructure. The below example will provision your resources in your defined VPC, with your defined subnets, and have a custom domain.

import * as pulumi from "@pulumi/pulumi";
import * as metabase from "@pulumi/metabase";

const metabaseService = new metabase.Metabase("metabaseService", {
    vpcId: "vpc-123",
    networking: {
        ecsSubnetIds: [
            "subnet-123",
            "subnet-456",
        ],
        dbSubnetIds: [
            "subnet-789",
            "subnet-abc",
        ],
        lbSubnetIds: [
            "subnet-def",
            "subnet-ghi",
        ],
    },
    domain: {
        hostedZoneName: "example.com",
        domainName: "metabase.example.com",
    },
});

export const url = metabaseService.dnsName;

Delivery in 30 minutes or less

To get started unlocking your data with Metabase, head on over to the Metabase Package. Follow the quick walkthrough and have your Metabase service running in 30 minutes or less (in most cases). Once your service is up and running, we recommend popping over the Metabase Documentation to learn How to Set Up Metabase .

If you encounter an issue or have a feature request, please file an issue in the Package’s Github Repo .


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK