12

example docker compose for postgresql with db init script

 1 year ago
source link: https://gist.github.com/onjin/2dd3cc52ef79069de1faa2dfd456c945
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

example docker compose for postgresql with db init script · GitHub

Instantly share code, notes, and snippets.

example docker compose for postgresql with db init script

In my case file is copied, but sql file is not executed? what is the config required for running it too.

@kamal-github: Try to remove data volume for postgres data and up your container again.

Not working for me either, my docker-compose file looks like this. I see that file present in docker-entrypoint-initdb.d but I think its is not executing it.

postgres:
image: onjin/alpine-postgres:9.5
restart: unless-stopped
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/1-init.sql
ports:
- "5432:5432"

Thanks,
Rambabu

getting same error as @g-bastianelli . I am on windows 8.1 using docker toolbox and getting this:

psql:/docker-entrypoint-initdb.d/init.sql:0: could not read from input file: Is a directory

I have windows 10 and i am using postgresql 10.3 image and also get this message
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
here my compose file
version: '2' services: gateway-postgresql: image: postgres:10.3 volumes: - ./postgres-init/:/docker-entrypoint-initdb.d/ #also tried - ./postgres-init:/docker-entrypoint-initdb.d environment: - POSTGRES_MULTIPLE_DATABASES=db1,db2 - POSTGRES_USER=abc - POSTGRES_PASSWORD=abc ports: - 5432:5432
I have folder postgres-init/initdb.sh

Ordering the scripts alphabetically works but there are a few caveats. If you are ordering using numbers be aware that if you have more than 9 files then the script name that begins with 10 is executed before the script whose name begins with 1

the thing here is that no mather what we have in docker-entrypoint-initdb.d in my case a single file called init,sh, seems that all are ignored there.

rjurney commented on Jun 20, 2018

edited

My docker-compose.yaml file looks like this below. I've tried all combinations of possible configurations.

        volumes: 
            - ./dumps:/docker-entrypoint-initdb.d/dumps
            - ./dumps/portal.sql:/docker-entrypoint-initdb.d/portal.sql
            - ./dumps:/docker-entrypoint-initdb.d/dumps.sql

And ./dumps contains portal.sql. However, I get this error: /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/dumps

I am following the directions, so what gives?

UPDATE: I got it to work!

Here is what works:

        volumes: 
            - ./dumps/portal2.sql:/docker-entrypoint-initdb.d/1-portal.sql
...
volumes:
    dumps:

This works. Apparently you have to have a subdirectory which you make a volume down below and then your scripts sit in it and can link as volumes directly inside the service.

@poloche make sure you aren't setting the PGDATABASE environment variable. I had a script that created this database, but got an error that the database didn't exist before the init scripts had a chance to run

I was facing the same issue of init scripts getting ignored.

I HAVE FOUND A SOLUTION if it helps anyone !!

version: '2'
services:

  common_db:
    image: postgres:10.4
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres123
      - POSTGRES_MULTIPLE_DATABASES=db1,db2
    volumes:
      - ./db.sql:/docker-entrypoint-initdb.d/db.sql
      - ./postgres-data:/var/lib/postgresql/data
    ports:
      - 5436:5432
    networks:
       - app-network

volumes:
  postgres-data:
networks:
  app-network:
    driver: bridge`

Now as you can see I have postgres-data: volume mounted.
If i remove this directory before starting docker-compose file things work !!.

I that directory/volume is present the init scripts are ignored !!
enjoy!!

init scripts are run when there postgres-data folder is created. If the postgres-data folder is initialized, the init scripts will be ignored.

My docker-compose.yaml file looks like this below. I've tried all combinations of possible configurations.

        volumes: 
            - ./dumps:/docker-entrypoint-initdb.d/dumps
            - ./dumps/portal.sql:/docker-entrypoint-initdb.d/portal.sql
            - ./dumps:/docker-entrypoint-initdb.d/dumps.sql

And ./dumps contains portal.sql. However, I get this error: /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/dumps

I am following the directions, so what gives?

UPDATE: I got it to work!

Here is what works:

        volumes: 
            - ./dumps/portal2.sql:/docker-entrypoint-initdb.d/1-portal.sql
...
volumes:
    dumps:

This works. Apparently you have to have a subdirectory which you make a volume down below and then your scripts sit in it and can link as volumes directly inside the service.

@rjurney are you sure that is the reason? I wouldn't think a top-level named volume would have anything to do with a local folder. Does anyone know?

init scripts are run when there postgres-data folder is created. If the postgres-data folder is initialized, the init scripts will be ignored.

nice caveat, thanks

I'm running into an issue where my db init process is in fact running in the container, at least batching a bunch of CREATES and INSERTS of my dump data, but then is abruptly shut down and restarted.

strends_psql | CREATE DATABASE
strends_psql | You are now connected to database "rivers" as user "usr".
strends_psql | CREATE SCHEMA
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | CREATE TABLE
strends_psql | INSERT 0 4
strends_psql |
strends_psql |
strends_psql | 2019-04-29 17:43:06.633 UTC [41] LOG:  received fast shutdown request
strends_psql | waiting for server to shut down....2019-04-29 17:43:06.637 UTC [41] LOG:  aborting any active transactions

When I get a shell into my db container, the database rivers was created but is empty, with no tables, relations, or data from my batched inserts.

@honestcomrade did you manage to fix this?

I'm running into an issue where my db init process is in fact running in the container, at least batching a bunch of CREATES and INSERTS of my dump data, but then is abruptly shut down and restarted.

strends_psql | CREATE DATABASE
strends_psql | You are now connected to database "rivers" as user "usr".
strends_psql | CREATE SCHEMA
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | SET
strends_psql | CREATE TABLE
strends_psql | INSERT 0 4
strends_psql |
strends_psql |
strends_psql | 2019-04-29 17:43:06.633 UTC [41] LOG:  received fast shutdown request
strends_psql | waiting for server to shut down....2019-04-29 17:43:06.637 UTC [41] LOG:  aborting any active transactions

When I get a shell into my db container, the database rivers was created but is empty, with no tables, relations, or data from my batched inserts.

I had the same issue but after a few hours of trying, I found a working combination of docker-compose and init script. Btw, I read here that the container shutting down and restarting is (apparently) part of the initialization process.
This is part of my initdb.sh:

#!/bin/bash
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" -d "$POSTGRES_DB"  <<-EOSQL
     create schema if not exists $SCHEMA;
     create table $SCHEMA.todos (
        id serial primary key,
        done boolean not null default false,
        task text not null,
        due timestamptz
     );
     create role $ANON nologin;
     create role $AUTHENTICATOR noinherit login password '$POSTGRES_PASSWORD';
     grant $ANON to $AUTHENTICATOR;
EOSQL

This is part of my docker-compose.yml:

version: '3.7'
services:
  db:
    image: postgres
    restart: always
    volumes:
      - ./initdb.sh:/docker-entrypoint-initdb.d/initdb.sh
      - ./pgdata:/var/lib/postgresql/data
    ports:
      - 8002:5432
    environment:
       POSTGRES_PASSWORD: docker
       POSTGRES_DB: mydb
       SCHEMA: public
       ANON: web_anon
       AUTHENTICATOR: authenticator

@Juliannnnshipit this looks like it might help me out, thanks a bunch for the reply.

Hi there! Thank you! Is there way to update my sql script and then update existing image, without data loss? for example I need to add trigger for existing postgres image...

Author

onjin commented on Sep 15, 2019

Hi there! Thank you! Is there way to update my sql script and then update existing image, without data loss? for example I need to add trigger for existing postgres image...

Hi, initdb.sh scripts are called only if there is no database set, so if you need to migrate current database it's better to run this sql migrations from psql cli for already created databases, and initdb.sh/ scripts will be run for new clean instances.

FYI: do not pass the host (localhost) in the initialization script or you'll likely to encounter an error about not being able to connect to database.

harem234 commented on Jan 9, 2020

edited

in docker-compose.yml
patch /file.sql:/patch/to/docker-entrypoint-initdb.d/
in fact, point the file to the directory
hint: remember where the work dir is base on your docker file for the image you are using to compose this container
volumes:

  • ./init.sql:/docker-entrypoint-initdb.d/

did anyone get this to work properly. i am currently facing similar issues

it is easier to make a docker file and COPY the init.sql to /docker-entrypoint-initdb.d

did anyone get this to work properly. i am currently facing similar issues

well what i did which worked was that i deleted all previous containers of the same image.
docker rm -f -v postgres_container_name.
restarted and it worked like a charm after that

What is the best way to add tables from sql files after the first init?

at the moment i have a docker compose file with mariadb und phpmyadmin
docker-entrypoint-initdb.d works the first time

but i need the possibilty to creates tables from sql files after docker-compose up
can you help me?

Author

onjin commented on Apr 16, 2020

after init, I'm using a separate file with database migrations (using app specific db migrations system) and run them by application entrypoint.sh, from time to time then I merge migrations to init, and clear migrations file.

I had some trouble getting it to work. My docker-compose.yml looks like this:

  volumes:
    - ./:/app

services:
  db:
    image: postgres:12
    restart: unless-stopped
    volumes:
      - ./project/data/init.sql:/docker-entrypoint-initdb.d/init.sql
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=picker
      - POSTGRES_PASSWORD=#
      - POSTGRES_DB=pricepicker-v2

volumes:
  postgres_data:

This is because Postgres does not execute init scripts if postgres-data folder is present. So I just renamed it to postgres and it works like a charm!

Thanks @Juliannnnshipit for your help. I got it working.

My setup.sh file looks something like this:

#!/usr/bin/env bash
psql "postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@$POSTGRES_HOST/$POSTGRES_DB?sslmode=disable" <<-EOSQL
-- YOUR SQL Statements here
EOSQL

docker-compose.yml

postgresdb:
    image: postgres:9.4
    container_name: postgresdb
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASS}
      - POSTGRES_DB=${DB_NAME}
      - DATABASE_HOST=${DB_HOST}
    ports:
      - 5433:5432
    volumes:
      - ./contrib/setup.sh:/docker-entrypoint-initdb.d/initdb.sh
      - ./pgdata:/var/lib/postgresql/data

Thank you so much. It saved my time


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK