Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing locale script for postgres:15.3-alpine3.18 #1082

Closed
belgattitude opened this issue May 14, 2023 · 6 comments
Closed

Missing locale script for postgres:15.3-alpine3.18 #1082

belgattitude opened this issue May 14, 2023 · 6 comments

Comments

@belgattitude
Copy link

I'd like to initialize locale with ICU support and alpine latest.

Not sure if it's a problem, but there's a warning about missing locale script

image

Docker compose example:

version: '3'

services:
  postgres:
    image: postgres:15.3-alpine3.18
    environment:
      POSTGRES_USER: strapi
      POSTGRES_PASSWORD: strapi
      POSTGRES_DB: strapi_test
      POSTGRES_INITDB_ARGS: '--locale-provider=icu --icu-locale=en-US --no-sync'
    restart: always
    ports:
      - '5432:5432'
    volumes:
      - pgdata_test:/var/lib/postgresql/data:rw
    healthcheck:
      test: ["CMD-SHELL", "sh -c 'pg_isready -U strapi -d strapi_test'"]
      interval: 2s
      timeout: 5s
      retries: 5
volumes:
  pgdata_test:
Full log 
strapi-postgres-1  | The files belonging to this database system will be owned by user "postgres".
strapi-postgres-1  | This user must also own the server process.
strapi-postgres-1  | 
strapi-postgres-1  | The database cluster will be initialized with this locale configuration:
strapi-postgres-1  |   provider:    icu
strapi-postgres-1  |   ICU locale:  en-US
strapi-postgres-1  |   LC_COLLATE:  en_US.utf8
strapi-postgres-1  |   LC_CTYPE:    en_US.utf8
strapi-postgres-1  |   LC_MESSAGES: en_US.utf8
strapi-postgres-1  |   LC_MONETARY: en_US.utf8
strapi-postgres-1  |   LC_NUMERIC:  en_US.utf8
strapi-postgres-1  |   LC_TIME:     en_US.utf8
strapi-postgres-1  | The default database encoding has been set to "UTF8".
strapi-postgres-1  | The default text search configuration will be set to "english".
strapi-postgres-1  | 
strapi-postgres-1  | Data page checksums are disabled.
strapi-postgres-1  | 
strapi-postgres-1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
strapi-postgres-1  | creating subdirectories ... ok
strapi-postgres-1  | selecting dynamic shared memory implementation ... posix
strapi-postgres-1  | selecting default max_connections ... 100
strapi-postgres-1  | selecting default shared_buffers ... 128MB
strapi-postgres-1  | selecting default time zone ... UTC
strapi-postgres-1  | creating configuration files ... ok
strapi-postgres-1  | running bootstrap script ... ok
strapi-postgres-1  | sh: locale: not found
strapi-postgres-1  | 2023-05-14 11:03:30.936 UTC [30] WARNING:  no usable system locales were found
strapi-postgres-1  | initdb: warning: enabling "trust" authentication for local connections
strapi-postgres-1  | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
strapi-postgres-1  | performing post-bootstrap initialization ... ok
strapi-postgres-1  | 
strapi-postgres-1  | Sync to disk skipped.
strapi-postgres-1  | The data directory might become corrupt if the operating system crashes.
strapi-postgres-1  | 
strapi-postgres-1  | 
strapi-postgres-1  | Success. You can now start the database server using:
strapi-postgres-1  | 
strapi-postgres-1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
strapi-postgres-1  | 
strapi-postgres-1  | waiting for server to start....2023-05-14 11:03:31.264 UTC [36] LOG:  starting PostgreSQL 15.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit
strapi-postgres-1  | 2023-05-14 11:03:31.265 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
strapi-postgres-1  | 2023-05-14 11:03:31.268 UTC [39] LOG:  database system was shut down at 2023-05-14 11:03:31 UTC
strapi-postgres-1  | 2023-05-14 11:03:31.270 UTC [36] LOG:  database system is ready to accept connections
strapi-postgres-1  |  done
strapi-postgres-1  | server started
strapi-postgres-1  | CREATE DATABASE
strapi-postgres-1  | 
strapi-postgres-1  | 
strapi-postgres-1  | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
strapi-postgres-1  | 
strapi-postgres-1  | waiting for server to shut down....2023-05-14 11:03:31.427 UTC [36] LOG:  received fast shutdown request
strapi-postgres-1  | 2023-05-14 11:03:31.428 UTC [36] LOG:  aborting any active transactions
strapi-postgres-1  | 2023-05-14 11:03:31.429 UTC [36] LOG:  background worker "logical replication launcher" (PID 42) exited with exit code 1
strapi-postgres-1  | 2023-05-14 11:03:31.429 UTC [37] LOG:  shutting down
strapi-postgres-1  | 2023-05-14 11:03:31.430 UTC [37] LOG:  checkpoint starting: shutdown immediate
strapi-postgres-1  | 2023-05-14 11:03:31.453 UTC [37] LOG:  checkpoint complete: wrote 918 buffers (5.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.010 s, sync=0.012 s, total=0.025 s; sync files=250, longest=0.001 s, average=0.001 s; distance=4222 kB, estimate=4222 kB
strapi-postgres-1  | 2023-05-14 11:03:31.458 UTC [36] LOG:  database system is shut down
strapi-postgres-1  |  done
strapi-postgres-1  | server stopped
strapi-postgres-1  | 
strapi-postgres-1  | PostgreSQL init process complete; ready for start up.
strapi-postgres-1  | 
strapi-postgres-1  | 2023-05-14 11:03:31.583 UTC [1] LOG:  starting PostgreSQL 15.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit
strapi-postgres-1  | 2023-05-14 11:03:31.583 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
strapi-postgres-1  | 2023-05-14 11:03:31.583 UTC [1] LOG:  listening on IPv6 address "::", port 5432
strapi-postgres-1  | 2023-05-14 11:03:31.585 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
strapi-postgres-1  | 2023-05-14 11:03:31.589 UTC [52] LOG:  database system was shut down at 2023-05-14 11:03:31 UTC
strapi-postgres-1  | 2023-05-14 11:03:31.592 UTC [1] LOG:  database system is ready to accept connections
strapi-postgres-1  | 2023-05-14 11:08:31.686 UTC [50] LOG:  checkpoint starting: time
strapi-postgres-1  | 2023-05-14 11:08:35.862 UTC [50] LOG:  checkpoint complete: wrote 44 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.119 s, sync=0.027 s, total=4.177 s; sync files=12, longest=0.024 s, average=0.003 s; distance=252 kB, estimate=252 kB
@tianon
Copy link
Member

tianon commented May 15, 2023

I don't think the Alpine images have ever included locale -- the end result appears to work successfully in spite of this?

$ docker run -it --rm --env POSTGRES_INITDB_ARGS='--locale-provider=icu --icu-locale=de-DE' --env LANG='de_DE.utf8' --env POSTGRES_PASSWORD=example postgres:15-alpine3.18
Unable to find image 'postgres:15-alpine3.18' locally
15-alpine3.18: Pulling from library/postgres
8a49fdb3b6a5: Already exists 
397880efeff2: Pull complete 
7981ad2d616c: Pull complete 
521df6ab8516: Pull complete 
46ac16d4613e: Pull complete 
3c0b7c938b8a: Pull complete 
230669f5a759: Pull complete 
4dbd9486f09b: Pull complete 
Digest: sha256:f632b07cad142a12360e5c4c084a3cf62fc95f3ee0be285647b4d3744840cade
Status: Downloaded newer image for postgres:15-alpine3.18
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with this locale configuration:
  provider:    icu
  ICU locale:  de-DE
  LC_COLLATE:  de_DE.utf8
  LC_CTYPE:    de_DE.utf8
  LC_MESSAGES: de_DE.utf8
  LC_MONETARY: de_DE.utf8
  LC_NUMERIC:  de_DE.utf8
  LC_TIME:     de_DE.utf8
The default database encoding has been set to "UTF8".
The default text search configuration will be set to "german".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... sh: locale: not found
2023-05-15 18:27:04.821 UTC [30] WARNING:  no usable system locales were found
ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....2023-05-15 18:27:05.323 UTC [36] LOG:  starting PostgreSQL 15.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit
2023-05-15 18:27:05.325 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-05-15 18:27:05.331 UTC [39] LOG:  database system was shut down at 2023-05-15 18:27:05 UTC
2023-05-15 18:27:05.335 UTC [36] LOG:  database system is ready to accept connections
 done
server started

/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

waiting for server to shut down....2023-05-15 18:27:05.448 UTC [36] LOG:  received fast shutdown request
2023-05-15 18:27:05.454 UTC [36] LOG:  aborting any active transactions
2023-05-15 18:27:05.455 UTC [36] LOG:  background worker "logical replication launcher" (PID 42) exited with exit code 1
2023-05-15 18:27:05.455 UTC [37] LOG:  shutting down
2023-05-15 18:27:05.457 UTC [37] LOG:  checkpoint starting: shutdown immediate
2023-05-15 18:27:05.468 UTC [37] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.004 s, sync=0.002 s, total=0.014 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
2023-05-15 18:27:05.471 UTC [36] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2023-05-15 18:27:05.575 UTC [1] LOG:  starting PostgreSQL 15.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit
2023-05-15 18:27:05.575 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-05-15 18:27:05.575 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-05-15 18:27:05.578 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-05-15 18:27:05.583 UTC [50] LOG:  database system was shut down at 2023-05-15 18:27:05 UTC
2023-05-15 18:27:05.587 UTC [1] LOG:  database system is ready to accept connections

And then in another terminal:

$ docker exec -it vigorous_albattani psql -U postgres
psql (15.3)
Type "help" for help.

postgres=# \l
                                                List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    | ICU Locale | Locale Provider |   Access privileges   
-----------+----------+----------+------------+------------+------------+-----------------+-----------------------
 postgres  | postgres | UTF8     | de_DE.utf8 | de_DE.utf8 | de-DE      | icu             | 
 template0 | postgres | UTF8     | de_DE.utf8 | de_DE.utf8 | de-DE      | icu             | =c/postgres          +
           |          |          |            |            |            |                 | postgres=CTc/postgres
 template1 | postgres | UTF8     | de_DE.utf8 | de_DE.utf8 | de-DE      | icu             | =c/postgres          +
           |          |          |            |            |            |                 | postgres=CTc/postgres
(3 rows)

@belgattitude
Copy link
Author

Can't say for sure, but I can confirm on the postgres side it seems to work properly.

@belgattitude
Copy link
Author

Link to gliderlabs/docker-alpine#144

@tianon
Copy link
Member

tianon commented Dec 18, 2023

Closing since this appears to be working correctly / as intended.

@tianon tianon closed this as completed Dec 18, 2023
@oerp-odoo
Copy link

oerp-odoo commented Oct 29, 2024 •
edited
Loading

Maybe I am missing something, but it looks like sorting is still done using default "C" locale even if I set custom one.

I set locale like this:

    environment:
      POSTGRES_INITDB_ARGS: "--locale-provider=icu --icu-locale=en-AU"
      LANG: en_AU.utf-8
      LC_COLLATE: en_AU.utf-8
      LC_CTYPE: en_AU.utf-8
      LC_ALL: en_AU.utf-8

Then in logs I see same thing as mentioned above:

The database cluster will be initialized with this locale configuration:
   provider:    icu
   ICU locale:  en-AU
   LC_COLLATE:  en_AU.utf-8
   LC_CTYPE:    en_AU.utf-8
   LC_MESSAGES: en_AU.utf-8
   LC_MONETARY: en_AU.utf-8
   LC_NUMERIC:  en_AU.utf-8
   LC_TIME:     en_AU.utf-8
 The default database encoding has been set to "UTF8".
 The default text search configuration will be set to "english".
 
 Data page checksums are disabled.
 
 fixing permissions on existing directory /var/lib/postgresql/data ... ok
 creating subdirectories ... ok
 selecting dynamic shared memory implementation ... posix
 selecting default max_connections ... 100
 selecting default shared_buffers ... 128MB
 selecting default time zone ... UTC
 creating configuration files ... ok
 running bootstrap script ... ok
 performing post-bootstrap initialization ... sh: locale: not found
 2024-10-29 06:58:55.692 UTC [30] WARNING:  no usable system locales were found

If I check datcollate, I see it is showing as set:

odoodb=# select datname, datcollate from pg_database;
  datname  | datcollate  
-----------+-------------
 postgres  | en_AU.utf-8
 odoodb    | en_AU.utf-8
 template1 | en_AU.utf-8
 template0 | en_AU.utf-8

But sorting still works the same as using C:

odoodb=# select default_code from product_template
order by default_code;

I see this:

default_code 
--------------
 A-HS
 AA-A
 E-COM06
 E-COM07
 E-COM08

If Australian locale would be used, it should have AA-A before A-HS as it should preffer letters first instead of special symbols, right?

Also if I try to explicitly use collate in sorting, I get error saying there is no such collate:

odoodb=# select default_code from product_template
order by default_code collate "en_AU.utf-8";
ERROR:  collation "en_AU.utf-8" for encoding "UTF8" does not exist
LINE 2: order by default_code collate "en_AU.utf-8";

or

odoodb=# select default_code from product_template
order by default_code collate "en_AU";
ERROR:  collation "en_AU" for encoding "UTF8" does not exist
LINE 2: order by default_code collate "en_AU";

Though it works if I explicitly specify "C"..

To me it looks that 2024-10-29 06:58:55.692 UTC [30] WARNING: no usable system locales were found means that no locale is actually set, even if it is showing as set?..

@tianon
Copy link
Member

tianon commented Oct 29, 2024

See https://wiki.musl-libc.org/open-issues#Locale-limitations (it "works" but I don't know that it actually functions correctly, as you've experienced, and certainly isn't actually an officially supported platform by PostgreSQL upstream, to my knowledge)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tianon @belgattitude @oerp-odoo