Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pgloader hangs on copying rows when migrating from maria db. #1599

Open
3 tasks
coreylarew opened this issue Jul 24, 2024 · 3 comments
Open
3 tasks

Pgloader hangs on copying rows when migrating from maria db. #1599

coreylarew opened this issue Jul 24, 2024 · 3 comments

Comments

@coreylarew
Copy link

I am currently trying to migrate my Apache Guacamole DB, which currently resides in MariaDB, to PostgresSQL. The schema and all tables are created. However when it starts the copy of rows process it just hangs there.

I deploy postgres, mariadb, and pgloader in kubernetes.

PGLOADER Deployment yml:

apiVersion: batch/v1
kind: Job
metadata:
name: pgloader
namespace: paas-admin
spec:
template:
metadata:
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: pgloader
image: ghrc.io/dimitri/pgloader
command: ["pgloader", "--verbose", "--debug", "/tmp/myload/my.load"]
#command: ["/bin/bash", "-c", "--"]
# args: [ "while true; do sleep 30; done;"]
volumeMounts:
- name: config-volume
mountPath: /tmp/myload
volumes:
- name: config-volume
configMap:
name: myload
items:
- key: my.load
path: my.load
restartPolicy: Never
backoffLimit: 4

PGLOADER Load File

load database
from mysql://user:pass@mariadb-svc:3306/guacamole_db
into postgresql://user:pass@paas-pg-pgpool:5432/guacamole_db

WITH workers = 8, concurrency = 1, create no indexes, multiple readers per thread, rows per range = 10000

CAST type int when (= 11 precision) to "integer" drop typemod

ALTER SCHEMA 'guacamole_db' RENAME TO 'public';

Log Where Process Hangs

024-07-24T14:18:05.569471023Z 2024-07-24T14:18:05.384998Z DEBUG SET application_name TO 'pgloader'
2024-07-24T14:18:05.395998Z SQL SET search_path TO public;
2024-07-24T14:18:05.580995116Z 2024-07-24T14:18:05.396998Z INFO pgsql:copy-rows-from-queue[7]: public.guacamole_connection (connection_id
2024-07-24T14:18:05.581022490Z connection_name
2024-07-24T14:18:05.581027103Z parent_id protocol
2024-07-24T14:18:05.581029974Z proxy_port
proxy_hostname
2024-07-24T14:18:05.581036006Z proxy_encryption_method
2024-07-24T14:18:05.581038819Z max_connections
2024-07-24T14:18:05.581042002Z max_connections_per_user
2024-07-24T14:18:05.581046202Z connection_weight
failover_only)

  • pgloader --version

    pgloader version 3.6.10~devel
    
  • did you test a fresh compile from the source tree?

    No

  • did you search for other similar issues?
    Yes I did not find any Similar

@coreylarew
Copy link
Author

full log found here

full_log.txt

@zentavr
Copy link

zentavr commented Aug 23, 2024

I might think that this is because of scram-sha-256 which the tool not supports.

@zentavr
Copy link

zentavr commented Aug 23, 2024

...dealing with the same issue BTW

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants