Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid path in mkdirs.sh #141

Open
ksiddi01 opened this issue Jul 23, 2020 · 3 comments
Open

Invalid path in mkdirs.sh #141

ksiddi01 opened this issue Jul 23, 2020 · 3 comments
Assignees

Comments

@ksiddi01
Copy link

pg_rman restore is failing due to incomplete path in mkdirs.sh file.

Invalid paths:
mkdir -m 700 -p ts
mkdir -m 700 -p ts/PG_10_201707211
mkdir -m 700 -p ts/PG_10_201707211/16395
mkdir -m 700 -p ts/PG_11_201809051
mkdir -m 700 -p ts/PG_11_201809051/16402

Corrected paths:
mkdir -m 700 -p /pgsql/data/ilmrep1p_ts
mkdir -m 700 -p /pgsql/data/ilmrep1p_ts/PG_10_201707211
mkdir -m 700 -p /pgsql/data/ilmrep1p_ts/PG_10_201707211/16395
mkdir -m 700 -p /pgsql/data/ilmrep1p_ts/PG_11_201809051
mkdir -m 700 -p /pgsql/data/ilmrep1p_ts/PG_11_201809051/16402

After mkdirs.sh is modified, restore works properly. Is it because of "_" in folder name (ilmrep1p_ts) ?

mkdirs.sh.txt

@MoonInsung
Copy link
Contributor

Dear @ksiddi01

Could you share the method of reproduction?
It seems that the issue is not related to the file name.

Best regards.
Moon.

@mikecaat
Copy link
Contributor

Hi,

We could reproduce the above situation.

The root cause seems to be that the logic to check whether the tablespace or $PGDATA is wrong.
The problem occurs if the prefix of tablespace path has $PGDATA path.

@ksiddi01
I assume that the conditions of your environments are like the following, right?

  • $PGDATA: /pgsql/data/ilmrep1p
  • tablespace path: /pgsql/data/ilmrep1p_ts

The following is the reproduction way.

# reproduced environments
> pg_rman --version
pg_rman 1.3.13

> psql -c "SELECT version();" 
                                               version
------------------------------------------------------------------------------------------------------
 PostgreSQL 14.0 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit
(1 row)

# create PostgreSQL server
> vim tmp_env
export BACKUP_PATH=/tmp/pg_rman_catalog
export PGDATA=/tmp/pgdata
> source tmp_env
> initdb;
> echo "archive_mode = on 
archive_command = 'cp %p /tmp/data.arch/%f'                                                                                                                                                 
restore_command = 'cp /tmp/data.arch/%f %p'                                                                                                                                                 
" >> ${PGDATA}/postgresql.conf
> mkdir /tmp/data.arch
> pg_ctl start

# create table space
> vim create_tablespace.sql
-- the prefix of the location directory is the *same* of PGDATA=/tmp/pgdata
CREATE TABLESPACE tblspc LOCATION '/tmp/pgdata_tblspc';
CREATE DATABASE test_db TABLESPACE tblspc;
> mkdir /tmp/pgdata_tblspc      # the path is "${PGDATA}_tblspc"
> psql -f create_tablespace.sql

> pg_rman init; pg_rman backup -b full; pg_rman validate 
> pg_ctl stop
> pg_rman restore
ERROR: could not open restore target file "/tmp/pgdata/pg_tblspc/16384/PG_14_202107181/16385/112": そのようなファイルやディレクトリはありません

> cat ${BACKUP_PATH}/*/*/mkdirs.sh
mkdir -m 700 -p base
mkdir -m 700 -p base/1
mkdir -m 700 -p base/13009
mkdir -m 700 -p base/13010
mkdir -m 700 -p global
mkdir -m 700 -p pg_commit_ts
mkdir -m 700 -p pg_dynshmem
mkdir -m 700 -p pg_logical
mkdir -m 700 -p pg_logical/mappings
mkdir -m 700 -p pg_logical/snapshots
mkdir -m 700 -p pg_multixact
mkdir -m 700 -p pg_multixact/members
mkdir -m 700 -p pg_multixact/offsets
mkdir -m 700 -p pg_notify
mkdir -m 700 -p pg_replslot
mkdir -m 700 -p pg_serial
mkdir -m 700 -p pg_snapshots
mkdir -m 700 -p pg_stat
mkdir -m 700 -p pg_stat_tmp
mkdir -m 700 -p pg_subtrans
mkdir -m 700 -p pg_tblspc
mkdir -m 700 -p pg_twophase
mkdir -m 700 -p pg_wal
mkdir -m 700 -p pg_wal/archive_status
mkdir -m 700 -p pg_xact
mkdir -m 700 -p tblspc
mkdir -m 700 -p tblspc/PG_14_202107181
mkdir -m 700 -p tblspc/PG_14_202107181/16385

rm -f pg_tblspc/16384
ln -s /tmp/pgdata_tblspc pg_tblspc/16384

Regards,

mikecaat added a commit to mikecaat/pg_rman that referenced this issue Oct 21, 2021
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
mikecaat added a commit to mikecaat/pg_rman that referenced this issue Oct 21, 2021
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
@huangfumingyue
Copy link
Contributor

Thanks.
It looks good for me

mikecaat added a commit to mikecaat/pg_rman that referenced this issue Sep 2, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue pushed a commit that referenced this issue Sep 5, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue pushed a commit that referenced this issue Sep 5, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue pushed a commit that referenced this issue Sep 5, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue pushed a commit that referenced this issue Sep 5, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue pushed a commit to huangfumingyue/pg_rman that referenced this issue Sep 8, 2022
Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.
huangfumingyue added a commit that referenced this issue Sep 8, 2022
…L_10_STABLE (#233)

* Fix the fail if a tablespace path has PGDATA(#141) (#210)

Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.

* add start function

Co-authored-by: mikecaat <[email protected]>
Co-authored-by: huangfumingyue <[email protected]>
huangfumingyue added a commit that referenced this issue Sep 8, 2022
…L_10_STABLE (#233)

* Fix the fail if a tablespace path has PGDATA(#141) (#210)

Previously, if the server has a tablespace and
its path has PGDATA's path, the restore
fails because mkdirs.sh doesn't work.

For example, the following case fails to restore.

* PGDATA path: /tmp/pgdata
* tablespace path: /tmp/pgdata_tblspc

So, this patch fixes the case. The root cause
is the logic checking whether the file taking backup
is in tablespace or PGDATA.

Because it only considers their prefix, if the prefix
path of tablespace is the same as PGDATA, it decides
the file is in PGDATA.

So, this patch changes the logic to check it has '/'
after the prefix.

* add start function

Co-authored-by: mikecaat <[email protected]>
Co-authored-by: huangfumingyue <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants