Improve pg_dump/pg_restore/pg_upgrade handling of large objects

4713

Title

CI (CFBot)

Not processed

Stats (from CFBot)

Unknown

Topic

Performance

Created

2023-12-20 23:49:21

Last modified

2024-04-01 20:46:58 (10 months, 2 weeks ago)

Latest email

2024-04-01 19:37:18 (10 months, 2 weeks ago)

Status

2024-03: Committed

2024-01: Moved to next CF

Target version

Authors

Tom Lane (tgl)

Reviewers

Nathan Bossart (bossartn)Become reviewer

Committer

Tom Lane (tgl)

Links

Emails

Re: pg_upgrade failing for 200+ million Large Objects: First at 2021-03-08 11:02:04 by "Tharakan, Robins" <tharar at amazon.com>
Latest at 2024-04-01 19:37:18 by Nathan Bossart <nathandbossart at gmail.com>
Latest attachment (v11-0001-Some-small-preliminaries-for-pg_dump-changes.patch) at 2024-03-15 23:18:41 from Tom Lane <tgl at sss.pgh.pa.us>
    Attachment (v11-0001-Some-small-preliminaries-for-pg_dump-changes.patch) at 2024-03-15 23:18:41 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (v10-0001-Some-small-preliminaries-for-pg_dump-changes.patch) at 2024-01-26 16:44:26 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (v9-005-parallel_pg_restore.patch) at 2024-01-02 17:33:00 from "Kumar, Sachin" <ssetiya at amazon.com> (Patch: Yes)
    Attachment (v9_1M_100kb_restore_times.png) at 2023-12-28 11:38:46 from Robins Tharakan <tharakan at gmail.com> (Patch: No)
    Attachment (v9_dump.png) at 2023-12-27 13:28:01 from Robins Tharakan <tharakan at gmail.com> (Patch: No)
    Attachment (v9-0001-Some-small-preliminaries-for-pg_dump-changes.patch) at 2023-12-20 23:47:44 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (v8-0001-restore-transaction-size-option.patch) at 2023-12-11 01:42:42 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (pg_upgrade_improvements_v7.diff) at 2023-12-04 16:07:59 from "Kumar, Sachin" <ssetiya at amazon.com> (Patch: Yes)
    Attachment (pg_upgrade_improvements_v6.diff) at 2023-11-13 15:06:31 from "Kumar, Sachin" <ssetiya at amazon.com> (Patch: Yes)
    Attachment (pg_upgrade_improvements_v5.diff) at 2022-08-25 00:32:27 from Nathan Bossart <nathandbossart at gmail.com> (Patch: Yes)
    Attachment (pg_upgrade_improvements.v4.diff) at 2021-03-24 16:05:27 from Jan Wieck <jan at wi3ck.info> (Patch: Yes)
    Attachment (pg_upgrade_improvements.v3.diff) at 2021-03-23 12:51:32 from Jan Wieck <jan at wi3ck.info> (Patch: Yes)
    Attachment (pg_upgrade_improvements.v2.diff) at 2021-03-22 18:07:34 from Jan Wieck <jan at wi3ck.info> (Patch: Yes)
    Attachment (upgrade-test-dbownership.patch) at 2021-03-21 19:29:10 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (pg_restore-preserve-datdba.v1.diff) at 2021-03-21 17:50:45 from Jan Wieck <jan at wi3ck.info> (Patch: Yes)
    Attachment (pg_upgrade-preserve-datdba.v1.diff) at 2021-03-21 16:50:46 from Jan Wieck <jan at wi3ck.info> (Patch: Yes)

History

When	Who	What
2024-04-01 20:46:58	Tom Lane (tgl)	Closed in commitfest 2024-03 with status: Committed
2024-04-01 20:46:58	Tom Lane (tgl)	Changed committer to tgl
2024-02-01 17:55:14	vigneshwaran C (vignesh.postgres)	Closed in commitfest 2024-01 with status: Moved to next CF
2023-12-21 03:16:45	Nathan Bossart (bossartn)	Added bossartn as reviewer
2023-12-20 23:49:37	Tom Lane (tgl)	Changed authors to Tom Lane (tgl)
2023-12-20 23:49:21	Tom Lane (tgl)	Attached mail thread a9f9376f1c3343a6bb319dce294e20ac@EX13D05UWC001.ant.amazon.com
2023-12-20 23:49:21	Tom Lane (tgl)	Created patch record