custom compression methods for toast

Edit

Title

Topic

Server Features

Created

2020-10-30 11:03:57

Last modified

2021-04-08 15:30:47 (3 years, 4 months ago)

Latest email

2021-03-30 14:30:32 (3 years, 5 months ago)

Status

2021-03: Committed

2021-01: Moved to next CF

2020-11: Moved to next CF

Target version

Authors

Dilip Kumar (dilip.kumar), Ildus Kurbangaliev (ildus)

Reviewers

Justin Pryzby (justinpryzby)Become reviewer

Committer

Robert Haas (rhaas)

Links

Emails

Custom compression methods: First at 2017-09-07 16:42:36 by Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru>
Latest at 2021-01-19 09:53:55 by Dilip Kumar <dilipbalaut at gmail.com>
Latest attachment (v19-0002-alter-table-set-compression.patch) at 2021-01-19 09:53:55 from Dilip Kumar <dilipbalaut at gmail.com>
    Attachment (v19-0002-alter-table-set-compression.patch) at 2021-01-19 09:53:55 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v18-0002-alter-table-set-compression.patch) at 2021-01-13 08:44:01 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v17-0003-Add-support-for-PRESERVE.patch) at 2021-01-04 11:27:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v17-0007-Add-Lz4-compression-to-WAL-FPIs.patch) at 2020-12-29 17:48:49 from Andrey Borodin <x4mmm at yandex-team.ru> (Patch: Yes)
    Attachment (v16-0007-Add-Lz4-compression-to-WAL-FPIs.patch) at 2020-12-27 07:10:34 from Andrey Borodin <x4mmm at yandex-team.ru> (Patch: Yes)
    Attachment (v16-0002-alter-table-set-compression.patch) at 2020-12-25 09:34:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v15-0002-alter-table-set-compression.patch) at 2020-12-09 12:07:27 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v14-0005-new-compression-method-extension-for-zlib.patch) at 2020-11-11 14:39:37 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v13-0003-Add-support-for-PRESERVE.patch) at 2020-11-08 10:59:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v12-0002-alter-table-set-compression.patch) at 2020-10-29 06:37:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v11-0004-Create-custom-compression-methods.patch) at 2020-10-28 07:46:31 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v10-0002-alter-table-set-compression.patch) at 2020-10-27 05:24:32 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v9-0002-alter-table-set-compression.patch) at 2020-10-22 12:26:55 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v8-0002-alter-table-set-compression.patch) at 2020-10-21 08:29:50 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v7-0002-alter-table-set-compression.patch) at 2020-10-17 06:04:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v6-0001-Built-in-compression-method.patch) at 2020-10-12 08:58:43 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v5-0002-alter-table-set-compression.patch) at 2020-10-08 14:24:42 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v4-0002-alter-table-set-compression.patch) at 2020-10-07 11:30:12 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v4-0001-Built-in-compression-method.patch) at 2020-10-05 07:48:14 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v3-0001-Built-in-compression-method.patch) at 2020-10-04 10:31:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v3-0002-alter-table-set-compression.patch) at 2020-09-28 10:48:07 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0002-alter-table-set-compression.patch) at 2020-09-19 07:49:48 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v1-0001-Add-support-for-setting-the-compression-method.patch) at 2020-08-13 11:48:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (custom_compression_methods_v23.patch) at 2019-07-08 12:00:21 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (jsonbd-master-fix.patch) at 2019-03-21 19:59:56 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v22.patch) at 2019-03-15 11:52:03 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v21.patch) at 2019-02-28 15:44:15 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v20.patch) at 2018-12-03 12:43:32 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v20.patch) at 2018-11-01 11:54:37 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v19.patch) at 2018-09-06 15:27:13 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v17.patch) at 2018-07-02 12:56:24 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v16.patch) at 2018-06-18 14:30:45 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v15.patch) at 2018-03-30 16:50:45 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v14.patch) at 2018-03-26 17:38:25 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v14.patch) at 2018-03-14 13:31:23 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v13.patch) at 2018-02-26 12:25:56 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v12.patch) at 2018-02-06 10:47:31 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v11.patch) at 2018-01-29 11:44:52 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v10.patch) at 2018-01-25 14:24:57 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v9.patch) at 2018-01-23 13:04:54 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v8.patch) at 2018-01-14 21:49:30 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (Screen Shot 2017-12-03 at 11.46.14.png) at 2017-12-03 02:52:13 from Oleg Bartunov <obartunov at gmail.com> (Patch: No)
    Attachment (data-sorted-exceeds-ram.png) at 2017-11-29 23:30:37 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v7.patch) at 2017-11-28 13:29:18 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v6.patch) at 2017-11-23 09:38:49 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (pg_lz4.tgz) at 2017-11-21 20:42:37 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v5.patch) at 2017-11-21 14:47:17 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (pg_lz4.tgz) at 2017-11-19 23:04:53 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v4.patch) at 2017-11-14 13:23:56 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v3.patch) at 2017-11-02 12:28:36 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v2.patch) at 2017-09-12 14:55:05 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v1.patch) at 2017-09-07 16:42:36 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
Re: [HACKERS] Custom compression methods: First at 2021-01-19 19:07:20 by Justin Pryzby <pryzby at telsasoft.com>
Latest at 2021-03-30 14:30:32 by Tom Lane <tgl at sss.pgh.pa.us>
Latest attachment (v4-0001-Show-compression-method-in-index-describe.patch) at 2021-03-25 09:44:36 from Dilip Kumar <dilipbalaut at gmail.com>
    Attachment (v4-0001-Show-compression-method-in-index-describe.patch) at 2021-03-25 09:44:36 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v3-0003-ALTER-SET-COMPRESSION-for-index-columns.patch) at 2021-03-24 11:45:11 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Show-compression-method-in-index-describe.patch) at 2021-03-24 10:10:58 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v1-0001-Fix-attcompression-for-index-expression-columns.patch) at 2021-03-24 08:54:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (toast_failedassertion.txt) at 2021-03-24 07:52:00 from Jaime Casanova <jcasanov at systemguards.com.ec> (Patch: No)
    Attachment (v3-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patch) at 2021-03-22 20:57:18 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patch) at 2021-03-22 20:33:55 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (toast-compression-guc-rmh.patch) at 2021-03-22 17:38:36 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v1-0001-fix-slice-decompression.patch) at 2021-03-22 15:27:59 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v3-0001-Add-docs-for-default_toast_compression.patch) at 2021-03-21 23:55:45 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (fix-partial-lz4-decompression.patch) at 2021-03-21 23:52:09 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (v1-0001-fix-compression-test.patch) at 2021-03-21 05:22:14 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-fix-compression-test.patch) at 2021-03-21 03:40:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Add-docs-for-default_toast_compression.patch) at 2021-03-20 23:20:39 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (0001-Use-valid-compression-method-in-brin_form_tuple.patch) at 2021-03-20 15:13:47 from Tomas Vondra <tomas.vondra at enterprisedb.com> (Patch: Yes)
    Attachment (0001-Fixup-dump-toast-compression-method.patch) at 2021-03-20 07:52:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Add-docs-for-default_toast_compression.patch) at 2021-03-20 07:44:20 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (comments.patch) at 2021-03-19 22:38:43 from Alvaro Herrera <alvherre at alvh.no-ip.org> (Patch: Yes)
    Attachment (redo-lz4-configuration.patch) at 2021-03-19 22:13:59 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (non-default-toast-compression-only-v1.patch) at 2021-03-19 21:49:37 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patch) at 2021-03-19 14:11:03 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patch) at 2021-03-18 19:57:21 from Robert Haas <robertmhaas at gmail.com> (Patch: No)
    Attachment (v38-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-18 14:22:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v37-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-18 11:10:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v36-0002-Expand-the-external-data-before-forming-the-tupl.patch) at 2021-03-17 11:40:44 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (test1.sql) at 2021-03-16 10:37:22 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-15 12:14:33 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patch) at 2021-03-12 05:15:06 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patch) at 2021-03-11 18:25:26 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (v34-0002-Expand-the-external-data-before-forming-the-tupl.patch) at 2021-03-11 10:51:38 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Built-in-compression-method.patch) at 2021-03-11 02:28:58 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v33-0004-default-to-with-lz4.patch) at 2021-03-10 11:51:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v32-0001-Built-in-compression-method.patch) at 2021-03-08 10:02:39 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v31-0001-Built-in-compression-method.patch) at 2021-03-06 15:29:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v30-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-04 13:43:13 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v29-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-03-01 15:23:09 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v28-0004-default-to-with-lz4.patch) at 2021-02-27 09:14:54 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v27-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-26 14:40:29 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v26-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-20 10:46:48 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v25-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-19 16:12:29 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-18 21:13:14 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (v1-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-16 14:20:35 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v24-0001-Built-in-compression-method.patch) at 2021-02-14 20:28:31 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v24-0001-Built-in-compression-method.patch) at 2021-02-14 18:49:40 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (POC_fix_compression_in_rowtype.patch) at 2021-02-11 12:36:28 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (fixups.patch) at 2021-02-10 21:56:17 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v23-0003-Add-support-for-PRESERVE.patch) at 2021-02-10 14:51:59 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v22-0001-Built-in-compression-method.patch) at 2021-02-07 11:45:42 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v21-0001-Built-in-compression-method.patch) at 2021-02-05 14:41:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v21-0001-Built-in-compression-method.patch) at 2021-01-30 07:16:39 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v20-0002-alter-table-set-compression.patch) at 2021-01-29 12:37:24 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)

History

When	Who	What
2021-04-08 15:30:47	David Steele (dsteele)	Changed targetversion to 14
2021-04-08 15:30:40	David Steele (dsteele)	Closed in commitfest 2021-03 with status: Committed
2021-04-08 15:28:46	David Steele (dsteele)	Changed targetversion to None
2021-03-20 20:59:49	Justin Pryzby (justinpryzby)	New status: Ready for Committer
2021-03-20 20:59:05	Justin Pryzby (justinpryzby)	Closed in commitfest 2021-03 with status: Committed
2021-03-20 20:59:05	Justin Pryzby (justinpryzby)	Changed committer to rhaas
2021-03-06 20:14:50	Justin Pryzby (justinpryzby)	Changed targetversion to 14
2021-02-15 15:10:35	Justin Pryzby (justinpryzby)	Attached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-15 15:10:15	Justin Pryzby (justinpryzby)	Detached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-09 18:49:49	Justin Pryzby (justinpryzby)	Attached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-01 12:46:19	Masahiko Sawada (masahikosawada)	Closed in commitfest 2021-01 with status: Moved to next CF
2021-01-22 13:29:28	Masahiko Sawada (masahikosawada)	New status: Needs review
2021-01-10 17:32:10	Justin Pryzby (justinpryzby)	New status: Waiting on Author
2021-01-04 16:17:29	Justin Pryzby (justinpryzby)	Changed authors to dilip kumar (dilip.kumar), Ildus Kurbangaliev (ildus)
2021-01-04 15:23:15	Justin Pryzby (justinpryzby)	Added justinpryzby as reviewer
2020-12-01 11:10:48	Anastasia Lubennikova (lubennikovaav)	Closed in commitfest 2020-11 with status: Moved to next CF
2020-11-03 09:23:52	Magnus Hagander (mha)	Changed authors to Ildus Kurbangaliev (ildus)
2020-10-30 11:03:58	Dilip Kumar (dilip.kumar)	Attached mail thread 20170907194236.4cefce96@wp.localdomain
2020-10-30 11:03:57	Dilip Kumar (dilip.kumar)	Created patch record

Edit