custom compression methods for toast

Edit
Comment/Review
Change Status
ID 2813
Title custom compression methods for toast
Topic Server Features
Created 2020-10-30 11:03:57
Last modified 2021-04-08 15:30:47 (3 years, 7 months ago)
Latest email 2021-03-30 14:30:32 (3 years, 7 months ago)
Status
2021-03: Committed
2021-01: Moved to next CF
2020-11: Moved to next CF
Target version 14
Authors Dilip Kumar (dilip.kumar), Ildus Kurbangaliev (ildus)
Reviewers Justin Pryzby (justinpryzby)Become reviewer
Committer Robert Haas (rhaas)
Links CFbot results (CirrusCI) CFbot GitHub
Checkout latest CFbot patchset Go to your local checkout of the PostgreSQL repository and run: 
git remote add commitfest https://github.com/postgresql-cfbot/postgresql.git
git fetch commitfest cf/2813
git checkout commitfest/cf/2813
Emails
Custom compression methods
First at 2017-09-07 16:42:36 by Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru>
Latest at 2021-01-19 09:53:55 by Dilip Kumar <dilipbalaut at gmail.com>
Latest attachment (v19-0002-alter-table-set-compression.patch) at 2021-01-19 09:53:55 from Dilip Kumar <dilipbalaut at gmail.com>
    Attachment (v19-0002-alter-table-set-compression.patch) at 2021-01-19 09:53:55 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v18-0002-alter-table-set-compression.patch) at 2021-01-13 08:44:01 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v17-0003-Add-support-for-PRESERVE.patch) at 2021-01-04 11:27:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v17-0007-Add-Lz4-compression-to-WAL-FPIs.patch) at 2020-12-29 17:48:49 from Andrey Borodin <x4mmm at yandex-team.ru> (Patch: Yes)
    Attachment (v16-0007-Add-Lz4-compression-to-WAL-FPIs.patch) at 2020-12-27 07:10:34 from Andrey Borodin <x4mmm at yandex-team.ru> (Patch: Yes)
    Attachment (v16-0002-alter-table-set-compression.patch) at 2020-12-25 09:34:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v15-0002-alter-table-set-compression.patch) at 2020-12-09 12:07:27 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v14-0005-new-compression-method-extension-for-zlib.patch) at 2020-11-11 14:39:37 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v13-0003-Add-support-for-PRESERVE.patch) at 2020-11-08 10:59:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v12-0002-alter-table-set-compression.patch) at 2020-10-29 06:37:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v11-0004-Create-custom-compression-methods.patch) at 2020-10-28 07:46:31 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v10-0002-alter-table-set-compression.patch) at 2020-10-27 05:24:32 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v9-0002-alter-table-set-compression.patch) at 2020-10-22 12:26:55 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v8-0002-alter-table-set-compression.patch) at 2020-10-21 08:29:50 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v7-0002-alter-table-set-compression.patch) at 2020-10-17 06:04:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v6-0001-Built-in-compression-method.patch) at 2020-10-12 08:58:43 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v5-0002-alter-table-set-compression.patch) at 2020-10-08 14:24:42 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v4-0002-alter-table-set-compression.patch) at 2020-10-07 11:30:12 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v4-0001-Built-in-compression-method.patch) at 2020-10-05 07:48:14 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v3-0001-Built-in-compression-method.patch) at 2020-10-04 10:31:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v3-0002-alter-table-set-compression.patch) at 2020-09-28 10:48:07 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0002-alter-table-set-compression.patch) at 2020-09-19 07:49:48 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v1-0001-Add-support-for-setting-the-compression-method.patch) at 2020-08-13 11:48:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (custom_compression_methods_v23.patch) at 2019-07-08 12:00:21 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (jsonbd-master-fix.patch) at 2019-03-21 19:59:56 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v22.patch) at 2019-03-15 11:52:03 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v21.patch) at 2019-02-28 15:44:15 from Ildus Kurbangaliev <i.kurbangaliev at gmail.com> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v20.patch) at 2018-12-03 12:43:32 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v20.patch) at 2018-11-01 11:54:37 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v19.patch) at 2018-09-06 15:27:13 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v17.patch) at 2018-07-02 12:56:24 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Make-syntax-changes-for-custom-compression-metho-v16.patch) at 2018-06-18 14:30:45 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v15.patch) at 2018-03-30 16:50:45 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v14.patch) at 2018-03-26 17:38:25 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v14.patch) at 2018-03-14 13:31:23 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v13.patch) at 2018-02-26 12:25:56 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v12.patch) at 2018-02-06 10:47:31 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v11.patch) at 2018-01-29 11:44:52 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v10.patch) at 2018-01-25 14:24:57 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v9.patch) at 2018-01-23 13:04:54 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v8.patch) at 2018-01-14 21:49:30 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (Screen Shot 2017-12-03 at 11.46.14.png) at 2017-12-03 02:52:13 from Oleg Bartunov <obartunov at gmail.com> (Patch: No)
    Attachment (data-sorted-exceeds-ram.png) at 2017-11-29 23:30:37 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v7.patch) at 2017-11-28 13:29:18 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v6.patch) at 2017-11-23 09:38:49 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (pg_lz4.tgz) at 2017-11-21 20:42:37 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v5.patch) at 2017-11-21 14:47:17 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (pg_lz4.tgz) at 2017-11-19 23:04:53 from Tomas Vondra <tomas.vondra at 2ndquadrant.com> (Patch: No)
    Attachment (custom_compression_methods_v4.patch) at 2017-11-14 13:23:56 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v3.patch) at 2017-11-02 12:28:36 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v2.patch) at 2017-09-12 14:55:05 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
    Attachment (custom_compression_methods_v1.patch) at 2017-09-07 16:42:36 from Ildus Kurbangaliev <i.kurbangaliev at postgrespro.ru> (Patch: Yes)
Re: [HACKERS] Custom compression methods
First at 2021-01-19 19:07:20 by Justin Pryzby <pryzby at telsasoft.com>
Latest at 2021-03-30 14:30:32 by Tom Lane <tgl at sss.pgh.pa.us>
Latest attachment (v4-0001-Show-compression-method-in-index-describe.patch) at 2021-03-25 09:44:36 from Dilip Kumar <dilipbalaut at gmail.com>
    Attachment (v4-0001-Show-compression-method-in-index-describe.patch) at 2021-03-25 09:44:36 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v3-0003-ALTER-SET-COMPRESSION-for-index-columns.patch) at 2021-03-24 11:45:11 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Show-compression-method-in-index-describe.patch) at 2021-03-24 10:10:58 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v1-0001-Fix-attcompression-for-index-expression-columns.patch) at 2021-03-24 08:54:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (toast_failedassertion.txt) at 2021-03-24 07:52:00 from Jaime Casanova <jcasanov at systemguards.com.ec> (Patch: No)
    Attachment (v3-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patch) at 2021-03-22 20:57:18 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patch) at 2021-03-22 20:33:55 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (toast-compression-guc-rmh.patch) at 2021-03-22 17:38:36 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v1-0001-fix-slice-decompression.patch) at 2021-03-22 15:27:59 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v3-0001-Add-docs-for-default_toast_compression.patch) at 2021-03-21 23:55:45 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (fix-partial-lz4-decompression.patch) at 2021-03-21 23:52:09 from Tom Lane <tgl at sss.pgh.pa.us> (Patch: Yes)
    Attachment (v1-0001-fix-compression-test.patch) at 2021-03-21 05:22:14 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-fix-compression-test.patch) at 2021-03-21 03:40:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v2-0001-Add-docs-for-default_toast_compression.patch) at 2021-03-20 23:20:39 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (0001-Use-valid-compression-method-in-brin_form_tuple.patch) at 2021-03-20 15:13:47 from Tomas Vondra <tomas.vondra at enterprisedb.com> (Patch: Yes)
    Attachment (0001-Fixup-dump-toast-compression-method.patch) at 2021-03-20 07:52:17 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Add-docs-for-default_toast_compression.patch) at 2021-03-20 07:44:20 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (comments.patch) at 2021-03-19 22:38:43 from Alvaro Herrera <alvherre at alvh.no-ip.org> (Patch: Yes)
    Attachment (redo-lz4-configuration.patch) at 2021-03-19 22:13:59 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (non-default-toast-compression-only-v1.patch) at 2021-03-19 21:49:37 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patch) at 2021-03-19 14:11:03 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patch) at 2021-03-18 19:57:21 from Robert Haas <robertmhaas at gmail.com> (Patch: No)
    Attachment (v38-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-18 14:22:05 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v37-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-18 11:10:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v36-0002-Expand-the-external-data-before-forming-the-tupl.patch) at 2021-03-17 11:40:44 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (test1.sql) at 2021-03-16 10:37:22 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-15 12:14:33 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patch) at 2021-03-12 05:15:06 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patch) at 2021-03-11 18:25:26 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (v34-0002-Expand-the-external-data-before-forming-the-tupl.patch) at 2021-03-11 10:51:38 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Built-in-compression-method.patch) at 2021-03-11 02:28:58 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v33-0004-default-to-with-lz4.patch) at 2021-03-10 11:51:45 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v32-0001-Built-in-compression-method.patch) at 2021-03-08 10:02:39 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v31-0001-Built-in-compression-method.patch) at 2021-03-06 15:29:16 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v30-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch) at 2021-03-04 13:43:13 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v29-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-03-01 15:23:09 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v28-0004-default-to-with-lz4.patch) at 2021-02-27 09:14:54 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v27-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-26 14:40:29 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v26-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-20 10:46:48 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v25-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-19 16:12:29 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-18 21:13:14 from Justin Pryzby <pryzby at telsasoft.com> (Patch: Yes)
    Attachment (v1-0001-Disallow-compressed-data-inside-container-types.patch) at 2021-02-16 14:20:35 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v24-0001-Built-in-compression-method.patch) at 2021-02-14 20:28:31 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v24-0001-Built-in-compression-method.patch) at 2021-02-14 18:49:40 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (POC_fix_compression_in_rowtype.patch) at 2021-02-11 12:36:28 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (fixups.patch) at 2021-02-10 21:56:17 from Robert Haas <robertmhaas at gmail.com> (Patch: Yes)
    Attachment (v23-0003-Add-support-for-PRESERVE.patch) at 2021-02-10 14:51:59 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
    Attachment (v22-0001-Built-in-compression-method.patch) at 2021-02-07 11:45:42 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v21-0001-Built-in-compression-method.patch) at 2021-02-05 14:41:41 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: No)
    Attachment (v21-0001-Built-in-compression-method.patch) at 2021-01-30 07:16:39 from Justin Pryzby <pryzby at telsasoft.com> (Patch: No)
    Attachment (v20-0002-alter-table-set-compression.patch) at 2021-01-29 12:37:24 from Dilip Kumar <dilipbalaut at gmail.com> (Patch: Yes)
History
When Who What
2021-04-08 15:30:47 David Steele (dsteele) Changed targetversion to 14
2021-04-08 15:30:40 David Steele (dsteele) Closed in commitfest 2021-03 with status: Committed
2021-04-08 15:28:46 David Steele (dsteele) Changed targetversion to None
2021-03-20 20:59:49 Justin Pryzby (justinpryzby) New status: Ready for Committer
2021-03-20 20:59:05 Justin Pryzby (justinpryzby) Closed in commitfest 2021-03 with status: Committed
2021-03-20 20:59:05 Justin Pryzby (justinpryzby) Changed committer to rhaas
2021-03-06 20:14:50 Justin Pryzby (justinpryzby) Changed targetversion to 14
2021-02-15 15:10:35 Justin Pryzby (justinpryzby) Attached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-15 15:10:15 Justin Pryzby (justinpryzby) Detached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-09 18:49:49 Justin Pryzby (justinpryzby) Attached mail thread 20210119190720.GL8560@telsasoft.com
2021-02-01 12:46:19 Masahiko Sawada (masahikosawada) Closed in commitfest 2021-01 with status: Moved to next CF
2021-01-22 13:29:28 Masahiko Sawada (masahikosawada) New status: Needs review
2021-01-10 17:32:10 Justin Pryzby (justinpryzby) New status: Waiting on Author
2021-01-04 16:17:29 Justin Pryzby (justinpryzby) Changed authors to dilip kumar (dilip.kumar), Ildus Kurbangaliev (ildus)
2021-01-04 15:23:15 Justin Pryzby (justinpryzby) Added justinpryzby as reviewer
2020-12-01 11:10:48 Anastasia Lubennikova (lubennikovaav) Closed in commitfest 2020-11 with status: Moved to next CF
2020-11-03 09:23:52 Magnus Hagander (mha) Changed authors to Ildus Kurbangaliev (ildus)
2020-10-30 11:03:58 Dilip Kumar (dilip.kumar) Attached mail thread 20170907194236.4cefce96@wp.localdomain
2020-10-30 11:03:57 Dilip Kumar (dilip.kumar) Created patch record
Edit
Comment/Review
Change Status