Improve dead tuple storage for lazy vacuum

git remote add commitfest https://github.com/postgresql-cfbot/postgresql.git
git fetch commitfest cf/3687
git checkout commitfest/cf/3687

[PoC] Improve dead tuple storage for lazy vacuum ×

First at 2021-07-07 11:46:38 by Masahiko Sawada <sawada.mshk at gmail.com>
Latest at 2024-04-08 13:17:54 by John Naylor <johncnaylorls at gmail.com>
Latest attachment (fix-anonymous-union.patch) at 2024-04-08 12:26:40 from John Naylor <johncnaylorls at gmail.com>

    Attachment (fix-anonymous-union.patch) at 2024-04-08 12:26:40 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (tidstore-quickfix-alignment.diff) at 2024-04-07 19:07:31 from Andres Freund <andres at anarazel.de> (Patch: Yes)
    Attachment (v84-0005-Introduce-a-bump-memory-allocator.patch) at 2024-04-07 09:44:56 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v83-0001-store-offsets-in-the-header.patch) at 2024-04-07 02:08:40 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v82-0001-Use-TidStore-for-dead-tuple-TIDs-storage-during-.patch) at 2024-04-01 02:53:28 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (comm-msg-jcn.txt) at 2024-03-29 07:21:32 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v81-0001-Use-TidStore-for-dead-tuple-TIDs-storage-during-.patch) at 2024-03-28 05:55:16 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v80-0001-Rethink-create-and-attach-APIs-of-shared-TidStor.patch) at 2024-03-27 08:43:37 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v79-0006-Address-review-comments-on-vacuum-integration.patch) at 2024-03-25 13:06:46 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (fix_radixtree.patch) at 2024-03-25 01:02:18 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v78-0005-Rethink-create-and-attach-APIs-of-shared-TidStor.patch) at 2024-03-22 05:19:44 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v77-0003-PoC-Make-shared-TidStore-create-its-own-DSA-area.patch) at 2024-03-21 09:02:47 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (tidstore_creates_dsa.patch.nocfbot) at 2024-03-21 07:02:05 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v76-0001-Use-TidStore-for-dead-tuple-TIDs-storage-during-.patch) at 2024-03-21 02:37:16 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v75-0001-Add-TIDStore-to-store-sets-of-TIDs-ItemPointerDa.patch) at 2024-03-19 16:40:25 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v74-ART.tar.gz) at 2024-03-19 03:23:47 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (use-fewer-blocks.patch.nocfbot) at 2024-03-18 23:35:39 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v73-ART.tar.gz) at 2024-03-18 04:12:07 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v72-ART.tar.gz) at 2024-03-15 07:36:10 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v71-0002-DEV-Debug-TidStore.patch) at 2024-03-13 11:04:51 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v70-ART.tar.gz) at 2024-03-12 02:20:18 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v69-ART.tar.gz) at 2024-03-11 08:13:17 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (WIP-3-embeddable-TIDs.patch.nocfbot) at 2024-03-11 07:32:20 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v68-ART.tar.gz) at 2024-03-07 15:34:48 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (fix-long-int-format.patch) at 2024-03-07 07:21:03 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v67-ART.tar.gz) at 2024-03-06 11:20:35 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (cast-signed-for-gcc.patch) at 2024-03-06 08:33:44 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v66-ART.tar.gz) at 2024-03-05 09:41:30 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v65-ART.tar.gz) at 2024-03-04 06:05:12 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v64-ART.tar.gz) at 2024-03-01 06:58:07 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v63-ART.tar.gz) at 2024-02-20 06:59:04 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v62-ART.tar.gz) at 2024-02-16 00:51:39 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v61-ART.tar.gz) at 2024-02-15 11:26:13 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v60-ART.tar.gz) at 2024-02-15 03:20:46 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (vacuum-test-lookup-int-2024.sql) at 2024-02-10 12:29:10 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v58-ART.tar.gz) at 2024-02-06 02:57:58 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v57-ART.tar.gz) at 2024-02-02 11:47:02 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v56-ART.tar.gz) at 2024-01-31 05:49:21 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v55-ART.tar.gz) at 2024-01-23 07:48:25 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v55-ART.tar.gz) at 2024-01-23 03:58:00 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v54-ART.tar.gz) at 2024-01-22 07:23:43 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v51-addendum-Put-shared-radix-tree-for-tidstore-in-its-own-tr.patch.nocfbot) at 2024-01-19 11:27:09 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (rt-find-callback-interface.patch.nocfbot) at 2024-01-18 04:30:01 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v53-ART.tar.gz) at 2024-01-17 03:32:25 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v52-ART.tar.gz) at 2024-01-16 06:17:35 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v51-addendum-size-class.patch.txt) at 2024-01-10 06:40:40 from John Naylor <johncnaylorls at gmail.com> (Patch: Yes)
    Attachment (v51-ART.tar.gz) at 2024-01-10 02:05:06 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v50-ART.tar.gz) at 2024-01-09 02:40:03 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v49-ART.tar.gz) at 2024-01-03 14:10:11 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v48-ART.tar.gz) at 2024-01-02 13:01:07 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v47-ART.tar.gz) at 2023-12-21 07:41:37 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v46-ART.tar.gz) at 2023-12-20 11:35:38 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v45-ART.tar.gz) at 2023-12-14 00:22:11 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v44-ART.tar.gz) at 2023-12-11 06:17:37 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v43-ART.tar.gz) at 2023-10-28 08:56:42 from John Naylor <johncnaylorls at gmail.com> (Patch: No)
    Attachment (v42-ART.tar.gz) at 2023-08-28 14:43:22 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v41-ART.tar.gz) at 2023-08-27 12:53:00 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v40-ART.tar.gz) at 2023-08-16 11:04:34 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v39-ART.tar.gz) at 2023-08-15 11:53:05 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (fix_oom.patch) at 2023-08-15 02:33:38 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v38-ART.tar.gz) at 2023-08-14 11:05:21 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v37-ART.tar.gz) at 2023-08-01 08:12:05 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v36-ART.tar.gz) at 2023-07-08 02:54:19 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v35-0001-Introduce-helper-SIMD-functions-for-small-byte-a.patch) at 2023-06-23 09:54:21 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v34-ART.tar.gz) at 2023-06-14 06:23:07 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v33-ART.tar.gz) at 2023-05-23 23:16:54 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v32-0015-vacuum-Miscellaneous-updates.patch) at 2023-04-17 15:20:16 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (dsa_init_max_block_size.patch.txt) at 2023-03-20 14:34:08 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v31-0013-Add-min-and-max-classes-for-node3-and-node125.patch) at 2023-03-10 15:26:18 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v30-0008-Remove-the-max-memory-deduction-from-TidStore.patch) at 2023-03-09 06:51:05 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (vacuum-test-lookup-int.sql) at 2023-02-27 17:07:23 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (add_assertions.patch.txt) at 2023-02-24 05:50:19 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v29-0011-Debug-TIDStore.patch.txt) at 2023-02-22 06:15:30 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v29-0006-Use-TIDStore-for-storing-dead-tuple-TID-during-l.patch) at 2023-02-20 05:56:15 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v2899-0002-Small-improvements-for-radixtree-and-tests.patch.txt) at 2023-02-16 03:24:16 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v28-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2023-02-14 12:36:51 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v27-0001-Introduce-helper-SIMD-functions-for-small-byte-a.patch) at 2023-02-14 11:24:38 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v2699-0002-Do-less-work-when-encoding-key-value.patch.txt) at 2023-02-11 05:33:42 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (change_tidstore_for_v15.patch) at 2023-02-10 07:15:46 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (0001-Add-bench_tidstore_load.patch.txt) at 2023-02-09 07:08:03 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v25-addendum-try-no-maintain-order.txt) at 2023-02-07 10:22:59 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v25-addendum-try-no-maintain-order.txt) at 2023-02-07 09:25:44 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v24-0005-Add-read-write-lock-to-radix-tree-in-RT_SHMEM-ca.patch) at 2023-01-31 14:42:32 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (remove-intermediate-variables.txt) at 2023-01-28 11:32:50 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v23-0016-Add-read-write-lock-to-radix-tree-in-RT_SHMEM-ca.patch) at 2023-01-26 14:47:40 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v22-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2023-01-23 12:29:33 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v21-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2023-01-23 11:20:06 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v20-0013-Use-TIDStore-for-storing-dead-tuple-TID-during-l.patch) at 2023-01-19 15:18:02 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (link_pgport_src.patch) at 2023-01-18 15:49:04 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v19-0001-introduce-vector8_min-and-vector8_highbit_mask.patch) at 2023-01-17 11:05:59 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v18-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2023-01-16 05:01:57 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (memory.py) at 2023-01-16 02:52:30 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)
    Attachment (v17-0010-fix-shmem-support.patch) at 2023-01-12 14:50:32 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v17-0005-Template-out-inner-and-leaf-nodes.patch) at 2023-01-09 08:59:04 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v16-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2022-12-23 11:47:08 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v15-0008-PoC-calculate-memory-usage-in-radix-tree.patch) at 2022-12-20 05:03:58 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v14-0005-tool-for-measuring-radix-tree-performance.patch) at 2022-12-19 07:13:45 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v13-0002-Move-some-bitmap-logic-out-of-bitmapset.c.patch) at 2022-12-06 10:32:08 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v12-0007-PoC-lazy-vacuum-integration.patch) at 2022-12-02 16:41:31 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v11-0006-addendum-bitmapword-node128.patch.txt) at 2022-11-30 05:51:03 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v11-0004-Preparatory-refactoring-for-decoupling-kind-from.patch) at 2022-11-24 14:54:06 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v10-0003-tool-for-measuring-radix-tree-performance.patch) at 2022-11-21 08:06:56 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v901-0002-Make-node32-variable-sized.patch.txt) at 2022-11-18 07:48:49 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (add-random-node-search-test.patch.txt) at 2022-11-16 07:39:00 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v9-0003-tool-for-measuring-radix-tree-performance.patch) at 2022-11-14 08:43:40 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v8-0005-PoC-lazy-vacuum-integration.patch) at 2022-11-04 15:24:23 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v8-0001-introduce-vector8_min-and-vector8_highbit_mask.patch) at 2022-10-31 05:46:53 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (0002-Add-radix-implementation.patch) at 2022-10-24 05:53:36 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (radix-v6-addendum-jcn1.tar.gz) at 2022-10-10 05:54:50 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (fix_bench_radix_tree.patch) at 2022-10-07 08:08:35 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v65-0001-Turn-on-per-node-counts-in-benchmark.patch) at 2022-10-07 05:29:11 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v6addendum-bench-node16.diff.txt) at 2022-09-21 06:17:21 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v6-0001-Support-pg_lsearch8_eq-and-pg_lsearch8_ge.patch) at 2022-09-16 06:00:31 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (radixtree_v5.patch) at 2022-07-22 01:43:09 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (radixtree_wip_v4.patch) at 2022-07-05 03:30:30 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (v3-radix-review-diff-20220627.txt) at 2022-06-27 11:12:13 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (radixtree_wip_v3.patch) at 2022-06-16 04:56:55 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (radixtree_wip_v2.patch) at 2022-05-25 02:48:16 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (radixtree.patch) at 2022-05-10 01:51:46 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: Yes)
    Attachment (bench_intset2.txt) at 2021-07-29 16:49:22 from Yura Sokolov <y.sokolov at postgrespro.ru> (Patch: No)
    Attachment (0001-integerset2.patch) at 2021-07-29 15:29:51 from Yura Sokolov <y.sokolov at postgrespro.ru> (Patch: Yes)
    Attachment (0001-svtm-specialized-vacuum-tid-map.patch) at 2021-07-25 16:07:18 from Yura Sokolov <y.sokolov at postgrespro.ru> (Patch: Yes)
    Attachment (0001-Fix-build-warnings.patch) at 2021-07-19 23:49:15 from Andres Freund <andres at anarazel.de> (Patch: Yes)
    Attachment (vacuum.svg) at 2021-07-07 11:46:38 from Masahiko Sawada <sawada.mshk at gmail.com> (Patch: No)