Speed up verifying UTF-8

Edit
ID 2996
Title Speed up verifying UTF-8
Topic Performance
Created 2021-02-20 21:13:00
Last modified 2021-12-20 14:26:44 (2 years, 9 months ago)
Latest email 2021-12-20 14:24:40 (2 years, 9 months ago)
Status
2022-01: Committed
2021-11: Moved to next CF
2021-09: Moved to next CF
2021-07: Moved to next CF
2021-03: Moved to next CF
Target version
Authors John Naylor (john.naylor)
Reviewers Heikki Linnakangas (heikki), Amit Khandekar (amitdkhan)Become reviewer
Committer John Naylor (john.naylor)
Links CFbot results (CirrusCI) CFbot GitHub
Checkout latest CFbot patchset Go to your local checkout of the PostgreSQL repository and run:
git remote add commitfest https://github.com/postgresql-cfbot/postgresql.git
git fetch commitfest cf/2996
git checkout commitfest/cf/2996
Emails
[POC] verifying UTF-8 using SIMD instructions
First at 2021-02-01 17:32:23 by John Naylor <john.naylor at enterprisedb.com>
Latest at 2021-12-20 14:24:40 by John Naylor <john.naylor at enterprisedb.com>
Latest attachment (v25-0001-Add-fast-path-for-validating-UTF-8-text.patch) at 2021-12-13 15:39:37 from John Naylor <john.naylor at enterprisedb.com>
    Attachment (v25-0001-Add-fast-path-for-validating-UTF-8-text.patch) at 2021-12-13 15:39:37 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v24-0001-Add-fast-path-for-validating-UTF-8-text.patch) at 2021-10-19 21:42:40 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v23-0001-Add-fast-paths-for-validating-UTF-8-text.patch) at 2021-08-26 15:35:54 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v22-addendum-32-bit-transitions.txt) at 2021-08-24 16:00:28 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v22-0001-Add-fast-paths-for-validating-UTF-8-text.patch) at 2021-08-04 11:22:57 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (mbverifystr-threshold.sql) at 2021-07-30 01:12:33 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v21-0001-Add-fast-paths-for-validating-UTF-8-text.patch) at 2021-07-28 18:12:11 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v20-0001-Add-a-fast-path-for-validating-UTF-8-text.patch) at 2021-07-26 11:09:00 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v2-0001-XXX-Make-SIMD-code-more-platform-neutral.txt) at 2021-07-22 00:07:26 from Thomas Munro <thomas.munro at gmail.com> (Patch: Yes)
    Attachment (0001-XXX-Make-SIMD-code-more-platform-neutral.txt) at 2021-07-21 15:29:21 from Thomas Munro <thomas.munro at gmail.com> (Patch: Yes)
    Attachment (v19-rewrite-pg_verify_str-for-speed.patch) at 2021-07-20 21:24:33 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v18-0001-Use-pure-DFA.patch) at 2021-07-19 01:26:47 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v17-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-07-17 00:02:33 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v16gamma-Rewrite-pg_utf8_verifystr-for-speed.txt) at 2021-07-15 22:00:05 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v16alpha-Rewrite-pg_utf8_verifystr-for-speed.txt) at 2021-07-15 18:12:43 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v15-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-07-12 19:45:39 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v14-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-06-30 11:18:32 from Heikki Linnakangas <hlinnaka at iki.fi> (Patch: Yes)
    Attachment (v13-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-06-29 11:20:38 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v12-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-06-10 12:45:01 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (Chinese62.txt) at 2021-06-07 12:39:40 from John Naylor <john.naylor at enterprisedb.com> (Patch: No)
    Attachment (v11-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-06-06 19:21:51 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v10-0001-Rewrite-pg_utf8_verifystr-for-speed.patch) at 2021-06-02 16:26:41 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v9-0001-Replace-pg_utf8_verifystr-with-two-faster-impleme.patch) at 2021-04-01 14:22:06 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v8-0001-Add-noError-argument-to-encoding-conversion-funct.patch) at 2021-03-19 19:24:06 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v7-0001-Add-noError-argument-to-encoding-conversion-funct.patch) at 2021-02-24 21:50:50 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v6-0001-Add-noError-argument-to-encoding-conversion-funct.patch) at 2021-02-24 16:25:49 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v4-0001-Add-noError-argument-to-encoding-conversion-funct.patch) at 2021-02-20 21:10:58 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (addendum-01-8-byte-stride.patch) at 2021-02-19 00:43:04 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v4-SSE4-with-autoconf-support.patch) at 2021-02-17 05:40:32 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v3-SSE4-with-autoconf-support.patch) at 2021-02-16 01:32:52 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v2-add-portability-stub-and-new-fallback.patch) at 2021-02-13 01:31:33 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (utf-sse42-demo.patch) at 2021-02-09 21:12:22 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (simdjson-utf8-hack.patch) at 2021-02-08 10:17:11 from Heikki Linnakangas <hlinnaka at iki.fi> (Patch: Yes)
    Attachment (v1-0001-Add-an-ASCII-fast-path-to-multibyte-encoding-veri.patch) at 2021-02-07 20:24:16 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
    Attachment (v1-verify-utf8-sse-ascii.patch) at 2021-02-01 17:32:23 from John Naylor <john.naylor at enterprisedb.com> (Patch: Yes)
History
When Who What
2021-12-20 14:26:44 John Naylor (john.naylor) Closed in commitfest 2022-01 with status: Committed
2021-12-13 19:18:34 John Naylor (john.naylor) Added john.naylor as committer
2021-12-13 19:18:25 John Naylor (john.naylor) New status: Needs review
2021-12-08 18:12:16 John Naylor (john.naylor) New status: Waiting on Author
2021-12-02 11:32:46 Daniel Gustafsson (d_gustafsson) Closed in commitfest 2021-11 with status: Moved to next CF
2021-10-19 21:43:20 John Naylor (john.naylor) New status: Needs review
2021-10-18 14:57:44 John Naylor (john.naylor) New status: Waiting on Author
2021-10-06 13:16:44 Jaime Casanova (jcasanov) Closed in commitfest 2021-09 with status: Moved to next CF
2021-08-04 11:24:00 John Naylor (john.naylor) New status: Needs review
2021-08-03 02:46:29 Masahiko Sawada (masahikosawada) New status: Waiting on Author
2021-08-03 02:27:17 Masahiko Sawada (masahikosawada) Closed in commitfest 2021-07 with status: Moved to next CF
2021-07-26 11:09:34 John Naylor (john.naylor) New status: Needs review
2021-07-22 03:17:48 John Naylor (john.naylor) New status: Waiting on Author
2021-07-12 19:46:02 John Naylor (john.naylor) New status: Needs review
2021-07-04 20:02:41 John Naylor (john.naylor) New status: Waiting on Author
2021-06-30 12:06:37 Heikki Linnakangas (heikki) Added heikki as reviewer
2021-06-02 16:30:48 John Naylor (john.naylor) Changed name to Speed up verifying UTF-8
2021-04-08 15:44:06 David Steele (dsteele) Closed in commitfest 2021-03 with status: Moved to next CF
2021-03-03 14:07:19 Amit Khandekar (amitdkhan) Added amitdkhan as reviewer
2021-02-21 00:07:31 John Naylor (john.naylor) Changed authors to John Naylor (john.naylor)
2021-02-20 21:13:00 John Naylor (john.naylor) Attached mail thread CAFBsxsEV_SzH+OLyCiyon=iwggSyMh_eF6A3LU2tiWf3Cy2ZQg@mail.gmail.com
2021-02-20 21:13:00 John Naylor (john.naylor) Created patch record
Edit