Bug 211880 - Use templates to reduce duplicate code in Persistence::Decoder and Persistence::Encoder classes
Summary: Use templates to reduce duplicate code in Persistence::Decoder and Persistenc...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Template Framework (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: David Kilzer (:ddkilzer)
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-05-13 21:25 PDT by David Kilzer (:ddkilzer)
Modified: 2020-06-07 21:47 PDT (History)
17 users (show)

See Also:


Attachments
Patch v1 (13.51 KB, patch)
2020-05-13 21:36 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v2 (14.74 KB, patch)
2020-05-14 08:57 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v3 (14.74 KB, patch)
2020-05-14 09:11 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v4 (14.75 KB, patch)
2020-05-14 10:15 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v5 (14.75 KB, patch)
2020-05-14 10:27 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v6 (14.80 KB, patch)
2020-05-14 11:29 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v7 (14.81 KB, patch)
2020-05-14 13:28 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v8 (16.32 KB, patch)
2020-05-14 13:48 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v9 (14.82 KB, patch)
2020-05-14 15:39 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v10 (15.26 KB, patch)
2020-05-23 09:54 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v11 (17.70 KB, patch)
2020-05-23 11:24 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v12 (17.71 KB, patch)
2020-05-23 11:49 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v13 (18.12 KB, patch)
2020-05-23 20:03 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v14 (18.12 KB, patch)
2020-05-23 20:53 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v15 (19.04 KB, patch)
2020-05-24 10:05 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v16 (19.04 KB, patch)
2020-05-24 10:22 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v17 (18.76 KB, patch)
2020-05-24 11:01 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v18 (18.74 KB, patch)
2020-05-24 12:15 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v19 (18.68 KB, patch)
2020-05-24 12:32 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v20 (18.39 KB, patch)
2020-05-24 13:31 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v21 (18.82 KB, patch)
2020-05-25 08:06 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v22 - v19 with Checksum:: updateChecksumForNumber() inlined - WinCairo compiled! (18.83 KB, patch)
2020-05-25 08:09 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v23 (18.47 KB, patch)
2020-05-25 09:20 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v24 (18.72 KB, patch)
2020-05-25 09:52 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
Patch v25 (24.16 KB, patch)
2020-05-25 10:34 PDT, David Kilzer (:ddkilzer)
no flags Details | Formatted Diff | Diff
v2 + OPENSSL_NO_SHA and OPENSSL_NO_SHA1 (15.47 KB, patch)
2020-06-07 21:46 PDT, Fujii Hironori
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Kilzer (:ddkilzer) 2020-05-13 21:25:03 PDT
Use templates to reduce duplicate code in Persistence::Decoder and Persistence::Encoder classes.
Comment 1 Radar WebKit Bug Importer 2020-05-13 21:25:30 PDT
<rdar://problem/63213462>
Comment 2 David Kilzer (:ddkilzer) 2020-05-13 21:36:19 PDT
Created attachment 399331 [details]
Patch v1
Comment 3 David Kilzer (:ddkilzer) 2020-05-13 21:40:18 PDT
Comment on attachment 399331 [details]
Patch v1

View in context: https://bugs.webkit.org/attachment.cgi?id=399331&action=review

> Source/WTF/wtf/persistence/PersistentDecoder.h:79
>      }

Is this static_assert() still needed now that the Decoder::operator<<() method is a template?

    template<typename E, std::enable_if_t<std::is_enum<E>::value>* = nullptr>
    Decoder& operator>>(Optional<E>& result)
    {
        static_assert(sizeof(E) <= 8, "Enum type T must not be larger than 64 bits!");
        [...]
    }

> Source/WTF/wtf/persistence/PersistentEncoder.h:51
> +        static_assert(sizeof(E) <= sizeof(uint64_t), "Enum type must not be larger than 64 bits.");

Is this static_assert() still needed now that the method is a template?
Comment 4 Alex Christensen 2020-05-13 22:04:41 PDT
Comment on attachment 399331 [details]
Patch v1

View in context: https://bugs.webkit.org/attachment.cgi?id=399331&action=review

>> Source/WTF/wtf/persistence/PersistentEncoder.h:51
>> +        static_assert(sizeof(E) <= sizeof(uint64_t), "Enum type must not be larger than 64 bits.");
> 
> Is this static_assert() still needed now that the method is a template?

This is still needed.  In the decoder we should change 8 to sizeof(uint64_t).
We also can't make this change (uint64_t to std::underlying_type) because existing persisted data has encoded enums as 64-bit values.
Comment 5 David Kilzer (:ddkilzer) 2020-05-14 08:57:16 PDT Comment hidden (obsolete)
Comment 6 David Kilzer (:ddkilzer) 2020-05-14 08:57:56 PDT
Comment on attachment 399331 [details]
Patch v1

View in context: https://bugs.webkit.org/attachment.cgi?id=399331&action=review

>>> Source/WTF/wtf/persistence/PersistentEncoder.h:51
>>> +        static_assert(sizeof(E) <= sizeof(uint64_t), "Enum type must not be larger than 64 bits.");
>> 
>> Is this static_assert() still needed now that the method is a template?
> 
> This is still needed.  In the decoder we should change 8 to sizeof(uint64_t).
> We also can't make this change (uint64_t to std::underlying_type) because existing persisted data has encoded enums as 64-bit values.

Fixed in v2.  Thanks!
Comment 7 David Kilzer (:ddkilzer) 2020-05-14 09:11:44 PDT Comment hidden (obsolete)
Comment 8 David Kilzer (:ddkilzer) 2020-05-14 09:12:30 PDT Comment hidden (obsolete)
Comment 9 David Kilzer (:ddkilzer) 2020-05-14 10:15:55 PDT Comment hidden (obsolete)
Comment 10 David Kilzer (:ddkilzer) 2020-05-14 10:16:32 PDT Comment hidden (obsolete)
Comment 11 David Kilzer (:ddkilzer) 2020-05-14 10:27:14 PDT Comment hidden (obsolete)
Comment 12 David Kilzer (:ddkilzer) 2020-05-14 11:21:02 PDT Comment hidden (obsolete)
Comment 13 David Kilzer (:ddkilzer) 2020-05-14 11:29:51 PDT Comment hidden (obsolete)
Comment 14 David Kilzer (:ddkilzer) 2020-05-14 13:28:23 PDT
Created attachment 399399 [details]
Patch v7
Comment 15 David Kilzer (:ddkilzer) 2020-05-14 13:42:10 PDT
Comment on attachment 399399 [details]
Patch v7

View in context: https://bugs.webkit.org/attachment.cgi?id=399399&action=review

> Source/WTF/wtf/persistence/PersistentEncoder.h:-87
> -    template <typename Type> struct Salt;

Made Encoder::Salt public to try to fix WinCairo build.
Comment 16 David Kilzer (:ddkilzer) 2020-05-14 13:48:58 PDT Comment hidden (obsolete)
Comment 17 David Kilzer (:ddkilzer) 2020-05-14 14:10:09 PDT
Comment on attachment 399399 [details]
Patch v7

View in context: https://bugs.webkit.org/attachment.cgi?id=399399&action=review

>> Source/WTF/wtf/persistence/PersistentEncoder.h:-87
>> -    template <typename Type> struct Salt;
> 
> Made Encoder::Salt public to try to fix WinCairo build.

That didn't work.  I'm out of ideas.  I think the WinCairo EWS VC++ compiler is out-of-date.  Every other platform (including AppleWin) build these patches.
Comment 18 David Kilzer (:ddkilzer) 2020-05-14 15:16:57 PDT
I need C++ template help to fix the WinCario build errors.  I don't understand what Visual C++ is complaining about:

[32/422] Building CXX object Source\WebCore\CMakeFiles\WebCore.dir\platform\network\curl\CertificateInfoCurl.cpp.obj
FAILED: Source/WebCore/CMakeFiles/WebCore.dir/platform/network/curl/CertificateInfoCurl.cpp.obj 
[...]
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): error C2672: 'WTF::Persistence::Encoder::updateChecksumForNumber': no matching overloaded function found
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentCoders.h(261): note: see reference to function template instantiation 'WTF::Persistence::Encoder &WTF::Persistence::Encoder::operator <<<double,0x0>(T)' being compiled
        with
        [
            T=double
        ]
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): error C2893: Failed to specialize function template 'void WTF::Persistence::Encoder::updateChecksumForNumber(WTF::SHA1 &,T)'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(76): note: see declaration of 'WTF::Persistence::Encoder::updateChecksumForNumber'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): note: With the following template arguments:
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): note: 'T=T'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): error C2672: 'WTF::Persistence::Encoder::updateChecksumForNumber': no matching overloaded function found
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentCoders.h(267): note: see reference to function template instantiation 'WTF::Persistence::Decoder &WTF::Persistence::Decoder::operator >><double,0x0>(WTF::Optional<double> &)' being compiled
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): error C2893: Failed to specialize function template 'void WTF::Persistence::Encoder::updateChecksumForNumber(WTF::SHA1 &,T)'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(76): note: see declaration of 'WTF::Persistence::Encoder::updateChecksumForNumber'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): note: With the following template arguments:
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): note: 'T=T'
[36/422] Building CXX object Source\WebCore\CMakeFiles\WebCore.dir\platform\network\curl\CurlContext.cpp.obj
FAILED: Source/WebCore/CMakeFiles/WebCore.dir/platform/network/curl/CurlContext.cpp.obj 
[...]
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): error C2672: 'WTF::Persistence::Encoder::updateChecksumForNumber': no matching overloaded function found
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentCoders.h(261): note: see reference to function template instantiation 'WTF::Persistence::Encoder &WTF::Persistence::Encoder::operator <<<double,0x0>(T)' being compiled
        with
        [
            T=double
        ]
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): error C2893: Failed to specialize function template 'void WTF::Persistence::Encoder::updateChecksumForNumber(WTF::SHA1 &,T)'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(76): note: see declaration of 'WTF::Persistence::Encoder::updateChecksumForNumber'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): note: With the following template arguments:
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(66): note: 'T=T'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): error C2672: 'WTF::Persistence::Encoder::updateChecksumForNumber': no matching overloaded function found
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentCoders.h(267): note: see reference to function template instantiation 'WTF::Persistence::Decoder &WTF::Persistence::Decoder::operator >><double,0x0>(WTF::Optional<double> &)' being compiled
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): error C2893: Failed to specialize function template 'void WTF::Persistence::Encoder::updateChecksumForNumber(WTF::SHA1 &,T)'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentEncoder.h(76): note: see declaration of 'WTF::Persistence::Encoder::updateChecksumForNumber'
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): note: With the following template arguments:
C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentDecoder.h(54): note: 'T=T'

Here are pages explaining the two error codes:

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-2/compiler-error-c2672?view=vs-2019
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-2/compiler-error-c2893?view=vs-2019
Comment 19 David Kilzer (:ddkilzer) 2020-05-14 15:29:48 PDT
Hold on.  I think I know what's going on.

Encoder::updateChecksumForNumber() used `typename Type` instead of `typename T`, which differed from other template functions in the Persistence::Encoder class.
Comment 20 David Kilzer (:ddkilzer) 2020-05-14 15:39:02 PDT
Created attachment 399420 [details]
Patch v9
Comment 21 David Kilzer (:ddkilzer) 2020-05-14 18:39:49 PDT
(In reply to David Kilzer (:ddkilzer) from comment #19)
> Hold on.  I think I know what's going on.
> 
> Encoder::updateChecksumForNumber() used `typename Type` instead of `typename
> T`, which differed from other template functions in the Persistence::Encoder
> class.

And I was wrong.
Comment 22 Stephan Szabo 2020-05-18 20:24:24 PDT
As an observation from playing with it, with the Coder<Seconds> and Coder<WallTime> bodies in PersistentCoders.cpp rather than the .h, many of the errors went away, but the Coder<CertificateInfo> still failed (either in the h or the cpp) and it seems like the cases that fail are in files that were including CertificateInfo. I don't see why that'd be different than anything else that might be including the persistence files though.
Comment 23 David Kilzer (:ddkilzer) 2020-05-23 09:54:53 PDT Comment hidden (obsolete)
Comment 24 David Kilzer (:ddkilzer) 2020-05-23 11:24:32 PDT
Created attachment 400129 [details]
Patch v11
Comment 25 David Kilzer (:ddkilzer) 2020-05-23 11:27:13 PDT
(In reply to David Kilzer (:ddkilzer) from comment #24)
> Created attachment 400129 [details]
> Patch v11

Style bot is just complaining about having the salt() methods defined on one line:

ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:90:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:91:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:92:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:93:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:94:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:95:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:96:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:97:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:98:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:99:  More than one command on the same line  [whitespace/newline] [4]
ERROR: Source/WTF/wtf/persistence/PersistentEncoder.h:100:  More than one command on the same line  [whitespace/newline] [4]
Total errors found: 11 in 5 files
Comment 26 David Kilzer (:ddkilzer) 2020-05-23 11:46:38 PDT
(In reply to David Kilzer (:ddkilzer) from comment #24)
> Created attachment 400129 [details]
> Patch v11

Still not sure why WinCairo build is failing.

I just don't understand the error message that MSVC++ is printing.
Comment 27 David Kilzer (:ddkilzer) 2020-05-23 11:49:44 PDT
Created attachment 400131 [details]
Patch v12
Comment 28 David Kilzer (:ddkilzer) 2020-05-23 11:50:25 PDT
(In reply to David Kilzer (:ddkilzer) from comment #27)
> Created attachment 400131 [details]
> Patch v12

One last attempt (which I tried when struct Encode::Salt existed):

-        Encoder::updateChecksumForNumber(m_sha1, value);
+        Encoder::updateChecksumForNumber<U>(m_sha1, value);
Comment 29 David Kilzer (:ddkilzer) 2020-05-23 20:03:46 PDT
Created attachment 400147 [details]
Patch v13
Comment 30 David Kilzer (:ddkilzer) 2020-05-23 20:07:39 PDT
(In reply to David Kilzer (:ddkilzer) from comment #29)
> Created attachment 400147 [details]
> Patch v13

I noticed in this error message:

C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/persistence/PersistentCoders.h(261): note: see reference to function template instantiation 'WTF::Persistence::Encoder &WTF::Persistence::Encoder::operator <<<double,0x0>(U)' being compiled
        with
        [
            U=double
        ]

That the template values are <double,0x0> for operator<<(), while updateChecksumForNumber() would only be <double>, so I tried adding a std::is_arithmetic<> check to updateChecksumForNumber() to see if that fixes it.

I also moved the implementation of updateChecksumForNumber() into the class itself, although I don't think that matters.
Comment 31 David Kilzer (:ddkilzer) 2020-05-23 20:53:51 PDT
Created attachment 400149 [details]
Patch v14
Comment 32 David Kilzer (:ddkilzer) 2020-05-23 20:56:24 PDT
(In reply to David Kilzer (:ddkilzer) from comment #31)
> Created attachment 400149 [details]
> Patch v14

Remove extra template parameter to Encoder::updateChecksumForNumber() (although I've tried this before):

-            Encoder::updateChecksumForNumber<U>(m_sha1, value);
+            Encoder::updateChecksumForNumber(m_sha1, value);

Use unique template parameter for Encoder::updateChecksumForNumber() and Encoder::salt():

-    template <typename U, std::enable_if_t<std::is_arithmetic<U>::value>* = nullptr>
-    static void updateChecksumForNumber(SHA1& sha1, U value)
+    template <typename V, std::enable_if_t<std::is_arithmetic<V>::value>* = nullptr>
+    static void updateChecksumForNumber(SHA1& sha1, V value)

-    template <typename U> static constexpr unsigned salt();
+    template <typename V> static constexpr unsigned salt();

This probably won't matter.  Will just change the "U=U" message in the error to "U=V" for the WinCairo build.
Comment 33 David Kilzer (:ddkilzer) 2020-05-24 10:05:06 PDT
Created attachment 400165 [details]
Patch v15
Comment 34 David Kilzer (:ddkilzer) 2020-05-24 10:07:55 PDT
(In reply to David Kilzer (:ddkilzer) from comment #33)
> Created attachment 400165 [details]
> Patch v15

Move updateChecksumForData() and updateChecksumForNumber() into their own WTF::Persistence::Checksum class since they're used in both the Decoder and Encoder classes.  (The Checksum class could probably be moved to its own header/source files so that PersistentDecoder.h doesn't have to include PersistentEncoder.h.)
Comment 35 David Kilzer (:ddkilzer) 2020-05-24 10:22:15 PDT
Created attachment 400166 [details]
Patch v16
Comment 36 David Kilzer (:ddkilzer) 2020-05-24 10:23:57 PDT
(In reply to David Kilzer (:ddkilzer) from comment #35)
> Created attachment 400166 [details]
> Patch v16

Moved Checksum::updateChecksumForNumber() implementation out-of-line of the class in case it needs the Checksum::Salt<> templates to be defined first with MSVC++ (even though the error doesn't mention Checksum::Salt<> at all).
Comment 37 David Kilzer (:ddkilzer) 2020-05-24 11:01:57 PDT
Created attachment 400167 [details]
Patch v17

Template the whole WTF::Persistence::Checksum class.  I don't like this solution, but I want to see if the WinCairo build still fails.

I wonder if this whole issue has to do with trying to template a static class method in a header.  Technically you can only have one copy of the static method, so maybe that's what MSVC++ is actually complaining about.

In that case, we may just need to move the method out of a class and into a stand-alone function.
Comment 38 David Kilzer (:ddkilzer) 2020-05-24 12:14:19 PDT
(In reply to David Kilzer (:ddkilzer) from comment #37)
> I wonder if this whole issue has to do with trying to template a static
> class method in a header.  Technically you can only have one copy of the
> static method, so maybe that's what MSVC++ is actually complaining about.

Specifically, the issue may be due to calling a templated static method from a templated instance method since MSVC++ realizes that this could cause a scenario where multiple copies of the same static method could be created, thus causing a linker error later.

In Patch v18, I'll try adding `inline` to Checksum::updateChecksumForNumber() based on Patch v16.
Comment 39 David Kilzer (:ddkilzer) 2020-05-24 12:15:35 PDT
Created attachment 400173 [details]
Patch v18

Patch v16 with the addition of the `inline` keyword to Checksum::updateChecksumForNumber().
Comment 40 David Kilzer (:ddkilzer) 2020-05-24 12:32:42 PDT
Created attachment 400174 [details]
Patch v19
Comment 41 David Kilzer (:ddkilzer) 2020-05-24 12:34:22 PDT
(In reply to David Kilzer (:ddkilzer) from comment #40)
> Created attachment 400174 [details]
> Patch v19

Replace Checksum class with Checksum namespace so formerly static class methods may now be inlined as stand-alone functions.
Comment 42 David Kilzer (:ddkilzer) 2020-05-24 13:31:10 PDT
Created attachment 400176 [details]
Patch v20

Patch v20 is Patch v19 with a second template argument for Checksum::Salt:

-template <typename T>
+template <typename T, typename U = T>
 inline void updateChecksumForNumber(SHA1& sha1, T value)
 {
-    auto typeSalt = Salt<T>::value;
+    auto typeSalt = Salt<U>::value;
     sha1.addBytes(reinterpret_cast<uint8_t*>(&typeSalt), sizeof(typeSalt));
     sha1.addBytes(reinterpret_cast<uint8_t*>(&value), sizeof(value));
 }

Perhaps MSVC++ doesn't like to mix full (updateChecksumForNumber()) and partial (Salt) template specialization?
Comment 43 David Kilzer (:ddkilzer) 2020-05-25 08:06:29 PDT
Created attachment 400199 [details]
Patch v21

Patch v21 manually inlines Checksum::updateChecksumForNumber() to eliminate it to find out what MSVC++ will complain about next.  It is based on Patches v20 and v21.

I should have probably tried this about 10-15 patches ago.
Comment 44 David Kilzer (:ddkilzer) 2020-05-25 08:09:20 PDT
Created attachment 400200 [details]
Patch v22 - v19 with Checksum:: updateChecksumForNumber() inlined - WinCairo compiled!

Patch v22 is v21 with build fixes.  (Forgot to build locally first!)
Comment 45 David Kilzer (:ddkilzer) 2020-05-25 09:16:23 PDT
(In reply to David Kilzer (:ddkilzer) from comment #43)
> Created attachment 400199 [details]
> Patch v21
> 
> Patch v21 manually inlines Checksum::updateChecksumForNumber() to eliminate
> it to find out what MSVC++ will complain about next.  It is based on Patches
> v20 and v21.
> 
> I should have probably tried this about 10-15 patches ago.

So Patch v22 (v21 with build fixes) compiles with MSVC++ on WinCairo EWS bot.

This issue seems to have something to do with partial specialization of struct Salt vs. full specialization of Checksum::updateChecksumForNumber().

The more experiments I try, the more this seems like an MSVC++ compiler bug.
Comment 46 David Kilzer (:ddkilzer) 2020-05-25 09:20:59 PDT
Created attachment 400205 [details]
Patch v23

Patch v23 is Patch v19 with a fallback full template added for struct Salt (which is unsafe; if we use this fix, I'd change struct Salt back to a template function so we could static_assert() that the fallback is not implemented if something tries to use it):

+template <typename T> struct Salt { static constexpr unsigned value = static_cast<unsigned>(-1); };

Let's see if this compiles with MSVC++ on WinCairo.
Comment 47 David Kilzer (:ddkilzer) 2020-05-25 09:47:31 PDT
(In reply to Stephan Szabo from comment #22)
> As an observation from playing with it, with the Coder<Seconds> and
> Coder<WallTime> bodies in PersistentCoders.cpp rather than the .h, many of
> the errors went away, but the Coder<CertificateInfo> still failed (either in
> the h or the cpp) and it seems like the cases that fail are in files that
> were including CertificateInfo. I don't see why that'd be different than
> anything else that might be including the persistence files though.

The Source/WebCore/platform/network/curl/CertificateInfo.h header includes Source/WebCore/platform/network/CertificateInfoBase.h, which includes <wtf/Seconds.h>.  Note that <wtf/WallTime.h> also includes <wtf/Seconds.h>.

The interesting thing about <wtf/Seconds.h> is that it includes encode/decode methods that templatize the Encoder and Decoder classes, presumably to reuse the same code for Persistence::Encoder/Persistence::Decoder and IPC::Encoder/IPC::Decoder;

    template<class Encoder>
    void encode(Encoder& encoder) const
    {
        encoder << m_value;
    }

    template<class Decoder>
    static Optional<Seconds> decode(Decoder& decoder)
    {
        Optional<double> seconds;
        decoder >> seconds;
        if (!seconds)
            return WTF::nullopt;
        return Seconds(*seconds);
    }

    template<class Decoder>
    static WARN_UNUSED_RETURN bool decode(Decoder& decoder, Seconds& seconds)
    {
        double value;
        if (!decoder.decode(value))
            return false;

        seconds = Seconds(value);
        return true;
    }

I wonder if there are any other classes that do this?

Perhaps this issue is as simple as the <wtf/Seconds.h> header being included before the <wtf/persistence/Persistent{En,De}coder.h> headers, so MSVC++ claims that it can't find a template specialization?

We can test this theory by including <wtf/persistence/Persistent{De,En}coder.h> in <wtf/Seconds.h> to see if that magically fixes the build.
Comment 48 David Kilzer (:ddkilzer) 2020-05-25 09:48:10 PDT
Comment on attachment 400205 [details]
Patch v23

(In reply to David Kilzer (:ddkilzer) from comment #46)
> Created attachment 400205 [details]
> Patch v23
> 
> Patch v23 is Patch v19 with a fallback full template added for struct Salt
> (which is unsafe; if we use this fix, I'd change struct Salt back to a
> template function so we could static_assert() that the fallback is not
> implemented if something tries to use it):
> 
> +template <typename T> struct Salt { static constexpr unsigned value =
> static_cast<unsigned>(-1); };
> 
> Let's see if this compiles with MSVC++ on WinCairo.

Nope.  Did not compile on WinCairo.
Comment 49 David Kilzer (:ddkilzer) 2020-05-25 09:52:02 PDT
Created attachment 400207 [details]
Patch v24

Patch v24 is the same as Patch v19, but includes Persistent{De,En}coder.h in <wtf/Seconds.h> to try to fix the WinCairo build:

diff --git a/Source/WTF/wtf/Seconds.h b/Source/WTF/wtf/Seconds.h
index c1be3927382..ce281977d32 100644
--- a/Source/WTF/wtf/Seconds.h
+++ b/Source/WTF/wtf/Seconds.h
@@ -27,6 +27,8 @@
 
 #include <wtf/MathExtras.h>
 #include <wtf/Optional.h>
+#include <wtf/persistence/PersistentDecoder.h>
+#include <wtf/persistence/PersistentEncoder.h>
 
 namespace WTF {
 

See Comment #47 for details.
Comment 50 David Kilzer (:ddkilzer) 2020-05-25 10:05:06 PDT
(In reply to David Kilzer (:ddkilzer) from comment #49)
> Created attachment 400207 [details]
> Patch v24
> 
> Patch v24 is the same as Patch v19, but includes Persistent{De,En}coder.h in
> <wtf/Seconds.h> to try to fix the WinCairo build:
> 
> diff --git a/Source/WTF/wtf/Seconds.h b/Source/WTF/wtf/Seconds.h
> index c1be3927382..ce281977d32 100644
> --- a/Source/WTF/wtf/Seconds.h
> +++ b/Source/WTF/wtf/Seconds.h
> @@ -27,6 +27,8 @@
>  
>  #include <wtf/MathExtras.h>
>  #include <wtf/Optional.h>
> +#include <wtf/persistence/PersistentDecoder.h>
> +#include <wtf/persistence/PersistentEncoder.h>
>  
>  namespace WTF {
>  
> 
> See Comment #47 for details.

Of course that introduced a cyclical dependency in header includes, so we can't do that!
Comment 51 David Kilzer (:ddkilzer) 2020-05-25 10:34:23 PDT
Created attachment 400208 [details]
Patch v25

Patch v25 is the same as Patch v19, but does the following:

- Extracts WTF::Persistence::Checksum methods into their own header:  Source/WTF/wtf/persistence/PersistentChecksum.h.
- Adds PersistentChecksum.h to build CMake and Xcode build systems.
- Includes <wtf/persistence/PersistentChecksum.h> in <wtf/Seconds.h> to try to fix the WinCairo build.  See Comment #47 for details.
- Breaks cyclical header include (introduced by adding PersistentChecksum.h to Seconds.h) by moving SHA1::addBytes() that takes a CString from SHA1.h to SHA1.cpp.
- Has nice side benefit of removing include of <wtf/persistence/PersistentEncoder.h> from PersistentDecoder.h in favor of <wtf/persistence/PersistentChecksum.h>.
Comment 52 David Kilzer (:ddkilzer) 2020-05-26 10:28:05 PDT
Hi QuellaZhang, I CC-ed you on this radar because I've been trying to fix a build failure only seen on MSVC++ for the WinCairo port where gcc and clang (and an older MSVC++ for Apple's Windows port) work fine.

Via Slack, the WinCairo bots should be on MSVC 16.4:  <https://webkit.slack.com/archives/CU64U6FDW/p1589492625078800?thread_ts=1589490687.076600&cid=CU64U6FDW>

I will probably work around this by manually inlining the updateChecksumForNumber() method in the two places it's used, but I thought you might wand to take a look at this.
Comment 53 Fujii Hironori 2020-06-07 21:46:35 PDT
Created attachment 401311 [details]
v2 + OPENSSL_NO_SHA and OPENSSL_NO_SHA1

It seems that OpenSSL SHA1 function is conflicting with WTF::SHA1.
Defining OPENSSL_NO_SHA1 and OPENSSL_NO_SHA macros before including OpenSSL headers seems to solve the issue.