Bug 199835 - [WHLSL] checkRecursiveType should not have exponential complexity.
Summary: [WHLSL] checkRecursiveType should not have exponential complexity.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGPU (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Robin Morisset
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-07-16 12:49 PDT by Robin Morisset
Modified: 2019-07-17 19:16 PDT (History)
5 users (show)

See Also:


Attachments
Patch (4.21 KB, patch)
2019-07-16 14:22 PDT, Robin Morisset
no flags Details | Formatted Diff | Diff
Patch (4.21 KB, patch)
2019-07-16 15:08 PDT, Robin Morisset
mmaxfield: review+
mmaxfield: commit-queue-
Details | Formatted Diff | Diff
Patch (4.22 KB, patch)
2019-07-16 15:57 PDT, Robin Morisset
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from webkit-cq-01 for mac-highsierra (3.39 MB, application/zip)
2019-07-17 14:18 PDT, WebKit Commit Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Morisset 2019-07-16 12:49:25 PDT
Consider the case where you have:

typedef A1 = X1;
typedef A2 = X1;
...
typedef An = X1;
typedef X1 = X2;
typedef X2 = X3;
typedef X3 = X4;
...
typedef XnMinusOne = Xn;
typedef Xn = int;

Currently we will go n times through the entire chain X1 -> X2 -> ... -> Xn -> int
So our total runtime will be in n^2, while the program size is only proportional to n.

This is easy to fix, just do a proper DFS, not revisiting parts of the graph we've already explored.

Note: this is basically the same problem, and the same solution, as https://bugs.webkit.org/show_bug.cgi?id=199688.
Comment 1 Robin Morisset 2019-07-16 14:22:14 PDT
Created attachment 374240 [details]
Patch
Comment 2 EWS Watchlist 2019-07-16 14:24:20 PDT
Attachment 374240 [details] did not pass style-queue:


ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:107:  Missing space around : in range-based for statement  [whitespace/colon] [4]
ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:109:  Missing space around : in range-based for statement  [whitespace/colon] [4]
Total errors found: 2 in 2 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 Robin Morisset 2019-07-16 15:08:24 PDT
Created attachment 374246 [details]
Patch
Comment 4 EWS Watchlist 2019-07-16 15:10:37 PDT
Attachment 374246 [details] did not pass style-queue:


ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:107:  Missing space around : in range-based for statement  [whitespace/colon] [4]
ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:109:  Missing space around : in range-based for statement  [whitespace/colon] [4]
Total errors found: 2 in 2 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 5 Myles C. Maxfield 2019-07-16 15:43:32 PDT
Comment on attachment 374246 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=374246&action=review

> Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:68
> +#define START_VISITING(t) \
> +do { \
> +    if (m_finishedVisiting.contains(t)) \
> +        return; \
> +    auto resultStartedVisiting = m_startedVisiting.add(t); \
> +    if (!resultStartedVisiting.isNewEntry) { \
> +        setError(); \
> +        return; \
> +    } \
> +} while (false);
> +
> +#define END_VISITING(t) \
> +do { \
> +    auto resultFinishedVisiting = m_finishedVisiting.add(t); \
> +    ASSERT_UNUSED(resultFinishedVisiting, resultFinishedVisiting.isNewEntry); \
> +} while (false);

Can we make this general so we can share it with RecursionChecker? And maybe put it in ScopedSetAdder.h?

Also, can we use templates instead of preprocessor macros?
Comment 6 Myles C. Maxfield 2019-07-16 15:44:57 PDT
Comment on attachment 374246 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=374246&action=review

> Source/WebCore/ChangeLog:3
> +        [WHLSL] checkRecursiveType should not have quadratic complexity.

I think it's exponential, not quadratic
Comment 7 Myles C. Maxfield 2019-07-16 15:48:53 PDT
Comment on attachment 374246 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=374246&action=review

> Source/WebCore/ChangeLog:8
> +        The change is very similar to that in https://bugs.webkit.org/show_bug.cgi?id=199698.

I don't think this is the right URL
Comment 8 Robin Morisset 2019-07-16 15:50:24 PDT
As noted by Myles, the worst case is actually exponential:
struct A1 {
  A2 x;
  A2 y;
}
struct A2 {
  A3 x;
  A3 y;
}
...
struct AnMinusOne {
  An x;
  An y;
}
typedef An = int;

would traverse An 2^n times.
Comment 9 Robin Morisset 2019-07-16 15:51:02 PDT
(In reply to Myles C. Maxfield from comment #7)
> Comment on attachment 374246 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=374246&action=review
> 
> > Source/WebCore/ChangeLog:8
> > +        The change is very similar to that in https://bugs.webkit.org/show_bug.cgi?id=199698.
> 
> I don't think this is the right URL

Thanks, the right URL is actually https://bugs.webkit.org/show_bug.cgi?id=199688.
I will fix the Changelog.
Comment 10 Robin Morisset 2019-07-16 15:57:10 PDT
Created attachment 374253 [details]
Patch
Comment 11 Robin Morisset 2019-07-16 15:58:53 PDT
(In reply to Myles C. Maxfield from comment #5)
> Comment on attachment 374246 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=374246&action=review
> 
> > Source/WebCore/Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:68
> > +#define START_VISITING(t) \
> > +do { \
> > +    if (m_finishedVisiting.contains(t)) \
> > +        return; \
> > +    auto resultStartedVisiting = m_startedVisiting.add(t); \
> > +    if (!resultStartedVisiting.isNewEntry) { \
> > +        setError(); \
> > +        return; \
> > +    } \
> > +} while (false);
> > +
> > +#define END_VISITING(t) \
> > +do { \
> > +    auto resultFinishedVisiting = m_finishedVisiting.add(t); \
> > +    ASSERT_UNUSED(resultFinishedVisiting, resultFinishedVisiting.isNewEntry); \
> > +} while (false);
> 
> Can we make this general so we can share it with RecursionChecker? And maybe
> put it in ScopedSetAdder.h?
> 
> Also, can we use templates instead of preprocessor macros?

It is not obvious how to make it both clean and not use preprocessor macros because it needs to be able to return early. One option would be to make it a higher-order function that takes a lambda that would contain the actual body of the function.
I thought it was more effort than it was worth (and likely even less readable than macros), but I can do the change. What do you think should be done? This, something else, or just keep the macros?
Comment 12 Robin Morisset 2019-07-17 12:59:42 PDT
Comment on attachment 374253 [details]
Patch

Myles said ok offline to landing this patch.
Comment 13 WebKit Commit Bot 2019-07-17 14:04:30 PDT
The commit-queue encountered the following flaky tests while processing attachment 374253 [details]:

http/tests/IndexedDB/collect-IDB-objects.https.html bug 199881 (author: youennf@gmail.com)
The commit-queue is continuing to process your patch.
Comment 14 WebKit Commit Bot 2019-07-17 14:04:31 PDT
The commit-queue encountered the following flaky tests while processing attachment 374253 [details]:

The commit-queue is continuing to process your patch.
Comment 15 WebKit Commit Bot 2019-07-17 14:18:03 PDT
Comment on attachment 374253 [details]
Patch

Rejecting attachment 374253 [details] from commit-queue.

New failing tests:
storage/indexeddb/dont-wedge.html
Full output: https://webkit-queues.webkit.org/results/12760805
Comment 16 WebKit Commit Bot 2019-07-17 14:18:05 PDT
Created attachment 374329 [details]
Archive of layout-test-results from webkit-cq-01 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the commit-queue.
Bot: webkit-cq-01  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 17 Robin Morisset 2019-07-17 15:44:49 PDT
Comment on attachment 374253 [details]
Patch

Let's try again, I don't see how I can have caused a bug in IndexedDB.
Comment 18 WebKit Commit Bot 2019-07-17 17:36:46 PDT
Comment on attachment 374253 [details]
Patch

Clearing flags on attachment: 374253

Committed r247549: <https://trac.webkit.org/changeset/247549>
Comment 19 WebKit Commit Bot 2019-07-17 17:36:48 PDT
All reviewed patches have been landed.  Closing bug.
Comment 20 Radar WebKit Bug Importer 2019-07-17 17:37:21 PDT
<rdar://problem/53230131>
Comment 21 Simon Fraser (smfr) 2019-07-17 18:46:31 PDT
This broke my build:

In file included from /Volumes/Data/Development/system/webkit/OpenSource/WebKitBuild/Release-iphoneos/DerivedSources/WebCore/unified-sources/UnifiedSource164.cpp:1:
./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:107:40: error: member access into incomplete type 'WebCore::WHLSL::Program'
    for (auto& typeDefinition : program.typeDefinitions())
                                       ^
In file included from /Volumes/Data/Development/system/webkit/OpenSource/WebKitBuild/Release-iphoneos/DerivedSources/WebCore/unified-sources/UnifiedSource164.cpp:1:
In file included from ./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:27:
./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.h:35:7: note: forward declaration of 'WebCore::WHLSL::Program'
class Program;
      ^
Comment 22 Simon Fraser (smfr) 2019-07-17 19:01:11 PDT
Build fix in https://trac.webkit.org/r247553
Comment 23 Robin Morisset 2019-07-17 19:16:00 PDT
(In reply to Simon Fraser (smfr) from comment #21)
> This broke my build:
> 
> In file included from
> /Volumes/Data/Development/system/webkit/OpenSource/WebKitBuild/Release-
> iphoneos/DerivedSources/WebCore/unified-sources/UnifiedSource164.cpp:1:
> ./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:107:40: error: member
> access into incomplete type 'WebCore::WHLSL::Program'
>     for (auto& typeDefinition : program.typeDefinitions())
>                                        ^
> In file included from
> /Volumes/Data/Development/system/webkit/OpenSource/WebKitBuild/Release-
> iphoneos/DerivedSources/WebCore/unified-sources/UnifiedSource164.cpp:1:
> In file included from
> ./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.cpp:27:
> ./Modules/webgpu/WHLSL/WHLSLRecursiveTypeChecker.h:35:7: note: forward
> declaration of 'WebCore::WHLSL::Program'
> class Program;
>       ^

Sorry for this mistake, for some reason it worked fine for me and the commit queue.
Thank you for the build fix.