Bug 261712 – Add an official Searchfox instance for WebKit

• Component: WebKit Website
• Status: NEW

[sideshowbarker]

https://searchfox.org/wubkat/source/ is a great search tool known and used by a lot of WebKit contributors — but not well-known or used much be anyone outside the project. There are times when others outside the project — in particular, other implementors — want to search through the WebKit sources. For a concrete example, see the comment at https://github.com/whatwg/html/issues/3699#issuecomment-1724114438 where a Blink/Chrome implementor says, “I didn't see any equivalent code in Gecko or WebKit after a quick look, though I admit I don't have good source code search tools for WebKit so I'm definitely less sure there”.

Perhaps a “Sources” tab could be added to the top menu: Downloads | Feature Status | Documentation | Sources | Policies | Contribute | Blog

…where the Sources tab expands to show “Sources in GitHub” (https://github.com/WebKit/WebKit/) and “Search the sources” (https://searchfox.org/wubkat/source/) links

[Emilio Cobos Álvarez (:emilio)]

If that's going to be something semi-official we should probably rename it from /wubkat to /webkit. The misnaming was to make emphasis that it wasn't quite official :)

[Alexey Proskuryakov]

Agreed, it's clearly not meant to be official (not yet?)

I'm also curious about why benefits it has over GitHub.

[sideshowbarker]

(In reply to Alexey Proskuryakov from comment #2)
 
> I'm also curious about why benefits it has over GitHub.

@ntim or @rego could speak to that better than I could personally

[Ahmad Saleem]

1) Faster to search 'comments' compared to GitHub Search
2) Shows function name in the search output to make it easier to find
3) Separate field to filter on file types

^ Some of benefits for which reason, I use it.

[Karl Dubost]

Created attachment 467770 [details]
symbolic search

In my own usage

Wubkat:
* for symbol search, aka following functions and tracking them with symbolic references  
* Make it easy to share a search


GitHub:
* Useful for blame search 
* Linking to precise PRs/commits.
* Having the Review comments.


My local code editor 
Useful for me and has the capabilities to do the things above BUT you can't share with a link the results of your searches. 

For example, compare
* https://searchfox.org/wubkat/search?q=symbol:E_%3CT_WebCore%3A%3ACSSSelector%3A%3APseudoElementType%3E_PseudoElementFirstLetter&redirect=false
* https://github.com/search?q=repo%3AWebKit%2FWebKit%20PseudoElementFirstLetter&type=code

[Diego Pino]

The main difference is that Searchfox builds the source code while GitHhub doesn't. My understanding is that GitHub only supports textual search, which Searchfox also supports through livegrep.

Building the sources allows Searchfox to build a dictionary of symbols. That allows symbol highlighting and symbol navigation. Also, since the source code is built searches include generated files.

Searchfox also provides blame information per line of code. Hovering on the left side of a file shows a popup window with blame information. It wouldn't be hard to tune that pop-up window to show a 'Show bug' link or a link to the corresponding GitHub pull-request.

Lastly, notice that the WebKit setup of Searchfox builds WebKitGTK with certain settings (https://github.com/mozsearch/mozsearch-mozilla/blob/master/wubkat/build#L45). That means that code paths or files that depend on having certain features enabled won't be compiled, that means there's no symbol highlighting or symbol navigation for those files or code paths. The same applies for port specific files other than WebKitGTK.

[Radar WebKit Bug Importer]

<rdar://problem/116028978>

[Andrew Sutherland]

Created attachment 468114 [details]
example searchfox diagram of `calls-between:'FrameLoader::loadInSameDocument' calls-between:'dispatchWindowEvent'`

Searchfox has a few other capabilities that are also worth calling out, with apologies that I wrote this assuming this bugzilla instance supported markdown:

- (experimental) Diagrams!  For example, the attached image is of what I currently see for the query [calls-between:'FrameLoader::loadInSameDocument' calls-between:'dispatchWindowEvent'](https://searchfox.org/wubkat/query/default?q=calls-between%3A%27FrameLoader%3A%3AloadInSameDocument%27+calls-between%3A%27dispatchWindowEvent%27).  (The source of this choice of symbols is from https://googleprojectzero.blogspot.com/2022/06/an-autopsy-on-zombie-in-wild-0-day.html)  Interestingly, the diagram has gotten a lot more complex since [the last time I screenshotted the diagram](https://www.visophyte.org/blog/wp-content/uploads/2022/11/20221109-233521-713x1536.png).
   - Note that currently diagram functionality on exists on searchfox.org as an opt-in via https://searchfox.org/mozilla-central/pages/settings.html where the user must set their "default feature gate" to "alpha" or the "diagramming feature gate" to "alpha".  Although technically the functionality is pre-alpha.
   - Enhancements to diagrams are coming soon, for example, on my searchfox hobby branch that should be folded into trunk during 2023H2, we have (all Gecko mozilla-central examples, but these would potentially apply to webkit when landed):
     - Inheritance diagrams like: [inheritance-diagram:nsGlobalWindowInner](https://asuth.searchfox.org/mozilla-central/query/default?q=inheritance-diagram%3AnsGlobalWindowInner)
     - Class diagrams like: [class-diagram:'mozilla::dom::ReadableStreamDefaultReader' depth:4](https://asuth.searchfox.org/mozilla-central/query/default?q=class-diagram%3A%27mozilla%3A%3Adom%3A%3AReadableStreamDefaultReader%27+depth%3A4)
- First-class understanding of IDL languages and their bindings and related semantics like IPC send/receive pairs.  For example the query [calls-between:'ClientSource::Focus' calls-between:'WindowClient_Binding::focus' depth:11](https://asuth.searchfox.org/mozilla-central/query/default?q=calls-between%3A%27ClientSource%3A%3AFocus%27+calls-between%3A%27WindowClient_Binding%3A%3Afocus%27+depth%3A11) demonstrates searchfox understanding the control flow from a WebIDL binding[1] call made in a content process, traversing IPC into the parent process, and then traversing IPC back into a content process.
   - This doesn't just benefit diagrams.  Search results and context-menus when clicking on symbols in source listings benefit as well.
- I blogged last year at https://www.visophyte.org/blog/2022/11/10/andrews-searchfox-roadmap-2022/ that potentially provides some other context and directions searchfox is headed.

1: Note that searchfox currently does not understand Gecko's WebIDL in a first-class way, just XPIDL and IPDL, although WebIDL support isn't far off (but would not automatically work for webkit).

[Alexey Proskuryakov]

Yes, these are pretty substantial differences. I'm going to retitle though, as this seems a bigger undertaking than adding a link.