Bug 161212 - Web Inspector: unify Main.html and Test.html sources and generate different copies with the preprocessor
Summary: Web Inspector: unify Main.html and Test.html sources and generate different c...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Inspector (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: Brian Burg
URL:
Keywords: InRadar
Depends on: 161767 161573 161770 161772
Blocks:
  Show dependency treegraph
 
Reported: 2016-08-25 14:52 PDT by Brian Burg
Modified: 2017-11-08 03:37 PST (History)
11 users (show)

See Also:


Attachments
Proposed Fix - Make-only (124.83 KB, patch)
2016-08-29 11:02 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews101 for mac-yosemite (1.09 MB, application/zip)
2016-08-29 11:47 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews104 for mac-yosemite-wk2 (1.34 MB, application/zip)
2016-08-29 11:52 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews112 for mac-yosemite (857.44 KB, application/zip)
2016-08-29 12:01 PDT, Build Bot
no flags Details
Proposed Fix (Make-only) (125.25 KB, patch)
2016-08-29 17:04 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (CMake+Make) (146.60 KB, patch)
2016-08-30 16:07 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (CMake+Make) (146.62 KB, patch)
2016-08-30 16:15 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (CMake+Make) (147.18 KB, patch)
2016-08-30 20:11 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (CMake+Make) (147.42 KB, patch)
2016-08-30 20:17 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (CMake+Make) (147.43 KB, patch)
2016-08-31 16:22 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (except Windows) (163.94 KB, patch)
2016-09-01 14:25 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (164.03 KB, patch)
2016-09-01 21:00 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix - Debugging Win EWS (164.28 KB, patch)
2016-09-01 23:56 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix - Debugging Win EWS (164.27 KB, patch)
2016-09-02 09:44 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix - More EWS debugging (164.31 KB, patch)
2016-09-02 10:58 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix - More EWS debugging (164.56 KB, patch)
2016-09-02 12:38 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix - More EWS debugging (164.56 KB, patch)
2016-09-02 13:12 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix (164.02 KB, patch)
2016-09-02 16:48 PDT, Brian Burg
no flags Details | Formatted Diff | Diff
Proposed Fix, now with less VERBATIM (163.92 KB, patch)
2016-09-03 15:36 PDT, Brian Burg
commit-queue: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Burg 2016-08-25 14:52:11 PDT
Let's end the annoying syncing thing.
Comment 1 Radar WebKit Bug Importer 2016-08-25 14:52:36 PDT
<rdar://problem/28017961>
Comment 2 Brian Burg 2016-08-26 12:48:28 PDT
This is kind of blocked on moving at least some code generation to CMake + DerivedSources.make. I am not going to attempt to edit the Windows-only scripts.
Comment 3 Alex Christensen 2016-08-26 13:17:55 PDT
Windows uses CMake.  The internal build just blindly copies everything, which works after https://bugs.webkit.org/show_bug.cgi?id=161221 and we could probably consolidate the WebInspectorUI.vcxproj, but verifying it is annoying.  Don't worry about that.
Comment 4 Brian Burg 2016-08-29 11:02:54 PDT
Created attachment 287282 [details]
Proposed Fix - Make-only
Comment 5 Brian Burg 2016-08-29 11:04:56 PDT
Comment on attachment 287282 [details]
Proposed Fix - Make-only

This patch was tested on macOS Sierra engineering and production builds. Next up is getting CMake to work, I'll be prototyping on GTK port.
Comment 6 Build Bot 2016-08-29 11:46:58 PDT
Comment on attachment 287282 [details]
Proposed Fix - Make-only

Attachment 287282 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1966861

Number of test failures exceeded the failure limit.
Comment 7 Build Bot 2016-08-29 11:47:03 PDT
Created attachment 287295 [details]
Archive of layout-test-results from ews101 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 8 Build Bot 2016-08-29 11:52:28 PDT
Comment on attachment 287282 [details]
Proposed Fix - Make-only

Attachment 287282 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1966866

Number of test failures exceeded the failure limit.
Comment 9 Build Bot 2016-08-29 11:52:31 PDT
Created attachment 287299 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 10 Build Bot 2016-08-29 12:01:43 PDT
Comment on attachment 287282 [details]
Proposed Fix - Make-only

Attachment 287282 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1966857

Number of test failures exceeded the failure limit.
Comment 11 Build Bot 2016-08-29 12:01:46 PDT
Created attachment 287302 [details]
Archive of layout-test-results from ews112 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 12 Brian Burg 2016-08-29 17:04:59 PDT
Created attachment 287353 [details]
Proposed Fix (Make-only)
Comment 13 Brian Burg 2016-08-30 16:07:53 PDT
Created attachment 287445 [details]
Proposed Fix (CMake+Make)
Comment 14 Brian Burg 2016-08-30 16:15:05 PDT
Created attachment 287446 [details]
Proposed Fix (CMake+Make)
Comment 15 Brian Burg 2016-08-30 16:24:52 PDT
Comment on attachment 287446 [details]
Proposed Fix (CMake+Make)

View in context: https://bugs.webkit.org/attachment.cgi?id=287446&action=review

> Source/PlatformWin.cmake:12
> +    DEPENDS JavaScriptCore WebCore ${WebInspectorUI_RESOURCES}

Blah, I think we need a fake target in WebInspectorUI to force Main.html to be generated earlier. It can't be generated here because WebInspectorUI_RESOURCES is scoped to WebInspectorUI. Windows will have the same problem.
Comment 16 Brian Burg 2016-08-30 20:11:46 PDT
Created attachment 287472 [details]
Proposed Fix (CMake+Make)
Comment 17 Brian Burg 2016-08-30 20:17:29 PDT
Created attachment 287473 [details]
Proposed Fix (CMake+Make)
Comment 18 Brian Burg 2016-08-31 16:22:06 PDT
Created attachment 287555 [details]
Proposed Fix (CMake+Make)
Comment 19 Joseph Pecoraro 2016-08-31 23:08:47 PDT
Comment on attachment 287555 [details]
Proposed Fix (CMake+Make)

View in context: https://bugs.webkit.org/attachment.cgi?id=287555&action=review

r=me. Patch looks good to me.

It does seem as though there is a Legit Windows build error on the bot that needs to be addressed:

  Error copying file "C:/cygwin/home/buildbot/WebKit/WebKitBuild/Release/DerivedSources/WebInspectorUI/inspector/InspectorBackendCommands.js" to "C:/cygwin/home/buildbot/WebKit/WebKitBuild/Release/bin32/WebKit.resources/WebInspectorUI/Protocol".

I got a little lost on the GTK resource bundles, but it doesn't appear to have changed behavior.

> Source/WebInspectorUI/CMakeLists.txt:5
> +set(WebInspectorUI_RESOURCE_PATTERNS
> +    ${WEBINSPECTORUI_DIR}/UserInterface/*.html
> +    ${WEBINSPECTORUI_DIR}/UserInterface/Base/*.js

Any chance a double glob would work here?  (I realize this is not new, but just concerning)

    ${WEBINSPECTORUI_DIR}/UserInterface/*.html
    ${WEBINSPECTORUI_DIR}/UserInterface/**/*.js
    ${WEBINSPECTORUI_DIR}/UserInterface/**/*.css

This seems like it will be a real pain to maintain. I also have no idea what it is used for...

> Source/WebInspectorUI/CMakeLists.txt:44
> +# Copy InspectorBackendCommands.js from JavaScriptCore to WebInspectorUI.
> +add_custom_command(
> +    OUTPUT ${DERIVED_SOURCES_WEBINSPECTORUI_DIR}/UserInterface/Protocol/InspectorBackendCommands.js
> +    DEPENDS ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/inspector/InspectorBackendCommands.js
> +    COMMAND ${CMAKE_COMMAND} -E copy ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/inspector/InspectorBackendCommands.js ${DERIVED_SOURCES_WEBINSPECTORUI_DIR}/UserInterface/Protocol/InspectorBackendCommands.js
> +)

Any idea why the CMake ports do this? Why not just use the JavaScriptCore one? This just ends up getting copied again in Source/PlatformEfl.cmake.

> Source/WebInspectorUI/PlatformGTK.cmake:18
> +    COMMAND ${PYTHON_EXECUTABLE} ${TOOLS_DIR}/gtk/generate-inspector-gresource-manifest.py --output=${DERIVED_SOURCES_WEBINSPECTORUI_DIR}/InspectorGResourceBundle.xml ${WebInspectorUI_RESOURCES}

Wow, this is a lot of arguments on the command line. I hope this doesn't run into issues. (I realize this is not new, but just concerning)

> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:-170
> -    # Remove Debug JavaScript and CSS files in Production builds.

This was the only user of "--strip". That could be removed from Scripts/combine-resources.pl.

> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:254
> +    my $jsMinScript = File::Spec->catfile($sharedScriptsRoot, 'jsmin.py');
> +    my $cssMinScript = File::Spec->catfile($sharedScriptsRoot, 'cssmin.py');

If this uses the jsmin/cssmin from JavaScriptCore we should remove the ones in WebInspectorUI/Scripts. They don't appear to be used anymore.

> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:299
> +    copy(File::Spec->catfile($derivedSourcesDir, 'Main.html'), File::Spec->catfile($targetResourcePath, 'Main.html'));
> +    copy(File::Spec->catfile($derivedSourcesDir, 'Test.html'), File::Spec->catfile($targetResourcePath, 'Test.html'));

Do these fail if the file doesn't exist? That may be useful to catch a port having problems with generating the derived sources.

> Source/WebInspectorUI/Scripts/generate-webinspectorui-derived-sources:1
> +#!/bin/sh

No copyright? Not sure what is common for Build Phase scripts, but all the other scripts have one...

> Source/WebInspectorUI/Scripts/generate-webinspectorui-derived-sources:16
> +    make --no-builtin-rules -f "${WebInspectorUI}/DerivedSources.make" -j `/usr/sbin/sysctl -n hw.activecpu` SDKROOT=${SDKROOT}

This SDKROOT seems to come out of nowhere. Was it needed for something in particular, or just good to have / common?

> Source/WebInspectorUI/Scripts/preprocess-main-resource.pl:25
> +
> +use Getopt::Long;

We should `use strict`.

> Source/WebInspectorUI/Scripts/preprocess-main-resource.pl:31
> +$input = shift or die "Usage: $0 --defines=\"...\" FILENAME.html.in\n";

If using strict, this will need `my $input = ...`.

> Source/WebInspectorUI/UserInterface/Inspector.html.in:694
> +#endif // INCLUDE_UI_RESOURCES
> +
> +#if INCLUDE_UI_RESOURCES

These are the same. I guess it is worth keeping them separate blocks because they are separate blocks.

It would probably be nicer to move these INCLUDE_UI_RESOURCES Controllers down past the Managers. They don't seem to need before the non UI Controllers.

> Source/WebInspectorUI/WebInspectorUI.xcodeproj/project.pbxproj:40
>  		1C435CAC14E7B287004E10EA /* cssmin.py */ = {isa = PBXFileReference; lastKnownFileType = text.script.python; path = cssmin.py; sourceTree = "<group>"; };
>  		1C435CAD14E7B287004E10EA /* jsmin.py */ = {isa = PBXFileReference; lastKnownFileType = text.script.python; path = jsmin.py; sourceTree = "<group>"; };

These are the ones that can/should be removed.
Comment 20 Brian Burg 2016-09-01 10:33:23 PDT
Comment on attachment 287555 [details]
Proposed Fix (CMake+Make)

View in context: https://bugs.webkit.org/attachment.cgi?id=287555&action=review

Thanks for looking!

>> Source/WebInspectorUI/CMakeLists.txt:5
>> +    ${WEBINSPECTORUI_DIR}/UserInterface/Base/*.js
> 
> Any chance a double glob would work here?  (I realize this is not new, but just concerning)
> 
>     ${WEBINSPECTORUI_DIR}/UserInterface/*.html
>     ${WEBINSPECTORUI_DIR}/UserInterface/**/*.js
>     ${WEBINSPECTORUI_DIR}/UserInterface/**/*.css
> 
> This seems like it will be a real pain to maintain. I also have no idea what it is used for...

Right now, its used to generate a manifest for GResource. It's basically GTK's version of xxd.pl, but for multiple resources.

In the future, WebInspectorUI_RESOURCES will reflect any port-independent minification of files, so it could have just the combined files as input to the port-specific packaging.

>> Source/WebInspectorUI/CMakeLists.txt:44
>> +)
> 
> Any idea why the CMake ports do this? Why not just use the JavaScriptCore one? This just ends up getting copied again in Source/PlatformEfl.cmake.

A few loosely related reasons:

- projects outside JavaScriptCore can't see the custom command that generates inspector files, they can only see what's in derived sources.  So we would have to force the thing to be generated in JavaScriptCore anyway. A dependency in WebInspectorUI cannot be fulfilled as we don't know how to build it from outside JavaScriptCore.
- Internal Windows builds have trouble sharing files between projects (though this may be orthogonal)
- some port-specific packaging (specifically, GTK) expects all input files to have a few common root directories. If we don't copy it over, the relative path inspector/ will not match the expected path in the main resource UserInterface/Protocol/.

>> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:-170
>> -    # Remove Debug JavaScript and CSS files in Production builds.
> 
> This was the only user of "--strip". That could be removed from Scripts/combine-resources.pl.

ok

>> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:254
>> +    my $cssMinScript = File::Spec->catfile($sharedScriptsRoot, 'cssmin.py');
> 
> If this uses the jsmin/cssmin from JavaScriptCore we should remove the ones in WebInspectorUI/Scripts. They don't appear to be used anymore.

ok.

>> Source/WebInspectorUI/Scripts/copy-user-interface-resources.pl:299
>> +    copy(File::Spec->catfile($derivedSourcesDir, 'Test.html'), File::Spec->catfile($targetResourcePath, 'Test.html'));
> 
> Do these fail if the file doesn't exist? That may be useful to catch a port having problems with generating the derived sources.

no but they probably should. It would only catch Mac failures at the moment.

>> Source/WebInspectorUI/Scripts/generate-webinspectorui-derived-sources:1
>> +#!/bin/sh
> 
> No copyright? Not sure what is common for Build Phase scripts, but all the other scripts have one...

Oops, I'll add the block.

>> Source/WebInspectorUI/Scripts/generate-webinspectorui-derived-sources:16
>> +    make --no-builtin-rules -f "${WebInspectorUI}/DerivedSources.make" -j `/usr/sbin/sysctl -n hw.activecpu` SDKROOT=${SDKROOT}
> 
> This SDKROOT seems to come out of nowhere. Was it needed for something in particular, or just good to have / common?

I don't think it's needed, copypasta.

>> Source/WebInspectorUI/Scripts/preprocess-main-resource.pl:25
>> +use Getopt::Long;
> 
> We should `use strict`.

OK

>> Source/WebInspectorUI/UserInterface/Inspector.html.in:694
>> +#if INCLUDE_UI_RESOURCES
> 
> These are the same. I guess it is worth keeping them separate blocks because they are separate blocks.
> 
> It would probably be nicer to move these INCLUDE_UI_RESOURCES Controllers down past the Managers. They don't seem to need before the non UI Controllers.

I'll try it.

>> Source/WebInspectorUI/WebInspectorUI.xcodeproj/project.pbxproj:40
>>  		1C435CAD14E7B287004E10EA /* jsmin.py */ = {isa = PBXFileReference; lastKnownFileType = text.script.python; path = jsmin.py; sourceTree = "<group>"; };
> 
> These are the ones that can/should be removed.

ok
Comment 21 Brian Burg 2016-09-01 14:25:44 PDT
Created attachment 287679 [details]
Proposed Fix (except Windows)
Comment 22 Brian Burg 2016-09-01 21:00:56 PDT
Created attachment 287726 [details]
Proposed Fix
Comment 23 Joseph Pecoraro 2016-09-01 22:08:34 PDT
Comment on attachment 287726 [details]
Proposed Fix

View in context: https://bugs.webkit.org/attachment.cgi?id=287726&action=review

> Source/WebInspectorUI/UserInterface/Inspector.html.in:696
> +#if INCLUDE_UI_RESOURCES
> +
> +#endif // INCLUDE_UI_RESOURCES

This is empty and can be removed.
Comment 24 Brian Burg 2016-09-01 23:51:26 PDT
Still somehow fails on windows. It works locally. I'll try another clean build just to be sure. It looks like this part fails:

 Generating ../../DerivedSources/WebInspectorUI/UserInterface/Main.html
  Usage: C:/cygwin/home/buildbot/WebKit/Source/WebInspectorUI/Scripts/preprocess-main-resource.pl --defines="..." FILENAME.html.in
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 255. 


It would be way more helpful to see the actual invocation of the script to see what led it to have no leftover arguments.
Comment 25 Brian Burg 2016-09-01 23:56:28 PDT
Created attachment 287746 [details]
Proposed Fix - Debugging Win EWS
Comment 26 WebKit Commit Bot 2016-09-01 23:58:45 PDT
Attachment 287746 [details] did not pass style-queue:


ERROR: Source/WebInspectorUI/CMakeLists.txt:54:  One space between command "endforeach" and its parentheses, should be "endforeach ("  [whitespace/parentheses] [5]
Total errors found: 1 in 21 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 27 Brian Burg 2016-09-02 09:44:47 PDT
Created attachment 287766 [details]
Proposed Fix - Debugging Win EWS
Comment 28 WebKit Commit Bot 2016-09-02 09:47:06 PDT
Attachment 287766 [details] did not pass style-queue:


ERROR: Source/WebInspectorUI/CMakeLists.txt:54:  One space between command "endforeach" and its parentheses, should be "endforeach ("  [whitespace/parentheses] [5]
Total errors found: 1 in 21 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 29 Brian Burg 2016-09-02 10:58:57 PDT
Created attachment 287778 [details]
Proposed Fix - More EWS debugging
Comment 30 WebKit Commit Bot 2016-09-02 11:01:21 PDT
Attachment 287778 [details] did not pass style-queue:


ERROR: Source/WebInspectorUI/CMakeLists.txt:55:  One space between command "endforeach" and its parentheses, should be "endforeach ("  [whitespace/parentheses] [5]
Total errors found: 1 in 21 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 31 Brian Burg 2016-09-02 12:38:34 PDT
Created attachment 287800 [details]
Proposed Fix - More EWS debugging
Comment 32 WebKit Commit Bot 2016-09-02 12:40:30 PDT
Attachment 287800 [details] did not pass style-queue:


ERROR: Source/WebInspectorUI/CMakeLists.txt:55:  One space between command "endforeach" and its parentheses, should be "endforeach ("  [whitespace/parentheses] [5]
Total errors found: 1 in 21 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 33 Brian Burg 2016-09-02 13:12:54 PDT
Created attachment 287806 [details]
Proposed Fix - More EWS debugging
Comment 34 WebKit Commit Bot 2016-09-02 13:15:09 PDT
Attachment 287806 [details] did not pass style-queue:


ERROR: Source/WebInspectorUI/CMakeLists.txt:55:  One space between command "endforeach" and its parentheses, should be "endforeach ("  [whitespace/parentheses] [5]
Total errors found: 1 in 21 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 35 Brian Burg 2016-09-02 16:48:30 PDT
Created attachment 287835 [details]
Proposed Fix
Comment 36 Brian Burg 2016-09-03 15:36:09 PDT
Created attachment 287870 [details]
Proposed Fix, now with less VERBATIM
Comment 37 WebKit Commit Bot 2016-09-03 17:47:16 PDT
Comment on attachment 287870 [details]
Proposed Fix, now with less VERBATIM

Rejecting attachment 287870 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'land-attachment', '--force-clean', '--non-interactive', '--parent-command=commit-queue', 287870, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
    -> origin/master
Partial-rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc ...
Currently at 205412 = 1f7f8a2610dc5544a3ccd8596ceb5211a5a9ffac
r205413 = f80bb23180ba205719a98ede391f9b973733ded2
r205414 = 4f9556cfeebe3f738abbdcf28ea1f17a55018237
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.

Full output: http://webkit-queues.webkit.org/results/2003178
Comment 38 Brian Burg 2016-09-03 21:36:57 PDT
Committed r205415: <http://trac.webkit.org/changeset/205415>
Comment 39 WebKit Commit Bot 2016-09-03 23:55:24 PDT
Re-opened since this is blocked by bug 161573
Comment 40 Brian Burg 2016-09-04 14:31:14 PDT
The failure on bots was often:

WebInspectorUI.framework/Resources/TestCombined.js:11280: CONSOLE ERROR SyntaxError: Unexpected end of script
WebInspectorUI.framework/Resources/Test.html:39:21: CONSOLE ERROR ReferenceError: Can't find variable: WebInspector
Wait on notifyDone timed out, process com.apple.WebKit.WebContent.Development pid = 87730

So I think there was a stale resource or something was not built correctly. I cannot reproduce locally on trunk.
Comment 41 Brian Burg 2016-09-04 14:32:26 PDT
And for some reason EWS was not affected either, so my best guess right now is that a clean build is needed. I tried to force a clean build on a bot, but it just pulled down and archive and re-ran tests without rebuilding, so that's not very helpful.
Comment 42 Carlos Alberto Lopez Perez 2017-11-08 03:37:38 PST
(In reply to Brian Burg from comment #41)
> And for some reason EWS was not affected either, so my best guess right now
> is that a clean build is needed. I tried to force a clean build on a bot,
> but it just pulled down and archive and re-ran tests without rebuilding, so
> that's not very helpful.

You have to force a clean build on the "(Build)" bot so it generates a new build product. The "(Test)" one will download it later.

What I usually do in this cases is just to trigger clean builds on all the bots. I open a new tab per bot and then click-click on the clean-build form. Just in case. It ends working.

Just my 2 cents.