Bug 128742

Summary: [GTK] Build failure caused by missing jsmin module
Product: WebKit Reporter: ChangSeok Oh <changseok>
Component: WebKitGTKAssignee: ChangSeok Oh <changseok>
Status: RESOLVED FIXED    
Severity: Normal CC: berto, calvaris, commit-queue, gustavo, mrobinson, pnormand, zan
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch none

Description ChangSeok Oh 2014-02-13 06:52:41 PST
../../autogen.sh --prefix=/usr/local --enable-gles2 --enable-egl --enable-developer-mode;make -j4

  GEN      DerivedSources/WebCore/XMLNames.cpp
  GEN      DerivedSources/WebCore/XMLNSNames.cpp
  GEN      DerivedSources/WebCore/UserAgentStyleSheets.h
Traceback (most recent call last):
  File "../../Source/WebCore/Scripts/make-js-file-arrays.py", line 28, in <module>
    from jsmin import JavascriptMinify
ImportError: No module named jsmin
make: *** [DerivedSources/WebCore/UserAgentScripts.h] Error 1
make: *** Waiting for unfinished jobs....
mkdir -p DerivedSources/JavaScriptCore/inspector 

Perhaps https://bugs.webkit.org/show_bug.cgi?id=127559 is related?
Comment 1 ChangSeok Oh 2014-02-13 06:59:42 PST
Created attachment 224067 [details]
Patch
Comment 2 ChangSeok Oh 2014-02-13 21:27:34 PST
I've tested with python3 for a while. But no luck there too. Another failure happend like following.
  GEN      DerivedSources/WebCore/XLinkNames.cpp
  GEN      DerivedSources/WebCore/XMLNames.cpp
  File "../../Source/WebCore/Scripts/make-js-file-arrays.py", line 48
    print 'Error: must provide at least 3 arguments'
                                                   ^
SyntaxError: invalid syntax
make: *** [DerivedSources/WebCore/UserAgentScripts.h] Error 1
make: *** Waiting for unfinished jobs....
  GEN      DerivedSources/WebCore/InjectedScriptCanvasModuleSource.h

But this can also be resolved with the patch.
I guess if gtk bots uses cmake build system, they may not complain.
Comment 3 ChangSeok Oh 2014-02-13 23:11:28 PST
o.k I found an another clue, https://bugs.webkit.org/show_bug.cgi?id=123097
MediaControlScript affects the failure.
Comment 4 ChangSeok Oh 2014-02-14 01:46:48 PST
Created attachment 224180 [details]
Patch
Comment 5 ChangSeok Oh 2014-02-14 01:50:50 PST
@calvaris, Isn't this what you intended?

> $(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
Comment 6 Xabier Rodríguez Calvar 2014-02-14 03:07:07 PST
Comment on attachment 224180 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review

> Source/WebCore/GNUmakefile.am:271
> -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
>  DerivedSources/WebCore/UserAgentScripts.h: $(WebCore)/Scripts/make-js-file-arrays.py $(USER_AGENT_SCRIPT_FILES)
> -	$(AM_V_GEN)$(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> +	$(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)

I intented that, yes, but for some reason I couldn't make it work because I was prepending the PYTHONPATH to AM_V_GEN and that didn't work. If this does, it makes sense.

Anyway, is there a problem with this? According something I read, this was the recommended way of enabling an environment variable for only one make target.
Comment 7 ChangSeok Oh 2014-02-14 09:07:36 PST
(In reply to comment #6)
> (From update of attachment 224180 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> 
> > Source/WebCore/GNUmakefile.am:271
> > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> >  DerivedSources/WebCore/UserAgentScripts.h: $(WebCore)/Scripts/make-js-file-arrays.py $(USER_AGENT_SCRIPT_FILES)
> > -	$(AM_V_GEN)$(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> > +	$(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> 
> I intented that, yes, but for some reason I couldn't make it work because I was prepending the PYTHONPATH to AM_V_GEN and that didn't work. If this does, it makes sense.
> 
> Anyway, is there a problem with this? According something I read, this was the recommended way of enabling an environment variable for only one make target.

Yes. The failure still happens on my system, Ubuntu 13.10 using python 2.7.x
It's really odd that gtk bot doesn't catch this failure if it uses autotool as default. I tried to build after bumping python up to 3.x. It was not helpful, on the contrary it causes two other problem
1. some webkit scripts in ./Tools are not working with python 3.x
2. Bumping the version just causes another break while generating DerivedSources

Anyway the make-us-xxx script needs jsmin.py so we should indicate where it is or add $(INSPECTOR...)/jsmin.py explicitly as an argument of python.
Comment 8 Philippe Normand 2014-02-17 02:56:53 PST
Comment on attachment 224180 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review

> Source/WebCore/GNUmakefile.am:-270
> -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)

How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?
Comment 9 Xabier Rodríguez Calvar 2014-02-17 03:04:49 PST
(In reply to comment #8)
> (From update of attachment 224180 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> 
> > Source/WebCore/GNUmakefile.am:-270
> > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> 
> How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?

I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.
Comment 10 Philippe Normand 2014-02-17 05:58:48 PST
(In reply to comment #9)
> (In reply to comment #8)
> > (From update of attachment 224180 [details] [details])
> > View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> > 
> > > Source/WebCore/GNUmakefile.am:-270
> > > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> > 
> > How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?
> 
> I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.

Ah ok I didn't know about that trick... Perhaps the GNU make version ChangSeok uses has an issue with that syntax? Anyway, the patch looks harmless to me.
Comment 11 Alberto Garcia 2014-02-17 06:13:26 PST
Not quite related to this, but it would also be awesome to replace jsmin with a more recent version:

https://bugs.webkit.org/show_bug.cgi?id=123665
Comment 12 ChangSeok Oh 2014-02-17 08:09:33 PST
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > (From update of attachment 224180 [details] [details] [details])
> > > View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> > > 
> > > > Source/WebCore/GNUmakefile.am:-270
> > > > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> > > 
> > > How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?
> > 
> > I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.
> 
> Ah ok I didn't know about that trick... Perhaps the GNU make version ChangSeok uses has an issue with that syntax? Anyway, the patch looks harmless to me.

Yes. you guys are right. The system GNU make I used was 3.81 which is a little old, so that the grammar Xabier wrote was unrecognizable(even though it was correct!). If we build through build-webkit script. the script set Dependencies/Root/bin to the PATH env variable. So actually make used is the one in Dependencies/Root/bin, the version 3.82. That's why gtk bots doesn't complain. :P
I like a tolerant approach, But I will not commit this if somebody feels this change unnecessary. :)
Comment 13 Xabier Rodríguez Calvar 2014-02-17 08:14:23 PST
(In reply to comment #12)
> Yes. you guys are right. The system GNU make I used was 3.81 which is a little old, so that the grammar Xabier wrote was unrecognizable(even though it was correct!). If we build through build-webkit script. the script set Dependencies/Root/bin to the PATH env variable. So actually make used is the one in Dependencies/Root/bin, the version 3.82. That's why gtk bots doesn't complain. :P
> I like a tolerant approach, But I will not commit this if somebody feels this change unnecessary. :)

I don't mind changing it as long as it works.
Comment 14 ChangSeok Oh 2014-02-17 11:11:50 PST
Comment on attachment 224180 [details]
Patch

I checked this hurt nothing for make 3.81 & 3.82. Thanks for the r+!
Comment 15 WebKit Commit Bot 2014-02-17 11:43:19 PST
Comment on attachment 224180 [details]
Patch

Clearing flags on attachment: 224180

Committed r164239: <http://trac.webkit.org/changeset/164239>
Comment 16 WebKit Commit Bot 2014-02-17 11:43:22 PST
All reviewed patches have been landed.  Closing bug.