Bug 128742 – [GTK] Build failure caused by missing jsmin module

• Component: WebKitGTK
• Status: RESOLVED FIXED

[ChangSeok Oh]

../../autogen.sh --prefix=/usr/local --enable-gles2 --enable-egl --enable-developer-mode;make -j4

  GEN      DerivedSources/WebCore/XMLNames.cpp
  GEN      DerivedSources/WebCore/XMLNSNames.cpp
  GEN      DerivedSources/WebCore/UserAgentStyleSheets.h
Traceback (most recent call last):
  File "../../Source/WebCore/Scripts/make-js-file-arrays.py", line 28, in <module>
    from jsmin import JavascriptMinify
ImportError: No module named jsmin
make: *** [DerivedSources/WebCore/UserAgentScripts.h] Error 1
make: *** Waiting for unfinished jobs....
mkdir -p DerivedSources/JavaScriptCore/inspector 

Perhaps https://bugs.webkit.org/show_bug.cgi?id=127559 is related?

[ChangSeok Oh]

Created attachment 224067 [details]
Patch

[ChangSeok Oh]

I've tested with python3 for a while. But no luck there too. Another failure happend like following.
  GEN      DerivedSources/WebCore/XLinkNames.cpp
  GEN      DerivedSources/WebCore/XMLNames.cpp
  File "../../Source/WebCore/Scripts/make-js-file-arrays.py", line 48
    print 'Error: must provide at least 3 arguments'
                                                   ^
SyntaxError: invalid syntax
make: *** [DerivedSources/WebCore/UserAgentScripts.h] Error 1
make: *** Waiting for unfinished jobs....
  GEN      DerivedSources/WebCore/InjectedScriptCanvasModuleSource.h

But this can also be resolved with the patch.
I guess if gtk bots uses cmake build system, they may not complain.

[ChangSeok Oh]

o.k I found an another clue, https://bugs.webkit.org/show_bug.cgi?id=123097
MediaControlScript affects the failure.

[ChangSeok Oh]

Created attachment 224180 [details]
Patch

[ChangSeok Oh]

@calvaris, Isn't this what you intended?

> $(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)

[Xabier Rodríguez Calvar]

Comment on attachment 224180 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review

> Source/WebCore/GNUmakefile.am:271
> -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
>  DerivedSources/WebCore/UserAgentScripts.h: $(WebCore)/Scripts/make-js-file-arrays.py $(USER_AGENT_SCRIPT_FILES)
> -	$(AM_V_GEN)$(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> +	$(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)

I intented that, yes, but for some reason I couldn't make it work because I was prepending the PYTHONPATH to AM_V_GEN and that didn't work. If this does, it makes sense.

Anyway, is there a problem with this? According something I read, this was the recommended way of enabling an environment variable for only one make target.

[ChangSeok Oh]

(In reply to comment #6)
> (From update of attachment 224180 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> 
> > Source/WebCore/GNUmakefile.am:271
> > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> >  DerivedSources/WebCore/UserAgentScripts.h: $(WebCore)/Scripts/make-js-file-arrays.py $(USER_AGENT_SCRIPT_FILES)
> > -	$(AM_V_GEN)$(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> > +	$(AM_V_GEN) PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR) $(PYTHON) $< $@ DerivedSources/WebCore/UserAgentScriptsData.cpp $(USER_AGENT_SCRIPT_FILES)
> 
> I intented that, yes, but for some reason I couldn't make it work because I was prepending the PYTHONPATH to AM_V_GEN and that didn't work. If this does, it makes sense.
> 
> Anyway, is there a problem with this? According something I read, this was the recommended way of enabling an environment variable for only one make target.

Yes. The failure still happens on my system, Ubuntu 13.10 using python 2.7.x
It's really odd that gtk bot doesn't catch this failure if it uses autotool as default. I tried to build after bumping python up to 3.x. It was not helpful, on the contrary it causes two other problem
1. some webkit scripts in ./Tools are not working with python 3.x
2. Bumping the version just causes another break while generating DerivedSources

Anyway the make-us-xxx script needs jsmin.py so we should indicate where it is or add $(INSPECTOR...)/jsmin.py explicitly as an argument of python.

[Philippe Normand]

Comment on attachment 224180 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review

> Source/WebCore/GNUmakefile.am:-270
> -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)

How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?

[Xabier Rodríguez Calvar]

(In reply to comment #8)
> (From update of attachment 224180 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> 
> > Source/WebCore/GNUmakefile.am:-270
> > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> 
> How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?

I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.

[Philippe Normand]

(In reply to comment #9)
> (In reply to comment #8)
> > (From update of attachment 224180 [details] [details])
> > View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> > 
> > > Source/WebCore/GNUmakefile.am:-270
> > > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> > 
> > How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?
> 
> I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.

Ah ok I didn't know about that trick... Perhaps the GNU make version ChangSeok uses has an issue with that syntax? Anyway, the patch looks harmless to me.

[Alberto Garcia]

Not quite related to this, but it would also be awesome to replace jsmin with a more recent version:

https://bugs.webkit.org/show_bug.cgi?id=123665

[ChangSeok Oh]

(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > (From update of attachment 224180 [details] [details] [details])
> > > View in context: https://bugs.webkit.org/attachment.cgi?id=224180&action=review
> > > 
> > > > Source/WebCore/GNUmakefile.am:-270
> > > > -DerivedSources/WebCore/UserAgentScripts.h: PYTHONPATH=$(INSPECTOR_SCRIPTS_DIR)
> > > 
> > > How could this work? :) Perhaps build-webkit somehow sets the PYTHONPATH, I suppose you don't use it, ChangSeok?
> > 
> > I read it here: http://stackoverflow.com/questions/15229833/set-environment-variable-inside-target and it worked for me and for the bots, it seems.
> 
> Ah ok I didn't know about that trick... Perhaps the GNU make version ChangSeok uses has an issue with that syntax? Anyway, the patch looks harmless to me.

Yes. you guys are right. The system GNU make I used was 3.81 which is a little old, so that the grammar Xabier wrote was unrecognizable(even though it was correct!). If we build through build-webkit script. the script set Dependencies/Root/bin to the PATH env variable. So actually make used is the one in Dependencies/Root/bin, the version 3.82. That's why gtk bots doesn't complain. :P
I like a tolerant approach, But I will not commit this if somebody feels this change unnecessary. :)

[Xabier Rodríguez Calvar]

(In reply to comment #12)
> Yes. you guys are right. The system GNU make I used was 3.81 which is a little old, so that the grammar Xabier wrote was unrecognizable(even though it was correct!). If we build through build-webkit script. the script set Dependencies/Root/bin to the PATH env variable. So actually make used is the one in Dependencies/Root/bin, the version 3.82. That's why gtk bots doesn't complain. :P
> I like a tolerant approach, But I will not commit this if somebody feels this change unnecessary. :)

I don't mind changing it as long as it works.

[ChangSeok Oh]

Comment on attachment 224180 [details]
Patch

I checked this hurt nothing for make 3.81 & 3.82. Thanks for the r+!

[WebKit Commit Bot]

Comment on attachment 224180 [details]
Patch

Clearing flags on attachment: 224180

Committed r164239: <http://trac.webkit.org/changeset/164239>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.