Bug 21279 – Support ime-mode (CSS property)

• Component: CSS
• Status: NEW

[Jungshik Shin]

IE and Firefox3 support ime-mode to control the activation of input method in some text input field.

http://developer.mozilla.org/en/CSS/ime-mode

For instance, Google Sites use it to limit the set of characters in URL fields (well, it's too restrictive, but that's a separate issue). 

1. Go to sites.google.com
2. Create or open an existing site
3. From Home page, click on 'More actions' > 'Page settings'
4. Move focus to 'Page URL' text box

In IE and FF3, you cannot activate an input method because the field is styled with 'ime-mode: disabled'.

[Jungshik Shin]

Created attachment 23984 [details]
a simple test file

[Jungshik Shin]

It's http://crbug.com/4490 as well. 

[Jungshik Shin]

In IE, it's either -ime-mode or -ms-ime-mode. 

Related to (but separate from) this is 'Windows InputScope' support ( http://blogs.msdn.com/tsfaware/archive/2007/07/10/input-scopes.aspx
http://msdn.microsoft.com/en-us/library/ms698133.aspx ), but I'm not sure if that belongs to bugs.webkit.org (perhaps more likely to belong to each embedder's bug tracker). 

[Kenichi Ishibashi]

Hi, I'd like to implement CSS ime-mode property to WebKit. I started this on SnowLeopard. I'm going to send a patch after this comment.

Here are some notes about the patch:
- The patch uses Text Input Source Services, which is a part of Carbon's HIToolbox. Actually, I don't like to use Carbon framework, but it seems that there is no APIs on Cocoa for handling input methods. (Please let me know if there are APIs for handling input methods in Cocoa. I'll be happy to re-implement by using it.)
- The patch will work on Mac OS v10.5 or later because the patch depends on the Text Input Source Services, which was introduced in Mac OS X v10.5
- The patch added a method into WebHTMLView class to force exposing input context because it is needed to make TSMGetActiveDocument() return non-nil, which is required to enable/disable input methods correctly. I'm not sure this is the right way so I'd like to ask some advice on this.
- I'm also planning to implement the ime-mode property for Windows (and hopefully Linux) once I could successfully add supporting this property on Mac.

Thanks,

[Kenichi Ishibashi]

Created attachment 66681 [details]
Proposed Patch V0

[Alexey Proskuryakov]

What is the use case for ime-mode? The one provided in bug description seems like a bad idea - I definitely don't want Web pages to disable input methods in URL fields. A page does not know what input method available on the platform do - for example, Kotoeri Romaji input mode on the Mac is used exactly to type Roman characters, as in URLs, and there is no reason to switch out of Kotoeri just for that.

[Kenichi Ishibashi]

(In reply to comment #6)

Hi,

Thank you for your response. As you mentioned (and MDC<https://developer.mozilla.org/en/CSS/ime-mode> also noted), it's not appropriate in general to control the activation of input method. However, it would be helpful for developers who have to develop a web application which works as front end of a system that accepts only certain input(e.g. ASCII only or non-ASCII only text). In fact (and unfortunately), such systems are fairly common in Japan, at least as far as I know. Providing a common-used way for controling activation of input method (because IE and FF3 support this property) might be helpful for developing such web applications.

[Alexey Proskuryakov]

OK, but doesn't preventing input methods mean that
1) Kotoeri Romaji won't work,
2) but other keyboard layouts (such as Russian) will still work?

Neither is desirable in your use case.

[Kenichi Ishibashi]

(In reply to comment #8)
> OK, but doesn't preventing input methods mean that
> 1) Kotoeri Romaji won't work,
> 2) but other keyboard layouts (such as Russian) will still work?
> 
> Neither is desirable in your use case.

Sorry for lack of description. The patch doesn't disable Kotoeri Romaji input mode even if a field is styled with 'ime-mode: disable' since the input mode is so called "ASCII-capable (This term used in the reference document on ADC) input mode," which behaves just like ASCII keyboard. The patch disables only non ASCII-capable input methods when the field is styled with "ime-mode: disabled." So, in that case, Kotoeri Romaji input mode doesn't switch out while Russian input mode do (Russian input mode is not ASCII-capable).

[Kenichi Ishibashi]

Yet another use case is an address book application on mobile phones. Typical address book applications on modern mobile phones for CJK people automatically enables/disables input mode depending on the input field. For example, input method is automatically enabled when the user should input the address, and is disabled automatically when the user should input the phone number. Once WebKit supports ime-mode property, mobile phones which uses WebKit as UI component such as iPhone or Android can provide easy way to implement this behavior. IMHO, since this behavior seems to be familiar with CJK people, supporting ime-mode property might be helpful for iPhone and Android developers.

[Hironori Bono]

(In reply to comment #9)
> (In reply to comment #8)
> > OK, but doesn't preventing input methods mean that
> > 1) Kotoeri Romaji won't work,
> > 2) but other keyboard layouts (such as Russian) will still work?
> > 
> > Neither is desirable in your use case.
> 
> Sorry for lack of description. The patch doesn't disable Kotoeri Romaji input mode even if a field is styled with 'ime-mode: disable' since the input mode is so called "ASCII-capable (This term used in the reference document on ADC) input mode," which behaves just like ASCII keyboard. The patch disables only non ASCII-capable input methods when the field is styled with "ime-mode: disabled." So, in that case, Kotoeri Romaji input mode doesn't switch out while Russian input mode do (Russian input mode is not ASCII-capable).

Even though I'm not a WebKit reviewer (just a claimer), I'm interested in the behavior of your change when we use a JIS keyboard in an element which has 'ime-mode: disable;' property.

Regards,

Hironori Bono

[Kenichi Ishibashi]

(In reply to comment #11)

Bono-san,

> Even though I'm not a WebKit reviewer (just a claimer), I'm interested in the behavior of your change when we use a JIS keyboard in an element which has 'ime-mode: disable;' property.

Thank you for the question. I also should mention that. Basically, the behavior of this change doesn't depend on the physical keyboard layout but on input mode that currently used because we can use any input mode regardless of the physical keyboard layout on Mac. In an element which has 'ime-mode: disabled' property, the input mode will switch out when we use non ASCII-capable input mode (otherwise not), regardless of the keyboard layout that we use.

Regards,

[Alexey Proskuryakov]

> because IE and FF3 support this property

Are you aware of any Web sites that are misbehaving due to our lack of support for ime-mode?

> e.g. ASCII only or non-ASCII only text

You are saying that this is a common issue in Japan. Could you please provide some examples? It doesn't seem i18n-friendly to provide a way to special case ASCII, and only ASCII. In fact, I can't think of many examples where text should be ASCII-only. Even when typing code in some programming language, there can be non-ASCII string literals, at least.

> For example, input method is automatically enabled when the user should input the address, and is
> disabled automatically when the user should input the phone number.

If the actual use case is switching to a numeric input mode in mobile devices, then it doesn't seem like we should be making any changes on desktop. And why forcefully disable Russian keyboard? It's perfectly good for phone numbers.

In HTML5, there is <input type=tel>, and a mobile browser should be able to recognize that, and set input mode accordingly if needed.

[Kenichi Ishibashi]

(In reply to comment #13)

Hi Alexey,

Thank you for your comments.

> Are you aware of any Web sites that are misbehaving due to our lack of support for ime-mode?

Here is a web page that uses ime-mode property (it's written in Japanese, sorry for your inconvenience):

https://www.aeoncredit.co.jp/NetBranch/cardinit.do

This page is the registration form for becoming a member of a credit card club. Most of input elements in this page are styled with ime-mode property because the system requires users to provide information with certain format. For instance, name of the user must contain only Katakana (it is quite common to force providing one's name with Katakana or Hiragana in Japan to identify how to read his/her name). On the other hand, the card number and the phone number must provide as ASCII characters (I think it's not appropriate because they can automatically convert non-ASCII digits characters to ASCII digits characters, but unfortunately such restriction is also commonly used in Japanese pages).

> You are saying that this is a common issue in Japan. Could you please provide some examples? It doesn't seem i18n-friendly to provide a way to special case ASCII, and only ASCII. In fact, I can't think of many examples where text should be ASCII-only. Even when typing code in some programming language, there can be non-ASCII string literals, at least.

I think above page is could be an example. Please note that ime-mode property not only for to force disabling input methods. IMHO, ime-mode property should be used for providing suitable input mode by default, rather than forcing enabling/disabling input methods. We can provide suitable default input mode by using 'ime-mode: active' or 'ime-mode: inactive' and even if these style are specified, the user can turn on/off the input method whenever the user want.

In addition, there are many demands on controlling input methods can be seen on the web. I've searched with query "html 入力制限" (means "html input restrict") and got about 2,800,000 results, and most of search results on the first page are related with controlling input methods.

> If the actual use case is switching to a numeric input mode in mobile devices, then it doesn't seem like we should be making any changes on desktop. And why forcefully disable Russian keyboard? It's perfectly good for phone numbers.

Sorry for leading to misunderstanding, but mobile devices is one of an example. As mentioned above, there are also demands on desktop web applications.

> In HTML5, there is <input type=tel>, and a mobile browser should be able to recognize that, and set input mode accordingly if needed.

<input type="tel"> is very useful, but it's not a substitute of ime-mode property because ime-mode not only for inputting numbers.

[Alexey Proskuryakov]

> name of the user must contain only Katakana

Thank you for the example!

How does one input Katakana without using an IM, if they have a U.S. keyboard? That's of course possible with Kotoeri IM, but only if input methods are not disabled.

> providing suitable input mode by default

Is ime-mode capable of providing Russian keyboard layout by default, for fields that are expected to be in Russian?

><input type="tel"> is very useful, but it's not a substitute of ime-mode property because ime-mode not only for inputting numbers.

I think that it also serves as an example of why ime-mode would be harmful. Although pages may want to disable input methods for telephone numbers, a better answer to this is the opposite - a browser would provide a custom input method, e.g. one that lets the user look up phone numbers in address book, or at least format them on the fly.

At this point, I think that this discussion should really go to WHATWG mailing list. I'm almost certain that we don't want to provide an explict way to play with OS input method mechanisms, but there may be other ways to solve the problems that need solving.

[Hironori Bono]

These are just random thoughts of mine after reading the discussion in this page. Please feel free to ignore them.

As for the usage of this "ime-mode" property, I thought of a somewhat crazy scenario: using input methods on non-editable elements, such as <canvas> elements. For example, when implementing an on-line game with <canvas> elements, I would like to enable input methods and to receive composition events so we can print composed characters with fillText().

By the way, to test this "ime-mode" property on IE, it does not change the keyboard layout but it just disables the IMEs attached to the keyboard. That is, we can type Cyrillic characters even in an input elememt whose "ime-mode" value is "disabled" when we use Russian keyboards. To emulate this behavior, I'm wondering if we should control TIS and change the keyboard layout as we do for password fields. (In terms of Cocoa, this "ime-mode" property just controls whether we should call [interpretKeyEvents:] in [WebHTMLView _interceptEditingKeyEvent:shouldSaveCommand:]?)

Regards,

Hironori Bono

[Kenichi Ishibashi]

Hi Alexey,

Thank you for discussing about this matter. I'm grateful to you.

> How does one input Katakana without using an IM, if they have a U.S. keyboard? That's of course possible with Kotoeri IM, but only if input methods are not disabled.

I might misunderstand your point, but IMs will not be disabled when the input element requires Katakana and they can use Kotoeri IM even if they use U.S. keyboard.

> Is ime-mode capable of providing Russian keyboard layout by default, for fields that are expected to be in Russian?

Basically, ime-mode property just turn on/off input methods. When an element which is styled with "ime-mode: active" gets focus, the current implementation of the patch works as follows:
- if there is the previously-used input method, turn it on.
- otherwise, looking for an input method based on the languages that the system preferred.
Therefore, when the system prefers Russian than other languages, Russian keyboard layout would activate by default.

> I think that it also serves as an example of why ime-mode would be harmful. Although pages may want to disable input methods for telephone numbers, a better answer to this is the opposite - a browser would provide a custom input method, e.g. one that lets the user look up phone numbers in address book, or at least format them on the fly.

I absolutely agree with you that we should use <input type="tel"> rather than ime-mode property for that purpose. However, I think that providing custom input methods also harmful because it would increase the cost of developing web pages that I mentioned in the previous comment.

> At this point, I think that this discussion should really go to WHATWG mailing list. I'm almost certain that we don't want to provide an explict way to play with OS input method mechanisms, but there may be other ways to solve the problems that need solving.

Thank you for suggesting that. It might be better to follow your suggestion. I should post this matter to WHATWG.

Regards,

[Kenichi Ishibashi]

(In reply to comment #16)

Bono-san,

> As for the usage of this "ime-mode" property, I thought of a somewhat crazy scenario: using input methods on non-editable elements, such as <canvas> elements. For example, when implementing an on-line game with <canvas> elements, I would like to enable input methods and to receive composition events so we can print composed characters with fillText().

Thank you for providing another possible usage of ime-mode property. I didn't realize that, but it's definitely a good example of using ime-mode property!

> By the way, to test this "ime-mode" property on IE, it does not change the keyboard layout but it just disables the IMEs attached to the keyboard. That is, we can type Cyrillic characters even in an input elememt whose "ime-mode" value is "disabled" when we use Russian keyboards. To emulate this behavior, I'm wondering if we should control TIS and change the keyboard layout as we do for password fields. (In terms of Cocoa, this "ime-mode" property just controls whether we should call [interpretKeyEvents:] in [WebHTMLView _interceptEditingKeyEvent:shouldSaveCommand:]?)

Thank you for letting me know that. I'm going to investigate it.

[Alexey Proskuryakov]

Comment on attachment 66681 [details]
Proposed Patch V0

Marking r- to get the patch out of review queue while it's being discussed whether we want ime-mode support.

[Jungshik Shin]

Just to help the discussion going:  

Mozilla's ime-mode was implemented by Masayuki Nakano. 

See https://bugzilla.mozilla.org/show_bug.cgi?id=279246

hi everyone

There will be an important defect on the keypress,keydown,keyup events , if users open their IMEs, because ime will eat  these key events. 
So please add it to Webkit.

Try this example on Webkit based browser:
If ime off, the example work well.
If ime on, the example can not work.


<input id='castId' size="6" type="text" maxlength="6" onkeyup="checkCastId(this)" style="ime-mode: disabled;" />
<button id="openCastIdBtn" disabled="disabled">Open Cast</button>
<script>
		function checkCastId(me){
			console.log(me.value);
			if(/[6-9]{6}/.test(me.value)){
				document.getElementById("openCastIdBtn").disabled = "";
			}
			else{
				document.getElementById("openCastIdBtn").disabled = "disabled";
			}
		}
</script>

[Kent Tamura]

http://www.w3.org/TR/2012/WD-css3-ui-20120117/#ime-mode

ime-mode is going to be standardized.

[Kang-Hao (Kenny) Lu]

(In reply to comment #22)
> http://www.w3.org/TR/2012/WD-css3-ui-20120117/#ime-mode
> 
> ime-mode is going to be standardized.

Though that section is pretty much copied from MDN, and it even contains less info than the MDN entry for 'ime-mode'.

Having said that, I just want to mention that I sent a review of this property to www-style[1]. Just in case anyone is interested in sending comments. I'll note that this is probably the first mail ever about 'ime-mode' on www-style. So sad.

I read previous discussions on webkit-dev about this property[2] and it seems that there are two potential actionable items for WebKit people:

* Say WebKit's is never going to implement 'ime-mode' and ask if Firefox and IE would drop it, or ask the editor to either drop this from css3-ui or postpone to the next level.
* Ask the editor to add a sentence in the description of 'disabled': "UAs SHOULD/MAY treat this value as 'inactive'". Whether that's a SHOULD or MAY is going to be debatable and I have no opinion because I don't know the use cases of this property (yet).

In any case, I can't speak for WebKit people so feel free to raise some discussions there!

[1] http://lists.w3.org/Archives/Public/www-style/2012Feb/0759
[2] https://lists.webkit.org/pipermail/webkit-dev/2010-October/014613.html

[James Craig]

(In reply to comment #22)
> http://www.w3.org/TR/2012/WD-css3-ui-20120117/#ime-mode
> 
> ime-mode is going to be standardized.

It's status is listed as "at-risk", which means it may be dropped from the specification. Given the number of objections, it probably won't be standardized in its current form.

My $0.02 addition.
http://lists.w3.org/Archives/Public/www-style/2013May/0075.html

[Takayoshi Kochi]

(In reply to comment #24)
> (In reply to comment #22)
> > http://www.w3.org/TR/2012/WD-css3-ui-20120117/#ime-mode
> > 
> > ime-mode is going to be standardized.
> 
> It's status is listed as "at-risk", which means it may be dropped from the specification. Given the number of objections, it probably won't be standardized in its current form.
> 
> My $0.02 addition.
> http://lists.w3.org/Archives/Public/www-style/2013May/0075.html

For the case that web pages want to *disable* IME may be covered by
other input form validation or HTML inputmode attribute, but is there
any good alternative to explicitly *enable* IME?

For example, a form has your email address and your name fields,
IME should be turned off for email and turned on for name.

One automatic mode change can save huge amount in total considering
the number of IME users, instead of imposing users on obvious and
tedious manual mode change on users.

[Alexey Proskuryakov]

> For example, a form has your email address and your name fields,
> IME should be turned off for email and turned on for name.

This is incorrect. For example, there is absolutely no reason to disable handwriting input methods for e-mail field. In fact, even Asian input methods are appropriate for non-ASCII addresses, which are perfectly valid.

[Takayoshi Kochi]

(In reply to comment #26)
> > For example, a form has your email address and your name fields,
> > IME should be turned off for email and turned on for name.
> 
> This is incorrect. For example, there is absolutely no reason to disable handwriting input methods for e-mail field. In fact, even Asian input methods are appropriate for non-ASCII addresses, which are perfectly valid.

In terms of 'validity' you are correct, but if you consider actual usage
(if you count mobile users) People using handwriting or
Asian input methods for email address is pretty rare,
though I don't have concrete number handy.

ime-mode is not a property to both force a fixed mode
but it still allows user to change mode unless it is 'disabled'.

This is about setting default modality hint for convenience for most users.

To count handwriting input methods as an IME is a separate issue.
On some mobile OS it may consider English keyboard is an IME, therefore
'ime-mode:disabled' does not make sense.

The original intention for this was for traditional CJK input methods
on desktop OSs.  So there are many counter examples for modern mobile OSs or
even desktop OSs.

Has there been any developments in this context?