Confirmed.

Try init=/bin/stty -a, that will show

-iutf8

So there is little wonder that canonical mode does not work as expected.

Try init=/bin/sh, from that shell run stty iutf8. Then things will work
fine.  The fix is thus just to make the VT's tty initial iutf8 setup
follow vt.default_utf8.

Samuel
--

Will try that on a more recent install. Mine's stty does not support
this option. Your analysis makes quite a lot of sense, and such a fix
would wipe part of my annoyances/anger with this recent change.

Thanks,
Willy

--

Hello,


Can you give the patch below a try?
Dynamic per-VT utf-8 switch should also work, provided that you reopen
the VT (i.e. log out).

Samuel



Set IUTF8 as appropriate on VT tty open.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

--- linux/drivers/char/vt.c.orig	2008-05-04 00:37:50.000000000 +0100
+++ linux/drivers/char/vt.c	2008-05-04 00:47:39.000000000 +0100
@@ -2723,6 +2723,10 @@ static int con_open(struct tty_struct *t
 				tty->winsize.ws_row = vc_cons[currcons].d->vc_rows;
 				tty->winsize.ws_col = vc_cons[currcons].d->vc_cols;
 			}
+			if (vc->vc_utf)
+				tty->termios->c_iflag |= IUTF8;
+			else
+				tty->termios->c_iflag &= ~IUTF8;
 			release_console_sem();
 			vcs_make_sysfs(tty);
 			return ret;
@@ -2899,6 +2903,8 @@ int __init vty_init(void)
 	console_driver->minor_start = 1;
 	console_driver->type = TTY_DRIVER_TYPE_CONSOLE;
 	console_driver->init_termios = tty_std_termios;
+	if (default_utf8)
+		console_driver->init_termios.c_iflag |= IUTF8;
 	console_driver->flags = TTY_DRIVER_REAL_RAW | TTY_DRIVER_RESET_TERMIOS;
 	tty_set_operations(console_driver, &con_ops);
 	if (tty_register_driver(console_driver))
--

I guess this is suitable for the stable trees of 2.6.24 and 2.6.25
(where UTF-8 is by default now).




Set IUTF8 as appropriate on VT tty open.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

--- linux/drivers/char/vt.c.orig	2008-05-04 00:37:50.000000000 +0100
+++ linux/drivers/char/vt.c	2008-05-04 00:47:39.000000000 +0100
@@ -2723,6 +2723,10 @@ static int con_open(struct tty_struct *t
 				tty->winsize.ws_row = vc_cons[currcons].d->vc_rows;
 				tty->winsize.ws_col = vc_cons[currcons].d->vc_cols;
 			}
+			if (vc->vc_utf)
+				tty->termios->c_iflag |= IUTF8;
+			else
+				tty->termios->c_iflag &= ~IUTF8;
 			release_console_sem();
 			vcs_make_sysfs(tty);
 			return ret;
@@ -2899,6 +2903,8 @@ int __init vty_init(void)
 	console_driver->minor_start = 1;
 	console_driver->type = TTY_DRIVER_TYPE_CONSOLE;
 	console_driver->init_termios = tty_std_termios;
+	if (default_utf8)
+		console_driver->init_termios.c_iflag |= IUTF8;
 	console_driver->flags = TTY_DRIVER_REAL_RAW | TTY_DRIVER_RESET_TERMIOS;
 	tty_set_operations(console_driver, &con_ops);
 	if (tty_register_driver(console_driver))

--

On Sun, 4 May 2008 11:25:54 +0100

That changelog is pretty darn terse :(  I'll often go through
the email ladder and try to extract the missing information
but this time I don't really see it there.

Things like: what is the kernel's current behaviour, why does
it behave that way, how does the patch fix it?

Thanks.
--

Well, it's more an implementation than a fix. Let's try again:



For e.g. proper TTY canonical support, IUTF8 termios flag has to be set
as appropriate.  Linux used to not care about setting that flag for VT
TTYs.

This patch fixes that by activating it according to the current mode
of the VT, and sets the default value according to the vt.default_utf8
parameter.



Samuel
--

You should have added:  CC: stable@kernel.org here so that the stable
team automatically gets notified when it's merged into mainline.

Thanks!
Willy

--

Hi Samuel,


I confirm that your patch works perfectly for me. Now backspace correctly
removes multi-byte characters. My bash is still fooled though but as Alan
explained it, it's readline which has to be upgraded now.

Thanks!
Willy

--

Yes of course, but here the purpose was _not_ programs like bash, but
the canonical mode (i.e. programs like cat etc.), for which the LANG
variable has no effect, only iutf8 has.

Samuel
--

exactly, thanks for understanding my problem Samuel :-)

Willy
--

You may also need to select a UTF-8 locale (e.g. LANG="en_US.UTF-8") for
programs like bash to handle this correctly.

-- 
Chris Adams <cmadams@hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
--