login
Header Space

 
 

Re: When will OpenBSD support UTF8?

Previous thread: 4.2 Trouble with HP Notebook by Frans Haarman on Thursday, November 1, 2007 - 10:19 am. (9 messages)

Next thread: multipath routing with OpenBGPD by Florian Fuessl on Thursday, November 1, 2007 - 11:17 am. (3 messages)
To: openbsd-misc <misc@...>
Date: Thursday, November 1, 2007 - 10:51 am

Hi gang.

Is there any priority for having OpenBSD support UTF8?

// peter


      Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail.  Click on Options in Mail and switch to New Mail today or register for free at http://mail.yahoo.ca
To: openbsd-misc <misc@...>
Date: Thursday, November 1, 2007 - 11:22 am

utf-8 isn't an OS-level thing. You need to do it in every app.
Googling, the first result brings up
http://osdir.com/ml/os.openbsd.ports/2004-02/msg00376.html as an
example.
To: Nick Guenther <kousue@...>
Cc: openbsd-misc <misc@...>
Date: Friday, November 2, 2007 - 4:30 am

The sad thing is that the man pages don't mention that OpenBSD's libc
doesn't quite support locale, multibyte/wide char conversions thus
Unicode.

E.g. if you look at mbstowcs(3) you'd say: okay, I can use that... but
looking at the code behind it you'll see its a pure stab that does a
simple memcpy from chars to ints (or wchar_ts as they modernly call it
in C99).
To: Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>
Cc: openbsd-misc <misc@...>
Date: Friday, November 2, 2007 - 7:41 am

To get back to the practical nature of my original request, if someone
can let me know how to write French characters in a terminal (via an
SSH connection) I would be very grateful.  I would like to use a
terminal emulator that uses UTF8 and I believe xterm does this but I
can't find an OpenBSD package (or port) for it.

// juan


      Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail.  Click on Options in Mail and switch to New Mail today or register for free at http://mail.yahoo.ca
To: Juan Miscaro <scry_mr@...>
Cc: Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>, openbsd-misc <misc@...>
Date: Friday, November 2, 2007 - 4:03 pm

You would REALLY be surprised how much of a difference this `simple stub'
does... it allows us to compile *a lot* of code that helps support utf8
in ports land.

And in reality, this part of OpenBSD is C99-compliant. There's absolutely
*nothing* in the standard that says you have to support any locale except
the C locale (which we do).

If something has to be documented, it's probably that we just support
8 bit locales for now...

That said, this will eventually improve, and yes, this is a long road.

The xterm in OpenBSD can do it. It supports the utf8 option.

You will need an editor that supports utf8 characters as well.
Both vim and emacs do.

There are lots of programs in ports that have fairly decent level of
locale support. Heck, I can actually write japanese in OpenBSD, for
instance, and that's a *whole lot* more complicated than just french
characters.
To: <espie@...>
Cc: Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>, openbsd-misc <misc@...>
Date: Monday, November 5, 2007 - 5:06 pm

Thank you.  I also have a need to be able to write UTF-8 on my non-X
systems.  Do you have any thoughts on the matter?

// juan


      Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail.  Click on Options in Mail and switch to New Mail today or register for free at http://mail.yahoo.ca
To: Juan Miscaro <scry_mr@...>
Cc: Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>, openbsd-misc <misc@...>
Date: Monday, November 5, 2007 - 5:16 pm

We don't have a console that supports utf8 for now.
ncurses also needs some complicated update before it will
be able to deal with wide characters...

Things are not as easy as they would seem...
To: openbsd-misc <misc@...>
Cc: Juan Miscaro <scry_mr@...>, Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>
Date: Monday, November 5, 2007 - 7:16 pm

I'm sorry I can't just shut up and code, but if UTF-8 support is
planned, could it be implemented in such a way that it doesn't cause a
slow-down on old hardware if it is not needed.  I ran into that on
Debian when they defaulted to UTF-8.  Luckily, I was able to remove
their locales package after setting LANG=C; with locales and UTF-8
support, every app was very slow on my PII-233.

Doug.
To: openbsd-misc <misc@...>
Cc: Juan Miscaro <scry_mr@...>, Paul Irofti <bulibuta@...>, Nick Guenther <kousue@...>
Date: Friday, November 2, 2007 - 4:55 pm

Would supporting UTF-8 in OpenBSD change the apparent speed at which it
runs on older hardware?  Debian Etch does UTF-8 by default and it crawls
on my P-II; they told me it was because of UTF-8 and locales support.  I
changed the locale to C and removed the locales support and it speeded
up dramatically.  That type of tweaking of a base install isn't as easy
in OpenBSD.

Doug.
To: openbsd-misc <misc@...>
Date: Friday, November 2, 2007 - 12:55 pm

xterm comes with OpenBSD. just run 'xterm'
You could try rxvt (which is in packages) if that doesn't work for some reason.

-Nick
To: Nick Guenther <kousue@...>, openbsd-misc <misc@...>
Date: Thursday, November 1, 2007 - 12:37 pm

Thanks.

I saw that post before resorting to the list but as it was 3.5 years
ago I thought its info had a good chance of being outdated.
Previous thread: 4.2 Trouble with HP Notebook by Frans Haarman on Thursday, November 1, 2007 - 10:19 am. (9 messages)

Next thread: multipath routing with OpenBGPD by Florian Fuessl on Thursday, November 1, 2007 - 11:17 am. (3 messages)
speck-geostationary