Kbd-1.15 doesn't map U+2010 to U+002d in lat1-16 font

[Matthew Burgess]

As shown in the screenshot at ​http://www.linuxfromscratch.org/~matthew/test-results/preconv.en_utf8.jpg, if one is in a Unicode/UTF-8 environment, the line-breaking character that is automatically generated by man/groff is not displayed correctly. This has been reported upstream at ​http://lists.altlinux.org/pipermail/kbd/2009-February/000162.html.

[Matthew Burgess]

This isn't actually a Groff issue, it's just the fact that my font, lat1-16, doesn't contain the symbol U+2010 (UTF-8 \xe2 \x80 \x90).

If I am reading ​http://homepage.ntlworld.com/zarniwhoop/consolefonts/sigma.html correctly, a console font can only contain 512 glyphs (including bold variations of the same character).

Ken, any ideas if it's possible to work out from the .psfu file, whether there's space to add another couple of glyphs to lat1-16? GCC spits out some '?' too, when it prints some fancy warnings out, so it'd be nice to get that symbol added, while we're at it.

Or maybe we should add sigma-consolefonts to the book (assuming it contains the correct glyphs)? :)

[ken@…]

Replying to matthew@…:

This isn't actually a Groff issue, it's just the fact that my font, lat1-16, doesn't contain the symbol U+2010 (UTF-8 \xe2 \x80 \x90).

If I am reading ​http://homepage.ntlworld.com/zarniwhoop/consolefonts/sigma.html correctly, a console font can only contain 512 glyphs (including bold variations of the same character).

Ken, any ideas if it's possible to work out from the .psfu file, whether there's space to add another couple of glyphs to lat1-16? GCC spits out some '?' too, when it prints some fancy warnings out, so it'd be nice to get that symbol added, while we're at it.

Or maybe we should add sigma-consolefonts to the book (assuming it contains the correct glyphs)? :)

I no longer remember how to look at .psfu files - in the past, I googled for something like "psf file format" and found enough to be able to read whatever I was looking for. I know that the current kbd maintainer takes patches, but I don't know how to edit these fonts. I'm guessing fontforge might be the right tool.

Sigma-consolefonts maps U+2010 (hyphen), as well as U+2012, U+2013, U+2212, to U+002D (minus sign) in all of the screen font maps except 'drdos'. Possibly, you can do the same addition of multiple characters for the same glyph using 'dd' once you've worked out the format ;-)

The attachment is the prog I wrote to list what was in a psf. Dunno if it will help.

As to whether the book should use sigma-consolefonts, I have no idea. I'm sure many people will hate the output, either because it is a lot less bold, or because the more useful mappings are 512-byte and that screws up their bold colours.

I haven't looked at the trunk version of the book yet, but previously we used to do

sed -i -e 's/2010/002D/' -e 's/2212/002D/' \
       -e 's/2018/0060/' -e 's/2019/0027/' font/devutf8/R.proto

before we configured groff to get rid of these awkward characters.

[Matthew Burgess]

Hi Ken,

That sed used to be in the book until groff-1.20.1 went in. As of that version, the sed does nothing, as the format/conventions used in that file appear to have completely changed. I couldn't work out how to adapt the sed to work, but I suspect that's what we'd need to do again, or get lat1-16 fixed up. I've emailed Alexey Gladkov (kbd maintainer) and will report back as and when I hear back.

[Matthew Burgess]

The fix here is to do the following (thanks to Lefteris Dimitroulakis on the kbd list):

cd /lib/kbd/consolefonts
gunzip lat1-16.psfu.gz
psfgettable lat1-16.psfu
sed -i -e 's/U+002d/& U+2010 U+2012 U+2013 U+2212/' table.txt
psfaddtable lat1-16.psfu table.txt lat1-16.psfu.new
mv lat1-16.psfu.new lat1-16.psfu
gzip lat1-16.psfu

I'm not recommending this for the book, although if others feel it would be beneficial, I certainly won't mind putting it in. I worry that we might have to audit all of the existing psfu fonts to ensure that they correctly map to the hyphen character though. The resultant psfu.gz file has been submitted upstream and should be downloadable from ​http://lists.altlinux.org/pipermail/kbd/2009-February/000164.html

We may also want to figure out what the funny character is that GCC prints out in some of its warnings, and map that to a backtick or single quote.