Force Pidgin online

The Network Mananger 0.9 series has made some changes which break current Pidgin. After I installed network-manager 0.8.999 Pidgin won’t connect, stalling with “waiting for network connection”.

Turns out there is a workaround in Pidgin: you can force it to ignore what it thinks network availability by running it as:

$ pidgin -f

There’s no GUI way in gnome-shell to edit a launcher at the moment, fine; The old “edit menus” trick didn’t seem to work either. So to do that manually:

$ cp /usr/share/applications/pidgin.desktop .local/share/applications/
$ vi .local/share/applications/pidgin.desktop

And change the Exec line to:

Exec=pidgin -f

It won’t take effect until the desktop recaches things. Reload gnome-shell by typing “r” in the Alt+F2 run dialog and you’ll be on your way.

I’m sure upstream will catch up with the Network Manager changes in due course but I can live without network availability detection for now and this gets me back online.

I’ve been using Empathy for instant messaging for a long time, but I still love Pidgin for IRC. Go figure. So two clients it is.

AfC

Upgraded PGP key

I have commenced using key id 0x5CB48AEA with fingerprint:

    7EFA 6058 DECC 8E97 F39D  2EC4 D500 10FB 5CB4 8AEA

you can download the key from my website or [modulo syncing] from one of the public key servers.

This supersedes key 0x57F6E7BD. I have no grounds to believe that the old key was compromised. While it’s disappointing to begin retiring such a well connected key (top 100!) it’s getting on in years and for the same reason as everyone else, it’s high time I generated a stronger one.

I have signed the new key with my old one.

AfC

java-gnome 4.0.19 released

This post is an extract of the release note from the NEWS file which you can read online … or in the sources from Bazaar.


java-gnome 4.0.19 (14 Feb 2011)

What do you mean that’s not the font I asked for?

This release includes some minor feature enhancements.

Preliminary coverage of Pango’s Font object. Font is Pango’s abstraction describing a typeface, and is what is actually loaded. We’ve exposed the methods that allow you to find out what was actually loaded for a given FontDescription request. You do this with Context’s loadFont() and then Font’s describe(). Thanks to Behdad Esfahbod for explaining how all this works.

Exposed a few utility functions, including one to find out if your program is running in a terminal or from the Desktop directly.

GTK improvements

Further improved some corner cases involved in using Actions, and now you can make them with named Icons.

There are some odd corner cases, especially with TextView, where idle handlers need to run before you have the calculations you need ready to query. One workaround appears to be letting the main loop cycle, so we’ve exposed Gtk.mainIterationDo() and the Gtk.eventsPending() which wraps it.

Build improvements

Building java-gnome on Mandriva now works! Thanks to Liam Quin for helping QA the top level configure script.


You can download java-gnome’s sources from ftp.gnome.org_, or easily checkout a branch from_ ‘mainline:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

AfC

Loading extensions into SQLite

It’s fairly common in financial systems to represent monetary amounts as cents rather than trying to represent dollars and cents with floating point [and all the rounding errors and inaccuracy that comes from using base 2 floating point to represent concrete base 10 numbers]. Call it fixed point if you like. Anyway, in an SQLite database I’m working with, the schema represents amounts as integer numbers of cents.

  3995

is

$39.95

I found myself constantly having to convert a number of cents to a number with two decimal places. Would you believe I came up with this?

SELECT
date(timestamp),
CASE
WHEN amount = 0 THEN
    '0.00'
WHEN amount % 10 = 0 THEN
    CAST (amount / 100.0 AS TEXT) || '0'
ELSE
    CAST (amount / 100.0 AS TEXT)
END,
description
FROM ...

I mean, cool that you can do that with SQL, but yikes. What I really wanted was a SQL function that would just do the conversion from cents to money, all nicely formatted.

SELECT date(timestamp), money(amount), description FROM ...

SQLite has a very straight-forward extension mechanism. Admittedly I already had a grip on the basics of the SQLite C API because I’d written a hyper-thin Java wrapper for it, but it writing an extension in C to define a new scalar SQL function was pretty simple.

Implement function

You create a funciton with a call to sqlite3_create_function() and end up implementing it by specifying poitners to functions with the signature void (*xFunc)(sqlite3_context* ,int , sqlite3_value**). Alright, should be easy:

static void 
convert
(
        sqlite3_context* context,
        int argc,
        sqlite3_value** argv
)
{
        ...

That’s all good. You get the argument passed with one of the sqlite3_value() functions:

        num = sqlite3_value_int(argv[0]);

do something with it, say:

        dollars = num / 100;
        pennies = num % 100;

        snprintf(str, 11, "%7d.%02d", dollars, pennies);

and send it back with a sqlite3_result() function:

        sqlite3_result_text(context, str, -1, SQLITE_TRANSIENT);
}

Easy enough API to use. I should add, the main SQLite C API is lovely — you get a database connection, you prepare a statement, then you step through the results. Of course, that’s what you do with any database engine in any language, but doing it from C against SQLite is really clean.

To register my function I just have to name it, say how many parameters it has, and pass a function pointer to it, like this:

        sqlite3_create_function(db, "money", 1, ..., convert, NULL, NULL);

Some details omitted there, but you get the idea. Simple enough. Except for one thing. Where do you make that call from? Some sort of entry point, presumably, but what? And meanwhile, where do I get that sqlite3*, the database connection, from?

Loading extension

Hunting around, there’s a “load extension” function. Ok, that’s promising, and it says that the default entry point that will be looked for is “sqlite3_extension_init” but it’s not entirely obvious from the documentation for sqlite3_load_extension() what (if anything) the signature for the entry point is. The usual void (*)(void)?

void
sqlite3_extension_init
()
{
        sqlite3_create_function(db, "money", ...);
}

That’s not going to cut it. Where do I get db from?

Well, I put two and two together and guessed that actually the entry point function is invoked with a pointer to the open database connection on the call stack, because, well, you need one. And sure enough it seems to work. Eventually I found the actual signature described on the reference page for sqlite3_auto_extension()one of the few times it was really hard to find something.

int
sqlite3_extension_init
(
        sqlite3* db
)
{
        sqlite3_create_function(db, "money", ...);
        return SQLITE_OK;
}

There we go. Compile it up into a nice little shared library and you’re on your way.

$ gcc $CFLAGS -shared -lsqlite -o money.so money.c

Last bit, get it loaded. Programmatically you’d use that load extension function, but if you’re working from SQLite’s command line interface, you use the .load instruction. The catch here is that although the sqlite3 interface does filename completion, this doesn’t work:

$ sqlite3
sqlite> .load m<TAB>
sqlite> .load money.so
Error: money.so: cannot open shared object file: No such file or directory
sqlite>

Bah Humbug. It seems to be looking on the LD_LIBRARY_PATH. Ok, fair enough, but a bit annoying that it’d complete a file it can’t load. Whatever:

sqlite> .load ./money.so
sqlite>

There. Now we can try it:

sqlite> SELECT money(3995);
39.95
sqlite>

Nice!

That’s obviously a lot of work just to coerce two decimal places, but it also means in future I can do custom things like currency symbols, thousands separators, etc. Cool. I’m not about to write actual reports in SQL, but this is really handy for debugging and exploring the schema I’m working with.

Incidentally, there’s no .unload so if you change your sources and recompile you’ve got to .quit sqlite3, restart, and .load again.

AfC

Postscript

The extension API is actually more powerful than this — you can override the inherited behaviour of SQLite functions themselves. You do that by including sqlite3ext.h and placing the following macros:

#include <sqlite3ext.h>
SQLITE_EXTENSION_INIT1

int
sqlite3_extension_init
(
        sqlite3* db,
        const char** err,
        const sqlite3_api_routines* api
)
{
        SQLITE_EXTENSION_INIT2(api)
        ...

which is more correct than what I was able to get away with above. Hooray for the lack of type safety of dynamic symbols. There’s a wiki page that describes the correct usage further.

The enduring unhelpfulness of “font size”

I have to mix serif, sans, and mono face fonts all the time. It’s quite common in technical writing. But it’s also witheringly difficult to get right:

MS core fonts

That was Times New Roman, Arial, and Courier New from the ever relied upon “corefonts” all at “20 pt”. These are so widely used that they have come to be the very definition of serif, sans-serif, and monospaced. Which is sad, because they are completely useless together: each of those bits of text were written at the same “size”, but clearly they are not. There is no resemblance between the different faces. The weights are completely different. And the text itself is radically different sizes. So much for being able to “just pick a bunch of fonts at the same size”.

It turns out that the root cause problem is that the point size of a font is, for historical reasons, something that isn’t directly measurable from the appearance of a single line of text. It was the body size, the size of the metal bits that were used to compose a line of text. Unfortunately there may be more (or less) spacing between lines, so the body size is not necessarily the distance between consecutive baselines. From the good ship Wikipedia’s article on typefaces, “when specified in typographic sizes the height of an em-square, an invisible box which is typically a bit larger than the distance from the tallest ascender to the lowest descender, is scaled to equal the specified size.”

Ok, so let’s see how that looks here:

point size illustration

Sure enough, the height of the em-square here is bigger than the ascender plus descender. It’s also wider than the letter M (traditionally an em was the width of a capital M). The em-square happens to match the spacing between baselines, though whether that’s the font or the render engine is hard to say.

In any event, the size of the font, here shown to be the size of an em-square, has no relationship to the height of the letters or the proportions between upper case letters and lower case ones:

metrics illustration

I first learned about all this because the corefonts, from Microsoft, are cost-free to use, but not libre for changes or redistribution and thus cannot be shipped in Linux distributions like Debian or Fedora. As a replacement, Red Hat obtained and then generously released the Liberation font family. They were advertised as being a libre substitute for the corefonts which were “metric compatible”. Silly me, of course, assumed that they meant aesthetically compatible with each other, but of course they meant what they said — that they were individually metric compatible with the corresponding proprietary Microsoft corefont. Fair enough; they were meant as a “drop in” replacement so your presentations and word docs wouldn’t get screwed up if you switched to Linux. I have my doubts about even that, but in any event as you can see here it didn’t help matters in the relative size department:

Liberation replacement fonts

There it’s Liberation Serif, Liberation Sans, and Liberation Mono. The problem is that again font metrics aren’t calibrated to correspond to each other: the font size (which as we’ve seen is the size of the em-square) has no relation to the cap-height, the x-height, or the ratio between them.

At best, you can hope that the height of the capital letters will be the same, but even that’s a stretch. Needless to say this is chaos for the font libraries to have to deal with; you see mis-sized text all the time in desktop applications; and meanwhile authors of web browsers haven’t got a chance when some things are specified in points, others in pixels, and relative % sizes appearing from time to time just to completely drive your poor renderer to the wall. Worse of all, us poor users trying to write documents achieve nothing but a muddy mess trying to achieve something which should be simple. I just want text that looks right!

Looking at these examples, you realize that what you wan’t isn’t the capital letter height to be aligned (who cares!) but for the lower case letters to be aligned. In typography this is called the “x-height”, the size of the lower case letter x.

Screen writing

We happen to have a font family that was designed to look good together on screen:

Deja Vu font family

That’s Deja Vu Serif (that almost no one seems to know about; try it as your Document font!), Dejav Vu Sans (which has excellent Unicode coverage and as the default GNOME Application font is thus very widely used), and Dejav Vu Sans Mono (the complementary monospaced version, also very nice, and a good choice for your terminals and source code editor). You can see the relative sizes are almost in sync; in fact, on a computer monitor at 10pt at 90-120 dpi, they are aligned, because these are screen fonts designed for low resolution display:

Deja Vu on screen

More to the point, they are clearly a family that go together. That’s hard to come by, and as a family are fantastic on screen. [Droid Serif, Droid Sans, Droid Mono are also nice as a family and are fine on small device screens with even lower density, but for a proper laptop or desktop screen they’re a bit lacking in refinement].

But these are all screen fonts, and, frankly, just don’t look right printed at 1200 dpi on paper — look at the glyph spacing and the blockiness of the serifs. To be expected; Deja Vu (based on Bitstream Vera) was designed for readability at low res; things like crossbars on ts and fs and the ear on the g have to be placed so that they work in the limited number of pixels available to draw each glyph. The whats? Found a really nice illustration of typeface anatomy (it’s 1920×1200 though).

Printing for fun and profit

So we’re left trying to find printer fonts that go well together. They don’t have to be from the same typeface family, or originate from the same foundry. They just have to look right.

One of the things that pisses me off about conventional word processors is having to specify fonts for bloody everything. Every time you turn around you have to specify the font for a span of text, a new paragraph… want to create a new style for something? Pick your font. Again. And meanwhile you get a list with every last font installed on the system.

First things first. You don’t need eighteen fonts. You need about three.

  • A beautiful serif font for your normal text. A good one will have excellent Unicode coverage and will handle italics and bold well as a matter of course.

  • Next you need a monospaced font for program source fragments, code literals, and other computer-y stuff.

  • And finally you need a sans-serif font as a tool to indicate things like proper names.

And that’s all you should need to pick!

Of course once we’ve selected a sans font and a mono font we have to figure out different “sizes” for each such that they correspond with the serif font we’ve picked, otherwise we’ll end up with the mishmash we’ve seen above. But once that’s done, we shouldn’t have to do any more font work. In particular, there’s nothing worse than the occasional line being taller than the rest just because there happens to be a superscript or some other font present. No. There should only be one line height (leading) and now that you’ve picked a serif font, you want it to control that line spacing. You know, font size.

How do you get the sizes for sans and mono figured out? Ultimately you’ve got to print to paper and make sure you’re happy with the aesthetics of your choices, but as you are picking fonts, why not get a preview of of the three fonts together so you can converge the x-heights? I’ve been working for a while on a document editor called Quill and Parchment whose rendering engine configuration offers something along these lines:

FontHeightDisplay widget

Once you’ve converged the heights of the Sans and Mono selections, you’re done! The renderer can figure out everything for you from there.

Parchment defaults with Linux Libertine and Inconsolata

Ah, there we go :)

Here we have Linux Libertine as the serif font, Liberation Sans for sans-serif spans, and Inconsolata for monospaced text. And now you can mix and match these as necessary and you know they’ll fit together. You don’t have to specify the metrics and everything else just because you happen to put a filename in the flow of your text. You tell Quill that the span of text is a filename. The default Parchment render engine is configured to use the mono font in an italicized style. And then it Just Does The Right Thing™

The font choices themselves aren’t the point. Getting the relative sizes aligned is; I could have achieved this with just the Liberation family or even the unfortunately ubiquitous Corefonts. The set above are just the defaults suggested out of the box. I think they’re pretty good together, but ultimately the whole point is that you can and will make your own choices. To assess the suitability of fonts — both individually, and to see whether they are compatible working together — you really do have to print out to paper. Laser printers don’t bleed ink like moveable type presses did, but it’s not until you look at the printed page can you decide whether your choices are good ones. Screen previews — including the illustrations here — don’t do high resolution fonts justice. Not at all.

But you shouldn’t have to go through the hassle of selecting a suitable font and figuring out what size it should be every time you create a different semantic markup for a piece of text. Nope. That’s up to the renderer. Funny how the “styles” mechanisms in WYSIWYG word processors — and web browsers — seem to fall short in all this.

Anyway, next time you’re writing a document and battling fonts trying to make them look good together, try ignoring “font size” and instead zoom in and see if you can align the x-heights. You might be pleased with the result.

AfC

Use commas in FontDescriptions

Tip: always use a comma ',' when constructing a new Pango FontDescription. Otherwise, if your font family has a space in its name (as most do), the font name parser may mistakenly assume the last word of the font name is a size or variant or…

The error I was getting was that

desc = new FontDescription("Times New Roman");

was giving me "Times New" — which, since it doesn’t exist, was resulting in a fallback to the system default font "Deja Vu Sans". Shit. The fix was to use the comma separator even though I wasn’t specifying weight or size in the string:

desc = new FontDescription("Times New Roman,");

Of course, if I read the documentation I wrote years ago for our binding of pango_font_description_from_string() I’d see that I said:

The trailing ‘,’ on the family list is optional but a good idea when a font family you’re targeting includes a space in its name

so apparently I already knew this. Grr. Turns out if I’d just used setters I would have been ok too:

desc = new FontDescription();
desc.setFamily("Times New Roman");
desc.setSize(16.0);

and so on. But if you’re using the useful pango-view command line tool, be aware that the --font option is a font description string that’ll be parsed and you’ll need the comma.

Don’t worry. I’m not using corefonts for anything important :). Just needed it to make make a screenshot.

AfC

Where’s the black shiny cuboid?

Monolith

2010 has come and gone, and still no monolith. There’s no rotating space station, no psychotic computer, and Zarathustra hasn’t spoken up yet. I’m a tad disappointed, really.

AfC