Operations and other mysteries

A blog by Andrew Cowie

Wednesday, 17 Mar 2010

java-gnome 4.0.15 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources from Bazaar.

java-gnome 4.0.15 (16 Mar 2010)

Radio things

This has mostly been a bug fix cycle with numerous internal quality improvements being made. A few developer visible API additions have been made, summarized below.

Unified radio handling

There are a number of controls in GTK that exhibit the “radio” behaviour of being in a group of which only one can be selected: RadioButton, RadioMenuItem, RadioToolButton and RadioAction. We originally had a class called RadioButtonGroup which was used when constructing RadioButtons to indicate which group they were a member of. In introducing overage of the other radio types, Guillaume Mazoyer implemented a generic grouping class called RadioGroup which is now used for all the radio types.

XDG utility functions

A number of utility functions (aka static methods) were added to Glib allowing you to access the various user and systems directories as specified by the XDG specification. These are Glib.getUserConfigDir(), Glib.getUserDataDir() and friends.

Miscellaneous improvements

Better control of positioning when popping up context menus; you can specify co-ordinates when calling Menu’s popup().

The no-arg “convenience” packing methods we invented for HBox and VBox were causing more trouble than they were worth because people we not understanding the implications of the “default” packing values. So these are deprecated; the full four-argument packStart() and packEnd() have been present for a long time and are to be used in preference.

Serkan Kaba spent a bit of time working with the text version of the ComboBox API. Apparently no one had needed removeText() so he added that.

A number of improvements to the coverage of GDK’s event masks were merged. This is work originally by Vreixo Formoso necessary to support the Widget.MotionNotifyEvent signal, though it has use in other applications. Widget now has addEvents() and setEvents() to this end.

Finally, Guillaume needed to handle selections in IconViews. The API is similiar to that in TreeView, but doesn’t use the same TreeSelection helper class; the methods are directly on IconView. Boo for asymmetry. Anyway, we’ve exposed isSelected() on both TreeSelection and IconView so you can find out if a given TreePath is currently selected.

Better initialization checking

Logic checking that the library has been properly initialized has been refactored, making it harder to accidentally misuse java-gnome when getting started [or when starting a new program after you’ve been using the library for years and forgotten the basics :)].

Last but not least, a number of bug fixes; one in Enchant was developer-visible; Enchant.requestDictionary() now returns null like it claimed to if you request an unknown dictionary. Nice catch, Serkan.

All the source files have the full (ie traditional) GPL v2 header text now; the files comprising the library as used at run-time have GPL headers text + the Classpath Exception text. Needed to be done. Makes for a large diff this release, but makes the Ohloh‘s page for java-gnome happy too :).

You can download java-gnome’s sources from ftp.gnome.org, or easily checkout a branch from ‘mainline’:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

Enjoy!

AfC

Wednesday, 16 Dec 2009

java-gnome 4.0.14 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources from Bazaar.

java-gnome 4.0.14 (16 Dec 2009)

You have to compose in order to enchant

Access to Enchant spell checking API

Coverage of the Enchant spell checking facade (which was already an implicit dependency arising from our GtkSpell coverage) is now included in java-gnome. It’s a lovely library with a simple to use API which in turn fronts for various back end spelling providers.

More detailed input handling

GTK’s handling of complex input methods is extraordinarily powerful, and of course present by default in the Entry and TextView text entry Widgets. If you’re doing your own text based work, however, you might need to capture the results of an input method being used to compose characters or words. InputMethod.Commit is where the result of a compose sequence is captured and delivered to the application.

We’ve also made numerous improvements down in GDK where events are processed; as a Java library we represent many naked low-level native entities with strongly-typed classes, and have improved our coverage here, notably with new Cursor constants representing the common use cases of changing the pointer.

Improved text rendering

Other minor improvements are present across the text rendering stack, notably with the ability to introspect where a Pango Layout has made its line breaks when wrapping via LayoutLine’s getStartIndex() and getLength() methods.

Guillaume Mazoyer finished up a work by Serkan Kaba resulting in us having coverage of the special LinkButton subclass of Button which can be used to present clickable URLs.

You can now create custom PaperSize objects, which is handy if you need to use Cairo to output PDF documents with a non-standard paper format.

Other changes

You can now use gdk-pixbuf to query an image on disk for its dimensions via Pixbuf getFileInfo() function calls.

There were of course miscellaneous improvements to various long established core classes, mostly fixing typos.

We now have the methods necessary to have ImageMenuItems actually show images (there’s a GNOME bug whereby suddenly icons are not showing in menus. So you need to either explicitly tell an ImageMenuItem that it should always show its image, or use the global Settings to say that Menus and Buttons should always have their icons showing).

The internal initialization sequence has been tweaked to ensure that GLib’s threads are initialized before anything else. This means java-gnome apps will work if glib 2.22.3 is installed; this is a workaround for bug 603774.

Headless testing

For a long time we’ve had to be careful in our test suite not to do anything that would cause a Window to appear or otherwise popup while the tests were running. But some for some test cases this is unavoidable, especially if the main loop needs to cycle. We now run the java-gnome test suite within a virtual X server (ie Xvfb), and as a result distros packaging the library can run the test suite on their headless build servers if they wish. There’s a new base class for java-gnome TestCases needing to run in this environment.

Looking ahead

What’s ahead for java-gnome? That’s always a good question. At this point java-gnome provides a comprehensive API for development of user interfaces suitable for the GNOME desktop, and it’s incredibly rewarding to see people using the library for their own programs.

Development of java-gnome has continued pretty steadily, driven by people finding they need additional features from some of the underlying GNOME and FreeDesktop libraries we already expose. As a community we also work to fine-tune the performance and quality of the library through continuous improvement of the code base and its algorithms. There are also people quietly working on experimental coverage of more unusual libraries such as GStreamer and Clutter which is pretty exciting to see.

Anyone using java-gnome are always welcome to join us in #java-gnome to ask questions or just hang out! So happy hacking, and see you soon.

You can download java-gnome’s sources from ftp.gnome.org, or easily checkout a branch from ‘mainline’:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

Enjoy!

AfC

Monday, 31 Aug 2009

java-gnome 4.0.13 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.13 (27 Aug 2009)

Unicode. It’s bigger than you think.

This is a bug fix release to address a serious weakness in Java’s handling of Unicode characters.

Unicode handling

It turns out that Java’s chars are not pure Unicode codepoints. Most people know that Java String objects are arrays of Java chars, but in aggregate they are encoded in UTF-16 in order to deal with the fact that there are Unicode characters whose index is higher than 0xFFFF and which need more than two bytes to identify them. It’s a problem that an application developer has to deal with if they’re using high-range “supplementary” Unicode characters, but wasn’t something that would break java-gnome…

Except it turns out that the Java VM does not do UTF-8 translation properly. It has a hard wired limitation preventing it from writing out UTF-8 sequences longer than 3 bytes. Who knows what crack they were smoking when they decided that one. But things like TextView / TextBuffer work in characters, so we need characters. (actually, they work in UTF-8 bytes, but the offsets in our public API are the characters variants).

Luckily, we can get at the raw UTF-16 arrays backing Strings, and so in combination with GLib’s character set conversion functions, we’ve been able to redo our string handling internally so as to have correct treatment of Unicode codepoints. Lots of testing.

This surgery was almost entirely internal; Strings returned by java-gnome methods are of course still Java String objects.

New coverage

This release also features the work of Guillaume Mazoyer exposing some of the new features available in Entry Widgets, including displaying icons and showing progress bars in the background. This compliments other minor enhancements to various miscellaneous classes.

With this release, java-gnome now requires GTK 2.16 or newer.

This release is already packaged for Gentoo and Ubuntu, which is sweet.

You can download java-gnome’s sources from ftp.gnome.org, or easily checkout a branch from ‘mainline’:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

Enjoy!

AfC

Tuesday, 28 Jul 2009

java-gnome 4.0.12 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.12 (24 Jul 2009)

Being Uniquely Notified while Spelling the Sources you are Viewing is good for the soul.

In addition to ongoing improvement in our coverage of the GTK widget toolkit, the next release of java-gnome begins to realize our vision to offer coverage of the broad suite of libraries making up the GNOME desktop.

New Coverage

The TextView text editing Widget has received two significant capability boosts. With the work of Stefan Schweizer, we now have coverage of the powerful GtkSourceView library with its impressive built-in multi [programming] language source highlighting features.

And with the contribution of GtkSpell coverage by Serkan Kaba, we can now offer spell checking in TextViews as well.

Two other GNOME libraries feature in this release. Serkan also contributed excellent coverage of LibNotify, enabling an application to create and send popups to be displayed by the desktop notification mechanism.

And, we expose LibUnique, which offers DBus-powered machinery enabling a developer to ensure only one instance of their application is running.

Continuing improvement

Lots of minor changes and enhancements throughout the core GTK libraries. Highlights include improved mouse button handling, filtering when choosing files, and further refinement to Pixbuf. Thanks to Peter Mossveld, Kenneth Prugh, Vreixo Formoso, Serkan Kaba.

Finally, thanks to Guillaume Mazoyer we now have coverage of EntryCompletion, the feature of Entry Widgets whereby available possible completions are offered to the based on characters the user has typed so far.

Looking ahead

There are a number of people working on various branches — some small feature extensions, and some major coverage additions that add signifant capabilies — but which haven’t quite made it to the point where they can be merged into java-gnome. We’ll see how these pieces of work fare in the coming months, but nevertheless the Java bindings for GNOME have reached a significant level of maturity and we are pleased to see people starting to use them for serious application work.

This release is already packaged for Gentoo and Ubuntu, which is sweet.

You can download java-gnome’s sources from ftp.gnome.org, or easily checkout a branch from ‘mainline’:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

Enjoy!

AfC

Sunday, 03 May 2009

java-gnome 4.0.11 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.11 (1 May 2009)

This is a bug fix release.

We made a few mistakes in our handling of the PangoAttribute structures’ memory which resulted in VM crashes [unfortunately, the normal GNOME way of debugging things is to SIGSEGV. That’s fine for a C program but Bad™ for your average Java Virtual Machine as it takes out the entire process. We therefore work rather hard to avoid — or at least trap — this sort of thing]. Releasing corrections for these bugs was a priority.

Concurrently a significant internal improvement in our handling of accumulating Attributes into AttributeLists was made. While this is not user visible per se, we were able to drop the requirement that the text you were formatting already be in the Pango Layout before assigning the range that the Attribute would cover. You also no longer need to pass that Layout to setIndices() when building up Attributes. This makes things a great deal easier if you are simultaneously aggregating the text and assigning markup.

Martin Garton contributed some documentation quality improvements and new coverage. Serkan Kaba made some minor fixes to ensure the unit tests run in a Turkish locale. And as ever there are small incremental improvements to various classes, including a number of additional properties exposed care of the work of new contributor Thijs Leibbrand.

Looking ahead

This was a brief cycle; as noted we’re mostly pushing some bug fixes. Meanwhile there are a number of significant branches underway by various hackers; which, hopefully, will feature prominently in the next release.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’d best follow the HACKING guidelines.

AfC

Saturday, 07 Mar 2009

java-gnome 4.0.10 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.10 (5 Mar 2009)

Sculptures made of letters

This release is a landmark, because it features coverage of the Pango text rendering library and so, along with our coverage of Cairo, brings us to a complete solution for drawing graphics with text — be they custom Widgets, PDF documents, or beautiful vector illustrations.

In addition to merging Pango work from numerous contributors, we have made massive improvements to our Cairo coverage, support for writing PDFs, and better access to the system Clipboard.

Drawing API improvements

We’ve done a huge amount of refinement to our APIs for the ever fabulous Cairo Graphics drawing library, including full coverage of matrix translations, rotations, and scaling and more integrated ways of setting the source pattern to be used in stroke, paint, and fill operations. Thanks to Kenneth Prugh for having seen this through and having done the lion’s share of the testing.

We’ve also had a number of improvements in the lower levels of GTK relating to image rendering. The gdk-pixbuf library contains a capable image parser and we can now feed it directly from a Pixbuf constructor. Thanks to new contributor Martin Garton for his hard work getting that tested and accepted.

Drawing graphics often also requires drawing text. This release features coverage of Pango, GNOME’s powerful text rendering library. Pango is not just about drawing mere glyphs; it also includes a sophisticated paragraph layout engine which gives us a capable solution for drawing text onto Cairo Surfaces.

Thanks are due to Serkan Kaba and Kenneth Prugh have both been really helpful testing; it was Vreixo Formoso who did the original leg work in April that identified Layout as the key class that we needed to concentrate on and cleaned up much of the underlying infrastructure.

As we were working on this we had occasion to continue the polish in and around the TextView / TextBuffer APIs, including a number of convenience methods for inserting text while simultaneously applying multiple TextTags, and supporting changing default fonts.

We’ve also modelled “notify signals” for when a property changes; see the TextBuffer.NotifyCursorPosition signal for this in action.

Finally, thanks to contributions from Zak Fenton and Kamil Szymala we have support for applications running in composited window managers to have transparent backgrounds — which is an exceptionally cool effect, even if it doesn’t have much to do with creating HIG compliant usable applications :).

Handling cut & paste

The GTK clipboard APIs are fairly complex, in no small part because the underlying implementations in X are really rather complex. Our coverage here in java-gnome is still fairly basic relative to that, but it’s been enough for people working on applications like text editors to write text to the system clipboard, get notification when the clipboard changes, and read text back out again.

Printed document support

The work on Pango noted above represented the essential machinery necessary to make effective use of Cairo’s PDF backend, and this has started us down the road of covering the GNOME printing APIs. You can now generate .pdf documents with java-gnome, and we’ve included an fun example showing this in action.

Thanks to Nathan Strum for permission to use one of his sketches in the example (you might find reading the blog where we found this interesting — it’s a fascinating exposé of a graphic artist in action).

Continuing improvement

Thanks to new contributor Stefan Schweizer we now have better coverage of signals relating to Notebook style Containers, and to new contributors Miloud Bel and Bruno Dusausoy for numerous small improvements in and around ProgressBar.

Miscellaneous documentation improvements and minor feature improvements always feature in our releases, with minor changes having accrued in many classes.

Just as important as new coverage is getting rid of cruft that we don’t need. Libraries like GTK and GDK are, like any other mature project, full of deprecated, obsolete, and unnecessary code — not to mention things that we just plain don’t need in a Java binding of these underlying native libraries. We’re already pretty good about not generating translation Java or JNI C code for such things; this release continues our refinements in this area with a considerable refinement of the .jar and .so which is ultimately what we ship.

Finally, it’s worth noting that java-gnome’s test suite is now 40 unit test classes with 186 individual test fixtures. Combined with over 30 screenshot generating programs and 17 example programs, we actually have surprisingly good test coverage of our library. It’s high time they — and more to the point the people who work hard to create such tests — got as much credit as the visible public APIs do.

Anyone can run the tests (and anyone wanting to submit a patch had better run the tests!); they’ve been accumulating since java-gnome 4.0.0; it’s just:

$ make test

and

$ make doc

If you’re working in Eclipse launch class UnitTests as a JUnit test suite.

Build improvements

We have some fixes from new contributor Przemysław Grzegorczyk ensuring we include the correct headers in a few corner cases. This came to us via work on a GNOME Love bug, which is a first for us. Thanks!

Meanwhile, the ever industrious Serkan Kaba has added some changes to ensure that our magically locating the native shared library component of java-gnome works properly even under strange and unusual conditions.

Configuration and prerequisite detection on Open Solaris is improved thanks to reports from new contributor Kamil Szymala.

Instructions for how to optimally lay out your branches when working with java-gnome have been moved to the HACKING file. People intending to hack on the bindings from Eclipse are really urged to set things up this way as it will make their lives much easier. Also, using the latest release of Bazaar (ie bzr >= 1.12) is definitely recommended — it’s getting really fast. We’ve upgraded our public branches, so you will need 1.9 at a minimum.

Note to those creating packages for distributions: java-gnome now depends on the current stable GTK (ie gtk+ >= 2.14).

Looking ahead

Coverage of Poppler, GNOME’s PDF rendering library is well along, but didn’t quite make it in time for this release. It could very likely feature in the next version of java-gnome, as might coverage of GConf, GNOME’s simple store for application configuration options. People are also known to be working on coverage of GtkSourceView, libwnck, and librsvg. We’ll see what gets contributed!

The most exciting part about java-gnome at the moment, though, is the number of people working hard on applications using it. It takes a long time to write a mature, well rounded, robust end-user program, but there are some pretty cool projects ticking away out there, and it has been terrific to see the authors of these projects joining our community.

Happy developing,

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

though if you’re going to do that you’d best follow the HACKING guidelines.

AfC

Monday, 22 Dec 2008

Positive Y

The excellent Cairo graphics library has a simple function to draw arcs; in C it’s cairo_arc(); from java-gnome it’s Context’s arc() method, etc.

Quite unsurprisingly they define increasing angles as going from the positive x axis on toward the positive y axis. Nothing unusual about that. The only thing that was surprising is that they even mention this in their documentation.

I should know better.

What I totally missed was the implication of this. I didn’t quite clue in that the positive y direction in screen positioning and page drawing is down, and so increasing angles go clockwise. Using cr.arc() to go from 0 to say π/3 radians does not give a rise of 60° like I expected; it gives this:

cairo arc positive

Whoa. This is not the counter-clockwise increasing θ like we’re all used to seeing in normal Cartesian or Polar co-ordinates. But it is indeed increasing toward the positive y axis. Oops. Oh well :)

So I made this illustration and added it to the documentation for Context’s arc() method. Really it’s mostly about pointing out which direction positive y is, but when I’ve learned something like this the hard way, I do my best to try and incorporate that knowledge into our public API. With any luck others can be spared my folly.

Drawn with Cairo, of course!

AfC

Update: Some people have pointed out that you can use a transformation matrix, and if you happen to (say) mirror across the horizontal axis then the clockwise notion would no longer apply. Fair enough; but if you have forgotten that +y starts out going down, then you’re not going to think to do such a flip in the first place.

Tuesday, 18 Nov 2008

A nice way to wake up

I’m the lead author and maintainer of java-gnome, an Open Source project that enables Java developers to write native GNOME applications using the GTK user interface toolkit. Like any such effort, it’s a huge amount of work, much of which goes on unseen. So there are few better ways of starting off your morning than finding a message like this in your Inbox:

I just want to thank you for this great API to GTK.

It feels entirely natural in Java, is very comfortable and very approachable. The names and APIs are as you’d think them up when you’d draw a widget set API in your mind, thus they are easy to find, too. They also work as expected (as least so far, I am still new).¹ The API documentation for each class has a nice prose introduction text, which is very helpful and saves most “tutorials”. I also appreciate that it points out just the right level of footangles to be aware of.

All in all a really great piece of work. When you talked about “quality” on your website, I was suspect, as that’s most of the time just words, but you really deliver quality. You show an good example of what quality means in the context of an API.

Sun should take a good example from you and many in the open-source world, too. Such brightness and care is rare.

Hey, thanks!

Quality versus Barrier to Entry

Regardless of the underlying economic circumstances, most Open Source projects are labours of love. But they can go one of two ways.

One tendency is to just accept anything, and hope that “someone” will fix it later. This approach has the beneift of a low barrier to entry, making it easy for people to contribute. But it can result in a muddle; code that is not yet up to scratch but whose authors have moved on to other things. The only time that Open Source fails to live up to its promise (as a software development methodology) is when someone starts something and doesn’t finish it. A situation nobody wants, but it is easy to have this inflicted upon your project if you accept incomplete contributions from other people — it can be hard to incent them to come back and finish the job.

The other way is to maintain a high standard, putting emphasis on internal consistency and clarity and asking people to rise to that level. While setting a high bar does put some people off, the contributions from those who invest the effort to rise to the occasion tend to be really good.

I tend to favour the second way. If your initiative is worth working on at all, then it is the kind of thing you work hard at making perfect because it deserves nothing less. You’re not going to let it down, and you don’t want anyone else to either.

Anyway, it’s all a lot of work. Sometimes I hate it. But when you receive a note like this from someone who appreciates the effort you’ve put in, it really does make your day — and makes you feel that result of staying clearly focused has been worth it.

AfC

¹ I pointed out to him in my reply that there was still plenty of time for him to trip over something :)

Monday, 13 Oct 2008

java-gnome 4.0.9 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.9 (13 Oct 2008)

The pen is mightier than the sword

New coverage

This is the first release with coverage of GTK’s powerful TextView/TextBuffer multi-line text display and editing Widget. This has been the result of several months of careful effort to present a clean and self-consistent API while remaining faithful to the underlying implementation. This bulk of this work was done by Stefan Prelle and Andrew Cowie, with contributions from Kenneth Prugh and testing by many people in the #java-gnome community.

The snapshot at right is from ExampleInstantMessenger, included with the sources. It is a somewhat detailed example showing the use of TextView, TextBuffer, and related classes.

Other improvements

Continuous improvement to various classes, especially in our documentation. Incremental changes have occurred in a number of places. In the TreeView/TreeModel APIs, some useful methods for translating TreeIters from one model to another have been added.

Also thanks to the persistent work of Stefan Prelle, we now have nice coverage of GTK’s Assistant (aka druid, wizard, etc), along with better support for doing popup context menus, including some bug fixes. Thanks to Srichand Pendyala for taking care of this and to Owen Taylor for having explained out some of the underlying implementation details.

As usual, incremental improvements to core classes continue. Virtually every class has been touched in one way or another; many changes are cosmetic but they add up to a nice delta overall.

Reducing memory pressure

Internally, java-gnome maintains a lookup table so that pointers coming from the C side can be converted into proxy objects for the case where a proxy has already been created. In any library there a great number of transient and temporary objects and structures allocated, and in wrapping native GNOME libraries we are no different. It turned out that registering these temporary objects was putting pressure on the lookup table. While these objects were properly weak referenced and being garbage collected (and thence freed), there were nevertheless an enormous number of temporary objects being inserted and removed from the lookup table — and that sort of thing causes hash tables to grow overly large.

To do something about this we have split the former hierarchy root into two classes. Only structures which have a persistent identity (which, in practise, means only GObjects and certain Cairo entities) are registered so they can be looked up by address later as necessary. The rest of the Java side proxies aren’t registered, essentially eliminating the transient pressure on the lookup table.

Thanks to Vreixo Formoso for doing the bulk of the leg-work on this one.

Making it easier to run java-gnome programs

Because java-gnome is directly binds to underlying system libraries, it has a native shared library component. This led to the usual development hassle of having to specify where this library is to be found if it were anywhere other than /usr and of course the nightmare of ensuring a VM used the right library in the event you were developing against or hacking on a newer version of java-gnome; in Java this meant:

$ java -classpath /opt/local/share/java/gtk-4.0.jar:. -Djava.library.path=/opt/local/lib com.example.Program

No longer!

The native shared library part of java-gnome is now located deterministically and loaded automatically. You don’t need to faff about with java.library.path on the command line or in your IDE any more!

$ java -classpath /opt/local/share/java/gtk-4.0.jar:. com.example.Program

Our native component is completely coupled to the specific release you are using, so sufficient version information is embedded in the .so name to ensure that the right library (and only the right library) is loaded.

There are no changes if you are simply working against an “in-place” development build of java-gnome, be it from command line, or in an IDE like Eclipse. Things will Just Work™. Again, no -Djava.library.path.

This whole issue turned out to be a real stumbling block for new developers attempting to use the bindings; it’s not something that many Java programmers have had to deal with, so engineering around this problem to make the native library loading transparent is a big win for us.

Build system improvements

Serkan Kaba has contributed a number of internal improvements allowing the top level ./configure script to be precise about the versions of various GNOME dependencies we require.

Thanks to some hard work from Serkan Kaba and new contributor George McLachlan, java-gnome correctly builds against GTK 2.14 without any problems due to deprecations.

Note that java-gnome releases do not set GTK_DISABLE_DEPRECATED (this is a change from 4.0.8); thanks to Mart Raudsepp of the Gentoo Linux desktop team for pointing out why this would be better. These macros are still enabled for builds checked out from version control so hackers working on the bindings so will be able to keep up with ensuring we react to future deprecations (it’s always awesome when downstream is a part of the upstream community; Serkan and Kenneth are also Gentoo packagers, and take care of the java-gnome .ebuild for us).

Looking ahead

We’re pretty happy with the state of the java-gnome right now. Coverage of the most important parts of GTK are in place. Our treatment of the underlying drawing library, Cairo, still has a bit to go, but the basics are there and a firm foundation to build from. More interesting are the remaining areas; the more general GNOME utility libraries and other parts of the Free Desktop stack that might be needed by an end-user application. It’ll be interesting to see how these areas evolve in the coming months.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

AfC

HTTP response code nuttiness

The technique of using a 404 error handler to actually render a web page from some underlying source or data is widely used. We use it on the java-gnome website to render prettily marked-up versions of our text meta documents (NEWS, README, HACKING, etc). The document you’re asking for (“README.html“) doesn’t exist, but a text source (“README“) does and so it reads that file in, runs the markup processor, and then serves the result. Yawn.

If you’re already in an error handler, then the default status code as far as the web server is concerned is, of course, HTTP 404. So you need to change that. In PHP, that’s as simple as:

which is actually treated a bit magically to tweak the response code. Too easy. This worked fine on our test site and on the production site for, well, years.

We ran into a weird problem that at some point over the last few weeks that suddenly I was seeing garbage appearing at the beginning and end of documents being so rendered on our SourceForge website. Huh?

I still have no idea what is wrong, and won’t detail all the diagnostics it took to track down the problem, but I ended up isolating it to the header() call. If I skipped that, then the document would render and serve correctly. But that’s no good, because then you are serving documents with an HTTP 404 error code, which will inhibit search indexing and caching even if it doesn’t screw with your browser.

The workaround ended up being this:

I have no idea why, but with whatever SourceForge has done to their webservers lately, this made it work fine. Go figure.

The real inanity, though, is that using wget to fetch the document while showing HTTP headers the result is this:

$ wget -x -S http://java-gnome.sourceforge.net/4.0/README.html
--2008-10-13 14:11:28--  http://java-gnome.sourceforge.net/4.0/README.html
Resolving java-gnome.sourceforge.net... 216.34.181.96
Connecting to java-gnome.sourceforge.net|216.34.181.96|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Server: nginx/0.6.31
  Date: Mon, 13 Oct 2008 03:11:29 GMT
  Content-Type: text/html; charset=UTF-8
  Connection: close
  X-Powered-By: PHP/5.2.6
  Last-Modified: Sun, 12 Oct 2008 14:02:37 GMT
Length: unspecified [text/html]
Saving to: `java-gnome.sourceforge.net/4.0/README.html'

    [    <=>                                ] 16,052      22.3K/s   in 0.7s    

2008-10-13 14:11:30 (22.3 KB/s) - `java-gnome.sourceforge.net/4.0/README.html' saved [16052]
$

Their web server sent an HTTP/1.1 response anyway! Talk about adding insult to injury. What a waste of time.

I’m sure this is just one of those transient conditions that’ll be gone by this time next week, but it sure as hell frustrated me. If you’re seeing something weird going on with PHP pages on one of your SourceForge sites, perhaps have a look and see if this workaround helps.

{shrug}

AfC

Tuesday, 19 Aug 2008

java-gnome 4.0.8 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.8 (15 Aug 2008)

Cleanups and fixups.

This release is mostly to push out bug fixes and internal improvements, setting the stage for some major new feature development. We’ve also taken the opportunity to introduce a major change to the way we connect handlers for signals.

New coverage and continuing improvement

With thanks to new contributors Stefan Prelle and Andreas Kuehntopf we have a number of small improvements to the TreeView/TreeModel APIs. As always, Widget and Window saw a bunch of work, with Window.ConfigureEvent now being available and a number of additional property setters and methods relating to window type. Widgets that scroll around a view of a broader underlying canvas have seen a fair bit of activity related to controlling that scrolling.

New features include refinements and new coverage of methods in a variety of lower level classes including that further support drawing operations. Bug fixes, debugging improvements, and defencive enhancements to our thread safety measures have also featured largely.

Signal naming change

The names of the inner interfaces used to specify the prototypes of the methods which receive signal callbacks have changed to the pattern Button.Clicked, this being more appropriate to Java type naming conventions and providing better consistency between the signal name, the method to be implemented to receive the callback, and the method that can be used to emit this signal.

API compatibility to previous releases in the 4.0 series has been preserved. The old signal interfaces and connect() methods are @deprecated. Developers are encouraged to migrate quickly; new coverage will of course all follow the new pattern. Making the transition is is easy, especially in an IDE. Most of the people we’re aware of using the library have already ported their code. Just turn assertions on to double check.

Build changes

java-gnome now defines C compiler flags like GTK_DISABLE_DEPRECATED to ensure we are not linking against code that will be unavailable in GTK 3.0. Many thanks are due to new contributor Kenneth Prugh for having done some terrific grunt work to prune deprecated classes and methods from our .defs data so that java-gnome compiles without using these APIs.

The build system internally now ensures that multiple runs don’t occur simultaneously, fixing a number of annoyances that cropped up when using IDEs which tend to like trying to frequently re-run the build even if a previous one hasn’t finished.

Documentation, examples, and testing

Our API documentation and the growing set of example code have all been updated to reflect the new signal interface names. Doing so forced us to review a wide swath of the documentation, and so along the way a huge number of minor improvements were made. Given how detailed our JavaDoc is, this sort of painstaking work really makes a genuine contribution to overall quality.

There has been steady growth in our test suite, which is great. When combined with the snapshots used to illustrate our documentation, the coverage level is substantial.

Error handling continues to improve. In the (hand written) public API wrapper layer we do our best to catch misuses of the library before they get sent to the native code. But that’s not always possible, and in 4.0.7 we introduced a mechanism whereby GLib error messages get translated into Java Exceptions and thrown. As of 4.0.8, in addition to ERROR and CRITICAL, we also throw WARNINGs as Exceptions. Getting a stack trace this way has proved very useful in helping developers track down where they are making mistakes in using the library.

Conclusion

You can see the full changes accompanying a release by grabbing a copy of the sources and running:

$ bzr diff -r tag:v4.0.7..tag:v4.0.8

Because of the API changes to signal handling this release touches just about every public class in the library and so isn’t quite as clean (as a summary) as in previous releases — but it does show you everything that changed. :)

Looking ahead

Most of the contributors to java-gnome are working on branches that didn’t reach sufficient maturity to be merged in time for 4.0.8; that’s the way it goes sometimes. Major effort continues on implementing coverage of GTK’s powerful TextView/TextBuffer APIs, along with further drawing capabilities in Cairo and Pango. There have also been a surprising level of interest on other areas of the GNOME stack, with new contributors working on adding support to java-gnome for Nautilus, GStreamer, and even WebKit. Exciting stuff!

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

AfC

Sunday, 06 Jul 2008

Using Eclipse’s source code formatter from the command line

One of the many wonderful capabilities built into the side of Eclipse is a powerful code formatter. Adjusting source code to match one’s particular preferences is nothing terribly new; what makes Eclipse’s so excellent is the degree of configurability and the preview of the effect each setting will have.

An Eclipse project has several metadata files:

.project
.classpath
.settings/org.eclipse.cdt.core.prefs
.settings/org.eclipse.jdt.core.prefs
.settings/org.eclipse.jdt.ui.prefs

Which seems a bit of a mess, but whatever. The .settings/ stuff gets written down when you decide to tell a Project to have settings custom that override the user’s defaults. It goes without saying that for a collaboratively developed project like the Java bindings we have the Eclipse project metadata for compiler warnings tuned, the code formatter settings, how to invoke the build system etc all stored in Bazaar along side the real sources.

Anyway, this’s all fine and nice for people working on my software in Eclipse, but what about people wishing to use a different editor?

For a few versions now, Eclipse has exposed the the ability to run its code formatter from the command line; the work was done by Ben Konrath, someone I met a few years ago at a java-gnome hackfest. Since I use Eclipse I hadn’t needed to bother to figure out how to use whatever it was that he’d done. Since becoming maintainer of java-gnome, though, I’ve noticed a bit of unnecessary formatting churn in the contributions we receive. We do have a style guide, but it is secondary to the fact that Eclipse just normalizes the code every time I save something (nice). But for people not using Eclipse I thought I might have a look.

A bit of searching around turned up this blog post by one Peter Friese. It included two important points that are non-trivial. Firstly, that configuration the code formatter from command line will use can be controlled by the pointing it at the .settings/org.eclipse.jdt.ui.prefs file which in turn can be created by the “Enable project specific settings” described above (we’d already done that, so no problem, but how on earth would anyone have guessed this?). Secondly, the command line that one has to incant to make it go is pretty voodoo, so I’m pretty thankful that he described what he’d done.

In our environment, this works out to:

$ /opt/local/eclipse/eclipse -nosplash
    -application org.eclipse.jdt.core.JavaCodeFormatter
    -verbose
    -config .settings/org.eclipse.jdt.core.prefs
    src/ tests/ doc/examples/

Gah!

For a brief moment I wondered if it would be easier just to run the Java class in question. So I interrupted the process and had a look in the process table.

/usr/lib/jvm/sun-jdk-1.6/bin/java
    -Dosgi.requiredJavaVersion=1.5
    -client
    -Xms40m
    -Xmx256m
    -jar /opt/local/eclipse/plugins org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar
    -os linux
    -ws gtk
    -arch x86
    -launcher /opt/local/eclipse/eclipse
    -name Eclipse
    --launcher.library /opt/local/eclipse/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.0.2.R331_v20071019/eclipse_1021.so
    -startup /opt/local/eclipse/plugins/org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar
    -exitdata 84001b
    -application org.eclipse.jdt.core.JavaCodeFormatter
    -verbose
    -config .settings/org.eclipse.jdt.core.prefs
    src/ tests/ doc/examples/
    -vm /usr/bin/java
    -vmargs
    -Dosgi.requiredJavaVersion=1.5
    -client
    -Xms40m
    -Xmx256m
    -jar /opt/local/eclipse/plugins/org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar

Good Lord. What the hell is all that? I mean, I know Java environments tend to get a bit unwieldy because there is still no link-to-binary facility, and I know Eclipse is an OSGi framework and what not, but that is pathetic. I guess running the command as described at the top isn’t such a bad idea after all.

I’ve got an even better idea: how about a convenience target in our Makefile front end. Yup. So I’ve added a format target:

$ make format

(or

$ ECLIPSE=/opt/local/eclipse/eclipse make format

in my case). Running that before committing will do the trick quite nicely.

I agree with the bug mentioned by one Nicholas Irving in his post on the topic: it is somewhat asinine that this bit of Eclipse bumps the modification time of every file it formats; it would be nice if it only changed files in which it actually found something it needed to alter. Fortunately our internal build system is actually md5 hash (not Make) based, so it doesn’t hurt us.

AfC

Monday, 05 May 2008

java-gnome 4.0.7 released!

This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.7 (30 Apr 2008)

Draw some.

In addition to the usual improvements to our coverage of the GNOME libraries, this release introduces preliminary coverage of the Cairo Graphics drawing library, along with the infrastructure to make it work within a GTK program.

Drawing with Cairo

Example

The trusty Cairo context, traditionally declared as a variable named cr in code, is mapped as class Context. Various Cairo types such as different surfaces and patterns are mapped as an abstract base class (Surface, Pattern) along with various concrete subclasses (ImageSurface, XlibSurface, and SolidPattern, RadialPattern, etc). Error checking is implicit: the library status is checked internally after each operation and an Exception thrown if there is a failure.

Thanks in particular to Carl Worth for having reviewed our API and having helped test our implementation.

New coverage and continuing improvement

The single option choice buttons in GTK are called RadioButtons and have now been exposed. When using them you need to indicate the other buttons they are sharing a mutually exclusive relationship with, and this is expressed by adding them to a RadioButtonGroup.

RadioButton

The usual steady refinements to our coverage of the GtkTreeView API continue. There’s a new DataColumn type for Stock icons, and TreeModelSort is now implemented, along with minor changes to various other miscellaneous classes.

Considerable internal optimizations have been done, especially relating to ensuring proper memory management, with notable refinements to make use of “caller owns return” information available in the .defs data. This fixes a number of bugs. Thanks to Vreixo Formoso for having driven these improvements.

Error handling has been improved for GLib based libraries as well. If an ERROR or CRITICAL is emitted, our internals will trap this and throw an exception instead, allowing the developer to see a Java stack trace leading them to the point in their code where they caused the problem.

Internationalization support

java-gnome now has full support for the GNOME translation and localization infrastructure, including the standard _("Hello") idiom for marking strings for extraction and translation, but combined with some of the powerful support for positional parameters available from Java’s MessageFormat as well. There’s a fairly detailed explanation in the Internationalization utility class.

Build changes

Note that as was advertised as forthcoming some time ago, Java 1.5 is now the minimum language level required of your tool chain and Java virtual machine in order to build and use the java-gnome library.

Thanks to Colin Walters, Manu Mahajan, Thomas Girard, Rob Taylor, and Serkan Kaba for contributing improvements allowing the library to build in more environments and for their work on packages for their distributions.

The download page has updated instructions for getting either binary packages or checking out the source code.

Documentation, examples, and testing

Refinements to the API documentation continue across the board, notably improving consistency. A large number of javadoc warnings have also been cleaned up.

While not a full blown tutorial, the number of fully explained examples is growing. There are examples for box packing and signal connection, presenting tabular data, and basic drawing, among others. See the description page in the doc/examples/ section.

This code, together with the not inconsiderable number of unit tests and the code for generating snapshots of Widgets and Windows means that a large portion of the public API is tested within the library itself. The number of non-trivial applications making use of java-gnome is starting to grow, which are likewise providing for ongoing validation of the codebase.

Summary

You can see the full changes accompanying a release by grabbing a copy of the sources and running:

$ bzr diff -r tag:v4.0.6..tag:v4.0.7

Looking ahead

It’s probably unwise to predict what will be in future releases. The challenge for anyone contributing is that they need to understand what something does, when to use it (and more to the point, when not to!), and be able to explain it to others. This needs neither prior experience developing with GNOME or guru level Java knowledge, but a certain willingness to dig into details is necessary.

That said, I imagine we’ll likely see further Cairo improvements as people start to use it in anger. It shouldn’t take too long until the bulk of the functionality needed for most uses is present in java-gnome. In particular, forthcoming coverage of the Pango text drawing library will round things out nicely.

There are a number of other major feature improvements we’d like to see in java-gnome. Conceptual and design work is ongoing on for bindings of GConf, GStreamer, and even support for applets. Within GTK, there have been a number of requests made for various things to be exposed, for example, the powerful GtkTextView / GtkTextBuffer text display and editing capability. Some of these have preliminary implementations; whether or not any given piece of work is acceptable in time for any particular future release will remain to be seen and depends on the willingness of clients to fund us to review and test such work.

In the mean time, people are happily using the library to develop rich user interfaces, which is, of course, the whole point. We’re always pleased to welcome new faces to the community around the project. If you want to learn more, stop by #java-gnome and say hello!

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr clone bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

AfC

Sunday, 06 Apr 2008

Cairo drawing from java-gnome

We’ve added support for the amazing Cairo Graphics library to the Java bindings for GTK and GNOME.

Adding Cairo is a feature we’ve been working on for about 6 months now, and so I’m pretty pleased that during the 4.0.7 development cycle we’ve been able to land it. Cairo is a huge library, of course, but we’ve put enough coverage in place to ensure that things are working.

Cairo has lots of convenience functions and tons of obscure uses; no surprise (and no apology) that there’s still lots that will need doing. If you want to help make sure it has what you need, then grab ‘mainline‘ and see org.freedesktop.cairo.Context and ExampleCairoDrawingInExposeEvent.

Huge thanks go out to Behdad Esfahbod and Carl Worth; Behdad was really critical in explaining some basic Cairo concepts to get me started when we were working together at the GNOME Summit back in October in Boston, and at the GTK hackfest in Berlin in March, Carl Worth checked our preliminary APIs and helped sort me out as we were working our way through to create some examples. Awesome!

example

java-gnome 4.0.7 should be out in the next week or two, depending on how long it takes to finish up testing, QA, and to do the release engineering.

AfC

Tuesday, 12 Feb 2008

java-gnome 4.0.6 released!

This blog post is the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.6 (12 Feb 2008)

Finding the missing methods.

Most of our effort recently has simply been fleshing out areas of the public API. The focus for this work as been getting the coverage needed to allow us to port some of our in-house applications to java-gnome 4.0. It’s not especially glamorous — if anything it has been tedious as hell — but the result has been a large body of improvements to java-gnome as a whole which we’re pleased to release as java-gnome 4.0.6

The bulk of this development took place on the ‘missing‘ branch, so named because that’s where I was working on what was missing :).

Continuing Improvement

Notable public changes include coverage additions to enable key stroke and mouse button handling

Rather than exposing the int keyvals that bubble up out of the X server, we have wrapped these as constants of type Keyval (thereby being consistent with the rest of java-gnome in our working to the strengths of Java as a strongly-typed language; MouseButton was created for the same reason, helping developers understand just what on earth mouse button “1” is, anyway). Along with ModifierType, this gives enough to deal with the KEY_PRESS_EVENT and KEY_RELEASE_EVENT signals when the developer wishes to deal with key strokes.

We’ve finally gotten around to providing proper coverage of the box packing model which underlies every aspect of how GTK presents user interfaces. To understand the size-request/size-allocation process, you might start with Widget’s setSizeRequest().

We’ve also added coverage for SizeGroup, which, when used in concert with nested VBoxes and HBoxes, can work wonderful magic and is often far better than messing around with Table when doing complex layouts.

After dithering for several releases, we’ve settled on how we’re going to deal with ComboBox and family. The underlying GtkComboBox presents something of a nightmare as it is really two classes in one with more-or-less incompatible APIs. So, not surprisingly, we’ve presented it as two sets of classes, with the text-only convenience API spliced out of ComboBox and ComboBoxEntry into TextComboBox and TextComboBoxEntry respectively.

We’ve added a few new features in our coverage of GTK’s TreeView / TreeModel API, and many other classes involved have also seen improvements. The persistent reference to a row provided by TreeRowReference is now available as is model type TreeModelFilter.

Support for the actual filtering in TreeModelFilter is notable for having been quite tricky. The underlying C library use a function pointers rather than a GObject signal emission, and we don’t have any mechanism to handle that. We do, however, have a fantastic capability to marshal signals, so we dealt with the problem by creating a custom signal and then passing a function which emits it when the TreeModelFilter wants to ask the developer whether to include a row or not.

The new classes include support for TreeModel columns storing long data as well as setting properties of that type.

It should also be noted that most of the methods taking a TreeViewColumn have been converted to taking an argument of type CellLayout (an interface implemented by TreeViewColumn). This has no change to how you use our TreeView API, but was necessary to support ComboBox properly.

Finally lots and lots of minor additions to both public APIs and internals deeper down in the GDK part of the toolkit.

As ever, you can see the full changes accompanying a release by grabbing a copy of the sources and running:

$ bzr diff -r tag:v4.0.5..tag:v4.0.6

Documentation

We’ve always had HTML JavaDoc for the current stable release at doc/api/ on the java-gnome website. We’re going to change that a bit, though. As fixes to the explanatory documentation happen quite frequently to classes and methods all over the place, we’ve decided to generate the JavaDoc from ‘mainline‘ periodically and upload that instead. This means that there will, of course, be descriptions of some methods which aren’t yet available in a released version of the library, but they will clearly identifiable by virtue of having a @since tag showing a version number greater than the most recent release. We’ll revert back to more traditional behaviour once we hit 4.2.0, but in the mean time we can give people access to the best information we can online.

The idea of having up-to-date illustrations of the various Widgets has proved popular, and we’ve continued to update the suite of snapshots. Doing that is also tedious, but it does provide a good opportunity to test APIs we are exposing especially where unit tests are less suitable.

Statusbar

Looking ahead

Almost as complex as the TreeView / TreeModel API are GTK’s powerful TextView / TextModel classes, collectively a Widget used to display and edit large text documents. Working out the java-gnome coverage for TextView will take a fair bit of consideration, but TreeView provides a road map, and, as with the coverage in 4.0.5 and 4.0.6 (which was driven largely by existing applications we were porting), we have some significant uses of GtkTextView which will guide us on our way.

The next release will also feature significant work outside of GTK; we should be in a position to merge our coverage of the excellent Cairo drawing library soon, and likewise we have tentative work in place letting people store configuration and settings data in GConf. Both the ‘cairo‘ and ‘gconf‘ branches need more QA and documentation work, but they’re looking good and will definitely be featured in java-gnome 4.0.7.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from ‘mainline‘ using Bazaar:

$ bzr clone bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

Well, now that I have this out of the way, I can finally stop worrying about having so much unreleased code and get back to the business of doing what I do for a living.

On a personal note, I mentioned that much of this coverage was driven by one of our legacy apps. About half way though this cycle, however, we decided we would be abandoning that program. Needless to say, it was more than slightly depressing to be putting so much work into writing new coverage for java-gnome in order to port such a large codebase in the certain knowledge that the application was not going to be resurrected after all, especially as we have not yet managed to raise any revenue to offset our costs in taking on the Java bindings for GNOME.

Cognitive Dissonance.

Nevertheless, I kept at it largely because getting that app to run again represented a good test of the quality and comprehensiveness of the Java bindings. It needed doing, and that has ultimately been the motivation behind all our work re-engineering java-gnome. I gotta say it was wonderful seeing that program come back to life bit by bit as the errors slowly fell away as missing classes and methods were added to the library. That monster running again after almost two years represents the 4.0 version of java-gnome having surpassed the level of coverage that we used from the old 2.x version, and that’s something to be very positive about indeed.

Of course, now all I have to do is to continue to resist the temptation to go back to working on that beast. :) No, better things to be about.

AfC

Monday, 26 Nov 2007

java-gnome 4.0.5 released!

This blog post is the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.5 (26 Nov 2007)

TreeView is here!

It’s always a great feeling when you bag a milestone, and with this release we have reached a major goal on our way to having outstanding Java bindings for the GNOME platform: coverage of GTK’s powerful yet complex TreeView & TreeModel API.

TreeView

TreeViews are a central part of almost every application. GUIs use lists for all sorts of things, and so a significant goal was to make coding TreeViews and their backing TreeModels as straight forward as possible.

The most challenging and complex part was to design the Java side API, which was no small matter. As a native library, the GtkTreeView API is complex and very much written with programming in the C language in mind, and as such our algorithmic mapping of the underlying libraries into Java doesn’t entirely fit. Long experience with the TreeViews in the previous bindings had made it clear just how nasty to use the API could be, and so the hardest part of the work was to come up with a mapping and a usage pattern that would be both faithful to GTK and be sensible to use.

The other significant challenge was to document the work effectively. Our Java side API documentation is a major feature of java-gnome, and merely exposing classes and methods is not sufficient; they need to be clearly explained in our JavaDoc as well.

along with numerous test cases in our unit test suite, and several comprehensively worked examples.

This was a monster patch, and the culmination of not just three months direct effort, but also where we’ve been heading since we first started the re-engineering of Java bindings for GNOME. Although largely written by Andrew Cowie, a significant contribution was made by Srichand Pendyala who not only exhaustively evaluated the design but also threw in some serious chunks of code. The work benefited from comprehensive input from Peter Miller on the modelling and design, and the comments of Bryan Clark, Owen Taylor, and Hanna Wallach were all really positive and helped us know that we’d gone in the right direction. Finally, thanks to Behdad Esfahbod and the GNOME Foundation who made it possible for us to meet in Boston at the GNOME Summit and so accomplish much of the final pulling together of this branch.

Continuing Improvement

Meanwhile, steady work continues on to the fundamental base classes, with a whack of additional signals and methods on Widget and especially Window, along with expansion of coverage in numerous other classes.

We’ve begun to get the basics of image handling in place,

One nice piece of contributed work came from Vreixo Formoso and Thomas Schmitz with coverage of the Dialog Window functionality in GTK. It took a bit of doing to map the int response codes used by GTK into something suitably strongly-typed, but all good.

Finally, minor improvements to all sorts of stuff, notably from new contributor Mario Torre. Awesome!

For further details you can always grab a copy of the sources and run

$ bzr diff -r tag:v4.0.4..tag:v4.0.5

to see the complete code delta.

Screenshots

For fun we built in a capability to create demonstrations to be captured as screenshots to illustrate various things. It doesn’t get more basic than the example on the Window documentation page, but it’s a nice touch. :) We’ve screenshots for a number of Dialog classes and one for the TreeView page. I imagine we’ll build up a nice library of images in the next few months (yes, dear contributors, you can add snapshots to the list of things we’ll be expecting along with well written documentation and unit tests when submitting additions to the public API).

Building and requirements

The library now depends on GTK >= 2.12. Those packaging java-gnome for their distributions please take note.

Looking ahead

Continuing to expand the coverage levels in the classes already exposed will continue to dominate our attention; there’s still a long way to go but we’re pleased with the progress we’ve made so far; you can definitely build real applications with java-gnome now.

The next release also ought to include preliminary coverage of GConf and Cairo. Doing each justice will again take a serious amount of work, but will continue to grow the fun things you can do with java-gnome.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from mainline using Bazaar:

$ bzr clone bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

This release marks a particularly satisfying milestone for me. I have three+ years of work that’s been waiting to be ported to java-gnome 4.0, but I’ve needed a new TreeView binding in place before I could do so. So this is a big deal for me. At last I can start thinking about getting back to actually hacking on the applications I actually want to be working on, and not spending so much of time on the underlying bindings themselves. There’s still a lot that needs doing on java-gnome, but it’s getting closer.

AfC

Wednesday, 21 Nov 2007

Automatically rendering and capturing screenshots

One of the things I’ve often noted in the API documentation for the various GNOME libraries is that the examle screenshots always seem out of date. I think this is mostly because the look and feel we all interpret as “modern” is just that which happens to be shipping with the current version of the GNOME Desktop we’re using — and that default changes fairly frequently.

As I’ve been working on the GTK coverage in java-gnome, I’ve had in the back of my mind that it would be really cool if we had illustrations of the various Widgets and whatnot under discussion. More than cool, actually; a lot of the people developing with java-gnome are experienced programmers but new to Linux and the GNOME Desktop, so it’s probably pretty important that we help them along in learning which controls on their desktop correspond to which Widget classes. Capturing such screenshots manually is a bit of a pain, though, and even if we did that we’d still end up with them looking out of date within a year or so.

So I had a bit of fun last week and knocked something together that would automatically take screenshots of individual Windows or fragments thereof such that we could use them in the java-gnome API documentation.

Snapshots

First I cooked up a scheme to make it easy to write the little programlets that draw the Windows and Widgets (I used the term Snapshot to refer to these; nothing as elegant as the JUnit API, but hopefully it’ll improve over time). Then I wrote a harness to instantiate the sequence of Snapshot programs and to capture a screenshot of each one in turn.

GNOME ships with a tool in gnome-utils called gnome-screenshot which is handy, but a bit too focused on being a user driven program and is not available as a library in any event. So we pulled out the parts that actually capture the image (my Lord, talk about ninja voodoo X programming — I remember working directly with Xlib back in the late 80s;I am so glad we don’t have to do that anymore) and made it accessible programmatically. So all Harness has to do is something like:

window = snapshot[i].getWindow();
window.showAll();
window.present();

pixbuf = Screenshot.capture();

and then save the Pixbuf to disk in the right place so that <img> tags in our JavaDoc can pick them up. I did up examples for Window, Button, a few MessageDialogs, and TreeView. They’ll be in java-gnome 4.0.5 which will be out within the next week or so.

Off screen

If you did this as a zillion Windows popping up in front of you, though, you’d probably end up capturing the wrong thing if you accidentally changed apps or something. So instead I tried to figure out how to do it off screen. The solution I came up with was using the virtual X server that ships with xorg, Xvfb. By doing:

$ Xvfb :1 -ac -screen 0 640x480x24 -wr

we have a new X server running on :1, and then by passing --display=:1 to the Gtk.init() call used to initialize GTK, the Windows rendered by the program will be sent to the virtual server, not the desktop running on :0, and then we can take our screenshots there.

I’ve had this in mind for a while as a way to do unit tests that absolutely need to have the Window mapped to the screen. Unlike the unit test scenario, though, there’s a catch: you want the Windows to look right, ie just as they would if they were on a user’s desktop. That doesn’t happen in a naked X server though; you need a window manager running and also the magic program that makes the GTK themes according to your current preferences:

$ metacity --display=:1
$ gnome-settings-daemon --display=:1

otherwise you get that rather ugly default built-in GTK fallback theme and no window decorations, which would defeat the whole point of the exercise. So hook that all up to be exec()d when the Harness program starts and everything works great!

I’ll have to remember to switch themes back to Clearlooks before generating the screenshots for our next release… :)

AfC

Wednesday, 26 Sep 2007

java-gnome 4.0.4 released!

This blog post is the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.4 (26 Sep 2007)

Coverage increasing!

Most of our work continues to be on infrastructure and architecture, improving the code generator that outputs the translation Java layer and JNI C layer which allow bindings hackers to reach the underlying native libraries. Nevertheless, there have been a number of publicly visible improvements across the board, so we wanted to push out a release highlighting these contributions!

Documentation improvements

Continuing our effort to have extensive developer friendly tutorial style documentation, there have been major additions to a number of existing classes. Of particular note is the Window class, containing the various utility methods used to ask the window manager to do things for you (we’ve also started exposing some of the deeper parts of the GTK toolkit, though only a few things that were immediately related to window management).

While the topic of thread safety was discussed at considerable length in the last release, we have added some of the more relevant information to the code documentation to reinforce its importance.

New coverage

Numerous people have been hard at work developing new coverage. The standards for accepting patches which expose public API are high, so it’s awesome to see bundles accepted for being merged to mainline from new contributors Thomas Schmitz, Wouter Bolsterlee, and Nat Pryce.

The infrastructure for a number of areas important to supporting applications including Menus, Toolbars, and Actions has been put in place.

A number of Container related Widgets have been added, though coverage is preliminary. There have, of course, also been a number of minor improvements in other existing classes, miscellaneous constants and wrappers around the stock item identifiers.

Internals

Vreixo Formoso carried out an important refactoring to the type database and Generator family of classes in the code generator, with the result that more of the array passing and out-parameter cases are now being handled correctly. This kind of work is usually thankless and taken for granted, but it’s hugely appreciated!

The real gains are in internal quality. A number of serious bugs and limitations have been overcome (Glade is working again, for example). The generated code now guards against improper use (you can’t pass a null pointer unless it’s allowed by the underlying library). Related to this is handling of “GError” — Java side, bindings hackers will get GlibException which they can then re-throw as an appropriate Java Exception, say FileNotFoundException in the case of not being able to open a file.

This all goes along with numerous build system fixes by Srichand Pendyala to make for an increasingly robust project. Thanks guys!

Looking ahead

As mentioned above, we have mostly been focused on areas other than public API, but it is expanding steadily. The hard work on infrastructure, however, is starting to pay off, and the next release should include coverage of TreeView, GTK’s powerful but complex list Widget.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from mainline using Bazaar:

$ bzr branch bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

Note switch to bzr:// protocol.

AfC

Wednesday, 01 Aug 2007

java-gnome 4.0.3 released!

As promised at GUADEC, the landmark release of Java bindings for GTK and GNOME based on generated internals has indeed been made in July! (with like five and half minutes to spare) As is traditional now, this blog post is the release note from the NEWS file which you can read online … or in the sources, of course!

java-gnome 4.0.3 (31 Jul 2007)

The code generator has landed!

Work has been underway for several months to develop the next stage of the new java-gnome: the code generator that will output the tedious translation and native layers that allow us to glue our public API to the native GNOME libraries. With this release we’re pleased to announce that the code generator is a reality!

Generated translation and JNI layers

The primary goal of the java-gnome 4.0 re-engineering effort has been to switch to an architecture whereby we could generate the bulk of the machinery necessary to take make native calls into the GNOME libraries from Java.

Extensive prototyping was done to establish the detailed design and to validate the architecture we had developed. Releases 4.0.0 through 4.0.2 contained this work along with mockups of the “translation” layer (the Java code that downshifts from our Proxy objects to primitives suitable to pass over the JNI boundary, along with the native declarations necessary in order to call methods actually written in C) and of the “JNI layer” (the C code that implements the methods declared in the translation layer which in turn coverts parameters into GLib terms and then makes the actual function call into the appropriate GNOME library).

With a solid foundation proving that our design was sound, we subsequently began the long effort to implement a code generator which would output these Java and C layers, allowing us to replace the mockup/ directory and at last leave behind the shackles of entirely hand written bindings. Over the past five months, the java-gnome hackers have been steadily working on the ‘codegen’ branch. The nature of the challenge meant that we had to have most of the code in place before any of it would be useful — never an enviable task to be working on. Thanks to the hard work of Andrew Cowie, Vreixo Formoso Lopes, and Srichand Pendyala, we reached the point where the output Java code compiled in May, and the output C code successfully compiled in by the end of June. Tremendous.

We’ve been bug hunting and refining since then, pushing towards the point where we could merge back to ‘mainline’, at last replacing the hand written mockup code. We are today pleased to announce the culmination of that work with the release of java-gnome 4.0.3.

This post on the development of the java-gnome code generator contains further details should you be interested; the file 5a-Architecture.txt in the doc/design/ directory of the source code explains the rationale and origin of the engineering design.

New coverage

Although our focus has evidently been on getting the generator into working order, there have nevertheless been a few minor coverage additions along the way which are largely to the credit of Sebastian Mancke for having submitted them and Srichand Pendyala for having fixed up their JavaDoc. There have also been steady improvements to a number of other classes; notably further signals and utility methods along with some preliminary coverage of the lower level GDK event machinery.

As is reasonable given our focus on writing the actual translation and JNI layer generators, most of these present only one or two methods from the underlying native class. Coverage will steadily improve as people contribute their knowledge and experience in documentation form.

People upgrading from 4.0.2 will actually notice that there are stubs for all of the public API classes; this was necessary to make the generated code compile. Most of these are empty as yet.

GList and friends

GList and GSList are the native GLib data structures used to represent lists and are the return type from quite a few methods across GTK. Vreixo Formoso Lopes worked out how to handle and working our wrap/unwrap functions for us to use in the JNI layer made a significant contribution to reducing our blacklisted method count.

Flags

We’ve also worked out handling of native types that, while type defined as enums, are actually bit fields.

Flags subclasses present an or() function allowing you to combine individual Flags constants into a new composite Flags object. Yes it’s a lot of machinery to do a logical |, but being strongly typed is a hallmark of java-gnome.

Build improvements

No release would be complete without mentioning that the code builds on more systems than it did before! Thanks to Maciej Piechotka and Srichand Pendyala for fixing problems resulting from Debian and Ubuntu strangeness.

The build internally now uses an optimized script that takes into account that even though the code generator may have run the translation and jni files may not actually be different. This was causing problems as Make only looks at file modification time. Instead, build/faster (great name, huh?) will only rebuild a target when source contents have changed. This was necessary for bindings hackers working in Eclipse; every time a file was saved Eclipse would merrily spawn off an auto-build which sooner or later would block the IDE UI. Yuk. Addressing this has also resulted in a faster build for everyone; all good.

The internal build script should be transparent; you still run ./configure and make as before. If you experience problems let us know.

Thread safety

In addition to the code generator, java-gnome 4.0.3 incorporates a comprehensive thread safety strategy.

None of the major Java graphical toolkits out there let you make GUI calls from threads other than the “main” one; they’re all single threaded. Even if all you want to do is a quick worker thread to carry out some input validation in the background after the user presses “OK”, you have to jump through horrific contortions to do so safely, resulting in cumbersome, clunky code.

By contrast, the new Java bindings of GTK presented in java-gnome are transparently thread safe, the first and only graphical user interface widget toolkit for Java to be so! We integrate properly with the underlying GDK thread lock and as a result you can safely make calls to various GTK methods from worker threads! This has been a long sought after goal and we hope a significant contribution to helping developers write elegant code.

Every call made to the native libraries is protected by entering the “GDK lock” [that’s gdk_threads_enter/leave() for those familiar with the C side of things]. The lock used is actually a Java side synchronized monitor and therefore reentrant; nested calls all behave properly. When in a signal handler callback the GDK lock is already held (you’re “in” the main loop when a callback happens), but since it just works transparently you don’t need to worry about it. If you do find a need to take the lock into account explicitly in your own code, see Gdk.lock in org.gnome.gdk.

It is worth noting that we have been warned that there are certain to be places in the underlying libraries that do not yet live up to the requirements of the GDK threads model — thus we will likely end up tripping over such things as we slowly add API coverage. We regard such inevitable instances as an opportunity to help contribute to improving the stability of the underlying libraries and will actively work with their maintainers to identify and resolve such issues. Nevertheless, in testing thus far our multi-threaded use of GTK has been rock solid. See these posts on GTK thread “awareness” and java-gnome’s thread strategy if you wish further details on our approach to the thread safety question.

Thanks in particular to Owen Taylor for having helped us navigate these waters!

Looking ahead

Taken together, these innovations represent the culmination of an immense amount of work towards realizing java-gnome as a viable platform for GTK and GNOME development.

Obviously with the generated translation layer in place the opportunity at last exists to start dramatically improving our coverage level, and we welcome contributions to this end. Prospective hackers are cautioned, however, that simply wrapping generated methods is insufficient — public API will only be added when it is clearly documented and meets the approachability criterion.

There are still areas where the code generator needs to be improved; we need to improve our handling for arrays, lists, and out-parameters — there are numerous permutations with all sorts of ugly corner cases.

Now that Free Java with support for generics is becoming widely available, 4.0.3 will be the last release holding the language level to Java 1.4; starting the next cycle 1.5 will be the minimum language requirement and we will be leveraging generics and other 1.5 features from here on.

Most importantly, the primary focus of the next few months will be developing a quality binding for the backbone of many applications: the TreeView Widget and the underlying TreeModel which powers it. The APIs in the native library is hideously complicated and has long been the source of confusion and pain for developers in C; it has long been a major goal amongst the java-gnome hackers to present a public API with as friendly and usable an interface as possible. It’ll be a good challenge.

You can download java-gnome from ftp.gnome.org or easily checkout a branch from mainline using Bazaar:

$ bzr branch http://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome

Back to work!

AfC

Friday, 08 Jun 2007

The code generator cometh

Calling C libraries from Java is somewhat arcane. It’s not so much that it’s “hard” as it is laborious. Once you’ve got your mind around it, however, and put the build infrastructure in place into your project (admittedly nasty learning curves), it’s not such a big deal to add another call or three on the margin. But for a library like java-gnome, which aspires to present a vast native libraries like GTK, each incremental function means an enormous amount of mundane and repetitive work.

Old way

In the old java-gnome, this Java and C code was hand written for every single method. Needless to day half the bugs that were in there were the kinds of errors that arise from madly doing so much cutting and pasting that your brain shuts off. Adding new coverage was insanely painful. No kidding it was abandoned.

New way

The primary motivation to re-engineer the Java bindings for GTK and GNOME was to switch from having to hand write all this glue code to an architecture whereby we could generate these complicated middle layers.

I released original prototype in November after 6 months of design work, and have been solidifying it in the java-gnome 4.0 releases so far. Obviously we had to mock up the parts that will be generated to make sure that we had the architecture right, which was no fun, because that’s right back to hand writing all the hideous boring shit in the middle. I’ve been pretty happy with it, though. The overall design has done a good job of handling unexpected corner cases as they’ve come up, and that’s always a good sign. Best of all has been the positive feedback from people who have looked at the code and complimented the design (we even had one guy come into #java-gnome the other day and say “java-gnome is really sexy”. Nice!).

But hand writing the glue code necessary to expand coverage sucks, especially when the whole point was to be generating all the boring bits. As you can see from this overview slide of the java-gnome 4.0 architecture,

it’s up to the code generator to do the bulk of the work. We needed to figure out the detailed design first, obviously, but clearly the next job is to get the machinery to output the necessary Java (code to translate from our public API into types we can safely ship across the Java/C boundary, and the native method declarations) and the corresponding C (the JNI code necessary to cast parameters and then actual call to the underlying library) … and of course the reverse direction for return values and callbacks like signals.

About 3 months ago we got started on it.

The codegen branch

The code generator has been evolving slowly but steadily. It started life as a little spike I wrote last September in Perl. That wasn’t going to scale, so Srichand Pendyala ported it to Python. He and I worked out some design issues there, but then we realized that that wasn’t going to scale either, so I ported it to Java. It’s been going gang busters since then, and we’ve since had lots of help from Vreixo Formoso and new contributor Sebastian Mancke. It’s getting to be quite the beasty.

The full stream of .defs data is massive; obviously the goal we’re pushing towards is to reach the point where the code generator can parse all the (define...) blocks therein and then output the appropriate Java and C code to implement each block.

So many corner cases. First time I tried it on the full data set it was not pretty :|. But we’ve been working away at it, and making painful but steady progress. Eventually I adopted the expediency of just blacklisting types that we didn’t know what to do with (an out-parameter that returns an array of arrays of GObject pointers? Fuck off!) — and finally last week I got a run that completed generation over the set of .defs files that represent GTK. That was an outstanding moment.

As of this writing, the java-gnome code generator outputs a translation layer of 475 classes containing 5454 methods (with 405 blacklisted due to requiring types that we haven’t figured out what to do with yet) and 733 callbacks (signal handler prototypes, mostly). There are still 162 odd compiler errors to go (some trivial, some really nasty), but we’re getting there!

Public API

This, of course, is just the glue in the middle. But this success means we are getting closer to the point where humans can get down to the serious work of writing the wrapper code and JavaDoc necessary to present those methods and signals we wish to expose in our public API.

One of the less appealing things about GTK in C is that numerous internals are unavoidably visible. In a language binding like java-gnome we don’t have to expose any of that — indeed, one of our primary design criteria has been zero such leakage. Thus there are a not insignificant number of entities in the .defs data that we don’t need to expose. GTK is huge, however, and it will take a long time to get our coverage up to a respectable level.

Where are we now? 58 public methods, 0.01%. That’d be radically embarrassing to admit, except that we gotta start somewhere. But it’s onwards and upwards from here — and although there are still quite a number of engineering design issues that have yet to be tackled, I don’t have to hand write JNI code any more, and neither do you.

The code generator cometh.

AfC

We make this service available to our staff in order to promote the discourse of ideas especially as relates to the development of Open Source worldwide. Blog entries on this site, however, are the musings of the authors as individuals and do not represent the views of Operational Dynamics. All times UTC.