GTK: OSX a11y support

Everybody knows that I have always been a firm believer in Gtk+’s potential to be a great cross platform toolkit beyond Linux. GIMP and Inkscape, as an example, are loyal users that ship builds for those platforms. The main challenge is the short amount of maintainers running, testing and improving those platforms.

Gtk+ has a few shortcomings one of them, one of the biggest ones is lack of a11y support outside of Linux. Since I have regular access to a modern OSX machine I decided to give this a go (and maybe learn son Obj-C in the process).

So I started by having a look at how ATK works and how it relates to the GTK DOM, my main goal was to have a GTK3 module that would walk through the toplevels and build an OSX accessibility tree.

So my initial/naive attempt is in this git repo, which you can build by installing gtk from brew.

Some of the shortcomings that I have found to actually test this and move forward:

  • Running gtk3-demo creates an NSApp that has no accessibility enabled, you can tell because the a11y inspector that comes with XCode won’t show metadata even for the window decorator controls. I have no idea how to enable that manually, it looks like starting an actual NSApp, like Inkscape and GIMP do would give you part of that.
  • Inkscape and GIMP have custom code to register themselves as an acutal NSApp as well as override XDG env vars in runtime to set the right paths. I suspect this is something we could move to G and GtkApplication.
  • The .dylib I generate with this repo will not actually load on Inkscape for some reason, so as of now I am stuck with having to somehow build a replacement gtk dylib for Inkscape with my code instead of through an actual module.

So this is my progress thus far, I think once I get to a point where I can iterate over the concept, it would be easier to start sketching the mapping between ATK and NSAccessibility. I would love feedback or help, so if you are interested please reach out by filing an issue on the gitlab project!

Chapter #Next: Kindle

Since the beginning of December I started working for Amazon in Madrid to run the team responsible for the Kindle experience on PC and the web. This is a bit of a career shift for me, for a while I’ve been wondering how would it be like to deliver a user experience of a successful consumer product, I have always been working on the seat of the OS/platform provider and more recently I have learned the hardware end by working with OEMs as part of my last role at Red Hat. However I have never been in the shoes of an ISV delivering an app for a vertical market, on top of that the position was advertised for Madrid for an onsite team, while working from home has many advantages running an entire team onsite is also a refreshing change for me, all in all it seemed like a cool opportunity where I could learn lots and try something different so I made the jump.

https://cdn.vox-cdn.com/thumbor/z96qIpPgQsSx9mHfH0-D1IW20DU=/0x0:1920x1080/920x613/filters:focal(807x387:1113x693):format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/57280131/kindle_app_logo.0.jpg

By the way, my team is hiring software engineers in Madrid, Spain, so if this is an area you are interested in please DM me on twitter.

 

Hanging the Red Hat

This is an extract of an email I just sent internally at Red Hat that I wanted to share with the wider GNOME and Freedesktop community.

After 6+ wonderful years at Red Hat, I’ve decided to hang the fedora to go and try new things. For a while I’ve been craving for a new challenge and I’ve felt the urge to try other things outside of the scope of Red Hat so with great hesitation I’ve finally made the jump.

I am extremely proud of the work done by the teams I have had the honour to run as engineering manager, I met wonderful people, I’ve worked with extremely talented engineers and learned lots. I am particularly proud of the achievements of my latest team from increasing the bootloader team and improving our relationship with GRUB upstream, to our wins at teaching Lenovo how to do upstream hardware support to improvements in Thunderbolt, Miracast, Fedora/RHEL VirtualBox guest compatibility… the list goes on and credit goes mostly to my amazing team.

Thanks to this job I have been able to reach out to other upstreams beyond GNOME, like Fedora, LibreOffice, the Linux Kernel, Rust, GRUB… it has been an amazing ride and I’ve met wonderful people in each one of them.

I would also like to make a special mention to my manager, Christian Schaller, who has supported me all the way in several ways both professionally and personally. There is this thing people say: “people do not leave companies, they leave managers”. Well this is certainly not the case, in Christian I have found not only a great manager but a true friend.

images

As for my experience at Red Hat, I have never lasted more than 2 years in the same spot before, I truly found my place there, deep in my heart I know I will always be a Red Hatter, but there are some things I want to try and learn elsewhere. This job switch has been the hardest departure I ever had and in many ways it breaks my heart to leave. If you are considering joining Red Hat, do not hesitate, there is no better place to write and advocate for Free Software.

I will announce what I will be doing next once I start in December.

Lessons when creating a C API from Rust

I have recently created a C API for a library dealing with Boot Loader Spec files as well as the GRUB environment file. In the process I have learnt a few things that, coming from a C background, were not obvious to me at all.

Box to control unsafe ownership

Say we have this simple Rust API:

pub struct A {
  counter: u8
}

impl A {
  pub fn new(count: u8) -> A {
    A { counter: count }
  }
}

Let’s start with the new method wrapper:

#[no_mangle]
pub extern "C" fn a_new(count: u8) -> *mut A {
  let boxed_a = Box::new(A {counter: count});
  Box::into_raw(boxed_a)
}

A Box is basically a smart pointer, it allows us to control the lifetime of the data outside of the boundaries of Rust’s borrow checker. Box::into_raw returns a pointer to the allocated A instance. Let’s see how to access that data again:

#[no_mangle]
pub extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = unsafe { Box::from_raw(a) };
  let count = a.counter;
  Box::into_raw(a);
  count
}

Box::from_raw is an unsafe method that turns a pointer into an owned Box, this allows us to access the pointer data safely from Rust. Note that Box is automatically dereferenced.

UPDATE: Sebastian Dröge has rightly pointed out that the above method is wrong, note that this is how I found most StackOverflow and other people explain how I should use the data again, but if Sebastian says it is wrong then I know it to be wrong ;-).

Turns out that casting the pointer as a reference inside an unsafe block is enough:

#[no_mangle]
pub unsafe extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = &*a;
  a.counter
}

 

Now we need to give the C user a deallocator for instances of A, this is relatively straightforward, we wrap the object around a Box and since we don’t call into_raw again, as soon as the Box is out of scope the inner contents are dropped too:

#[no_mangle]
pub unsafe extern "C" fn a_drop(a: *mut A) {
  Box::from_raw(a);
}

Strings

In Rust there are two standard ways to interact with strings, the String type, a dynamic utf-8 string that can be modified and resized, and &str, which basically is a bare pointer to an existing String. It took me a while to realize that internally a String is not null terminated and can contain many null characters. This means that the internal represenation of String is not compatible with C strings.

To address this, Rust provides another two types and referenced counterparts:

  • OsString and &OsStr: a native string tied to the runtime platform encoding and sizing
  • CString and &CStr: a null terminated string

My main grudge with this model when creating a C API is that it creates friction with the C boundary for a couple of reasons:

  • Internal Rust APIs often expect String or &str, meaning that at the C API boundary you need to allocate a CString if you use String as the internal representation, or the other way around if you use CString as the internal representation
  • You can’t “transparently” pass ownership of a CString to C without a exposing a deallocator specific to CString, more on this on the next section.

This means that compared to a C implementation of the API your code will liekly use more allocations which might or might not be critical depending on the use case, but this is something that struck me as a drawback for Rustification.

UPDATE: I am talking solely about the C API boundary, Rust is _great_ at giving you tools to avoid extra allocations (yay slices!), you can create a parser of a large chunk of text without allocating any extra strings to chunk the source text around.

Allocator mismatch

Something else I stumbled upon was that Rust does not use malloc/free, and that mismatch has annoying side effects when you are trying to rustify an API. Say you have this C code:

char* get_name() {
  const char* STATIC_NAME = "John";
  char* name = (char*)malloc(sizeof(STATIC_NAME));
  memcpy(name, STATIC_NAME, sizeof(STATIC_NAME));
}

int main () {
  char * name = get_name();
  printf("%s\n", name);
  free(name);
  return 0;
}

Now if you want to Rustify that C function, the naive way (taking into account the String vs. CString stuff I mentioned before) would be to do this:

#[no_mangle]
pub extern "C" fn get_name() -> *mut std::os::raw::c_char {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  name.into_raw()
}

But this is not exactly the same as before, note that in the C example we call free() in order to drop the memory. In this case we would have to create a new method that calls CString::from_raw() but that won’t be compatible with the original C API.

This is the best I was able to came up with:

/* You can use the libc crate as well */
extern {
  fn malloc(size: usize) -> *mut u8;
  fn memcpy(dest: *mut u8, src: *const u8, size: usize) -> *mut u8;
}

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  let length = name.as_bytes_with_nul().len();
  let cname = unsafe { malloc(length) };
  unsafe { memcpy(cname, name.as_bytes_with_nul().as_ptr(), length) };
  cname
}

Note that STATIC_NAME is just an example, usually the data comes from a String/&str in your Rust API. The problem here is that we allocated an extra CString to then copy its contents using malloc/memcpy and then drop it immediately.

However, later, while working on creating UEFI binaries from Rust, I learned that Rust allows you to override its own allocator and use a custom one or the native system one. This would be another way to achieve the same and save the malloc/memcpy step, but don’t trust me 100% here as I am not sure whether this is entirely safe (if you know, let me know in the comments).

UPDATE: Many people have pointed out that overriding the allocator to use the system is absolutely fine:

use std::alloc::System;

#[global_allocator]
static GLOBAL: System = System;

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME).expect("Multiple null characters in string");
  name.into_raw() as *mut u8
}

Traits as fat pointers

Let’s say we have the following API with two types and a trait implemented by both:

pub struct A {}
pub struct B {}

impl A {
  pub fn new () -> A { A{} }
}

impl B {
  pub fn new () -> B { B{} }
}

pub trait T {
  fn get_name(&self) -> std::ffi::CString;
}

impl T for A {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am A").expect("CString error")
  }
}

impl T for B {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am B").expect("CString error")
  }
}

Now the problem is, if we want a single wrapper for T::get_name() to avoid having to wrap each trait implementation family of functions, what do we do? I banged my head on this trying to Box a reference to a trait and other things until I read about this in more detail. Basically, the internal representation of a trait is a fat pointer (or rather, a struct of two pointers, one to the data and another to the trait vtable).

So we can transmute a reference to a trait as a C struct of two pointers, the end result for type A would be like this (for B you just need another constructor and cast function):

#[repr(C)]
pub struct CTrait {
  data: *mut std::os::raw::c_void,
  vtable: *mut std::os::raw::c_void
}

#[no_mangle]
pub extern "C" fn a_new() -> *mut A {
  Box::into_raw(Box::new(A::new()))
}

#[no_mangle]
pub extern "C" fn a_drop(a: *mut A) {
  unsafe{ Box::from_raw(a) };
}

#[no_mangle]
pub extern "C" fn a_as_t (a: *mut A) -> CTrait {
  let mut boxed_a = unsafe { Box::from_raw(a) };
  let ret: CTrait = {
    let t: &mut dyn T = &mut *boxed_a;
    unsafe { std::mem::transmute::<&mut dyn T,CTrait> (t) }
  };
  Box::into_raw(boxed_a);
  ret
}

#[no_mangle]
pub extern "C" fn t_get_name(t: CTrait) -> *mut u8 {
  let t = unsafe { std::mem::transmute::<CTrait, &mut dyn T> (t) };
  t.get_name().into_raw() as *mut u8
}

The C code to consume this API would look like this:

typedef struct {} A;
typedef struct {
  void* _d;
  void* _v;
} CTrait;

A*      a_new();
void    a_drop(A* a);
CTrait  a_as_t(A* a);
char*   t_get_name(CTrait);

int main () {
  A* a = a_new();
  CTrait t = a_as_t(a);
  char* name = t_get_name(t);
  printf("%s\n", name);
  free(name);
  a_drop(a);
  return 0;
}

Error reporting

Another hurdle has been dealing with Result<> in general, however this is more of a shortcoming of C’s lack of standard error reporting mechanism. In general I tend to return NULL to C API calls that expect a pointer and let C handle it, but of course data is lost in the way as the C end has no way to know what exactly went wrong as there is no error type to query. I am tempted to mimick GLib’s error handling. I think that if I was trying to replace an existing C library with its own error reporting mapping things would become easier.

Conclusions

I am in love with Rust and its ability to impersonate C is very powerful, however it is note entirely 0 cost, for me, the mismatch between string formats is the biggest hurdle as it imposes extra allocations, something that could become really expensive when rustifying C code that passes strings back and forth from/to the API caller. The other things I mentioned were things that took me quite some time to realize and by writing it here I hope I help other people that are writing Rust code to expose it as a C API. Any feedback on my examples is welcome.

GNOME Performance Hackfest

We’re about to finish the three days long first GNOME Performance Hackfest here in Cambridge.

We started covering a few topics, there are three major areas we’ve covered and in each one of those there has been a bunch of initiatives.

photo_2018-05-15_16-05-20

photo_2018-05-16_16-45-16

 

 

GNOME Shell performance

Jonas Adahl, Marco Trevisan, Eric Anholt, Emmanuele Bassi, Carlos Garnacho and Daniel Stone have been flocking together around Shell performance. There has been some high level discussions about the pipeline, Clutter, Cogl, cairo/gtk3 and gtk4.

The main effort has been around creating probes across the stack to help Christian Hergert with sysprof (in drm, mutter, gjs…) so that we can actually measure performance bottlenecks at different levels and pinpoint culprits.

We’ve been also looking at the story behind search providers and see if we can rearchitect things a bit to have less roundtrips and avoid persistent session daemons to achieve the same results. Discussions are still ongoing on that front.

GNOME Session resource consumption

Hans de Goede put together a summary of the resource consumed in a typical GNOME session in Fedora and tweaks to avoid those, you can check the list in the agenda.

There are some issues specific to Fedora there, but the biggest improvement that we can achieve is shutting down’s GDM’s own gnome-shell instance, for which Hans already has a working patch. This should reduce resource consumption by 280megs of RAM.

The second biggest target is GNOME Software, which we keep running primarily for the shell provider. Richard Hughes was here yesterday and is already working on a solution for this.

We are also looking into the different GNOME Settings Daemon processes and trying to figure out which ones we can shut down until needed.

Surely there’s stuff I’ve missed, and hopefully we’ll see blogposts and patches surfacing soon after we wrap up the event. Hopefully we can follow up during GUADEC and start showing the results.

On Tuesday we enjoyed some drinks out kindly hosted by Collabora.

collabora-logo-small
I’d like to thank the Eben Upton and the Raspberry Pi Foundation for sponsoring the venue and sending Eric Anholt over.

raspberry-pi-logo-8240ABBDFE-seeklogo.com

Rust 💙 GNOME Hackfest: Day 1

Hello everyone,

This is a report of the first day of the Rust 💙 GNOME Hackfest that we are having in Madrid at the moment. During the first day we had a round of introductions and starting outlining the state of the art.

IMG_20180418_095328
A great view of Madrid’s skyline from OpenShine’s offices

At the moment most of the focus is around the gobject_gen! macro, the one that allows us to inherit GObjects. We already have basic inheritance support, private structure, signals and methods. The main outstanding issues are:

  • Properties support, which danigm is working on
  • Improve compiler errors for problems within the macro, which antonyo and federico worked on
  • Interface implementation and declaration
  • Cover more Rust<->GObject/GLib/C type conversion coverage

I’ve been focusing mostly on improving the support for the private structure of GObject classes. At the moment this is how you do it:

struct MyPrivateStruct {
        val: Cell<u32>,
}

gobject_gen! {
    class Foo {
        type InstancePrivate = MyPrivateStruct;
    }

    pub fn my_method(&self) {
        self.get_priv().val.set(30);
    }
}

Which I find rather cumbersome, I’m aiming at doing something like this:

gobject_gen! {
    class Foo {
        val: Cell<u32>;
    }

    pub fn my_method(&self) {
        self.val.set(30);
    }
}

Which is a bit more intuitive and less verbose. The private struct is generated behind the scenes and the de-referencing is done through the Deref trait. The challenge now is how to allow for default values for the private structure. This was done through the Default trait on the current implementation, but I’m not sure this is the best way forward to make this easy to use.

By the way, the hackfest is being kindly hosted by our friends at OpenShine, which besides being free software enthusiasts and supporters, are great at deploying kubernetes, Big Data and analytics solutions.
logo-alta-alpha1

1+ year of Fedora and GNOME hardware enablement

A year and a couple of months ago, Christian Schaller asked me to pivot a little bit from working full time on Fleet Commander to manage a new team we were building to work on client hardware enablement for Fedora and GNOME with an emphasis on upstream. The idea was to fill the gap in the organization where nobody really owned the problem of bringing up new client hardware features vertically across the stack (from shell down to the kernel), or rather, ensure Fedora and GNOME both work great on modern laptops. Part of that deal was to take over the bootloader and start working closer to customers and hardware manufacturing parnters.

15525894974_17d28e4e5c_z.jpg
Sound Blaster 16 PnP, Danipuntocom @ Flickr, CC BY-NC 2.0

At first I hesitated as I wasn’t sure I could do a good job, y’all know how the impostor syndrome works specially outside of the comfort zone, also had very little engineering experience on the kernel or hardware related fields outside the hardware design I did at uni.

However after some thinking I thought this was a terribly exciting prospect and I had some ideas as to how to go about it and do a decent job.

Fast forward 16 months and I’m loving it, in a relatively short period of time we’ve been able to build an amazing team that have been able to execute quite a few important highlights to make Fedora and GNOME work better on laptops:

  • Peter Jones and Javier Martinez are taking care of the bootloader stack for Fedora, from GRUB2 to the UEFI tooling, including the secure boot and the low level bits of the firmware update mechanisms. Our current efforts revolve around http boot, enabling TPM2 in the bootloader for Trusted Boot and implement the Boot Loader Spec in Fedora across the supported architectures to improve reliability when updating kernels, BLS prevents you from needing to generate a GRUB configuration file every time a kernel is installed or removed.
  • Hans de Goede has been working on some neglected areas hardware support wise. When he was transferred he was still working on improving optimus support for NVIDIA hardware. You’ve probably seen two major highlights from Hans’ work lately, one is us spending some time to help VirtualBox upstream their guest drivers since we wanted to make Fedora and the Linux ecosystem at large work out of the box (pun not intended), this is really important as VirtualBox is the first approach many Windows and Mac users have to a Linux operating system and desktop, so we’ve decided to treat it as an important hardware platform for us as we do with KVM/QEMU/GNOME Boxes. He has also been working  and most recently he has hit headlines with his amazing and thorough work on improving battery life in Fedora by trying to gather data on which power saving defaults we can enable safely on which devices.
  • Christian Kellner has been doing tons of vertical integration, first he revamped GNOME Battery Bench, to improve battery teststing and gather better data about consumption  when we try new laptops. He has also been looking at fingerprint, bluetooth and pulseaudio issues. But more recently he has taken the torch to implement Thunderbolt 3 Security levels. This is a pretty big deal and has required a ton of design work with jimmac and Bastien. For those unaware, this is a feature enabled by default in modern laptops that will prevent Thuderbolt devices to be enabled just by plugging them and it will require some sort of authorization from the user. This is important as Thunderbolt is basically a PCI-e bus slave that has access to your entire system so malicious peripherals or cables could access sensitive data.
  • Last but not least, Benjamin Berg has been doing a lot of work behind the scenes to improve laptop testing and coming up with a testing suite people can use to test a laptop and bring back a standardized results to Fedora to keep track of regressions and gaps on specific laptop models. We’re trying to automate as much as we can but we still have to write a lot of manual tests for this. This is ongoing work but I think it’s going to help Fedora and the larger Linux ecosystem to be more thorough when it comes to testing hardware and preventing regressions.

Beyond the engineering efforts, we are working with OEMs and silicon vendors as well to try to educate them in the difficult transition from the proprietary OS model to contribute upstream. Some of them are doing really great work and others need improvement. While I can’t share any specific details of these conversations, I must say it’s an incredibly exciting moment for Linux on laptops/workstations and if we are able to push enough silicon vendors to have a more fluent relationship with upstream I think we really have a chance to reduce the problems people have with newer hardware at least on the enterprise offerings at first.

I have to say, it is an incredibly humbling experience to work with this team, I’m learning a lot about the space and I’m excited about the things we’re planning for the next couple of years and the opportunities those efforts could bring for the free software desktop.