Hanging the Red Hat

This is an extract of an email I just sent internally at Red Hat that I wanted to share with the wider GNOME and Freedesktop community.

After 6+ wonderful years at Red Hat, I’ve decided to hang the fedora to go and try new things. For a while I’ve been craving for a new challenge and I’ve felt the urge to try other things outside of the scope of Red Hat so with great hesitation I’ve finally made the jump.

I am extremely proud of the work done by the teams I have had the honour to run as engineering manager, I met wonderful people, I’ve worked with extremely talented engineers and learned lots. I am particularly proud of the achievements of my latest team from increasing the bootloader team and improving our relationship with GRUB upstream, to our wins at teaching Lenovo how to do upstream hardware support to improvements in Thunderbolt, Miracast, Fedora/RHEL VirtualBox guest compatibility… the list goes on and credit goes mostly to my amazing team.

Thanks to this job I have been able to reach out to other upstreams beyond GNOME, like Fedora, LibreOffice, the Linux Kernel, Rust, GRUB… it has been an amazing ride and I’ve met wonderful people in each one of them.

I would also like to make a special mention to my manager, Christian Schaller, who has supported me all the way in several ways both professionally and personally. There is this thing people say: “people do not leave companies, they leave managers”. Well this is certainly not the case, in Christian I have found not only a great manager but a true friend.

images

As for my experience at Red Hat, I have never lasted more than 2 years in the same spot before, I truly found my place there, deep in my heart I know I will always be a Red Hatter, but there are some things I want to try and learn elsewhere. This job switch has been the hardest departure I ever had and in many ways it breaks my heart to leave. If you are considering joining Red Hat, do not hesitate, there is no better place to write and advocate for Free Software.

I will announce what I will be doing next once I start in December.

Lessons when creating a C API from Rust

I have recently created a C API for a library dealing with Boot Loader Spec files as well as the GRUB environment file. In the process I have learnt a few things that, coming from a C background, were not obvious to me at all.

Box to control unsafe ownership

Say we have this simple Rust API:

pub struct A {
  counter: u8
}

impl A {
  pub fn new(count: u8) -> A {
    A { counter: count }
  }
}

Let’s start with the new method wrapper:

#[no_mangle]
pub extern "C" fn a_new(count: u8) -> *mut A {
  let boxed_a = Box::new(A {counter: count});
  Box::into_raw(boxed_a)
}

A Box is basically a smart pointer, it allows us to control the lifetime of the data outside of the boundaries of Rust’s borrow checker. Box::into_raw returns a pointer to the allocated A instance. Let’s see how to access that data again:

#[no_mangle]
pub extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = unsafe { Box::from_raw(a) };
  let count = a.counter;
  Box::into_raw(a);
  count
}

Box::from_raw is an unsafe method that turns a pointer into an owned Box, this allows us to access the pointer data safely from Rust. Note that Box is automatically dereferenced.

UPDATE: Sebastian Dröge has rightly pointed out that the above method is wrong, note that this is how I found most StackOverflow and other people explain how I should use the data again, but if Sebastian says it is wrong then I know it to be wrong ;-).

Turns out that casting the pointer as a reference inside an unsafe block is enough:

#[no_mangle]
pub unsafe extern "C" fn a_get_counter(a: *mut A) -> u8 {
  let a = &*a;
  a.counter
}

 

Now we need to give the C user a deallocator for instances of A, this is relatively straightforward, we wrap the object around a Box and since we don’t call into_raw again, as soon as the Box is out of scope the inner contents are dropped too:

#[no_mangle]
pub unsafe extern "C" fn a_drop(a: *mut A) {
  Box::from_raw(a);
}

Strings

In Rust there are two standard ways to interact with strings, the String type, a dynamic utf-8 string that can be modified and resized, and &str, which basically is a bare pointer to an existing String. It took me a while to realize that internally a String is not null terminated and can contain many null characters. This means that the internal represenation of String is not compatible with C strings.

To address this, Rust provides another two types and referenced counterparts:

  • OsString and &OsStr: a native string tied to the runtime platform encoding and sizing
  • CString and &CStr: a null terminated string

My main grudge with this model when creating a C API is that it creates friction with the C boundary for a couple of reasons:

  • Internal Rust APIs often expect String or &str, meaning that at the C API boundary you need to allocate a CString if you use String as the internal representation, or the other way around if you use CString as the internal representation
  • You can’t “transparently” pass ownership of a CString to C without a exposing a deallocator specific to CString, more on this on the next section.

This means that compared to a C implementation of the API your code will liekly use more allocations which might or might not be critical depending on the use case, but this is something that struck me as a drawback for Rustification.

UPDATE: I am talking solely about the C API boundary, Rust is _great_ at giving you tools to avoid extra allocations (yay slices!), you can create a parser of a large chunk of text without allocating any extra strings to chunk the source text around.

Allocator mismatch

Something else I stumbled upon was that Rust does not use malloc/free, and that mismatch has annoying side effects when you are trying to rustify an API. Say you have this C code:

char* get_name() {
  const char* STATIC_NAME = "John";
  char* name = (char*)malloc(sizeof(STATIC_NAME));
  memcpy(name, STATIC_NAME, sizeof(STATIC_NAME));
}

int main () {
  char * name = get_name();
  printf("%s\n", name);
  free(name);
  return 0;
}

Now if you want to Rustify that C function, the naive way (taking into account the String vs. CString stuff I mentioned before) would be to do this:

#[no_mangle]
pub extern "C" fn get_name() -> *mut std::os::raw::c_char {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  name.into_raw()
}

But this is not exactly the same as before, note that in the C example we call free() in order to drop the memory. In this case we would have to create a new method that calls CString::from_raw() but that won’t be compatible with the original C API.

This is the best I was able to came up with:

/* You can use the libc crate as well */
extern {
  fn malloc(size: usize) -> *mut u8;
  fn memcpy(dest: *mut u8, src: *const u8, size: usize) -> *mut u8;
}

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME)
               .expect("Multiple null characters in string");
  let length = name.as_bytes_with_nul().len();
  let cname = unsafe { malloc(length) };
  unsafe { memcpy(cname, name.as_bytes_with_nul().as_ptr(), length) };
  cname
}

Note that STATIC_NAME is just an example, usually the data comes from a String/&str in your Rust API. The problem here is that we allocated an extra CString to then copy its contents using malloc/memcpy and then drop it immediately.

However, later, while working on creating UEFI binaries from Rust, I learned that Rust allows you to override its own allocator and use a custom one or the native system one. This would be another way to achieve the same and save the malloc/memcpy step, but don’t trust me 100% here as I am not sure whether this is entirely safe (if you know, let me know in the comments).

UPDATE: Many people have pointed out that overriding the allocator to use the system is absolutely fine:

use std::alloc::System;

#[global_allocator]
static GLOBAL: System = System;

#[no_mangle]
pub extern "C" fn get_name() -> *mut u8 {
  const STATIC_NAME: &str = "John";
  let name = std::ffi::CString::new(STATIC_NAME).expect("Multiple null characters in string");
  name.into_raw() as *mut u8
}

Traits as fat pointers

Let’s say we have the following API with two types and a trait implemented by both:

pub struct A {}
pub struct B {}

impl A {
  pub fn new () -> A { A{} }
}

impl B {
  pub fn new () -> B { B{} }
}

pub trait T {
  fn get_name(&self) -> std::ffi::CString;
}

impl T for A {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am A").expect("CString error")
  }
}

impl T for B {
  fn get_name(&self) -> std::ffi::CString {
    std::ffi::CString::new("I am B").expect("CString error")
  }
}

Now the problem is, if we want a single wrapper for T::get_name() to avoid having to wrap each trait implementation family of functions, what do we do? I banged my head on this trying to Box a reference to a trait and other things until I read about this in more detail. Basically, the internal representation of a trait is a fat pointer (or rather, a struct of two pointers, one to the data and another to the trait vtable).

So we can transmute a reference to a trait as a C struct of two pointers, the end result for type A would be like this (for B you just need another constructor and cast function):

#[repr(C)]
pub struct CTrait {
  data: *mut std::os::raw::c_void,
  vtable: *mut std::os::raw::c_void
}

#[no_mangle]
pub extern "C" fn a_new() -> *mut A {
  Box::into_raw(Box::new(A::new()))
}

#[no_mangle]
pub extern "C" fn a_drop(a: *mut A) {
  unsafe{ Box::from_raw(a) };
}

#[no_mangle]
pub extern "C" fn a_as_t (a: *mut A) -> CTrait {
  let mut boxed_a = unsafe { Box::from_raw(a) };
  let ret: CTrait = {
    let t: &mut dyn T = &mut *boxed_a;
    unsafe { std::mem::transmute::<&mut dyn T,CTrait> (t) }
  };
  Box::into_raw(boxed_a);
  ret
}

#[no_mangle]
pub extern "C" fn t_get_name(t: CTrait) -> *mut u8 {
  let t = unsafe { std::mem::transmute::<CTrait, &mut dyn T> (t) };
  t.get_name().into_raw() as *mut u8
}

The C code to consume this API would look like this:

typedef struct {} A;
typedef struct {
  void* _d;
  void* _v;
} CTrait;

A*      a_new();
void    a_drop(A* a);
CTrait  a_as_t(A* a);
char*   t_get_name(CTrait);

int main () {
  A* a = a_new();
  CTrait t = a_as_t(a);
  char* name = t_get_name(t);
  printf("%s\n", name);
  free(name);
  a_drop(a);
  return 0;
}

Error reporting

Another hurdle has been dealing with Result<> in general, however this is more of a shortcoming of C’s lack of standard error reporting mechanism. In general I tend to return NULL to C API calls that expect a pointer and let C handle it, but of course data is lost in the way as the C end has no way to know what exactly went wrong as there is no error type to query. I am tempted to mimick GLib’s error handling. I think that if I was trying to replace an existing C library with its own error reporting mapping things would become easier.

Conclusions

I am in love with Rust and its ability to impersonate C is very powerful, however it is note entirely 0 cost, for me, the mismatch between string formats is the biggest hurdle as it imposes extra allocations, something that could become really expensive when rustifying C code that passes strings back and forth from/to the API caller. The other things I mentioned were things that took me quite some time to realize and by writing it here I hope I help other people that are writing Rust code to expose it as a C API. Any feedback on my examples is welcome.

GNOME Performance Hackfest

We’re about to finish the three days long first GNOME Performance Hackfest here in Cambridge.

We started covering a few topics, there are three major areas we’ve covered and in each one of those there has been a bunch of initiatives.

photo_2018-05-15_16-05-20

photo_2018-05-16_16-45-16

 

 

GNOME Shell performance

Jonas Adahl, Marco Trevisan, Eric Anholt, Emmanuele Bassi, Carlos Garnacho and Daniel Stone have been flocking together around Shell performance. There has been some high level discussions about the pipeline, Clutter, Cogl, cairo/gtk3 and gtk4.

The main effort has been around creating probes across the stack to help Christian Hergert with sysprof (in drm, mutter, gjs…) so that we can actually measure performance bottlenecks at different levels and pinpoint culprits.

We’ve been also looking at the story behind search providers and see if we can rearchitect things a bit to have less roundtrips and avoid persistent session daemons to achieve the same results. Discussions are still ongoing on that front.

GNOME Session resource consumption

Hans de Goede put together a summary of the resource consumed in a typical GNOME session in Fedora and tweaks to avoid those, you can check the list in the agenda.

There are some issues specific to Fedora there, but the biggest improvement that we can achieve is shutting down’s GDM’s own gnome-shell instance, for which Hans already has a working patch. This should reduce resource consumption by 280megs of RAM.

The second biggest target is GNOME Software, which we keep running primarily for the shell provider. Richard Hughes was here yesterday and is already working on a solution for this.

We are also looking into the different GNOME Settings Daemon processes and trying to figure out which ones we can shut down until needed.

Surely there’s stuff I’ve missed, and hopefully we’ll see blogposts and patches surfacing soon after we wrap up the event. Hopefully we can follow up during GUADEC and start showing the results.

On Tuesday we enjoyed some drinks out kindly hosted by Collabora.

collabora-logo-small
I’d like to thank the Eben Upton and the Raspberry Pi Foundation for sponsoring the venue and sending Eric Anholt over.

raspberry-pi-logo-8240ABBDFE-seeklogo.com

Rust đź’™ GNOME Hackfest: Day 1

Hello everyone,

This is a report of the first day of the Rust đź’™ GNOME Hackfest that we are having in Madrid at the moment. During the first day we had a round of introductions and starting outlining the state of the art.

IMG_20180418_095328
A great view of Madrid’s skyline from OpenShine’s offices

At the moment most of the focus is around the gobject_gen! macro, the one that allows us to inherit GObjects. We already have basic inheritance support, private structure, signals and methods. The main outstanding issues are:

  • Properties support, which danigm is working on
  • Improve compiler errors for problems within the macro, which antonyo and federico worked on
  • Interface implementation and declaration
  • Cover more Rust<->GObject/GLib/C type conversion coverage

I’ve been focusing mostly on improving the support for the private structure of GObject classes. At the moment this is how you do it:

struct MyPrivateStruct {
        val: Cell<u32>,
}

gobject_gen! {
    class Foo {
        type InstancePrivate = MyPrivateStruct;
    }

    pub fn my_method(&self) {
        self.get_priv().val.set(30);
    }
}

Which I find rather cumbersome, I’m aiming at doing something like this:

gobject_gen! {
    class Foo {
        val: Cell<u32>;
    }

    pub fn my_method(&self) {
        self.val.set(30);
    }
}

Which is a bit more intuitive and less verbose. The private struct is generated behind the scenes and the de-referencing is done through the Deref trait. The challenge now is how to allow for default values for the private structure. This was done through the Default trait on the current implementation, but I’m not sure this is the best way forward to make this easy to use.

By the way, the hackfest is being kindly hosted by our friends at OpenShine, which besides being free software enthusiasts and supporters, are great at deploying kubernetes, Big Data and analytics solutions.
logo-alta-alpha1

1+ year of Fedora and GNOME hardware enablement

A year and a couple of months ago, Christian Schaller asked me to pivot a little bit from working full time on Fleet Commander to manage a new team we were building to work on client hardware enablement for Fedora and GNOME with an emphasis on upstream. The idea was to fill the gap in the organization where nobody really owned the problem of bringing up new client hardware features vertically across the stack (from shell down to the kernel), or rather, ensure Fedora and GNOME both work great on modern laptops. Part of that deal was to take over the bootloader and start working closer to customers and hardware manufacturing parnters.

15525894974_17d28e4e5c_z.jpg
Sound Blaster 16 PnP, Danipuntocom @ Flickr, CC BY-NC 2.0

At first I hesitated as I wasn’t sure I could do a good job, y’all know how the impostor syndrome works specially outside of the comfort zone, also had very little engineering experience on the kernel or hardware related fields outside the hardware design I did at uni.

However after some thinking I thought this was a terribly exciting prospect and I had some ideas as to how to go about it and do a decent job.

Fast forward 16 months and I’m loving it, in a relatively short period of time we’ve been able to build an amazing team that have been able to execute quite a few important highlights to make Fedora and GNOME work better on laptops:

  • Peter Jones and Javier Martinez are taking care of the bootloader stack for Fedora, from GRUB2 to the UEFI tooling, including the secure boot and the low level bits of the firmware update mechanisms. Our current efforts revolve around http boot, enabling TPM2 in the bootloader for Trusted Boot and implement the Boot Loader Spec in Fedora across the supported architectures to improve reliability when updating kernels, BLS prevents you from needing to generate a GRUB configuration file every time a kernel is installed or removed.
  • Hans de Goede has been working on some neglected areas hardware support wise. When he was transferred he was still working on improving optimus support for NVIDIA hardware. You’ve probably seen two major highlights from Hans’ work lately, one is us spending some time to help VirtualBox upstream their guest drivers since we wanted to make Fedora and the Linux ecosystem at large work out of the box (pun not intended), this is really important as VirtualBox is the first approach many Windows and Mac users have to a Linux operating system and desktop, so we’ve decided to treat it as an important hardware platform for us as we do with KVM/QEMU/GNOME Boxes. He has also been working  and most recently he has hit headlines with his amazing and thorough work on improving battery life in Fedora by trying to gather data on which power saving defaults we can enable safely on which devices.
  • Christian Kellner has been doing tons of vertical integration, first he revamped GNOME Battery Bench, to improve battery teststing and gather better data about consumption  when we try new laptops. He has also been looking at fingerprint, bluetooth and pulseaudio issues. But more recently he has taken the torch to implement Thunderbolt 3 Security levels. This is a pretty big deal and has required a ton of design work with jimmac and Bastien. For those unaware, this is a feature enabled by default in modern laptops that will prevent Thuderbolt devices to be enabled just by plugging them and it will require some sort of authorization from the user. This is important as Thunderbolt is basically a PCI-e bus slave that has access to your entire system so malicious peripherals or cables could access sensitive data.
  • Last but not least, Benjamin Berg has been doing a lot of work behind the scenes to improve laptop testing and coming up with a testing suite people can use to test a laptop and bring back a standardized results to Fedora to keep track of regressions and gaps on specific laptop models. We’re trying to automate as much as we can but we still have to write a lot of manual tests for this. This is ongoing work but I think it’s going to help Fedora and the larger Linux ecosystem to be more thorough when it comes to testing hardware and preventing regressions.

Beyond the engineering efforts, we are working with OEMs and silicon vendors as well to try to educate them in the difficult transition from the proprietary OS model to contribute upstream. Some of them are doing really great work and others need improvement. While I can’t share any specific details of these conversations, I must say it’s an incredibly exciting moment for Linux on laptops/workstations and if we are able to push enough silicon vendors to have a more fluent relationship with upstream I think we really have a chance to reduce the problems people have with newer hardware at least on the enterprise offerings at first.

I have to say, it is an incredibly humbling experience to work with this team, I’m learning a lot about the space and I’m excited about the things we’re planning for the next couple of years and the opportunities those efforts could bring for the free software desktop.

Fleet Commander is looking for a GSoC student to help us take over the world

Fleet Commander has seen quite a lot of progress recently, of which I should blog about soon. For those unaware, Fleet Commander is an effort to make GNOME great for IT administrators in large deployments, allowing them to deploy desktop and application configuration profiles across hundreds of machines with ease through a web administration UI based on Cockpit. It is mostly implemented in Python.

One important aspect of large deployments is their identity management systems to handle large numbers of users, groups and hosts. On the free software end, FreeIPA is the project that we’ve integrated Fleet Commander with. FreeIPA is an administrative interface and sets of APIs that integrates LDAP directory, DNS and other related services together. Another way to describe FreeIPA is as the Linux’s counterpart of Microsoft’s Active Directory.
logoactivedirectory.png
And that’s precisely what the GSoC idea we want a student for is about, we think that the best way to encourage GNOME usage in large organizations is to have tools that ease the migration, many organizations may have an existing Microsoft Windows deployment managed by Active Directory, so we want Fleet Commander to be able to use Active Directory as well as FreeIPA as the identity management system and data store for the profile data (by using Group Policy Objects).

This project would be mostly implemented in Python and it will require talking to AD’s LDAP server and CIFS/Samba storage, we are fairly confident that it can be achieved during the GSoC term.

If you are looking for a fun GSoC project, you’re skilled in Python and are interested in becoming a GNOME contributor by helping it reach a larger user base and take it one step closer to world domination and make some money in the process you should apply!

We’re hanging around in the #fleet-commander IRC channel in irc.freenode.net if you want to approach us to get a better understanding of the idea, look for ogutierrez, fidencio and aruiz if you have any questions.

GUADEC 2017: GNOME’s Renaissance

NOTE: This is a blog post I kept as a draft right after GUADEC to reflect on it and the GNOME project but failed to finish and publish until now. Forgive any outdated information though I think the post is mostly relevant still.

I’m on my train back to London from Manchester, where I just spent 7 amazing days with my fellow GNOME community members. Props to the local team for an amazing organization, everything went smoothly and people seemed extremely pleased with the setup as far as I can tell and the venues seemed to have worked extremely well. I mostly want to reflect on a feeling that I have which is that GNOME seems to be experiencing a renaissance in the energy and focus of the community as well as the broader interest from other players.

35538262073_de005dfb31_z
Source: kitty-kat @ flickr CC BY-SA 4.0

Peak attendance and sponsorship

Our attendance numbers have gone up considerably from the most recent years, approximately 260 registrations, minus a bunch of people who could not make it in the end. That is an increase of ~50-60 attendants which is really encouraging.

On top of that this years’ sponsorships went up both in the number of companies sponsoring and supporting the event, this is really encouraging as it shows that there is an increased interest in the project and acknowledgement that GUADEC

Comebacks

There are two comebacks that were actually very encouraging, first, Canonical and Ubuntu community members are back, and it was really encouraging to see them participating. Additionally, members of the Solaris Desktop team showed up too. Also, having Andrew Walton from VMWare around was encouraging too.

Balance was brought back to the force

Historically Red Hat would have a prominent presence at GUADEC and in GNOME in general, it was really amazing to see that Endless is now a Foundation AdBoard member, but also, that the amount of Endless crew at GUADEC matched that of Red Hat.

Contrary of what people may think, Red Hat does not generally enjoy being the only player in any given free software community, however since Nokia and Sun/Oracle retreated their heavy investment in GNOME, Red Hat was probably the most prominent player in the community until now. While Red Hat is still investing as heavily as ever in GNOME, we’re not the only major player anymore, and that is something to celebrate and to look forward to see expanded to other organizations.

It was particularly encouraging to see the design sessions packed with people from Endless, Canonical and Red Hat as well as many other interested individuals.

Flathub

It feels to me that Flatpak, and specially the current efforts around Flathub have helped focused the community towards a common vision for the developer and the user story and you can feel the common excitement that we’re onto something and its implications across the stack and our infrastructure.

Obviously, not everybody shares this enthusiasm around flatpak itself, but the broad consensus is that the model around it is worth pursuing and that it has the potential to raise considerably the viability of the Free Software desktop and personal devices, not to mention, it gives a route towards monetization of free software and Linux apps.

GitLab, Meson and BuildStream

Another batch of modernization in our stack and infrastructure. First and foremost, the GitLab migration which we believe it will not only improve the interaction of newcomers and early testers as well as improving our Continuous Integration pipeline.

Consensus around Meson and leaving autotools behind is another big step and many other relevant free software projects are jumping to the bandwagon. And last but not least, Tristan and Codethink are leading an effort to consolidate continuous and jhbuild into BuildStream, a better way to build a collection of software from multiple source repositories.

Looking ahead

I think that the vibe at GUADEC and the current state of GNOME is really exciting, there are a lot of things to look forward to as well. My main take away is that the project is in an incredibly healthy state, with a rich ecosystem of people committing entire companies on products based on what the GNOME community writes and a commitment to solve some of the very hard remaining problems left to make the Free Desktop a viable contender to what the industry offers right now.

Promising times ahead.