Wayland utilizing Android GPU drivers on glibc based systems, Part 2

In this blog series, I am presenting a solution that I've developed that enables the use of Wayland on top of Android hardware adaptations, specifically the GPU drivers, but without actually requiring the OS to be Bionic based.

This is part 2 and will cover the actual server side (and a little bit about generic EGL implementation) of the solution. The first part can be read here. The third and last blog post will revolve around the client side solution and how you can use it today, as well as future work. There are a -lot- of links in this blog, please take a look at them to fully understand what is being explained.

This work was and is done as part of my job as Chief Research Engineer in Jolla, which develop Sailfish OS, a mobile-optimized operating system that has the flexibility, ubiquity and stability of the Linux core with a cutting edge user experience built with the renowned Qt platform.

The views and opinions expressed in this blog series are my own and not that of my employer.

The aim is to have documented the proof of concept code and published it under a "LGPLv2.1 only" license, for the benefit of many different communities and projects (Sailfish, OpenWebOS, Qt Project, KDE, GNOME, Hawaii, Nemo Mobile, Mer Core based projects, EFL, etc).

This work is done with the hope that it will attract more contribution and collaboration to bring this solution and Wayland in general into wider use across the open source ecosystem and use a large selection of reference device designs for their OS'es.

Rendering with OpenGL ES 2.0 to a screen with Android APIs

In Android, when SurfaceFlinger wants to render to the screen, it utilizes a class named FramebufferNativeWindow which it passes to eglCreateWindowSurface. As I mentioned in my previous post, on Android, when you use eglCreateWindowSurface you utilize a type/'class' named ANativeWindow. FramebufferNativeWindow implements this type. This then means it gets buffers utilizing FramebufferNativeWindow, renders to them within the OpenGL ES 2.0 implementation and queues them to be shown on the screen utilizing the same FramebufferNativeWindow.

But what happens under the hood? I'll try to explain with libhybris' "fbdev" windowing system as an example.

We're back to ANativeWindow - what libhybris' "fbdev" windowing system does is practically to do an implementation of ANativeWindow.

When a OpenGL ES 2.0 implementation wants to have a buffer to render into, it will call the dequeueBuffer method of an ANativeWindow. This usually happens upon surface creation or when you have done eglSwapBuffers and would like a new fresh buffer to render into.

You may have heard of fancy things like 'vsync' and you know that you have to follow signaling of vsync to avoid things like tearing. On the occasion that you do not have any buffers available (as some might be waiting to be posted to the framebuffer), you will need to block and wait for a non-busy buffer to be available within a dequeueBuffer implementation - don't just return NULL. Use pthread conditions and be CPU-friendly. This also makes sure you will block in eglSwapBuffers()

A quick note for implementors of ANativeWindow: Many OpenGL ES drivers are very temperamental. When it relays the information to you that it wants to set your buffer count to 4 buffers, it means that it wants 4 buffers and only to see those 4 buffers in the lifetime until usage or format changes. Mess up and it will happily crash on you - these drivers do not come with debug symbols.

When you want to allocate graphical buffers you naturally need gralloc to do so - gralloc is a module that is accessible through Android's libhardware API - in practice, gralloc is a shared object that libhardware dlopen()s, see /system/lib/hw/ for examples of these (gps, lights, sensors, etc).

When loading gralloc you will naturally get the interface of the gralloc module itself, but when you initialize gralloc, you get an allocation device interface where you can allocate and free buffers with, by specifying parameters such as width, height, usage, format. Usage is important since we'd like to allocate buffers for use with the framebuffer - so when we allocate a buffer, we allocate with usage 'usage | GRALLOC_USAGE_HW_FB'.

The return value of the alloc() call is a integer value indicating if it was a success, a native handle in the provided memory location (read my previous blog post for an explanation on what this is) and stride of the buffer.

We then wrap the handle and related information in a ANativeWindowBuffer structure and pass it back to the caller. Please note two things in this structure. incRef and decRef - they are very important. You will need to implement reference counting and you will need to increase/decrease your reference counting matching your own references to it. When reference count reaches 0, the buffer should destruct.

Eventually we will then get the buffer back from the caller in queueBuffer -- but how do we now send it to the framebuffer to be displayed?

In the initialization of our framebuffer window, we should have also opened the framebuffer with the libhardware API. It is in the same hw_module_t as gralloc. The framebuffer interface includes handy information such as the width, height, format, dpi and a few methods to actually utilize the framebuffer. The most important one for us is post(). This allows us to flip an actual buffer to the screen - utilizing the buffer handle, provided it has the same width, height/format as framebuffer and is allocated with appropiate usage (framebuffer usage). This call will on occasion block.

We have to be careful not to deliver the current front buffer to the caller in dequeueBuffer until we have replaced it with another at the front of the screen or we may see flickering.

A note to users of libhybris: there may be some Android adaptations that implement a custom framebuffer interface requiring extra implementation to achieve sane posting of frames that blocks. Check your FramebufferNativeWindow.cpp for this. This does not seem to be pervasive but I've encountered it on HP Touchpad with CyanogenMod/ICS.

Server-side Wayland enablement

The wayland protocol has two sides, server - and client. But unlike X, there is no "Wayland server". The implementation of the protocol communication for each side is implemented in respectively libwayland-server and libwayland-client. When implementing a compositor, you then utilize libwayland-server API to create server sockets, do communication, etc.

But how does the EGL stack get to be connected to a Wayland server instance when the associated EGLDisplay the stack is connected to, probably isn't a Wayland display? (note: may be in nested compositors - ie, a Wayland compositor running as a client to another Wayland compositor) That's where the next topic comes in:

EGL extensions - EGL_WL_bind_wayland_display

In order to connect your EGL stack to a Wayland display, you need to bind to one - you do this with eglBindWaylandDisplay(EGLDisplay, struct wl_display *) from the EGL_WL_bind_wayland_display. In libhybris, we provide this extension when it has been configured with --enable-wayland and available in most windowing systems (we provide an environment variable EGL_PLATFORM to select between 'windowing systems). Since the extension is not just part of the Wayland windowing system, it is possible to do nested Wayland compositors.

But what happens in libhybris when you bind to a Wayland display? We call the server_wlegl_create method in server_wlegl.cpp. What this does is to add a global object to the Wayland protocol - with a certain interface, - but where is this interface defined? As it has to be shared between both client and server; it is specified in a xml format file that is then converted by a tool called 'wayland-scanner' into .c files that are then linked into your client or server part. We then implement the actual interfaces for server side in our code.

Creation of buffers

When a client requests to create a buffer, it first creates an Android native handle object on server side and shares the client's native handle through the Wayland protocol (utilizing the support for fd passing) and then actually asks to create the buffer (note: we actually created the buffer on client side, now we're just sharing it with the compositor and letting it know the details).

When that happens on libhybris side, is when a handle is created, we construct an Android native_handle_t on server side - with the correct fd and integer information. We then map this with registerBuffer into the compositor's address space so the buffer is available to EGL and related stacks.

Once we have the handle in place, we can now create an Android native buffer - like we made ANativeBuffer on client side in the framebuffer/generic window scenario, we make one representing the remote buffer with the reconstructed handle. Finally we construct a Wayland object referencing this Android buffer and increase the reference counter of the Android buffer - and pass the buffer back to the Wayland client.

When we want to destroy the buffer again (well, when the reference count reaches 0), we unregister the buffer and close the native handle.

Utilizing a Wayland-Android buffer as part of your scenegraph

When a compositor would like to utilize a Wayland buffer in general, it uses eglCreateImageKHR with the EGL_WAYLAND_BUFFER_WL target and passing the (server-side) wl_buffer. This means that the compositor does not have to worry about the factual implementation of the wl_buffer behind the scene.

In our case, our wl_buffer is the one we indicated above - so we know it actually encapsulates a ANativeWindowBuffer with a handle/width/height etc. The knowledged reader might realize that eglCreateImageKHR in the Android EGL implementation does not support EGL_WAYLAND_BUFFER_WL. It does however support EGL_NATIVE_BUFFER_ANDROID

The way that this is handled is that we wrap eglCreateImageKHR and when we see EGL_WAYLAND_BUFFER_WL, we call the real eglCreateImageKHR with EGL_NATIVE_BUFFER_ANDROID - with the ANativeWindowBuffer. And we get the Wayland client's buffer as part of our OpenGL scenegraph.

This way we can also easily implement method such as eglQueryWaylandBufferWL as we know the attributes of the Android buffer.

An implementor's note: the destructor of a buffer is first called with wl_buffer_destroy coming from client side. You'll have to remember reference counting and not just delete the buffer

Conclusion

Thanks for reading this (rather technical) second blog post, the third one should follow quite soon. The code is already published and continually developed in http://github.com/libhybris/libhybris but it's not easy to approach or use for general users or developers right now.

The final post will describe how you can use this solution together with QtCompositor on top of Mer Core as well as describe how the Wayland client side works to make all this tie together. You can already now study, comment or flame the client side implementation. But for the explanation and for a description of what is missing, you'll have to wait for next one :)

Feel free to join us in #libhybris on irc.freenode.net to discuss and contribute to this work.

Wayland utilizing Android GPU drivers on glibc based systems, Part 1

In this blog series, I will be presenting a solution that I've developed that enables the use of Wayland on top of Android hardware adaptations, specifically the GPU drivers, but without actually requiring the OS to be Bionic based.  This is part 1.

This work was and is done as part of my job as Chief Research Engineer in Jolla, which develop Sailfish OS, a mobile-optimized operating system that has the flexibility, ubiquity and stability of the Linux core with a cutting edge user experience built with the renowned Qt platform.

The views and opinions expressed in this blog series are my own and not that of my employer.

At the end of the series, the aim is to have finished cleaning up the proof of concept code and published it under a "LGPLv2.1 only" license, for the benefit of many different communities and projects (Sailfish, OpenWebOS, Qt Project, KDE, GNOME, Hawaii, Nemo Mobile, Mer Core based projects, EFL, etc).

QML compositor, libhybris, Wayland on top of Qualcomm GPU Android drivers

The blog series seeks to explain and document the solution and many aspects about non-Android systems, Wayland and Android GPU drivers that are not widely known.

(Ignore the tearing, old demo video)

This work is done with the hope that it will attract more contribution and collaboration to bring this solution and Wayland in general into wider use across the open source ecosystem and use a large selection of reference device designs for their OS'es.

Why am I not releasing code today? Because that code alone doesn't foster collaboration. There's more to involving contributors into development - such as explaining reasons why things are the way they are. It's also my own way to make sure I document the code and clean it up, to make it easier for people to get involved.

Now, let's get to it..

The grim situation in mobile hardware adaptation availability

One of the first thing somebody with a traditional Linux background realizes as he tries to make a mobile device today when he meets with an ODM is that 99% of chipset vendors and hence ODMs - will only offer Android hardware adaptations to go along with the device designs.

When you ask about X11 support within the GPU drivers or even Wayland they'll often look blankly at you and wonder why anybody would want to do anything else than Android based systems. And when you go into details they'll either tell you it can't be done - or charge you a massive cost to have it done.

This means that OS'es and devices that are non-Android will be not able to take into usage the huge availability of (often low cost) device designs that are out there, increasing the time to market and R&D cost massively.

Libhybris

In August 2012, I published my initial prototype for 'libhybris'. What is libhybris? Libhybris is a solution that allows non-Android systems such as glibc-based systems (like most non-Android systems are) to utilize shared objects (libraries) built for Android. In practice this means that you can leverage things like OpenGL ES 2.0 and other hardware interfacing provided within Android hardware adaptations.

I had developed libhybris initially in my idle hours at home and the big question you might have is: Why did I open source it instead of keeping it to myself and earn on it as it obviously was the holy grail for non-Android systems?

The simple answer is this: by working together on open source code, it would help accelerate the development of libhybris and testing of the software for everybody's mutual benefit.

I didn't feel good about libhybris initially, it's not the most perfect solution to the problem: many around me in the open source community were and are fighting to have chipset vendors provide Wayland or X11 adaptations for mobile chipsets or even GPU drivers for non-Android systems in the first place.

But I felt that this was the required road that had to be taken before non-Android systems turned completely irrelevant in the bigger picture. When we again have volume of non-Android devices, we can have our own dedicated HW adaptations again

Open sourcing worked quite well - a small group of people got together, tested it,  improved it, got it running on a lot of multiple chipsets - thanks to OpenWebOS, Florian Haenel (heeen), Thomas Perl (thp), Simon Busch (morphis) and others. It turned the project from a late night hacking project into a viable solution for building device OS'es on top of. Or even running Android NDK applications using.

Earlier this year however, I discovered that a well-known company had taken the code - disappeared underground with it for several months, improved upon it, utilized the capability in their advertisements and demos and in the end posted the code utilizing their own source control system, detached from any state of that of the upstream project's. Even to the extent some posters around the web thought libhybris was done by that company itself.

That kind of behavior ruined the initial reason I open sourced libhybris in the first place and I was shocked to the point that I contemplated to by default not open source my hobby projects any more. It's not cool for companies to do things like this, no matter your commercial reasons. It ruins it for all of us who want to strengthen the open source ecosystem. We could have really used your improvements and patches earlier on instead of struggling with some of these issues.

But, I will say that their behavior has improved - they are now participating in the project, discussing, upstreaming patches that are useful. And I forgive them because they've changed their ways and are participating sanely now.

Now for a few words on my story with Wayland..

Wayland

My journey with Wayland started in late 2011. It was my belief around that time that Wayland was a bit dry and boring - it was just a protocol. I was not fully appreciating the power and simplicity that it provided for embedded UI. That it was a building block for much more exciting things - like libhybris turned out to be.

Being in embedded Linux and exploring Qt Lighthouse, I had learnt of Qt Scenegraph and a curious new thing called QtCompositor. QtCompositor was what sold me on Wayland - it enabled amazing capabilities on embedded and ease of development of effects, window management and homescreens. Things that previously would take several manyears to develop for embedded devices was made easy to do. And allowed stack builders to have similar graphical stacks on SDK/virtual machines for development as on target devices.

If you don't know QML, it's a declarative UI language to design beautiful and functional UIs with. What QtCompositor did, was to enable that you first off could get events about windows appearing, changing size, etc - but also that each window, - the graphical buffer, became just another item in your UI scenegraph like an image or a label would be. It could even make your graphical buffer be a widget inside your traditional UI.

Screenshot from http://blog.qt.digia.com/blog/2011/03/18/multi-process-lighthouse/

This could naturally be expanded to much more curious things, such as 3D Wayland compositors. If you'd like to hear more about QtCompositor, you can also watch the following talk from Qt Developer Days by Andy Nichols. Capable QtCompositor technology is something that is here today. Not something that has to be developed from scratch or roadmapped.

I was doing the role of maintainer of the Nokia N900 hardware adaptation for MeeGo at the time and I wanted to see if I could get Wayland working on it - it had a PowerVR SGX GPU. I reached out to #wayland on irc.freenode.net and was met with open arms, guidance and a lot of help from krh (Wayland founder), jlind, capisce (QtWayland/QtCompositor), Andrew Baldwin, Mika Bostrom and many other talented people and I was able to get started very quickly with Wayland.

To get things working with Wayland, what I needed to do was figure out:

• How to render an image with OpenGL ES 2.0 into GPU buffers that I had under my control
• Share that GPU buffer with another process (the compositor)
• Include that GPU buffer as part of a OpenGL scenegraph, a texture - and display this to the screen (in the compositor)
• And for performance, flip a full screen GPU buffer straight to the screen, bypassing the need to do OpenGL rendering

To be able to render into a specific GPU buffer under your own control, you usually need to get inside the EGL/OpenGL ES implementation. On some chipsets, it's possible to use specific EGL operations that allows shared (across two processes) images to be rendered into - such as on Raspberry Pi.

In the EGL implementation, you should be able to follow the path of the buffer, it's attributes (size, stride, bpp/format) and when the client has requested to do eglSwapBuffers. 

On the PowerVR SGX, there was an API provided called WSEGL, for making plugins that were windowing systems (X11, Framebuffer, QWS, Wayland..) that allowed me to do just that. 

Sharing that buffer is sometimes a bit more difficult - you effectively need to make the same GPU buffer appear in two processes at once. On SGX, this was simple - you could request a simple integer handle to the buffer and share that value using whatever protocol you wanted. In the target process you then just map in the GPU buffer through a mapping method.

Wanting to standing on the shoulders of giants, I looked at how Mesa had implemented their Wayland protocol for DRM - it too had simple handles and shared these buffers through a custom Wayland protocol.

Even if it was a custom protocol for buffer handling (creation, modification, etc), the same operations for handling buffers in Wayland still applied to it. I didn't need to do anything extra for compositor or client for the buffers in particular - I could piggyback on existing infrastructure available in Wayland protocol.

Wayland made it easy for me to take existing methods for the techniques/needs listed above into use and made it possible to quickly and easily implement Wayland support for the chipset.

Now, to something a little different, but quite related:

Android and it's ANativeWindow

When you use eglCreateWindowSurface, as in, creating a EGL window surface for later GL rendering with Android drivers, you have to provide a reference to the native window you want to do it within. In Android, the native window type is ANativeWindow.

As you know, Android's graphics stack is roughly application -> libEGL that sends GPU buffers to SurfaceFlinger that either flings the buffer to the screen or composites it with OpenGL again with libEGL.

Why not just include all the functionality and code in the EGL stack which communicates with SurfaceFlinger? The answer is that you need to sometimes target multiple types of outputs - be it video/camera streaming to another process, framebuffer rendering, output to a HW composer or communication with SurfaceFlinger.

One of the good things about Android graphics architecture is that through the use of ANativeWindow, they have made it possible to flexibly keep the code that does this work outside the EGL drivers - that is, open source and available for customization for each purpose. That means that EGL/OpenGL drivers are less tied to the Android version itself (sometimes API versions of ANativeWindow changes) and can be reused in binary form easily across upgrades.

ANativeWindow provides handy hooks for a windowing system to be managing GPU buffers (queueBuffer - send a buffer, I'm done rendering, dequeueBuffer - I need a buffer for rendering, cancelBuffer - woops, I didn't need it anyway, etc) - and it gives the methods you need to accomplish things, like I did on PowerVR SGX.

This is the entry point used to implement Wayland on top of Android GPU drivers on glibc based systems. Some fantastic work in this area has already been done by Pekka Paalanen (pq) as part of his work for Collabora Ltd. (Telepathy, GStreamer, WebKit, X11 experts) which proved that this is possible. Parts of the solution I will publish is based on their work - their work was groundbreaking in this field and made all this possible.

A note on gralloc and native handles

The graphical buffer allocation in Android is handled by a libhardware module named 'gralloc'. This is pretty straightforward, allocate buffer - get a buffer handle, dealloc, register (if you got the buffer from a separate process and want to map it in), but most documentation pieces don't talk about buffer_handle_t and what it actually is.
If you do a little bit of detective work, you'll find out that buffer_handle_t is actually defined as a native_handle_t* .. and what are native handles?

The structure is practically this: number of integers and a number of file descriptors plus the actual integers and file descriptors. How do you share a buffer across two processes then?

You have to employ something as obscure as "file descriptor passing". This page describes it as "socket magic" which it truly is. It takes a file descriptor from one process and makes it available in another.

The android GPU buffers are typically consisting of GPU buffer metadata (handle, size, bpp, usage hints, GPU buffer handle) and then file descriptors mapping GPU memory or otherwise shared memory into memory. To make the buffer appear in two processes, you pass the handle information and the related file descriptors.

The good news however is that Wayland already supports file descriptor passing so you don't have to write obscure code handling it yourself for your custom Wayland compositor.

Conclusion

This concludes the first blog post in this series, to give a bit of background about how Wayland, libhybris and GPU drivers for Android can work together.  Next blog post will talk more about the actual server side implementation of this work. Last blog post will talk about direction of the future work on it - and what you can do with it today and how as well as explaining the .

If you'd like to use, discuss and participate in the development of this solution, #libhybris on irc.freenode.net is the best place to be. A neutral place for development across different OS efforts.

Tizen conference, wrapping up

The Linux Foundation sponsored me to travel to and attend the Tizen Conference 2012 in San Francisco and as part of this sponsorship, I'll be blogging about the conference and my insights and thoughts of the talks and keynotes at it. This is my last post in a two-parter about the conference.

When attending a conference, or a music festival, or any other event with multiple tracks, there will always be sessions that you for some reason do not end up attending, be it because of meeting somebody suddenly at the coffee table while a session you'd like to see is about to start, or because there's a session that you'd rather see, or simply because you decided to take a break. The solution to this is the conference recording the sessions on video, which is not often performed very well.

I've encountered the usual screw-ups in conferences: session recording that does not include the slides at all, session recording where the A/V people forgot to wire up the microphone to the recording equipment(!), or so bad audio that you couldn't even hear the speaker. It's also more difficult, when viewing the session afterwards, to having to focus on three things - the movements of the speaker, his/her voice and the slide content.

That's why I welcome a better format, which is what the Tizen conference will be utilizing it seems - recording the speaker audio and along with that, the slide content, but not the speaker him/herself. And exporting it in podcast format, allowing you to catch up while on the move on the newest technology. Without having to dedicate your full attention to it.

Which brings me to that I didn't participate in that many sessions during the last day. But I got to attend the ones that mattered to me the most: Open Build Service—Facts, Features and Future, by adrianS and mls from openSUSE and Next Gen OS Initialization Done Right, by Auke Kok. And missed out on Tizen IVI architecture with Mikko Ylinen.

Meeting the OBS guys is always a pleasure - you get to sync on what their ideas and plans are and they listen to what your expectations and sometimes crazy ideas are. Compared to many other distribution build systems, OBS does not only function or serve their own community (openSUSE) -  it is entirely usable, deployable and fantastic for building other distributions. OBS has leveled the amount of infrastructure you previously needed in other to run/roll your own distribution. And that's why we love it in Mer. It enables anyone with a minimum of OBS knowledge to maintain their own customized distribution. And for ISVs, to build against your distribution. 

As with most technical discussions - the hallway track is the most interesting. The questions and concerns the audience comes up with during the session seeds the ground for the continued discussion in the hallway afterwards. One concern, that was raised by Dominique Le Foll, of Intel OTC was that OBS-to-OBS links are simply too fragile and too often causes build stalls and problems, was actually a matter we approached at first with Mer too, given the very unstable nature of the meego.com API - we needed a way to synchronize and access the OBS projects MeeGo consisted of, offline. Their need was simply needing a way to export MeeGo (well, now Tizen) releases to customers in a reliable manner and allowing them to modify it too.

What we invented there was a piece of software called FakeOBS (now Mer Delivery System) which, to make a long story short, serves up a HTTP/REST interface that is similar enough to the OBS-to-OBS protocol as to make it able to have another OBS connect to it and it thinking it's a remote OBS. 

While in fact, it is a cache of sorts - we extracted through the OBS API the entire OBS project history of sources, built binaries and put it into a on-disk format that FakeOBS could then use to provide those over the OBS-to-OBS protocol, giving us effectively offline access - leaving us able to not have to care about external entities. You can view the latest iteration of it here. There's also a file called 'gitmer.py' which is how we deal with the git-based approach that Mer uses, for sources.

When we generate Mer releases, we do not only export the built binaries, we also export this on-disk format for FakeOBS, allowing anyone to re-create Mer and re-build it in their own OBS, along with the additional use case that we have built OBS package repositories for ISVs to build against. Meaning that even if Merproject.org shuts down, anybody can resurrect the project. As well as that vendors do not have to rely on merproject.org being up.

Next session was Auke Kok's systemd session. Auke's one of my personal open source heroes, always working on quite interesting things. As some of you might know, the traditional way that most UIs are launching applications and daemons on boot are through D-Bus and /etc/xdg/autostart .desktop files. In Mer and MeeGo, this was accomplished with uxlaunch (another of Auke's inventions)

But what if you could use the same flexibility that systemd offers you, in order to create a proper dependency tree for proper optimized booting within the user session? Well, guess what Auke invented :) 

Instead of starting uxlaunch, you'd instead start systemd --user as a user, which would properly start up X, perhaps services that do not need X before X starts,  and ability to indicate session-internal dependencies. Which leads to amazing results. You can check out the systemd discussion on this mailing list thread. 

Another thing that happened was the availability to all conference attendees (except for Intel & Samsung, it seems) of a Tizen reference device, the so-called Lunchbox. It's an amazing piece of hardware, 1280x720 AMOLED display, 4.3", with dual-core Samsung Exonys 4210, Cortex-A9, 1gb ram, 16gb eMMC, microSD slot, sim slot (though modem ability is unknown), Mali GFX chipset, u-boot boot loader, 8mp back camera, 1mp front camera, PN544 NFC chip (unsure how to use), GPS chip, WLAN (WiFi Direct possible).. so, a quite nice kit. And a possible replacement for the N800 as well.

And if you're wondering if we've tried to put Mer on it. Now, of course we have. We've found most interesting pieces, including a "Boot to SD card" mode (with no success just yet - press power key, volume up and volume down at same time), kernel source code (2.6.36) and investigated the system which uses currently Xorg 1.9.3 with a Xorg driver we can't find source for yet. But it'd surprise me if it wasn't somehow similar to the Mali Xorg driver. Once we've figured out SD card boot, it should be a breeze to run Mer on there. Even with X11-GLESv2 acceleration.

That's all I have to say about the conference, will look forward to the next one.

Tizen Conference 2012, first days

The Linux Foundation sponsored me to travel to and attend the Tizen Conference 2012 in San Francisco and as part of this sponsorship, I'll be blogging about the conference and my insights and thoughts of the talks and keynotes at it. While some parts may seem negative, please keep reading as I have a point with those parts that is illustrated later on.

While some people might think that I see Mer as a competitor to Tizen, or think that I don't like Tizen, in fact, I see the benefit of Tizen for the entire open source community that they're pushing a proper, fast and working, standards compliant web runtime for Linux based devices. And I hope we'll see people putting this on top of Mer. Mer plays a different game than Tizen and hence not a competitor - it's a solid mobile core that everybody can build on and most importantly, rely on despite the moods of corporate giants, to make their devices - where solid delivery, ability to productise and code is king.

There's always a challenge in how to properly launch a commercial open source project. Especially in mobile Linux, there is unnecessary high media and developer expectations to the features and ability of a mobile handset stack that often enough causes mobile OS projects to break their necks and die.

But Rome wasn't built in a day. And so weren't the mobile stacks you see around the today - Android took over 3 years to get to a remotely functioning stack. And this has a tendency to make mobile OS developers develop in 'closed mode' in order to launch properly on day one.

Tizen has drawn a lot of crap from their complete silence and secret cathedral building behaviour up to 1.0 release. But I can say that if I was in their shoes, having to launch a handset device .. and handset stack. I would probably end up doing the same things as they had. In a world where you will see your semi-ugly alpha release screenshots laughed at in news articles about your 1.0 launch, when you have a perfectly working and very shiny final release that nobody seems to bother to even check out, it's hard to argue for transparent development from day zero.

Luckily I'm not in their shoes and I can concentrate on technical core side of things and make it easier for people to make exciting new devices. I'm not a fan of whole-stack projects - for the reasons listed above. And that's why Mer doesn't contain UIs and hardware adaptations - we let people base and develop on a sane core that we can maintain the openness of, while people work on exciting new devices and UIs in seperate projects from the Mer project.

Reputation and history matters a lot as well - There was a lot of disappointment in the demise of MeeGo and the messaging on MeeGo site was that Tizen was the new direction. And people expected there'd be some kind of relation. But there's not - Tizen, at least the handset side (IVI is more related on the Core side), is so very different from anything you knew in MeeGo - your employees would need complete retraining to work with it. The stack is based off Samsung Linux Platform and actually says it's that, when booting up, last I saw. That should lead to interesting discussions with your company lawyers.

And with reputation and history in mind - many from MeeGo remember last years keynote, Monday Morning with MeeGo.. February 11 had happened months before and there was still a fighting spirit in the community, we needed people who were showing passion in their work, the same fighting spirit. And we got something that was closer to Monday Mourning with MeeGo. Which left many people depressed and unimpressed. A talk that spoke more about the fantastic deployments of the platforms that MeeGo was in practical competition with, than about MeeGo itself and it's qualities and achievements.

When a last moment change in the Tizen conference schedule came in, that moved the first keynote which was supposed to be Imad from Intel and JD from Samsung to the morning after and instead, we got a recycled keynote, void of genuine and documented passion for Tizen, with the same recycled material as in the Monday Morning with MeeGo talk and the same speaker as last year - with him even talking about that if people had noticed he would be on schedule, there'd probably be fewer in the room. I was left unimpressed and depressed, again.

I'm ashamed to say it, but I've grown to appreciate the famous "Developers! Developers! Developers!" dance. Somebody showing passion for their work - people who truly believe in their product and their stack. Someone who wants to stirr excitement in developers, CTOs and platform integrators alike. And I harbour a lot of respect for people who truly believe in what they do and that they want to do a good product and good software.

And we got that in the morning after keynote, Imad, the head of Intel OTC and JD, EVP of Samsung, the Tizen technical steering group. They showed their excitement about what they were doing, showing excellent live demos and they understood the challenges in presenting a mobile OS and explained to depth both the challenges and reasoning behind their choices for the Tizen stack - even the controversial ones. I can recommend watching the video when it gets put online.

This is what the first keynote of the conference should have been like. And I was more excited again.

The second keynote, from Tizen Association was interesting, showing the need of operators to provide and control content and the benefit of having to write to one platform (HTML5). Which is funny, when there were also talks about "The web always wins" .. that in practice, the walled garden never wins, the properitary standards will not either. That content from outside the walled garden will always be more interesting and varied than your own content. Doesn't anybody remember AOL? :)

I might be an idealist, but I have difficulty seeing the word 'open' together with the working method of the Tizen Association. When their terms of membership states at http://www.tizenassociation.org/en/tizen-association/board-of-directors
that there is an annual due of membership of Two Hundred and Twenty Thousand USD. And talking about confidentiality clauses.

But I was reminded of a quote from my former boss, Harri Hakulinen, from his MeeGo blog, speaking of Nokia's choice of WP7: "... What I had witnessed from the side, thanks to Sami J. Mäkinen, was of course the birth of Linux. And, in so many ways, I feel that we have now "replayed" the same with MeeGo in last 12 months.
Co-incidentally, at that time I had a summer job where we replaced some ancient Nixdorf and IBM mainframes with another new OS, called Windows NT. It was hard to explain to Sami and others, that from the point of view of those companies I worked for, Windows NT was, in fact, like "open environment", compared to those things that they had been using before."

And perhaps this is how it's like with Tizen. That compared to how it has been before, this is actually more open than the previous situation for the ecosystem. At least the code is open, actively developed, looking to be intended to be developed in the open, not only codedrops, with a proper code submission process. So perhaps that's an improvement on top of what we have had with Symbian and Android.

What I am more encouraged by in terms of openness, is what I learned in the Tizen Architecture talks, where there is a strong push towards that there will actually be as little Tizen specific WebAPIs as possible. That there's an active desire to use W3C standards where possible and to submit APIs to the standards process. Which makes me hope that for the parts that really matters - the developer facing APIs, that there will no vendor lock down. In the end, the winner would be the Web. And even if Tizen did go away one day, that the code would be living on. Just like MeeGo's code has. Because it's open source.

From an entirely architectural point of view, Tizen architecture (SLP-based) has a lot of NIH in it - it's own telephony stack instead of oFono (though I hear that's coming), and a lot of self-made libraries and methods. Which may impact the portability of the web runtime.

I'm hoping that Tizen will realize the importance of separating the web runtime from OS stack. That one of the reasons that the web standards have developed so fast, has been the ability to get many people to upgrade their browsers over the air.

We probably cannot assume we'll see Tizen 1.0 to Tizen 2.0 upgrades on our handsets - aftermarket hardware adaptation is really hard.  But we should make sure that at least for a few generations, we can keep on delivering the web runtime to older devices. To foster a standards compliant and secure web runtime for our apps - just like the browsers has fostered it for the web. And I think this is one area Tizen could potentially be very agile in, comparing to other platforms.

The last talk of the day was on Tizen security - Ryan Ware was presenting along with his Samsung counterpart, where compared to many other platforms, the focus has been to ensure end-user privacy, and not on the need to protect content owner content. That's the right direction - especially in a world where devices are turning very promiscuous and interacting in sometimes unsafe manners with other devices.

One of the most important parts of a conference is not so much the presentations, but the people there and the discussions in the hallway - the 'hallway track'. This is a concept that any conferences doesn't always understand - the importance that people can discuss outside the conference rooms. Tizen conference has as usual done a great job on this (thanks to Amy Leeland's fantastic work), providing good space and hacker lounges with whiteboards for people to properly discuss. I've sat down together with many key people in my line of work and had intellectual discussions about the challenges we meet. And I've gotten to know more people that I previously didn't.

And of course - when working on a distributed-across-the-world project, I've been able to go out and have our traditional icecream shop visit (since 2009) with the rest of the participants of my project. So far it has been a good conference, looking forward to be seeing more presentations today.

Mer is back!

So, we are back. Please take some time to read the following mailing list post:

http://lists.meego.com/pipermail/meego-dev/2011-October/484215.html

See you in #mer on irc.freenode.net and on http://www.merproject.org

Why Meego on N900 is the right and future-proof direction for the community

So, I think it's about time for me to really weigh in why it is I'm pushing MeeGo on N900 and why this is the right direction. This is probably going to be a bit long. Some people asked me to repost it on my blog as to not have it lost in the maelstrom of talk.maemo.org.

This is also probably my last long post on the matter as I'm now fully in a MeeGo role, no longer maemo.org distmaster.

The fact is that the model of 770 -> N800 -> N810 -> N900 has been that of one device per OS version. There hasn't been resources for keeping a full product team for multiple devices at a time.

This has caused what some describe as Rapid Obsolescence Syndrome, where the device OS rapidly goes into maintenance mode after device release while mostly everyone involved moves on to the next product/device. This obviously leaves users unhappy.

Now, there are multiple suggestions around maemo.org:

* Fremantle community SSU
* Petition for open sourcing most of Fremantle to keep it maintained
* Harmattan HE for N900
(etc)

And my main point in the following is that those directions will be like pissing your pants to keep yourself warm. Quick review:

Community SSU can work on the open source parts (it has happened on N8x0 just fine). Maybe you can find some ability together with Fremantle aficionados inside Nokia who could be interested in contributing with upgrades of some binary packages -
I doubt there's any problem if those exist to slip some packages out.

But the main point is: no upstream work will really happen. I highly doubt you can find enough people that are still around to keep things working. Even with documentation, the code can be quite hairy to add new features to. So that bends down to a resourcing problem.

Open sourcing most of Fremantle - I simply don't think it's going to happen. Too much work for too little benefit. And the same problems as with normal SSU when the code does arrive.

Harmattan HE. This would already be obsolete by the time we were done as a more modern platform would be available in MeeGo. As well as at some point, no more upstream development.

The main point - you'll piss your pants with these projects to keep yourself warm for a while, but you'll just keep on feeding the Rapid Obsolescence Syndrome to some extent - that you'll never be able to play catch-up to new technologies and satisfy the requirements of your users. It is a physical impossibility as there's many more man-hours poured into MeeGo than you can provide here.

MeeGo and Qt has changed the status quo that existed that:
* Any Linux-based OS platform from Nokia is tied to a certain device
* Any applications from Nokia is tied to a certain device by fact that they were released for a certain OS platform.

It's a game-changer because now, for any given 'Nokia' OS for these devices, this will after Harmattan be based off MeeGo (platform from MeeGo.com).

Now, what are we actually doing in MeeGo for N900?

First off - we're making the hardware adaptation for N900 maintainable.

This means that we are actively upstreaming N900 drivers to the mainline Linux kernel as well as getting those few blobs we have into the MeeGo non-oss repositories with redistributable license. This is actively happening.

Second, we're running QA daily to make sure that N900 still works with the current MeeGo platform state. This means we're always at the front of the platform development. MeeGo platform doesn't move forward if a change breaks N900 - it is a reference device

That's not pissing your pants to keep warm. That's making a bonfire and collecting wood to make us sustainably warm. And that's one of the reasons why I hope for more people to contribute to the MeeGo effort.

Now for the users. Look forward a bit - Nokia's obviously going to make a OS release based on 'real' MeeGo.com. The same code that'll power future handsets will be based on the same code that N900 already now supports. It's the same infrastructure that builds images and software that MeeGo for N900 is made using, as any given future Nokia OS. Instead of Nokia having applications tied to Fremantle or to Diablo or whatever, they now tie to MeeGo - the same platform you have on your N900.

If you want to make a real bonfire and eat your cake too, you will want to do the following:

* Contribute to MeeGo and MeeGo on N900 in the short term - this will improve matters for Nokia N900 obviously.
* Petition for Nokia early on, to consider providing the following things:
** Executive summary: access to the binaries making up the Nokia differentiation
** Weekly (or other interval) repository releases (binaries) available to Nokia N900 users of their future MeeGo.com-based OS under the usual 'this is not for end-users, bla bla'. Kinda like the old 'Sardine'
** Kickstart files for these weekly releases so you can edit them into your own images for the N900 using the Nokia bits.
** Community can be possibly be more involved in the QA process this way and contribute in general.
** If branding is a problem, have community themes and icons.
** Suggest new ways to have both a 'closed' vendor OS with all the goodies based on MeeGo.com while at same time having your power user community close to you.

Now, this is a way that can keep you ahead of the curve regarding your device. The N900 hardware isn't going obsolete for many years and is still very capable.

I think this might be possible to pull off, but it requires people to start gaining skills within MeeGo - and help contribute to MeeGo on N900. It's a similar approach to Harmattan HE, but more involved and more future proof. So I wouldn't say it's impossible.

So - it's your choice: Live in the future instead of the past.

I'll be working to make the future happen - see you on http://www.meego.com (and http://wiki.meego.com/ARM/N900 ).

First instance of 'What have I done for the N8x0's lately'

So, this is a new series of blog series on what I've done for the N8x0's lately. So people better see what I've actually tried to do for the owners of these devices.

* I've spent a lot of time with the MBX 3d driver to try to get it in good shape and set up gitorious repo for it and forward ported it to 2.6.33. At the current stage,
* Started a page for the MeeGo N8x0 hardware adaptation team and added plans/milestones
* Working on stabilising MeeGo N8x0 startup, building Xomap for it. I have it running with Xfce4 on my device and hope to show it after 1.0 release.
* Worked with tmr to try and get DSP working on 2.6.33 for N8x0. He forward ported the patches himself.

These are fairly technical things, but should hopefully manifest in actual benefits sooner or later.