Planet Mandriva

When I released the version 3.0.0, some people started to report issues at restore time, that I didn’t saw during my tests. These problems were mentioned as related to LVM restoration, or partition table restoration.

After looking at the logs they send (kind reminder, that’s a mandatory info if you want to get any form of support !), I saw that in their case, the resizing factor was incorrect, even sometimes 0, leading to empty partitions.

This should have been fixed with revision 2932, and I have released a beta of 3.0.1 that people encountering this problem should use. Let me know if you want to test the fix for a distribution not yet published.

I’ve also repackaged mindi-busybox (with tag 2) for all the deb distributions I manage, in order to solve a dependency issue when upgrading, which was not seen on RPM based systems (for once I have an advantage )

I’d like to have more rapid cycles for this 3.0 branch to reach a very stable point asap, allowing me to work on other branches. Feel free to give feedback so that I could publish 3.0.1 quickly.

The decision that should have been taken today has been postponed to Friday, January 27th. The deadline for the decision on the proposal has been extended by the proposing entity upon request of some shareholders.

La décision qui devait être prise ce jour a été ajournée au vendredi 27 janvier 2012. La validité de l’offre reçue a été prolongée par l’entité l’ayant présentée, donnant suite à la requête de certains actionnaires.

When working with the git version control system and editing .git/config to add a new remote, some people may be tempted to copy and change the origin remote that reads something like:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = git@github.com:shlomif/perl.git

However, note that origin also appears at the fetch = and needs to be changed there as well, or else all the branches will be placed in remotes/origin. Maybe there's a better way to add a new remote using the git config commands.

Otherwise, I should note that there doesn't seem to be a consensus among git users whether git pull --rebase is better than a simple git pull: the perl people told me to use --rebase and the Amarok people and someone on Freenode's ##programming told me not to use it. Now I'm just confused.

Je déteste transformer ce blog en rubrique nécrologique. Mais après Montserrat Figueras et Alain Recordier, j’ai appris par un article du Monde de Renaud Machart la disparition du maître Gustav Leonhardt.

Alors même si cela ne fera encore pas les gros titres du 20h, il faut affirmer qu’il a été celui par qui la révolution de la musique ancienne est arrivée, au même titre que Nikolaus Harnoncourt, David Munrow, ou Thomas Binkley et Jordi Savall, chacun dans leur domaine de prédilection respectif. Pour Gustav Leonhardt, ce fut le clavecin qui fut enfin considéré comme un instrument majeur grâce à lui et non pas comme un ersatz de pianoforte mal dégrossi. Et cela parce qu’il a su faire chanter la musique sur cet instrument comme personne avant lui. Et heureusement, comme beaucoup après qui lui ont emboité le pas.

Je me souviens encore des concerts que j’allais voir de lui sur Paris, ou je me retrouvais dans la même rame de métro qu’un Pierre Hantaï qui venait le voir aussi ! Gustav Leonhardt mérite le titre de maître que je lui donne plus haut, car justement, il a joué un rôle de passeur, tant dans l’enseignement, que dans la redécouverte de pans oubliés de répertoire (j’écoute en ce moment même son second disque consacré à Johann Jakob Froberger qu’il m’a fait découvrir comme à des milliers de mélomanes dès son premier enregistrement 1962 (!) pour Harmonia Mundi) ainsi que par la publication d’un ouvrage consacré à l’art de la fugue de Johann Sebastian Bach.

Puisque l’on parle de Bach, c’est évidemment ce compositeur qu’il aura le plus illlustré au disque. J’ai encore un frémissement rétrospectif lors de ma découverte de la fantaisie chromatique et fugue sous les doigts du maître hollandais ! Quel adéquation pour moi entre la hauteur de vue du compositeur et de cet interprète. Ma femme se souvient encore des 500 kms aller-retour faits pendant les vacances une année pour aller l’entendre jouer à l’abbaye de St Guilhem le désert où il avait interprèté de façon magistrale diverses pièces de Bach, dont une de ses propres transcriptions de sonates pour violon seul au clavecin.

Ce qui m’a toujours frappé dans ses interprétations, c’est l’extrême lisibilité des voix, dans la musique si riche de Bach, qui rendait l’oeuvre à l’écoute évidente, tout en lui laissant sa richesse et sa complexité visible. Et les mots qui me viennent naturellement en tête quand je pense à ses enregistrements, c’est non pas austérité, dont on l’a souvent affublé, à tort, mais au contraire fulgurence (2è prélude du clavier bien tempéré de Bach), noblesse (son disque Louis Couperin chez Harmonia Mundi), architecture (son concerto italien de Bach), gravité (ce premier disque de Froberger chez Harmonia Mundi), virtuosité (son Scarlatti chez DHM ou son Duphly pour Séon), probité (Suites anglaises de Bach chez Séon).

Je crois que je n’ai jamais autant écouté des disques que ceux de ces partitas de Bach (HM) d’abord en 33T, puis en CD. C’est l’un des sommets de la musique enregistrée (meilleur pour moi que sa seconde version pour Virgin où de nombreuses reprises manquent). Et je ne peux plus écouter toute cette musique autrement qu’au clavecin depuis ma fréquentation assidue des enregistrements qu’il a réalisés (comme ceux du regretté Scott Ross, inoubliable lui aussi dans Domenico Scarlatti, Jean-Philippe Rameau, François Couperin et pour moi tellement complémentaire de la discographie de Gustav Leonhardt). J’ai failli travailler le clavier vers 17 ans, après tant d’heures passé à écouter la musique qu’il avait enregistrée, mais mon emploi du temps ne m’a pas permis de le faire vraiment et je reste hélas seulement un auditeur. Mais je suis toujours capable, comme à Saintes, de passer une demi-heure rien qu’à écouter un claveciniste s’accorder (Trevor Pinnock a vraiment dû se demander ce qui se passait ce jour là )

N’oublions pas non plus ses apports en tant qu’animateur d’ensemble. Quelle messe en si de Bach d’une majesté, d’une ferveur que peu d’autres que lui peuvent se permettre. Et que dire de son Requiem de Biber !! J’ai encore le frisson de la fois où je l’ai entendu en concert, et où il avait demandé à ne pas applaudir à la fin du concert. Le silence qui s’en est suivi, la concentration qui avait été accumulée par le public était simplement d’une densité palpable.

Ce maître artiste a dédié sa vie à la musique, jusqu’à en faire le sacrifice en se produisant jusqu’à la fin. Il m’a fait aimer le clavecin plus que mon propre instrument (la flûte à bec qu’un Franz Brüggen a si bien illustré avec son accompagnement). Il a contribué à graver une intégrale des cantates de Bach qui a fait date et reste, par sa diffusion sur France Musique par Jacques Merlet, une de mes nombreuses initiations à la musique, de celles qui vous marquent pour la vie. Et il laisse un corpus d’enregistrements pour le clavecin et l’orgue qui est toujours la base d’un discothèque idéale.

Alors, à l’heure où votre descente et arrêt brutal sur le clavier qui illustre la chute de Mr Blancrocher résonne à mes oreilles en fin de disque, un grand merci pour tout ce que vous avez apporté, Monsieur Leonhardt, à la musique et puisse le son de vos enregistrements longtemps susciter des vocations de mélomanes, d’amateurs et de professionnels. Ils ne pourraient mieux choisir leur modèle.

In deference to Adam Young, I’m going to try and write a series of broken-down posts on FUDCon, rather than one or two giant mish-mash-y summaries.

So, this one’s about the presentation I gave, titled ‘Cloud 0.1′, with a subtitle I haven’t quite nailed down yet, but which is something like ‘Why Not to Spend Lots of Time and Energy Running Your Own Infrastructure Much Worse Than Google Would, And How To Do It If You Insist’.

I’ve had the idea for a while now, but being lazy, didn’t write anything at all until the day before FUDCon, nor make any slides. Then I pitched it. To my surprise, it got enough votes to be scheduled. To my consternation, it got scheduled in the very first timeslot – so I had no time to finish my half-written notes, make any slides, do a runthrough, or generally do any of the stuff that would make it into a good talk.

Instead I got up, read my introduction, then improvised inexpertly for an hour. Many thanks to the dozen people who showed up and managed to avoid falling asleep or throwing rotten fruit.

The way I presented the talk was to spend a while talking about the many reasons it’s not a good idea to run your own infrastructure and the few reasons it is, then spend quite a while giving a 10,000 foot overview of how to set up a mail and web server, then spend the last 15 minutes briefly going over some rather neat webapps I run on my servers, and IRC/IM proxying. However, in hindsight, I think the most valuable bits are the consideration of whether you should run your own infrastructure, and the notes on neat, not-necessarily-well-known webapps and so on you can use if you do. The mailserver / webserver stuff is just too complex for a one hour presentation. So, since my notes are terrible, personal shorthand gibberish, and I have no slides, instead of giving you those, I’ll write a post about the same topics. Deal?

When I talk about ‘infrastructure’ I’m talking about the services that support your computing. The classic, old-school example is running your own mail server; other bits that come into the talk are a personal web server and IRC / IM proxying servers.

In the past it was pretty hard to find managed ways of doing any of those things, and it was fairly common for geeky types with personal internet connections to DIY. If you look at the internet, of, say, 1995, it was kind of designed as a giant interoperable network of nodes which would provide these kind of services to a group of users, and geeky types would essentially act as a node unto themselves – they were a service provider of one, providing services to themselves, and maybe a few friends and family, instead of relying on mail and web hosting services provided by their ISPs, which were inevitably crappy and limited.

These days it’s much less common, for a good reason: you can almost always get someone else to do it for you, much cheaper and better than you would do it yourself.

This forms the ‘why you probably shouldn’t do this’ side of the argument. There is just about nothing you can achieve by hosting your own mailserver which Google won’t do much better in exchange for sending you some ads and assimilating your personal information into the future Skynet, or which a service like Fastmail won’t do much better in exchange for a frankly pretty small cost – a cost which will almost certainly be less than the value of the time and money you’ll invest into doing it yourself. This is not surprising. There are huge, huge economies of scale built into infrastructure provision. Doing it for a user base of 1-5, on a hobbyist basis, is unsurprisingly vastly less efficient than doing it for ten million people on a very very professional basis.

The other disadvantages to self-hosting really just derive from this fact. You will almost certainly screw up more than a hosted provider will: you will break the server by deploying some dodgy app or an untested update. You will have less capacity (wave goodbye to your self-hosted blog when you get slashdotted, for e.g.) You will almost certainly have less redundancy – I know I don’t have any kind of failover on this webserver. You will almost certainly fail to take adequate backups. All these are boring, menial things which any decent hosted provider will do better just because it’s part of doing a professional job. You won’t because you’re doing this for fun, and those things are not fun.

Briefly, paid or ‘free’ (ad-supported / personal data supported) hosting services can provide you with almost anything you can host yourself, and do it much more efficiently. So why would you ever want to do it yourself? There are only a few reasons:

Necessity. I’m sticking this up at the top to make sure you don’t miss one of the best bits of this lengthy post. There are some things you can self-host that, to the best of my knowledge, you can’t actually get from a paid provider. The thing I know about is IRC/IM proxying. There’s no hosted provider of this that I know of. There’s a bit of this post down the bottom which explains what this is and, briefly, how to do it. If you’re a heavy user of IRC and/or IM you may well want to do it, because it’s really useful. So if you skip a lot of this post, do read that bit.

Education. You can learn quite a lot about how the internet (still, more or less) works by doing this stuff yourself. It will certainly teach you things. The internet is a somewhat different beast in practice these days, with so much of it existing inside Google’s and Facebook’s monstrously internally complex domains, but at a certain level it still works _more or less_ how the RFCs of the Internet Past declare it works, and running your own services will teach quite a lot about that.

Control. Obviously, the higher the level of functionality that you outsource, the less control you have over the implementation. This seems like a really big reason, but it often isn’t. When it comes to mail, a hosted mail provider will almost always provide everything you want. You just don’t need really fine grained control over the server configuration. You do not need to control the maximum simultaneous connection count to the IMAP server. You want a service that delivers your mail, allows you to send mail, allows you to organize your mail, and filters spam out for you. That’s really pretty much it. Gmail certainly achieves all these things. So do dozens of other services. Again, when it comes to web hosting, often what you want is a WordPress instance. You do not need deep control over the server’s PHP configuration. It’s more likely to irritate you than help you. There are cases where you actually need such control, as opposed to just maybe finding it cool that you have it, but those cases are fairly uncommon.

Fun. Yeah, it’s worth mentioning this. Some of us have very strange mindsets which find battling obscure MTA configuration to be an interesting way to spend our time. I’ve checked with medical professionals, and this is an incurable condition. Sorry. We just have to live with it. If you’re a fellow sufferer, you may self-host for no reason other than that you enjoy doing it.

Privacy. This is probably the largest remaining really valid reason. If you use a ‘free’ service for your infrastructure, you should always keep in mind that you almost certainly no longer own your stuff in any practical sense. If you use Gmail, Google pretty much owns your email. You don’t. They can look at it, use it to develop Skynet, send it to the government, and just generally do whatever the hell they like with it. In strict point of fact this is not entirely true – there are some legal restraints on what they can do with ‘your’ data – but I find it’s an excellent rule of thumb to work from. When dealing with such services I find it pays more or less to assume that everything you put into them will immediately be forwarded to the police and all your worst enemies, and then used to generate large amounts of advertising that will be mailed to you. Doing so avoids you being shocked in future when some of those things actually happen.

Paid services are a somewhat different ball of wax, in that you are not offering up your data in exchange for some services, but actually paying for the services. You therefore have a reasonable expectation that you will retain most of the ownership of your data. If you use a decent service provider, the contract you have with them may even possibly bear this out. However, there are still several problems, mostly legal ones. Your hoster can almost certainly be obliged to nuke your services and probably turn over your data to law enforcement under the terms of various bits of legislation, depending on where you are and where they are. Even if they’re not obliged to, they may well do so if asked by a sufficiently powerful body (like the government, or Universal Studios), on the basis that pissing you off is probably less damaging to them than pissing off the government. If you host your own services, this becomes much more unlikely.

It remains only to point out that, in brutal point of fact, this is often unlikely to be a consideration, but it is still worth bearing in mind, and though it’s not a huge issue for me, I do still value the fact that it’d be quite difficult for anyone to kill or forcibly access my mail or private web content.

In relation to this last point, it’s worth remembering that ‘self-hosting’ vs ‘using a provider’ is more of a spectrum than a binary state. Even those of us who ‘self-host’ are inevitably going to be outsourcing some stuff to someone. I use No-IP for DNS registration, for instance, so in theory someone could at least knock happyassassin.net offline by leaning on No-IP. I don’t have control over that level of things. But still, No-IP doesn’t own or even have access to any of my actual data, only my DNS records.

At the general level, even if you decide you want to ‘self-host’, you have a lot of flexibility in terms of what level you want to control yourself and what you want to pay someone else to look after for you. You don’t have to actually buy physical hardware and host everything off an internet connection you personally control. If that’s at, or near, the extreme ‘self-hosting’ end of the spectrum, then moving towards ‘completely managed’, we have:

* Stick your own hardware in a co-lo (i.e. you outsource the physical internet connection)
* Use a service like Slicehost where you get full root access to a bare virtual server (i.e. you outsource the physical connection and the ‘hardware’ provision)
* Use a service which gives you access somewhere higher up the stack

Everything else is a variant on that last one. It really only matters what level you get access at. Maybe you get a pre-set web server instance in which you can run whatever webapps you want. So-called ‘PaaS clouds’, like Openshift, are really just this kind of managed hosting, in a way; ‘IaaS clouds’ are pretty much like Slicehost. Maybe you just get a managed instance of some specific app or service, like WordPress (or ‘email’). It comes down to how much control and privacy you need, with the trade-off for more control and privacy usally being more expense and complexity.

So, there’s the theoretical for-and-against of self-hosting. It comes down to the broad conclusion that you probably don’t want to do it, and even if you do, you’re probably better off going for something in the middle of the spectrum – Slicehost, or one of the new public clouds, or something like that – than really doing (almost) everything yourself.

Assuming you self-host, or are going to start trying, despite all the above: here’s some notes on actually doing it.

Domain

Getting a domain of your own is pretty much the Point 0 of self-hosting. It’s also, fortunately, pretty simple. You can find a lot of confusing information on the topic but essentially it boils down to: buy a domain name and then set up the information that says ‘this domain is associated with this IP address’ – DNS records. It is much simpler to do these two things together, through one service. I use No-IP – their prices are reasonable and I’ve had no problem with their service. There are many other providers. It’s really as simple as picking a domain – like my happyassassin.net – paying your fee, and then filling out a little form which says ‘www.happyassassin.net should point to IP address xxx.xxx.xxx.xxx, mail.happyassassin.net should point to IP address xxx.xxx.xxx.xxx’, and so on. If you’re going to host mail for your domain, you’d also need an MX record, which says ‘mail for any address at happyassassin.net goes to IP address xxx.xxx.xxx.xxx’. And that’s really pretty much it. If you’re really self-hosting, as in you own the machines and they’re hanging off your own internet connection, all those IP addresses should be your own IP address. You’re going to want a static IP, for that.

Mail

Mail is the most complex thing to self-host and probably the least sensible, as hosted mail providers really do have it all figured out. I’m not going to turn this into a comprehensive ‘how to host your own mail’ walkthrough, because there are many of those already, and if you’re going to do it, what you should do is get a hold of a good guide and follow it carefully. But I do have one thing to contribute. I find it helps to bear in mind there are broadly three functions of a mail server, at least in my mental model, and you can pretty much treat them separately:

1: Retrieve messages from your existing mail accounts and serve them back out via IMAP for you to read on your client machines

I do this using fetchmail to actually retrieve the mail, procmail to sort it into folders and spam-test it via spamassassin, and dovecot to serve it back out via IMAP. I would strongly recommend the use of dovecot, it really is the best IMAP server around. It’s efficient, actively developed, highly standard-compliant, and supports things like IDLE very well. Other IMAP servers generally fail at at least one of those things. The retrieving and serving out are kind of different functions, but it makes no sense to do one without the other, really. There’s no point aggregating the mail from your various accounts in one place without also setting up a convenient interface – i.e. a server – for you to access it with.

2: Act as an SMTP server for your outgoing mail

When you want to send mail you send it through an SMTP server, right. Most people know that. Running your own SMTP server, for your personal use, has the advantage that you don’t have to keep changing to an SMTP server that’s accessible from the network you’re currently on. (Though, of course, if you just use Gmail, you can send outgoing mail from anything…)

3: Accept incoming mail from anyone to mail addresses at a domain you own

This is the most complicated case, probably. The fact that I’m set up to do this is why you can mail me at happyassassin.net, my own domain. When you send a mail there, your mail provider sees that mail to happyassassin.net is supposed to go to an IP address I own, and sends it there. That IP address actually is my own IP address, and connections to port 25 on that IP address are forwarded by my router to my mail server, which accepts the mail and sticks it into my mail folders just like fetchmail/procmail do for the email addresses I don’t administer myself.

I’m not going to explain in detail how to achieve all the above, but the key point is to remember these functions are distinct – you can do any one of them without doing the others. Where it’s easy to get confused is that you usually would use the same application, the same process, to do functions 2 and 3. I use postfix, because it’s marginally less insane than sendmail. But it’s best to think of them as two separate operations, and do one and then the other. If you think in terms of ‘how do I set up postfix’, you’re likely to get confused – finding guides for function 3 when what you really wanted was function 2. I know I did.

Another little note on that topic: the sketch of happyassassin.net mail I gave is, strictly speaking, incorrect. Your mail provider doesn’t really see that mail for happyassassin.net should go to my IP address: it sees that mail for happyassassin.net should go to No-IP. Why? Well, because I host my servers off my home internet connection, and that has port 25 blocked. Most home internet connections do. The way email actually works, mail for a domain is always initially delivered on port 25. The DNS record which says ‘mail for happyassassin.net goes to IP XXX’ cannot say ‘IP XXX on port 26′. It just says ‘IP XXX’. The port is hard-coded in the standards. So if you have a connection on which port 25 is blocked, you really can’t be the server that initially receives mail for your own domain. No-IP provide a neat service to get around this, called mail reflector. Essentially you set up your DNS records so that mail for your domain goes to No-IP’s server, and you tell No-IP the actual port of your server. Then No-IP’s server simply forwards mail straight through to your server. They don’t store it or have any access to it, except in the case that your server is down – they will keep it on theirs until your server comes back up, then forward it on. It’s a neat way around the port 25 problem, which costs $40 a year – at which price you could instead have fastmail handle your entire mail setup, including your own domain’s mail. Again, like I said, self-hosting is almost never actually economically sensible.

Web

Setting up a web server, at the 10,000 feet scale, isn’t very difficult. Basically, you do ‘yum install httpd’ (or equivalent), and you’re done. You already registered www.mydomain.com and pointed it to your server’s IP address. Now you set your router to forward traffic on port 80 to the appropriate box, and you’re done. People going to www.mydomain.com will see a ‘hello world!’ post that’s the default homepage for Apache. Oh, and you do want to use Apache. There are alternatives, but they’re rarely what you want for self-hosting, and you will find much more help with configuring Apache than configuring anything else.

These days, you’re likely not going to be faffing around creating static content and dumping it in /var/www/html on your server. You really want to run webapps – you probably want to run a WordPress blog, for instance. Essentially your web server is providing useful services for you.

The 10,000 foot overview of how to install web apps is similarly simple: yum them. The most common ones are packaged. WordPress is: you can just do ‘yum install wordpress’. There are guides for the finicky bits of configuration.

There’s one stumbling block you’ll hit for most webapps, so I’ll mention it quickly: they almost all need a database. Web apps rarely store things as files on your local disk, because that’s silly. They want access to an SQL database instead, and they’ll store their configuration, your blog posts, and whatever else in there. You almost certainly want to use MySQL for this. MySQL will be packaged in any sane distro. Once you install it, it will probably be configured with no root password and a guest account. You will want to set a root password and destroy the guest account. There are guides to how to do this in the excellent MySQL documentation. Then, for each webapp you install, you’ll likely create a new database specially for that webapp, with a user account specially for that webapp which has access to the database. You can do this with a single one line command. The webapp will ask for a MySQL username and password as part of its setup process; feed it the username you created especially for it. That way, no webapp can access another’s data; only root will have access to all the databases, and you should only use the root account for any manual poking of the database you personally have to do. Never give the root password to any webapp (or any other person). The most popular webapps, like WordPress, tend to have the MySQL setup well documented, and you can apply the documentation to any other webapp which just needs a simple MySQL config to work. Which is most of them.

That’s web serving. Here are some of the webapps I run on my server. You may not have known about some, and find ‘em useful.

WordPress. Well, everyone knows about WordPress. It’s a blogging platform. If you want to have a blog on your server, you’re probably going to want to run WordPress. It’s well documented, easy to set up, hugely popular (and hence well supported), does everything you need from a blog, and has a bewildering array of plugins. Of course, if all you want is to have a blog, it’s almost certainly a better idea just to get it hosted by wordpress.com than faff around with setting up your own web server.

Roundcubemail. This is a webmail front end. Combined with my mailserver, it’s the last puzzle piece in extremely painfully replicating the functionality of Gmail – it gives me a pretty snazzy web front end to my mail, for the rare cases where I’m on someone else’s system and don’t want to set up an IMAP client, or something. It also came in quite handy at one FUDCon when the port blocking was so tight that IMAP clients didn’t actually work. Roundcube is a very very good webmail app, it has all the functionality of a desktop mail client, is pretty fast, and has a very snazzy interface. The old-school choice, Squirrelmail, is about as functional but nowhere near as pretty.

tt-rss is a news reader webapp. Running it is like hosting your own Google Reader, essentially. It’s a lot nicer than just running separate news reader clients on each of your client machines, because it means your read/unread state is always in sync. But of course, you could always just…use Google Reader. It’s not like knowledge of what RSS feeds you like is likely to be astonishingly private information.

MyTinyToDo is a very simple todo list webapp. I tried for years to find a big stonking egroupware suite – contacts, calendaring, and tasks, essentially – which would cover those things and sync well with my desktop clients and my phone. I never quite did. But mytinytodo handles one piece of the equation – tasks – just fine. I haven’t bothered trying to sync it with desktop clients / phone because you can just use the web interface very easily on any of those devices, it renders nicely on phones. Of course, you could always just use a hosted service like Remember The Milk.

OwnCloud is a ‘personal cloud’ server, or to avoid the buzzwordiness, it’s basically just a file server webapp. You point it at a place where files live and it makes them available through a web frontend and also via WebDAV (which lets you mount them as a shared drive on most OSes). It pretty much just does that, but it does it quite well and easily. At FUDCon, Jeroen gave me a long list of things that are wrong with it, and Jeroen is massively smarter than me so I’m sure he’s right, but all I know is it does what I ask it to. It’s handy for, say, storing your (encrypted!) password database, or a document you want accessible from anywhere. I store a lot of my notes in it. Your hosted equivalent would be, say, Dropbox.

Finally (man, 4000 words? Anyone still awake?) we come to the one thing I host myself, find useful, and could not find a hosted-provider equivalent of: IRC and IM proxying.

This achieves for IRC and IM what using a mail server achieves for mail, or using a web feed reader achives for news: you can use many clients without them conflicting, and with the state preserved between them. How it works is essentially that you run an app which acts as both an IRC client and an IRC server. It connects to all your IRC servers, and then on your client machines, instead of connecting directly to Freenode or EFnet, you connect to the proxy, which also acts as an IRC server. It then forwards all the traffic to you.

What does this get you? Well, you can sign in from six different clients at once – and instead of each looking to the rest of the world like a separate user, they all act as ‘you’. You can have part of a conversation from your laptop, part from your phone, and part from your desktop, and the outside world won’t know the difference.

Also, as the proxy’s always logged in, you can disconnect all your client machines, and the proxy will keep storing conversations, including any private messages. Then the next time you connect a client, you’ll get a log of all the channel traffic that happened while you were away, and any PMs you got sent will show up. It’s very handy.

Finally it’ll give you a handy central store of logs. It’s just a much better way to IRC.

I use Bip as an IRC proxy. It’s very easy to set up – really, you just install it and give it a list of IRC networks and channels you use, and tell it your nickname. Then you run it, and set up your IRC clients to connect to it, not directly to the networks. And you’re done. It’s probably the easiest thing you can self-host, as well as being the most useful.

On the same machine I run Bitlbee, which is an IM proxy – it connects out to MSN, Jabber, ICQ, AIM and so on, and also acts as an IRC server, effectively turning IM traffic into IRC traffic. I then have Bip use my Bitlbee server, so when I’m using MSN, my desktop is connected to my Bip instance, which is connected to my Bitlbee instance, which is connected to MSN. Fun, huh? Bitlbee can also actually connect to Twitter and Identi.ca, effectively turning your ‘social network’ traffic into IRC. You can tweet just by typing a message into your IRC client, and tweets from people you follow pop up as IRC messages. It’s a fun interface if you’re used to using IRC.

So…that’s my self-hosting story. Why you probably shouldn’t do it, and some things you might want to run if you do. Hope it’s helpful!

Just pushed an updated Clementine-1.0 to Mandriva 2011 Main/backports, while testing i noticed this weird analyser:

Do not use it after lunch!

We would like to inform that a proposal to acquire Mandriva has been submitted by an external entity. As required in such a situation, the major shareholders have been asked to determine their position. As per today, Mandriva has not received every determination in written form and will, in consequence, wait until January, 23rd to decide on the future of the company.

Ces quelques mots pour vous informer que Mandriva a fait l’objet d’une offre de rachat par un tiers. Comme une telle offre est sujette à l’acceptation des actionnaires de référence, et que ceux-ci ne se sont pas tous déterminés par écrit pour l’instant, il a été décidé d’attendre jusqu’au 23 janvier pour choisir l’orientation future de la société.

Cela fait un peu plus de deux ans que je me suis mis au jeu de rôle Star Wars, un peu par hasard. En cherchant des idées de sorties sur le net, je me suis rendu compte qu'un maître de jeu (celui qui anime les parties de jeu de rôle) habitait à une rue de chez moi. Et bon, il fallait bien que je complète la panoplie du Geek :-p. Depuis le virus m'a contaminé...

Dans un jeu de rôle, pour représenter les différentes caractéristiques de son personnage, on utilise une feuille de personnage. Sur cette fiche sont regroupés ses attributs (dans Star Wars: dextérité, savoir, mécanique, perception, vigueur, technique) et les compétences qui découlent de ces attributs. Par exemple, la compétence "Sabre Laser" qui permet de manier un sabre laser est une compétence de dextérité.

J'ai utilisé plusieurs fiches en 2 ans. J'ai commencé avec une photocopie du modèle de personnage que j'avais choisi, un "Jedi Raté". Ce modèle est fourni avec le livre de règles. J'ai ensuite évolué vers celui de Robin Defives (merci à lui) que vous pouvez trouver sur scenariotheque.org. Il a mis deux fiches à disposition : une fiche de personnage, une pour un vaisseau (utile pour incarner un pilote).

Étant gêné par quelques limitations, j'ai il y a quelques temps commencé à faire évoluer ma fiche de personnage. J'ai par exemple modifié la fiche pour en faciliter la lecture, notamment en optant pour un meilleur alignement des différentes zones. J'ai aussi regroupé les zones qui avaient un lien, comme celles utilisée lorsque l'on est touché lors d'un combat: "vigueur" et "protection" (pour vérifier si le coup/tir reçu blesse notre personnage), et "santé", à modifier lorsque l'on est blessé.

Feuille de personnage Star Wars D6 (pdf)

Incarnant un "Jedi raté", je trouvais aussi les zones permettant d'indiquer les pouvoirs de la Force du personnage vraiment trop limitées. J'ai donc opté pour une page à part (ce qui m'a permis de me perfectionner dans l'utilisation d' Inkscape par une nuit d'insomnie). La présentation en diagramme plutôt qu'en texte permet de repérer plus facilement les dépendances entre chaque pouvoir, et de quelle capacité de la Force elles dépendent (Contrôle, Sens, Altération). Les pouvoirs représentés sont ceux de la seconde édition:

Diagramme des pouvoirs de la Force (pdf)

Il y a également la fiche des caractéristiques du vaisseau que vous pilotez. N'ayant pas orienté (pour l'instant) mon personnage vers le pilotage, je n'ai pas modifié celle de Robin:

Feuille de vaisseau Star Wars D6 (pdf)

Selon le personnage que vous incarnez (sensible ou non à la Force, pilote ou non) vous pourrez vous faire une fiche recto-verso sur mesure avec un site de fusion de PDF comme PDF Merger. Bonnes aventures ;-) !

Freecell Solver version 3.10.0 has been released. It is available in the form of a source tarball from the download page.

This release fixes two bugs - one with the --max-iters affecting only the last instance, and one with reading foundations with 0, and implements many small optimisations and cleanups. It also adds some experimental code with the so-called delta-states, where states are compactly encoded based on the original state. This functionality is not available in the main solver yet, but it powers the experimental on-disk-key/value-databases-based solver, which end up not scaling very well during testing.

Enjoy!

Maybe first i should put this code on github?

The code is now on github:  https://github.com/bedi1982/Get-Stuff

Décidément, mes rares articles en français sur la musique servent à communiquer autour de tristes nouvelles. J’ai appris par l’association Exultate la disparition d’Alain Recordier, tromboniste et sacqueboutier.

Alain Recordier à la sacqueboute en 2010

Ce fin connaisseur de la musique ancienne avait crée en 1986 l’ensemble Musicque de Joye (du nom d’un recueil publié vers 1550 à Lyon par Jacques Moderne). Il avait aussi été membre de l’ensemble de cuivres Da Camera, du quintette Arban, de l’ensemble Guillaume de Machaut et était trombone solo de l’orchestre d’Orléans.

Il avait participé à de récentes académies de musique sacrée de la Renaissance d’Etampes, dirigées par Jean Belliard

Alain Recordier lors de l'académie de la Renaissance 2007

Comme l’a si fort bien dit le mail que j’ai reçu: “Que de beaux concerts avons nous donnés ensemble ! Sa discrétion et sa grande gentillesse faisaient l’unanimité.”

Alain n’avait pas besoin d’épater la galerie. Il vivait sa musique intensément, donnant un air chaloupé aux canzons de Gabrieli avec énergie et velouté. Sa connaissance transparaissait dans les remarques qu’il sonnait à ses partenaires lors des répétitions avec son accent méditerranéen. Un court exemple pour mettre tout le monde d’accord, et on repartait dans la musique.

J’espère qu’aujourd’hui il joue avec tous ses pairs dans l’orchestre céleste ! Il me restera la mémoire auditive de quelques enregistrements que nous avons faits lors des académies, où je retrouverai la qualité de son son. Et le souvenir d’un musicien humble au service de son art.

(Images personnelles hébergées sur flickr.com)

From December 19 to December 28 zarb.org main server was down. This server host(s|ed) many things including this blog, Mageia website, PLF, ... The reason why it took so long is that the server is in the south of France, kindly hosted by Lost Oasis and we have no one nearby to physically access it, and in this case we had lost our main raid array.

This server (kindly donated by HP almost 10 years ago) has a remote administration card (P1218A) but it is not really usable for anything except rebooting the machine. The remote console more or less works with some of the java versions from sun, but most of the time it only displays the top third of the screen, until next refresh when it goes black, and misses many keystrokes. This made it unsuitable for accessing the RAID BIOS and finding the problem.

After about a week, for some unknown reason (I could have done it many times over the last 10 years), I thought of looking at the communications between the applet and the management card. Everything was clear text and very simple. The next days I wrote a ruby-gtk client for the card, accessed the BIOS, found that the 4 disks had been marked has failed without errors and were correctly syncronized, and put them back online.

Login
The first (and longest) part was to find how to login and get the session cookie. The exchange looks like:
GET /cgi/challenge HTTP/1.1
<?xml version='1.0'?><?RMCXML version='1.0'?><RMCLOGIN><CHALLENGE>DJRhNVfOWfuB8fS/6PFazg==</CHALLENGE><RC>0x0</RC></RMCLOGIN>
GET /cgi/login?user=FOO&hash=UtPRDzFS36s0jJBgTmtS4JDR HTTP/1.1

Challenge was obviously 16 bytes of data base64 encoded. Response was called hash and was 18 bytes whatever the password is. Given that it was written more than 10 years ago, I supposed it would be md5, even if it only gives 16 bytes.

I then wrote a small ruby application trying various combinations (md5(challenge + password), md5(xor(callenge,password)), xor(challenge,md5(password)), ...) and found that md5(xor(challenge,md5(password))) was giving me the correct first 16 bytes.

I then used an online CRC calculator to find that the remaining 2 bytes are "CRC-CCITT (XModem)".

Console
The other big part was the remote console.

Getting the current screen content is quite easy, it's a GET on /cgi/scrtxtdump (with an optional force=1 parameter).

In my initial tests there was 0x10 between each character so I just filtered them out. I found later that it actually gives attributes for the character (bold, color, ...) and now support the ones I have seen so far.

Sending a keypress is quite easy too, it's a POST to /cgi/bin with data being <RMCSEQ><REQ CMD="keybsend"><KEYS>space separated scancodes</KEYS></REQ></RMCSEQ>.

The result

The code is now online, still very ugly, but hopeful helpful :)

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/utsname.h>
#include <libgen.h>
 
#define VERSION "2.2.0.35"
 
 
int recursiveDelete(char* dirname) {
	DIR *dp;
	struct dirent *ep;
 
	char abs_filename[FILENAME_MAX];
 
	dp = opendir (dirname);
	if (dp != NULL)
	{
		while (ep = readdir (dp)) {
			struct stat stFileInfo;
 
			snprintf(abs_filename, FILENAME_MAX, "%s/%s", dirname, ep->d_name);
 
			if (lstat(abs_filename, &stFileInfo) < 0)
				perror ( abs_filename );
 
			if(S_ISDIR(stFileInfo.st_mode)) {
				if(strcmp(ep->d_name, ".") && 
						strcmp(ep->d_name, "..")) {
					printf("%s directory\n",abs_filename);
					recursiveDelete(abs_filename);
				}
			} else {
				printf("%s file\n",abs_filename);
				remove(abs_filename);
			}
		}
		(void) closedir (dp);
	}
	else
		perror ("Couldn't open the directory");
 
	remove(dirname);
	return 0;
 
}
 
void arch()
{
	struct utsname un;
	uname(&un);
 
	if (!strcmp(un.machine, "x86_64")){
		printf ("You are on a 64bit system, so, additional dependencies are requested (only2)\n");
		printf ("Be sure you have the Main32 media enabled! If they are, press ok:\n");
		system ("edit-urpm-sources.pl –expert");
		system ("urpmi libxscrnsaver1 libxv1 libxrender1 libXrandr2 libfreetype6 libfontconfig1 libglib2.0_0");
	}
}
 
void erase()
{
	chdir ("/opt");
	unlink ("skype_static-"VERSION);
 
	if (chdir("/opt/skype_static-"VERSION) == 0){
 
		recursiveDelete("/opt/skype_static-/"VERSION);
 
		unlink ("/usr/bin/skype");
		unlink ("/usr/share/applications/skype.desktop");
		unlink ("/usr/share/icons/skype.png");
		printf ("Clean\n");
	}
}
 
void install()
{
	arch();	
 
	chdir ("/opt");
	unlink ("skype_static-"VERSION".tar.bz2");
 
	//Check if wget is present//
	FILE *wget = fopen("/usr/bin/wget","r");
	if (!wget){
		system("urpmi wget");
	}
	/////////////////////////////
 
	system ("wget http://download.skype.com/linux/skype_static-"VERSION".tar.bz2");
	printf ("Downloading skype into /opt...\n");
	system ("tar -jxvf /opt/skype_static-"VERSION".tar.bz2");
 
	symlink ("/opt/skype_static-"VERSION"/skype", "/usr/local/bin/skype");
	symlink ("/opt/skype_static-"VERSION"/skype.desktop", "/usr/share/applications/skype.desktop");
	symlink ("/opt/skype_static-"VERSION"/icons/SkypeBlue_48x48.png", "/usr/share/icons/skype.png");
}
 
 
int main()
{
	int choose = 0;
 
	if(geteuid() != 0){
		printf ("Run as root\n");
	}else{
		printf ("1 - to Erase\n");
		printf ("2 - to Install\n");
		scanf ("%d", &choose);
 
		switch (choose){
			case 1: erase(); break;
			case 2: erase(); install(); break;
		}
	}
	return 0;
}

Download:  http://users.mandriva.com.br/~bedi/C_crap/get_skype.c

C'est un peu tard pour le signaler, mais le 31 décembre 1996, Blizzard North engendra ce monstre qu'est Diablo. C'était il y a 15 ans. A l'époque, j'ai entendu parler du jeu sur Micro Kids, une des premières émissions de télé dédiée aux jeux vidéos, dans les années 90, dont on peut d'ailleurs retrouver de vieilles vidéos sur abandonware-videos.org. Je n'ai commencé à jouer à Diablo que l'année suivante, une fois que j'ai eu mon premier PC (j'étais auparavant un Amigaïste comme certains le savent peut être). Déjà à l'époque, la sauce prenait bien. La collectionnite aigüe d'armes, et d'items en tous genres dans une ambiance bien glauque fonctionnaient du tonnerre : qui n'a pas frémi devant The Butcher et son "Ah... Fresh meat !"... Certains se rappellent de ces longues heures passées à finir ce jeu (là où certains mettent 3 minutes 12 secondes).

Et puis en juin 2000, il y a eu Diablo II. Peut être un poil moins attachant, mais au moins votre personnage pouvait enfin courir plutôt que se traîner, utiliser des sets complets d'armure (c'est beau, le vert), de nouvelles classes, un système de compétences complètement revu et bien d'autres choses. C'est le seul autre jeu (le premier étant Starcraft) auquel j'ai joué en LAN party entre potes. Ah, évidemment en te temps là j'étais jeune et sous Windows. Autre temps, autres moeurs...

Depuis, Diablo III a été annoncé, est en beta test... Sans qu'on ne sache quand il sortira vraiment. Le jeu est encore plus beau, mais... toujours pour Windows et Mac OS. Bon, de toute façon, quand je vois la configuration minimale requise (dual core à 4GHz), je me dis que je ne vais pas changer de machine pour un jeu. Mais il y a peut être de petits Linuxiens qui ont le matériel qu'il faut, eux. Et j'entends leurs grosses larmes perler le long de leurs joues... Pour ce profil rare et atypique qu'est le linuxien-gamer, un espoir subsiste pourtant: Diablo I et II étaient tout de même plus ou moins jouables sous WINE, alors qu'en est-il de Diablo III ? Hé bien Diablo III semble fonctionnel sous Linux, toujours via WINE. Bien des nuits blanches (autant pour l'installer que pour jouer j'imagine :-p) et des destructions de souris en perspectives^^.

En prime, Blizzard nous offre une petite rétrospective Diablo (à défaut d'une date de sortie).

Last week, I followed an NBIC course on Managing Life Science Information. The course included a tutorial on creating an ontology with TopBraid Composer.
A number of course participants, me included, couldn’t run this (Eclipse-based) application because it complained about an incorrect Java VM. Turns out, it only wants to run on the official Sun/Oracle JVM and not on the OpenJDK that’s included with most Linux distributions.
For other people running into this problem: the way to solve it is to:

• Install Sun/Oracle Java from your distribution’s repositories, if available
• Otherwise, download Java SE 7 from Oracle as a tar.gz file

If you had to download the tar archive from Oracle, extract it somewhere on a logical path, such as /usr/java.

When you have done so, you can use the update-alternatives system to switch between Java versions. If you installed the Sun/Oracle JDK through your distribution’s system utilities then chances are, it will be already added to the alternatives database.

Quoting anujjaiswal’s blog:

/usr/sbin/alternatives --install "/usr/bin/java" "java" "/usr/java/default/bin/java" 2
/usr/sbin/alternatives --install "/usr/bin/javac" "javac" "/usr/java/default/bin/javac" 2

And to configure systemwide changes use

/usr/sbin/alternatives --configure java
/usr/sbin/alternatives --configure javac

But why, exactly, doesn’t TopBraid run with OpenJDK? According to Jeremy Carroll, it’s because ‘We have places where we have to mess with JVM internals, mainly for clean up.’.
I doubt OpenJDK’s internals are that much different from Sun’s, but even if that were true– if you need to mess with JVM internals, in my humble opinion, you’re doing it Wrong^tm.

After migrating 2 of my kids and my wife from Kmail to Thunderbird last year, I finally decided this week-end to finish the last migration for my first daughter on her Mageia distribution.

I previously made unsuccessful tries, as her environement was different, with many more subdirectories, and special chars, so it didn’t work with the previous version of the script.

Now with the revisions 1389 and 1390 of the md2mb.pl script, I have successfully migrated her environment, without any manual intervention.

Hopefully, seeing the number of times the previous post was looked at, it will be again useful (even more now that it works better:-)) for others. I even clarified the license in revision 1391 for you to use more easily.

Happy migration !

To be honest the first packages appeared before Christmas as I was hoping to have everything ready as a gift ! But I met a certain number of issues trying to build all packages for the 99 different distributions I’m trying to build for ! This is due to my upgrade to Mageia 1 where the QEMU/KVM version proposed work differently from the previous Mandriva 2010.2 I was using.

Some i386 VMs are now freezing, so I had to find new correct parameters for them. Then autoconf wasn’t generating a correct content for all Mandrake/Mandriva build for mondo, so I had to call for these distro now %configure2_5 as a macro, instead of %configure.

And I still have some issues remaining, with busybox on SLES 9, Mandriva 2009.1, and RHEL 3, with some old SuSE (10.1-11.0) and old Asianux 2, RH 7.3/9, RHAS 2.1 … So Project-Builder.org gained at this occasion a new feature which consists in enumerating on the remote repo which packages have been built correctly or not. And chain the result to a sbx2vm option through the new –rebuild option, which will trigger the rebuild of all not correctly built packages. Very handy ! And will be used to finish publishing what is missing and still useful.

But I already delayed too much the delivery of that important evolution in the project life, so it was time to officially introduce MondoRescue 3.0.0 to the world !

And finally looking at all the modifications since latest stable, MondoRescue really deserve it’s 3.0.0 label ! I won’t be able to cope with the Linux kernel, now at 3.2, but hopefully you’ll find that new version usefull. It fixes a lot of issues brought recently on the mailing list. Remains to work on the Xen kernel support more precisely, but most of what I wanted to fix is in it, including OBDR fixes, RHEL 6.2 fixes, SSSTK ProLiant support improved, loop mount issues, bootable USB keys, mdadm support for metadata, a grub install fix among many others.

You’ll need to use mondo 3.0.0 with mindi 2.1.0 and mindi-busybox 1.18.5 to have a working environemnt as underlined on our Wiki.

And even if it’s a 3.0.0 number, I consider it stable and in the line of latest 2.2.9.x versions. I’d like to avoid copying my Red Hat friends with their .0 versions

Happy New Year and Disaster Recovery with MondoRescue !

I have nice kids, that love Internet. Sometimes a bit too much too our state, and with my wife we decided that after a certain hour during the week, it was wise to shutdown the Web access for the kids’ machines so they could start thinking sleeping

As I have a transparent proxy setup, I looked at how to do that with Squid 2.5 (old I know !) and used google to find examples.

The ones I found first turned out to be wrong in their explanations and the squid doc lacks of an example here.

Especially, the names of days should NOT be separated by spaces. Instead you rather need to use other examples which give good advises, and also explain the constraints on the hours.

So I came up with this configuration:

#Time
acl bfore_time time SMTWH 00:00-06:00
acl after_week time SMTWH 22:30-24:00
acl after_wend time FS 23:30-24:00
#Kids
#acl punition src 192.168.x.y/32
acl kids src 192.168.x.y1/32 192.168.x.y2/32 192.168.x.y3/32
#http_access deny punition
http_access deny kids bfore_time
http_access deny kids after_week
http_access deny kids after_wend


That way they are restricted to using the Web between 10:30PM and 6:00AM during the week, and later (11:30PM) the week-end. And I can ban one easily in case of punishment

Of course, that’s not a completely blocked context, as they can use our machines, or try to change the IP address (but they aren’t root and dn’t know that yet – but could learned which would be great !!). But it’s a kind reminder when it’s becoming late.

ACLs in Squid are very powerfull, and combined with Squiguard, you can also activate parental control if you want.

Les lutins statisticiens chez WordPress.com ont préparé un rapport annuel 2011 pour ce blogue.

Voici un extrait:

La salle de concert de l’Opéra de Sydney contient 2 700 personnes. Ce blog a été visité environ 16 000 fois en 2011. Si c’était un concert à l’Opéra de Sydney, il faudrait environ 6 représentations à guichets fermés pour pour qu’autant de personnes le voient.

Cliquez ici pour voir le rapport complet.

Maintenant la question que je me pose est pourquoi y a-t-il autant de personnes qui lisent ce blog Bonne année 2012 à vous tous, et bonnes futures lectures !

Statisticians at WordPress.com have prepared the yearly 2011 report for this blog.

Here is an extract:

The concert hall at the Syndey Opera House holds 2,700 people. This blog was viewed about 16,000 times in 2011. If it were a concert at Sydney Opera House, it would take about 6 sold-out performances for that many people to see it.

Clic here to read the full report

Now the question is why are so many people reading this All the best for 2012 and happy future reading

Yeah, 2011 is coming to the end, and had already switched place with 2012 in most countries already.

Somewhere on the road to curitiba, starting 2011....

For me, 2011 was extremely busy, exhausting but also very fruitful. I started it in a completely different situation from 2010, when I was a mere developer in Mandriva company. And during the course of 2010 and, later, 2011, I got the chance to work as engineering team leader for Conectiva, then development manager and, finally, technical diretor for Mandriva. And, after this experience, I got to move into Intel, working with even more challenging and amazing projects.

So yes, it was a hugely crazy and great experience. Thank you 2011 – despite all the problems and obstacles, you was great after all if we sum up everything.

See you all in 2012!

Yeah, 2011 is coming to the end, and had already switched place with 2012 in most countries already.

For me, 2011 was extremely busy, exhausting but also very fruitful. I started it in a completely different situation from 2010, when I was a mere developer in Mandriva company. And during the course of 2010 and, later, 2011, I got the chance to work as engineering team leader for Conectiva, then development manager and, finally, technical diretor for Mandriva. And, after this experience, I got to move into Intel, working with even more challenging and amazing projects.

And finishing 2011 in this landscape...

So yes, it was a hugely crazy and great experience. Thank you 2011 – despite all the problems and obstacles, you was great after all if we sum up everything.

See you all in 2012!

Yeah, 2011 is coming to the end, and had already switched place with 2012 in most countries already.

For me, 2011 was extremely busy, exhausting but also very fruitful. I started it in a completely different situation from 2010, when I was a developer and impossible-mission-solver in Mandriva. And during the course of 2010 and, later, 2011, I got the chance to work as engineering team leader for Conectiva, then development manager and, finally, technical diretor for Mandriva. And, after this experience, I got to move into Intel, working with even more challenging and amazing projects.

And finishing 2011 in this landscape...

So yes, it was a hugely crazy and great experience. Thank you 2011 – despite all the problems and obstacles, you was great after all if we sum up everything.

See you all in 2012!

This is my final rank (among 7897 contestants) on the last Artificial Intelligence challenge on aichallenge.org. That is not the best result ever, and I'm a bit disapointed as I was aiming for the top 500 but I'm fairly new to the AI world...

I definitely had difficulties in chosing between developping my own algorithms or seeing what was seen as interesting directions to explore, like the anti-objects approach. Being a noob in the AI field, the Wikipedia articles on pathfinding algorithms were extremely useful, as well as some Linux Magazine articles on the same field.

Unfortunaltely, I became less active a few weeks before the end, as a work colleague had a much better program on his first attempt, while I still had worked a great amount of time on that... Good lesson for next time: the only competitor I'm trying to surpass is myself. The end of the contest being at Christmas, I couldn't try to catch up, as I was late in buying the Christmas presents too :-).

The version that was used for the final tournament was version 12, even if I had since then done some small improvements and bug fixing. The whole code is available on github, under the WTFPL 2.0 license.

My overal stats:

• 1342 amont 7897 (top 17% of contestants - with an undefined amount of really active contestants)
• 77th French contestant among 295 (top 27%).
• 35th C user among 248 (top 15%)

I learned a lot of things and had lots of fun, so expect me to come back on the next contest, with better basics ;-)

Running 1,266 km  (786 miles):

Cycling 3865 km (2401 miles):

Let’s see if I can improve for next year

(The Hebrew text will be followed by an English one).

English Version

Please note the change of venue. This is the building where we started having TA.pm, and not the one which we used for some of the recent meetings.

On 28 December, 2011 (Wednesday), the Tel Aviv Perl Mongers will hold their monthly meetup, and this time it is going to be special. We meet at 18:30 and the talks begin at 19:00. The address is: Shenkar College, main building on Anna Frank street, Ramat Gan, Room 300.

One can find more details on the web-site of the Tel Aviv Perl mongers.

This meeting will hold the following talks:

• Visualizing the brain of Vim by Ran Eilam - You loved him in "Game frameworks with SDL, Moose and Coro", you cried over his performance in the after-meeting get-togethers, and you will absolutely adore him in "Visualizing the brain of Vim". All-star legend Ran Eilam will talk to us (both beginners and experts) about Vim and how to fathom this incredible beast. I warn you, the first two lines in the audience will be damaged by Emacs flames.

• A bunch of modules which made sense writing - by Sawyer X: I'm going to cover some utility modules I've written, why they were written and when they are useful. At the end, you'll have to judge whether they were worth writing at all. There will be voting booths available! (We'll cover Algorithm::Diff::Callback, App::Genpass, Data::PowerSet::Hash and Module::Version.)

• To upgrade or not to upgrade, Perl 5.6 vs. Perl 5.14 - ik vs. Sawyer X: A source of great concern in the Perl users community is whether to use an up-to-date Perl and what version should be considered "too old". On one hand, we have the management that wants as little cost and changes as possible (sometimes along with systems administrators), and on the other hand, the developer who wants to use the latest technologies, and is often stuck on obsolete systems.

  In light of this serious consideration, we're going to take turns poking at each other with a stick, wearing funny hats, damning the unbelievers and heretics until they submit to one side: 5.6 or 5.14!

The entrance to the meeting is free-of-charge, and everyone are welcome to attend. See you there!

Yeah, I admit that my semi-periodic updates about Intel Linux Graphics got more “seldom-periodic” than “truly-periodic” for the past weeks. But have no fear – they are back! And I am still on my self-appointed bi-weekly schedule estimate. This is what’s good about semi-periodic schedule – one never can run too much out of it .

So starting with the coolest news – the Mesa team is getting close to the GL 3.0 milestone! Yeah, with latest GL_ext_transform_feedback patches from Paul Berry, the last major piece of GL 3.0 spec is getting into place. There are still some extensions missing and lots of smaller tasks to be done, but it is possible to say that we are almost there. I think that this is really exciting for both us, and for all the Linux and open-source users in the world – so yeah – we’ve been good boys and girls during the year and Santa Claus gift has materialized itself in form of almost-full GL 3.0 support in Mesa.

Who knows, maybe prior to the Chinese new year we’ll receive the 2nd part of this gift (in other words, mesa 8.0 release ).

On kernel side, the 3.2-rc6 release brought lots of awesome changes to our drivers. Yes, I am talking about everyone’s favorite rc6 and semaphores features. They are on by default on Ivy Bridge architecture, and are also enabled on Sandy Bridge if VTd is disabled. So most of you should enjoy greatly improved battery life, considerable faster performance and also enhanced stability within the i915 driver when Linux 3.2 will be released.

Besides those patches, work has started on collecting patches for the 3.3 merge window. Daniel Vetter sent his pending patches in a form of tiny 43-patches series. Those patches bring PPGTT support, improve debugfs handling, enhance pread/pwrite performance, fix swizzling for SNB/IVB, improve forcewake operations and enhance debugging support for cases when GPU rings get stuck.

Ben has also sent his patches for scheduling/throttling, but they haven’t received much interest except from myself and phoronix . Those patches add support for more fine-grained GPU scheduling and rings load distribution between individual process. I am really interested in this work, and I hope that they will be accepted into the main kernel in the foreseeable future.

Also on kernel, Rodrigo Vivi and Paulo Zanoni sent out some patches which finally fix some corner cases for TV-out and SDVO outputs. This certainly should make many users happy out there just in time for Christmas.

And finally, for the kernel size, Chris Wilson came with a patch which works around the missed IRQs issues on Ivy Bridge platform. With this patch, and with semaphores being enabled on Ivy Bridge by default, I am very happy to say that we don’t have any blocking bugs for Ivy Bridge in our bugzilla. I think that it comes as a nice Christmas gift for all the users out there (the ones who already have an Ivy Bridge machine, and the ones who will get it by its launch – which is still 4 months away). Of course, I won’t talk much about it prior to its official launch, but trust me – Ivy Bridge rocks!. I can’t wait to have an Ultrabook based on this platform for myself…

Besides mesa and kernel, it is worth mentioning that on the 2D side, Zhigang added full Glamor support into the driver. The Glamor acceleration is still considered very experimental and non-stable, but now it is available for the world to take a peek on it and witness how it works with their own eyes.

So I think that this is pretty much it. We have hundreds of patches floating around for all the projects, thousands of emails and millions of users in the world – and we are working hard to make all of them happy with the results of our work. 2011 was extremely productive and rewarding for us – and I hope that the year of 11111011100 (a.k.a., 0x7DC or 2012_base10 for the ones still reading in decimal numbers out there ) will be even more interesting*!

See you!

(*) Assuming the world won’t end in a core dump caused by the Mayan millennium counter overflow bug .

The nice thing when you register your camera on the Nikon Web site, you received mail when a Firmware update is available. That’s how I learnt that version 1.03 was available.

Of course, there is the easy way to perform the upgrade: you take out the camera’s card, put the firmware on it at the root, and voila.

But I tried to do that from my Linux system, with the camera directly connected to the box in USB. As it only works in PTP mode, it could a bit more chalenging

First, get the firmware from the download page and look at the instructions.

Once you have the right file downloaded, extract the firmware from it:

$ unrar x F-D7000-V103W.exe
Creating D7000Update OK
Extracting D7000Update/D7000_0103.bin OK
All OK

Then check that your camera is seen correctly:
$ gphoto2 --auto-detect
Modèle Port
----------------------------------------------------------
USB PTP Class Camera usb:008,005

$ gphoto2 -u D7000Update/D7000_0103.bin --folder /store_00010001/
[...]
*** Erreur ***
PTP Accès refusé
*** Erreur (-1 : « Erreur indéfinie ») ***
[...]


So I activated debug mode, and it turned out that you can’t really write with PTP mode that way on the card
Same thing if you use Dolphin and try to copy paste the file, the target is blocked !

So back to begining, I turned off the camera and removed the card from it.
Then using USB disk emulation, it was easy to copy the firmware at the root of my key placed in a card adaptor on the USB port of my system.
$ cp D7000Update/D7000_0103.bin /media/D7000

Then I put back the card in the camera and turned it on again. I went into the FW menu,just to realize that I didn’t had any way to upgrate, contrary to what was written in the doc. Going back and forth I finally had the idea to unconnect the camera from USB, and then the menu option appeared !
However, the battery wasn’t full anymore so had to charge it again

A couple of minutes later, the camera was upgrading, and I now have a fully patched one !

Mr Nikon: is it that difficult to provide direct USB disk support in your cameras ? That would be the most direct way for us, Linux users, to deal with your wonderful toy ! Maybe for the next firmware update !!!

The normal way to remove a completion for a Bash command (say “mv”) is to do “complete -r mv”. However, with the bash-completion package installed on Mageia Linux Cauldron, this is not enough because it also adds a default completion for every invoked command. So in order to override this behaviour, type “complete -r -D” and then you can remove the commands’ completions permanently, using “complete -r mv” or whatever.

After a busy first day, here is the report for my second day at LinuxCon EMEA 2011, which started directly with some sessions (I skept the plenary for once):

Distributed redundancy by Roopesh Keeppattu – Huawei

Redundancy is about availability, by duplicating components to avoid unavailability of the service.
Availability measured with ’9′.
4 nines is 1 hour per year, 5 nines means 5 minutes, 6 nines 32 seconds.
Major types of redundancy: standby (cold – the other server remains unpowered, warm – all servers powered, hot – all servers provide identical services).
ALso notion of N modular redundancy (N servers in parallel).
1:N redundancy = 1 standby for N active units.

Traditional redundancy: mainly based on backup HW systems, with similar capabilities so large CAPEX and OPEX.
So moving to distributes redundancy to reduce costs.
1:1 scenario taken in account
Instead of duplicating HW, there is a duplication of process instances on a set of servers
3 models available: live migration of OS/VM or of processes or of pre-distributed processes

• live migration OS/VM: preserve states, less complex from application point of view but higher migration time due to size of what to transfer.
• processes migration: you encounter more complex migration design, which has to be part of the application, and gain on the data to transfer. This method can also provide dynamic load distribution. But you need pre-failure detection for failover.
• pre-distributed process: you increase again the ressource optimization, availability and switching time but also the complexity, the need of additional SW to deal with states and the linkage to the application.

The future is in redundancy in the cloud, and ressource abstraction.

I was hoping for a more in depth presentation, and was not satisfied by this one, as it didn’t go into details, just remaining at the surface. However, this is a critical topic for most customers today in their Linux adoption.

Experiences booting 100s of thousands to millions of Linux VMs by Andrew Sweeney – Sandia National Lab

Managing a large number of VMs presents some challenges and horror stories (such as filling fill the switch CAM table, creating VM feed back loops, finding some unique bugs or odd behaviour). Even 0.01 % of error is 100 VMs in their case.

They tried multiple technologies such as lguest, QEMU, KVM, NOVA. They are using a mixed of technologies due also to hardware limitations.

Guests configurations are computed at runtime. Everything is stored in RAM. They treat VMs as an application process. They use standard tools, the same TCP stacks, kernel…

They are using VMatic to generate the images and boot 1000 VMs in < 3 minutes.
Another tool used is Gproc (Cluster Management tool written in Go) allowing O(ln(n)) execution time.. It scales beyond 200K+ instances. Web based interface.

The first cluster type was:

• Using PXE boot for booting physical host Hypervisors and then start the guests.
• In July 2009, they reached 1 Million VM with lguest and 4600 Dell Super Computer (256 lguest per node) bootleneck being RAM.

They then created KANE (sort of their own cloud approach) made of 520 nodes with 12 GB RAM with Video cards (because that was more expensive to remove them !!)) 13 racks, 40 nodes/rack, 1 PDU/rack.

Then they developed a strongbox ARM cluster made of 490 nodes 512 MB RAM little power needed with lguest.

Then they started Megatux 2.0 to reach a higher number of VMs. Everything is virtual. They even created a network creation language. Use virtual quagga & linux virtual routers (+ physical) and virtual VDE switches. It supports multiple OS normally, but for Windows they got many blue screen (ipconfig before IP is sup, ping before IP is up, …). They’re using KSM a lot (and made patches) and various approaches to reduce VM footprint. gproc is used after the initial boot to push the VM images and start the VM + aggressive KSM.
Cold boot to experiment is performed in 7 minutes. 1 daemon per host to regulate KSM, VM state.

Interesting problem to collect info from 1 Million of nodes overloaded and where to store it ? Using network sniff, VM inspection. For that they used a MongoDB backend populated at runtime. Data collected in in best effort mode.
They’re looking at using KVM tool instead of QEMU/KVM to reduce memory footprint and AXFS (Advanced XIP FS) combined with cramfs.
Next steps with Android, more realistic network usage, improved monitoring, data visualisation and error handling.

A good talk on very unusual context with some interesting issues to consider, even if far from being current problems as of now.

I then met with my colleague Sue Paylor, who is one of our excellent FLOSS expert in EMEA, and that was again a good talk exchanging about our respective customer experiences, how to improve HA with Linux, and lots of various topics.

Providing High Perfomrance Round table (instead of SuSE Keynote)

Ludek Safar, Ministry of Interior, Czech Republic approched the linux topic from the desktop side, and they’re now moving to the Data Center (Oracle instances on physical hardware and the rest in Xen VMs including java based custom devs.). They help by giving publicity for some FLOSS projects. The choice of an enterprise distribution is specifically to be the linkage with the communitites. He likes the embedded approach with regards to the fully integrated hypervisor which provides the perfect cloud solution for them.

Dr. Udo Seidel, Amadeus explained that they started 9 years ago with Linux. They have done lots of internal developments including lots of mission critical workloads. Participating to events is key to keep good technical exchanges, influence the developments, give feedback. He really likes the flexibility and the open mindset. However he is still missing a central approach around role based manageemnt (a la AD).

Andreas Pöschl, BMW explained that they started back in 2003 for servers, and in 2006 decided that Windows and Linux were the strategic OS on x86. They run SAP on Linux e.g. and desktops on Windows. They do virtualization (1000 VMs) with Xen, including 16 cores 64 GB VMs for SAP. They don’t do direct contributions, but rather provide use/test cases for large configurations, and rely on their distribution providers to do the return. Sharing what they do with Linux is also important to improve the ecosystem. He insisted on the freedom of choice which avoids vendor lock-in and also marked his appreciation for the large set of possibilities offered by FLOSS. He is still concerned by boot time. BMW has requirements around storage and scale out, so they appreciate the work done on Btrfs. He mentioned usage of Linux in GENIVI that will bring infotainment to the end users.

Nils Brauckman underlined that the SuSE company, is organized to take this feedback and make it available upstream, doing that since 20 years, as well as providing mission critical solutions to customers, and detailed the new features brought into Linux 3.0 (btrfs rollback, snapshots, trace capabilities, …) bridging the gap between Unix and Linux. He underlined also for SuSE the new agility brought by being back as a separate Business Unit, operating like a single company.

I like more and more this type of round table, as it gives concrete production example of FLOSS usage, and show how serious customers are today, and also how far they want to push their usage, which creates interesting challenges for us !

It takes a community/village to raise a Distribution by Tim Burke, Red Hat

"Unix was a job, Linux is a crusade" Tim said it’s awesome to be a part of RHEL as well as OLPC.
He started by showing a large set of stars in the sky (glibc, LVM, X.org, Linux), independant stars that only come together when gathered in a distribution, which give them visibility. Then he showed the various actors, hardware vendors, translators, designers, lawyers, testers, and distribution vendors as well. The real competitors of Red Hat are VMWare, Microsoft, not the other collaborative groups such as other distribution makers. He explained the relationship around the kernel between upstream, Fedora and RHEL. He also underlined the benefit of working upstream such as they did around the Real Time extensions, instead of coming with a large patch developped separated.

The role of distribution makers is also to coordinate with hardware vendors (I’m well placed to know that !). Distribution can help create communities such as for AMQP, which was a real common need among FSI companies, as they know how to do it.

Mantra is "get it upstream first". Being divergent is being ignored, costs more, represents more work.

Time then gave some numbers:

• 80% of Fortune 500 run Linux.
• 92% of supercomputers for healthcare or analytics run Linux.

He mentioned the OVA to bring up in the stack integrated solution based on KVM.
No keynote without cloud, so Tim had to mention it and noticed Linux usage in it, and the integration characteristics it requires, very near from the one you have to make a distro.

A good talk, but not as pushy as the one made by Jim Whitehurst

How Linux runs the World of Finance by Christoph Lameter, Graphe Inc.

Christoph started by explaining the various players (Stickes, traders, banks, …) and explained their needs of speed. This creates the need for certain technologies (Real Time, kernel, binaries and network optimisation, RDMA APIs, fast C++ code, processor caches). One problem is the limitation of speed of light (even if that may change !). That sounded like a joke first, but is very serious !! 200 µs to go round the earth. It creates limitations to signaling of events.

We’re moving from manual to automated trading. Hours vs ms, human vs compute/algo, 30-60 trades/min vs 1000/s. Manual is used as a backu p mechanism only today.

The case for Linux is because you can modify what you want, and such win against competitors by speed improvements. The first there wins ! Windows couldn’t make it in term of latency in its network stack. Linux was already used for Internet, large companies such as Amazon, Facebook, … All major stock exchanges are on Linux today. Commercial solutions vendors focus on Linux. Solaris is diminishing after Oracle bought Sun.

Distributions used are mainly RHEL, some SLES (Germany mainly), a bit of Gentoo and Ubuntu/Debian.

There are still some challenges for Linux in Finance: involvement upstream is rare, as they want to protect their advantages. Regression in kernel components is creating higher latencies (so some still run RHEL 3 !). Christoph Gave an example of a customer having a 200% regression moving from RHEL4 to RHEL5.

The Forward path is with direct access to hardware (OS bypass) to gain on latency. RT linux does not scale and increases average latency. RT linux is used by exchanges not traders.
Linux dominates finance for the forseeable future. Common hardware looks like supercomputers today (Numa). HPC goes mainstream. Offload technology is seen with suspicion by the community. So again no willingness to contribute these improvements upstream.

One of the best presentation of the day, with lots of anecdotes, and a visible knowledge of the topic end to end.

Where is the Money in Open Source? Business Models and the Marketing of Open Source Technologies by Nithya Ruff, Wind River Systems

Nithya created a story to illustrate this talk. 3 communities: producers, distributors, consumers.

• Producers are interested by solving problems. License used is key. It’s all about meritocracy. How do developers make money ?: being hired by a company, consulting contracts, venture funded, sponsorship/grants/donations.
• Why consumers use linux: no vendor lock-in, comparable perf and high quality, time to market and savings, choice and flexibility, empowerment,, innovation and transparency
• Distributors make it available for consumers with support, favour FLOSS adoption making it safe to use, employ developers, solve some issues and contribute back, market FLOSS, and serve as a liaison between consumer and developer. Successful business models are subscription, services fee, training, books but also proprietary extensions

Marketing FLOSS is different. You need to clearly articulate your added value in the ecosystem. So you have to add value. (TTM, ROI, Integration, risk mitigation)
Prediction, by 2021, 100000 infrastructure core endpoints and 1B mobile endpoints and 20B MtoM endpoints.
Even more need to collaboration between the various communities.

I was expecting a bit more from such a presentation. Good for beginers, but lacks new thoughts on our ecosystem.

ReaR by Dag Wieers

I was particularly interested by this presentation as ReaR is a MondoRescue competitor, and Dag is mister rpmforge, mrepo, … so was really curious to attend it.

Rear provides a Disaster Recovery Workflow in bash. Its framework is easy to use and extend. It supports HP SmartArray, SW Raid, DRBD (not MondoRescue !), LVM, multipath, ext2,3,4, xfs, jfs, vfat. It supports tape, ISO, USB, eSATA, NFS, CIFS, rsync, HTTP, FTP, SFTP. It also provides back-ends with TSM, HP DP, Bacula, …

ReaR works on RHEL4,5,6. It’s shipped with SLES (the one distribution on which it’s tested).

It saves storage info and network info. It has local GRUB integration, serial console support, network and SSH key integration, syslinux management.

Dag then explained the use case of the Belgian Federal Police (HP-UX to Linux migration using Ignite before):
Developers prefered USB usage for flexibility instead of OBDR (also lack of OBDR support by latest HP HW). It manages labels on tape and USB devices. For this project, they support a central DR server with PXE boot and control the HTTP PUT upload with ACLs.
They provide a tool to detect when changes are needed to relaunch ReaR by cron.

In the future they plan to work on: better rsync support (like rsnapshot or rbme), more backup backends, PXE integration, code base reorganization, release process, website+doc, dev tools.

Dag made backup and restore demos.

I really liked the presentation. Dag is an excellent presentor, and has accomplished a huge work to improve the tool.If only I could also have some brilliant contributors like hom for my project !!

So after the presentation, I introduced myself to Dag, and we ended up talking together most of the evening during the dinner organized in a central place of Prague. We talked not only about DR, on which we share a lot of common ideas, but also about a large set of other topics, some of them HP related such as webOS future, … I like making new relationships during evens like LinuxCon as you end up talking with luminaries and that helps a lot enrich your own vision.

Some pictures of this event are available on Picasa.

Sometimes, you read a news, and just found out that this is so great the only word that comes to your mouth is wow !

I must confess that since the last ten years I have not been that impressed by HP CEO decisions in the past. And especially recently. At that time I was thinking about WebOS: “Of course, in order to compete with Android, it would need to be Open Source IMO” and of HP: “HP needs to respect its willingness to really invest in R&D more as promised.” And I even concluded: “Even if I think that founders are leading their company with a unique perspective, there is no reason that another board and leaders can align and do the same. Maybe after re-reading the HP Way.“

To be honest, even if I’m trying to reach a “Strategist” level inside HP, I’d never have thought to be so spotty !!

And then came the wow effect: HP to Contribute webOS to Open Source.

And the bonus effect: “Meg: There will be milestones along the way, but one thing I know about technology is that if you believe in something, you have to have a longer term horizon than next week, next quarter, or next year.”

A CEO of a fortune 500 company is just saying that we need to think not only on a quarterly base, but *also* in term of years in technology. That’s a tremendous change, especially with regards to the Hurd period. Which makes me feel much more confident in HP’s ability to come back much stronger in the play after this cahotic year. That’s a fantastic evolution IMHO, for both HP employees and customers ! As this way of thinking will also serve other approaches in other BUs.

“Meg: Well first I want to set expectations about time frame. This is going to take some time. If you look back at the history of Mozilla or Red Hat — these things did not become giant platforms over night. This in my view is a 4 or 5 year timeframe, and I want to make sure we really communicate that.“

We now are taking point of comparison with Mozilla or Red Hat. Wow again !! This also seems to reinforce the recent talk she made at HP Discover in EMEA, insisting on our roots as a hardware manufacturer. And I was already thinking that of course, using Open Source much more with our power of great hardware manufacturer, and one of the best service company could just place us as being really successful in the coming years.

So I’m really forced to change my mind on the conspiration theory, and deliver on what I said then: “open sourcing a technology helps driving the business in favour of the open sourcer, and doesn’t reduce it. Especially when the most costly part (the investment in R&D) has already been done.“

So I promise that I’ll do all my best to help HP make this new Open Sourced WebOS successful as much as I can. And as a start, as I was so successful in my guess, I’m launching a new idea in the basket: HP should now contact the Tizen community (which has still not published its architecture diagram), as a member of the Linux Foundation, and propose them to join forces around webOS which exists, in order to add to it what Tizen wanted to get in such platform instead of re-inventing the wheel.

Let’s all work together for once, to have the biggest community behind a brilliant platform, which is now open source, and be here with a Linux based approach, as well as on the server side, the best offering for customers for tablets, phones, TVs, IVIs, …

As an example, I’d love to see some KDE apps available on top of WebOS, my favourite being tellico.

And in order to feast that, I’ve just accepted an enhancement request for project-builder.org to add as quickly as possible now !! My little stone to improve the ecosystem.

Never sure what the future will be, but this past decision is already a new reference point in IT history. Thanks Meg, Marc and all the others who helped obtaining that ! I can now be proud again of working at HP

Until now symbolic links were not handled in Nepomuk. Today I commited the last patch for the new symlink support in Nepomuk. The solution I chose is not the theoretically perfect one. That would have taken way to much effort while introducing all kinds of possible bugs, regressions, API incompatibilities, and so on. But the solution is nice and clean and simple.

Essentially each direct symlink is indexed as a separate file using the content of its target file. (This is necessary since a direct symlink might have a different file name than the target file.) The interesting part are the indirect symlinks. Indirect symlinks are files in a folder which is a symlink to another folder. An example:

/home/trueg/
|-- subdir/
   |-- thefile.txt
|-- link/ -> subdir/
   |-- thefile.txt

Here I have a folder “subdir” which contains a file “thefile.txt”. The folder “link” is a direct symlink to “subdir” whereas “link/thefile.txt” is an indirect symlink to “subdir/thefile.txt”.

Indirect symlinks are simply stored as alternative URLs on the target file resources using the kext:altUrl property. (The property is not defined in NIE since it is not theoretically sound with respect to the design of NIE. It needs to be considered a beautiful hack.)

The only situation in which the alternative URLs are actually needed is when searching in a specific folder. Imagine searching in “/home/trueg/link” only. Since there are no nie:url values which match that prefix we need to search the kext:altUrls, too.

The result of all this is that nearly no additional space is required except for the kext:altUrl properties, files are not indexed more than once, and files in symlinked folders are found in addition to “normal” files.

In my tests everything seems to work nicely but I urge you to test the nepomuk/symlinkHandling branches in kdelibs and kde-runtime and report any problems back to me. The more testing I get the quicker I can merge both into KDE 4.8.

Lastly the pledgie campaign is done but the search for funds goes on:

Les dépôts 2011 ont été signés avec la clé officielle Mandriva (branches main et contrib). Cela signifie qu’il n’y a plus de clé cooker dans les dépôts 2011.

Ce changement provoque une mise à jour majeure de tous les miroirs, car les RPMs signés ne sont plus les mêmes, ils sont donc resynchronisés sur tous les miroirs.

Vous devriez donc importer la clé publique officielle, enlever la clé cooker si vous le souhaitez, mais les RPMs déjà installés restent signés avec la clé cooker.

Vous pouvez utiliser n’importe quel mirroir 2011 pour importer la clé:

rpm --import ftp://ftp.proxad.net/pub/Distributions_Linux/MandrivaLinux/official/2011/i586/media/media_info/pubkey_contrib
rpm --import ftp://ftp.proxad.net/pub/Distributions_Linux/MandrivaLinux/official/2011/i586/media/media_info/pubkey_main

Pour retirer la clé cooker :

rpm -e gpg-pubkey-22458a98-3969e7de

D’autre part, libreoffice a été déplacé de media/testing vers main/release, pour corriger les problèmes #64224 et #64246. La validation QA a été faite selon la procédure habituelle.

Hello everyone!

As you probably know, Mandriva/ROSA officially supports only KDE as Desktop Environment. But as we promised earlier, we support any activity of our members with other Desktop Environments. For example, currently we have a good distro on the LXDE created by Alexander Kazancev. 

Today, I’m glad to announce a new distro based on the Mandriva/ROSA 2011 repository: ROSA Desktop 2011 GNOME Editon Alpha (ROSA Desktop 2011 GE, RDGE). This unofficial distro created by Arkady Shane and by me for users who, for any reasons, do not want to use official distribution with KDE and want to use GNOME. 

RDGE is an unofficial distro, it will be developed as community wants. So, if you are a GNOME user, and want to be involved in the RDGE development, or you just want to say an idea - feel free to write your comment here, in the my blog. Later, we’ll create a wiki page for the RDGE and if community is interested in RDGE and there are people who can take an active part in the distro making, we’ll give more opportunities for them (ROSA’s resources for building a distro, wiki, IRC, mailing lists, etc). 

OK. Let’s take a look at the distro. Default applications:

• GNOME 2.32 (yes, yes, a “good old” GNOME 2 :));
• Firefox 8, Evolution, Empathy, Ekiga, Deluge for internet surfing, mail, instant messaging and torrents;
• Shotwell and GIMP for photo editing;
• LibreOffice for documents making;
• Totem, Rhytmbox and EasyTag for playing and editing media files;
• Other software included by default: Gwibber, Evince, FBReader, SimpleScan, EOG, Cheese, File-roller, Brasero, Gnote, GNOME Games, GNOME Terminal; 
• Kernel 2.6.38.7 (will be updated to 2.6.39.4 soon);
• ROSA icons and ROSA-Elementary theme by default (with some bugs for now);

Some screenshots:

                      Pic. 1: ROSA Desktop 2011 GE default view

                     Pic.2: LibreOffice and Nautilus in ROSA Desktop 2011 GE

Some words about the distro. Because it is based on the Mandriva/ROSA 2011 repositories it is quite stable and “Alpha” prefix is more about stuffing rather than stability. So, I appeal to all our testers and GNOME users: please download iso-image and test it and give your feedback! Do you like it, default applications, theme? I think, as RDGE is an community-driven distro, community should decide all about it. 

Known bugs:

• ROSA theme is used not in all components, and has some bugs (for example, Nautilus looks not ideal sometimes);
• Old kernel.

You can download iso-images from here. 

NOTE: ROSA Desktop 2011 GE is an unofficial distro, ROSA/Mandriva will not provide support for it.

It is a typical problem: we downloaded images from a camera, maybe did not delete them from the camera instantly, then downloaded the same images again next time, maybe created an album by copying images into sub-folders (without Nepomuk Digikam can only do so much ;), and so on. Essentially there are a lot of duplicate photos lying around.

But never fear. Just let Nepomuk index all of them and then gather all the duplicates via:

select distinct ?u1 ?u2 where {
  ?f1 a nexif:Photo .
  ?f2 a nexif:Photo .
  ?f1 nfo:hasHash ?h .
  ?f2 nfo:hasHash ?h .
  ?f1 nie:url ?u1 .
  ?f2 nie:url ?u2 .
  filter(?f1!=?f2) .
}

Quick explanation: the query does select all nexif:Photo resources which have the same hash value but are not the same. This of course can be tweaked by adding something like

?f1 nfo:fileName ?fn .
?f2 nfo:fileName ?fn .

to make sure that we only catch the ones that we downloaded more than once. Or we add

?f1 nie:contentCreated ?cc .
?f2 nie:contentCreated ?cc .

to ensure that the photo was actually taken at the same time – although I suppose the probability that two different photos have the same hash value is rather small.

Maybe one last little detail. In theory it would be more correct to do the following:

?f1 nfo:hasHash ?h1 .
?f2 nfo:hasHash ?h2 .
?h1 nfo:hashValue ?h .
?h2 nfo:hashValue ?h .

However, with the introduction of the Data Management Service in KDE 4.7 similar hash resources are merged into one. Thus, the slightly simpler query above. Still, to be sure to also properly handle pre-KDE-4.7 data the above addition might be prudent.

Of course this should be hidden in some application which does the work for you. The point is that Nepomuk has a lot of power that only reveals itself at second glance. :)

Ever since the unicode bug in Virtuoso 6.1.3 many of us have broken unicode strings in our Nepomuk databases. Completely re-creating the database is IMHO not an option since that would mean loosing all manual annotations and things like download source URLs. One solution would be restoring a backup but I simply do not trust the Nepomuk backup until I had a deeper look into it. The perfect solution would be if Nepomuk could simply fix the data automatically. While that is of course my goal and I am looking into that it will take a while.

In the meantime I threw together a small desktop file which adds two new actions to the context menu of folders.

1. (Re-)index Folder contents will make the indexer update all the files in the folder indifferent of their state in Nepomuk. This includes fixed unicode strings.
2. (Re-)index Folder contents recursive does the same as the above except that it also recurses into sub folders.

Simply put the following into a file called “nepomuk-index-folder.desktop” and save it in “~/.kde/share/kde4/services/ServiceMenus”. At the next start of Dolphin or Konqueror the two new actions will be available.

[Desktop Entry]
Type=Service
X-KDE-ServiceTypes=KonqPopupMenu/Plugin,inode/directory
Actions=indexFolder;indexFolderRecursive;
X-KDE-Submenu=Desktop Search
Icon=nepomuk

[Desktop Action indexFolder]
Name=(Re-)index Folder contents
Icon=nepomuk
Exec=qdbus org.kde.nepomuk.services.nepomukfileindexer /nepomukfileindexer org.kde.nepomuk.FileIndexer.indexFolder %f 0 1

[Desktop Action indexFolderRecursive]
Name=(Re-)index Folder contents recursive
Icon=nepomuk
Exec=qdbus org.kde.nepomuk.services.nepomukfileindexer /nepomukfileindexer org.kde.nepomuk.FileIndexer.indexFolder %f 1 1

Update: The code above does only work for KDE 4.8 since we renamed the “strigi service” to “file indexing service”. So in order to make this work in KDE 4.7 and before replace “nepomukfileindexer” with “nepomukstrigiservice” and “FileIndexer” with “Strigi”.

C’est par un article de la Vie que ma femme m’a appris la nouvelle: Montserrat Figueras nous a quitté le 23 Novembre dernier.

Mes pensées sont tout de suite allées vers Jordi Savall, son époux et compagnon musical depuis le début de leur carrière commune en 1967. 44 années d’un parcours fabuleux, jalonné de disques tous plus magnifiques les uns que les autres, et de concerts où la beauté était en permanence conviée, avec cette générosité de l’âme qui faisait de chacun de ses moments partagés une fête absolue des sens.

J’ai découvert les deux artistes avec leur disque consacré au Llivre Vermell … de Montserrat ! Et depuis ce jour, je n’ai cessé d’être en admiration devant la voix de Montserrat Figueras et le son de la viole de Jordi Savall. J’ai acheté je crois la totalité de leur discographie au fur et à mesure de leurs parutions. Faible mais sincère moyen de soutenir leurs activités, surtout depuis la création de leur label Alia Vox.

Et puis, bien évidemment, j’ai assisté à des concerts, les Prohéties/Chants de la Sybille, où la voix de Montserrat ne sera jamais égalée, pour sûr. Mais aussi les cancioneros et villancicos de musiciens espagnols qu’ils affectionnent tant et ont fait découvrir à tant d’auditeur et immortalisé au disque également. Allez entendre Ay Luna par Montserrat pour comprendre combien cette voix colle de façon incomparable à la musicalité du texte et des mots. Mais elle état aussi fantastique dans son disque consacré à Merula où l’association de son timbre avec celui du cornet de Jean-Pierre Canihac donne le frisson. Et ces concerts à Beaune, où tous les répertoires furent abordés, de Flecha à Charpentier, de Monteverdi à Victoria. D’inoubliables parcelles d’Art absolu.

J’ai même eu la chance en 1995 de participer à un de leurs stages, en Espagne, et leur simplicité, leur bonté, autant que leurs qualités musicales, ont fait de cette semaine passée près d’eux un souvenir impérissable pour moi, qui n’a pu que me faire regretter d’être informaticien et ne pas pouvoir partager la complicité musicale de tels artistes plus souvent.

Alors ce soir, pour conjurer le spleen qui guette, j’écoute Jérusalem, que je n’avais pas encore mis sur ma platine, pour me réchauffer une fois encore au son si suave et doux de la voix de Montserrat Figueras, entre autres. Il est triste de savoir que plus jamais je ne pourrai l’entendre en concert, là où l’émotion véhiculée est à son comble, ni lui parler à la fin du concert pour encore la féliciter du bonheur qu’elle nous avait apporté comme j’ai eu la chance de le faire.

Mais c’est ici que le disque joue son vrai rôle: pas celui de promotion, Montserrat Figueras, pas plus que Jordi Savall n’en ont besoin. Mais celui de conserver pour le futur une trace de l’art inégalable des meilleurs d’entre les musiciens. Ce pour quoi il a toujours été conçu. Et ce qu’ils avaient très bien compris en formant leur propre maison, pour être libre au maximum de leurs choix, et produire des objets de mémoire et de culture. Bien plus que des disques. Des images de souvenirs partagés et vécus que nous transmettront nous mêmes à nos enfants en leur rappelant le rôle primordial que ce couple a eu dans la renaissance d’un très large pan du patrimoine sonore de plusieurs peuples du pourtour de la Méditerranée.

Puisse la musique continuer à accompagner Jordi, Arianna et Ferran, leurs enfants, et adoucir la peine qu’ils éprouvent. Qu’ils continuent leur chemin en portant l’esprit d’amour qui animait Montserrat Figueras et en sachant que de très nombreux mélomanes sont près d’eux dans cette épreuve . Comme moi, ils écoutent sa voix et l’admireront pour l’éternité.

It happened. Alf Rustad donated the missing 356€ which broken the magical barrier of 9000€ in the Nepomuk Fundraiser I started nearly three months ago.

While the actual goal – securing long-term funding for Nepomuk – has not been reached yet this is a great opportunity to thank Alexander, Alvar, Andreas, Andre, Andrew, Angelo, Anton, Antonio-J, Ardy123, arkub, Baltasar, Bernd, Bernhard, Calogero, Carl, Ceferino, Christopher, Christoph, Claude, Cristiano, Daniel, David, dunkelschorsch, Eduard, Efthymia, Elias, the two Enriques, Fabio, Felix, Florian, Francisco, Friedhelm, Fux, Gael, Giacomo, Giorgio, Guillaume, Günter, Hans, Han, Hartmut, Hector, Hendy, Huftis, Jaroslav, Jérôme, Jesus, Josep, Jos, Jramskov, Juan, Juanjo, Junichi, Kai, Kenneth, Kevin, Kilian, Kulomi, Leopoldo, Linopolus, Luca, Luis, Luiz, Maik, Manoel, Manuel, Marco, Marc, the three Markusses and Martins, Maxime, Mguel, the two Michaels, Mikael, Mike, Morgan, Nicolas, Olaf, Olivier, Orestes, the two Pauls, Paulo, the two Peters, Philipp, Pierre-Hugues, Régis, Robert and Robert, Rodrigo, samtuke, the Sebastians, Simone, Sören, Stefano, Steffen, Stian, tanghus, Thiago, Thomas, Thomas, and Thomas, Tiago, Timothy, Tommi, Tuukka, Ulrich, Wakeley, Xavier, Yaroslav, and all the anonymous doners for their support. You have given me time to keep looking.

A special thanks goes to Carl Symons for his great dot article, his many tips and continuous encouragement.

Thank you also to Peter, George, Ivan, Vishesh, Christian, Andrew, Martin, and Laura for their great developer comments on Nepomuk.

And last but not least thanks for all the positive feedback on my blog articles, the translations into strange and exotic languages such as spanish :P and all the encouraging words which showed how many actually get what the semantic desktop is all about and want Nepomuk to go on and change the way we work with information today.

Development goes on, on all the fronts, and time has come for some news about Intel Linux Graphics project.

For the Kernel side, some nice patch series have arrived to the list:

• Daniel Vetter sent his PPGTT enabling patches, which resulted in many interesting reviews and discussions. PPGTT, or Per-Process Graphics Translation Table, is a mechanism for remapping GPU memory into system one. Unlike traditional Graphics Translation Table, which has a global mapping for everything, PPGTT allows each process to have its own level of mapping. On practice, it should improve stability by a large margin and performance by a considerable value, and also is a nice thing to have in general, specially when hardware supports it natively (which it does, starting with Sandy Bridge). Also, if you are interested in learning the details of how GTT and GPU memory management works, you should check this excellent article from 2007 for a great introduction.
• Ben Widawsky has sent a new series of patches for forced GPU throttling and scheduling. I already described them in one of the previous posts, and I am really interested in seeing them accepted to the kernel.
• I’ve sent out some patches for userspace-controled RC6 enabling and tweaking, together with some patches for enabling semaphores and rc6 by default on newer generations of gfx hardware. Those patches, together with Ben’s ones, also raised an interesting point – we have many userspace-controllable items in the debugfs file system, which should probably belong to sysfs instead. This would require a proper definition of their usage and behavior before real userspace applications would be able to use them.
• And Wu Fengguang sent some patches for Display Port and HDMI fixes.

Moving up the stack to the 3D driver, some major news worth highlighting are:

• On Mesa side, the major news is Ian has released Mesa 7.11.2 with some additional GLES and EGL patches, and a small patch which fixed Mesa build on Mandriva (and also on other distributions which use -Werror by default, such as Mageia for example).
• Lots of work is happening in Mesa, targeting OpenGL 3.0 support by the end of the year. GLSL 1.30 is already among us, and most of GL 3.0-required extensions are in place, but there are many things to do. Hopefully they will be all done in the next few weeks.
• Speaking on Mesa, some major news happened for the drivers using the Gallium technology. As you probably heard through Phoronix already, the i965g, Cell and Failover gallium-based drivers were dropped from Mesa due to lack of love, support and care. Sad – but this is life. And in any case, i915g driver is still there, and of course, those events do not affect Intel’s 915 and 965 Mesa drivers at all.
• And also on Mesa, Ian has started the work on GLX_ARB_create_context and the layered GLX_ARB_create_context_profile extension, and came with a question whether it is still worthy to support non-XCB protocol, or if there are any platforms that can’t / don’t use XCB for X protocol yet. The overall feedback so far was to drop non-XCB bits sooner than later. So looks like XCB has won the X protocol war in the end .

And finally, on the xserver side, discussions were raised on the mailing list on the release dates for the next xserver release, and the Xinput 2.2 state in it. It is almost there, and will probably get merged until Christmas. Also, Xorg-server 1.11.2.901 (a.k.a., 1.11.3-RC2) was released.

The 2011 repository has been signed with the official Mandriva key (main and contrib media). This means there is no more cooker key in the 2011 repository.

This change will do a major update of all mirrors, because signed RPM are not the same, so they will be re-synced on all mirrors.

People should now import the official pubkey. They can remove the cooker one if they want, but installed RPMs on the system will stay signed with the cooker key.

You can use any 2011 repository to import the key:

rpm --import ftp://ftp.proxad.net/pub/Distributions_Linux/MandrivaLinux/official/2011/i586/media/media_info/pubkey_contrib
rpm --import ftp://ftp.proxad.net/pub/Distributions_Linux/MandrivaLinux/official/2011/i586/media/media_info/pubkey_main

To remove the cooker key:

rpm -e gpg-pubkey-22458a98-3969e7de

Moreover following the Oden’s recommendation libreoffice from media/testing has been moved to main/release, to fix #64224 and #64246. QA process has been done following http://wiki.mandriva.com/en/OOo_Quality_Assurance

Hello everyone!

Today ROSA team is glad to announce the our new product ROSA Media Player 1.0 Beta. What is the ROSA Media Player? 

ROSA Media Player (ROMP) is a new media player for the ROSA/Mandriva. Technically it is based on MPLayer and SMPlayer code, but considerably differs from the SMPlayer (yes, ROMP is fork of the SMPlayer). What for? The main reasons that made us to fork SMPlayer are:

• possibility to allow our users to just see a video without taking the trouble with many features (yes, we don’t want to refuse the our “pure choice” concept);
• impossibility to put our code to upstream project because of different ideology;
• modern look and best usability; 

Just see the screenshots: 

Pic.1: ROSA Media Player is playing video

Pic.2: ROSA Media Player cutting video

Pic.3: ROSA Media Player Plugin allows to show streaming video

As you can see, we made not only the player itself, we also made the ROSA Media Player Plugin (ROMPP) - plugin for the Firefox/Chromium web-browser to show the streaming video. The ROMPP is created from scratch. 

Some words about stability. Because ROMP is based on the SMPlayer it has a good stability and word “beta” in the title speaks more exactly about new features and usability that we want to do before we will release the ROMP 1.0 final. So, you can use it without a fear that it will be unstable.  

If we speak about our further plans, I can mark out several important (in my opinion) features:

• possibility to record video from the screen (like recordMyDesktop);
• possibility to strip the video by size;
• possibility to connect, download and play a video from some video services like Youtube (as in Miro);
• New interface (maybe on the QT Quick);

And this is not all! Today we want to ask you, our users: what features do you want to see in the video player? How do you see your best video player? Please, feel free to send me emails or comment on this blog post with any your idea. We don’t promise that we’ll implement all of your ideas, even right now, but you can be sure - we will consider any of your ideas!

P.S. The packages (rosa-media-player and rosa-media-player-plugin) already in the Mandriva Cooker and Mandriva/ROSA 2011 ‘main/backports’ repositories. 

UPD: The source code for the ROMP is here and for the ROMPP here.