There has been a lot of chatter lately regarding Amarok’s switch to MySQL as its only SQL backend. A decent amount is FUD — either by people simply pushing back against change, or by people that simply don’t understand the decision. Some of it (particularly Adriaan’s blog post) has been insightful and interesting, but miss the mark in terms of why this change was made. This post attempts to explain why this decision was made, what it really means for you the end-user, and why you should have a cup of tea and relax.

我想首先指出的是,我说过MySQL将会成为Amarok的唯一SQL后端。Amarok2的收藏系统很强大。看看有多少各种各样的音乐源吧,来自Shoutcast的,Jamendo的,Magnatune的,Ampache的,MP3Tunes的,还有来自本地的,比如来自iPod和本地的文件系统,它们在Amarok2当中都被同等对待了。Amarok1的收藏仅仅是个收藏,并且被它自己所声明的功能限制了(当然,他能提供它自己的特定功能)。虽然我不这么想,但是同时还有一个基于Nepomuk的收藏选项没有被启用。(注:这个i don’t think看的我一头雾水)所以振作一点,这个变化只是影响Amarok的内部SQL收藏,但不包括其他来源(尽管如果那些来源乐意,他们也可以把信息存储在SQL数据库中)。

I want to point out first that I said that MySQL is going to be Amarok’s only SQL backend. A2’s collection system is very powerful. Just take a look at how varied music sources from Shoutcast, Jamendo, Magnatune, Ampache, MP3Tunes, as well as local sources like iPods and your local file system, are treated as equals in A2. A collection is a collection, and is limited only by what capabilities it advertises it can support (and of course, it can supply its own custom capabilities). It’s not currently enabled, I don’t think, but there’s a Nepomuk-based collection option too. So take heart — this change only affects Amarok’s internal SQL collection, and not other sources (although those sources can store information in the SQL database if they wish to cache information).

由于我提到了Nepomuk,我们也来谈谈另一个普遍的问题/需求/抱怨:KDE已经有了很好的Strigi-Nepomuk在工作,为什么我们不用它来搜索音乐并且存储信息?这有很多原因。第一个原因是Strigi和Nepomuk是可选的,并非必要的。(更新:Strigi是必要的,但Soprano不是,所以Nepomuk从整体上来说还是可选的。)我们不能指望用户安装了这些,而且就算他们安装了,我们不能指望用户正确地配置了他们(注意由于我们的计划是跨平台,这使得这件事更不大可能了)。第二个原因是速度:Amarok的自定义收藏扫描的速度非常快,并且利用taglib收集特定的信息。和Strigi比较起来,Strigi的速度就太慢了(它需要计算所有文件的哈希值,这意味着它将会读取整个文件),并且取得较少的信息。(更新:照Strigi开发者的说法,而不是在, 维基百科,甚至在作者自己主页上的描述,Strigi默认并不计算文件的哈希值。这使得对于Strigi来说,如果配置得当的话,和Amarok的内置扫描相比能够一样快。尽管我不知道它是否将所有需要的信息都提取出来。但如果它配置成需要计算所有文件的SHA1哈希值,那么它肯定会非常慢。)在本地硬盘上,这并不是一个大问题,但是当你将基于网络的存储纳入考虑范围内的时候,这就是个大问题了,而且这是一个非常普通的场景。同时尽管我不记得细节了,但我也听说,和一般的SQL数据库相比,对Nepomuk的查询是很慢的。同时记得当基于Nepomuk的收藏完成的时候,来源于基于Nepomuk的收藏将会把它们的元数据修改存回Nepomuk当中。所以SQL收藏并不是Nepomuk的替代——它们是完全独立的。(更新:我忘记提到基于Nepomuk的收藏已经存在了。它已经由GSoCer在暑假开发完成。我不确定它的状态是否可以赶上2.0的发布,但是我们这些Amaroker都很喜欢Strigi/Nepomuk,并且对于打开应用程序之后所有音乐都正确可用,并且不需要任何预先配置这个点子感到兴奋。但是SQL收藏也确实占有一席之地。正如我说的:它们都是很好的技术。)

Since I mentioned Nepomuk, it’s time to discuss another common question/demand/complaint: KDE has this nice Strigi-Nepomuk thing going on…why aren’t we using it for scanning music and storing information? There are a couple main reasons. The first is that Strigi and Nepomuk are optional, not required. (Update: Strigi is required, but Soprano isn’t, so Nepomuk as a whole is still optional.) We can’t rely on the user installing them, and even if they are installed, we can’t rely on the user to configure them properly (remember that we’re going cross-platform, making it even less likely). The second reason is speed: Amarok’s custom collection scanner is extremely fast and pulls out specific pieces of information with TagLib. Strigi is, by comparison, very slow (it calculates hashes of all files, which means it needs to read the entire file) and pulls out less information. (Update: According to the Strigi developer, and despite what is said on, Wikipedia, and even the author’s own home page, it does not calculate hashes by default. So it’s possible that Strigi, if properly configured, could be as fast as Amarok’s internal scanner, although whether it would pull out all necessary information, I don’t know. If it’s configured to calculate SHA1 hashes of all files, then it will indeed be far slower.) On a local hard drive, it may not be a big issue, but it sure is a huge issue when you throw networked storage into the picture, which is a very common scenario. I’ve also heard, though don’t remember specifics, that querying and such through Nepomuk is rather slow, compared to a normal SQL database.  Regardless, though, remember that when the Nepomuk-based collection is finished, tracks sourced through a Nepomuk-based collection will have their metadata changes saved back to Nepomuk. So, it’s not that the SQL collection is in place of Nepomuk — they are entirely independent. (Update: I forgot to mention that a Nepomuk collection already exists. It was developed by a GSoCer over the summer. I’m not sure what its status is as far as making the 2.0 release, but we Amarokers both like Strigi/Nepomuk and are excited about the idea of opening up the app and having all your music available right then and there with no pre-configuration. But there is a place for the SQL collection too. As I said: they are complimentary technologies.)


With those topics out of the way, on to the meat.


First, it is important to understand an important pair of facts. Number one: we are not database guys. Sure, we can store data in them, and more or less come up with a working schema, but none of us are gurus/wizards/jedis/etc. This leads in to number two: maintaining three databases was driving us crazy. Every time a minor schema change was needed, it had to be coded up for all three types of databases. Modifying a schema could be trivial for one database type, and super difficult (or impossible) for another. People would report bugs that we couldn’t reproduce, only to find out that it was because we didn’t quite understand how one database or another behaved (or in some cases, none of the active devs were using that type). And so on. So from the beginning of A2 development (and in our fantasies during A1 development) we knew we wanted just one database.


(We did actually look at abstraction layers like QtSQL and others. I’m not going to comment on them much, as I didn’t do the evaluation, but in general they were found to not be flexible enough to handle all of our needs without doing some custom SQL coding (especially in the cases of things like schema changes), which kind of defeats the point. If you want to know more/want to insist that they are, try asking eean, as I think he did the evaluations.)


Now we had to choose the type. At first, SQLite seemed like a good choice. Using transactions, it’s decently fast. It’s pretty stable (those that complain about odd MySQL bugs should talk to markey, as he, being the SQLite maintainer in 1.4, can attest that SQLite’s had its fair share). However, there were a few problems that in the end knocked it out of the running. The first problem is performance. Although for people with small collections it performs fairly well, people with large collections that switched to the MySQL or PostgreSQL backends in A1 would report enormous speed gains when operations performing complex or many queries were performed, such as adding many entries to the playlist, scanning files, or filtering/searching in the collection. Since we want to accommodate users with large collections just as well as those with smaller collections, and since digital music collections aren’t getting smaller, the speed increase for our users with large collections was quite important. Many of our developers, after the switch to mysqle (as we call it, though that’s not the official name), have noticed huge speed increases in their day-to-day use of A2, so that speed increase is carrying through to the embedded server as well as the normal server. That was the first knock against SQLite.


The other blow for SQLite came for a totally different reason. Many users (myself included) have multiple computers sharing a single Amarok database. Assuming all the computers have access to the music at the same mount point (and a few other things are configured right), this allows you to scan once, play everywhere, update the same ratings no matter where you play it, and more. Even if your aren’t sharing the database among multiple computers, many users want their database stored on a particular server for speed, security, or backup reasons. If you think either of these isn’t a common use-case, you’d be quite wrong. MySQL and PostrgreSQL were quite happy with this workload. It’s a total no-go for SQLite, simply because it’s designed for a different purpose. So SQLite had two big knocks against it. K.O.


However, just as we can’t rely on the user to set up Strigi/Nepomuk correctly, we can’t rely on them to get their tables set up in MySQL or PostgreSQL. So we needed the database to be embeddable, so that it could just work for the user without any setup necessary on their part. MySQL, with libmysqld, had the seeds of this in the 4.1 series, it works decently in 5.0, and it’s becoming fully supported (AFAIK) in 5.1. PostgreSQL, on the other hand, does not have any such thing. (They have an interesting and cool concept of their own of embedded SQL though. Update: apparently that is part of the SQL standard. Still pretty cool. Still totally different from what we mean when we are talking about an embedded server.)


So this leaves us with — as you guessed — MySQL. It may not be any particular person’s favorite database (although it is for plenty), and I don’t know how much overhead it really has in embedded form, but it fit the bill. It’s both embeddable and can run standalone on the local or a separate machine (yes, this is not supported yet in A2, but it will be). It is fast and robust for large collections. It is well understood by the development team. And most of all, it is a single-backend solution that fills all of our needs.


If you’re still unhappy about our decision, I’m sorry. We try to please most and can’t please everyone. But we’re the ones that develop and support this thing, and so we made a decision based both upon our needs as developers and the real-world use-cases from the collective feedback of thousands of users that have contacted us over the last few years. Please remember that even if most of the comments on the Dot, or to this post, (i.e. much of the sudden visible feedback) are from people that are unhappy with our decision, it is a decision that will actually suit the vast, vast majority of our users better than the other options we currently have.


We’re a project that is known for being good to our users — we listen to them, we try to implement features they want, try to be responsive with support. It’s one of the things that got us where we are today. So please, dear readers — put some faith in us. This has not been an easy decision — we’ve discussed, we’ve argued, we’ve thrown things, we’ve made up, we’ve had an after-the-make-up orgy or two — but in the end it’s what we collectively felt was the right way to go, and we feel that, in the long run, it will make Amarok even mores awesomer. Hopefully you’ll feel that way too.

This entry was posted in KDE and tagged , . Bookmark the permalink.

12 Responses to Amarok2中的MySQL-真相(渣翻译)

  1. 右京样一 says:
    Internet Explorer 8.0 Windows XP




  2. csslayer says:
    Firefox 4.0b3 Windows 7

    说到nihui大神,nihui竟然在MagicLinux里面的kde drop掉Nepomuk……这是我没有想到的……这导致我恐怕永远都不会去试试MagicLinux……

  3. 右京样一 says:
    Internet Explorer 8.0 Windows XP

    国外每秒几兆十几兆乃至几十兆的网速,云计算大概是够了。像我这只体验过512k DSL的人么……通常能用本地就不用网络……

  4. csslayer says:
    Google Chrome 5.0.375.127 Windows 7

    关于XIM不改真没辙。谁让X Server下面的输入法就这破德行呢。。ubuntu只不过提供了这么一个工具,用于帮你配置那些个环境变量。所以只能说多多推广这东西吧,顺便做出个图形化的界面来。(fedora貌似就有图形化的……)

  5. 右京样一 says:
    Internet Explorer 8.0 Windows XP


  6. csslayer says:
    Firefox 4.0b4 Windows 7

    sunpinyin又不是没有linux版……况且open solaris这是要死啊。外壳不外壳什么的,scim和ibus不知道是多少输入法外壳了……况且fcitx的优点又不在于此。另外老输入法并不会扔掉。


  7. 右京样一 says:
    Google Chrome 5.0.375.127 Windows XP

    外壳这东西,SCIM和ibus的定位就是输入法外壳,并没有附带官方的“输入法”(好像是吧,至少我没听过);FCITX是个“Input Toy”。不过原来的算法不丢掉太好了。

  8. csslayer says:
    Firefox 4.0b4 Windows 7


    整句输入法是算法问题,嘛,和数学也有关系。sunpinyin用的算法起码在基础上和sogou,google pinyin是同出一门,都是通过建立语言模型。fcitx现有算法打比方相当于落后一个时代,也就是和智能abc一个级别吧。

    老实讲,我还想说现在真的还有人管kimpanel吗?所以你看aur上的kimpanel-plasmoid-svn是有几个patch的,能解决几个还比较严重的问题……我也找不着人,report bug给kde也没人鸟我。看看这里,你会发现,真的,没人维护(那些都是老外,估计仅仅是修改编译错误的)。


  9. 右京样一 says:
    Google Chrome 5.0.375.127 GNU/Linux


  10. nihui says:
    Firefox 3.6.4 Fedora

    其实,我也是默认设置控,程度和 csslayer 差不多的,MagicLinux kde 的所有包都没有自带额外的配置文件,去掉 nepomuk 是因为发行单张 CD 的原因,需要为放其它更有用的软件腾出空间,另外就是我的测试机器只有 200MB 的内存,性能上的微小差异在我这边会被放得很大,mysql 也是。
    我如今越来越不喜欢用 amarok 了,juk 更容易一些。
    新背景真的很好看 ^^:)

  11. 右京样一 says:
    Google Chrome 5.0.375.127 GNU/Linux


  12. csslayer says:
    Firefox 4.0b4 Windows 7




Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.