Archive for the ‘Summer of Code 2009’ Category
GSoC 2009: wrap up
Since last monday, the 2009 edition of the Summer of Code is over, so it’s time for a summary of the work of my project, what has been done, what has not been done, and what I am doing right now; while reading this, keep in mind the picture you see above, taken from one of the most recent code snapshots of the smartsave dialog system.
The project: a little reminder
My project for the GSoC had the aim of developing an alternative semantic load an save dialog system, through which the user would be able to save meta-informations and interrelations about its files, like associating the current context or the current project, or the pdf version of an OpenDocument file; during the opening operation, a new view of the file system would be produced, so the user should navigate and filter through the previously created metadata, narrowing the possible range of files until it finds out what he/she is looking for.
Why is this important/useful?
This is a question that I often hear, especially from those people who think that a semantic desktop engine is not different from, for example, Google Desktop Search, or similar file crawlers; well, a semantic engine is very different from a crawler: the latter just passively crawls through tons of files, extracting text and allowing a search through keywords, while the former can combine its informations, using mathematical logic rules in a process called reasoning, for expanding the metadata associated to a document and allowing a user to search it through paths he/she hadn’t thought about when saving it.
The semantic desktop is not just this, and a very good introduction to it can be found here: it is worth reading it, believe me.
The new dialog system jumps straight into this view: saving files in the current way is somehow old, and it requires the user to create a reasonable directory path, so he/she will be able to find its file back someday, when he/she will need it; in my view, it should be very useful if we are able to associate to a document some informations, about its contents or about the contest in which we have created it, without bothering with a file system structure: wouldn’t it be easier to search for it later?
What has been done in these three months
Ok, so what do we have now? During this GSoC, I have created a new UI, similar in certain aspects but different in others to the current one; an interface to some plugins which are able to analyze file content and extract possible metadata, also querying some online services like OpenCalais; the user is also able to associate personalized metadata.
The open dialog presents a full list of files, comprehensive of those indexed by Strigi, with a list of filters for narrowing the search; the filter system is very similar to the faceted browsing system which you can find on many Web sites.
Currently, files are saved in ~/Nepomuk, with a file path that can be explored also through the standard file manager; the name and the path automatically given by the dialog backend is not *that* beautiful for now, but there are some development already planned.
What has not been done
Reading from the original application, there is just the optional goal which has not been developed, and it consisted in a new view also for the file manager, allowing a file browsing similar to the open dialog one; unfortunately, time has not been enough to develop it too, but other developers are also working on a better file search UI for Dolphin, so there will surely be news in this field in the next weeks.
What is developed now (and where is the code)
The code for this dialog system is in playground, in the Nepomuk-KDE section, but currently it cannot be used as is
But there is a good reason for this: I am currently working in a little hack of kdelibs, for having the dialog shown as an alternative in all the KDE programs (like Okular in the screenshot, or KOffice in a previous post), so the code is now in an intermediate form and you will have to wait a little more to try it out.
This is currently the main development target, with also a couple of todo items still in my list; there will also be new code coming up for the elaboration of the “real” file path, and more plugins for extracting more and more metadata automatically from the document to be saved.
Wrapping up
Here are some credits: I have to thank Nicola Vitucci, a great friend who gave me some very good advices, especially when I wrote my application; Sebastian Trueg, my mentor, for having helped me along these months (and for still helping me
), and for having stood for my very basic questions on Qt and KDE libraries; Adam Kidder and its GSoC project about kio-slaves, which I am heavily using in the open dialog, and all those people who commented my posts and discussed their views in chat with me: your opinions and ideas has been very important, and I hope they will be of the same importance also in the next days that I will spend with KDE and its community (yes: I’m staying with you)!
So, stay tuned: some good things are coming out!
Disclaimer: the following guide may overwrite some of the files installed through the standard packages of your distribution, so it may be better if the work is done on a newly installed machine different from your production one, or in a different directory along with kdesupport, kdelibs and kdebase from SVN trunk. You have been warned…
As promised, while I’m currently tweaking a few things out in the dialogs, here there are the compilation instructions for trying my code. No .deb precompiled packages (yet?), a couple of passages were a little more complicated in relation to the time that I can give them, so you need to get a little dirty, but I’m sure that this is not a problem.
As a premise, I assume you are a little confident with compiling sources, and you know what cmake is; given that, here we go!
The following passages work for sure in Ubuntu Jaunty, updated to KDE 4.3 (oh, by the way: you need KDE 4.3 or better (that is: trunk) for all this to work), and I know this because my new shining laptop arrived last wednsday, so I did this to work on my code.
Prerequisites
You need at least kdelibs and kdebase packages (let me repeat: minimum version is 4.3, so if you are on Jaunty, you do need the backports repository and the upgrade operation, please refer to www.kubuntu.org for instructions), plus their development versions: kdelibs5-dev and kdebase-workspace-dev (pay attention: kdebase-dev is from 4.2.2); for any other missing library, cmake will tell you what you need.
Source code
Premise: if you don’t have the latest version of Strigi, you need to get it from SVN, too: for example, in Jaunty is 0.6.4 and in Karmic, at least a couple of days ago, was 0.6.5, but you need at least 0.6.95. So, if you need it, then get it:
svn co svn://anonsvn.kde.org/home/kde/trunk/kdesupport/strigi
cd strigi
mkdir build
cd build
cmake ..
make
(as root) make install
You can install it in /usr/local, but this thing is probably going to stop your strigi filesystem analysis: if you don’t care about it, then go and compile it; otherwise, I think you will need to install trunk in some different directory and work with it.
Then, you need to checkout from SVN Adam’s kioslave:
svn co svn://anonsvn.kde.org/home/kde/branches/work/soc-virtualfolders
cd soc-virtualfolders
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make
(as root) make install
As you may notice, you need to install that code in the same folder you have KDE, unfortunately it does not seem to work in /usr/local or similar folders (but any suggestion in this sense is accepted): as a consequence, it will overwrite the current kioslave from your packages. Again, if you don’t like it, then compile trunk in a directory different from the one of your standard installation.
After the last command, everything should work.
Now, let’s go with playground:
svn co svn://anonsvn.kde.org/home/kde/trunk/playground/base/nepomuk-kde
cd nepomuk-kde
mkdir build
cd build
cmake ..
Ok, now you have your building makefiles ready; usually I only compile what I really need from this code, also because there is the need to install the annotation plugins, because they are searched at runtime by the dialog system, so here it is the sequence of commands to compile and install them and their dependencies:
cd kcompleter
make
make install
cd ../pimo
make
make install
cd ../nepomukcompletion
make
make install
cd ../scribo
make
make install
cd ../annotationplugins
make
make install
cd ..
Ok, with this you should have installed everything you need (of course, if needed, install commands have to be launched as root); now we can build smartsave and test it:
cd smartsave
make
make install
The installation passage is optional, because it will not install the testing application.
If you want to try the dialog, enter the test directory and launch the texteditor app: it is just the same simple notepad from Techbase tutorials, but with my open and save dialogs instead of the standard KDE ones.
Troubleshooting
Some things can go wrong: the first one is that you see the standard dialogs in the test app, and if this happens, then you probably have shut down Nepomuk, please restart it from the system settings dialog.
The second one is that you don’t get any suggestion in the save suggestion box: if this is true, then you can try by restarting the kde daemon, by invoking kdeinit4 from a console, so that the plugins installed before are registered.
The third one is that you cannot see any file in the open dialog: if this happens, then you have to check that the kioslave has been correctly installed in your KDE main directory (/usr on a standard system, or something else if you work directly on trunk).
What now?
Now that you have a text editor with the dialogs, what can you do? Well, I’m going to prepare a screencast, probably in the following week, to show a couple of features; bear in mind that there are some tweaks on which I’m working on, so this is not a perfect system or a system ready to be shipped in a stable release (far from that
), so be patient and test, and feel free to report any kind of trouble you may find along the way.
GSoC status
Well, one week left: the dialog system surely has a base on which to work for both UI and some backend code; there are a couple of things that I’d like to work out before August the 17th; then I may try to add these dialogs to a real KDE application, instead of a simple test program, but we will see what will come up in the next few days…
Yes, as you can see, now the open dialog is working!
It took more time than expected, especially because of a couple of different approaches tried along the way (and some signal loops between models), but now a first basic version is there: as you can see, on the right there is a standard KDirModel, which is using Adam’s code, the other GSoCer of Nepomuk, on kio-slaves, and it shows files; on the left, there is the filter view.
The filter view is still a work in progress, but it is quite similar to the faceted browsing systems you can see in many Web sites: there are properties and their current value, taken from the files shown on the right side, and the user can check them and restrict those files (and consequently the filters themselves become less in number). Of course, their visualization is very very _very_ basic for now, but it will become better, don’t worry about it!
There have been also a couple of changes in the save dialog, especially in how the user can enter personalized informations, I don’t have a screenshot right here but a screencast will come, I promise, as soon as some things will be refined.
Now, about the roadmap: only three weeks of official GSoC are left (17th is the last day “on the job”), so for that date I need to do a couple of important refinements in the base code, and some good work will go in the UI part, filters as one of the priorities; anyway the base dialog system is there, and that’s the first important thing. So, stay tuned!
P.S.: some people asked about how to try the code: in short, you need to compile the virtualfolders branch and the nepomuk playground, but some things need to be installed to be used, so I will publish some detailed guide, at least as I compile and use it; I also thought about creating some debs and putting them on my account on Launchpad, if I can I will do it and tell you about it: they will be Ubuntu debs, because I use that distribution
Yes, I know, it’s been a while since my last post, and today, too, I cannot say so much about the code: the open dialog is on its way, and a good milestone in the project is just behind the corner, but it’s exam session here at Politecnico and it’s been hard to find some good time to code in the last two weeks…
Anyway, tomorrow I’ll have another exam, and the last of the summer session is waiting for me on the 20th, and then it will be only SoC! And I promise to achieve that milestone next week, so expect a screencast coming, stay tuned!
New screenshot, old screenshot: the dialog you can see here is almost the same as before, except that it now shows some recent annotations added to previously saved files, and a few suggestions have been loaded, thanks to the new shining Scribo plugin (by Sebastian Trueg) and to the OpenCalais public services (for those you will need an API key, after a free registration to their site, but anyway a nice popup will inform you of this, when you will try to use it for the first time).
In that particular scene, I was saving lyrics from the song “Ordinary people” by Neil Young, and OC pointed out a few related terms, like the Las Vegas city written in the text, and others.
Ok, not so much work (on my side) this week, but past weekend with another public election (and me working for it) and a presentation made on tuesday for a university course haven’t left so much time for working on this…
Anyway, I have also made a little roadmap from now to the end of the GSoC (the BIG deadline), so the next important step will be a working load dialog (besides some little work on a couple of pseudo-bugs), so there will be a starting point for thinking about interaction with the user and UI.
Last but not least, my Commander-In-Chief already blogged about the project on which Sebastian Faubel is working, and some ideas are very interesting, like for example a template-based filesystem tree for giving files a good place in which they can be saved (and browsed traditionally later).
So, stay tuned!
Does this remember you something? The answer, of course, is yes, and I can tell you what is: the draft I published here a few weeks ago. The difference? It’s that now that dialog is able to save files.
Ok, UI may suck, but frankly that was not the purpose, the purpose was to have a functioning backend, so the engine can (hopefully) produce some suggestions based on the text written in the notepad, the user himself can add new tags with a line edit field, and in the end, with a label and a comment, a text file will be saved into ~/Nepomuk/PlainTetxtDocument/label (so please, add a label). This directory is created and the full path is given to the caller application.
The first idea is to split files based on types defined in NFO ontology; things may get more complicated later on, but for these testing purposes that’s good enough. A doxygen configuration file is in place, and executing it you can get the API: for saving a file, you just need to pass a NFO url, as you can see in test 1 code, so the dialog itself will know what is the type of the document; later on, there will be the possibility to specify in a more detailed way a document subtype.
Next improvements: implementation details for suggestions and recent annotations, then a simple open dialog is the next step into development. The next weekend (usually my commit-and-publish moment) will be a little complicated because of elections, again, here in Italy, and I will be at the polling station (if wordreference.com does not lie, and this is the correct translation (it does not look as good as I thought)); anyway, stay tuned!
A brief note, just because lots of things changed during this weekend (this long weekend, two days of holidays here in Italy): I have dropped the metadata model that I started last week, because the annotation plugin already had one, and a couple of hours ago I also dropped the SuggestionProvider plugin system.
Recent annotations will be implemented through an AnnotationPlugin subclass, and related annotations will be provided with an utility class, probably later on.
Still working on save dialog UI, though, and I’m trying to add views in a KFileDialog subclass: task which seems not so easy (any advice?), some functioning code will shortly come, I promise!
The first official SoC week is gone! A bit of a hard work, but good results obtained…
With the last smartsave commit, all the stubs are done and real code is coming out: all the base classes are being populated, and in a few days some UI may come up.
I’m especially working in the save part of the system, still from the backend point of view, with models and signals that come and go through the classes (test 2 already shows something of this), updating views when new annotations come up. A hard link to the annotation framework (in playground) is coming out, and probably one of the two exposed classes will be collapsed to that framework, when my experiments and my messing up the API will stabilize a little.
The most difficult part, I must admit, has been to adapt to Qt frameworks (signal-slot, model-view, d-pointers), which are pretty different from my last C++ works or from the Java approach, so a big thank goes to Sebastian Trueg for his advices on them.
Besides, I hope not to flame in any way, but I really have some difficulties in understanding why the d-pointer idea… for example, I see the model-view as some kind of a derivative of MVC, and signal-slot as an alternative to a callback system; I see why calling getter methods with the variable name (C# does something similar), but why can’t I put my private members directly in the class? I read about the binary compatibility, but those are still private members, so the outside world cannot see them. So why complicate the classes structure with all those private classes with public members?
So, C++/Qt masters, I’d really like to read your opinion on that, not for flaming (I repeat), just for deeply understanding this way of programming
And, stay tuned for UI updates in the next few days!
This week has seen my first (and second, by the way) commit into KDE SVN! Another milestone reached
There are mostly stubs into the new smartsave/ folder, and a bit of documentation for the interfaces; there also is a test program, taken from one of Techbase tutorials: a very simple notepad, which will be the very first application to use the new dialog system, and it will be useful for trying functionalities.
No, don’t try it now: it just falls back to the standard KFileDialog class; let me write some real implementations before toying with it!
For now, the idea is to have an abstract class, Nepomuk::SuggestionProvider, which may be subclassed by any application (there’s, for now, just a method to be implemented), and an instance of it will be passed to the dialog UI for getting personalized suggestions, in form of Nepomuk::Annotation objects. These objects need to be created by the developer in the way he wants, by hand or using any Nepomuk::AnnotationPlugin, but things may change later on.
The second important class is Nepomuk::KSmartDialog (I called it ‘K’ just to mimic KFileDialog, but if you don’t like it, I can drop that ‘K’), which is a subclass of the actual file dialog, and which will provide a couple of static methods for getting or setting document metadata; of course, each of these methods will use the SuggestionProvider subclass as stated before.
These are the two important exported classes, at least today; expect some working code in the next days!
Oh, by the way: tomorrow is the official GSoC start! Godspeed to any GSoCer around PlanetKDE!





