wm161.net

Qt in College

Some might know I go to the University of Akron in Akron, Ohio. Its probably just me but it seems that the northeast part of ohio is a very Microsoft-favored region as far as software developers go.

However: My favorite professor, Dr. Collard, is teaching the “Introduction to Object Oriented Programming” (which should really be called Object Oriented Design and Analysis). I understand that sounds rather boring but the super cool part is the book and framework we’re using:

The full source of the book (which itself is open source) is available at http://cartan.cas.suffolk.edu/oopdocbook/.

Dr. Collard is pretty excited about it. He emailed the author to acquire the teaching materials, which led to us being a ‘beta test’ class that gets to try out the next pre-release version of the book, as soon as he gets the copy emailed to him.

Not sure if its a good thing or not, but he refuses to use the proper pronunciation of ‘cute’ “even if you write for KDE” because he refuses to call such a good framework ‘cute’.

The stage is set for domination

And that, my friends, is the last bit needed to get webcams into KDE. This morning around 3 AM or so I finished implementing the VideoDataOutput class in the gstreamer backend. It allows a phonon user to capture frames from a gstreamer stream. That picture was captured with the following code:

Once I test it against some code that doesn’t use video capture, it’ll be golden.

Webcams in Phonon: The GStreamer Edition

Hey, look, its a working webcam in Phonon. Again. My significant other is away for the week being a camp counsellor, so instead here is a picture of us drawn by an artist we met at Ohayocon earlier this year in Columbus, OH. I appear to be playing Cave Story, and she’s wearing her MOMO costume.

The first picture was much more exciting, I think. But this one still has potential. It was created using the GSOC project I previously discovered. The code is actually one line shorter:

QList devices = Phonon::BackendCapabilities::availableVideoCaptureDevices();
mediaObject->setCurrentSource(devices[0]);

Assuming that no super huge changes happen with the API, I plan to work on the Xine backend once I finish implementing the bits I left out from my code, such as device removal notification and proper device icons. My code is up for ridicule at Gitorious, under my personal repository. Make sure you check out the ‘webcam’ branch.

Before I go, a certain “Steve Schilz” has contacted me with an interest in writing support for the new capture API into the DirectShow backend. I gave him some pointers on where to start, so all those still using Windows can rejoice.

In which our hero discovers his work already done

Phonon already has video capture support. But nobody thought to tell kde-multimedia@lists.kde.org, #phonon, or the planet(s). I’m also slightly miffed that I wrote a proposal for the same exact thing and never got considered.

So karma decided to deliver my big box of homemade DDR machine fun from Newegg 2 days earlier. Such is life.

Compared to that merge request, my changes were miniscule, but I feel that I should still explain what I did, what I had planned, and how Casian’s changes fall (more or less) right in line. Hopefully my input will help.

The nucleus of my design was the CaptureDevice. I planned on adding a constructor to the MediaSource that would take this newly invented CaptureDevice class. CaptureDevices would be abstract devices and could represent any well-defined hardware-based source of media. Here’s what I wrote:

I wanted it to be super simple so anyone could just shout, “I need a webcam”, and *bam* you’d get the default webcam instead of the tuner card. Or even better, just ask for any kind of capture device and in 99.99% of user environments, you’d get something with an audio stream. The 0.01% is to account for users without any kind of sound card or sound capturing device. You’ll also note the special “Desktop” type for capturing your desktop. I think its safe to say that pretty much all platforms implement this in their media layer.

In addition, you could pass in some string that the platform could understand (eg. “/dev/video0″ or “plughw:0″). That way, you could grab a Device from Solid and pass that around.

Looking at his code, all that is already there. So the only real difference between my planned design and Casian’s was that there wouldn’t be a distinction between audio and video capture unless you absolutely needed it (such as for changing TV channels), which is what type() is for. The way I thought about it was that with a simple CaptureDevice to cover both cases, there’d only be one place to go for all your capture needs. In retrospect, splitting it up this way is a much better idea. It keeps us from trying to squash the unrelated functions of TV tuning and microphone boosting into one class.

Besides that one detail, his implementation is almost verbatim. I look forward to when the merge request gets accepted so I can jump in and get the gstreamer and xine engines to work with it. The code I wrote for my CaptureDevice is literally one line of original code: gst_element_factory_make("v4l2src", "v4l2src");.

So congratulations and many thanks to Casian for doing what I was supposed to be doing for a while :P. Please direct your thanks to him instead of the comments.

Live: From Phonon!

Hey, remember me? Yeah, I was supposed to get webcams into Phonon. What was that, months ago? As everyone knows, life gets in the way at times. But the last week or so, things have settled down. In the last three days, I sat down and re-learned everything I forgot about phonon. Then I learned a little bit about gstreamer.

So, cue the drumroll.

Ladies and gentlemen.

I present to you, for your viewing pleasure, the first screenshot of a webcam being used with phonon:

ta da!

That would be Melanie, my girlfriend-turned-test-subject.

Its not much, but thats because it is only 14 lines of code.

Here’s the code, in as few lines as needed:

The other lines not there create a mainwindow, a video widget, etc.

The code in phonon though, is a little ugly. It’ll need cleaned up slightly before I can push out a commit to the git repo. Its also 2 AM, so I’ll post more details about how I’m planning on designing the system tomorrow.

Of course, I could be entirely dancing all over someone else’s toes. I had to drop out of the loop for a good while, but from what I understand, nobody has attempted tackling this since I last tried. Ping me if I’m wrong :).

Systemsettings Reorganization: Part II

If you didn’t glean it from my call for icons, then here’s what you missed: systemsettings needs reorganizing and some better icons.

Dario Andres and Ben Cooksley have a proposal up on TechBase that goes reasonably in-depth to explore a better organization of systemsettings. Today, I spent time with a friend of mine who isn’t a power user, but an ‘average’ KDE user. I sat down with her over the period of a few hours and came up with some interesting notes while I asked her to do various tasks, comment on why she thought certain things would be in places, and generally go through every icon in the 4.4 systemsettings layout.

The way I went through things was I asked her to perform some task, and I asked questions to explain her thoughts when she seemed confused, made a quick decision, or the wrong choice. Here’s what we came up with, roughly translated from my notes:

Task: Change the screensaver

Her first move was to jump to the appearance icon and poke around in there. The words “Splash Screen” confused her for a few seconds before she went back. After scanning the icons, she jumped over to Display and was thoroughly confused to not find it there. The reasoning behind the choice was her background of growing up using Windows. Eventually she found it after I told her to check out the Desktop group. After she found it, she commented that there was a bit of confusion between your display and your appearance. I asked what she’d think if moving the display icon to a new Hardware group would do, and she’d immediately know that the screensaver wasn’t in there.

Task: Change the battery level at which your laptop goes to sleep

Initially, she thought it might be under Desktop. It was the only one remotely related. I pointed out that there was an Advanced tab and she immediately found it there, commenting that she never had a clue that the tabs existed.

Task: Calibrate the new joystick you just plugged in

She was totally unable to find it. This is reasonable considering that it was buried under “Keyboard & Mouse”. I pulled up the screenshot from the reviewboard ticket and asked where she might find it then. In the screenshot, Keyboard & Mouse was renamed to Input Devices, which she said was a dead giveaway that the joystick would be in there. We both agreed that it needed a better icon, since the keyboard is but one of many devices.

Task: Configure the list of actions that show up when a storage device is plugged in

Her first choice was the “Default Applications” icon, explaining that it made sense you’d want to specify the default actions for a device. However, she couldn’t find it ’till I reminded her of the Advanced tab. Still in ‘find the icon’ mode, she scanned the list and went to Autostart first. I’d like to blame a poor explanation of the task for that one, but her claim was that the icon (a gear with a play button) implied some kind of connection between ‘do something’ and ‘hardware’. From there she jumped right at the Removable Devices (device-automounter) text. Going back, she picked the other ‘device’ text and found it.

Here, she took a break and asked why some of these things weren’t filed under the ‘Personal’ category. “I mean, they’re all about configuring my personal settings, right?” She was making references to the appearance, desktop, and display icons.

Task: In KDE, you can configure the system to send certain sounds to specific devices, such as notifications and voice chat to your headphones, but pipe your music to your speakers. Where do you change that?

Phonon’s speaker icon was a dead giveaway. Instant success.

Task: When you double-click on a window’s titlebar, you can have it either ’shade’ up into a little bar, or maximize the window. Change this.

She saw the text ‘Window Behavior’ and leapt for it. Again was ‘window behavior’, followed by ‘titlebar actions’.

Task: Change the keyboard shortcuts.

At first, she went for the Input Actions tile, explaining that it simply made sense for input to generate actions, and that the icon was a keyboard. Going back, she went for Keyboard & Mouse and found it.

Task: Configure your mouse gestures.

She was in disbelief that KDE had mouse gestures, but forged on anyways. She remembered which icon it was (Input Actions) and commented that it *really* should be called ‘Mouse Gestures’ and be put in with Keyboard & Mouse.

Task: Change the sounds Konversation plays when someone mentions your name.

She “wasn’t aware that the OS could let you do that through systemsettings”, and first insisted that she open up konversation. I changed the task to ‘make the windows make sounds when they minimize’, since that seemed a bit less tied to any one application. She went to the Notifications icon, clicked the dropdown and stared at it. Not seeing ‘window’ and not noticing the scrollbar, she closed it and assumed there was some other place, like maybe window behavior. I asked her about her choices here and she mentioned that the tiny dropdown was a real bad choice because the list was far too huge for the box. My suggestion here is that we change the tiny dropdown to a much bigger list view and add a search box.

That was the end of the list of tasks I had planned, so we moved on to looking at all the icons under the “Advanced” tab. I feel that there are some icons in there that simply don’t belong.

The first icon we looked at was the Login Manager one for KDM. She was confused by the word ‘login’, thinking that it meant you would configure your account from there. There really wasn’t a clear distinction between Login Manager and User manager, even considering the icons.

Next, we looked at the service manager entry. She thought it would be related to the services you’d find in /etc/init.d. Since it was under ‘advanced’, the term implied that they were related to the system and not the desktop. This gave me an interesting idea, that the various system services, like opencollaboration, desktop search, akonadi, solid, KDED services, and the session manager should all be combined into one ‘Desktop Services’ icon group.

Task Scheduler (crontab) was next. It was easy to identify what would be configured, but when asked about the icon she thought that the wrench and screwdriver had zero relation to the ‘time’ and ‘task’ aspects of scheduling a task. I asked for her idea of what a good icon would be, but she was clueless. Another CS student in the lab overheard me asking and drew a quick doodle of a stopwatch in front of a checklist.

After going through these icons (the rest were fine), I opened up the version I had modified as part of the reviewboard request. It was a slight improvement she said, but lots more needed to be done. However, she mentioned some other interesting ideas that don’t seem to be in Dario and Ben’s proposal:

• If Display is moved to hardware, the screensaver would be expected to be in appearance, since the display is hardware.
• Merge the two CD icons (the CDDB retrieval and cdaudio kioslave) into one group
• Desktop Theme Details should really be a button or sub-menu below the Workspace KCM that actually configures the desktop theme.
• In The KDE Wallet page, she expected that she’d be able to manage her password list in addition to what it does now – configure when it gets closed and opened.
• Default Applications, Device Actions, and File Associations seem to be the same topic and should be grouped somehow
• The Autostart icon “Gear+Play button” clearly means “configure something that runs”

I’d like to post my thoughts based on my friend’s input, but it is approximately 00:30 AM here with a 9 AM class and a super-busy Wednesday. I’ll save that for later today.

Call for icons

For the 4.4 release, my local ACM chapter had a small release event where I demoed all the neat new features. One question that came up was “Why are all the control panel pages so…disorganized?”

I’ve wanted to work on that since it seemed like it’d be really simple and quick to do. However for the past few weeks, I’ve been stuck working with an iRobot Create for my robotics class. [plug] My team and I have been using the libirobot library I started writing a long while ago to get our two projects done.[/plug] Since the library is completely unfinished and experimental, it took up all of my KDE hacking time. D’oh.

But after the two projects were done, I suddenly found myself with free time again. So instead of working on getting webcam support into phonon like I should’ve been doing, I fiddled around with the organization. (Here’s a WIP-quality screenshot too)

I asked a few folks who thought it was disorganized, and they seem to like the new ordering. The only problem though, is that we need a new icon for the new ‘input hardware’ category. I created a bug report, to which pinheiro replied:

ok valid bug, there is a bunch of icons in kcm that could be better a comprehensive list of those and also proposals to make them better would be fantastic.

pinheiro on bug 231270

I’m going to go through them this weekend with some other campus KDE users and a new design major friend I recently met (who has never used or seen KDE before) and look at what needs improvement.

’till then, if you’ve got a problem or suggestion for some icons, go say something in that bug.

Exciting Adventures in Windows

Over the weekend, I went to a small LAN party of 4 people, including myself where we played a ‘quick’ 5-hour game of Sins of a Solar Empire. Yes, the game really is that huge. The real fun though, was that my windows partition ate itself a few days ago and I forgot about fixing it till I arrived. Windows would boot up and show me a BSOD saying it couldn’t mount the boot partition. Someone burned a XP Pro installer CD which I used to enter the ‘recovery console’. Then it got exciting. For about an hour, my friend who is a Windows expert tried to help me figure out which windows ‘drive’ contained my windows install. I was terrified of using the disk checker tool on the wrong drive for fear that Windows would eat it alive, claiming it was all ‘bad data’. My windows bootloader was installed to the first partition, sda1. In Windows, this shows up as the I drive because of the multi-format USB card reader my machine came with. My linux partitions (/boot and /) show up as either J or O, I’m not sure which is which.

The recovery console didn’t care about that useful bit of information I’ve lived with comfortably for about a year. My friend told me that Windows sees the first NTFS partition as the C drive, and thats that. Problem was, that windows couldn’t recognize any drives as being my NTFS partition. Linux could *guess* that sda1 had an NTFS header, but the important bits of the header were corrupt so it couldn’t be mounted. In the recovery console, I, J, and O were non-existant. The only drives I could see were two missing floppies, my two DVD drives, and my three hard disk partitions. After an hour of jumping through logical problems such as “If windows marks the drives it sees in alphabetic order, but my windows drive is actually called I in real life, and chkdisk might find a partition, then maybe I’m probably not screwed.”

I gave up¹, rebooted to Linux in single user mode, and ran TestDisk. Problem solved.

Lesson learned here? You can’t use Windows to fix Windows. It just simply isn’t equipped to do so. I’ll gladly stick with software that gives me the tools to do what I want.

¹: My friend actually donated a spare harddrive for the night that I quickly installed a fresh copy of windows to, letting us finally start our game at 2am. I fixed things when I got home later that day.

Video4Linux2 in xine-lib

This morning around 2:30 AM I finally got a semi-working implementation of video4linux2 in xine-lib. Clone my hg xine-lib 1.1 repository from http://tdfischer.fedorapeople.org/xine-lib/ and have a look at src/input/input_v4l2.c. Thats one big roadblock out of the way towards getting webcam support in Phonon.

Having never hacked on xine before, the code isn’t nearly as nice as I want it to be. In fact, you’ll notice around line 215 that I magically multiply the image size by two and it suddenly works. Yay. There’s also a few other interesting bits, such as returning “v4l:/” as the media URI I’m currently playing, despite being told to play /dev/video0. This appears to be the only way to get the v4l demuxer to start demuxing my video stream. After that though, it Just Works.

So right now my plans are to get it included into xine-lib proper, which might mean porting it to the 1.2 unstable branch. In addition, it needs to be able to support more than just one format (including the MPEG streams that some capture devices use), radio devices, closed captioning, OSD output, and probably most importantly, audio. Bits of that sound to me like I’ll have to write a better demuxer, specific to v4l2 to better handle things. I’ve heard demuxers are easy though.

Lastly, in reply to all the comments on my last v4l-in-phonon post, thanks for the links, but the bit about wanting to write a photobooth clone was just a handy introduction to the real goal: getting webcams in phonon. I don’t really think its possible to simply ‘drop in’ one of those many libraries into Phonon to make things work, as those are all C++ (both gstreamer and xine are pure C), and its the backend’s job to handle getting video from the device anyways. But the kopete library is where I started, and its how I learned how video4linux2 works, so thanks for that.

Webcams in Phonon

For a good while, I’ve been wanting to write a KDE app sorta like Photobooth for the Mac. If you’ve never used it, the name says exactly what it is. It uses your webcam to turn your computer into a cool little photobooth, snapping pictures with different effects, backdrops, and even recording video. Cheese from GNOME does the same thing, but it lacks that ‘KDE’ feel.

With the holidays done with, I had some time today to toy around with the Video4Linux API. After a few hours, I had a small console program with Qt that opened up my webcam and…didn’t do much else. I read about the many different formats supported by the API, all of which looked like a pain to write support for. Then I remembered that Kopete has had webcam support in for a very long time. A little more digging later and I found rudimentary support for webcams in phonon/experimental.

Turns out the code in phonon was lifted almost verbatim from kopete, but it hadn’t been touched since 2008. That meant it didn’t work with my webcam right away. However, I committed some changes a little bit ago that port the kopete changes over to the phonon copy. Now Phonon has working rudimentary support for my webcam. Neat, huh?

But I’m not done yet. I’m only the second commit on the phonon-webcam stuff. While it does work, it probably isn’t ready for primetime yet. Theres bits and pieces (also from 2008) that seem unrelated to the kopete code in phonon, but they make passes at dealing with video capture.

Now here’s the question: is anyone still working on video capture in phonon? If there is, I’m here to help. If not, I’d like to jump in and get things going again.

The content of wm161.net by Trever Fischer is licensed under a Creative Commons Attribution 3.0 United States License unless noted otherwise.
See http://wm161.net/copyright for more details.