/* Baniya Ki Duniya */

Moving to Wordpress...

2010-10-19T16:19:00.001+05:30

My blog has moved to a new location: http://asimmittal.net

You will be redirected shortly...

Robin Hood of the Future: Robot Archer iClub

2010-10-08T13:00:00.001+05:30

Researchers at the Italian Institute of Technology have created the first robot archer. It grips a bow and uses computer vision to aim at the target. It learns on the fly from failed attempts. It was a project led by Petar Kormushev, and is a fruit of open-source development. The robot or humanoid looks really good, standing 3.5 feet tall and has illuminated facial features and big round eyes. It mimics the learning behavior of a growing child. Watch the robot in action:

My own domain ---> http://asimmittal.net

2010-10-07T02:17:00.001+05:30

Hey everyone, I’ve recently bought a domain name at justHost.com and my website goes live today. You can follow my blog through the site.

The URL is : http://asimmittal.net

The blog has been changed visually and is now integrated with the website. You can still follow it via RSS. The website will also contain views into my youtube channel and contain the most relevant project work that I’ve done in the last year or so.

Please feel free to write to me with suggestions on how to improve the look and feel of my website. I have gone with a “Text/Image” only version to speed up loading and achieve a neat and clean look.

Hope you like it!

Note to Self : Google Not Found!

2010-10-06T20:10:00.001+05:30

Today a peculiar thing happened. My web browsers (IE, Firefox and Chrome) couldn’t locate Google.com!

The fix was resetting the TCP/IP service using the NetShell utility from the command line. Here’s the syntax (do remember to run the command prompt in “Admin” mode):

> netsh int ip reset C:\resetlog.txt

Multi point "touchless" Workspace using Wii + Python

2010-10-02T18:46:00.005+05:30

Hey, due to popular demand, I'm uploading a video of the final section of my presentation @ PyCon India 2010. This video shows how the Wii can be combined with some nifty Python code to create a touch less workspace, with more than one point of control (a touch less multi touch... goofy!)

The source is available on the projects page (runPuzzle.py is the file you wanna peek into). I'm currently working on implementing a 3D workspace for viewing photos and playing movies, that would respond to gestures and the motion of my fingers, using the same principle of course. Anyone interested in contributing, give me a buzz. I'm gonna need a little help with OpenGL on Python.

Thanks everyone for making PyCon 2010 a roaring success! Ektitli.org is looking for volunteers to come forth and help us in any way they can. Contact Vaibhav @ ektitli.org for more info.

Going Open Source with "Python + Wii = Intuitive Control"

2010-09-28T15:05:00.003+05:30

Hey everyone, I have prepared a short document explaining the source code. The document is available along with the rest of the source and presentation files. Click here to download the same. If you are having trouble with the link, you can also visit my talk's page on the PyCon 2010 website and download the files from there. I've recently uploaded all the source onto the PyCon website. Click here to go to my PyCon page.

The documentation is about five pages and takes you through a line by line break down of the code for the first three applications of the presentation. Why only the first three!!!?!?! Well, I feel that they are the one's that lay the foundation of the concepts used in all the other applications. These were also the one's I spent the most time developing, simply as I had no precedents or references to begin with. The others just flowed as my understanding of the PyQt4 framework grew.

The applications which seemed to arouse maximum interest were the two games and the touchless workspace (or the puzzle). Those applications have NOT been covered in this document. I would like you to fathom those on your own. They are not very complicated. Those apps are more about the UI than about the Nintendo Wii. I would like anyone downloading this project to give those applications a shot after having fully understood the basics.

And besides, I'm only an email away ;-) Enjoy the Wii, and remember that you are only limited by your imagination!

My PyCon 2010, Bangalore

2010-09-25T22:34:00.003+05:30

It was my first PyCon, and honestly, I would be lying if I said that I wasn't the least bit nervous. I haven't felt the "butterflies" in my stomach for a while, and I was quite apprehensive about how my talk would be received by the Python community.

But the atmosphere at PyCon, really put me at ease and I'm grateful to one and all, organizer and attendee, for giving me the opportunity to share my ideas with programmers and enthusiasts from all over the country.

A special thanks to Noufal, who incidentally, was the first person I came across at the venue.

In tune with my promise of sharing the source of my talk, I have uploaded the source files and they are available for download here. The file is a tar file, simply decompress it and use the wiigrab module. You may follow the sequence of examples as demonstrated in the presentation. Please ensure that you have the following packages installed before running the source:

python-cwiid
xautomation
python-qt4

The source would run smoothly on Ubuntu, provided these dependencies are met. I will be publishing more documentation shortly, explaining the structure of the source (something I couldn't do during the talk due to the shortage of time).

Please feel free to drop me a line if you have any queries or suggestions. Thanks again!

How to make your blog mobile friendly in 5 mins!

2010-09-05T21:01:00.001+05:30

If you have a lot of traffic coming from cell phone users, you might want to optimise your content for the cell phone browser. Its smaller, less powerful and is content centric. Almost all blogs have some sort of syndication using RSS or atom feeds.

http://www.mippin.com is a site that lets you distribute your blog in a mobile friendly format, using your blog's feed. Its a simple 3 step procedure and you can even host your mobile site through mippin.com, just don't forget to put a link to the mobile site on your blog template. I've added mine using my blog description (yes, it is just HTML).

Do visit the mobile version, preferably using a cell phone. Happy moBLOGGING!

Blogging from my s60...

2010-09-05T17:50:00.002+05:30

My first blog post from my s60 phone. Feels like heaven. I've been searching for a mobile blogging client for quite a while now, and until today, i didn't have a viable solution. I came across Wavelog and within 5 minutes of using the trial version, i was hooked. You can even embed images and videos. Check out this video embedded straight out of youtube:

Perfect for blogging on the go! I am loving my new toy.

Ek Titli @ PyCon, India 2010

2010-09-03T16:40:00.004+05:30

Ektitli is an initiative that strives for a cleaner environment through proactivity, real world thinking and youthful effervescence. Today, I do mother earth proud by matching my step with EKT, hoping that one day, the grass is "green" everywhere I see.

I will be promoting their cause at PyCon India, 2010. Help me make this happen. On behalf of the EKT team, I call upon my fellow pythonistas, to open their minds and their code editors and help contribute towards the growth of this movement in anyway they can.

Together, we can redeem our planet... and what better way to do it than coding all the way! Visit ektitli.org for more info about the initiative.

Wii + Python = Motion Capture + Gesture recognition

2010-07-30T21:30:00.002+05:30

Here’s the second installment in the series of videos I’ve been uploading recently on using the Nintendo Wii and Python to provide intuitive means of controlling applications on Ubuntu.

I plan to demonstrate this and much more at PyCon India, 2010, if my proposal is accepted. The Nintendo Wii has an IR Camera built into it with a really powerful on board controller (an SOC) that basically does a lot of hard work to present simplified coordinate based tracking. Each controller can track up to four points in two dimensional space (there is an element of depth as well, but more on that later).

Using the “CWiiD” Project, an open source interface for the Wii remote on Linux, I have developed a small “sandbox” application that allows you to manipulate an object using my hands, as opposed to the traditional forms of control viz. the mouse and keyboard. What’s even more impressive is that the entire hardware will cost only under $50. This provides almost limitless possibilities for the amount of intuitiveness that we can add to an application.

Currently, there is no device driver in place, but with a little help from the open source community and a little “rocket science” (basically a library for recognizing commonly used gestures), we can change the way we interact with our PCs forever.

Here is the video. Hope you enjoy this as much as I enjoyed making it! Cheers to PyCon 2010.

If you like this video, comment on this post, share it with your peers and support PyCon in India and around the world.

Wii Remote + Python + Linux

2010-07-27T00:55:00.004+05:30

Here’s a teaser on the applications that I’ve been building recently for my proposal to present a talk at PyCon India, 2010. The cwiid project provides a simple python wrapper which allows application developers to interface with the Nintendo Wii remote over bluetooth.

I have build a small wrapper on top of the cwiid module that allows you to run a thread which connects to the Wii remote and sample data from it at regular intervals and feed that data to a designated callback routine. This approach improves the usability of the cwiid module and allows a developer to easily embed Wii remotes into their applications.

Intuitive control is something that never comes cheap! Touch screens cost a fortune and are not supported by every application. Using the Wii, and some clever programming, gesture and pattern recognition can be brought to almost every desktop. Brian Peek’s library has been a rich source for developers on Windows using the .NET framework, but it seems like there is little happening on Linux as far as expanding or maintaining a decent Wii library is concerned.

Here is a taste of what we can achieve with a little imagination and a few lines of code. Check out this video I made today, that allows me to set my desktop on fire :D

There will be more updates headed your way very soon. If you like what you’ve seen, comment on my blog or “like” this video on YouTube. Support PyCon India, this year and the next.

I dedicate this video to Brian Peek, whose work on creating a Windows based library for the Wiimote inspired many others to take it beyond the barriers of operating systems and languages. Cheers to you mate!

My thread burnt down my computer!!!

2010-07-24T16:28:00.002+05:30

I've been working on developing applications for my PC using the wii lately. In order to get me going, I found an open source library called "cwiid" with a python wrapper, "python-cwiid". Yesterday, while coding, I kept facing a peculiar problem. Every time I ran my application (which was merely something that connected the nearest wii remote and started receiving accelerometer and key press information from it), my laptop got extremely hot and my computer shut down.

I was quick to understand that this had something to do with the CPU being pushed into thermal cutoff. Interestingly, a previous application that I wrote came in handy. My TempMonitor application allowed me to get a visual comparison of the CPU usage and temperature against time. So I decided to see for myself, how brutally my CPU was being ravaged by the wii application I had made. The results were quite astounding. My application pushed my CPU upto a 100 degrees centigrade in under 15 minutes (coupled with a few other apps like games or presentations for which the remote was suppose to act as a new interface) and caused it to shut down.

So I took a deeper look at my code. My app was essentially a wrapper for the "Wiimote" class provided by python-cwiid. It allowed me to run a thread that received data from the wii remote and invoke a callback routine to handle the processing. This was extremely essential as it was the only way to abstract the nitty gritties of the wii from the actual application. After an hour or two of breaking my head, it hit me like a tonne of bricks.

My main routine started the thread and died, making a "orphaned zombie" (Now this might sound strange since, an orphan thread is one whose parent thread has died. A zombie process is a defunct or redundant entry in the process table. In our case, the cycle of calls is Python Interpreter --> main --> background process. Here, the main routine dies, but the interpreter doesn't. So the background thread has lost its parent but still has a grandparent viz. the interpreter. And technically, since the application is over after the death of the main routine, the python interpreter is a redundant entry that continues to live on in the process table. So the nomenclature!). I quickly added a semaphore to my main routine, that kept it alive until the background task was not complete, and "peace was once again restored to the empire". But then I decided to put my theory to the test. I wrote a script that mimiced the structure of my Wii application.

It had a background task with some heavy processing (well not as heavy as my app, but enough to put a little squeeze on the core), and in order to test its effect on the CPU load and temperature, i create three kinds of main routines:

the first one died after starting the background task, orphaning it
the second one had a semaphore but was a bigger time hog than the background task (i.e. it didn't sleep and fairly share time with the background task)
the third one had a semaphore that made the main routine doze every once in a while and provide more time to the background task

The processing was not too heavy, as i wanted to put just enough load to cause a steady rise in temperature. The background task's job was to open a JPEG and read out all the pixels (values between 0-255). Now the question to ask was, "what would be the threshold temperature"? I wanted each of the cases to eventually break down, and yet give comprehensive results on the effect a simple "sleep statement" can have. Normal range of operation of most cores is between 55 to 90 degrees centigrade. An average load on my computer is between 65-70% (which is higher than most regular computer users) and it usually works between 60 C to 70 C. So after a little deliberation, I chose 70 degrees centigrade as my threshold. The load that the application was going to put on my 2.8 GHz CPU was going to be much above ordinary and more importantly, it was going to be persistent (yes, that's something we normally don't face. Most applications have asynchronous demands which are resource heavy indeed but for brief periods of time. Therefore, we don't really feel the CPU running out of steam or heating up too much. Persistent applications are the best kind of load to make your CPU sweat!).

Here is the script:


import time,sys
from threading import Thread

#---------------------------------------------------------------------------
class backgroundTask (Thread):
 
 """ simple background task """

 def __init__(self):
  """ simple thread constructor """
  Thread.__init__(self)
  self.stop = False
  self.cnt  = 0
 #------------------------------------------------------------------
 def bizzareProcessing(self,num):
  """ some heavy duty processing for testing pruposes """
  fileRead = open('APPLE.jpg')
  image = fileRead.read()
  for eachbyte in image:
   print ord(eachbyte)
  fileRead.close() 
  print "_"*30
 #------------------------------------------------------------------
 def run(self):
  """ worker routine for the thread """

  print 'starting background task'  
  while not self.stop:
   
   self.bizzareProcessing(self.cnt)
   self.cnt+=1
   time.sleep(0.1)
  
  print 'exiting backround task'
 #------------------------------------------------------------------
 def stop(self):
  """ This is the kill switch """
  self.stop = True
 #------------------------------------------------------------------
 def getCount(self): return self.cnt
  

#--------------------------------------------------------------------------- 
def mainZombie():

 bgTaskZombie = backgroundTask()
 bgTaskZombie.start()
 #now the parent thread, i.e. this function will die, but the
 #task bgTaskZombie is still going to be running
 print '\nMainZombie is dying now'
#---------------------------------------------------------------------------
def mainHog():
 
 bgTaskHog = backgroundTask()
 bgTaskHog.start()
 
 while bgTaskHog.getCount ==  20000: pass
 #the semaphore above keeps the parent thread from dying before
 #the child thread, therefore not making it a zombie but no
 #sleep in the loop causes the main routine to hog cpu

 print '\nMainHog is dying now'
#---------------------------------------------------------------------------
def mainProper():
 bgTaskProper = backgroundTask()
 bgTaskProper.start() 
 
 while bgTaskProper.getCount == 20000: time.sleep(1)
 #same as the mainHog routine, only with a little nap for the main
 #routine. This causes the main routine to share time and stay alive

 print '\nMainProper is dying now'
#---------------------------------------------------------------------------
if __name__ == '__main__':
 
 arg = int(sys.argv[1])

 if arg == 1: mainZombie()
 elif arg == 2: mainHog()
 elif arg == 3:  mainProper()

The results are shown below. My temperature monitor application uses google charts API to generate a few graphs of CPU load (blue trace) and CPU temperature (green trace) against time. It records the changes in these two quantities over the last 400 seconds. I ran each of the main routines and clobbered the images together for a clear comparison. Now I did run each main routine over five times to make sure there was consistency in my readings. Here are the plots for the last run:

As you can see, it takes longer for the most properly structured application to reach the threshold, while the one with the zombie hanging around is quick to touch the threshold temperature in no time.

Problems like this may go undetected and surface in an untimely fashion to make your life a living hell. So structure your threads well. Make sure all the babies have their parents and everyone gets enough sleep.

Automated Wallpapers for your Ubuntu Desktop

2010-07-22T12:38:00.003+05:30

I was quite bored with the ill written dynamic wallpaper utility on Ubunutu. It uses the wallpapers stored in /usr/share/backgrounds directory in union with an XML file, to change your desktop wallpapers. If you wanted to add your own collection of wallpapers, you'd have to put them in that directory and edit the XML file and god save you if you accidentally deleted some wallpapers, 'cause then you're going to get a plain screen every time it tries to load a dead link.

So I wrote my own little desktop utility called "Wallshaker". It allows you to load multiple directories as sources of images, monitors them and doesn't load dead links. If new images are added to any of the monitored directories, they are picked up as well. You can even configure the timing between wallpapers (in seconds). But hey, this is pretty regular stuff. You get a truckload of applications that do just the same thing. "DRAPES" for one is a decent app and is available through the repositories. So what is it that sets this application aside.

Well, my little app has a checkbox that reads "Live World Map". If you click that, satellite images of the sunlight map of the world is grabbed from the web and set as your wallpaper. The map is refreshed multiple times during the day as and when new images are available. Cloud patterns are update every few hours and you can actually see how one part of the world moves from light to darkness during the course of the day.

Now you can simply look at your desktop and see which parts of the world are "awake" or "asleep" :D

But simply using this was not enough. So I wrote another utility that would grab my desktop every now and then and made a movie out of it. Its a small clip, spanning almost six to seven hours of the day squeezed into 15 seconds. But it clearly shows you how your desktop changes as the earth moves about its axis and as the day blends into the night.

Here's the link to the video. As always, you can download the app from my projects page.

=-=-=-=-=
Powered by Blogilo

Taking periodic snapshots of your Ubuntu Desktop

2010-07-21T14:03:00.015+05:30

I've been working an application (will be posting more about that a bit later), but in order to test it, i needed a tool that would constantly take snapshots of my desktop every few minutes and save those snapshots in my home folder.

I thought long about doing this, without support from external applications and using only those that were native to gnome. Unfortunately, that didn't work out too well, simply because the snapshot tool in gnome captures the image and pops up a save dialog box with a preview of the snapshot taken. I didn't want to sit and record a macro as they don't always work too well.

That's when the guys at google tossed out "imageMagick" a.k.a "import" at me. This is a utility that comes packaged with Ubuntu (i'm using 10.04 LTS). And it works purely off the command line without any confirmations or dialogs. Here's a sample of how its used:

shell$ import -window root some_filename.png

This saves the image "some_filename.png" into the current working directory. Its really compact and works inline at the shell. The next thing I wanted to do was to minimize all the applications (CTRL + ALT + D, show the desktop basically) before the snapshot was taken and restore them after my work was done. This is where I feel the .NET framework's SendKeys("keys") makes so much sense. You can use that to send any key combination to the OS. Unfortunately, none of that was available to me on Ubuntu, so I found another command line utility (a really really neat one at that!) called "wmctrl". It can be used for a variety of tasks, and here's how it can be used for minimizing and maximizing all windows:

shell$ wmctrl -k on #minimize all windows to show desktop

shell$ wmctrl -k off #maximize all windows

Finally, combining the two, I came up with a small python script, that allowed me to take snapshots of my desktop at regular intervals. The interval, in minutes, can also be specified at the shell. Here's the script.

Small program "logger" module in Python

2010-07-20T14:52:00.004+05:30

I was sitting around and a thought crossed my mind, "How can I get the function name of a function that the trace is in?". I thought about it for a few minutes, and said to myself, "The stack must know". But how do I read the function names of the stack in Python??
For all these system level hacks, usually the sys module has something to offer, and this case was no exception. Here's how you can do it (using something called a "frame object"):

print sys._getframe().f_code.co_name

Another useful modification of the snippet above can give us the name of the function that is calling the current function. This is how it looks:

print sys._getframe(1).f_code.co_name

So as you saw, the argument of sys._getFrame() can take you up or down the stack as your need may be. This utility of the sys module is extremely useful while logging for errors. Almost every good programmer uses some kind of logging in his code. So far most of my logging was using independant file writes or console writes. With my newfound knowledge, all that was going to change, so I wrote myself a small, yet handy logger for my forthcoming projects. Here's the script.

I integrated this into one of the apps I was making and well, it was simply fantastic. Now I just piped out the log messages to this module and it would automatically determine which function was sending the messages. I wrote an app with a GUI in PyQt4, and as will all GUI apps, tracking events of widgets is extremely important in figuring out what went wrong during debugging. This is what the output of my new logger looked like:

Tue Jul 20 12:32:53 2010: <loadAppData()> Trying to load application data
Tue Jul 20 12:32:53 2010: <loadAppData()> Values picked up from appdata file. Interval:43; Dirs:[]
Tue Jul 20 12:32:53 2010: <loadAppData()> Application data loaded successfully
Tue Jul 20 12:32:56 2010: <btnAdd_Click()> Add button clicked
Tue Jul 20 12:33:12 2010: <saveAppData()> Trying to save application data
Tue Jul 20 12:33:12 2010: <saveAppData()> Application data saved successfully!
Tue Jul 20 12:33:26 2010: <trayIcon_Click()> Tray icon clicked. Current visible status:True
Tue Jul 20 12:33:26 2010: <trayIcon_Click()> Tray icon event complete. Current visible status:False
Tue Jul 20 12:33:30 2010: <trayIcon_Click()> Tray icon clicked. Current visible status:False
Tue Jul 20 12:33:30 2010: <trayIcon_Click()> Tray icon event complete. Current visible status:True
Tue Jul 20 12:33:35 2010: <closeEvent()> Form close event raised
Tue Jul 20 12:33:35 2010: <saveAppData()> Trying to save application data
Tue Jul 20 12:33:35 2010: <saveAppData()> Application data saved successfully!
Tue Jul 20 12:33:35 2010: <closeEvent()> Program is exiting.
This is so neat :D

Remote control your PC using a Wii remote

2010-07-12T14:14:00.002+05:30

I’ve been tripping on the Wii lately, and another application that I cooked up over the last couple of days was something I like to call “WiiCONTROL”. Its an application that allows me to control navigation on Windows 7 using the Nintendo Wii remote.

I have built my app on top of Brian Peek’s Wiimote library. I hope you’re reading this Brian :D

The need for the application arose while I was watching a movie a night ago, and I realized that using a wireless keyboard while lying in the comfort of my bed was rather annoying. I needed something that didn’t take much space and was “like a remote”. Lying next to my computer was the Wiimote I’ve been using off late.

The application took me nearly a day to make. The major portion of the app was figuring out how the navigation on windows could work solely through keyboard events and then tying them up to the button press events generated by the Wiimote.

Here’s what it looks like:

You can clearly see which functions are tied up to which buttons and the battery indicator gives you a rough idea about how much juice you have left in your remote. The experience counter may be used to control how responsive (key de-bouncing is such a pain!) your remote should be. The preset value works really well for me. The box titled “function” can be used to check which key corresponds to which function. You push a button and it shows up on that box.

Here’s my personal favorite. When you click “test feedback”, the app sends a signal back to your remote causing it to vibrate momentarily :D confirming a closed loop between your PC and your remote.

The app iconizes to your system tray and can be brought back to focus by double clicking the “YODA” icon (heehee!!!) therein.

Whats cooler, is that if you use VLC player, you can use the same remote to Play/Pause/FF/Rewind/Control Volume.

Here’s a video I made last night of me using the app on my home computer. Its a bit fuzzy thanks to the compression, but you can see what the deal is all about.

Go to my projects page to download the app. Please note that you have to hook up your Wiimote to your PC before starting the app. If the Wii is not found, it will simply notify you and exit. For a tutorial on how to connect your Wiimote to your Windows PC, go here.

Scribbling and drawing on your computer's desktop

2010-06-25T15:50:00.003+05:30

This is a follow up to my previous post, where in, I demonstrated the use of Johnny Lee's application to manipulate any display or surface using an IR stylus. Since Lee's application was open source, I decided to extend that application and add a few features that presenters and educationists might find extremely useful. These features help increase the usability of the app by allowing the user to annotate on anything on his desktop.

Educationists might find the "Black board" feature extremely useful as it allows them to blank out the screen and draw and erase on it in multiple colors. The snapshot feature allows you to share your annotations in the form of images that you could create using any simple app like Microsoft Paint. My goal is to make this application feature rich and convenient to use. I need your help in understanding what you'd like to do with this application. If your suggestions are practical and feasible, you might find them in final release of the application and your name in the credits :-)

I have made a small video highlighting what the current feature set can allow users to do. Check it out and comment or mail me about what you think. If you like this app, then spread the word and let your friends know as well. This effort is made in a bid to dissuade users from using commercial applications which essentially work on the same technology at their core, but insist on a payment of around $35-$40. Great technology is one that is easy to use and that is easily available. I don't believe something so simple and so much fun should have a price tag on it. Help me in my endeavor to expand the scope of this application!

Here's the link to the video.

Make any display a Tablet... for under 40 Dollars!!!

2010-05-23T17:21:00.003+05:30

So yesterday, I was browsing around the web looking for Human Machine Interfaces, and I chanced upon Johnny Lee. The creator of a simple, yet powerful tool that allows you to use a Wii remote and a home-made IR pen, to create a tablet like (stylus sensitive) display out of any thing from a computer screen to a wall or any other surface that is capable of acting as a screen.

The hack is brilliant and Lee has gone a step ahead to build up a few crazy applications based on Wii's motion sensing technology.

Here's my video on how my Compaq CQ45 transformed into a tablet. After a preliminary run, I modified my IR stylus for more power and distance and started playing around with CAD. The experience is simply unlike anything I’ve used before. A little more development, and this could potentially change the way designers work.

My IR stylus allows more current flow through the LED (within tolerance limits), allowing me to perform, what I call a “grab” and “drop”. It lets me grab an object on screen and move it around from nearly a foot away. Almost felt like waving a magic wand!

Here’s a video I made while testing out my latest mod to Lee’s electronic whiteboard system.

This has already taken the world of electronic whiteboards and projection systems by storm. What's more, is that it costs you nothing but a remote and under a buck to make the IR Pen (if you can cook it up on your own, else commercial products are also available). Visit Johnny Lee's site for more info.

There is surely going to be a lot more “Wii” in my forthcoming projects!

Feeding password to sudo over command line

2010-05-22T02:20:00.001+05:30

I'm sure all linux users (ubuntu) have had a day when the security features of the OS, just seemed a tad more annoying than ever before. Specially when you have to put your password into sudo for every menial task. Or perhaps all you wanted to do was install a package using apt-get and didn't care to press "Y" in front of every question that followed.

Here's something you can do to minimize user intervention after you press the enter key in order to execute a command. To feed your password to sudo, use the following construct:

user-laptop$ echo <password> | sudo -S apt-get install blogilo

Using the -S switch allows you to feed the password through a pipe using echo over the command line. In order to skip the crazy prompts for apt-get, use the following construct:

user-laptop$ sudo apt-get -y install blogilo

Here's something that combines the two:user-laptop$ echo <password> | sudo -S apt-get -y install blogilo

P.S.:

This is my first post using "Blogilo". Just moved to Kubuntu (KDE4 for ubuntu. Download the kubuntu-kde4-desktop package if you're on v10.04). Loving the new UI and the cool desktop effects. Konsole (the default terminal for KDE, sucks... no transparency!) has still miles to go before i leave gnome-terminal. Empathy and Gnome-terminal are the only GNOME apps that I'm still hung over. The rest is good. Desktop effects is really the area where KDE takes the cake. You can toggle between desktop effects based on your power options too. So if you're running on battery power, you may wanna keep these effects to the minimum, and turn them back on when you're plugged into a power socket. You can have as many desktop workspaces as you want, and toggle between them using a mouse or keyboard. Miss the simplicity of GNOME, but for now, KDE is gonna stay :D !!

Check out the video below, to get a glimpse of KDE has to offer. This is just the tip of the iceberg.

KDE 4.3 desktop effects

FireUploader - Mass Media Uploader for Ubuntu

2010-05-16T11:45:00.003+05:30

I've been using Ubuntu for a long time and its evolution over time has been a thing of beauty. The latest version (10.04) is the best Ubuntu so far. It combines great look and feel with the stability and security of Linux. Canonical has worked hard at addressing the various issues and incompatibilities users around the world faced with the release of 9.10. I remember how I could never wake up from a hibernation as my keyboard and mouse would be rendered useless. Power consumption was another sore point, which has been improved in the latest release. Clearly, their proactive attitude has put Ubuntu on the path to becoming a household name among people who don't want to pay an exorbitant licensing fee, recover from system crashes or never be able to figure out which anti-virus to use for Big Brother's Windows.

The only drawback (arguably!), I feel, is the lack of support for non commercial applications. Windows, being the most popular operating system, is preferred by a major chunk of software developers and companies (specially based in Asia) as a target OS. Even companies like Google, who pioneered the integration of mankind and hyperspace, have now begun to turn their back on Ubuntu (Google video and voice!!!! SCREEEECH!!!). A lot of software and plugins for existing software, unfortunately never make it beyond Mac OS and Windows. Mass uploading of media onto various file sharing and microblogging sites is yet another grey area, with little to help the world's most popular free OS.

Uploading content to Facebook (pictures and videos) and Flickr was rather tideous. With Adobe's Flash Plugin adding insult to injury, one was restricted to using the basic uploading utilities which needed constant monitoring and were rather unsuccessful on many an occasion. I, personally, have faced so much trouble posting pictures onto Facebook using the basic uploader that I had given up taking pictures all together. While third party plugins were being released for Picasa, on Windows, allowing users to export their entire library or parts of it onto facebook and flickr, all I could do, was be patient or make unsuccessful attempts to "wine" these applications.

Alas, Mozilla stepped up with FireUploader. An add on that allows users on all platforms (windows or linux) to upload their media content onto a multitude of social networking and file sharing sites.

Its extremely easy to use and authorize. It's UI is simple and elegant and it has an upload manager that allows you to pause and resume your uploads as and when necessary. Here's what it looks like:

This add on, along with Download Helper and Yoonoo are among the most popular plugins for Firefox. I recommend users on Windows and Linux alike to give this a try. It definitely gets my vote!

Appellation - Smart renaming for your music files

2010-05-14T05:07:00.004+05:30

A while back I had posted a little command line tool called "Appellation", that would essentially help you rename your music files from meaningless default names (e.g. track 1 etc.) to more appropriate ones with artist and song title in the filename (e.g. JayZ-Empire State of mind.mp3).

Here is the link to that post.

This may not seem all that useful, but greatly improves locatability and file system indexing, yielding to faster results during a search of your hard drive. It also creates a more visually appealing structure to your music and allows you to store all your music files in one big folder without worrying about organising it through a buch of sub folders.
I was sitting around with a little itch that I simply had to scratch. I had to create a simple GUI for this utility. God knows console applications can be speedy but look worse than a 500 pounder without his underpants.

As usual, the GUI is in PyQt4. So in order to run this application, you'll need to have this toolkit installed along with Python 2.6 or higher. This application should work on both Windows and Linux without much difficulty provided all the dependancies are met.

Here's a screenshot of what the application looks like:

You can go to my projects page to download the source code for this application (look for "appellation"). The application scans through an entire directory recursively, searching for music files and trying to extract the artist and song names from the ID3 tag present in the file. If the ID3 tag present is not the latest version, it may not be read properly. Backward compatibility with older versions of the tag are not supported. Perhaps that is something I can toss out in the next version of this application.

The UI is fairly simple. Select the directory to search for music files using the Browse button. Click 'Start Renaming Music Files' to start the search and renaming process. At any time if you feel like stopping the search, click the small button marked 'X' near the progress bar, both of which come alive when the search is commenced. The summary of operations is displayed in the text browser widget and a total count of failures and files parsed is also displayed.

Good app to kick off your appetite on PyQt4. Hope you enjoy the app!

Review: Traffic Shaper for Windows (Freeware for limiting bandwidth usage)

2010-04-13T15:17:00.001+05:30

I was recently at the office blitzing a download for Visual Basic 2008 express on the High speed DSL line. Suddenly the folks around me started grimacing and cursing their computers in unison. I realized that my download was putting a lot of work on hold, and so I was forced to close it and wait for a more opportune time.

Whenever I’m in a situation like this, I start thinking about code. That makes me wonder if there was a way to control the traffic on every workstation in the network. I was quite sure that there would be many applications available to professional network administrators to aid in the task at hand. Making one from scratch would be a worthwhile waste of time :D But I had way too much work to sit and break my head over this right now. So a ready made solution it had to be…!

A few Google searches later I stumbled upon “Traffic Shaper XP” (click here to download). This is what it looks like:

To start checking your bandwidth, simply go to View> Options > Internet. Here choose a bandwidth limit for uploads and downloads from this computer. That’s all there is to it. Just be careful, as the settings you choose are in kiloBITS per second. So divide the number by 8 to get the limit value in kiloBYTES per second. I set my download limit to 512 kbits/sec allowing me upto 64 kBytes/sec. This download limit is now shared between all the active downloads on my computer.

So all you downloaders out there, download with ethics. “Surf and let surf”. Cheers!

Appellation - Smart rename utility for your music (Core Python)

2010-04-13T00:51:00.002+05:30

Haven’t you always wished that your music was well organized but were too lazy to do it yourself. Here is an app that addresses the first step towards organizing your music collection viz. “Nomenclature”. Personally, I have always preferred my music files to be saved on disk in “<artist name>-<song title>” format.

Recently I had written a post on extracting ID3 tags from music files (read it here). I decided to extend that module and create an application which could be used to rename all my music files in the format mentioned above. The application would simply accept the path of destination directory and recursively search it for music files. If the ID3 tags present in the file are the latest version, the renaming process would occur without much difficulty. However, if the version of ID3 tags is very old, the process might not occur, leaving the filename unaltered. Another case of failure is when the artist information is not present in the file (No frame id TPE1 found… the app won’t look for anything else as the performer name!)

I was too lazy to create a GUI… so you are most likely to see a console when you run this app. But fear not, there is little you have to do. Simply copy and paste the path of the destination directory into the console and press enter. The progress of the entire operation is displayed on the console itself.

You can download the application from my projects page (click here). Look for a file “appellation.rar”. Simply extract the files into a folder and run the launcher script “run.py” using python.

The app is open source, so feel free to browse through the meagre amount of code. Perhaps if you feel like extending the app, you can drop me a line. Someone please make a UI if possible. I’m too bummed out with GUI for the moment to make this any better than its current (and ugly) form :)

I ran this on my collection and got pretty decent results. The summary towards the end of the program displayed a sum total of 20 files (out of 16 GiB of music) that had failed. Hopefully you have similar or better results.

How To: PyQt4 - Temperature Monitor with Google Charts support for Ubuntu 9.10

2010-04-04T17:33:00.011+05:30

I was recently using various system monitors on my Ubuntu 9.10, the most popular being "System Monitor" (the default gnome application for this purpose), and I realised that neither of these utilities presented any information about the CPU core temperature.

Perhaps its not such a big requirement, but I'd really like to know how hot my CPU core gets under different load conditions. For instance, when I run a massive search on my filesystem, I find my laptop getting a little warmer (a little too warm sometimes!!). The CPU core will always be at a higher temperature than the body/ambient environment since it's always at work. The operating range of most CPUs is between 55 to 85 degree centigrade. So if the ambient temperature (temperature of the body) seems to have risen by a small margin, there is a good chance that the increase is considerable inside your core.

So I decide to compliment the existing utilities by making an application of my own - TempMon. I decided to use PyQt4 for my application as its easy to use and fits perfectly with Python, my language of choice. Before I began, I penned down the goals for this application. Here they are:

The application must display the core temperature of the CPU at all times
The application must also display the average CPU load (a.k.a CPU Usage). This is usually represented as a percentage by many system monitors. Since I have a dual core 64 bit AMD at the heart of my motherboard, I will compute the "AVERAGE" cpu load. Gnome's system monitor provides individual load percentages for each core
The application must allow me to trigger an alert/alarm if the core temperature exceeds a particular threshold value that can be user defined
The application must be dockable to the taskbar or system tray
Finally, all the data assimilated (temperature and usage info) over a certain period of time must be displayed using a plotter or a graph

Firstly, its important to understand your data source for this application. Where does the application get the temperature and CPU usage data from? On Linux, this information is available on the local filesystem on the following files:

'/proc/acpi/thermal_zone/TZ01/temperature' contains a string that contains the immediate value of the temperature. In fact, this file has only one line that says "temperature: 64 C". Makes my life a lot easier :)
'/proc/stat' contains the information about CPU usage. There is a lot of information available on this file and not all of it is relevant to our need. The first line is all we need. It looks something like 'cpu 936808 309 246585 14286660 39745 3447 13778 0 0'. Only the values in bold digits are important to us.
So how do we calculate the CPU usage? Its rather simple. In the above example, consider the four numbers (in bold). Let them be u1, s1, t1 and i1. At another time (usually on a successive read of this file), let the values for the same numbers be u2, s2, t2 and i2. Now we will calculate two quantities (since we need a ratio for a percentage): usage(U) and total usage(T). The ration of these two values multiplied by 100 will be the percentage of CPU usage
U = (u2-u1)+(s2-s1)+(t2-t1); T = U + (i2 - i1); CPU Usage = (U/T) x 100 %

I quickly opened up Qt Designer, and a few clicks later, the GUI was ready. Small and compact but very informative. Here's what the UI looked like:

Two labels that displayed the temperature and usage percentage, one spin box to set the threshold value for temperature, a checkbox and a push button. Thats pretty much it. Once I was done, I saved the file as "fontend.ui" and subsequently converted the GUI into a python source file using pyuic4:

shell$pyuic4 frontend.ui > frontend.py

A file "frontend.py" is generated and contains a class for this dialog. That class has a method called setupUi that draws the form shown above. Now, I'm ready to start coding the core of the app. I create another file called "tempmonitor.py". You can download the entire source code from here (Look for a file labelled "Temperature Monitor").

In the init routine of the main class of "tempmonitor.py", I created a system tray icon object:

self.trayIcon = QtGui.QSystemTrayIcon(QtGui.QIcon("trayicon.png"),self)

Following which I connected this tray icon to a slot where in I toggled the form's visibility:

self.connect(self.trayIcon,SIGNAL("activated(QSystemTrayIcon::ActivationReason)"),self.toggleVisibility)

This creates a system tray icon, which allows me to show/hide the form by simply clicking it. The self.toggleVisibility routine simply changes the visibility status of the form to the opposite of its current state.

Next up, its time to write the actual tasks of polling the temperature and usage values from the respective files. That is done using self.getTemp and self.getCpuUsage methods. Both these methods read the respective files (mentioned above), and process the data within to create two strings, one for temperature and another for CPU usage. Once all the hard work is done, each of these methods, emits a custom signal viz. "tempUpdate" and "cpuUpdate" respectively.

These signals are very important. Since these routines do the "heavy work", its only natural to run them as a thread in the background (in order to prevent the GUI event loop from being interrupted). Now if they're going to run as concurrent children threads, directly providing them with access to the widgets on the form is risky business. Signals help us solve this problem. The background threads emit signals which the parent thread (GUI event loop) comes to know about. Each of these signals is tied to a slot (a method basically, that allows us the parent thread to "do" something when it discovers the signal). The following statements help redirect these signals to their respective slots.

self.connect(self,SIGNAL("tempUpdate"),self.tempUpdate) self.connect(self,SIGNAL("cpuUpdate"),self.cpuUpdate)

Now these slots are tied up to the self.tempUpdate and self.cpuUdate slots respectively. These slots are simple methods of the class, which can now be used to update the text in the labels for temperature and CPU Usage.

So the heavy work is done. Its time to actually spawn a thread that will run do all the heavy work. This thread must run until the parent thread dies (application is closed). I put up all the methods which I want to run, into a single routine called self.idleTime, which is now run in the background using the following statement:

thread.start_new_thread(self.idleTime,())

The empty parentheses indicate that no arguments are being passed to this routine. This routine periodically runs all the other methods which will read the temperature and usage data from the files, process this information and emit signals to indicate that their work is done. After every iteration of these methods, I put the thread to sleep for a couple of seconds (this allows for context switching, and since the data we read isn't changing drastically every moment, this resolution is pretty good).

We're almost done, except for the last criterion that demands a graphical display of the assimilated data. If you notice, in the source for self.tempUpdate and self.cpuUpdate (slots for the signals we spoke about earlier), the values that are read are stored as tuples of (temperature, cpu percentage) in a list called data. This list is going to serve as our source of information when we plot the temperature and usage values graphically using Google Charts.

The Google Charts API is really easy to use and the best part is that it works easily using simple GET requests. So all you really have to do is take all the data you want to plot and formulate a URL. That URL can be opened in a web browser and your chart is ready. As a hack I used the QWebView widget to dynamically generate the URL for the chart every few seconds, thereby giving the appearance of a real time graph/plot (just like system monitor does). Anyway, that is not part of this tutorial, so let's stay on point.

The self.getChartLink method generates the URL based on the values present in the list ''data". This routine is also added to the idle task routine. So now, the worker thread that runs in the background will read the temperature, the CPU usage and generate the URL for a graph based on the values stored in data. Since Google Charts API has a limit to how many points can be plotted in a single chart, I've restricted the plot to the last 200 values that were sampled. Currently I'm sampling the values at an interval of 2 seconds (sleep time for the worker thread), thereby giving myself a plot of the temperature and CPU usage over the last 400 seconds or roughly 6 minutes. Here is what the chart looks like:

The green trace represents the CPU temperature (in centigrade) and the blue trace represents the CPU usage (in percentage).

Download the source from my projects page (look for Temperature Monitor). In order to run this project you will need to have PyQt4 installed along with Python (>=v2.5.0).

And that is how its done!

How To: Extracting ID3 Tags from MP3 or Music files

2010-03-31T17:39:00.002+05:30

Just the other day I was using Rhythmbox to play a few songs I had downloaded. I don’t remember what it was that struck me about the album cover image that was automatically loaded in the bottom left corner of my screen, but it made me wonder how that had happened? I mean, not all my songs had album cover images loading every now and then. Then what was special about this particular song (oh btw, the song was Amplifier by Imran Khan)? I also happened to notice that the recently downloaded music files were a little larger in size in comparison to the songs from yesteryear.

I knew about album art being put inside the same folder as the audio files (a lame image titled folder.jpg), but this was different. There was no such image present in the album directory. So I guessed that it must be part of the file header. A few Google searches later, I affirmed my hypothesis and found myself going through the entire spec for the latest version of ID3 tags at http://www.id3.org/Developer_Information

ID3 tags have been around for slightly over a decade with the first version ( a simplistic “TAG” appended towards the end of the file) released in 1997. Over the years, they have evolved along with the quality of sound they represent. The current ID3 tags (v3.2.4) are capable of storing more information about a song, than perhaps musicians and producers can even provide. What’s more interesting is that these tags allow for variable length fields. So your information is not confined (unlike the earlier versions where names and other information had to be squeezed within 30 bytes).

A review of the ID3 tags specification will tell you that all the metadata about a song is organized as “frames”. Each frame represents a unique field. For instance you have one frame (TPE1) telling you the name of the performing artist and another frame (TALB) bearing the name of the album. The album art may be found under yet another frame (titled APIC, yes.. the entire jpeg image is actually stored against this frame in the file). All frame IDs are four bytes long (TPE1, APIC, TALB, TRDC… etc).You can find a more comprehensive list of frames at http://www.id3.org/id3v2.4.0-frames

From a programmer’s perspective, what one needs to know is:

how much part of the file must be parsed? It makes little sense to go on from the first byte to the last when the info you need is only present in the beginning of the file
how do we know how much data to parse after each frame ID (the data is variable length right?)

Firstly, the size of the entire ID3 tag can be got by checking the size field of the ID3 tag. This info can be found in the ID3 header (first ten bytes of the file that look like this: “ID3 XXXXXXX”). Attention must be paid to the fact that the bytes that will be read in hexadecimal format, and if you’re going to read one byte at a time, you will have to clobber a few of them together to construct the size field. I had to write a small routine that would convert a hex string to an integer so that I could compute the value of the size field. So getting hold of this value will solve the first issue.

Each frame ID is followed by a size field as well. This value indicates the length of data corresponding to each frame. So we simply use that to solve the second issue.

Another point to note is that not all tags listed in the specification, will be present in the file stream. Most mp3 files will contain basic info about the performers and producers, the album and a jpeg for the album art if you’re lucky.

I created a class called TagParser, which basically creates a sliding window (4 bytes long) and rolls all the bytes in the file through this window. Whenever the window holds a valid frame id, it looks for the size field for that frame and proceeds to extract the metadata for that frame based on the value of the size field.

The extracted data is stored against the corresponding frame id in a dictionary. The parser scans the entire ID3 Tag in a file and extracts any frame that can be found. (I made an app using this module called "Appellation". Go to my project page and download the app.)

The object of this class invokes a method called ‘parser’ which accepts a path to a file as an argument and returns a dictionary containing the frame IDs as keys and a tuple of (description, metadata) as the value against each key.

Extracting audio from PWI files

2010-03-13T16:55:00.005+05:30

My father has been using a Windows mobile phone (IMATE Jasjar) for quite some time now, and makes voice recordings using the Notes application that comes along with it.

These notes are saved as .PWI files. The good part about the notes is that they combine text and voice all in the same file. He did a lot of reading online, about how to use MS Word to manipulate .pwi files. But when nothing really worked out for him, he came to me with the problem.

So I wrote him a command line utility in python, that would help him extract all the audio from his files. In order to do this, I first opened the file in a hex editor (I use Bless Hex Editor on Ubuntu). Within a minute I realized that the format was extremely simplistic. All the audio was saved in the files as RIFF Wave (yes.. really really old). And what was even more convenient was the fact that all the audio was stored after a "RIFF" tag in the file.

Another point to note here was, that a single PWI file could store several audio snippets, each one starting out with a 'RIFF' tag.

This is what the file looked like:

So all you have to do is extract the pieces of the file starting with the 'RIFF' tags. Python, as always, would prove to be the most efficient language for a task like this. Simply read the entire file. Split the content into pieces using the 'RIFF' identifier. Ignore the first piece. Each successive piece would be an individual audio file. Take each piece and write it into an independant wave file.


########################################################################
# filename :    pwi2wave.py
# utility  :    converts .pwi files to .wav files (handles multiple wave
#               files stored in the same pwi note)
# author   :    Asim Mittal
#
########################################################################

import os,sys

if __name__ == '__main__':

   filename = sys.argv[1]
   filename.lstrip().rstrip()

   if os.path.isfile(filename) is True: #check if the path provided was indeed a file
       name,extension = os.path.splitext(filename)
       try:
           fRead = open(filename,'rb')
       except:
           print '\n\nError reading file!! Exiting'
           sys.exit(0)

       content = fRead.read()    #read the entire file
       lstPieces = content.split('RIFF') #split the file into pieces using the RIFF tag
       fRead.close()      #close the file i/o stream


       #now lstpieces contains all the parts of the file separated
       #by the RIFF tag. The first element of this list is not important
       #so we'll take everything starting after.
       #also note, when storing the wave part of the file, we will have to
       #add the 'RIFF' tag again

       del lstPieces[0] #not of interest to us

       for index in range(0,len(lstPieces)):
           newname = name + '_wave_' + str(index) + '.wav'
           newContent = 'RIFF'+lstPieces[index]
           fWrite = open(newname,'wb')
           fWrite.write(newContent)
           fWrite.close()


       #the above loop creates as many wave files as there are audio segments
       #in the pwi file

   else:

       #the filename didn't point to a file. tell the user about it
       print '\n\nBad path provided. Please give the path to an actual file.'

And that is how its done!

How to: Building a front end for Python help

2010-03-10T05:19:00.004+05:30

I'm sure most python programmers out there who have used the Python interpreter's built in help utility and regretted not being able to perform a simple "Find" operation on everything that was visible. So I decided, that it was about time, this was taken care of.

A good front end for an extremely useful help utility was lacking in Python, and so, I named my application "Venom" (something a python doesn't have, but could really do wonders with!). It is a really simple tool that consists of a single widget form that looks something like this:

Its only a text box!!! Its simple design was inspired by the Google desktop search utility. All you have to do is enter the module name (as you would have done at the 'help' utility prompt) and press the enter key.

This tool also remembers all the searches made during every run, and you can scroll through earlier queries simply by using the up/down arrow keys. Another interesting feature is that, this tool can be hidden onto the taskbar or system tray. There is an icon placed in the system tray, clicking which toggles the visibility of the tool.

When you press enter, the query is validated and if the help utility does dish something out, it is stored and displayed to you in your web browser as a simple HTML file. You can then perform simple search/find operations, which were not possible when the interpreter was executed directly from the shell.

The entire utility is built using PyQt4. The application is completely cross platform and I have tested it out on Ubuntu and Windows 7. You must have the PyQt site-packages and a Python 2.6 interpreter installed. PyQt4 is free and may be downloaded from the Riverbank Software Website.

From a programmer's perspective, this utility was a great way to understand how to create and use a system tray icon in PyQt (using the QSystemTrayIcon class). The UI was designed using Qt Designer and converted into Python code using pyuic4.

Click here to download the source!! (look for Venom)

Hope you like it... Its really made searching for help on Python a lot simpler. A funny thing I noticed was that the system tray icon was appearing in full color only when the utility was being run from the terminal. When I ran this application from anywhere else, the system tray icon didn't appear or was vague, but space was made available in the system tray for this icon. In order to prevent more confusion, I added a tooltip. Clicking on the system tray icon will toggle the visibility of this application.

Any ideas on how this may be expanded further...?? Please comment on this post or mail me.

A Small PING utility to keep the network awake

2010-03-09T07:01:00.000+05:30

I recently set up a small home network on windows, and was using samba to access the same using Ubuntu.

I realized over a period of time that some computers on the network have a tendency to fall asleep, preventing me from being able to access shared data. A simple solution to this was a simple "ping". But the task of facing "access denied" and then running a terminal to ping the particular computer was getting too tedious. Moreover, since I have a wireless router in between, the IP addresses for all the machines keep hovering in an around my current IP address.

Today I decided that enough was enough, and I had to squash this issue once and for all. So I wrote something called "pingTEN" ('ten' being the reverse of 'net' .. heehee!)

Its a really simple utility which allows my computer to ping X number of IP addresses on either side of my current IP address. Since this operation can be a major CPU hog, you can also program Y number of seconds, for which the program will sleep once all the pings are complete. It performs a single packet quiet ping.

Since I usually have about two to three computers active on my home network at any time, I simply type this:

shell:~$python pingten.py 2 10

Now 2 (is X) and 10 (is Y). Another cute feature I've added is that everything that is put out on the console (if you're running it from the terminal), will be written down periodically into a file, with a timestamp. Why is this important?? Simply because, I can now choose to run this in the background (using the System > preferences > startup applications menu) and be able to monitor if it is still active. It runs as if it were a daemon or a background process.

If I simply open nautilus (or whatever file browser utility is available), I can actually see the file size changing from 270 bytes to 0 bytes to 270 bytes... and so on. With every set of pings, the file is overwritten with 270 bytes of data. The program then sleeps for ten seconds and rewrites the file when it wakes up, causing the size to get nullified.

Its a really simple way to keep track of my application, which I cannot see without the terminal.

Click here to download the source!

Adding your Windows partition to the GRUB

2010-03-06T11:31:00.003+05:30

So I've been toying around with Windows 7 and decided to install Ubuntu 9.10 to go with it. Obviously Windows 7 was the first to be installed (that way, I hoped that the GRUB loader would recognise a Windows installation and automatically add it to the list of operating systems to choose from); however, to my disbelief, the GRUB tricked me and Windows was nowhere to be found in the list.

My first reaction was of frustration, but I decided to use the existing installation of Ubuntu to help me figure out how to go about getting my access to Windows back.

Here's what I did:

Firstly, when Ubuntu loads up, mount the windows partition (as if it were just another hard drive). There is a folder boot (which may/may not contain a sub-folder called grub). However, this is of no consequence to us. Then there is another folder called "Boot" (capital 'B') which contains all the boot files.Rename the former ('boot', the not so important one) to XYZ and rename the latter ("Boot", more valuable) to 'boot'. Yes all we did was change the case of 'Boot' to 'boot'.

Now open a terminal, perform a simple 'fdisk':

Shell:~$ sudo fdisk -l /dev/sda

Device Boot Start End Blocks Id System
---------------------------------------------------------------------------------------------------------------------------
/dev/sda1 * 1 4080 32771576 7 HPFS/NTFS
/dev/sda3 8585 30401 175245052+ b W95 FAT32
/dev/sda5 4081 8323 34081866 83 Linux
/dev/sda6 8324 8584 2096451 82 Linux swap / Solaris

This is what I got. Here my Windows 7 is loaded on /dev/sda1 which is NTFS. Now create a file in /etc/grub.d/ called 11_Windows

Shell:~$sudo nano /etc/grub.d/11_Windows

Nano is my text editor of choice. You can use anything you like ('gedit' etc.). In this file, write the following:

#!/bin/sh -e
echo "Adding Windows">&2
cat<
menuentry "Windows 7"{
set root=(hd0,1)
chainloader --force+1
}
EOF

Save it. Close it and return to the terminal. Now let's recap. You made the "Boot" folder of your windows partition "boot" and created a file in the /etc/grub.d/ directory.

Now simply run the utility to update the grub. You can do this by entering the following at the terminal:

Shell:~$sudo update-grub

Output (on my system):

Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.31-20-generic
Found initrd image: /boot/initrd.img-2.6.31-20-generic
Found linux image: /boot/vmlinuz-2.6.31-14-generic
Found initrd image: /boot/initrd.img-2.6.31-14-generic
Adding Windows
Found memtest86+ image: /boot/memtest86+.bin
Found Windows 7 (loader) on /dev/sda1
done

Notice how the "Adding Windows" appears. That happens to be the cute little script we just wrote tipping the odds in our favor. And voila!! Windows 7 is found on /dev/sda1 (which we verified manually a little while back)

Thats all there is to it. Hope this helps.

Click, Drag and Drop widgets in wxPython

2010-02-24T03:57:00.004+05:30

Recently, I was running through a few posts at http://python-forum.org during my free time today and I came across someone who need help in creating a frame, where in the widgets (namely a push button) could be pressed and dragged around the frame, and dropped off at any part of the frame once it was depressed.

At first, I thought it was nothing to break my head over. In fact I wrote a little snipped of code in IDLE (on Windows Vista) and it worked! What was a bit disconcerting was the fact that the widget moved around alright, but didn't really flow smoothly along the frame. The transition from one point to the other was really jarring.

I have worked extensively on wxPython and know how faulty its behavior is on Windows. The true test for an application in wxPython is on a Linux box. I booted into my Linux partition, in a hope to recreate the effect that I had so easily created in windows. A few shell commands later, I could see the familiar window, but to my disappointment, the buttons didn't drag. They just sat there in one place, doing nothing.

Determined to solve this issue, I dug deeper into wxPython. Starting out with penning down the sequence of events that occur, I understood that we have to organize the events starting with the pressing of the left mouse button (only when the pointer is on the push button); followed by the drag/motion event of the mouse and finally concluding with the mouse up event.

However, what was challenging was the fact that once the mouse button was pressed down, the event loop didn't recognize any other mouse events on the button. So how does one go about recognizing the drag/motion event?? I tried to skip (event.Skip()) after receiving the mouse down event, but that didn't seem to work. After about an hour of google, I devised a mechanism to complete the desired effect.

After receiving the mouse down event, we turn on something called "mouse capture" - this focuses all the events coming from the mouse (motion, right click etc) to be directed to the frame and not to the button or the panel. That way, I was able to catch and handle the motion/dragging event. This was bound to the frame and not to any widget in particular. Once the dragging around of the button was complete, the mouse button is depressed causing the "mouse up" event to be released (again directed/bound to the frame, not to any widget in particular). In the handler for the mouse up event, I make a simple check to see if the mouse capture is turned on. If it is, then we simply "release capture" of mouse events, restoring the frame back to its original state.

Here is what the results looked like:

The source code for the same is available here (look for a file named wx_DragButtons.py).

The code contains the bulk of the documentation. This is a small, yet elegant solution, which only makes me wonder the kind of effort that goes into creating dynamic user interfaces. Alas, I hope the folks at the forum find this post useful.

Iron ‘Mic(rosoft)’ Python – Creating a simple app from ground up

2010-02-22T17:48:00.001+05:30

I have heard a lot about Microsoft’s initiative to include Python programming into their existing .NET framework. DotNET is selling like hot cake nowadays and almost every one I know writing professional application level software seems to have sold his soul to the devil.I have resisted this temptation time and time again, and perhaps, the inclusion of Python was what was needed to tip me over the cliff.

So before I signed the thousand page contract with Elizabeth Hurley, and watch her turn into a grotesque red faced monster, I decided to poke around and whet my appetite. I have been toying around with the idea of crossing over to .NET for a while, and I guess with the advent of IronPython, I had run out of excuses.

I started reading ‘Pro IronPython’ by Alan Harris. I always like to get my hands on some well organized reading material to get a feel of what I’m getting into. The book was surprisingly short (it was 300+ pages, but there was little worth reading in it!); and I skimmed through the nine chapters at leisure. The first five chapters were enough reading material for any experienced Python programmer to gauge what IPY was all about.

I downloaded “SharpDevelop”; an open source equivalent to Visual Studio, but only ten times lighter. Along with that I got a hold of the IronPython interpreter from http://codeplex.com/IronPython

That was perhaps the bare minimum that was needed to kick of some reptilian development. The next step is perhaps very important and I, like most programmers or computer geeks, like to delve into a new programming language by making an application of some sort. Something not too big but with a usage that spans many areas of programming. I had an idea loitering around in my cranium for a while, but sheer lassitude and lethargy was preventing me from coding it. I recently got a huge chunk of music from a friend, who was extremely meticulous in organizing his MP3 files. Over 16 GB of music was stored on my filesystem in multiple directories. Here’s how it looked:

The music files were segregated by language and then further classified based on the artist. So I had a whole load of files, stored in over 1000 folders (artist folders). I usually like my music stored in a single directory with the file name in this format :<artist name><hyphen><song>.mp3

Obviously, I wasn’t going to sit and manually cut and paste each file, and search utility on Vista is perhaps the lamest application ever. So I took it upon myself to create a small app to do this for me.

I had made a command line application for recursively searching, indexing and organizing my files before, using Python on Linux. I wanted to check if IPY would work with previously designed core python modules as well.

So, objectives for this application:

Get a feel of GUI design and associated generated code in IPY
Check if IPY plays well with core Python modules
Create an application that will allow the user to recursively search any directory for files of one or more extenstions/file types, and allow him to cut/copy all these files from their respective source directories to a specified destination
Understand IPY’s peculiarities with respect to worker threads and their independence of the UI threads
Assess convenience in application deployment

STEP 1 : Creating the GUI

Voila !!! EXODUS is born !!!

Download the source from my groups page (look for exodus.rar)!!

There was nothing to creating a GUI. The one thing (also the only thing) I’ve always loved about MS, was that they made GUI creation so simple. No long hours of coding… a few mouse clicks and my GUI was ready.

PITTFALL : There is however one problem with GUI creation and code generation in IPY. I didn’t face this while working with Visual Basic Express 2008. When you create a widget or tool (e.g. the push button, or a text box etc.); you must decide on the “name” of the object then and there. If you accidentally switch to the code view before appropriately naming the object, the code is generated for that widget/object with the standard names that the IDE gives them. So you’re going to have names like ‘textBox1’ and ‘button1’ sitting all over your GUI code. In Visual Basic, this wasn’t such a big issue, because you could go and rename the object later on, and the generated code would be changed accordingly… NO SUCH FEATURE WITH IPY!!! So name first, code later.

Barring that, there was no other problem in designing the GUI. You can pick out a widget of your choice from the toolbox panel (dockable… so look hard!)

The GUI code (auto generated) appears as a class with the class name as the name of the form. So I accidently ended up with a form called “MainForm” (default name given to the form) and the class created was also given the same name. This didn’t worry me so much as my application was only a single form application. But if you’re working on more than one form, things might get a bit difficult. So I re-iterate myself… name first, code later. In this form was a method titled InitializeComponent(self).

This method is very readable and neatly organizes the code for all the widgets that you have created. Some properties of these widgets are also assigned in this part of the code. This method is the first to be called from the class constructor or __init__(self) method.

STEP 2 – Creating the event handlers for widgets

The GUI is good to look at and that is all that it is worth, without any code a.k.a “Wiring”. My application had been designed keeping the following in mind:

Use at least four to five different widgets (buttons, radio buttons, checked style list box, text boxes, status strip with a progress bar)
There must be some kind of background task (a long running process), which in our case is the copying or moving of files from the source folder to the destination. This will help illustrate the effectiveness of the UI-worker thread independence
Use at least one file/folder/print/open/save dialog box – this will help us understand how dialog boxes return selected information
Use at least one message box or alert box to display passive messages to the user or draw attention to something

You can code a handler for a particular widget, or generate it by double clicking on it in the design view. Automatically, the code for the handler associated with that event and widget (e.g. click event for a button widget) will be generated. The body of this handler will be a simple “pass” statement. You have to write your code in that handler.

Note:

If you want to code an event handler, read this note. Consider a button widget with the name btnOK and you want to code the handler for a ‘Click’ event. Now all you have to do is bind the widget to its handler. Let’s also say that you have written a routine called ‘btnOKClicked(self,sender,event)’ which you want to invoke once this button is clicked.

To bind the method to the widget for the event, the code looks like this:

<widget_name>.<event_name> += self.<method_name>

In the above scenario, you’ll have:

self._btnOK.Click += self.btnOKClicked

The code generation scheme does exactly the same thing and saves you the trouble of doing it manually.

My application demanded that I’d be able to use a folder browse dialog box to select both source and destination directories; have a list box with commonly used extensions – for this I have used a checked list box so that one may select more than one extension; a text box is provided so that one may enter an extension (e.g: .rtf, .xyz etc) that is not present in the list – this allows flexibility since new file formats will come out every now and then and there is no way to know what type of files the user wants to move.

If you seek information about widgets and their associated members or methods, you can refer to any resource on C# or VB for the .NET framework. IPY implements the same methods as well. All the assemblies from .NET are available in IPY.

For instance the checked list box widget, implements a method called Add. Let’s assume that we have a widget with the name clbExtension (a checked list box). In Visual Basic, you’d probably access this as follows:

clbExtension.Items.Add(“Item number 1”)

In IPY, the usage is exactly the same. The only difference being that since there is a class in place for the entire form, all widgets on that form are members of this class. All event handlers are also part of the class for the form. Here’s how it goes in IPY, pay close attention to the last line:

So IPY preserves the same methods that are consistent with the rest of the .NET framework. If you’re familiar with widget objects and their members and have used them on VB or C#, you only have to master Python syntax and its libraries. Even if you’re not and are having a hard time finding resources specific to IronPython for a particular widget; fret not, all you have to do is find a VB or C# equivalent. That combined with a little common sense will have you coding away.

Please note, that in the snippet above, the only part that you actually code is the handler. The rest is generated by the IDE :-) just like it is in Visual Basic. Double click a particular widget or you can use the pull down menus at the top of the code view to generate an empty handler for a particular widget. Then simply feed in your code into that handler.

If you were paying close attention, you’d notice that in the snippet above, the component for the checked list box has been created and the code for the event handler has been put down. But there is no code to bind the handler to the widget. This is because, I typed out the snippet by hand. Be rest assured that the IDE will make no such blunders :D

If you’re familiar with Python syntax, you’d realize that the underscore (_) before the name of every widget object is used/generated to ensure that these widgets have private access only.

STEP 3 – Understanding a Background Worker

When designing GUI, one must understand, that any front end that responds to events (clicks, text entry etc.) has a process that runs in a continuous loop to sample these incoming events. This is commonly called the ‘UI Event loop’ or the UI thread.

If you interrupt this thread (run a really long for loop in the same thread), you’ll realize that the UI becomes unresponsive and freezes. To avoid this, all “heavy duty” work or long running or time consuming processes must run in separate threads that are concurrent with the UI thread. This will allow your application’s UI to remain responsive and prevent your operating system’s scheduler or process manager, from declaring your application as a passive process.

Here too, there are a few subtle techniques. Imagine a scenario with a form having a button. When clicked, the form starts downloading a really large file from the web, and reports its status using a progress bar. Another common feature is disabling this button when the download begins and enabling it when the download is complete.

Now, in order to do that you have to know the following:

when the download has commenced
the current status of the download
and when the download is complete

Since we have to run our download in a separate thread, to allow for the UI to be updated, the downloader thread must communicate with the UI thread. Each GUI toolkit offers some mechanism or the other for this communication to take place (PyQT for instance employs a signal and slot mechanism, it is one of the simplest and most effective mechanisms till date and offers the programmer complete flexibility). IronPython, like the remainder of the .NET framework uses something called a “Background Worker”

A background worker is basically an object with the following characteristics:

it has a property that stores the name of the routine which is the primary task viz. DoWork
it has a property that stores the name of the routine which must be invoked, when the primary task is complete viz. RunWorkerCompleted
and another property that stores the name of the routine which will update the UI on the progress of the primary task viz. ReportProgress (in my application I have not utilized this property. Instead I’ve directly invoked the routine to update the GUI)

Lets take a look at the code:

The first line creates a member called bgWorker which is an object of the type BackgroundWorker. The second line indicates that this worker object reports the progress of the task. The third and final line in the snippet above, indicate the two properties mentioned above. searchFilterAndMove & resetUI are two methods that perform the primary task (copying the files from source to destination) and reset the UI, once the primary task is complete (a cleanup task after the heavy work is over).

That was about the creation and initialization of the background worker. Now lets see how we can get some real work done. I have a button on my form that reads “MOVE IT!”; which when clicked, causes the movement of files from source to destination. Here is the event handler for that button:

The lines indicated the intermediary processing that happens before the background worker is told to start doing its work. The last line is the most important one. The RunWorkerAsync() method of the bgWorker object, causes the routine searchFilterAndMove to get activated. Moreover, this routine now runs asynchronously of the UI thread, thereby helping us achieve our objective. When the searchFitlerAndMove routine is complete, it will call the resetUI method as discussed earlier. The routine to update the GUI is called from within searchFilterAndMove.

You may look up the definitions of these routines to get a better understanding of how they work by downloading the source.

STEP 4 – Mixing up IPY and Core PY

At the beginning of this article, I had enlisted out the collaboration of IronPython with core Python modules as an important issue. Well, to say the least, its not smooth. But with a little work, anything is possible :-)

I had created a library (just another .py file) with some helpful functions. I call this module “helpermodules.py”

It contained a routine that would recursively scan a source directory for all its children, another routine to validate the extension (manual entry) and two routines to cut or copy files from one place to another.

In designing these, I had used modules such as os, shutils and glob.

I wanted to know how well the IPY interpreter would handle these. When I was working with the SharpDevelop IDE, I would use the in built compiler/interpreter, to run my code. Initially, while testing the widget related methods, all was well. But the moment I included routines that required core Python modules, a lot of errors started showing up. Therefore, I had a form with widgets that would look nice and seem hale and hearty on the UI front, but had little or no real functionality.

A few Google searches later, I realized that a lot of people were complaining about the use of various core Python modules. But I had come too far to turn back now. What was more disconcerting was the fact that imports of the core python modules worked brilliantly at the IPY interpreter shell, and failed miserably on the SharpDevelop IDE’s run. I finally found a post that bailed me out of my predicament. Apparently, there is a certain addition one has to make to allow IPY to use the core Python modules. Here it is:

The first two sections are what need to be added to the code generated by SharpDevelop. The logic of these statements is merely to provide a link between the IPY (which is the python interpreter) and the CLR (Common Language Runtime of the .NET framework). This way, all the resources that CLR (and hence, .NET) has to offer will be made available to the IronPython interpreter.

Another queer observation was that after this was done, the design file of the project (which is nothing but the design of the GUI which we have made by dragging widgets from the toolbox to the form etc.) was corrupted. Hence, SharpDevelop couldn’t display the design of my form. Thankfully, at this stage, I was through with the design of the UI, but if I needed to make a few changes later on, it would have been only through code and not through an interactive UI designer. I think this had more to do with the SharpDevelop IDE than with IronPython. Still trying to figure out why exactly did something likes this cause a ripple :D

Now this code file ran beautifully using the IronPython interpreter at the command line. The same effect can be achieved by running the project through SharpDevelop’s built in IronPython interpreter (Ctrl+Shift+W).

What is more important is that the core python modules worked flawlessly. The form takes a little time to pop up but that is simply because IronPython is built on C# and really doesn’t create an intermediate bytecode like Python does (which is also the reason for its superior performance).

There is a large part of core Python that is still not supported by IPY (there is a little dispute over this issue), but perhaps that will change in times to come.

My application is currently in its raw state and has not been deployed. I hope to understand deployment shortly and want to check out Microsoft’s IronPython Studio to see if they have “click once deployment” (like in Visual Basic 2008); that helps in creating a ready to use bundle for your application.

This project was purely for a didactical purpose. Hopefully, I was able to help you find what you were looking for. I will be immersing myself into IronPython in the days to come, and target more specific topics in forthcoming posts.

Summary

GUI design using SharpDevelop IDE
IronPython’s compatibility with Visual Basic.NET (or C#)
Creating event handlers
Using background worker objects
Mixing IPY with core Python

To download the source of the project, click here.