Gray Code Generator

I suppose I could finally talk about it!  I had my Google interview last July.  Was an embarassement (and to myself too).  This was the final question in my interview (with none other than Lars Rassmussen).  The question was the typical one – “there is a room from which at any time only one person can either enter or exit the room – and not at the same time.  So how would you model all the states the room can be in without repetitions”.

“Simple, isnt that just Graycode?” I blurt out.  At which point I was asked to write an algorithm to actually generate all graycodes for a n-bit string.   And this was where I floundered.  Mind you I had just been through 3 and a half hours and writing an algorithm was the last thing on my mind.  It wasnt that I hadnt known how to do it, I just blanked out then then and there – half an hour close to the finish of the interview and I had lost focus.  Stupid stupid me.

Anyway I just got back from a swim and for some reason the demon of the failed interview reared its ugly head and so I figured Id attempt it here (atleast to get some closure, sob sob).  Here goes.

Here are possible gray codes for N = 1, 2, 3 and 4:

N = 1	N = 2	N = 3	N = 4
0	00	000	0000
1	01	001	0001
	11	011	0011
	10	010	0010
		110	0110
		111	0111
		101	0101
		100	0100
			1100
			1101
			1111
			1110
			1010
			1011
			1001
			1000

Anyway this is only one combination. Any bit could be changed in generating the sequence. The pattern here is that the first half and the second half of the code are “kind of” the same but with the first bit being 0 and 1 respectively and the rest of the bits being reverse of each other.

ie with N = 2,
first half = 00 and 01
second half = 11 and 10

So a simple algorithm would be (in pseudo-haskell):

gc 1, reverse = ["1", "0"] if reverse, else ["0", "1"]
gc N, True =
      ["1" + x for x in gc(N - 1, True) ] ++ ["0" + x for x in gc(N - 1, False) ]
gc N, False =
      ["0" + x for x in gc(N - 1, False) ] ++ ["1" + x for x in gc(N - 1, True) ]

Problem with the above is the storage required in the recursion to store the graycodes for N – 1 (and that too twice). So in python we could avoid this with generators like:

def graycode(numbits, reverse = False):
    if numbits = 1:
        if reverse:
            yield "1"
            yield "0"
        else:
            yield "0"
            yield "1"
    else:
        if reverse:
            # all the "1"s start first
            gcprev = graycode(numbits - 1, True)
            for code in gcprev:
                yield "1" + code

            gcprev = graycode(numbits - 1, False)
            for code in gcprev:
                yield "0" + code
        else:
            # all the "0" start first
            gcprev = graycode(numbits - 1, False)
            for code in gcprev:
                yield "0" + code

            gcprev = graycode(numbits - 1, True)
            for code in gcprev:
                yield "1" + code

and finally an auxilliary function to print out the codes:

def gcgen(numbits, reverse = False):
    for codes in graycode(numbits, reverse):
        print "Code: ", codes

And that was 15 minutes of thinking. Damn it why didnt it come to me last year? Could have been the difference between a googler and foogler!!! Unless ofcourse it didnt really matter and they simply thought I was just a dick@#$D!!!

API Centric Design with Django and GAE

I wasnt really sure what to call this post.  My current situation is I write django apps (well so many words within that sentence can be loosely defined).  I guess after playing with django for last two and a half years (and Google AppEngine, GAE, ever since it came out) I am at a very interesting (or frustrating) crossroad.  What platform to choose and stick with – appengine or django.  Actually I had this conundrum about a year ago and I have essentially postponed this choice by what I call API centric design (ACD).  I am pretty sure ACD is just another fancy acronymn for existing techniques out there but still bear with me.

Django can be quiet easily used with GAE.  And infact that is what I have done.  I use django entirely within GAE.  The real impedence mismatch pops its ugly head thanks to the completely different backend schemes used by GAE and Django (key-value store vs relational DB).  As much as I am not a fan of the GAE, it is a pretty good platform for prototyping.  Well its free and hence, given the state of the competition, the best choice (for now) in the price range.   So how does one use GAE for prototyping an application, leaving it out there to collect feedback (and failing early and failing often) and move the application a platform of choice when the limits are reached and the constraints in GAE are just too much?  A redesign of an app to leverage non-GAE environments is certainly not the way to go.

What I have done is only expose an API from the app point of view.  Hide the back-end specific details behinds its own models.py (django uses models.py to hold the data models for an applications).  Its actually a lot easier than it sounds (if it already sounds easy, then it is even easier).  What is trickier is that one would have to exercise considerable discipline in following this.  Every call in the application should call only the exposed api instead of using the data-backend related calls provided by the platforms (GAE or Django or anything else).  Even leave the implementation and the optimisation to the specific backends.

As an example, let us take the famous “counter” example.  A typical way to have counters in traditional RDBMs based platforms is to have a table that stores the rows of name/counter pairs.  For DBMSes that are strong with write’s this is great.  GAE however recommends a different approach using shards.  Check Brett Slatkin’s video on this.  So how do you choose between these two?

Firstly settings.py needs to have a variable called USING_APPENGINE (or something similar) that will be set to True if the app is running in GAE and False otherwise (ie a native django app running on the test server or on other webservers).

Suppose our counters api is as follows:

def get_counter(counter_name)

def incr_counter(counter_name, byhowmuch = 1)

Now il need 3 files for the api – counters.py, djcounters.py and gaecounters.py.  Only counters.py will be imported in any other part of the application requireing counters.  djcounters.py will have the get and incr counter implementations for traditional RDBMBs and gaecounters.py will have the same for a GAE specific implementation.

counters.py will be:

from django.conf import settings

if settings.USING_APPENGINE:

from gaecounters.py import *

else:

from djcounters.py import *

Thats it.  In your calling app simply do:

import counters

counters.get_counter(“xxx”)

counters.incr_counter(“xxx”, 10)

This is only the beginning.  Clearly this extends to other types of backends as well (ie support TokyoCabinet in the future etc).  When writing an application you shouldnt have to worry about the optimisations.  Nothing stipulates that gaecounters.py will have to implement Brett Slatkin’s stragey when using counters in GAE, but having separate files allows for this optimisation in the future.  Let the api abstract it all away.

Ofcourse it may appear that this will explode the number of functions required in the API.  Not necessarily (ie functions having to bepeated to do filters or object creations or deletions).  Some of the common patterns are object creation, fetching an object by ID or by certain attrib/column values, deleting records.  There are others, but these are the most common ones.

So I have a base djhelpers and gaehelpers file which define object creations, deletions, searches for ANY kind of object class and each one is optimised (sort of) to that specific implementation.

For instance, in django objects can be created with the method:

def create_object(obj_class, **kwds):

    return obj_class.objects.create(**kwds)

while the GAE implementation would be:

def create_object(obj_class, **kwds):

    obj = obj_class(**kwds)

    db.put(obj)

    return obj

Note these are only basic ones.  Other implementations could go through the kwds list and get the parent and id parameters to set these on the object as required.  Alternatively, you could only save optionally and so on.

So an object creation in another part of the app is simply a matter of doing:

create_object(MyObjectClass, a = 1, b = 2, c = 3)

and the object is created without having to worry about the backend type.

This is again a simple way of hiding the implementation details.  The same could be implemented using metaclasses and properties to define a DBObject class whose metaclass would be the implementation specific class for the above and other object specific methods.

Sydney Django Users Group

Well we already do have a Sydney Python group.  So I was a bit apprehensive when I created the Sydney Django Users Group.  But my goal was not to compete with SyPy (actually most of our members are FROM SyPy and we regularly visit SyPy discussions – including one last night by Alex Dong on scaling web applications).

The main goal was to have a small group focusing purely on django activity in sydney.  So far so good.  Had our first meetup last tuesday.  Modest turn out and interesting talk by Sam Cavenagh and Shaon Diwakar.  Oh and thanks heaps for Shaon and Sam for actually pushing me to organise the meetup by getting me out of my lazy zone.

And here’s for lots more activity in the sydney django community.

StartupCampSydney II

Wow that was a doozer!

Last weekend was the StartupCampSydney.  A weekend where about 6-10 teams (each with 6-8 members) build, launch and pitch a Startup in one weekend.  Yep one weekend. To be honest I never thought it would work and when asked to be part of this one, I was trying to come up with an excuse to back out of it (before my wife who I hoped would have asked me not to go, urged me to participate…) …  the urging turned out to be a blessing.. that was one of the most intense, enriching, immersed and enlightening product development excercises Ive been in.

The sheer amount of talent from all backgrounds says it all.. and ofcourse meeting and working with so much talent just makes you do things that you never realised were possible.  Our startup is ThreeFeeds.com – a topic/news aggregator. Check it out, we would love your feedback (please forgive us in advance for the bugs).

Check out a full coverage of SCS by Ross Dawson.

Cant wait for the next one.

LunarProbe

At work I do games.   Again not the usual kind but the slot ones.  A good thing about where I work unlike others in the industry is that we actually have a bit of belief in technology and try to “catchup” with latest technology (usually mainstream – 5 years).  Now while this may be astonishing to some outside the industry, it is quite advanced within!  Anyhoo I am digressing.

Point was that with all this catching up we do, we recently (about 6months-1year ago) started using Lua to script a big chunk of the games so that only the core is handled by C++ and everything UI related is palmed off to Lua.  Obvious advantage is that extra builds are not required as compileable source code is not changed and configuration is a lot easier and lot less time consuming.  Again nothing new here.

The problem is that the lua scripts we invoke from C++ have no easy way of debugging them.  So our engineers had been toiling with this by using the first-class technique of peppering the code with print statements.  You would thing some methodologies would die eventually but nope.

So I figured why not simply have a debugger that takes advantage of Lua’s debug api and allow debugging of scripts remotely.  So was born LunarProbe.  My attemt at debugging lua scripts once embedded in a host language.  Even at the beginning I figured that this was a very generic thing and has no specific code for our games.  So I decided to release it on the Apache License.  You can find the code on google code.

So far it only supports C/C++ but hope to have more languages supported with the appropriate bindings in place.  The idea is simple:

1. The lua_State object (ie the lua stack or the lua context) is god in Lua.  Essentially it is what the interpreter uses to execute scripts and lua statements “on”.
2. A new lua stack can be opened with the lua_open command.
3. Usually following this command, you open all the standard libraries with the luaL_openlibs.

That is it.  You do usual things like loading lua files on a particular stack.  See the lua manual for more info on this.

So where does the debugging come in?  Well each lua stack can be given a debug hook function that is called by the interpreter after certain points at runtime. eg after a line has been executed, after a function is invoked, after a function has returned etc.

What LunarProbe does is registers itself as a hook function and deals with each of the above debug events and presents it via tcp (port 9999 by default) to a remote client.  The debug server is in fact api driven.  It accepts commands from the client and returns responses (but not HTTP – to keep it simple).  This means any UI could be written easily by simply following the debug server protocol.  A reference client is provided (written in python).  The python client is supposed to be like gdb (but no where near it!! GDB – please accept my apologies for even trying a comparison!!  I know I dont deserve it.)

I wanted to write a GUI or an Eclipse plugin, but we dont use eclipse plugin at work and wx widget toolkits werent installed by default here at work.  So doing either meant having all developers install extra packages which was a huge adminstrative overhead.  At a later stage, I would like to see more coming up.  Any helpers?

Again I digress.  So how would one enable LunarProbe in stacks they are opening?  Simple.

Instead of lua_open and luaL_openlibs, simply do this:

1. Include the lunarprobe header (ensure that lunarprobe/src is in the include path):

• #include “lpmain.h”

2. Instead of lua_open and luaL_openlibs simply do:

LUNARPROBE_NS::LuaUtils::NewLuaStack(true, true, “hello”)

This does a few things:

1. Creates a lua stack using lua_open
2. If the first parameter is true, then Opens standard libraries using luaL_openlibs.
3. and if the second parameter is true, registers LunarProbe by calling LUNARPROBE_NS::LunarProbe::Attach(stack, name), where “stack” was opened in step 1.
4. The fourth parameter (defaulting to “”) only gives a name to each of your stacks for debugging purposes.  This is very useful in the debug client to identify which stack your dealing with when the address of a stack may not be very useful or very clear.

Alternatively, you can simply call the step 3 instead of NewLuaStack if you do not want to pull out your own versions of lua_open and luaL_openlibs.

That it. This is the embedding/attaching part of the debugger.  Now the client.

The Client

lunarprobe/client includes a simple client – client.py.

I wont go into too much detail here as the “help” command is mostly self-explanatory.  Just the brief stuff.

Invoke this with:

• python client run

or if you would like to run it from within an interpreter then:

• import client
• client.run()

This brings up the LDB prompt: type in “help” to show all commands and their usage.

One distinction (actually an inconvinience) is that since multiple stacks can be debugged at the same time, any context (LP calls stacks as contexts) specific commands must pass in the address of the context.  This is not hard, just inconvinient.

To get a list of all the contexts simply type in the contexts at the prompt and press enter.  This will show all context names and their addresses.  Use these addresses with any context related commands (like step, next, continue etc).

Breakpoints at the moment can be set by filename (and linenumber) or by a function name.  However at this stage member functions are not differentiated from global functions.  So a function name breakpoint will affect ALL functions of the same name.

As mentioned this is very brief.  Apologies for that.  Il add more and more in as I improve this.

Well hopefully this project is useful for all Lua embedders out there.  Please let me know how I can improve this.  Also this project is on google code, so please let me know if you would like to contribute or extend LP.

Best of luck!!!

End of 2008

And another short one after ages.  Just a summary of what I had been upto.

August – Had my exam for Marketing Management and Stats.  For some reason i find that more I learn in a subject the worse I do in the exams.  I really need to learn to write academically!  (Yes I know it sounds like a looser and that most likely must be it!)

September – 3 weeks in India.  Wow Wow Wow!  Entrepreneurship is just springing up.. (ok the financial climate is not so great but that is really an issue of the system rather than the product/environment)… After “the” valley India is the place to be.  Actually not entirely true.  Even though many wouldnt agree with me, Id say china is still leading in that aspect as people are a lot more risk taking than in India but hey Ive never been to china so it is a very very brave and untested statement.

October – 2 weeks in HongKong for one of my subjects (Private Equity and Venture Capital)..  amazing course.. only sad thing was that I had to go all the way to HK to do a course in entrepreneurship, which was not offered by AGSM in sydney… what does that really say about the apetite for entrepreneurship in australia?  But loved HK – the city and the people… I cant say the same about the food.  See I am a vegetarian and this is something not easily catered.  Also I was a bit dissapointed with the prices.  Not exactly as cheap as I hoped them to be.  Oh well..

November – Had my exam for the PE and VC subject..  confirms my theory about learning being inversely proportional to my mark… well havent got my mark yet but not holding my breath for a great mark… damn

December – Had 2 concerts yesterday for SMC.  Tiring but lots of fun.  One more in Brisbane on the 13th and one possibly in melbourne during christmas.  Have to see.  Both very challenging and testing.  One at a time.

Ciao till later.  Will be updating on a lot of programming stuff Ive been hacking away on.

HongKong

Well I am in HK now..  Doing an elective on Private Equity and Venture Capital.  My first time here and the place is breath taking….  Ok the weakened Ausie dollar has just robbed a bit of my ability to buy but still a great place….

The course is very insightful… if you follow VC news on techcrunch and a whole bunch of blogs from the big guns in the Valley (no i am just a follower not a propogator), this course is very very complementary.  Ofcourse it is quite sad that we (students) in the Australian campus of AGSM have to resort to travelling overseas to undertake a course in anything entrepreneurial.  Seems the Australian focus is on management training more than anything else.  Could that explain the pitiable state of entrepreneurship in Australia?

I am Rich (I Wish)

Well wanted to write this last week, but wanted to read more about the reactions to Apple yanking out the “I am rich” application from its store.

Apple’s reaction to the app is hardly surprising, after all the company does have a reputation for masking its dubious actions of cencorship and spoon feeding its customers under the blanket motto of “quality”.  What was surprising (may be less so given the psyche of the iStuff customers) is the customers reaction.  All I have heard all around are “kudos Steve, great work.”

Forget cencorship, are we actually planning to get rid of the words “Common Sense” out of our vocabulary and lives for ever?  What ever happened to Buyer Beware?  Let me go through the arguments Ive heard.

1. It is Apple’s store and they can do what ever they want.

Ok No argument here.  I mean if M$ can do what they want and get away with it, why shouldnt Apple?  If this is all we care about, then stop reading right here.

2. You dont find $1000 news papers in a news-stand so why should we have them in the AppStore?

Marketing and Economics 101.  A news agent, has to prioritise the shelf space to maximise his/her profits.  A good indication of this is the customer demand for a product, not ethical or moral values.  You dont find $1000 news papers in a news stand simply because the market segment looking for $1000 news papers is not the one being targetted by the news stand.  If you are looking for expensive magazines try this.

3. Dilutes the quality of the App Store

Welcome to economics again.  Ever heard of supply and demand?  Ever heard of transaction costs? It was not exactly hard to “find” the product.  The App store enables you to search by price and (I am assuming) a few other criteria.  So we have 0 transactions costs for the users OR Apple (never mind that Apple made a cool 30% of the app!).  In an open market (which is what this SHOULD be) the best would bubble to the top and crap like this sinks to the bottom (ok I suddenly have a hankering for this app and I dont even have an iPhone!).

4. People are being fooled and tricked.

How?  Was the price not clear?  Was the product description not clear?  Sure the app is useless.  But was its uselessness ever a secret and mis-communicated?  The 8 who have actually bought the app, knew exactly what they were getting into (by the way how many of them are actually complaining about it?).

5. Mockery of the store

Wow another insightful argument for masking the envy at good (well ok going a bit far there) entrepreneurship.  So suddenly we are ok with Louis Vuitton bags (known mainly for their durability?) but an attempt at brand management and positioning (towards deep-pocketed) users is wrong and unethical and evil and dodgy (and a whole lot of other equally _____ adjectives).  Perhaps it is time we (developers) start making apps targeted at user needs (refer to PG’s “make something users want” mantra for this)?

————————

All said and done fellas, the marketing hype for the AppStore and Heinrich (the app’s author) have been phenominal.  Even with a sale to 8 customers, thats a massive revenue of 80,000.  Wow.  Kudos.  This is just capitalism guys.  Get used to it.  The thing with idiots is, if you cant beat them, you just take advantage of them.  And kudos again to Heinrich for doing just that.  Dont try to beat this guy, join him!!

The other reason I am excited about Apple having been foolish in dealing with this App, is that the need for a better market place for the iPhone is becoming more pronounced.  How long can Apple’s cencorship and treatment of its customers as children help insulate (and incubate?) its customers’ stupidity?

Google App Engine

Well ive been spending the last week and a bit with my first AppEngine project.  I had recieved my account on the day it was released (yes a shameless boast there )…  Unfortunately due to exams never got around to doing anything with it.

So as a simple project I figured Il write (yet another online) chess game – WizChess.  Note that there is already a Blitz Chess as part of the appengine samples page.  I wanted to write one just to get a feel for the engine and to largely to learn a lot of things myself.  There are a still a few things I have not implemented (eg on a castle, the server updates, but the client does not properly update UI.  en-passante-s are not implemented.  Timed game not hapening yet, chatting between players not yet happening and so on).

It was a very interesting experience.  The documentation was quite nice and full, but still I was not fully satisfied.  For some reason I found django a lot more satisfying as a web-app dev platform.  For the models for the game I had chosen to follow a more django style (relational) rather than harnessing the rich model extensions provided by appengine (mainly the Expando Models).  I had done this as I wasnt sure if I wanted to seal my app purely to the AppEngine platform.  And after finishing the project, I am still not sure that I do.

I had got a whole bunch of puzzle games from here.  Now I had needed a way to update the database with an initial data set.  So my way was to have a url on the webapp (something like /resetdb) that would load the initial board templates (and in the future games that are saved and so on).  However, since this loading would take some time, I had to resort to breaking this up into several parts and calling each of these individually.  eg:  Id nomally have:

class ResetDB(webapp.RequestHandler):
    def get(self):
        for board in my_boards:
            add_board_to_db(board)

Now as the number boards in my data set increased, the time to put them in the data store would naturally increase.  But there seems to a time limit on each request, which would make my request fail.  So I had to resort to writing a client side script that would call ResetDB with parameters (eg, firstboard and lastboard) and then also break em up as more saved games were restored etc.  Absolute Pain!  Its all good for appengine to be smart about things but to impose assumptions severely limits the usefulness of this platform and I hope in the future this is released.

Next thing was the model apis.  The Expando model is awesome, but once again I was not happy about the lockin.  But this was not my main concern.  The querying api is extremely poor.  Django has amazing querying facilities where one can specify pretty much arbitrary query filters (not a and c or a minus x and so on).  AppEngine does not support “not”s in its query.  Aaaaaaaaaaaaaaaaah.   Also I couldnt find an easy way of chaining queries.  Eg, to find a game, where either player0 or player1 (my way of saying white or black) was user, I had to do this:

games_with_user = []
games = models.Game.all()
games_with_user.extend([g for g in games if g['player0'] = current_user])
games_with_user.extend([g for g in games if g['player1'] = current_user])

Yuck!  Same thing in Django happens with:

games_with_user = models.Game.filter(Q(player0 = current_user) or Q(player1 = current_user))

(ok I did that syntax from memory so it may not be right, but it is something similar).

Now the final that REALLY annoyed me was lack of Comet (or any reverse ajax) support.  Essentially I poll the server every 5 seconds to get updates on whether the opponent has played.  Would have been great for the server to notify the client instead of having to keep polling and waste requests (I believe there is a limit of 6.5 million requests a day or is that 650K requests?)

But having said all this, the main (only?) advantage of the platform is the fact that we dont have to worry about scalability.  Unfortunately this is a HUGE advantage.  Normally I could write an app, and host it on my own boxes at home (if i want it to be free), but then il soon be hitting limits with 10 users and Il have to find a hoster and also handle scalability issues myself.  So migrating my own code to appengine at THAT point would be quite a nightmare.  By writing and prototyping it on appengine allows me to fail fast.  Looking at the pricing, app engine would charge about $40 per month per app (with reasonable traffic).  If my app ever got to that stage, I could always port it out of appengine (as long as I dont use Expando and stick to django style object modelling – with which I dont loose any power).

Please let me know if you would like the sources.  I am not sure how to upload files to wordpress here.

Also I would really appreciate it if you could give me feedback on my learning here.  As a python and web-app noob, Id be grateful for pointing any errors in my ways

Parur Style Techniques

Well I love Parur style violin.  Full-stop.  I would kill to master it.  It is extremely demanding and needs a kind of discipline that would make the marines look like amateurs at Paintball!

I have had the extreme good fortune of being pointed to a few clips on Youtube by Akella Mallikarjuna Sharma.  Here they are:

http://youtube.com/results?search_query=msakella

Hope you benefit from these.  Good luck.

Inbreeding – The Valley Style

So Marc Andreessen joins the Board of Facebook.  (Advance note of jealousy towards Mark Zuckerberg for getting someone of Marc’s stature.  Kudos).

What I find surprising is will there be any input from Marc on FB’s technical strategy?  I am specifically concerned about Marc’s Ning using OpenSocial and FB using its own FB API.  Will we see a merging of the two?  Surely it would be stupid (a quality I possess in abundance) to think FB would abandon FB API in favour of Ning for better tapping into Ning and its countless social networks hosted within.  However one cannot help see the allure in a wrapper to FB API to be OpenSocial compatible?

Or is this issue too puny and insignificant to deserve a mention?

The Greatest Entrepreneurs

The 80s and 90s, saw a huge influx of Indian migrants to Australia (the trend still continues).  Now there is something different about these migrants.  Now what is odd about this class of migrants is most of them were in their mid 30s to mid 40s, with very comfortable lifestyles back home (by which I mean respectable jobs/positions resulting from years of hard work).  Why I think this is odd is that you would not expect people/couples with such low risk lifestyles to risk it all and jump ship to sail to the uncertain shores of a foreign land.

It is pretty easy (or comforting atleast) sitting in your parents’ basement coding or being sponsored to spend summers on startup ideas sponsored by THE man PG himself (an opportunity Id kill for), but leaving all you have with very little promise of great rewards, purely for the sake of your family, is what I consider true entrepreneurship (well children are the ultimate startups anyway).  Costs of failures are very high (hardship for entire family, loosing face, etc).  Rewards are not staggering except that life for one’s family is slightly better.

I would like to thank two such migrants who risked very comfortable jobs for the sake of their two sons.  Myself and my brother.  Mum was a very respected high-school teacher (she was my math teacher – which could explain my interest in maths).  Dad was a successful accountant.  Like others they had left it all for us.  The main reason was reverse-discrimination in the society gripped by the plague of the caste system, existing only to empower already corrupt politicians (redundant adjective). Times were hard for the first two years – time they had spent unemployed, living on government assistance.  But I am glad for those times.  It had instilled in us a passion and respect for hardwork.  Cant complain at all.  No room for becoming spoilt brats.  In fact I wonder if I can ever replicate their own dedication and values on my own kids (one day).  Atleast I know that I dont have to look far in search of role-models!

So thanks mum and dad (hope this combined post suffices for mothers’ and fathers’ days).

gDay Mate

Well, no exception. Google dazzles us all with their supreme Mate technology – short for Machine Assisted Temporal Extraction.  Its the beginning of Future searches!!!  Just put in the keywords and search for items that may appear tomorrow!!

I was a bit skeptical to see how it actually works but man its amazing!  Looks like I wont have to be working ever again!!

Here’s the link!!

http://www.google.com.au/intl/en/gday/index.html

Android Hack – Parcellable in eclipse

Long story short – I have been playing around with android.

wow. Amazing platform. Ok so the sdk is not all that well documented but it is getting better by the day. Ofcourse due to its young age, it is nowhere near the flex sdk in terms of the ease of use, documentation and ide, but hey a great start. And there are tons of samples out there ofcourse.

Anyhoo today is about one of the many problems and a fix I stumbled across – Including Parcelables in eclipse projects.

Background:

It is very easy to create remote services in android. A remote service runs in the corner handling requests by activities (or other processes in general) that need to use them (essentially server like processes). By following the tutorial above, you will find that “aidl” files come in 2 flavours.

1. Include the interface definition of an interface.

2. Contain a single statement “parcelable xxxx”, indicating that class xxxx can be marshalled and unmarshalled (into Parcel objects) so instances of xxxx can be communicated between processes.

With step 1, an aidl file YYYYY.aidl automatically gets built into YYYYY.java which contains a lot of stub and other helper functions to take care marshalling and remote method invocation and yada yada yada. This seems to be an automatic thing with eclipse.

With step 2, the .java already exists, and hence the need to use the “parcelable” keyword to indicate that “there exists a corresponding XXXX.java that is already parcelable so do NOT create a .java file again”, but eclipse does not understand this.

Worse still, it is intuitive to place the aidl files along with all your other sources. The fix involves NOT doing this.

Your project will have usually have the following structure:

<project>

—- .classpath

—- .project

—- .settings/

—- bin/

—- src/

——– package1

————file1.java —> This is parcellable

————file2.java

——– package2

————interface1.java

————class2.java —-> This is parcellable

and so on

the key is to add another folder at the same level and structure as the “src” folder(I called mine aidl).

So the new structure looks like this:

<project>

—- .classpath

—- .project

—- .settings/

—- bin/

—- src/

——– package1

————file1.java —> This is parcellable
————file2.java

——– package2

————interface1.java

————class2.java —-> This is parcellable

—- aidl/

——– package1

————file1.aidl —> This is parcellable
——– package2

———— class1.aidl —-> a sample interface definition with no corresponding .java files ANYWHERE

————class2.aidl —-> This is parcellable
the aidl files that exist in your src folder, have their corresponding aidl files in the aidl folder in the same package structure.

Now good news is since the aidl folder exists in your <project> folder, it will be picked up by eclipse (on a refresh). However there is still one minor issue. aidl file class1.aidl will not be automatically compiled to class1.java. To do this, we need to add the entire aidl folder as a “source” folder. Simple. do this:

add an entry like so in the .classpath file!! (ah ha)

<classpathentry kind=”src” path=”aidl”/>

This essentially says, “i have another folder called ‘aidl‘ that is a source folder”.. voila, being a source folder it will get included in auto-builds.

Beauty of this is that for class2.aidl and file1.aidl (in your aidl folder), no .java file will be created as they already exist, however class1.java will be built.

Saves a huge pain in the rear side (i couldnt spell “posterior”)!

good luck. please let me know if ive missed something.

PS <08/03/2008>

Please see Comment # 2 for further clarifications (and speculations too).  Thanks for trickybit (Comment #1) for inspring that!

Google Captcha got my tongue!!

AAAAAAAAAAAAAAAAAAAAAARRRRRRRRGGHHHHH

Hate captcha in the google maps.  I can understand how it needs to protect itself against spam, but this is ludicrous.  I was playing with the google maps api writing a simple google maps mashup.  This meant editing my js file and reloading the pages a few times.  Somehow google flagged this as a sequence of spam requests.  So it shows me the dreaded captcha page.  I enter the damn captcha.  You would think a company like google would be smart enough to let you get on with your work.  Oh no, it brings up the captcha page again.  May be I got the captcha wrong originally so I tried again, no luck.  Now onto my 100th try in a row and still no luck.  Google still thinks my requests (from a FF3 browser) are all spam requests.

Is this the sign that gOoGle is finally going the MS way?

gprobe – Json Parser

Here is the code for the json parser I had described in an earlier post.  It is still to be fully tested (works for our purpose right now).

Essentially data is fed into the parser continuosly (as there are no blocked reads in flex from the socket).

Usage is:


var parser:GPJsonParser = new GPJsonParser()
parser.processBytes(byteArray)
var nextObj:* = parser.next();
while (nextObj != null)
{
// do something with nextObj
nextObj = parser.next();
}


nextObj will only be non-null if there were no syntax errors and if enough bytes were consumed to produced a complete json fragment.

package gprobe.utils
{


import flash.utils.ByteArray;/*
* On-demand parser for parsing json from a stream
*/
public class GPJsonParser
{
private static var STATE_EOF:int                  = -1;
private static var STATE_START:int                = 0;
private static var STATE_READING_IDENTIFIER:int    = 1;
private static var STATE_READING_NUMBER:int       = 2;
private static var STATE_READING_STRING:int       = 3;

private static var OBJ_NULL:int         = 0;
private static var OBJ_BOOL:int         = 1;
private static var OBJ_NUMBER:int       = 2;
private static var OBJ_STRING:int       = 3;
private static var OBJ_LIST:int         = 4;
private static var OBJ_STRUCT:int       = 5;
private static var OBJ_COMA:int         = 6;
private static var OBJ_COLON:int        = 7;
private static var OBJ_OPEN_BRACE:int   = 8;
private static var OBJ_OPEN_SQUARE:int  = 9;
private static var OBJ_CLOSE_BRACE:int  = 10;
private static var OBJ_CLOSE_SQUARE:int = 11;

/*
* Current string delimiter only valid if we are
* currently reading a string
*/
private var currDelim:int;

/*
* Current string - only if we are reading a string
*/
private var currString:String;

/*
* State in reading current string
*/
private var currStringState:int;

/*
* The byte buffer
*/
private var byteBuffers: Array = new Array();

/*
*
*/
private var currBuffer: ByteArray;

/*
* Number of bytes read since start of object
*/
private var bytesRead: int;

/*
* Current parser state
*/
private var currState: int;

/*
* How many unmatched open square brackets have we encountered?
*/
private var squareCount: int = 0;

/*
* How many unmatched open braces have we encountered?
*/
private var braceCount: int = 0;

/*
* Current parent object
*/
private var objStack: Array = null;

private static const CHAR_BSLASH:int = '\/'.charCodeAt(0);
private static const CHAR_DQUOTE:int = '"'.charCodeAt(0);
private static const CHAR_SQUOTE:int = '\''.charCodeAt(0);
private static const CHAR_PLUS:int = '+'.charCodeAt(0);
private static const CHAR_DOT:int = '.'.charCodeAt(0);
private static const CHAR_a:int = 'a'.charCodeAt(0);
private static const CHAR_A:int = 'A'.charCodeAt(0);
private static const CHAR_e:int = 'e'.charCodeAt(0);
private static const CHAR_E:int = 'E'.charCodeAt(0);
private static const CHAR_t:int = 't'.charCodeAt(0);
private static const CHAR_v:int = 'v'.charCodeAt(0);
private static const CHAR_b:int = 'b'.charCodeAt(0);
private static const CHAR_f:int = 'f'.charCodeAt(0);
private static const CHAR_r:int = 'r'.charCodeAt(0);
private static const CHAR_n:int = 'n'.charCodeAt(0);
private static const CHAR_z:int = 'z'.charCodeAt(0);
private static const CHAR_Z:int = 'Z'.charCodeAt(0);
private static const CHAR_0:int = '0'.charCodeAt(0);
private static const CHAR_9:int = '9'.charCodeAt(0);
private static const CHAR_SLASH:int = '\\'.charCodeAt(0);
private static const CHAR_SPACE:int = ' '.charCodeAt(0);
private static const CHAR_MINUS:int = '-'.charCodeAt(0);
private static const CHAR_UND:int = '_'.charCodeAt(0);
private static const CHAR_TAB:int = '\t'.charCodeAt(0);
private static const CHAR_CR:int = '\r'.charCodeAt(0);
private static const CHAR_LF:int = '\n'.charCodeAt(0);
private static const CHAR_COMA:int = ','.charCodeAt(0);
private static const CHAR_COLON:int = ':'.charCodeAt(0);
private static const CHAR_OSQ:int = '['.charCodeAt(0);
private static const CHAR_CSQ:int = ']'.charCodeAt(0);
private static const CHAR_OBRACE:int = '{'.charCodeAt(0);
private static const CHAR_CBRACE:int = '}'.charCodeAt(0);

/*
* Constructor
*/
public function GPJsonParser()
{
reset();
}

/*
* Resets the parser
*/
public function reset(resetBytes:Boolean = true): void
{
objStack        = [];
currState       = STATE_START;
bytesRead       = 0;
currString      = "";
currStringState = 0;
squareCount     = 0;
braceCount      = 0;
if (resetBytes)
{
byteBuffers = new Array();
currBuffer  = null;
}
}

/*
* Process more bytes.
*
* Returns nothing.  Call next() to retrieve the next object
* from the stream (if any).
*/
public function processBytes(data:ByteArray): void
{
byteBuffers.push(data);
}

/*
* Return the next object from the stream.
* null if none (this is not necessarily an error, just means we do
* not have enough bytes given to us to extract the next object).
*/
public function next(): *
{
var currCh:*;
var readChar:Boolean = true;

while ( ! (objStack.length == 1 && squareCount == 0 && braceCount == 0))
{
if (currBuffer == null || currBuffer.bytesAvailable == 0)
{
if (byteBuffers.length == 0)
{
return null;
}
currBuffer = byteBuffers.pop();
}
if (readChar)
{
if (currBuffer.bytesAvailable == 0)
return null;
bytesRead++;
currCh = currBuffer.readByte();
}
else
{
readChar = true;
}

if (currState == STATE_READING_STRING)
{
processStringBytes(currCh);
}
else if (currState == STATE_READING_IDENTIFIER)
{
if ((currCh >= CHAR_a && currCh <= CHAR_z)         ||
(currCh >= CHAR_A && currCh <= CHAR_Z)     ||
(currCh >= CHAR_0 && currCh <= CHAR_9)     ||
currCh == CHAR_UND)
{
currString += String.fromCharCode(currCh);
}
else
{
if (currString == "true")
{
shift(OBJ_BOOL, true);
}
else if (currString == "false")
{
shift(OBJ_BOOL, false);
}
else if (currString == "null")
{
shift(OBJ_NULL, null);
}
else
{
shift(OBJ_STRING, currString);
//throw new Error("Invalid identifier: " + currString);
}

currState = STATE_START;
readChar = false;
}
}
else if (currState == STATE_READING_NUMBER)
{
if ( ! processNumberBytes(currCh))
{
readChar = false;
}
}
else if (currState == STATE_START)
{
if (currCh == CHAR_SPACE || currCh == CHAR_TAB ||
currCh == CHAR_CR || currCh == CHAR_LF)
{
// ignore white spaces
}
else if ((currCh >= CHAR_a && currCh <= CHAR_z)         ||
(currCh >= CHAR_A && currCh <= CHAR_Z)     ||
currCh == CHAR_UND)
{
currState = STATE_READING_IDENTIFIER;
currString = String.fromCharCode(currCh);
}
else if (currCh == CHAR_MINUS || (currCh >= CHAR_0 && currCh <= CHAR_9))
{
currState = STATE_READING_NUMBER;
currStringState = 0;
processNumberBytes(currCh);
}
else if (currCh == CHAR_SQUOTE || currCh == CHAR_DQUOTE)
{
currDelim = currCh;
currString = "";
currState = STATE_READING_STRING;
}
else if (currCh == CHAR_COMA)
{
shift(OBJ_COMA);
}
else if (currCh == CHAR_COLON)
{
shift(OBJ_COLON);
}
else if (currCh == CHAR_OSQ)
{
squareCount++;
shift(OBJ_OPEN_SQUARE);
}
else if (currCh == CHAR_OBRACE)
{
braceCount++;
// shift(OBJ_STRUCT, new Object());
shift(OBJ_OPEN_BRACE);
}
else if (currCh == CHAR_CSQ)
{
squareCount--;
shift(OBJ_CLOSE_SQUARE);
}
else if (currCh == CHAR_CBRACE)
{
braceCount--;
shift(OBJ_CLOSE_BRACE);
}
}
}

if (objStack.length == 1)
{
var obj:Object = objStack.pop();
reset(false);
return obj['value'];
}

return null;
}

/*
* Shifts a token onto the stack and immediately reduces it (the
* stack)
*/
private function shift(tok:int, value: * = null): void
{
if (tok == OBJ_CLOSE_SQUARE || tok == OBJ_CLOSE_BRACE)
{
reduce(tok);
}
else
{
objStack.push({'type': tok, 'value': value});
}
}

/*
* Reduces the parse stackby utilising the next token.
* Reduction can happen more than once.
*/
private function reduce(topType:int): void
{
var t1: *;
var t2: *;

if (topType == OBJ_CLOSE_SQUARE)
{
// we are building a list
var newArray: Array = new Array();
while (objStack.length > 0 &&
objStack[objStack.length - 1]['type'] != OBJ_OPEN_SQUARE)
{
t1 = objStack.pop();
if (t1['type'] == OBJ_COMA)
t1 = objStack.pop();
newArray.push(t1['value']);
}
if (objStack.length == 0 || objStack[objStack.length - 1]['type'] != OBJ_OPEN_SQUARE)
{
throw new Error("Could not find '['");
}
objStack.pop();
newArray.reverse();
objStack.push({'type': OBJ_LIST, 'value': newArray});
}
else
{
// we are building a map
var newObj: Object = new Object();
while (objStack.length > 0 &&
objStack[objStack.length - 1]['type'] != OBJ_OPEN_BRACE)
{
t1 = objStack.pop();
if (t1['type'] == OBJ_COMA)
{
t1 = objStack.pop();
}

t2 = objStack.pop();
if (t2['type'] != OBJ_COLON)
{
throw new Error("':' expected but not found");
}
t2 = objStack.pop();

newObj[t2['value']] = t1['value'];
}
if (objStack.length == 0 || objStack[objStack.length - 1]['type'] != OBJ_OPEN_BRACE)
{
throw new Error("Could not find '{'");
}
objStack.pop();
objStack.push({'type': OBJ_STRUCT, 'value': newObj});
}
}

private function processNumberBytes(currCh: *): Boolean
{
var NUM_START:int               = 0;
var NUM_READING_INT:int         = 1;
var NUM_READING_DOT:int         = 2;
var NUM_READING_FRAC:int        = 3;
var NUM_READING_EXP:int         = 4;
var NUM_READING_EXP_DIGITS:int  = 5;

if (currStringState == NUM_START)
{
currString         = "";
currStringState = NUM_READING_INT;
if (currCh == CHAR_MINUS)
{
currString     = '-';
return true;
}
}

if (currStringState == NUM_READING_INT)
{
if (currCh >= CHAR_0 && currCh <= CHAR_9)
{
currString += String.fromCharCode(currCh);
return true;
}
else
{
currStringState = NUM_READING_DOT;
}
}

if (currStringState == NUM_READING_DOT)
{
if (currCh == CHAR_DOT)
{
currString += '.';
return true;
}
else if (currCh == CHAR_e || currCh == CHAR_E)
{
currString += 'e';
currStringState = NUM_READING_EXP;
return true;
}

currStringState = NUM_READING_FRAC;
}

if (currStringState == NUM_READING_FRAC)
{
if (currCh >= CHAR_0 && currCh <= CHAR_9)
{
currString += String.fromCharCode(currCh);
return true;
}
else if (currCh == CHAR_e || currCh == CHAR_E)
{
currString += 'e';
currStringState = NUM_READING_EXP;
return true;
}
}

if (currStringState == NUM_READING_EXP)
{
currStringState = NUM_READING_EXP_DIGITS;
if (currCh == CHAR_PLUS)
{
currString += '+';
return true;
}
else if (currCh == CHAR_MINUS)
{
currString += '-';
return true;
}
}

if (currStringState == NUM_READING_EXP_DIGITS)
{
if (currCh >= CHAR_0 && currCh <= CHAR_9)
{
currString += String.fromCharCode(currCh);
return true;
}
}

currState = STATE_START;
shift(OBJ_NUMBER, Number(currString));

// invalid char, must be end of numeric
// value so let the caller take care of the
// rest by not consuming the character
return false;
}
private function processStringBytes(currCh: *): void
{
var STR_NORMAL:int = 0;
var STR_SLASH:int  = 1;

if (currStringState == STR_SLASH)
{
currStringState = STR_NORMAL
if (currCh == CHAR_SLASH)
{
currString += '\\';
}
else if (currCh == CHAR_t)
{
currString += '\t';
}
else if (currCh == CHAR_n)
{
currString += '\n';
}
else if (currCh == CHAR_r)
{
currString += '\r';
}
else if (currCh == CHAR_b)
{
currString += '\b';
}
else if (currCh == CHAR_f)
{
currString += '\f';
}
else if (currCh == CHAR_BSLASH)
{
currString += '\/';
}
else if (currCh == CHAR_DQUOTE)
{
currString += '"';
}
else if (currCh == CHAR_SQUOTE)
{
currString += '\'';
}
else
{
currString += ("\\" + currCh);
}
}
else if (currCh == currDelim)
{
currState = STATE_START;
shift(OBJ_STRING, currString);
}
else
{
currString += String.fromCharCode(currCh);
}
}
}

}
Essentially, since data is read asynchronously, it needs to be "fed" data as it is recieved by calling

gprobe – Part 1 – Message Formats

Message transport (between the gprobe, the game server I had written for my work in my spare time and the front end client) was using json or  Action Message Format, AMF.  Reason for the duality, was that for quick testing, I could login into the server with simple telnet and send (textual) commands and recieve data in json format – Simple.  However, I could not find a built in JSON parser for Flex and writing one (again, as I had already written one in C++ for the server side) – didnt just seem too appealing.  Luckily, Flex is capable of serializing and deserializing objects in binary format over a stream – yep – the Action Message Format or AMF.  Adding the AMF serializer to the server was fairly simple (did take few hints on the integer encoding from pyamf – thanks guys).

Now the problem was that I could not find a way of debugging the decoding (on the flex side) of the AMF stream I had generated from my server.  Why do I need debugging you ask?  Well, I was having random errors in the AMF stream (no doubt due to my encoding) while encoding strings.  Due to time-constraints I had decided to switch to JSON.  This meant I had to write a json parser for flex (as I could not find one at the time).  Not too bad as it gave me a lot of insight into what was being sent and so on.  One day when I earn more free time at work I may decided to debug my AMF encoder.

Happy 3rd of January

Happy new year and a happy 3rd of January to all.  No, nothing significant about that date except that it happens to be the date of my first blog (hopefully of hundreds, cough cough) this  year.

Been a great vacation.  Drove down to Melbourne to visit the parents.  Boy that was relaxing and tiring at the same time.  Cant believe time just dissapeared before I knew it!

Any hoo a happy 2008 to all.  Hope 2007 was a great year for all of you and hope 2008 is much better!

Flex

At work I write games.  No not the cool ones like Xbox or PS3 or Wii, but pokie machines ones.  Enough said.  Anyway, each build of the game takes about 5-10 minutes and each start takes upto 3 minutes.  Needless to say, the bug-fixing and configuration is quite the pain in the posterior.

An obvious realisation was, why not write a server that could listen to commands from “somewhere” and make changes to the game (objects) at run time.  This would mean a build is only necessary on a critical improvent.  And a restart (of the the game) would only be required on a rebuild!  This would be useful for modifying Sprite locations and properties and just about anything which the custom server can get hold off.  Now such a tool did exist.  Unfortunatly it was a custom written http server, which printed out the entire Sprite tree on each request.  Worse still, each request opened a new connection and then closed it.  Did I mention that this server only allowed querying, not modifying or creation of sprites (or other game objects)?

So I embarked on a new probing utility for doing just that.  The server was written in C++ using pthreads for multi-threading.  No big deal.  That is not what I am going to talk about – better people have written and preached about the art of server writing.  Thing that got me tickled was I used this opportunity to learn flash, so I begain writing the gui in flash/flex.  Main reason was that I wanted a web based UI for the client-side.  The experience was amazing.  The API is fantastic and intuitive, the IDE (well painful was having to move from VIM to Flex Builder/Linux on Eclipse due to intellisense).  Conversely without intellisense, the experience is tortuous.

Il talk more about the features in future posts each dealing with a subchallenge in the project.

/* No Comment */

I think the new title certainly reflects what I usually have to say about things and how often I say it.  So figured why not.  On a “deeper” level (not much more though),  this was our team name back in uni when we had participated in the ACM programming competitions.  Jees, I think I did everything to avoid actually going to classes.  Good times, those were.  The intense atmosphere in the comps..  ah dont think Id be doing justice describing it.  Those were the days.  And plus, did I mention the free food?

Any body interested in discussing some of the ACM problems?

OpenSocial

Google’s announcement of OpenSocial is fairly old news (atleast by blog time-lines anyway).  It was amazing how many companies had been  briefed earlier on, while Facebook had been kept in the dark.  If it was intended as fair platforms for all social network sites out there, why had Google decided to work with MySpace “secretly” over the past year?

And am I the only one finding it odd (or just finding) that it is merely a glorified version of the Google Gadget API?

Harry Potter and the Sickly Realisation

Well, I am a huge fan. I owe HP, or JKR rather, for getting me into the reading habbit! Yes it is embarassing, but I am proud of it. By the way there are a couple of spoilers so beware.

Anyway, I had finished the final installment HP and the Deathly Hallows last night at 3 AM. Thanks to the flu and an assignment I couldnt get into it earlier. But it is done. What an absolutely fantastic finale to the series. This is testament to the fact that she deserves every penny of the couple of billion pounds she has amassed thanks to the magical world she has woven. While the ending itself is not as surprising or brilliant as that of the Prisoner of Azkaban or the Goblet of Fire, it still ranks as one of the best books of all times. I am so glad that my theory (well not uniquely mine ofcourse) about Snape’s innocence and Id even say greatness, came true.

Ofcourse the deaths of Dobby was I think unnecessary. I loved the poor guy, and to have him killed especially after all that… well no words… but I think his death is made up by the house-elves’ participation in the final battle. The other deaths I beleived unnecessary were Tonk’s and Fred’s. I mean Moody and Lupin’s departure was inevitable, but Tonks and Fred’s? Surely we could have done without Charlie (no offence), but to deprive George of his significant other, thats just too cruel.

But deaths aside, an absolutely magnificent plot and character development. One thing that was a bit dissapointing was the epilogue. I could have used a bit more on what happened to more of the characters. JKR only had briefly described how Harry, Ron, Hermione and Ginny end up (with a couple of others). Hopefully these will be cleared out in an interview.

The hardest part was however the sickly realisation (which began as I had started the first page), that once I had gone through the 600 pages, that would be the end of HP world for ever. And boy was that painful. Actually it still is. I suppose this is nothing new. All good things must come to an end, but one can still wish otherwise.

By the way who is the individual who discovers magic late in his life (as JKR had pointed out in her interview?)

C++ – Python Bindings

Version 3.0 of omnifs saw the onset of python bindings for the omnifs library (written in C++ using Curl). In this post I am going to talk about what I had learnt in writing python bindings for C++. What this means essentially is that a library written in C++ can now be used from python. This is ideal for situations where the core modules can be written in a more efficient/lower level languages and can be controlled in a higher level scripting language. This, some (including myself) believe would assist in testing and development in general.

I will be referring to libomnifs, part of the omnifs package, in this posting for any examples. I wont be showing the details of libomnifs, and will focus on what additions were required to enable python bindings.

I have used the Boost.python library for creating the python bindings to omnifs. There other ways of doing this, but I wont go into them here. I had chosen Boost.python as it was fairly well documented and easy to follow (plus it looked cool).

Prerequistes

1. First download and install the boost library. I had used version 1.34.0. Please follow the installation guide for Boost on how to do this.
2. Then read the bjam tutorial. bjam is the build system created and used by Boost. It is very very powerful and can make life a lot simpler (once you get the hang of it).
3. Read the boost.python tutorial – This is a must. In fact this highlights everything one would need to create the bindings. I will only document what “extras” I have learnt in the process and will try to keep the repititions to a minimum.
4. Install python if you do not already have it.

ONCE AGAIN – The boost.python tutorial is fantastic so please have a look at it.

Step 1: First Make a list of Exportable Classes

In my case, one of the classes I had chosen to export was OMNI_Action. OMNI_Action class is superclass of all actions that can be taken agains/on the omnidrive server. For example “ListFolder”, “UploadFile” etc.

The public interface of the OMNI_Action is (protected, static and private members are not shown):

class OMNI_Action
{
public: // Virtual methods and constructors
OMNI_Action(OMNI_Session &p_Session);
virtual ~OMNI_Action();
virtual bool Perform() = 0;

public: // Non virtual methods
int Result();
const char * Message();
OMNI_Session * Session();
};

I have chosen the Action object as the example (rather than the Session object which is required by all Actions) as it is a class that is extended and thus requires more work to create the bindings:

• every OMNI_Action derived class, needs to only implement the Perform method (to *surprise surprise* perform the method).
• Each OMNI_Action object also needs a OMNI_Session object – the session in which omnifs is running. T his is another class that is exported but not shown.

Step 2: Include necessary Boost headers

Include the required boost headers as follows:

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
#include <boost/python/class.hpp>
#include <boost/python/args.hpp>
#include <boost/python/overloads.hpp>
#include <boost/python/docstring_options.hpp>
#include <boost/python/enum.hpp>
#include <boost/python/pure_virtual.hpp>

#include <boost/python/return_internal_reference.hpp>
#include <boost/python/copy_const_reference.hpp>
#include <boost/python/copy_non_const_reference.hpp>
#include <boost/python/return_value_policy.hpp>

or alternatively include ALL python related headers by include just python.hpp as:

#include <boosts/python.hpp>

Also enable the use of the python namespace:

using namespace boost::python;

Note that this could be avoided if you prefer to explicitly qualify items with the python namespace each time.

Step 3: Create the module

All the generated python bindings happen inside a module definition. This is done as:

BOOST_PYTHON_MODULE(module_name)

{

// Put binding definitions here – ie class exports and method exports

}

This is it. no need for a main function or anything. Think of the BOOST_PYTHON_MODULE bit as the main entry point of the generated shared object.

Step 4: Export the Classes

As shown in the “Exposing Classes” section of the boost.python tutorial, exporting OMNI_Action is simply a matter of doing:

class_<OMNI_Action>("Action")

.def("Perform",
pure_virtual(&OMNI_ActionAuthenticate::Perform),
"The function that performs the specific action.")
.def("Result", &OMNI_Action::Result,
"Returns the result of the action after \"Perform\" is called.")
.def("Message", &OMNI_Action::Message,
"Returns the message associated with the return result \n"
"after \"Perform\" is called.")
.def("Session", &OMNI_Action::Session, return_internal_reference<>(),
"Reference to the session object on which the action "
"object is valid.")

;

Now there is on SMALL problem with the above. The Perform method is a pure virtual method. Which means that due to a “no implementation” the above class cannot actually be exported. In order to do this, a wrapper class (called ActionWrap) is created for OMNI_Action, and THIS wrapper class is exported instead as shown below:


struct ActionWrap : OMNI_Action, wrapper<OMNI_Action>
{
bool Perform()
{

#if BOOST_WORKAROUND(BOOST_MSVC, <= 1300) // Workaround for vc6/vc7
return call<int>(this->get_override(“Perform”).ptr());
#else
return this->get_override(“Perform”)();
#endif

}

};

This is it. Now we have provided a proxy implementation of the Perform method which essentially calls the derived Perform method when invoked and we have gotten ridden of the pure virtual method.

The exporting of this class is:

class_<ActionWrap, boost::noncopyable>(“Action”,
“A wrapper for classes deriving from the Action object.\n”
“The Action class is the super class of all classes that \n”
“perform work on the omnidrive server.”, no_init)
.def(“Perform”,
pure_virtual(&OMNI_ActionAuthenticate::Perform),
“The function that performs the specific action.”)
.def(“Result”, &OMNI_Action::Result,
“Returns the result of the action after \”Perform\” is called.”)
.def(“Message”, &OMNI_Action::Message,
“Returns the message associated with the return result \n”
“after \”Perform\” is called.”)
.def(“Session”, &OMNI_Action::Session, return_internal_reference<>(),
“Reference to the session object on which the action “
“object is valid.”)
;

Note

• how all the methods of the original OMNI_Action class are also exposed (ie Perform, Result, Message, and Session)
• The noncopyable specifies that the ActionWrap cannot be copied or instantiated
• The “return_internal_reference<>” specifies that the return value of the “Session” object is infact a pointer to a class member, which means that the lifetime of the return value (a session object) is tied to the lifetime of the Action object. So unless the Action object is destroyed, the reference count of the return Session instance cannot be reduced! Please refer to Call Policies in the Boost.python tutorial for more information on this.
• The string values are essentially documentation strings in exported python module (docstrings).
  

Once this is done, the derived classes can be exposed normally.
Step 5: The Build Process

Once you have exposed all the classes in the above fashion, it is time for setting up your builds!

The builds are performed using bjam – the Boost build system. bjam looks for the file Jamroot in the current directory the same way that make looks for a Makefile.

bjam is a lot more complex than make and the bjam tutorial is an excellent source of all the information you are going to need.

I will go through my Jamroot file line by line:

1. Specify the Boost directory: This is the location where boost was installed.

use-project boost
: /opt/boost_1_34_9 ;

2. Specify the requirements and default builds. To make debug the default build, simply specify that in the default-build line.

project
: requirements <library>/boost/python//boost_python
: default-build release
;

3. Specify a dependency on the libomnifs C++ library. The C++ library is infact built in ../bld/<build_mode>

The following two rules mention that in the debug mode the debug build of the libomnifs is to be used and similar in the release mode, the release build of the libomnifs is to be used, as indicated by the “variant” flags.

lib libomnifs
:
: <file><locateion of omnifs source tree>/bld/debug/libomnifs.so <variant>debug
;

lib libomnifs
:
: <file> <omnifs_build_dir_path>/bld/release/libomnifs.so <variant>release
;

4. This specifies the installation target. Unlike in “make” where installations are arbitrary actions, they have a bit more meaning in bjam. These (in this rule) simply specify the installation location (in this case the current directory “.”) and types (install a library instead of an executable):

install dist : omnipy : <location> . <install-type>LIB ;

5. Finally the following specifies that the object being built is a python-extension library (of the name omnipy). Many other types of projects are also possible. Please refer to the bjam tutorial for a full list of build types. The .cc files are the files in which I had divided the export declarations. It is recommended to break up the class export declarations into multiple files so that changes in one class will not require the compilation of the entire source – there by speeding up the build process.

python-extension omnipy
: omnipy.cc omnipy-logger.cc omnipy-utils.cc omnipy-session.cc
omnipy-connection.cc omnipy-memory.cc omnipy-actions.cc libomnifs
;

Step 6 – In Action

Finally to see a glimpse of the library in action, simply start python and run the following:

– Import required modules including the one we just created

>>> import omnipy, sys, os

– create a session object by calling the Session constructor (refer to the actual distribution)
>>> session = omnipy.Session()

– create an actual action object – the action for creating a folder:

>>> action = omnipy.ActionCreateFolder(session, “new_folder1″, “”)

– Perform the action

>>> action.Perform()

and so on and so forth.

Please refer to the example.py and utils.py in the omnifs source distribution for more details.

Step 7: Gotchas and Todos

There a still a few things I havent yet figured out. One of them is how to export FILE * and streams so that we could use Python file handles instead. Once Im done with that Il put that up in here as well.

---

Well that concludes my little tip on how to create python bindings (more of a self-learning set of notes more than anything else). I hope this really saves you a lot of time. Please let me know if you find anything I am missing or forgot or just plainly got wrong. Id be glad to learn from it myself.

OmniFS

Today marks the release of omnifs. I had started trying omnidrive – a storage drive on the web in mid April. I am still on the free service (which comes with 1Gb of Storage and 5Gb of transfers/month). It is still in beta mind you, but not bad at all. They even have a client for windows and mac. The windows client lets you mount omnidrive as a special folder which can be accessed from windows explorer.

However, for Linux, I had to install a Java client. Now I am not against Java, but wondered why couldnt this just be a mountable drive? So being the fidder (or is it twiddler?) that I am, I thought, “if you cant find it, make it”. I set about making a mountable drive for omnidrive. Luckily I had come across the omnidrive developer API. It is not bad but could really be better by being clearer. The result of this work is omnifs. This is small program that uses the Fuse (Filesystem over User Space) library to mount your omnidrive account as a directory in Linux! By doing so, normal file operations can be performed (eg cp, mv, ls and also developer functionility like fXXX functions in the C library). Please do try it out. Would love to here what you think about it and ofcourse to fix it up more.

The general goal of omnifs is to transition into a “webfs” so that it would eventually support WebDAV and other online storage accounts!

Back On

Well after a long long gap. Here I am back on. Had a few things going on. Had my exams going, bought a house, tried out OmniDrive and made a mountable drive for OmniDrive. Will post more about it soon.