Message transport (between the gprobe, the game server I had written for my work in my spare time and the front end client) was using json or  Action Message Format, AMF.  Reason for the duality, was that for quick testing, I could login into the server with simple telnet and send (textual) commands and recieve data in json format – Simple.  However, I could not find a built in JSON parser for Flex and writing one (again, as I had already written one in C++ for the server side) – didnt just seem too appealing.  Luckily, Flex is capable of serializing and deserializing objects in binary format over a stream – yep – the Action Message Format or AMF.  Adding the AMF serializer to the server was fairly simple (did take few hints on the integer encoding from pyamf – thanks guys).

Now the problem was that I could not find a way of debugging the decoding (on the flex side) of the AMF stream I had generated from my server.  Why do I need debugging you ask?  Well, I was having random errors in the AMF stream (no doubt due to my encoding) while encoding strings.  Due to time-constraints I had decided to switch to JSON.  This meant I had to write a json parser for flex (as I could not find one at the time).  Not too bad as it gave me a lot of insight into what was being sent and so on.  One day when I earn more free time at work I may decided to debug my AMF encoder.

At work I write games.  No not the cool ones like Xbox or PS3 or Wii, but pokie machines ones.  Enough said.  Anyway, each build of the game takes about 5-10 minutes and each start takes upto 3 minutes.  Needless to say, the bug-fixing and configuration is quite the pain in the posterior.

An obvious realisation was, why not write a server that could listen to commands from “somewhere” and make changes to the game (objects) at run time.  This would mean a build is only necessary on a critical improvent.  And a restart (of the the game) would only be required on a rebuild!  This would be useful for modifying Sprite locations and properties and just about anything which the custom server can get hold off.  Now such a tool did exist.  Unfortunatly it was a custom written http server, which printed out the entire Sprite tree on each request.  Worse still, each request opened a new connection and then closed it.  Did I mention that this server only allowed querying, not modifying or creation of sprites (or other game objects)?

So I embarked on a new probing utility for doing just that.  The server was written in C++ using pthreads for multi-threading.  No big deal.  That is not what I am going to talk about – better people have written and preached about the art of server writing.  Thing that got me tickled was I used this opportunity to learn flash, so I begain writing the gui in flash/flex.  Main reason was that I wanted a web based UI for the client-side.  The experience was amazing.  The API is fantastic and intuitive, the IDE (well painful was having to move from VIM to Flex Builder/Linux on Eclipse due to intellisense).  Conversely without intellisense, the experience is tortuous.

Il talk more about the features in future posts each dealing with a subchallenge in the project.

Version 3.0 of omnifs saw the onset of python bindings for the omnifs library (written in C++ using Curl). In this post I am going to talk about what I had learnt in writing python bindings for C++. What this means essentially is that a library written in C++ can now be used from python. This is ideal for situations where the core modules can be written in a more efficient/lower level languages and can be controlled in a higher level scripting language. This, some (including myself) believe would assist in testing and development in general.

I will be referring to libomnifs, part of the omnifs package, in this posting for any examples. I wont be showing the details of libomnifs, and will focus on what additions were required to enable python bindings.

I have used the Boost.python library for creating the python bindings to omnifs. There other ways of doing this, but I wont go into them here. I had chosen Boost.python as it was fairly well documented and easy to follow (plus it looked cool).

Prerequistes

1. First download and install the boost library. I had used version 1.34.0. Please follow the installation guide for Boost on how to do this.
2. Then read the bjam tutorial. bjam is the build system created and used by Boost. It is very very powerful and can make life a lot simpler (once you get the hang of it).
3. Read the boost.python tutorial – This is a must. In fact this highlights everything one would need to create the bindings. I will only document what “extras” I have learnt in the process and will try to keep the repititions to a minimum.
4. Install python if you do not already have it.

ONCE AGAIN – The boost.python tutorial is fantastic so please have a look at it.

Step 1: First Make a list of Exportable Classes

In my case, one of the classes I had chosen to export was OMNI_Action. OMNI_Action class is superclass of all actions that can be taken agains/on the omnidrive server. For example “ListFolder”, “UploadFile” etc.

The public interface of the OMNI_Action is (protected, static and private members are not shown):

class OMNI_Action
{
public: // Virtual methods and constructors
OMNI_Action(OMNI_Session &p_Session);
virtual ~OMNI_Action();
virtual bool Perform() = 0;

public: // Non virtual methods
int Result();
const char * Message();
OMNI_Session * Session();
};

I have chosen the Action object as the example (rather than the Session object which is required by all Actions) as it is a class that is extended and thus requires more work to create the bindings:

• every OMNI_Action derived class, needs to only implement the Perform method (to *surprise surprise* perform the method).
• Each OMNI_Action object also needs a OMNI_Session object – the session in which omnifs is running. T his is another class that is exported but not shown.

Step 2: Include necessary Boost headers

Include the required boost headers as follows:

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
#include <boost/python/class.hpp>
#include <boost/python/args.hpp>
#include <boost/python/overloads.hpp>
#include <boost/python/docstring_options.hpp>
#include <boost/python/enum.hpp>
#include <boost/python/pure_virtual.hpp>

#include <boost/python/return_internal_reference.hpp>
#include <boost/python/copy_const_reference.hpp>
#include <boost/python/copy_non_const_reference.hpp>
#include <boost/python/return_value_policy.hpp>

or alternatively include ALL python related headers by include just python.hpp as:

#include <boosts/python.hpp>

Also enable the use of the python namespace:

using namespace boost::python;

Note that this could be avoided if you prefer to explicitly qualify items with the python namespace each time.

Step 3: Create the module

All the generated python bindings happen inside a module definition. This is done as:

BOOST_PYTHON_MODULE(module_name)

{

// Put binding definitions here – ie class exports and method exports

}

This is it. no need for a main function or anything. Think of the BOOST_PYTHON_MODULE bit as the main entry point of the generated shared object.

Step 4: Export the Classes

As shown in the “Exposing Classes” section of the boost.python tutorial, exporting OMNI_Action is simply a matter of doing:

class_<OMNI_Action>("Action")

.def("Perform",
pure_virtual(&OMNI_ActionAuthenticate::Perform),
"The function that performs the specific action.")
.def("Result", &OMNI_Action::Result,
"Returns the result of the action after \"Perform\" is called.")
.def("Message", &OMNI_Action::Message,
"Returns the message associated with the return result \n"
"after \"Perform\" is called.")
.def("Session", &OMNI_Action::Session, return_internal_reference<>(),
"Reference to the session object on which the action "
"object is valid.")

;

Now there is on SMALL problem with the above. The Perform method is a pure virtual method. Which means that due to a “no implementation” the above class cannot actually be exported. In order to do this, a wrapper class (called ActionWrap) is created for OMNI_Action, and THIS wrapper class is exported instead as shown below:


struct ActionWrap : OMNI_Action, wrapper<OMNI_Action>
{
bool Perform()
{

#if BOOST_WORKAROUND(BOOST_MSVC, <= 1300) // Workaround for vc6/vc7
return call<int>(this->get_override(“Perform”).ptr());
#else
return this->get_override(“Perform”)();
#endif

}

};

This is it. Now we have provided a proxy implementation of the Perform method which essentially calls the derived Perform method when invoked and we have gotten ridden of the pure virtual method.

The exporting of this class is:

class_<ActionWrap, boost::noncopyable>(“Action”,
“A wrapper for classes deriving from the Action object.\n”
“The Action class is the super class of all classes that \n”
“perform work on the omnidrive server.”, no_init)
.def(“Perform”,
pure_virtual(&OMNI_ActionAuthenticate::Perform),
“The function that performs the specific action.”)
.def(“Result”, &OMNI_Action::Result,
“Returns the result of the action after \”Perform\” is called.”)
.def(“Message”, &OMNI_Action::Message,
“Returns the message associated with the return result \n”
“after \”Perform\” is called.”)
.def(“Session”, &OMNI_Action::Session, return_internal_reference<>(),
“Reference to the session object on which the action “
“object is valid.”)
;

Note

• how all the methods of the original OMNI_Action class are also exposed (ie Perform, Result, Message, and Session)
• The noncopyable specifies that the ActionWrap cannot be copied or instantiated
• The “return_internal_reference<>” specifies that the return value of the “Session” object is infact a pointer to a class member, which means that the lifetime of the return value (a session object) is tied to the lifetime of the Action object. So unless the Action object is destroyed, the reference count of the return Session instance cannot be reduced! Please refer to Call Policies in the Boost.python tutorial for more information on this.
• The string values are essentially documentation strings in exported python module (docstrings).
  

Once this is done, the derived classes can be exposed normally.
Step 5: The Build Process

Once you have exposed all the classes in the above fashion, it is time for setting up your builds!

The builds are performed using bjam – the Boost build system. bjam looks for the file Jamroot in the current directory the same way that make looks for a Makefile.

bjam is a lot more complex than make and the bjam tutorial is an excellent source of all the information you are going to need.

I will go through my Jamroot file line by line:

1. Specify the Boost directory: This is the location where boost was installed.

use-project boost
: /opt/boost_1_34_9 ;

2. Specify the requirements and default builds. To make debug the default build, simply specify that in the default-build line.

project
: requirements <library>/boost/python//boost_python
: default-build release
;

3. Specify a dependency on the libomnifs C++ library. The C++ library is infact built in ../bld/<build_mode>

The following two rules mention that in the debug mode the debug build of the libomnifs is to be used and similar in the release mode, the release build of the libomnifs is to be used, as indicated by the “variant” flags.

lib libomnifs
:
: <file><locateion of omnifs source tree>/bld/debug/libomnifs.so <variant>debug
;

lib libomnifs
:
: <file> <omnifs_build_dir_path>/bld/release/libomnifs.so <variant>release
;

4. This specifies the installation target. Unlike in “make” where installations are arbitrary actions, they have a bit more meaning in bjam. These (in this rule) simply specify the installation location (in this case the current directory “.”) and types (install a library instead of an executable):

install dist : omnipy : <location> . <install-type>LIB ;

5. Finally the following specifies that the object being built is a python-extension library (of the name omnipy). Many other types of projects are also possible. Please refer to the bjam tutorial for a full list of build types. The .cc files are the files in which I had divided the export declarations. It is recommended to break up the class export declarations into multiple files so that changes in one class will not require the compilation of the entire source – there by speeding up the build process.

python-extension omnipy
: omnipy.cc omnipy-logger.cc omnipy-utils.cc omnipy-session.cc
omnipy-connection.cc omnipy-memory.cc omnipy-actions.cc libomnifs
;

Step 6 – In Action

Finally to see a glimpse of the library in action, simply start python and run the following:

– Import required modules including the one we just created

>>> import omnipy, sys, os

– create a session object by calling the Session constructor (refer to the actual distribution)
>>> session = omnipy.Session()

– create an actual action object – the action for creating a folder:

>>> action = omnipy.ActionCreateFolder(session, “new_folder1″, “”)

– Perform the action

>>> action.Perform()

and so on and so forth.

Please refer to the example.py and utils.py in the omnifs source distribution for more details.

Step 7: Gotchas and Todos

There a still a few things I havent yet figured out. One of them is how to export FILE * and streams so that we could use Python file handles instead. Once Im done with that Il put that up in here as well.