Building a telepresence Robot - part 3

My last post left off where I was playing around with the Parallax Propeller micro controller. There are two official ways to program the controller these are either via SPIN or via assembler and a number of unofficial unsupported ways. I played a bit with both and as I am more comfortable with C, I decided to use the Catalina C compiler package which includes the Code::Blocks editor. Unfortunately there is no easy way to use SPIN objects obtained from the Object Exchange with C so I had to rewrite the Object I downloaded to do the Wheel Control in C. The propeller chip has 8 cores, cogs as they are officially named, this is great as one can have an isolated routine run on each cog.



As you can see in the photo I have installed some of the ultrasonic pings, on the front and a single one on the back. I still have a few more but I will install them when I need them. The base kit came with an additional mounting board for a battery but as it had holes that matched the proto board I mounted the proto board on it and the board over the motors. I had to make a quick trip to Home Depot to get a couple of 2.5" stainless bolts and a few nuts.
When I ordered the proto board I got also ordered the adapter kit to allow me to connect a VGA monitor to it. This makes debugging a lot easier as you don't need to fire up a serial terminal every time to see what is going on.
Using each component is pretty simple - you compile your program on a PC and you download it to the controller board via a USB cable. The pings are also simple to use - you signal the pin that it is attached to and count the number of processor ticks it takes to get a response. This gives the distance to an object in clock cycles which can be converted to time and distance. The motors are a little more complex - therefore I just downloaded a routine to manage them. You pass a distance to the motor controller and it rotates the wheels until you cover the distance.
The next step in this project will be to use the information I am getting from the pings and have the robot guide itself through the office without hitting anything. I will cover this in the next post on this blog.

Building a telepresence Robot - part 2

Well got of to a good start - had to find an air pump to inflate the wheels to 36psi. One of the screws on one of the motor assemblies needed a washer else the bearing would take too much strain. The overall quality of the parts is exceptional - every part is precission machined out of aluminum and every hole aligns correctly. This makes assembly a walk in the park.



Now that I have the major hardware installed I am going to play around with the micro controller and see how much code I need to write to control the servo's and read information from the ultrasonic pings.

Building a telepresence Robot - part 1

After recent reading articles in the IEEE spectrum and having experience with OpenCV and a background in Electronic Engineering I decided it would be a fun exercise to see if I could hack together a robot proxy using off the shelf components and Open Source software.

I found a great starting point at Parallax Inc which provides a couple of embedded processor boards and a really sturdy base. For power I decided to use a 12v lead acid UPS battery, I have lying around, for the motors together with a universal BEC which would give me 6 volts for the servos and microprocessor board.

I ordered and received the parts, now the fun really begins - follow this blog to keep track of how the project progresses.

Converting from StarTeam to Subversion - Part 2

After spending another month on this I finally managed to get a reasonable import for each project. Links were tricky and I ended up just saving the link location to a file so that I could go back and manually add them to SVN when I was done.
Starteam allowed multiple files with the same name which caused a lot of issues. I also discovered that revision and version labels were handled differently and a revision label could contain files that were dated after it's creating. Doing a difference between a revision and version label would often end up causing a huge mess.
My final working code only used the difference between revision labels to get history. I processed every revision in every branch completely creating tags for every version label as I hit the time stamp for that label. This made it easier to create branches from the tags than try and determine which version of which file should be included.

I hope I never have to do this type of conversion again - a simple task (or so I thought) of converting 15+ years of starteam projects to SVN took me about 3 months of development time which far exceeded the 2 weeks I had set asside for the project.

Our largest project was just not reliably convertable, for this I just took a snapshot of each branch and commited that to SVN - if somone wanted more history they could get it from Starteam :).

Converting from StarTeam to Subversion - Part 1

Being a long time user of StarTeam (since 2000) we have been reasonable content with our StarTeam installation. We do however use a number of different operating systems and with Borland no longer giving appropriate support to OSx and the fact that the integration with Visual Studio is extremely bulky and only available on the top tier of the product  I decided it was time to switch to something new. A number of our developers had used Subversion in the past so this was the obvious choice - the repository would however need to be converted. Having gone through the conversion from MKS to starteam in 2000 I thought I would take responsibility for the StarTeam to Subversion conversion.

I found a utility written in Java called svnimport from Polarion and it did a reasonable job until it hit our main project and kept throwing out of heap exceptions. I tried all the Java tricks of setting a large heap size and aggressive heap usage but it would always die at about 1.3GB of allocated memory (I had 12 GB so I think the issue was the way the utility was written). After a bit of searching I found the source code for svnimport. I downloaded a copy of myeclipse and started debugging svnimport to see if I could possibly fix the issues or change the way it was using memory. I am however not a big fan of Java so I decided to convert the project to C# and use the Borland .Net libraries.

The first issue I discovered with svnimport was that it did not do its branching or tagging intelligently if you had 10 branches that were created at different times there would be 10 copies of everything up to the branch time. this meant that if you had a 1MB file with 10 revisions in your main trunk it would be present 10 times in every branch. This design caused one project to take 18 hours to dump and 4 days to import creating a 24GB subversion repository

The first thing I changed was to build up a list of branches and tags sorted by creation date (you need to check the attributes and see if the branch is based on a label and then get the appropriate date from the label item).

Svnimport builds up a list of actions/commits based on the revision of each file, I changed this to only process files that had a modified time > than the creation time of the branch - this cut down on the issue above of having duplicate files. now with these 2 changes I went about updating the dump file creation. The commits are sorted by date so all I had to do was insert code that would check to see if a commit date was larger than a branch or label and if so I would inset an action to create a branch/label that was copied from the current revision. Tags were easy as they are just a snapshot in time.

I did a couple of test runs - one project that was generating a 18MB dump file was now generating a 3MB dump, and the import into subversion was solid.

I could now start working on getting a more reliable conversion going.

continued in part 2...

Finding missing ClearQAM streams

Often, after installing a new Digital TV Tuner into your system Media Center will not detect any channels on it. You run 100's of  'Scan for more Channels' and just end up with 'No Channels Found. To add insult to injury you attempt to add the channels manually but you have absolutely no luck as guessing the channel numbers and sub numbers is just that - a guess.

Figure 1 The Dreaded - No channels found - screen

Well I had enough of that and I decided to write a small utility to scan every channel that media center scans and display the information in the hopes that someone else can find it useful - I know I did.
To understand why your channels are not detected you need to understand how Media Center 'detects' a unique channel. The first step is of course scanning for channels usually this starts by setting the digital tuner to channel 55.25 MHz (if you live in the USA) Media Center uses frequencies for BDA tuners to set the channel. It then jumps in increments of roughly 6 Mhz searching for a digital signal - there are some places where it jumps a different amount but this is outside the scope of this discussion.
Every time the tuner locks onto a signal the transport stream is scanned for a Program Association Table (PAT) table - the PAT table contains each program number and the PID associated with that program. After the PAT table is detected the Program Map Table (PMT) needs to be found. The PMT contains the information (PID and type) of each elementary stream i.e. video, audio, etc.
Now you are wondering what is he talking about what's all this talk about programs when we are talking transport streams…well a transport stream can carry many different streams that are multiplexed, each stream is identified by a PID which is contained in the PAT.

Now Media Center has determined that this is a digital stream and it wants to get 'Digital' information. The stream is now scanned for PSIP (Program and System Information Protocol) information. This starts off with the Master Guide Table (MGT) – this table, similar to how PAT contains information about the streams, contains information about PSIP tables. Most PSIP packets have a PID of 0x1ffb (8187) so detecting the correct table is just a matter of detecting the table field. The MGT will define if a stream contains a Terrestrial Virtual Channel Table (TVCT) used by ATSC transmissions or a Cable VCT (CVCT) which is obviously used by Cable transmissions. These two VCT tables along with the MGT are all that is used for channel detection. There are other tables – the Event Information Table (EIT) is used for guide information if a channel match cannot be found, the Extended Text Table (ETT), etc.
If everything panned out MCE would have detected your channels and you would not be reading this postJ. However there is one caveat occasionally program with duplicate program numbers may be overwritten. I have tested and verified this in a couple of times.

Figure 2 Channel information returned

As you can see in Figure 2 channel 72 two channels with PSIP information were detected and 3 without – the Call Signs are displayed along with the channel numbers that Media Center will present the channel as on the guide the other may be displayed as C72.97, C72.20 and of course C72.235. This of just depends of course on no other channels show up with the same program number and overwrite them.

Figure 3 Media Center Channel edit

As you can see in Figure 3 the information detected on the utility is reflected in Media Center. The pad-lock on the right shows that the channels are encrypted if there is no padlock you can check the box and the channel will show up on your guide without any information. You can then watch it for a while to determine what channel it is and change the listing manually.
If you do not see your program and you know more or less what channel it is you will be able to add it manually. Select Add a QAM channel enter the missing channel number which is either the PSIP number or the "channe .program number". You can select QAM256 by default it does not really make a difference as he tuner handles the demodulation not Media Center.

Figure 4 Adding Channel manually

Unfortunately Media Center will never map a xx.yy channel that it detected via a scan to a xx channel that you will see if you are using a cable card or a STB. This is because Media Center is treating every program as if is a ATSC program – scanning the internals of Media Center also reflects this.
The utility I wrote just scans the channels for now – I am busy porting some of this code to C# so that I can combine it with the code in my previous article to allow editing of the Media Center Guide listings directly. This will also give the ability to select a full channel number as opposed to a x.y number.

Here are the links to the files:
32bit ClearQAM Scanner.
64bit ClearQAM Scanner.

This files are offered without warranty, however they are not public domain please leave any comments re bugs etc. here.

To install run the registerdll.bat file then run the exe file - it will grab the first QAM capable tuner that Media Center is aware of i.e. you must have set up Media Center to use the Digital Cable Tuner before running this utility.

Detecting and fixing Channels in Media Center 7

One of the to-do's I have on my Streaming Tuner is to detect the channel call signs on the client computer for the encoded analog channels that are now digital. The second to-do was to get the channels to match up with guide information.

I managed to get the channels detected by scrubbing the transport stream (TS) created by the mpeg-2 encoder in the graph. I needed to edit the PAT and PMT - to change the program number of the stream and add additional streams (MCE does not like duplicate program numbers - I discovered after many hours of head banging). Then I needed to inject PSIP packets (namely MGT, CVCT and EIT) which would enable MCE to detect the callsigns of the channels. Needless to say I now consider myself to be somewhat of an expert in PSIP having thoroughly read the specifications and the book PSIP: Program And System Information Protocol: Naming, Numbering, and Navigation for Digital Television from cover to cover.

To get the callsigns turned out to be a much more complex the MCE 6 SDK has a few annoying features - one of which is the inability to provide channel names in cases where there are multiple sub-channels on the same channel number. As can be seen in the first image. Using the GetCallSigns method returns a serviceID and a callsign then by using the GetServiceIds method, which returns serviceID and channel number, one should theoretically be able to map the callsign to the channel number by using the serviceID and a common variable.

Figure 1 Channel 5.x on server

However as I mentioned earlier this does not work because for channel 5 only the serviceID of WPTVDT2 is returned with a channel number of 5.

I then shifted my focus to using XMLTV but that now requires a subscription for US guide data and the used would have to configure XMLTV as well as Media Center which would leave way to much space for mistakes. In addition to that I had not found a way for the client to match callsign to guide data yet.

Figure 2 Channel detection on client

Finally I decided to revisit getting the information from the media center database. I imported mcepg.dll into a project and voila I saw almost everything I needed. After a few hours I was able to pull the scanned analog lineup out of the database into an XML file.

 
 

Figure 3 Part of channel configuration file

Now all that was needed was to get the client side to edit the guide listing programmatically. I knew this was possible because you can import MXF formatted XML into media center. I had a look at that but it seemed more effort than it was worth. I decided to use the mcepg.dll again to edit the guide on the client side. This was also a pretty simple task except for the fact that I was getting exceptions every time I called the
Microsoft.MediaCenter.Guide.MergedChannel.AddChannelListings(Channel) method. I wrapped this in try catch and it worked fine - but I don't like hiding problems in code so I spent about a day finding out what was causing the exception (luckily I learnt to program in the days when asm was needed) - I narrowed it down to a inactive DeviceGroup. I tried a number of workarounds and finally managed to find one that would allow me to make my changes without corrupting the database of causing exceptions.

Figure 4 Client guide after channel scanning

Not only have I managed to programmatically remap scanned channels to a legitimate guide channel I have also managed to change a channel.subchannel into a channel number without any sub channels. The last image shows the guide after I did the first 2 channels 1.2 and 1.3. I know no one uses channel 1.x for anything so I just sent the channel on PSIP and major=1 and subchannel=analog channel number.

Figure 5 First 2 channels fixed

Next week I will finish up the installer and clean up a couple of issues so that I can make an alpha build and send it off to everyone that requested it. I think I might just also create a GUI for people to change their Guide information as well. I know I hate going through the "edit listing" menu and searching for a matching listing to map my clearQAM to my cablecard channels.