Some years back I worked on a project concerned with combining the information from several sources into something a pilot could use in a "high pressure" situation.

At the state-of-the-art then, about the best we could come up with after quite a few years of tinkering was to convert the various sources of data into a common form and show it on a single display.

Some of the information was radar data, and some came from radio receivers. The radar data gave more-or-less accurate info on range, direction, heading, and altitude of various objects; the receivers gave an idea of range for radio transmitters (e.g. targeting radar and transponders, if applicable , but only a very vague idea of range of the transmitters.

So the final solution was to convert everything to a range and direction with error bands, and show a kind of merged density plot on a multi-colour display. It looked pretty much like a radar plot in false colours, with the addition of showing where transponders and other transmitters were "likely" to be. The pilot just kept out of the red "hot" zones.

Nowadays this is the  hot area of "data fusion", and the US military is still spending big bucks for people to come up with ways of doing it.

In the case above, we really pushed the "fusion" part of the project onto a human. But in robotics we’d like the machine to do the work and just present a summary to any human observer in those unusual cases where anyone cares.

So I’ve been burning the midnight candles trying to cook up some simple methods to combine info from several robot sensors. And it’s starting to make some progress.

On my Inex Robo-11 "tank" I now have several different sensors. Some of them overlap in the kind of info they provide, and some of them are disjoint. But all can be used to gain info about the robot’s environment.

Presently I have
(1) an IR ranger. It’s the Sharp "120" model, and is good out to about 3 m.
(2) a sonar ranger. This is a recent "SRF05" model, and is good to 4 m.
(3) 2 bump sensors on the front. Range approx 0.
(4) A magnetic compass. This neat module from Inex allows the bot to tell the direction its headed, within a few deg.
(5) Wheel sensors. Just the usual black-and-white stripes glued to a couple of wheels and observed by an IR sensor. Accurate to 6 mm, allegedly.
(6) IR line detectors. I’ve more-or-less permanently mounted 2 under the tank, left and right.

There are a few more devices waiting in the wings. But the assignment may already be overloaded.

My initial experiments simply involve combining the info from the 3 sources of ranging. I.e. (1)-(3).

Obvious methods include a simple combining function like max, min, arith avg, geom avg, median, etc. And this kind of "suck it and see" bottom-up approach can be quite fruitful.

Since all ranging detects something at a given range (and, usually, directly in front of the sensor — immediately in front of the bot in the above setup), using a "min" on the ranges might seem like a reasonable idea. If the bumpers say "something at 0, boss" and the IR says "3 m", we’d prefer to accept to "something at 0" message.

But this simple idea doesn’t take into account the noisiness of the envirnoment. The sensors are not perfectly accurate, to start with, and it took quite a while of fiddling with the SRF05 to get it to +- 10% over its entire range 0 .. 4 m. But other "noise" is introduced by what might be viewed as "false echoes".

Especially using a sonic ranger, the cone of outgoing "radition" means there are numerous paths that (e.g.) bounce between walls and produce phantom objects at close range. (The same thing was obsvered in my radar experience, with things like basketball poles on school playgrounds having jet fighters "see" other fighters approaching from the north at high speed .

And the IR ranger is also susceptible to unusual returns. Some objects absorb IR and appear further away than they really are; and contrarywise.

So another idea is to use a probabalistic model & information theory to get the "maximum information" from the available data. I.e. you produce a conditional probability function for each sensor that gives the prob an object (e.g. "the closest object") is really at distance "D" given the sensor says it is at "d".

Then these sources can be combined using (e.g.) maximum likelihood or other methods.

Another idea is to use Game Theory. I.e. attribute a cost to getting measurements right or wrong, and use (e.g.) a genetic method to *evolve* a combining method.

But maybe the idea that will yield fruit soonest is an adaptation of the "vision 101" methods I’ve been espousing elsewhere. I.e. create the simplest geometric model that is consistent with all observations. (The "simplest model" is not only well-know after Wm of Occam, but has a formal underpinning first given by Monash’s Chris Wallace). This has the effect of filtering noise as well as combining the info in a low-entropy-preserving optimal way.

I’ll present all of these methods in coming days.