Current developments in autonomous vehicle systems typically consider solutions to single problems like road detection, road following and object recognition individually. The integration of these individual systems into a single package becomes difficult because they are less compatible as integration was not their purpose. This work introduces a generic Integrated Neural System for Autonomous Vehicles (INSAV) package solution with processing blocks that are compatible with each other and are also suitable for hardware implementation. The generic INSAV is designed to account for important problems such as road detection, road structure learning, path tracking and obstacle detection. The paper begins the design of the generic INSAV by building its two most important blocks: the Road Structuring and Path Tracking Blocks. The obtained results from implementing the two blocks demonstrate an average of 92% accuracy of segmenting the road from a given image frame and path tracking of straight roads for stable motion and obstacle detection.

The system is designed based on the Psychophysical Evidences. These systems involve Hierarchical Processing, Neural Configurability, Adaptive Response and Attention Models. Hierarchical Processing allows the system to have different levels of analysis and processing. These levels typically include Sensory, Perceptual, Syntactic and Semantic Analyzers. Neural Configurability enables the system to use previously learned information to understand its current environment. Adaptive Response provides the perceptual ability for the system, e.g. structure and motion analysis. Attention Models guide the system to focus on anticipated areas for faster responsivity.

The INSAV is hierarchical processor with analyzers connected in a hierarchical way.It consists of a Memory Block to implement the neural configurability. The adaptive response is implemented by the Perceptual and Conceptual Analyzers. The attention modeling is not considered in the design.

The Sensory Analyzer consists of an Edge Detection Block and Road Structuring Block. It provides the system with the current road structure.

The Perceptual Analyzer consists of a Path Tracking Block, a Motion Classifier Block and an Obstacle Detection Block. This analyzer perceives and classifies the motion of the objects in its view as potential obstacles or non-potential obstacles.

The Conceptual Analyzer consists blocks that enable the system to understand the road structure as specific types such as a straight road, bending road, cross-road or T-junction, etc, and also learn new road types like tarmac roads of different colors or non-tarmac roads of mud, stony, etc, types.

The Road Structuring Block is responsible for providing the system with the information about the structure of the road at all times. This is accomplished using edge detection and pixel connecting to form the neural structure of the road. Each pixel on the image is considered a neuron with R, G, and B of the RGB color space as inputs. Then given a color set, the pixel neurons belonging to the given set fire. The fired pixel neurons (FPNs) connect with each other using the 8-neighborhood technique. The edge detection algorithm partitions the connections made by the fired pixel neurons with one of these partitions representing the structure of the road.

The color set given to the pixel neurons is some spherical set that represents the color of the road. This spherical set has a center-of-gravity point and a radius in the RGB space. The reason for choosing a spherical set comes from the understanding that points in a space closer to each other have similar characteristics. The level of similarity is defined by the euclidean distance between the points.

This work currently assumes that such sets are created, say,during a factory training stage. In this stage, several images of roads of different types are given to the system for learning with some supervision. For initial experimental purposes, a road color of a single type was used. Test with more road colors (i.e. with an extended vocabulary of road colors) will be the next experiment.

The INSAV is currently designed with a capability to modify what it learned previously with new data it receives on the run. This makes the INSAV an adaptable system, a fundamental requirement of neural systems.

The Path Tracking Block is responsible for estimating the vanishing point of the road. The vanishing point is the point showing where the road is headed. For straight roads the task is simple but for curved roads this is tricky. For curved roads, the portion of the road that is straight must be identified first. The advantages of knowing the vanishing point of the road are 1) to ensure motion stability of the vehicle and 2) to classify motions of other objects on the road (important for obstacle detection). Therefore the remaining parts of the Perceptual Analyzer majorly depend on the output of this block. The remaining dependency belongs to an effective Motion Computation Block.

The vanishing point estimation is accomplished using two neurons namely the Left-Road-Boundary-Neuron (LRBN) and Right-Road-Boundary-Neuron (RRBN). These neurons are neurons on the image plane and so they have weights wx and wy for the X and Y dimensions respectively and bias b. The weights of the neurons form lines on the image plane with the point of intersection representing the vanishing point.

The algorithm developed for this estimation considered the following specifications: 1) The estimation should fundamentally work for straight road structures with no noise, 2) The estimation should give an acceptable result for straight road structures with some additive noise and 3) The estimation should still give a good enough result for straight road structures deformed by vehicles on the road.

For the first two specifications, existing algorithms such as the LMS would work well but would fail for the third specification. Hence this work developed an algorithm that would satisfy all the three specifications.

The experimental results of the above algorithms on real road scenes is given below.

Test Samples

Result of Road Structuring Block

Result of Path Tracking Block

Publications:

• U. Ravishankar, M. Manic, "A Hardware Suitable Integrated Neural System for Autonomous Vehicles - Road Structuring and Path Tracking," in Intl. Joint Conf. on Neural Networks, IJCNN (accepted in 2011)Papers Abstract PDF