La motivación de la sentencia

7.1 Computer Vision

Computer vision is ability of machines to see and recognize objects. Image Processing , Pattern Recognition and Object Detection come under sub domains of computer vision.

Several techniques can be used to detect objects of particular size and shape. In this project, Hough transform is used.

7.2 Hough Transform

Hough transform¹ is extensively used in the computer vision community. It is primarily used for line detection. However generalized Hough transform can be used to detect any shape of interest[4].

Hough Linear Transform Consider the equation for a line

y = mx + c (7.1)

where (x,y) is the co-ordinates of the points on the line. Here m and b are the parameters of the line which represent the slope and the y intercept respectively.

Consider another representation of a line y = −cosθ

sinθ x + r

sinθ (7.2)

where r is the perpendicular distance of the origin from the line and θ is the angle between x axis and the perpendicular line passing through the origin. We can observe that there will be only one (r, θ) for a particular line, which means that if we have a (r, θ) space a line will be mapped to a point in that space.

Consider two points p1 and p2 in an x-y plane as shown in figure 7.1. The following analysis

1The Hough transform universally used today was invented by Richard Duda and Peter Hart in 1972, who called it a ”Generalized Hough Transform” after the related 1962 patent of Paul Hough

36

CHAPTER 7. MACHINE VISION 37 includes the Hough transform method to find the equation of the line connecting these two points.

Figure 7.1: Two point in x-y plane

Infinite number of lines can pass through a single point as shown in figure 7.2. Now when each line is mapped into (r, θ) space, a plot similar to that given in figure 7.3 is got.

Figure 7.2: Possible lines

In this figure, the two plots for point p1 and p2 are intersecting at a point (r⁰, θ⁰). If we plot a line with (r⁰, θ⁰) as parameters, we will get a line connecting p1 and p2 as shown in figure 7.3.

Hough transform for detecting circles Consider the equation of a circle

(x − a)²+ (y − b)² = r² (7.3)

where (a,b) is the center and r is the radius of the circle. For a line, there are 2 parameters r and θ, wherein for a circle there are 3 parameters - a,b and r i.e here the accumulator is a 3 dimensional space.

CHAPTER 7. MACHINE VISION 38

Figure 7.3: Mapping

Algorithm with matlab results 1. Reduce the size of the image

The size of the image is decreased to reduce the processing time. In this section a value of each pixel is replaced by the average value of eight pixels adjacent to it.

2. Edge detection (using sobel filter)

Sobel fiter is used for edge detection. This section is tested with canny filter also.

3. Hough circle transform

Circles in the image is detected using Hough transform 4. Isolated a particular coloured ball

In this section the values of the pixels inside the detected circles are averaged. Then this averaged values are compared with values of a specific colour. If we need to isolate a yellow coloured ball, the averaged values are compared with values of yellow colour.

Matlab results

The output of the Matlab program is shown in figure 7.5. The time given is in seconds.

Note that the time taken to process the image is considerably large.

CHAPTER 7. MACHINE VISION 39

7.3 Implementation in Java

Since MATLAB is a simulation software and as it consumes more time while processing the images, it can’t be used in real time systems. To increase the speed the program should be coded in any of the programming languages. As Java is fast and reliable, the basic platform for the system is changed from MATLAB to Java. Java is done in the IDE Eclipse provided as freeware by IBM.

7.4 Technologies Used for Java Design

Java is a language which can be used to develop any kind of applications. A Java program is created as a text file with the file extension .java. It is compiled into one or more files of byte codes with the extension .class. Byte codes are a set of instructions similar to the machine code instructions created when a computer program is compiled. The difference is that machine code must run on the computer system it was compiled for; byte codes can run on any computer system equipped to handle Java programs[5].

7.5 Features of Java

Java Virtual Machine(JVM)

The Java Virtual machine is a sort of interpreter that must be written for our particular computer. Byte code is a set of instructions to be run by the Java run-time system i.e the JVM. The security and portability of a Java program results from the fact that the output of a Java compiler is not an executable code; instead it is a byte code. It is this translation to byte code, which makes it very much easier to run a program in a wide variety of environments. This makes Java a platform independent language[9].

Java Packages

Packages in Java are a way of grouping together related classes and interfaces. The various classes used in this project are:

javax.media

This package is basically used for processing real time video and audio. It contains many classes which help in processing video frames such as FrameGrabbingControl, CaptureDe-viceManager etc. In our project, this package is used to grab the picture from the VGA web camera and convert it into frames taken over a fixed interval of time. These frames are then given to a ’ image processing ’ module for further processing.

javax.swing

Swing is a GUI toolkit for Java. Swing is one part of the Java Foundation Classes (JFC).

Swing includes graphical user interface (GUI) widgets such as text boxes, buttons,

split-CHAPTER 7. MACHINE VISION 40 panes, and tables. Swing widgets provide more sophisticated GUI components than the earlier Abstract Windowing Toolkit(AWT). Since they are written in pure Java, they run the same on all platforms, unlike the AWT which is tied to the underlying platform’s windowing system. Swing supports a pluggable look and feel - not by using the native platform’s facilities but by roughly emulating them. This means it provides any supported look and feel on any platform. The disadvantage of lightweight components is possibly slower execution. The advantage is uniform behavior on all platforms.

java.awt

AWT stands for Abstract Window Toolkit. It contains all of the classes for creating user interfaces and for painting graphics and images. A user interface object such as a button or a scrollbar is called, in AWT terminology, a component. The AWT Event class and its subclasses are used to represent the events that AWT components can fire. A container is a component that can contain components and other containers. A container can also have a layout manager that controls the visual placement of components in the container. The AWT package contains several layout manager classes and an interface for building our own layout manager.

java.io

It includes the input and output classes for writing to, and reading from streams and for handling files. Java programs perform I/O through streams. A stream is an abstraction that either produces or consumes information. Java implements streams within class hierarchies defined in the java.io package.

java.lang

It includes classes that apply to the language itself which includes the object class, the string class and the system class. All Java programs automatically import this package. The class called System encapsulates several aspects of the run-time environment. Systems contain three predefined stream variables, in, out, and err.

javax.comm.CommPort

CommPort is an abstract class that describes a communications port made available by the underlying system. It includes high-level methods for controlling I/O that are common to different kinds of communications ports. SerialPort and ParallelPort are subclasses of CommPort that include additional methods for low-level control of physical communications ports.

After a communications port has been identified and opened it can be configured with the methods in the low-level classes like SerialPort and ParallelPort. Then an IO stream can be opened for reading and writing data. Once the application is done with the port, it must call the close method.

CHAPTER 7. MACHINE VISION 41 javax.comm.CommPortIdentifier

There are no public constructors for CommPort. Instead an application should use the static method CommPortIdentifier.getPortIdentifiers to generate a list of available ports. It then chooses a port from this list and calls CommPortIdentifier.open to create a CommPort object. Finally, it casts the CommPort object to a physical communications device class like SerialPort or ParallelPort.

The main function of this class is Communications port management. CommPortIden-tifier is the central class for controlling access to communications ports. It includes methods for:

1. Determining the communications ports made available by the driver.

2. Opening communications ports for I/O operations.

3. Determining port ownership.

4. Resolving port ownership contention.

5. Managing events that indicate changes in port ownership status.

An application first uses methods in CommPortIdentifier to negotiate with the driver to discover which communication ports are available and then select a port for opening. It then uses methods in other classes like CommPort, ParallelPort and SerialPort to communicate through the port.

7.6 Process Flow

The AIBOT is set in free running state when the program starts. The sonar detects the presence of any obstacle in line with the AIBOT. The Java program running in the com-puter collects the data from the Master PIC through the serial port. The java program sends a value 34(address of SONAR) continuously to the master PIC. If the SONAR finds an obstacle, the master PIC sends the distance back to the Java program. The program checks whether the distance obtained is between 20cm to 34cm, the program sends signals to the Master PIC for applying brake to DC motors in the base. When the base stops, the program captures frame using the web cam and the ball detection program is initiated.

The algorithm for ball detection and tracking implemented is basically inspired by Hough transform algorithm already explained. Algorithm was first coded in MATLAB and was later converted to java. Java processing took around 1 sec while MATLAB took around 3-4 minutes. In the program a video player is first created, from the player a frame is grabbed and the frame is converted to an image and is saved in the PC.

All the images can be converted to a one dimensional array in Java and from that one dimensional array we can find out the RGB values of each pixel. The image is then resized to half the original size and converted to gray scale form. Then edges are found out using

CHAPTER 7. MACHINE VISION 42 the SOBEL filter implemented in java.. This image is converted to one dimensional array.

The one dimensional array is passed on to the Hough transform function to find out the circles present in the image. The possible circles with its centers and radius are achieved from this function. Then the average value of R, G and B in that circle are found out and the colour of the ball is found out.

If the obstacle is a white ball then the PIC sends values to the master PIC for the movement of hand. The Master PIC sends values to the PIC that controls the hand. The hand grabs the ball and brings it to a particular position. If the image captured does not contain the specfic coloured ball it takes a deviation from the obstacle and moves through its side. The Java program again checks for the obstacle and follows the above steps again.

The algorithm is shown in figure 7.8 The Addressing Scheme

The whole system consists of several PIC microcontrollers. Unique addresses are given to each PICs so that the master PIC could communicate with all the slave PICs easily. Java program directly communicates with the master PIC only using its USART module. The java program sends a set of seven values for communicating with the corresponding slave PIC. In all the sets of seven, first value corresponds to the address of the PIC. Following are the different addresses given to each PIC and the remaining values of that set. For example the value for the DC motor in the hand are as follows

7 - byte Data Format for Base 1 - Base PIC Address

7 - byte Data Format for Hand 1 - Hand PIC Address

CHAPTER 7. MACHINE VISION 43 7 - byte Data Format for SONAR

1 - 0X34 Constant for SONAR Communication 2 - SONAR PIC Address

3 - Not used 4 - Not used 5 - Not used 6 - Not used 7 - Not used

Master PIC receive the distance data from the SONAR modules after sending 7ˆth byte.

CHAPTER 7. MACHINE VISION 44

Figure 7.4: Matlab’s intermediate results

CHAPTER 7. MACHINE VISION 45

Figure 7.5: Matlab output No.1

CHAPTER 7. MACHINE VISION 46

Figure 7.6: Matlab output No.2

CHAPTER 7. MACHINE VISION 47

Figure 7.7: Java program screen shot

CHAPTER 7. MACHINE VISION 48

Figure 7.8: Java Algorithm

Chapter 8