STANDALONE TERRESTRIAL IMAGE PROCESSING ROBOT (STI-RO)

           Through this project, we are aiming more into image processing in embedded devices and platforms. This will enhance the mobility of the robot by reducing the hardware necessary for image processing using traditional methods.  Open CV or Open Computer Vision is a special kind of library developed in these days. This library is a cross-platform. It focusses mainly on real time image processing. If the library finds Intel’s integrated performance primitives on the system, it will use these proprietary optimized routines to accelerate itself.  The library will work on almost all OS like Windows7, Windows8, Android, Ubuntu, Debian etc. Since Open CV is light weight and can run in embedded platforms, our project aims to bring the image processing into new levels and will help to innovate more and bring more idea. Our project aims to make a mobile robot. The robot can process image in real time and can take decisions by itself. An arm is attached to the robot so that it can move, pick or place objects. This type of robots can be used to pick objects in Mars or other planet where data sending take ages.  It can also be used in military field for bomb diffusion.   

               Since our project is a Standalone robot which can be used terrestrially and is able to process image at real time, we named our project as STI-RO, which is Standalone Terrestrial   Image processing Robot. 

WORKING

Phase 1: CIRCUIT

01: Left Wheel motor
02: Right Wheel Motor
03: Arm Motor
04: Grip Motor
05: Motor Driver Circuit
06: BBB Rev C
07: Acceleromter (Not necessary)
08: Logic level Sifter
09: PIC16F877A
10: USB HUB
11:Web Cam

                      The image from webcam is fed to BBB . It process the image and out data to PIC. PIC will control the motors corresponding to the data. The feed-backs are also fed to PIC to analyse the current position of arm and grip of ROBOT.

                      Through this project, we are aiming more into image processing in embedded devices and platforms. This will enhance the mobility of the robot by reducing the hardware necessary for image processing using traditional methods.  Open CV or Open Computer Vision is a special kind of library developed in these days. This library is a cross-platform. It focusses mainly on real time image processing. If the library finds Intel’s integrated performance primitives on the system, it will use these proprietary optimized routines to accelerate itself.  The library will work on almost all OS like Windows7, Windows8, Android, Ubuntu, Debian etc. Since Open CV is light weight and can run in embedded platforms, our project aims to bring the image processing into new levels and will help to innovate more and bring more idea. Our project aims to make a mobile robot. The robot can process image in real time and can take decisions by itself. An arm is attached to the robot so that it can move, pick or place objects. This type of robots can be used to pick objects in Mars or other planet where data sending take ages.  It can also be used in military field for bomb diffusion.   

                       Since our project is a Standalone robot which can be used terrestrially and is able to process image at real time, we named our project as STI-RO, which is Standalone Terrestrial   Image processing Robot. 

Phase 2: PCB

TOP VIEW
Note: The figure below is not the final PCB coz I lost the PCB file. So add a 5 ohm resistor(not shown in figure) series with each voltage regulator (7805) output to maintain equal load bearing. 

BOTTOM VIEW

BOTTOM ( COPPER + SILK )

Phase 3: ASSEMBLY

This is the ROBOT we are talking about.

The webcam I’m using was not so small as my requirement.

So I took out the PCB.

Hm…. Better….!

The camera signal is fed to BBB

This PCB handles power signals as well as control signals.Microcontroller :PIC16F877a

Motor Driver :L293D

Yep.. This PCB powers BBB

Used only 3 IO pins BBB to transfer data in parallel.

Phase 4: PYTHON PROGRAM

1) The Python program need extra libraries to work like Opencv, numpy etc.
Refer this link to install them (Works perfectly in Ubuntu / Debian OS)
Link: http://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html

2) The program output data like “move STIRO right, left etc”. You need to out some values corresponding to this output through IO pins in Beaglebone Black.

Eg: moving forward = 010

moving right = 001

moving left = 100

pick ball = 111

Refer this PDF for setting and outing values through IO pins
Link: https://learn.adafruit.com/downloads/pdf/setting-up-io-python-library-on-beaglebone-black.pdf

Program
Note: STI-RO is my robot name

import numpy as np
import cv2
import time
cap =cv2.VideoCapture(1)
j=0
i=0
while(True):
j=(j+1)
ret, frame=cap.read()
output=frame
hsv=cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)
lower_blue=np.array([40,50,20])
upper_blue=np.array([100,240,240])
mask=cv2.inRange(hsv,lower_blue,upper_blue)
res=cv2.bitwise_and(frame,frame,mask=mask)
erode=cv2.erode(mask,None,iterations=5)
dilate=cv2.dilate(erode,None,iterations=5)
contours,hierarchy=cv2.findContours(erode,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
nw=0
nh=0
nx=0
ny=0
ncnt=0
i=0
for cnt in contours:
x,y,w,h=cv2.boundingRect(cnt)
if 1.2>w/h>0.8 or 1.2>h/w>0.8:
if nw<w:
ncnt=cnt
nw=w
nh=h
nx=x
ny=y
i=(i+1)
if (i!=0) and (nw>70):

leftmost = (ncnt[ncnt[:,:,0].argmin()][0])
rightmost = (ncnt[ncnt[:,:,0].argmax()][0])
topmost = (ncnt[ncnt[:,:,1].argmin()][0])
bottommost = (ncnt[ncnt[:,:,1].argmax()][0])
moments = cv2.moments(ncnt)
yc = moments['m01'] / moments['m00']
xc = moments['m10'] / moments['m00']
x_diff=((topmost[0:-1])-xc)
y_diff=((leftmost[1:])-yc)
if (-35<x_diff<15) and (-35<y_diff<15) and (-15<(nw-nh)<15):
print 'diff= ',nw-nh
print 'CONCLUSIONS FROM THE OBTAINED DATA'
print 'BALL FOUND'
print 'width in pixels= ',nw
if (300<xc<340):
print 'Distance form center= ',320-xc
print 'The ball is at center'
print '.................................'
if (260<nw<300):
print 'The ball is in capture range of 20cm'
print 'PICK THE BALL...!'
elif (nw<260):
print 'The ball far away from capture range'
print 'Move STI-RO forward'
else:
print 'The ball is more nearer than capture range'
print 'Move STI-RO backward'
elif (xc>340):
print 'Distance from center= ',320-xc,'pixels'
print 'The ball is in right side'
print 'Turn STI-RO right'
print '.................................'
print '.................................'
else:
print 'Distance from center= ',320-xc,'pixels'
print 'The ball is in left side'
print 'Turn STI-RO left'
print '.................................'
print '.................................'
else:
print 'diff= ',nw-nh
print 'CONCLUSIONS FROM THE OBTAINED DATA'
print 'BALL NOT FOUND'
print 'CONTINUE SEARCHING FOR BALL'
print '.................................'
print '.................................'
print '.................................'
print '.................................'
print '.................................'
print '____________________________________________________'
cv2.line(output,(310,200),(310,280),(0,255,0),5) #range left
cv2.line(output,(330,200),(330,280),(0,255,0),5) #range right
cv2.line(output,(180,140),(180,340),(255,0,0),15) #range capture
cv2.line(output,(460,140),(460,340),(255,0,0),15) #range capture
cv2.rectangle(output,(nx,ny),(nx+nw,ny+nh),[0,255,255],2) #box
cv2.line(output,(nx+nw/2,0),(nx+nw/2,480),(0,0,255),2) #target x
cv2.line(output,(0,ny+nh/2),(640,ny+nh/2),(0,0,255),2) #target y
cv2.imshow('output',output)
cv2.imshow('mask', mask)
cv2.imshow('res',res)
cv2.imshow('erode',erode)
cv2.imshow('dilatte',dilate)
if cv2.waitKey(1) &0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

Phase 5: PIC PROGRAM

This program will help to control the DC motors using the inputs from the BeagleBone Black.
Compiler : PICC
uC: PIC16f877a

#define FB_GRIP_C pin_A0

#define FB_GRIP_O pin_A1

#define FB_ARM_UP pin_A2

#define FB_ARM_DN pin_A3

#define L0 pin_B7

#define L1 pin_B6

#define L2 pin_B5

#define L3 pin_B4

#define ARM_0 pin_D2

#define ARM_1 pin_D3

#define GRIP_0 pin_C4

#define GRIP_1 pin_C5

#define LMOTOR_0 pin_C6

#define LMOTOR_1 pin_C7

#define RMOTOR_0 pin_D4

#define RMOTOR_1 pin_D5

void arm_up(int i)

{

   while (input(FB_ARM_UP)==0)

   {

      output_high(ARM_0);

      output_low(ARM_1);

      delay_ms(500);

      output_low(ARM_0);

      output_low(ARM_1);

      //delay_ms(500);

   }

  return;

}

void arm_dn(int i)

{

   while (input(FB_ARM_DN)==0)

   {

      output_low(ARM_0);

      output_high(ARM_1);

      delay_ms(500);

      output_low(ARM_0);

      output_low(ARM_1);

      //delay_ms(500);

   }

  return;

}

void grip_o(int i)

{  i=1;

   while (i<=7)

   {

      output_high(GRIP_0);

      output_low(GRIP_1);

      delay_ms(500);

      output_low(GRIP_0);

      output_low(GRIP_1);

      //delay_ms(500);

      i++;

   }

  return;

}

void grip_c(int i)

{  i=1;

   while (i<=7)

   {

      output_low(GRIP_0);

      output_high(GRIP_1);

      delay_ms(500);

      output_low(GRIP_0);

      output_low(GRIP_1);

      //delay_ms(500);

      i++;

   }

  return;

}

void move_fd(int i)

{

   output_low(LMOTOR_0);

   output_high(LMOTOR_1);

   output_high(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(200);

   output_low(LMOTOR_0);

   output_low(LMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void move_bd(int i)

{

   output_high(LMOTOR_0);

   output_low(LMOTOR_1);

   output_low(RMOTOR_0);

   output_high(RMOTOR_1);

   delay_ms(200);

   output_low(LMOTOR_0);

   output_low(LMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void move_rt(int i)

{

   output_low(LMOTOR_0);

   output_high(LMOTOR_1);

   output_low(RMOTOR_0);

   output_high(RMOTOR_1);

   delay_ms(100);

   output_low(LMOTOR_0);

   output_low(LMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void move_lt(int i)

{

   output_high(LMOTOR_0);

   output_low(LMOTOR_1);

   output_high(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(100);

   output_low(LMOTOR_0);

   output_low(lMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void tilt_rt(int i)

{

   output_low(LMOTOR_0);

   output_high(LMOTOR_1);

   output_low(RMOTOR_0);

   output_high(RMOTOR_1);

   delay_ms(100);

   output_low(LMOTOR_0);

   output_low(lMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void tilt_lt(int i)

{

   output_high(LMOTOR_0);

   output_low(LMOTOR_1);

   output_high(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(100);

   output_low(LMOTOR_0);

   output_low(LMOTOR_1);

   output_low(RMOTOR_0);

   output_low(RMOTOR_1);

   delay_ms(300);

}

void main()

{

  

   setup_adc_ports(NO_ANALOGS);

   setup_adc(ADC_OFF);

   setup_psp(PSP_DISABLED);

   setup_spi(SPI_SS_DISABLED);

   setup_timer_0(RTCC_INTERNAL|RTCC_DIV_1);

   setup_timer_1(T1_DISABLED);

   setup_timer_2(T2_DISABLED,0,1);

   setup_comparator(NC_NC_NC_NC);

   setup_vref(FALSE);

  

   output_c(0x00);

   output_d(0x00);

   delay_ms(1000);

   output_c(0xFF);

   output_d(0xFF);

   delay_ms(10000);

   output_c(0x00);

   output_d(0x00);

   delay_ms(1000);

   arm_up(1);

   grip_o(1);

while(1)

   {

   if (input(L0)==1 && input(L1)==1 &&  input(L2)==1)

      {

      arm_dn(1);

      grip_c(1);

      arm_up(1);

      grip_o(1);

      }

     

   else if (input(L0)==1 && input(L1)==1 &&  input(L2)==0)

      {

      tilt_lt(1);

      }

     

   else if (input(L0)==0 && input(L1)==1 &&  input(L2)==1)

      {

      tilt_rt(1);

      }

     

   else if (input(L0)==1 && input(L1)==0 &&  input(L2)==0)

      {

      move_lt(1);

      }

     

   else if (input(L0)==0 && input(L1)==0 &&  input(L2)==1)

      {

      move_rt(1);

      }

     

   else if (input(L0)==1 && input(L1)==0 &&  input(L2)==1)

      {

      move_bd(1);

      }

     

   else if (input(L0)==0 && input(L1)==1 &&  input(L2)==0)

      {

      move_fd(1);

      }

     

   else if (input(L0)==0 && input(L1)==0 &&  input(L2)==0)

      {

      move_rt(1);

      delay_ms(1000);

      }

   }

}