Message Boards Message Boards

[Hackathon] Tic-Tac-Toe with autonomous Delta Robot

Hello! Last weekend I was at a hackathon at Georgetown University where my team and I wrote a python program that allowed our home made delta robot to autonomously play tic tac toe with a human user. The Mathematica program I was working on was written to interface the arduino controlling the delta robot, and the python program which has the tic tac toe algorithm, by relaying information about the user input on the board through image processing.

enter image description here

The user input had to be from the computer however because I wasn't able to finish the image processing. The img was an image of the board taken by a logitech web cam attached to the main shaft of the robot. It didn't work reliably but did work generally.

Does anyone have a better way of doing this? Thanks!

P.S. here is the code: http://paste.ofcode.org/JCyYF2tJtpcmRta8YjMhmW

src = ColorConvert[img, "Grayscale"];
white = Closing[src, DiskMatrix[5]];
srcAdjusted = Image[ImageData[src]/ImageData[white]]
components = 
  ComponentMeasurements[
    ColorNegate@Binarize[srcAdjusted], {"ConvexArea", "Mask"}][[All, 
   2]];
largestComponent = Image[SortBy[components, First][[-1, 2]]]
mask = FillingTransform[Closing[largestComponent, 2]]
lY = ImageMultiply[
   MorphologicalBinarize[
    GaussianFilter[srcAdjusted, 3, {2, 0}], {0.02, 0.05}], mask];
lX = ImageMultiply[
  MorphologicalBinarize[
   GaussianFilter[srcAdjusted, 3, {0, 2}], {0.02, 0.05}], mask]
verticalGridLineMasks = 
  SortBy[ComponentMeasurements[
      lX, {"CaliperLength", "Centroid", "Mask"}, # > 100 &][[All, 2]], #[[2, 1]] &][[All, 3]];
horizontalGridLineMasks = 
  SortBy[ComponentMeasurements[
      lY, {"CaliperLength", "Centroid", "Mask"}, # > 100 &][[All, 2]], #[[2, 2]] &][[All, 3]];
centerOfGravity[l_] := 
 ComponentMeasurements[Image[l], "Centroid"][[1, 2]]
gridCenters = 
  Table[centerOfGravity[
    ImageData[Dilation[Image[h], DiskMatrix[2]]]*
     ImageData[Dilation[Image[v], DiskMatrix[2]]]], {h, 
    horizontalGridLineMasks}, {v, verticalGridLineMasks}];
    Show[src, 
 Graphics[{Red, 
   MapIndexed[{Point[#1], Text[#2, #1, {1, 1}]} &, gridCenters, {2}]}]]


tr = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{4, 1}, {4, 2}, {3, 1}}], {{0, 0}, {0, 1}, {1, 0}}]
ts = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{3, 1}, {3, 2}, {2, 1}}], {{0, 0}, {0, 1}, {1, 0}}]
tt = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{2, 1}, {2, 2}, {1, 1}}], {{0, 0}, {0, 1}, {1, 0}}]
tu = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{4, 2}, {4, 3}, {3, 2}}], {{0, 0}, {0, 1}, {1, 0}}]
tv = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{3, 2}, {3, 3}, {2, 2}}], {{0, 0}, {0, 1}, {1, 0}}]
tv2 = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{2, 2}, {2, 3}, {1, 2}}], {{0, 0}, {0, 1}, {1, 0}}]
tw = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{4, 3}, {4, 4}, {3, 3}}], {{0, 0}, {0, 1}, {1, 0}}]
tx = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{3, 3}, {3, 4}, {2, 3}}], {{0, 0}, {0, 1}, {1, 0}}]
tz = Last@
  FindGeometricTransform[
   Extract[gridCenters, {{2, 3}, {2, 4}, {1, 3}}], {{0, 0}, {0, 1}, {1, 0}}]
pics = {ImageTransformation[src, tr, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, ts, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tt, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tu, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tv, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tv2, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tw, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tx, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}], 
  ImageTransformation[src, tz, {200, 200}, DataRange -> Full, 
   PlotRange -> {{0, 1}, {0, 1}}]}
   find[img_] := 
  (croppy = ImageCrop[img, {180, 180}];
   bin = ColorNegate@
     Binarize[ImageAdjust[GaussianFilter[croppy, 5]]];
   components = 
    ComponentMeasurements[
     bin, {"Centroid", "Area", "FilledCircularity", 
      "EquivalentDiskRadius"}, #2 > 100 &]; 
   Show[croppy, 
    Graphics[{Thick {If[#[[3]] > 0.5, Red, Blue], 
          Circle[#[[1]], #[[4]]]} & /@ components[[All, 2]]}]] );
POSTED BY: Jenny Mandl
2 Replies
Sort By:

Hello! It sounds like you are trying to identify and find the location of the objects on a tic tac toe board. Here is a notebook which does that. I do not have your exact lighting conditions, so I used an intentionally bad tic tac toe board made in paint. It definitely is not perfect, but maybe it will give you some ideas.

The image I used to test (high-res in the notebook), intentionally bad paint image to try and replicate less than ideal conditions

enter image description here

(*Smoothing out the image, and getting components. This is similar to the original post. Ideally the x's, o's, and grid are separate.*)

srcAdjusted = Erosion[img, DiskMatrix[5]];
components = ComponentMeasurements[ColorNegate@Binarize[srcAdjusted], {"ConvexArea", "Mask"}][[All, 
   2]];

(*Getting the grid component, filling in the center*)

largestComponent = Image[SortBy[components, "First"][[-1, 2]]];
mask = FillingTransform[Closing[largestComponent, 3]]

(*subtracting the component from the filled in center to get just the box, then finding the corners of it*)

box = Opening[ImageSubtract[mask, largestComponent], DiskMatrix[3]];
corners = ImageCorners[box, 20, .01, 25, MaxFeatures -> 4];
Show[box, ListPlot[corners]]

(*remove the grid from components, and obtain the image equivalent of each sparse array*)

nogrid = Delete[SortBy[components, First], -1];
imgcomponents = Map[Image[#[[2]] // Normal] &, nogrid]

(*Sort the corners, get the furthest left/right/top/bottom*)

sortx = SortBy[corners, #[[1]] &];
sorty = SortBy[corners, #[[2]] &];
{left, top, right, bottom} = {Mean[{sortx[[1, 1]], sortx[[2, 1]]}], 
  Mean[{sorty[[3, 2]], sorty[[4, 2]]}], Mean[{sortx[[3, 1]], sortx[[4, 1]]}], 
  Mean[{sorty[[1, 2]], sorty[[2, 2]]}]}

(*Obtain the center of each, figure out what space on the tictactoe board it should be*)

centerOfGravity[l_] := ComponentMeasurements[Image[l], "Centroid"][[1, 2]]
getrowcol[{x_, y_}]  := {If[y >= top, 1, If[y >= bottom, 2, 3]], 
  If[x <= left, 1, If[x <=  right, 2, 3]]  }
locations = Map[{getrowcol[centerOfGravity[#]], 
    If[Length[ImageLines[#, .18, .15]] > 1, "x", "o"]} &, imgcomponents ]

(*Make an empty array, map in the above data, display it in matrix form*)

tictactoe = {{"", "", ""}, {"", "", ""}, {"", "", ""}};
Map[(tictactoe[[#[[1, 1]], #[[1, 2]]]] = #[[2]]) &, locations];
tictactoe // MatrixForm

enter image description here

Attachments:
POSTED BY: Garrett Ducharme

Hello, I edited the post. The python was the tic tac toe algorithm and the goal was to have mathematica interface between the python and the arduino controlling the delta robot which is a 3 axis arm typically used for pick and place operations in manufacturing (we made a fully functioning one with 3D printed parts and stepper motors)
POSTED BY: Jenny Mandl
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract