Hi Nagata Hannya.
It is always easier if you post all the code necessary to reproduce your problem or question.
That being said, if I understand you correctly, you have two problems, you want to put the label under your grid and you want to make it so that the whole image can be copied as one.
The first problem could be solved by using the Labeled[expr,lbl,pos] function. Your grid is the expression, the caption is the label and pos should be "Bottom" if you want it to be placed under your grid. Bottom is the default for Labeled so actually you don't have to specify a position.
Your second question is trickier and I'm sure someone else has a better suggestion than this but you could always Rasterize the complete image. This would combine your graphics with your text and allow it to be copied together (although the graphics would no longer be vectors, which could be a deal-breaker, depending on what you want to use it for) .
i = 0
Dynamic@Rasterize@Labeled[
Graphics[{
Thick, Green, Rectangle[{0, -1}, {2, 1}], ColorData[1][i],
Disk[], Blue, Circle[{2, 0}], Yellow,
Polygon[{{2, 0}, {4, 1}, {4, -1}}], Purple, Arrowheads[Large],
Arrow[{{4, 3/2}, {0, 3/2}, {0, 0}}], Black, Dashed,
Line[{{-1, 0}, {4, 0}}]}],
Style["Color Index: " <> ToString[i], FontColor -> ColorData[1][i]]]
Slider[Dynamic[i], {0, 15, 1}]
Attachments: