Band[{1, 1}] -> 0
can be omitted, certainly. Also, you can put in the assignments of the corner elements in SparseArray[]
itself. Thus,
With[{n = 5},
SparseArray[{Band[{2, 1}] -> I, Band[{1, 2}] -> I,
{1, -1} -> 1, {-1, 1} -> 1}, {n, n}]] // MatrixForm