Band[{1, 1}] -> 0 can be omitted, certainly. Also, you can put in the assignments of the corner elements in SparseArray[] itself. Thus,
With[{n = 5}, 
     SparseArray[{Band[{2, 1}] -> I, Band[{1, 2}] -> I,
                  {1, -1} -> 1, {-1, 1} -> 1}, {n, n}]] // MatrixForm