I think the problem is that, if you were doing this with tensor operations only, is that you would need a contraction operation akin to Diagonal. It seems like the syntax (currently allowed) for PartLayer is insufficient for this. Take the case with a=c=2, b=d=1, then you can reshape both tensors into 2x2 matrices. The product is a 2x2 matrix, but you only want the diagonal of the product.
Perhaps the right thing to do is to split both tensors into its parts, each part is its own layer. This might be tedious if you typed it in by hand, but you can create the layers programmatically (e.g. Table[PartLayer[n],{n,a}] ) and use some (hopefully not difficult) arithmetic to refer to the appropriate layers.
