Hi,
I think the construction you gave produces the left cosets. To get the right cosets use PermutationProduct[#2, #1]& in the third argument. This is because the action function f[x, g] in the third argument is a right action so it lets generators g of the group G act on elements x of the orbits (in your case elements of Sn). To construct the right cosets Gx we need to reverse the product. The left cosets xG do not need the reversal of the product.
You can check that the cosets are correct by showing that all elements of a given coset have the same canonical representative. A simple example with n = 4:
In[1]:= G = DihedralGroup[2];
In[2]:= S4list = GroupElements[SymmetricGroup[4]];
In[3]:= rightcosets = GroupOrbits[G, S4list, PermutationProduct[#2, #1] &];
In[4]:= leftcosets = GroupOrbits[G, S4list, PermutationProduct[#1, #2] &];
Here we see that all representatives (using RightCosetRepresentative) of the same coset coincide:
In[5]:= Length /@ DeleteDuplicates /@ Map[RightCosetRepresentative[G, #] &, rightcosets, {2}]
Out[5]= {1, 1, 1, 1, 1, 1}
That does not happen here:
In[6]:= Length /@ DeleteDuplicates /@ Map[RightCosetRepresentative[G, #] &, leftcosets, {2}]
Out[6]= {1, 4, 4, 4, 4, 1}
If this does not solve the issue, please let us know the particular example that seems to be wrong.
Jose.
