I agree this isn't related to encoding, it's best to stick to UTF-8. There is a hidden option to CSV and TSV Export "ForceDelimieters" that controls whether the setting for "TextDelimiters" is wrapped around text fields always, or only when needed. Since this isn't documented yet, this might change in future versions. The reason we always wrap non-numeric values in the delimiter is export would be much slower if it has to test each string for the presence of a comma, newline, or delimiter. Compare these outputs:
In[2382]:= ExportString[{{1, "B", "test,comma"}}, "CSV"]
Out[2382]= "1,\"B\",\"test,comma\"
"
In[2383]:= ExportString[{{1, "B", "test,comma"}}, "CSV",
"ForceDelimiters" -> False]
Out[2383]= "1,B,\"test,comma\"
"
In[2385]:= ExportString[{{1, "B", "test,comma"}}, "CSV",
"ForceDelimiters" -> False, "TextDelimiters" -> "'"]
Out[2385]= "1,B,'test,comma'
"
If your data does not contain any commas or newlines, then the best approach is to just disable text delimiters all together so that the string matching doesn't kick in (notice this mangles the field with a comma):
In[2387]:= ExportString[{{1, "B", "test,comma"}}, "CSV",
"TextDelimiters" -> None]
Out[2387]= "1,B,test,comma
"
P.S. I stumbled on this post by accident, and probably would have seen it sooner if CSV was in the title :)