I assuming that you are able to bring up a stylesheet that you wish to edit.

In one of my stylesheets I have a modified style for Section. I can't copy out what is displayed in the stylesheet but it says:

Local definition for style "Section" and then below that shows what the Section header actually looks like.

You could copy this down into your stylesheet using the "Choose a style" dropdown menu at the top. Or you could type "Section" in the input field at the top. Or you could copy and paste it from the Default stylesheet. Then select the Section cell bracket on the right and use Ctrl+Shift+E. That shows the underlying expression for the Section style. In my case it is:

Cell[StyleData["Section"],
 CellFrame->False,
 ShowGroupOpener->True,
 AutoItalicWords->{"Presentations", "Mathematica", "Tensorial"},
 FontFamily->"Book Antiqua",
 FontSize->22,
 FontWeight->"Plain",
 FontSlant->"Plain",
 FontVariations->{"StrikeThrough"->False,
 "Underline"->False},
 FontColor->GrayLevel[0]]

(This is just plain text typed into the cell and the contents may be different in your case.)

The style is defined by the various options. Not all of these are defined on the Format menu, which is why I prefer to edit the underlying expression. You only have to add the things you want changed. The option you are most interested in is "CellFrame -> False". After you make the changes you want, use Ctrl+Shift+E again to toggle back to the normal display. Any frame on the Section cell should now be gone. If you have a notebook open that is using the stylesheet it should also be gone there.

If the stylesheet is attached to the notebook, then you are done and you can close it. If this is a style sheet to be used by a number of notebooks you will have to either save it, using SaveAs, in the Stylesheets folder in an application, or use the "Install Stylesheet" button at the top save it in a system location.

Dean,

The notebook you attached is not a Stylesheet notebook.

I have an essay A Mathematica Style that discusses how to set up a Stylesheet for your own work. It also has links to a Mathematica test notebook and a Mathematica Stylesheet that can help you get started.

Otherwise you have to start with an open notebook and use Menu/Format/Edit Stylesheet to bring up an initial Stylesheet.

The Stylesheet discussion in the essay is in terms of an application, but you could instead use the Install button on the Stylesheet to save it.

I would, however, urge you to set up your work as an application in your $UserBaseDirectory/Applications folder. Just create a folder there with a name such as 'Dean Physics Study'. Then you can add a place for Stylesheets, palettes and even add a package if you get around to it. Mathematica will automatically find everything. Maybe initially the application will just be for yourself, but later you may want to share parts of it with other people. If they unzip it into their UserBaseDirectory/Applications folder then everything will automatically be available to them. You can also include folders for your study notebooks. It's just a convenient way to organize your work, instead of having everything scattered around.