Installed data will generally be in your user-specific base directory. You can open this in your OS with:
$UserBaseDirectory // SystemOpen
A package will generally be added under the "Applications"
sub-folder so delete it there. The only common case I can think of where it wouldn't be there is if it's a paclet. Then you'd find it here:
$UserBasePacletsDirectory // SystemOpen
Or better:
PacletFind["pacletName"]
But the Install...
menu item doesn't seem to do that.