Shortest way of doing it might be:

dataset[All, Append[#, "d" -> #["a"] + #["b"]] &]

but might not be fastest.

Or simpler notation:

dataset[All, <|#, "e" -> #a + #b|> &]

if your dataset is very big you probably don't want to use Dataset ;-) It is very handy and flexible but that comes at the expense of memory usage and speed in some cases…

It is very similar to a hash table indeed. But finding/manipulating data in "named" columns just takes more time than if it the can simply be accessed by an index. You can store the data as a 'matrix' (list of lists) and 'remembering' the columns yourself, generally much faster. Dataset can also handle data as just matrices, rather than associations.

That's the balance: error-prone/flexible/convenience vs speed/less memory. Associations are implemented very efficiently but if you can avoid it and you can use packed arrays than that is (generally) much faster.

Not quite, but what I'm asking for isn't much different than adding a column to a 2-D array which can be a bit tedious with

MapThread[Append,{twoDarray,newColumn}]

I can use MapThread if I convert the datasets to associations, do the MapThread, then turn it back into an association

Dataset@MapThread[Append, 
  Normal /@ {dataset, dataset[All, <|"d" -> #a + #b|> &]}]

Using Transpose instead

The route that seems a little more intuitive is to use Transpose. So for adding a column to a 2-D array

Transpose[Append[Transpose[twoDarray],newColumn]]

Which is analogous to the approach I'm using with "joinDataset". I think Datasets are a great way to keep track of a lot of data in a human-usable form.

Extracting column/row with key

One other thing I wonder about is, how do I retain the key when extracting a single value from the Dataset? So for instance, say I want to take dataset[All,"b"] and add it to a different Dataset dataset2? I can't do this:

dataset2 = dataset[All, {"a", "c"}]
joinDataset[{dataset2, dataset[All, "b"]}]

I have to map an association to each element first

joinDataset[{dataset2, <|"b" -> #|> & /@ dataset[All, "b"]}]