In A Short Talk on AI Ethics, Stephen Wolfram states

Well, if we can express laws in computable form maybe we can start telling AIs how we want them to act. Of course it might be better if we could boil everything down to simple principles, like Asimov’s Laws of Robotics, or utilitarianism or something.

But I don’t think anything like that is going to work. What we’re ultimately trying to do is to find perfect constraints on computation, but computation is something that’s in some sense infinitely wild. The issue already shows up in Gödel’s Theorem. Like let’s say we’re looking at integers and we’re trying to set up axioms to constrain them to just work the way we think they do. Well, what Gödel showed is that no finite set of axioms can ever achieve this. With any set of axioms you choose, there won’t just be the ordinary integers; there’ll also be other wild things.

And the phenomenon of computational irreducibility implies a much more general version of this. Basically, given any set of laws or constraints, there’ll always be “unintended consequences”. This isn’t particularly surprising if one looks at the evolution of human law. But the point is that there’s theoretically no way around it. It’s ubiquitous in the computational universe.

Question: Can someone explain Wolfram's argument more formally?

I've read NKS, as well as overviews on Godel's undecidability proofs. I don't see how this follows.

To me he seems to be saying that our practical ethics are incredibly complicated, and so cannot be captured by simple ethical laws. But that violates one of the whole points of NKS, which is that simple laws can generate incredible complexity. So why couldn't simple laws (something resembling Asimov's Laws, etc) turn out to compute our entire ethical framework?