Hi Daniel, it sounds like you got this working by creating a submission environment CloudFormation stack in a different region. For future reference, the region configured in the AWS service connection on your local machine (if you're using credential files, then that in the ~/.aws/config file, as described in this workflow) must match the region that the CloudFormation stack was created in. If a region isn't specified in ~/.aws/config, the default used is us-east-1.

I will make a note to improve the error message you encountered - it might be more clear if it said something like "The specified job definition arn:aws:batch:us-east-1:[...] does not exist in the XX-YYY-Z region."

I was able to get AWS to work. 1) I deleted all ssh-key pairs and created new ones. 2) re-ran aws configure. 3) Created a new service connection aws = ServiceConnect["AWS", "New"]

But, following the instructions in the workflow:

ec2 = aws["GetService", "Name" -> "EC2"]

and

ec2["DescribeInstanceTypes", "InstanceTypes" -> {"c5.12xlarge"}]

came back with

The DescribeInstanceTypes request failed with the following error from the server: "You are not authorized to perform this operation."

I'll update again if I make any progress.

Using "ServiceObject" -> "New" inside the association of the second argument of RemoteBatchSubmissionEnvironment:

RemoteBatchSubmissionEnvironment["AWSBatch", <|
  "JobQueue" -> 
   "xxx", 
  "JobDefinition" -> 
   "xxx-us-east-xxx\
WolframJobDefinition-xxx", 
  "IOBucket" -> 
   "xxx",
  "ServiceObject" -> "New"|>]

works.

However,

job = RemoteBatchSubmit[env, 2 + 2]

fails with

RemoteBatchSubmit::awsbatch-missingdefinition: The specified job definition xxx:us-east-xxx does not exist.

Progress. Will report back