AWS BATCH EXAMPLE

Sample code to get started with AWS Batch.

Prerequisites:

Install AWS CLI Install Docker Install maven Setup either Docker hub/AWS ECR

Run the code

Step 1:

Download Code -> Open command Line -> go to the folder where you downloaded the code Run command

 mvn clean package
mkdir temp
cp target/etl-1.0-SNAPSHOT.jar temp/
cp Dockerfile temp/
cd temp

Step 2:

1. Login yo your docker hub or AWS ECR

docker build -t sentiment . docker tag sentiment:latest your-ecr-repository docker push your-ecr-repository

Step3:

Build a compute Environment:

 1. Run aws batch create-compute-environment --generate-cli-skeleton >> computeEnv.json 2. Fill in the values in computeEnv.json
computeEnv.json should look like:
{"computeEnvironmentName": "C4OnDemand",
"type": "MANAGED",
"state": "ENABLED",
"computeResources": {
"type": "EC2",
"minvCpus": 0,
"maxvCpus": 2,
"desiredvCpus": 0,
"instanceTypes": [
"c4.large"
],
"imageId": "",
"subnets": [
"subnet-0c96cc3be30e80b43"
],
"securityGroupIds": [
"sg-00a45f7d1b790586f"
],
"ec2KeyPair": "XYZ",
"instanceRole": "arn:aws:iam::XXX:instance-profile/ecs-instance-role",
"tags": {
"owner": "sunitm"
}
},
"serviceRole": "arn:aws:iam::XXX:role/aws-batch-service-role"
}
3. Runaws aws batch create-compute-environment --cli-input-json file://computeEnv.json --region ap-southeast-2

Build a Job Queue:

 1. Run aws batch create-job-queue --generate-cli-skeleton >> queue.json 2. Fill in the values in queue.json
queue.json should look like: {
"jobQueueName": "HighPriority",
"state": "ENABLED",
"priority": 10,
"computeEnvironmentOrder": [
{
"order": 10,
"computeEnvironment": "C4OnDemand"
}
]
}
3. Run aws batch create-job-queue --cli-input-json file://queue.json --region <aws-region>

Register a Job definition

 1. Run aws batch register-job-definition --generate-cli-skeleton >> jobDef.json 2. Fill in the values in jobDef.json
jobDef.json show look like:
{
"jobDefinitionName": "senti2",
"type": "container",
"parameters": {},
"containerProperties": {
"image": "00099.dkr.ecr.ap-southeast-2.amazonaws.com/sentiment",
"vcpus": 1,
"memory": 1024,
"command": [
"java",
"-jar",
"batch.jar",
"AWS_BATCH_IP_SOURCE_BUCKET",
"source-ip-batch-cicd",
"AWS_BATCH_IP_SOURCE_BUCKET_KEY",
"22571a6aef63447b84bd812fcf155827.csv",
"AWS_BATCH_IP_DESTINATION_BUCKET",
"source-ip-batch-cicd",
"AWS_BATCH_IP_DESTINATION_KEY",
"sent.txt"
],
"jobRoleArn": "",
"volumes": [],
"environment": [],
"mountPoints": [],
"ulimits": [] },
"retryStrategy": {
"attempts": 1
},
"timeout": {
"attemptDurationSeconds": 3000
}
} 3. aws batch register-job-definition --cli-input-json file://jobDef.json --region <aws-region>

Submit a Job

 aws batch submit-job --job-name example --job-queue HighPriority --job-definition senti2 --region <aws-region>

Note:

Syntax : file:// In my case it is in the same directory. "file://" is necessary to avoid encoding issues that the awscli might give you.