Getting git experiences

Had to get a file contents dynamically from the AWS codecommit repo and supply those contents as an array list to the groovy shell in a jenkins job. This file, belongs to an AWS codecommit repo, is so small and just has some info as one word per line. Those items are related to the tags (modules) which are defined in the robot file destined for automation.

I can have the file put in the jenkins server manually and read it from the location from the Jenkins’s groovy shell. But that file might be modified and/or contents might be added in the due course whenever new modules get added for automation. The contents of that file will be shown as the checkbox items whenever you hit the jenkins job’s ‘build with parameters’ button to trigger the job and user has to supply those checkbox items (selecting one or many) as inputs to the jenkins job. So, getting that file from the codecommit repo dynamically whenever the Jenkins job runs is inevitable, otherwise managing the file manually would be costly laborious affair.

Well, that’s the background story. So, getting or downloading or checking out that module list items file from the codecommit repo without cloning the entire repo is the initial step. I can’t use the SCM step (which clones the entire repo) for the automation job because we will have to show the items as checkbox items before cloning the repo. Hope you got it.

Know that there are couple of ways to checkout a singe file from the repo by using git sparse checkout or git archives or some other ways. Even one can use wget on the git raw content url (for example, like, https://raw.githubusercontent.com/ambatigan/list-items/master/items_list.txt) if it is github repo. But this is AWS Codecommit repo. They will not provide the raw content urls like github. AWS provides some API calls for getting the blob of the file content though, but they should be authenticated first.

If you have svn installed in your server, you can also try ‘svn export’ on the github url. (for example, svn export https://github.com/ambatigan/list-items.git/trunk/items_list.txt). But this approach for the AWS codecommit needs the credentials to be supplied in the command itself.

Discovered that in the AWS CLI for codecommit (for the latest CLI version, 1.16.x ONLY), they introduced get-file subcommand for aws codecommit. The response of this get-file command output gives the fileContent in the base64 encoding. And you can use the default base64 decoder in the linux server, to decode it back to the original content.

For example, the response to the get-file command (ex: aws codecommit get-file –repository-name Testing-Automation –file-path /Jenkins/Dev/Resources/Input_data/tag_names.txt) is as follows:

[ec2-user@ip-60-0-1-94 ~]$ aws codecommit get-file –repository-name Testing-Automation –file-path /Jenkins/Dev/Resources/Input_data/tag_names.txt
{
“filePath”: “Jenkins/Dev/Resources/Input_data/tag_names.txt”,
“blobId”: “a6c7ac16cf059e739c3ad50efc2375d95feea03c”,
“commitId”: “e2a51016ce9b3f281504257124e6b6d72d3e338e”,
“fileSize”: 26,
“fileContent”: “QWxsCkxvZ2luClRyZW5kcwpTaXRlX21hcAo=”,
“fileMode”: “NORMAL”
}

And we can decode it like –

[ec2-user@ip-60-0-1-94 ~]$ echo QWxsCkxvZ2luClRyZW5kcwpTaXRlX21hcAo= | base64 -d
All
Login
Trends
Site_map
[ec2-user@ip-60-0-1-94 ~]$

Summary – echo ` aws codecommit get-file –repository-name Testing-Automation –file-path /Jenkins/Dev/Resources/Input_data/tag_names.txt|jq -r ‘.fileContent’`|base64 -d

Refer to https://github.com/ambatigan/list-items – to know how I implemented this core concept into the groovy shell in the Jenkins job.