Docker build fails with download_payloads.sh $'\r': command not found

[peterborcsok]

Describe the bug
The Docker build is broken due to an encoding / line ending issue of download_payloads.sh script.

To Reproduce
Steps to reproduce the behavior:

1. As per the readme, execute: git clone https://github.com/mitre/caldera.git --recursive
2. cd caldera
3. docker build . --build-arg WIN_BUILD=true -t caldera:latest

Expected behavior
Successful Docker build.

Error

 => [24/30] RUN ./update-agents.sh                                                                                                                      0.7
 => [25/30] RUN if [ ! -d "/usr/src/app/plugins/atomic/data/atomic-red-team" ]; then       git clone --depth 1 https://github.com/redcanaryco/atomic-  11.6
 => [26/30] WORKDIR /usr/src/app/plugins/emu                                                                                                            0.1
 => ERROR [27/30] RUN if [ $(grep -c "\- emu" ../../conf/local.yml)  ]; then     apt-get -y install zlib1g unzip;                    pip3 install -r   12.4
------
 > [27/30] RUN if [ $(grep -c "\- emu" ../../conf/local.yml)  ]; then     apt-get -y install zlib1g unzip;                    pip3 install -r requirements.t;                   ./download_payloads.sh;                         fi:
0.414 Reading package lists...
1.267 Building dependency tree...
1.490 Reading state information...
1.842 zlib1g is already the newest version (1:1.3.dfsg-3.1ubuntu2.1).
1.842 Suggested packages:
1.842   zip
1.874 The following NEW packages will be installed:
1.874   unzip
2.214 0 upgraded, 1 newly installed, 0 to remove and 14 not upgraded.
2.214 Need to get 174 kB of archives.
2.214 After this operation, 384 kB of additional disk space will be used.
2.214 Get:1 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 unzip amd64 6.0-28ubuntu4.1 [174 kB]
2.525 debconf: delaying package configuration, since apt-utils is not installed
2.562 Fetched 174 kB in 0s (363 kB/s)
2.585 Selecting previously unselected package unzip.
(Reading database ... 45526 files and directories currently installed.)
2.608 Preparing to unpack .../unzip_6.0-28ubuntu4.1_amd64.deb ...
2.613 Unpacking unzip (6.0-28ubuntu4.1) ...
2.650 Setting up unzip (6.0-28ubuntu4.1) ...
3.379 Collecting pyminizip==0.2.6 (from -r requirements.txt (line 1))
3.525   Downloading pyminizip-0.2.6.tar.gz (261 kB)
3.608      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 261.2/261.2 kB 3.3 MB/s eta 0:00:00
3.667   Installing build dependencies: started
6.801   Installing build dependencies: finished with status 'done'
6.804   Getting requirements to build wheel: started
7.041   Getting requirements to build wheel: finished with status 'done'
7.044   Preparing metadata (pyproject.toml): started
7.283   Preparing metadata (pyproject.toml): finished with status 'done'
7.288 Building wheels for collected packages: pyminizip
7.291   Building wheel for pyminizip (pyproject.toml): started
11.71   Building wheel for pyminizip (pyproject.toml): finished with status 'done'
11.71   Created wheel for pyminizip: filename=pyminizip-0.2.6-cp312-cp312-linux_x86_64.whl size=209120 sha256=7773e2662e8152f815fa4fc0aece2fe001b632b491e633c5ad18cf1d84f2ef
11.72   Stored in directory: /root/.cache/pip/wheels/c1/0c/f0/a04bf9aa244d08cd2df915863ecbf4452f636e3b88ab82eef8
11.72 Successfully built pyminizip
12.13 Installing collected packages: pyminizip
12.15 Successfully installed pyminizip-0.2.6
12.29 ./download_payloads.sh: line 7: $'\r': command not found
12.30 curl: (3) URL rejected: Malformed input to a URL function
12.31 unzip:  cannot find or open payloads/AdFind.zip, payloads/AdFind.zip.zip or payloads/AdFind.zip.ZIP.
12.31 cp: cannot stat 'payloads/AdFind.exe': No such file or directory
12.31 ./download_payloads.sh: line 11: $'\r': command not found
12.32 curl: (3) URL rejected: Malformed input to a URL function
12.33 ./download_payloads.sh: line 13: $'\r': command not found
12.33 curl: (3) URL rejected: Malformed input to a URL function
12.34 unzip:  cannot find or open payloads/NetSess.zip, payloads/NetSess.zip.zip or payloads/NetSess.zip.ZIP.
12.34 cp: cannot stat 'payloads/NetSess.exe': No such file or directory
12.34 ./download_payloads.sh: line 17: $'\r': command not found
12.35 curl: (3) URL rejected: Malformed input to a URL function
12.35 ./download_payloads.sh: line 19: $'\r': command not found
12.36 curl: (3) URL rejected: Malformed input to a URL function
12.36 cp: cannot stat 'payloads/psexec.exe': No such file or directory
12.36 ./download_payloads.sh: line 22: $'\r': command not found
12.37 curl: (3) URL rejected: Malformed input to a URL function
12.37 ./download_payloads.sh: line 24: $'\r': command not found
12.37 curl: (3) URL rejected: Malformed input to a URL function
12.37 ./download_payloads.sh: line 26: $'\r': command not found
12.38 curl: (3) URL rejected: Malformed input to a URL function
12.38 ./download_payloads.sh: line 28: $'\r': command not found
12.39 curl: (3) URL rejected: Malformed input to a URL function
12.39 unzip:  cannot find or open payloads/wce_v1_41beta_universal.zip, payloads/wce_v1_41beta_universal.zip.zip or payloads/wce_v1_41beta_universal.zip.ZI12.39 ./download_payloads.sh: line 31: $'\r': command not found
12.39 curl: (3) URL rejected: Malformed input to a URL function
12.40 ./download_payloads.sh: line 33: $'\r': command not found
12.40 curl: (3) URL rejected: Malformed input to a URL function
12.40 ./download_payloads.sh: line 35: $'\r': command not found
12.41 curl: (3) URL rejected: Malformed input to a URL function
12.41 unzip:  cannot find or open payloads/PSTools.zip, payloads/PSTools.zip.zip or payloads/PSTools.zip.ZIP.
12.41 md5sum: payloads/PSTools/PsExec64.exe: No such file or directory
12.41 ./download_payloads.sh: line 57: syntax error: unexpected end of file
------
Dockerfile:72
--------------------
  71 |     # If emu is enabled, complete necessary installation steps
  72 | >>> RUN if [ $(grep -c "\- emu" ../../conf/local.yml)  ]; then \
  73 | >>>     apt-get -y install zlib1g unzip;                \
  74 | >>>     pip3 install -r requirements.txt;               \
  75 | >>>     ./download_payloads.sh;                         \
  76 | >>> fi
  77 |
--------------------
ERROR: failed to solve: process "/bin/bash -c if [ $(grep -c \"\\- emu\" ../../conf/local.yml)  ]; then     apt-get -y install zlib1g unzip;
   pip3 install -r requirements.txt;                   ./download_payloads.sh;                         fi" did not complete successfully: exit code: 2 

Desktop (please complete the following information):

• OS: WSL2/Ubuntu
• Browser: N/A
• Version: master branch

Additional context
Script caldera\plugins\emu\download_payloads.sh has CRLF file ending while Ubuntu expects LF only.

1. Script can be converted to have Unix line endings in the repo
2. dos2unix conversion during execution

I recommend dos2unix conversion to be included during execution because it can be hard to keep a differently encoded file in the repository.
I'll open a PR to do that and reference this issue.

[github-actions]

Looks like your first issue -- we aim to respond to issues as quickly as possible. In the meantime, check out our documentation here: http://caldera.readthedocs.io/

[uruwhy]

This should no longer be an issue with the latest changes to Dockerfile in the main branch - @peterborcsok let us know if you are still seeing problems