orjanj / nmap-pkg-generation-analysis Goto Github PK

Create script for preparing the target to scans.

Methods needed:

• Change parameters if the machine is cloned from template:
  • Set static IP address (/etc/netplan/00-installer-config.yaml)
  • Change hostname of target host (hostnamectl set-hostname <hostname>)
  • Possible to run netplan apply to activate new settings
  • Change SSH config to listen to management NIC (/etc/ssh/sshd_config -> ListenAddress a.b.c.d)
• Connectivity checks to validate management listening connection
• Check if tcpdump is installed
• PKI (key exchange from scanner-host to worker-host
  • Add scanner-host public key to authorized keys
  • Add user config file on scanner host to connect to targets (~/ssh/config);

Host *-mng
   User <username>
   IdentityFile <path to SSH private key>

• Same private key on root user on scanner as regular user?
• Make sure that tcpdump is chmod to sticky so that regular users could run the command
• Change the hostname (if the machine is cloned from a template)

Have added _ after the task name to identify the uniqueness of the task.
Earlier the $TASK_NAME might return multiple tasks (e.g. having 100 similar tasks, and _7 will match also _7{0-9}).

This is a logical error within the deploy_tasks_to_worker() function where the status check for ongoing returns the same task value through each iteration.
Needs to be rewritten and logic should be implemented within worker host iteration instead.

If there exists results on the scanning host and worker host, there must be created a new file to prevent overwriting existing data files.
Other suggestions:

• Create script on the scanning side that fetches results from the scanning host and worker host and structure data in a backuped result directory
• Create script on the worker side to structure data in a result directory

Bug message:

# scan.sh: line 26: [: 13480: binary operator expected

The need for parsing bulk pcap files needed.
Have created a script for this procedure.

Using the tcpdump -U parameter will packet-buffer the output, so that the output is written to stdout in the end of each packet.
Ref: https://www.tcpdump.org/manpages/tcpdump.1.html#lbAE

It takes time to conduct scans with various template parameters.
Needs to implement threading to streamline the scanner script.

Running the following command on three different target test virtual machines makes the SSH service unreachable;

tcpdump -i ens33 -nn -s 0 -U -w pcapname.pcap ip and not port 22

When running nmap -oX <task_name>.xml <hostname> -T0 -sX the following error occurs:

mass_dns: warning: Unable to determine any DNS servers. Reverse DNS is disabled. Try using --system-dns or specify valid servers with --dns-servers

Noise is generated with the packet captures, which causes a more unreliable data set when conducting analysis.

Tasks

• Disable snapd services
  • snapd.socket
  • snapd.service
  • snapd.seeded
  • snapd.snap-repair.timer
• Turn off automatic date and time
• Disable automatic updates through APT
• Diagonse other services using NTP/DNS
• Disable IPv6 (see issue #21)

Example noise output

tcpdump: listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
23:37:58.611322 IP (tos 0xc0, ttl 64, id 2173, offset 0, flags [none], proto ICMP (1), length 90)
    192.168.2.1 > 192.168.2.104: ICMP 192.168.2.1 udp port 53 unreachable, length 70
	IP (tos 0x0, ttl 64, id 23659, offset 0, flags [DF], proto UDP (17), length 62)
    192.168.2.104.40982 > 192.168.2.1.53: [udp sum ok] 1670+ A? api.snapcraft.io. (34)
	0x0000:  000c 29b8 e4d8 0050 56c0 0001 0800 45c0  ..)....PV.....E.
	0x0010:  005a 087d 0000 4001 ebac c0a8 0201 c0a8  .Z.}..@.........
	0x0020:  0268 0303 82f2 0000 0000 4500 003e 5c6b  .h........E..>\k
	0x0030:  4000 4011 588a c0a8 0268 c0a8 0201 a016  @[email protected]......
	0x0040:  0035 002a 2bca 0686 0100 0001 0000 0000  .5.*+...........
	0x0050:  0000 0361 7069 0973 6e61 7063 7261 6674  ...api.snapcraft
	0x0060:  0269 6f00 0001 0001                      .io.....
23:37:59.930274 IP (tos 0xc0, ttl 64, id 2395, offset 0, flags [none], proto ICMP (1), length 88)
    192.168.2.1 > 192.168.2.104: ICMP 192.168.2.1 udp port 53 unreachable, length 68
	IP (tos 0x0, ttl 64, id 23952, offset 0, flags [DF], proto UDP (17), length 60)
    192.168.2.104.58574 > 192.168.2.1.53: [udp sum ok] 33570+ A? ntp.ubuntu.com. (32)
	0x0000:  000c 29b8 e4d8 0050 56c0 0001 0800 45c0  ..)....PV.....E.
	0x0010:  0058 095b 0000 4001 ead0 c0a8 0201 c0a8  .X.[..@.........
	0x0020:  0268 0303 82f0 0000 0000 4500 003c 5d90  .h........E..<].
	0x0030:  4000 4011 5767 c0a8 0268 c0a8 0201 e4ce  @[email protected]......
	0x0040:  0035 0028 7c0a 8322 0100 0001 0000 0000  .5.(|.."........
	0x0050:  0000 036e 7470 0675 6275 6e74 7503 636f  ...ntp.ubuntu.co
	0x0060:  6d00 0001 0001                           m.....

Stop tcpdump running on workers and the initiating SSH session on the scanner machine.
These have obviously different PID's so the following is needed:

• pkill on the scanner host
• pkill on the target host

Must be seen in accordance with #14, #13 and #12.

Should consider whether this should be in a Jupyter notebook with matplotlib graphs or a script.

Following must be fixed:

• Count distinct number of destinations a host communicates to (TCP/UDP/ICMP) - to filter out relevant data
• Port combination for TCP and UDP (which ports are scanned on a host?)
• Scan ordering
  • List of ports (as a timeline) to compare to another scans of the same type (incoming ports)
  • List of ports and with a time difference between the scans

tcpdump aren't exiting automaticly when done, though the scan does.
Create a script that crawls through the tasklist and updates status to the tasklist and shuts down tcpdump.

Create a function for changing tasks in tasklist from ongoing to completed.

Two tasks could in some cases end up having the same name.
This could happen when iteration of tasks are in progress (e.g. the tasklist is read into the script and not done processing) and the status of tasks using cleanup.sh are updated for the file using sed.

A object oriented parser is started on, though primarily focusing on #4 for now. This needs to be done later on.

When a scan is done the process stops, though the tcpdump does not. This must be controlled through a script, and status must be updated in tasklist.

Output such as the following are not wanted:

tcpdump: listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes

Some tasks are hanging when monitoring active TCP dumps compared to Nmap scans.
Example for monitoring ongoing Nmap tasks:

while true;
do
  ps -eo command | grep nmap | grep -v grep
  sleep 2; echo ""; echo ""
done

One use case:
If the task list is manipulated through an editor, the cleanup script changes the file, and the user changes are overwriting what the script is changing.
Second use case:
Tasks are hanging without manipulating the tasklist.
The tasks are enlisted as completed in the task list.
Examples:

ssh bsc17-mng tcpdump -U -i ens33 -w nmap_ping_scan_normal_83_202201161821.pcap 2>&1
ssh bsc18-mng tcpdump -U -i ens33 -w nmap_ping_scan_normal_84_202201161821.pcap 2>&1
ssh bsc19-mng tcpdump -U -i ens33 -w nmap_ping_scan_normal_85_202201161821.pcap 2>&1
ssh bsc20-mng tcpdump -U -i ens33 -w nmap_ping_scan_normal_86_202201161822.pcap 2>&1

It should look something like this (with both the ssh tcpdump process and the nmap process):

ssh bsc07-mng tcpdump -U -i ens33 -w nmap_xmas_scan_paranoid_202201152048.pcap 2>&1
nmap -oX ./results/nmap_xmas_scan_paranoid_202201152048.xml bsc07 -T0 -sX --system-dns

This error makes the worker busy for some time until the task is finally killed.