GSoC community ideas/Performance/Optimization specialist

• Optimize Nmap performance and resource usage
  • Particularly focus on measuring existing --min-rate behavior compared to Zmap and masscan. The common criticism of Nmap being slow usually ignores this scan mode.
  • Consider measuring for improvement of hostgroup sizes. Larger hostgroup sizes could improve parallelization for large scans, but has memory cost.
• A fully stateless ping scanner, à la zmap|masscan could be added for very large scale scans.
• Maybe do some of our large-scale scanning research too, like keeping top ports stats up to date.
• Analyze data dump from Carna botnet for Nmap improvements.

Some items from the todo list:

• We should figure out why (at least with Nping) raw ethernet frame sends seem to be taking significantly longer than raw socket sends (e.g. using --send-ip or the OS-provided ping utility). This has been reproduced on Linux and Windows. Here's a thread: http://seclists.org/nmap-dev/2012/q4/424
  • Note that David and I tried to reproduce this on his machine and on 'web' and 'research' machines and could not reproduce. Still happens with Fyodor's machine connected with WiFi. Fyodor should test on the same machine using wired and see if that changes anything.
• Make version detection and NSE timing system more dynamic so that the concurrency can change based on network conditions/ability. After all, beefy systems on fast connections should be able to handle far more parallel connections than slower systems. At a minimum, this at least warrants more benchmark testing.
• [NSE] Do some benchmarking of our brute.nse. We should check the performance with different levels of thread parallelism. Our initial results show that it isn't helping much for vnc-brute or for drda-brute (which is currently using the multi-thread feature directly rather than through brute.nse library). We should figure out why the threads aren't helping more, and whether there is something we can do to fix it. It would also be interesting to compare speed with Ncrack for services we have in common.
• Consider implementing both global and per-host congestion control in the IPv6 OS detection engine. Currently it handles congestion globally (one CWND and SSTHRESH shared by all hosts). This works fine but it may not be the most efficient approach: if the congestion is not in our network segment but in a target's and we are os-scanning hosts in different networks, then all hosts get "penalized" because there is congestion in another network, not in theirs.

• Look into whether we should loosen/change the global congestion control system to address possible cases of one target host with many dropped packets slowing down the whole group. See http://seclists.org/nmap-dev/2008/q1/0096.html .
  • Related possibility: Fix --nogcc to gracefully handle ping scans. Right now it seems to go WAY TOO FAST (e.g. several thousand packets per second on my DSL line).
  • [12/22/09] David says: It still is in one case that I've documented on my wiki. I had an idea to fix it, but on testing it it didn't work. The idea was to treat the global congestion limit differently. Instead of dropping it down to the minimum level on a drop as is done currently, I thought about only dropping it by the amount that the individual host limit drops. For example, if a host had a drop and its limit fell from 25 to 1, then the global limit would change (if it was at 100 to begin with) to 76, not all the way down to 2 or whatever it is. The idea being that the global limit is most important at the beginning of a scan, when there's no information to set host limits, and every host wants to send all its first probes at once. See http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive2#global-cc. I am convinced, though, that some sort of global control is necessary. There's a reason that a web browser limits the number of connections it will make, and doesn't try to download every image file at once and count on the fairness of TCP to sort it out.