Nsock engines

This page presents the nsock-engines subsystem. Nsock is one of the core libraries of nmap, designed to perform high performance parallel socket operations for massively client applications. Information contained there is an updated version of the documentation initially written for the experimental nsock-engines branch.

A quick look at nsock

Here are some keypoints about nsock, from both external (exported API) and internal (internal data structures and algorithms) standpoints.

Exported API

In order to use nsock, one first needs to create a ``nsock_pool``. Once the pool is created, the program can associate events to it:

• Request new TCP connection
• Create timer
• Wait for data to read
• ...

These events are non-blocking ones, each of them is registered with a callback function (the event handler). Once this is done, the user call the main loop. Nsock will look for events and call the associated callback of the events as soon as they occur.

Internals

From within the library, a pool is a structure that mainly references two things:

• The lists of events, ordered by type
  • Connection requests
  • Read events
• Write events
• Timers
• The list of IODs

An IOD is a nsock /super-socket/, basically every IOD has a corresponding file descriptor and might have several read/write events associated.

Each time the user creates a new event, a new item is added to the corresponding events list and an IOD is eventually created. Each event (but timers) in the lists has a pointer to the IOD it's associated with.

During the main event loop nsock will poll the active IODs for activity, then go through all the events, and call the handlers accordingly.

Context

The older versions of nsock used to rely upon the select(2) function to perform non-blocking I/O management. This function is highly portable but has several drawbacks:

• O(n) complexity
• Limited performances
• Inability to deal with more than FD_SETSIZE file descriptors (usually 1024)

Fortunately modern operating systems offer much better I/O management interfaces in term of performance. These are of course not portable and don't have the same API than select(2) (that would have killed all the fun!). The aim of the nsock-engines branch was therefore to both leverage these efficient functions when available *and* keep the library portable and efficient.

Fundamental design

The core idea behind nsock-engines is the notion of /engine/ (no surprise). I call /engine/ a set of functions, exporting a clearly defined interface to the rest of the library. Each engine leverages a specific IO management interface and is only compiled if compatible with the system. In case several engines are available, the one known to be the most efficient is chosen (though it's also possible to explicitly ask the library to instantiate a given engine).

In other words, all the engines which could be used on a system are actually compiled, and selection is made at runtime. This is not expensive as engines are sorted by preference (see nsock_engines.c): we just pick the first one of the list. With this design it would be possible to select the engine to use at runtime.

Here are the elements that an engine needs to export (as defined in nsock_internal.h):

 struct io_engine {
   /* human readable identifier for this engine. */
   const char *name;
 
   /* engine constructor */
   int (*init)(mspool *nsp);
 
   /* engine destructor */
   void (*destroy)(mspool *nsp);
 
   /* register a new IOD to the engine */
   int (*iod_register)(mspool *nsp, msiod *iod, int ev);
 
   /* remove a registered IOD */
   int (*iod_unregister)(mspool *nsp, msiod *iod);
 
   /* modify events for a registered IOD.
    *  - ev_set represent the events to add (if not set)
    *  - ev_clr represent the events to delete (if set) */
   int (*iod_modify)(mspool *nsp, msiod *iod, int ev_set, int ev_clr);
 
   /* main engine loop */
   int (*loop)(mspool *nsp, int msec_timeout);
 };

Nsock currently comes with four engines. The first one is of course based on select() and is the fallback engine, used in case nothing else is available. The select(2)-based engine is basically a refactoring of the old code under the form of an engine. Note that ncat always use this engine, as it offers the best compatibility with non-socket file descriptors.

The second one is the epoll(7)-based engine. Epoll is a Linux-specific I/O event notification facility aiming to deliver internal O(1) complexity, high performance and ability to poll over an unlimited number of sockets.

Then comes the kqueue(2)-based engine. kqueue(2) is available on numerous flavors of BSD (including MacOS). Like epoll, it's extremely efficient.

A poll(2)-based engine was also added. Poll(2) isn't as efficient as epoll and kqueue but is very portable (even MS windows has its version of poll) and doesn't suffer any limitation in term of maximum number of FDs.

The last three ones bring the ability to deal with very large sets of sockets.

Usage

You can see which engines are available on your system by typing the following command:

 $ nmap -V  
 [...]
 Available nsock engines: epoll poll select

Engines are sorted by efficiency, the first one in the list (here epoll) will be used by default. You can enforce the selection of another engine by passing the --nsock-engine=<name> option to nmap, ncat and nping.

e.g.:

 nmap --nsock-engine=select -sCV scanme.nmap.org