Using Git with Nmap

Nmap source code is under version control using the Subversion (SVN) version control system (VCS). SVN is a centralized client-server VCS, similar to CVS. Git is a Distributed VCS, like Mercurial. There are many excellent sources of information about Git. This page will help you use Git locally and interact with the official SVN repository remotely.

Obtain git-svn

Git-svn is a plugin for Git that allows it to track changes in a SVN repository. It can be obtained on Ubuntu and Debian OSs by running sudo apt-get install git-svn.

Git-svn workflows

Create an initial clone

Do not do this step! This is an example, but will take a long time and probably generate errors. A more complete, quicker, and less-error-prone method is outlined below in #Clone including nmap-exp branches.

 git svn clone https://svn.nmap.org/nmap nmap-git

Set up ignores

 git svn show-ignore >> .git/info/exclude

Get the latest source from SVN

 git checkout master
 git svn rebase

Make local changes

 git checkout -b topic-branch
 <make local changes>
 git commit -a

Commit to SVN

 git checkout master
 git svn rebase
 git checkout topic-branch
 git merge master
 <resolve merge conflicts>
 git checkout master
 git merge --squash topic-branch
 git diff HEAD #Check that your changes make sense
 git commit #Make one commit with one message
 git svn dcommit

Working with Github

This section pertains specifically to Github, but works for other git remotes.

1. Create a new repository on Github.
2. Create a remote for github with the "git remote add" command Github suggests,

except name it something besides "origin". Git-svn will have its own remote that you don't access with standard git commands, so when you want to push to Github do "git push github".

1. If you want to track your branches on github, do "git push github branch-name".

Clone including nmap-exp branches

Here is how to create a clone that includes branch history from nmap-exp branches. You can easily checkout an nmap-exp branch, make changes, and dcommit to them. The idea is from http://stackoverflow.com/a/573619.

One difficulty is that some nmap-exp subdirectories are themselves branches (level 1), while other subdirectories are named after a user with branches one level deeper (level 2). I tried to classify each subdirectory into one of these categories. If you don't do this, you get loads of bogus "branches" like nmap-exp/bhdc08/libdnet-stripped. Some subdirectories were used in both ways at different times in their history; those I have put in level 2 so that merges from their proper subdirectories are visible, even though it leaves you with some old bogus branches.

mkdir nmap-git
cd nmap-git
LEVEL1='{Sriharsha,bhdc08,calderon,colin-packet-fuzz,claudiu,diman,doug,ejlbell,fyodor-perf,fyodor,josh,kroosec,nsock-proxy,patrick_temp,philip,rpc-grind,scriptsuggest,soc07,stable-5.2,stable-5.5,yang}'
LEVEL2='{aca,brandon,claude,colin,d33tah,daniel,david,dev,devin,djalal,dmiller,drazen,henri,ioerror,ithilgore,jah,james,jay,joao,jurand,kirubakaran,kris,luis,majek04,michael,ncat,patrick,patrik,peter,rainmap,ron,sean,shinnok,sophron,sven,venkat,vladimir,weilin}'
git svn init https://svn.nmap.org/ --trunk=nmap --branches="trunk/*" --branches="{nbase,ncat,nping,nsock,umit,zenmap}" --branches="nmap-exp/$LEVEL1" --branches="nmap-exp/$LEVEL2/*"

Edit the file .git/config to add prefixes to the remote names. Change this:

[svn-remote "svn"]
        url = https://svn.nmap.org
        fetch = nmap:refs/remotes/trunk
        branches = trunk/*:refs/remotes/*
        branches = {nbase,ncat,nping,nsock,umit,zenmap}/*:refs/remotes/*
        branches = nmap-exp/{Sriharsha,bhdc08,calderon,colin-packet-fuzz,claudiu,diman,doug,ejlbell,fyodor-perf,fyodor,josh,kroosec,nsock-proxy,patrick_temp,philip,rpc-grind,scriptsuggest,soc07,stable-5.2,stable-5.5,yang}/*:refs/remotes/*
        branches = nmap-exp/{aca,brandon,claude,colin,d33tah,daniel,david,dev,devin,djalal,dmiller,drazen,henri,ioerror,ithilgore,jah,james,jay,joao,jurand,kirubakaran,kris,luis,majek04,michael,ncat,patrick,patrik,peter,rainmap,ron,sean,shinnok,sophron,sven,venkat,vladimir,weilin}/*:refs/remotes/*

to this:

[svn-remote "svn"]
        url = https://svn.nmap.org
        fetch = nmap:refs/remotes/svn/nmap
        branches = trunk/*:refs/remotes/svn/trunk/*
        branches = {nbase,ncat,nping,nsock,umit,zenmap}:refs/remotes/svn/*
        branches = nmap-exp/{Sriharsha,bhdc08,calderon,colin-packet-fuzz,claudiu,diman,doug,ejlbell,fyodor-perf,fyodor,josh,kroosec,nsock-proxy,patrick_temp,philip,rpc-grind,scriptsuggest,soc07,stable-5.2,stable-5.5,yang}:refs/remotes/nmap-exp/*
        branches = nmap-exp/{aca,brandon,claude,colin,d33tah,daniel,david,dev,devin,djalal,dmiller,drazen,henri,ioerror,ithilgore,jah,james,jay,joao,jurand,kirubakaran,kris,luis,majek04,michael,ncat,patrick,patrik,peter,rainmap,ron,sean,shinnok,sophron,sven,venkat,vladimir,weilin}/*:refs/remotes/nmap-exp/*/*

Now fetch the repository. This will take forever.

git svn fetch

If you don't care about having such deep history, you can fetch only newer revisions. r27104 is when nbase, ncat, nping, nsock, and zenmap were moved from externals into the nmap tree.

git svn fetch -r 27104:HEAD

Set up your ignores.

git svn show-ignore >> .git/info/exclude

You can now view all the possible remote branches and check one of them out:

git branch -r
  nmap-exp/aca/misc
  nmap-exp/aca/nmap
  nmap-exp/bhdc08
  nmap-exp/brandon/nmap-service-perf
  nmap-exp/calderon
  [...]
git branch
* master
git show-branch nmap-exp/david/xml-output master
git checkout -b xml-output nmap-exp/david/xml-output
Switched to a new branch 'xml-output'
git branch
  master
* xml-output
git show-branch HEAD master
gitk HEAD master

The main benefit of doing this is being able to easily diff a branch against master. This shows the changes in the current branch relative to the common ancestor of HEAD and master (http://stackoverflow.com/a/4944548):

git diff master...

Or if you're on master and want to see what's happening in a branch:

git diff ...nmap-exp/david/xml-output

Doing a dcommit does it in the current SVN remote.

git svn dcommit
Committing to https://svn.nmap.org/nmap-exp/david/xml-output ...

How to make an nmap-exp branch:

git svn branch --destination nmap-exp david/foo
Copying https://svn.nmap.org/nmap at rXXXXX to https://svn.nmap.org/nmap-exp/david/foo...

Noticing new remote SVN branches

If you find yourself editing .git/config to add new SVN remote branches, do this to fetch the branches locally. Edit .git/svn/.metadata and remove the line

        branches-maxRev = XXXXX

Or if you know the revision at which the branch you want to notice was created, then set XXXXX to one revision lower to save time. Then run

$ git svn fetch