4.3.2 Generic Enumeration filter description

Flexible enumeration can be achieved by giving an Enumeration=filter modifier to a RootNode URL. The filter is a program which takes URLs on standard input, and writes new RootNode URLs on standard output.

The output format is different than specifying a RootNode URL in a Gatherer configuration file. Each output line must have nine (9) fields separated by spaces. These fields are:

        URL
        URL-Max
        URL-Filter-filename
        Host-Max
        Host-Filter-filename
        Access
        Delay
        Depth
        Enumeration-Program

These are the same fields as described in section 4.3. Values must be given for each field. Use /dev/null to disable the URL-Filter-filename and Host-Filter-filename. Use /bin/false to disable the Enumeration-Program.