SiLK free download, SiLK download on software download

SiLK is the System for Internet-Level Knowledge. SiLK is the System for Internet-Level Knowledge. It is a collection of traffic analysis tools developed by the CERT Network Situational Awareness Team (CERT NetSA) to facilitate security analysis of large networks. The SiLK tool suite supports the efficient collection, storage and analysis of network flow data, enabling network security analysts to rapidly query large historical traffic data sets. SiLK is ideally suited for analyzing traffic on the backbone or border of a large, distributed enterprise or mid-sized ISP.SiLK consists of two sets of tools: a packing system and analysis suite. The packing system receives Netflow V5 PDU's and converts them into a more space efficient format, recording the packed records into service-specific binary flat files. The analysis suite consists of tools which can read these flat files and then perform various query operations, ranging from per-record filtering to statistical analysis of groups of records. The analysis tools interoperate using pipes, allowing a user to develop a relatively sophisticated query from a simple beginning.The vast majority of the current code-base is implemented in C, Perl, or Python. This code has been tested on Linux, Solaris, OpenBSD, and Mac OS X, but should be usable with little or no change on other Unix platforms.The SiLK software components are released under the GPL.What's New in This Release:· rwfilter can support filtering using expressions written in Python, and it is possible to manipulate SiLK Flow records from within Python. This feature requires Python 2.4 or later, and you must specify --with-python when you run configure. See the "PySiLK: SiLK in Python" language reference documentation, and the --python-expr and --python-file switches on rwfilter.· Preliminary support for IPv6 addresses can be included. Use the --enable-ipv6 switch on the configure script to include IPv6 support in SiLK. When IPv6 is present, rwfilter provides a --ip-version switch to filter on IPv4 and/or IPv6 addresses, and the tools rwuniq and rwcut provide a --ipv6-policy switch (and SILK_IPV6_POLICY environment variable) that controls the display of IPv6 addresses.· rwfilter now supports threads. Performance is greatly improved for queries that look at many files but return few records. Use the --threads switch on rwfilter or the SILK_RWFILTER_THREADS environment variable to control the number of threads. By default, rwfilter will use a single thread. Our testing has found that performance peaks around four threads per CPU, but performance will vary depending on the type of query and the number of records returned.· There are new binary SiLK file formats, and the format of every SiLK file has changed. SiLK-1.0.0 can read files created by earlier versions of SiLK; however, releases prior to SiLK-1.0.0 will not be able to read SiLK-1.0.0 files. Binary SiLK files now contain additional information in their headers, including the version of SiLK that produced the file.· Delimited textual output has changed in almost all tools. Note this is a POTENTIAL INCOMPATIBILITY and may break scripts. A new --no-final-delimiter switch prevents printing of the final delimiter in the textual output of rwaddrcount, rwbagcat, rwcount, rwcut, rwpmapcat, rwsetcat, rwstats, rwtotal, rwuniq. In addition, the --delimited switch now enables --no-final-delimiter, making it easier for the output to be parsed by other tools. If you need to maintain compatibility with earlier versions of SiLK, replace --delimited=X with --no-columns --column-sep=X.· Arbitrary notes (annotations) can be added to the headers of some SiLK files. Use the --note-add=TEXT to add a note, or --note-file-add=FILE to add text from a file. The rwfileinfo tool will view the notes. Notes are supported by rwbag, rwbagbuild, rwbagtool, rwcat, rwfilter, rwset, rwsetbuild, rwsettool.· Site information is completely determined at run-time. The rules that rwflowpack uses to categorize flows are now controlled by a run-time plug-in that rwflowpack loads. The name of the plug-in must be passed to rwflowpack via the --packing-logic switch, or set in the silk.conf file.· The sensor.conf file used by rwflowpack and flowcap has a completely different syntax. See the Installation Handbook and the rwflowpack(8) and sensor.conf(5) manual pages. The update-sensor-conf script converts the old syntax to the new.· A new rwidsquery tool is provided. rwidsquery takes a Snort alert log or rule file and invokes rwfilter with the appropriate arguments to find the SiLK flow records that match the input file.· Bugs have been fixed in processing times on Solaris when the machine's timezone was not UTC.· Configuring SiLK to use legacy timestamps by default is no longer supported. The --legacy-timestamps switch is still supported on the applications.· When looking for support files (such as country_codes.pmap), tools will look in $SILK_PATH/share/silk/ and $SILK_PATH/share/, but they no longer look in $SILK_PATH/.· buildset, readset, setintersect, rwset-union:- These symbolic links to rwsetbuild, rwsetcat, rwsetintersect, and rwsetunion are no longer created.· rwaddrcount:- See discussion of --no-final-delimiter above· rwbag:- See discussion of --note-add above· rwbagbuild:- The --output switch has been renamed to --output-path.- See discussion of --note-add above· rwbagcat:- The --output switch has been renamed to --output-path.- See discussion of --no-final-delimiter above· rwbagtool:- See discussion of --note-add above- The --output-file switch is deprecated. Use --output-path instead.· rwcat:- See discussion of --note-add above· rwcount:- Enhancement to support millisecond-sized bins. Specify a fractional value to the --bin-size switch: --bin-size=0.500- As a side effect of this millisecond capability, the output from the default load scheme (--load-scheme=4, splitting a flow by its active time) will now divide flows across each millisecond that the flow is active. This results in slightly different output.- New --end-epoch switch allows user to control the final bin to print.- The --delimiter switch has been removed. Use the --column-separator switch instead. (Note that the --delimited switch still exists).- See also discussion of --no-final-delimiter above· rwcut:- A new --all-fields switch causes all possible fields to be printed.- New --ipv6-policy switch controls how IPv6 flows are handled- See also discussion of --no-final-delimiter above· rwdedupe:- The --identical-fields switch has been renamed to --ignore-fields, and --sort-buffer-size has been renamed to --buffer-size.· rwfileinfo:- Output has changed to reflect new SiLK binary file headers.· rwfilter:- New --python-expr and --python-file switches- New --threads switch- See discussion of --note-add above- New tuple.so plug-in filters flow records based on any subset of the five-tuple {source-ip, destination-ip, source-port, destination-port, protocol}. The --ipport-any and --ippair-any switches are deprecated.- The --ippair-any and --ipport-any switches no longer work for files that use only TAB characters between the two columns of input. Change the TAB characters to spaces.- New --ip-version switch when IPv6 support is enabled.- Fix an issue where an error writing to the file system was not being correctly reported.- Fix a bug that caused the --site-config-file switch to be ignored· rwmatch:- New --unmatched switch allows unmatched records to be written to the output.- New --symmetric-delta switch allows either input file to contain the initiating flow· rwpmapbuild:- See discussion of --note-add above- rwpmapbuild has been rewritten as a C application.· rwpmapcat:- See discussion of --no-final-delimiter above· rwnetmask changes:- Enhancement so that it takes file names from the command line and produces a file as output.- Renamed switches to be more consistent with other tools but leave the old names for compatibility.· rwscan:- Existing output files are no longer overwritten.- Printing of each filename processed, thread creation, etc. is now only done when the user specifies --verbose-progress on the command line.- New --verbose-results prints information about each IP.- New switches allow setting the parameters used by the TWR algorithm- New --integer-ips switch to print IPs as integers- In the printed output, headers and output records now end with a delimiter by default. This can be turned off with --no-final-delimiter.- The --scandb switch enables --no-final-delimiter.- The --output-file switch has been renamed to --output-path.- Improved manual page.· rwset: POTENTIAL INCOMPATIBILITY.- Running rwset with no arguments will no longer produce an IPset. The IPset(s) to create MUST now be specified with the --sip, --dip, and/or --nhip switches.- See discussion of --note-add above· rwsetbuild:- See discussion of --note-add above· rwsetcat:- See discussion of --no-final-delimiter above· rwsettool:- See discussion of --note-add above· rwstats:- See discussion of --no-final-delimiter above· rwtotal:- See discussion of --no-final-delimiter above· rwuniq: POTENTIAL INCOMPATIBILITY.- The --threshold switch is no longer supported. Use the --flows switch instead.- The output from rwuniq may appear in a different order due from previous releases due to changes in the internal hash table.- The --sip-distinct and --dip-distinct switches are handled more efficiently for sparse IPs.- New --ipv6-policy switch controls how IPv6 flows are handled- See discussion of --no-final-delimiter above· Summary of changes that may break old scripts or usage patterns:- See the discussion of --no-final-delimiter above- rwbagbuild: The --output switch has been renamed to --output-path. Since --output is a legal abbreviation of --output-path, no end-user effects should be seen.- rwbagtool: The --output switch has been renamed to --output-path. Since --output is a legal abbreviation of --output-path, no end-user effects should be seen.- rwcount: The --delimiter switch has been removed. Use the --column-separator switch instead. (Note that the --delimited switch still exists).- rwdedupe: The --identical-fields switch has been renamed to --ignore-fields, and --sort-buffer-size has been renamed to --buffer-size.- rwtotal: The --delimiter switch has been removed. Use the --column-separator switch instead. (Note that the --delimited switch still exists).- rwuniq: The --threshold switch is no longer supported. Use the --flows switch instead.· For programmers:- The IP address is now an abstract object.- All access to the fields of an rwRec should occur through the rwRec* wrappers.- Time is now represented as an sktime_t (a signed 64bit integer), representing milliseconds since the UNIX epoch.- There have been many changes to the library functions.· The following incompatible changes exist in the packing tools:- The sensor.conf syntax is completely different.- rwflowpack: When processing PDU-files as input, you need to use --input-mode=pdufile instead of --input-mode=file.- rwflowpack: The --fc-address and --fc-port switches have been removed; use --flowcap-address and --flowcap-port instead.- flowcap: The --sensors switch has been removed. The --probes switch offers similar functionality, but takes the names of probes, not sensors.

SiLK Related Software