Home

nProbe User`s Guide

1. My Plugin Name shortName NULL version Plugin string description author email 0 always enabled 1 enabled PLUGIN_DONT_NEED_LICENSE myPlugin_init NULL Term myPlugin_conf myPlugin_delete 1 call packetFlowFctn for each packet NULL myPlugin_packet myPlugin_get_template myPlugin_export myPlugin_print NULL NULL myPlugin_help NULL 0 O and a function with the following format ifdef MAKE_STATIC_PLUGINS PluginEntryPoint myPluginEntryFctn void else PluginEntryPoint PluginEntryFctnivoid endif return amp myPlugin The fields of the PluginEntryPoint function have the following meaning e char nprobe_revision String to be defined as NPROBE_REVISION e char name Extended plugin name e char short_name Short plugin name e char family Plugin family name if any or NULL to use the short plugin name e char version Plugin version e g 1 0 29 nProbe User s Guide v 6 16 e char descr Plugin description in plain English e char author Plugin author name and email e u_int8_t always_enabled Set it to 1 to enable the plugin permanently regardless of its use in the template T command line option e u_int8_t enabled Do not touch it and set it to O used by nProbe e u_int8_t need_license Set it to 1 if a license for this plugin is needed or 0 if is not needed e PluginInitFctn initFctn Plugin initialization function ca
2. IPFIX 35632 334 IPFIX 35632 335 IPFIX 35632 336 IPFIX 35632 337 IPFIX 35632 338 IPFIX 35632 339 IPFIX 35632 340 IPFIX 35632 341 IPFIX 35632 349 IPFIX 35632 350 Plugin BGP Update Listener temp NFv9 57762 IPFIX 35632 290 NFv9 57763 IPFIX 35632 291 NFv9 57764 IPFIX 35632 292 NFv9 57765 IPFIX 35632 293 NFv9 57766 IPFIX 35632 294 NFv9 57767 IPFIX 35632 295 NFv9 57768 IPFIX 35632 296 NFv9 57769 IPFIX 35632 297 NFv9 57770 IPFIX 35632 298 NFv9 57771 IPFIX 35632 299 NFv9 57772 IPFIX 35632 300 NFv9 57773 IPFIX 35632 301 NFv9 57774 IPFIX 35632 302 NFv9 57775 IPFIX 35632 303 NFv9 57776 IPFIX 35632 304 NFv9 57777 IPFIX 35632 305 NFv9 57778 IPFIX 35632 306 NFv9 57779 IPFIX 35632 307 NFv9 57780 IPFIX 35632 308 NFv9 57781 IPFIX 35632 309 P P P LL LL LL LS Plugin DHCP Protocol templates NFv9 57825 IPFIX 35632 353 l NFv9 57826 IPFIX 35632 354 NFv9 57827 IPFIX 35632 355 P Plugin DNS Protocol templates NFv9 57677 IPFIX 35632 205 NFv9 57678 IPFIX 35632 206 NFv9 57679 IPFIX 35632 207 NFv9 57680 IPFIX 35632 208 NFv9 57681 IPFIX 35632 209 NFv9 57824 IPFIX 35632 352 P P P P P P Plugin FTP Protocol templates NFv9 57828 IPFIX 35632 356 NFv9 57829 IPFIX 35632 357 NFv9 57830 IPFIX 35632 358 NFv9 57831 IPFIX 35632 359 RAGMENTS LIENT_NW_DELAY_SEC LIENT_NW_DELAY_USEC
3. NetFlow accounts IP traffic only not counting layer 2 headers Using this option the layer 2 headers are also accounted in flow traffic statistics dump metadata lt file gt Dump metadata information into the specified file and quit This option is useful when users want to know the type of each information element exported by nProbe so that for instance they can properly import into a database event log lt file gt Dump relevant activities e g nProbe start stop or packet drop onto the specified file enable throughput stats When P is used with this option is also possible to generate throughput information The file has the following format lt epoch gt lt bytes gt lt packets gt Each line is printed every second and it contains the number of bytes and packets observed within minute ndpi proto ports lt file gt Read the nDPI custom protocol and ports configuration from the specified file Please refer to the nDPI manual for further information about the format of this file disable I7 protocol guess When nDPI is unable to detect a protocol nProbe uses the port information to guess the protocol This flag prevents nProbe from doing that so protocols are detected only by nDPI without relying on default ports db engine lt database engine gt In case flows are dumped on a MySQL database see later on this manual the default database engine used by nProbe is MyISAM With this option you can use anoth
4. amp OxIfff 0 catches only unfragmented datagrams and frag zero of fragmented datagrams This check is implicitly applied to the tcp and udp index operations For instance tcp 0 always means the first byte of the TCP header and never means the first byte of an intervening fragment Primitives may be combined using Aparenthesized group of primitives and operators parentheses are special to the Shell and must be escaped e Negation or not e Concatenation amp amp or and Alternation II or or Negation has highest precedence Alternation and concatenation have equal precedence and associate left to right Note that explicit and tokens not juxtaposition are now required for concatenation If an identifier is given without a keyword the most recent keyword is assumed For example not host vs and ace is short for not host vs and host ace which should not be confused with not host vs or ace Expression arguments can be passed to nProbe as either a single argument or as multiple arguments whichever is more convenient Generally if the expression contains Shell metacharacters it is easier to pass it as a single quoted argument Multiple arguments are concatenated with spaces before being parsed 7 1 Examples To select all packets arriving at or departing from sundown 38 nProbe User s Guide v 6 16 nprobe f host sundown To select traffic between helios and either hot
5. new_bucket is set to 1 for the first packet of the flow or 0 for the following packets it calls all active plugins to tell that a new flow is active in cache The plugin will then decide if the packet can be handled by the plugin or not This is done by looking at the packet header fields passed to the function or inspecting the packet payload payload point whose length is specified by payloadLen If a plugin decides that the packet cannot be handled by the plugin for instance because the packet protocol is not managed by the plugin no action is needed and the function must simply return Instead if the plugin can handle the packet at the beginning of the function the following code like must be specified in order to add the plugin to the list of plugins it should usually be 1 or 0 element long handling this flow 32 nProbe User s Guide v 6 16 iffnew_bucket This bucket has been created recently info gt pluginPtr void amp myPlugin pluginData info gt pluginData struct my_plugin_info malloc sizeof struct my_plugin_info iflinfo gt pluginData NULL traceEvent TRACE_ERROR Not enough memory freelinfo return Not enough memory else struct my_plugin_info myinfo struct my_plugin_info pluginData Reset fields memset myinfo 0 sizeof struct my_plugin_info info gt next bkt gt ext gt plugin info gt plugin_used 0 bkt gt ext gt plugin info Once a plugin is
6. IPFIX 35632 122 FLOW_SERVER_NAME NFv9 57598 IPFIX 35632 126 PLUGIN_NAME NUM_PKTS_TTL_EQ_1 NUM_PKTS_TTL_2_5 NUM_PKTS_TTL_5_32 NUM_PKTS_TTL_32_64 NUM_PKTS_TTL_64_96 NUM_PKTS_TTL_96_128 NUM_PKTS_TTL_128_160 NUM_PKTS_TTL_160_192 NUM_PKTS_TTL_192_224 NUM_PKTS_TTL_224_255 IN_SRC_OSI_SAP OUT_DST_OST_SAP lates SRC_AS_PATH_1 SRC_AS_PATH_2 SRC_AS_PATH_3 SRC_AS_PATH_4 SRC_AS_PATH_5 SRC_AS_PATH_6 SRC_AS_PATH_7 SRC_AS_PATH_8 SRC_AS_PATH_9 SRC_AS_PATH_10 DST_AS_PATH_1 DST_AS_PATH_2 DST_AS_PATH_3 DST_AS_PATH_4 DST_AS_PATH_5 DST_AS_PATH_6 DST_AS_PATH_7 DST_AS_PATH_8 DST_AS_PATH_9 DST_AS_PATH_10 DHCP_CLIENT_MAC DHCP_CLIENT_IP DHCP_CLIENT_NAME DNS_QUERY DNS_QUERY_ID DNS_QUERY_TYPE DNS_RET_CODE DNS_NUM_ANSWERS DNS_TTL_ANSWER FTP_LOGIN FTP_PASSWORD FTP_COMMAND FTP_COMMAND_RET_CODE nProbe User s Guide v 6 16 Number of fragmented flow packets Network latency client lt gt nprobe Network latency client lt gt nprobe sec deprecated residual usec msec sec deprecated residual usec Network Network Network latency latency latency client lt gt nprobe nprobe lt gt server nprobe lt gt server Network latency nprobe lt gt server residual msec Application latency sec deprecated Application latency residual usec deprecated Application latency msec packets whose size lt 128 packets whose size gt 128 and lt 256 packets whose size
7. IPFIX 35632 123 CLIENT_NW_DELAY_MS ERVER_NW_DELAY_SEC ERVER_NW_DELAY_USEC IPFIX 35632 124 SSERVER_NW_DELAY_MS IPFIX 35632 86 APPL_LATENCY_SEC IPFIX 35632 87 APPL_LATENCY_USEC IPFIX 35632 125 APPL_LATENCY_MS UM_PKTS_UP_TO_128_BYTES UM_PKTS_128_T0_256_BYTES UM_PKTS_256_T0_512_BYTES UM_PKTS_512_T0_1024_BYTES UM_PKTS_1024_T0_1514_BYTES UM_PKTS_OVER_1514_BYTES UMULATIVE_ICMP_TYPE 101 SRC_IP_COUNTRY NFv9 57574 IPFIX 35632 102 SRC_IP_CITY NFv9 57575 IPFIX 35632 103 DST_IP_COUNTRY NFv9 57576 IPFIX 35632 104 DST_IP_CITY NFv9 57577 IPFIX 35632 105 SFLOW_PROTO_PORT unknown NFv9 57578 IPFIX 35632 106 SUPSTREAM_TUNNEL_ID unknown NFv9 57579 IPFIX 35632 107 LONGEST_FLOW_PKT NFv9 57580 IPFIX 35632 108 SHORTEST_FLOW_PKT NFv9 57581 IPFIX 35632 109 SRETRANSMITTED_IN_PKTS NFv9 57582 IPFIX 35632 110 RETRANSMITTED_OUT_PKTS NFv9 57583 IPFIX 35632 111 OO0ORDER_IN_PKTS NFv9 57584 IPFIX 35632 112 O0O0ORDER_OUT_PKTS NFv9 57585 IPFIX 35632 113 SUNTUNNELED_PROTOCOL NFv9 57586 IPFIX 35632 114 UNTUNNELED_IPV4_SRC_ADDR NFv9 57587 IPFIX 35632 115 UNTUNNELED_L4_SRC_PORT NFv9 57588 IPFIX 35632 116 UNTUNNELED_IPV4_DST_ADDR NFv9 57589 IPFIX 35632 117 UNTUNNELED_L4_DST_PORT NFv9 57590 IPFIX 35632 118 L7_PROTO NFv9 57591 IPFIX 35632 119 L7_PROTO_NAME NFv9 57592 IPFIX 35632 120 DOWNSTREAM_TUNNEL_ID unknown NFv9 57593 IPFIX 35632 121 SFLOW_USER_NAME NFv9 57594
8. Note that the z flag affects only the TCP protocol i e UDP ICMP and other protocols are not affected M maximum number of active flows It is used to limit the maximum number of concurrent flows that the probe can sustain This is useful for preventing the probe from creating as many flows as needed and hence to take over all the available resources E netflow engine Specify the netflow enginelype engineld into the generated flows m minimum number of flows per packet In order to minimize the number of emitted packets containing flows it is possible to specify the minimum number of flows that necessarily need to be contained in a packet This means that the packet is not emitted until the specified number of flows is reached q flow sender address This option is used to specify the address and port from which the packets containing flows are coming from Usually the operating systems prevents people from sending packets from addresses different from those assigned to the network interfaces nProbe User s Guide v 6 16 S sample rate lt packet rate gt lt flow rate gt nProbe uses all the captured packets for calculating flows In some situations e g strong traffic conditions it is necessary to reduce the number of packets that need to be handled by nProbe This option specifies the sampling rate i e the number of packets that are discarded between two packets used to produce flows You can also specify the flow sa
9. RADIUS Accounting Input Packets RADIUS Accounting Output Packets First flow RTP Sync Source ID First flow RTP timestamp Last flow RTP Sync Source ID Last flow RTP timestamp RTP jitter ms 1000 RTP jitter ms 1000 Packet lost in stream Packet lost in stream RTP payload type RTP payload type Max delta ms 100 between consecutive pkts Max delta ms 100 between consecutive pkts SIP call id corresponding to this RTP stream RTP MOS value 100 RTP R_FACTOR value 100 RTP Round Trip Time ms SIP call id SIP Call initiator SIP Called party SIP RTP codecs SIP SysUptime epoch of INVITE SIP SysUptime epoch of Trying SIP SysUptime epoch of RINGING SIP SysUptime epoch of INVITE OK SIP SysUptime epoch of INVITE FAILURE SIP SysUptime epoch of BYE SIP SysUptime epoch of BYE OK SIP SysUptime epoch of CANCEL SIP SysUptime epoch of CANCEL OK SIP RTP stream source IP SIP RTP stream source port SIP RTP stream dest IP SIP RTP stream dest por SIP failure response code SIP Cancel Bye Failure reason cause SIP C IP adresses SIP Call State Mail sender Mail recipient Whois DAS Domain name For instance if you want to specify NetFlow v9 flows in a format similar to v5 flows you can do as follows nprobe T IPV4_SRC_ADDR IPV4_DST_ADDR IPV4_NEXT_HOP INPUT_SNMP OUTPUT_SNMP IN_PKTS JoIN_BYTES FIRST_SWITCHED LAST_SWITCHED L4_SRC_PORT L4_DST_PORT TCP_FLAGS PROTOCOL SRC_TOS SRC
10. RADIUS_ACCT_STATUS_TYPE RADIUS Accounting Status Type RADIUS_ACCT_IN_OCTETS RADIUS Accounting Input Octets RADIUS_ACCT_OUT_OCTETS RADIUS Accounting Output Octets RADIUS_ACCT_IN_PKTS RADIUS Accounting Input Packets RADIUS_ACCT_OUT_PKTS RADIUS Accounting Output Packets The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order radius dump dir lt dump dir gt Directory where Radius logs will be dumped radius exec cmd lt cmd gt Command executed whenever a directory has been dumped Note that 3GPP radius extensions are supported by the plugin 4 12 RTP Plugin This plugin dissects RTP traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements RTP_FIRST_SSRC First flow RTP Sync Source ID RTP_FIRST_TS First flow RTP timestamp RTP_LAST_SSRC Last flow RTP Sync Source ID RTP_LAST_TS Last flow RTP timestamp RTP_IN_JITTER RTP Jitter ms 1000 RTP_OUT_JITTER RTP Jitter ms 1000 27 nProbe User s Guide v 6 16 RTP_IN_PKT_LOST Packet lost in stream RTP_OUT_PKT_LOST Packet lost in stream RTP_IN_PAYLOAD_TYPE RTP payload type RTP_OUT_PAYLOAD_TYPE RTP payload t
11. b 2 e PluginSetupFctn setupFctn Function called after plugin initialization when not set to NULL if according to the specified template this plugin will be used e PluginHelpFctn helpFctn Function that is called when nprobe h is executed and that is supposed to print plugin information e PluginidleTaskFctn idleFctn If not set to NULL this function will be periodically called by the nProbe core to execute if any housekeeping activities 30 nProbe User s Guide v 6 16 e u_int8_t v4Templateldx v Templateldx Used by nProbe Set them to 0 Each plugin must define a template with the following format static V9V10TemplateElementid myPlugin_template 0 BOTH_IPV4_IPV6 FLOW_TEMPLATE LONG_SNAPLEN NTOP_ENTERPRISE_ID 0 STATIC_FIELD_LEN O O 0 NULL NULL NULL what will be then used by the following functions static V9V10TemplateElementid myPlugin_get_template char template_name int i for i 0 myPlugin_templatelil templateElementid 0 i if strcmp template_name myPlugin_templateli netflowElementName return amp myPlugin_templatelil return NULL Unknown static V9V10TemplateElementid myPlugin_conf void return myPlugin_template In the file template h are specified the flow identifiers to be used in v9V10TemplateElementid that is defined as follows e U_ int8_tisInUse Always set it to 1 or 0 if it is the last template element to indicate that no furth
12. i e those whose application protocol has been detected and unknown i e the application protocol is unknown e 1 Export only know flows discarding unknown flows e 2 Export only unknown flows discarding known flows v print version This flag is used to print the nProbe version number and date C flow export lock This is a simple way to implement high availability Start two probes capturing the same data The master probe emit flows the slave probe is started with C lt path gt As long as lt path gt exists the slave works but no flow is emitted If the lt path gt file is deleted e g using an external program for controlling the master slave such as heartbeat the slave starts emitting flows If the file is restored the slave is silent again h print help Prints the nProbe help quick mode nProbe is computing many statistics but if you care just about basic netflow i e V5 or V9 IPFIX flows with standard fields you can use this flag to expedite operations telling nProbe to avoid doing many unnecessary things e g handle L2 traffic Use this option if you care about speed dont nest dump dirs nProbe User s Guide v 6 16 nProbe dumps data on disk e g with P using a nested directory In essence the base directory will be partitioned in sub directories with lt year gt lt month gt lt day gt lt hour gt lt min gt structure use this option is you want nProbe to dump all data in the base directory
13. 4094 ifIdx is set to IN E 1 OUT E For odd outer VLAN tags 0O where 0 3 5 7 4095 ifIdx are set to IN 0 1 OUT 0 Note that this option superseedes the in out iface idx options In case you enable L7 proto detection e g add L7_PROTO to the template this options enables you not to export flows for which nDPI has not been able to detect the proto Mode values VLAN tag double 0 Export known unknown flows default 1 Export only known flows discard flows with unknown protos 2 Export only unknown flows discard flows with known protos Prints the program version If the flow lock file is present no flows are emitted This facility is useful to implement high availability by means of a daemon that can create a lock file when this instance is in standby Prints this help Interpret received packets to see if they contain flows development only Enable debugging development only In case JSON label is used e g with ZMQ labels instead of numbers are used as keys Micro nprobe use if need speed and do not need advanced traffic analysis Fake packet capture development only Drop flows that have not processed by a plugin Dump files won t be saved on nested dirs Enable performance tracing debug only Log to syslog as lt probe name gt default stdout Flows hash size default 131072 IPv6 packets will not be accounted Delay in ms between two flow exports default 1 Send this many
14. NFv9 57820 IPFIX 35632 348 RTP_SIP_CALL_ID NFv9 57842 IPFIX 35632 370 RTP_MOS NFv9 57843 IPFIX 35632 371 RTP_R_FACTOR NFv9 57852 IPFIX 35632 380 RTP_RTT Plugin SIP Plugin templates NFv9 57602 IPFIX 35632 130 SIP_CALL_ID NFv9 57603 IPFIX 35632 131 SIP_CALLING_PARTY NFv9 57604 IPFIX 35632 132 SIP_CALLED_PARTY NFv9 57605 IPFIX 35632 133 SIP_RTP_CODECS NFv9 57606 IPFIX 35632 134 SIP_INVITE_TIME NFv9 57607 IPFIX 35632 135 SIP_TRYING_TIME NFv9 57608 IPFIX 35632 136 SIP_RINGING_TIME NFv9 57609 IPFIX 35632 137 SIP_INVITE_OK_TIME NFv9 57610 IPFIX 35632 138 SIP_INVITE_FAILURE_TIME NFv9 57611 IPFIX 35632 139 SIP_BYE_TIME NFv9 57612 IPFIX 35632 140 SIP_BYE_OK_TIME NFv9 57613 IPFIX 35632 141 SIP_CANCEL_TIME NFv9 57614 IPFIX 35632 142 SIP_CANCEL_OK_TIME NFv9 57615 IPFIX 35632 143 SIP_RTP_IPV4_SRC_ADDR NFv9 57616 IPFIX 35632 144 SIP_RTP_L4_SRC_PORT NFv9 57617 IPFIX 35632 145 SIP_RTP_IPV4_DST_ADDR NFv9 57618 IPFIX 35632 146 SIP_RTP_L4_DST_PORT NFv9 57619 IPFIX 35632 147 SIP_FAILURE_CODE NFv9 57620 IPFIX 35632 148 SIP_REASON_CAUSE NFv9 57834 IPFIX 35632 362 SIP_C_IP NFv9 57835 IPFIX 35632 363 SIP_CALL_STATE Plugin SMTP Protocol templates NFv9 57657 IPFIX 35632 185 SMTP_MAIL_FROM NFv9 57658 IPFIX 35632 186 SMTP_RCPT_TO Plugin Whois Protocol templates NFv9 57823 IPFIX 35632 351 WHOIS_DAS_ DOMAIN nProbe User s Guide v 6 16
15. Tunnelld Control GTPV1_END_USER_IP GTPv1 End User IP Address GTPV1_END_USER_IMSI GTPv1 End User IMSI GTPV1_END_USER_MSISDN GTPv1 End User MSISDN GTPV1_END_USER_IMEI GTPv1 End User IMEI GTPV1_APN_NAME GTPvl APN Name GTPVI_RAIL MCC GTPv1 RAI Mobile Country Code GTPV1_RAILMNC GTPv1 RAI Mobile Network Code GTPVI_RAI LAC GTPv1 RAI Location Area Code GTPV1_RAI RAC GTPv1 RAI Routing Area Code GTPV1_ULI_MCC GTPv1 ULI Mobile Country Code GTPV1_ULI_MNC GTPv1 ULI Mobile Network Code GTPVI_ULI_CELL_LAC GTPv1 ULI Cell Location Area Code GTPV1_ULI_CELL_Cl GTPv1 ULI Cell CI GTPV1_ULI_SAC GTPv1 ULI SAC GTPV1_RESPONSE_CAUSE GTPv1 Cause of Operation The plugin supports the following command line options that are used to specify where the optional GTP log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order gtpvl dump dir lt dump dir gt Directory where GTP logs will be dumped gtpvl exec cmd lt cmd gt Command executed whenever a directory has been dumped Please note that GTP U is not handled by this plugin but rather by the nProbe core when the tunnel option is used 4 5 GTPv2 Plugin This plugin dissects GTPv2 signaling information GTP C and saves it in dump files as well export the information via NetFlow IPFIX using the following inform
16. a list of pcap files If you use this flag the i option will be ignored Bi directional flows export policy 0 export all flows 1 export bi directional flows only 2 export mono directional flows only Specify the text files separator see P Default is pipe Do not drop privileges changing to user nobody Force flows to be bi directional This option is not supported by NetFlow V5 that by nature supports only mono directional flows NetFlow accounts IP traffic only not counting L2 headers Using this option the L2 headers are also accounted Dump flow metadata into the specified file and quit Useful for knowking the IE handled Dump incoming packets on the specified dump Maximum number of lines on a dump file Default 10000 Specified the timestamp format on dump files Value 0 Unix Epoch 1 Unix Epoch with microseconds 2 Human readable timestamp Comma separated list of nDPI protocols to enable If not specified all known protocols are detected When used with GTP traffic and redis the user traffic is accounted per IMSI NSAPI mobile traffic only Dump relevant activities into the specified log file Filter applied to collected filters only 3 Filter format lt asx network mask gt means discard flows matching filter Example as12345 192 168 0 0 24 10 0 0 0 8 Aggregate IMSI traffic GTP traffic only Simulate storage to disk debug only Deliver flows to subscribers connected t
17. distributing a proprietary product either appliance GUI front end or application based on nProbe Just sell distribute it without nProbe and point your customers to http www ntop org products nprobe in order to have access to nProbe you cannot include nProbe into a non GPL derivative work Note that the above is a clarification of what we mean for derivative work in the scope of the GPL licensed nProbe This interpretation applies only to nProbe and not to any other GPL products if you have questions about nProbe licensing please contact license ntop org See COPYING and EULA files for more details Note that the EULA applies only to nProbe derived work 11 Appendix E EULA NTOP END USER LICENSE AGREEMENT THIS END USER LICENSE AGREEMENT EULA IS A BINDING LEGAL AGREEMENT BY USING THE SOFTWARES PROVIDED TOGETHER WITH THIS EULA THE SOFTWARE OR USING ANY AUTHORIZATION CODE PROVIDED BY THE LICENSOR YOU ACKNOWLEDGE THAT YOU HAVE READ THIS EULA THAT YOU UNDERSTAND IT AND THAT YOU AGREE TO BE BOUND BY ITS TERMS IF YOU DO NOT AGREE WITH THE TERMS AND CONDITIONS OF THIS EULA DO NOT MAKE ANY USE OF THE SOFTWARE OR ANY USE OF AUTHORIZATION CODES YOUR USE OF THE SOFTWARE AND YOUR USE OF ANY AUTHORIZATION CODE IS SUBJECT AT ALL TIMES TO 45 nProbe User s Guide v 6 16 THE TERMS AND CONDITIONS HERE INCLUDING ALL PROVISIONS REGARDING THE LIMITATIONS OF THE LICENSOR LIABILITY 1 Grant of License in F
18. dump_as_ip_proto dump_as_ipv4_address dump_as_ipv6_address dump_as_mac_address dump_as_epoch dump_as_bool dump_as_tcp_flags dump_as_hex dump_as_ascii e const char netflowElementName String with the symbolic network element name used in NetFlow V 9 e const char ipfixElementName String with the symbolic network element name used in IPFIX V 10 e const char templateElementDescr String that describes the element information type used by nProbe when the help h is printed Most plugin callbacks are straightforward and its logic can be understood simply having a look at examples of existing plugins The only function worth to describe is the one that processes packets as it is the most complex one static void myPlugin_packet u_char new_bucket int packet_if_idx 1 unknown void pluginData FlowHashBucket bkt FlowDirection flow_direction u_intl6_t ip_offset u_short proto u_char isFragment u_short numPkts u_char tos u_short vlanid struct eth_header ehdr IpAddress src u_short sport IpAddress dst u_short dport u_int len u_int8_t flags u_int32_ttcpSeqNum u_int8_t icmpType u_short numMplsLabels u_char mplsLabels MAX_NUM_MPLS_LABELS MPLS_LABEL_LEN const struct pcap_pkthdr h const u_char p u_char payload int payloadLen i This function processes a packet belonging to a flow handled by this plugin nProbe has no clue what plugins are doing this whenever a new flow is created
19. gt 256 and lt 512 packets whose size gt 512 and lt 1024 packets whose size gt 1024 and lt 1514 packets whose size gt 1514 Cumulative OR of ICMP type packets Country where the src IP is located City where the src IP is located Country where the dst IP is located City where the dst IP is located L7 port that identifies the flow protocol or if Upstream tunnel identifier e g GTP TEID or if Longest packet bytes of the flow Shortest packet bytes of the flow Number of retransmitted TCP flow packets src gt dst Number of retransmitted TCP flow packets dst gt src Number of out of order TCP flow packets dst gt src Number of out of order TCP flow packets dst gt src Untunneled IP protocol byte Untunneled IPv4 source address Untunneled IPv4 source port Untunneled IPv4 destination address Untunneled IPv4 destination port Layer 7 protocol numeric Layer 7 protocol name Downstream tunnel identifier e g GTP TEID or if Flow username of the tunnel if known Flow server name if known Plugin name used by this flow if any 1 packets with TTL packets with TTL gt 1 and TTL lt 5 packets with TTL gt 5 and TTL lt 32 packets with TTL gt 32 and lt 64 packets with TTL gt 64 and lt 96 packets with TTL gt 96 and lt 128 packets with TTL gt 128 and lt 160 packets with TTL gt 160 and lt 192 packets with TTL gt 192 and lt 224 p
20. interface nets gt 2 lt number gt 3 lt port gt 4 5 lt port gt 6 9 lt path gt black list lt networks gt pcap file list lt filename gt N lt biflows export policy gt dont drop privileges collector n lt host port none gt Address of the NetFlow collector s Multiple collectors can be defined using multiple n flags In this case flows will be sent in round robin mode to all defined collectors if the a flag is used Note that you can specify both IPv4 and IPv6 addresses If you specify none as value no flow will be export in this case the P parameter is mandatory Note that you can specify the protocol used to send packets Example udp 192 168 0 1 2055 tcp 10 1 2 3 2055 interface i lt iface pcap gt Interface name from which packets are captured or pcap file debug only lifetime timeout t lt timeout gt It specifies the maximum seconds flow lifetime default 120 idle timeout d lt timeout gt It specifies the maximum seconds flow idle lifetime default 30 queue timeout 1 lt timeout gt It specifies how long expired flows queued before delivery are emitted default 30 snaplen s lt snaplen gt Packet capture snaplen default 128 bytes aggregation p lt aggregation gt It specifies the flow aggregation level lt VLAN Id gt lt proto gt lt IP gt lt port gt lt TOS gt lt AS gt where each element can be set to O ignore or 1 ta
21. interfaces from which packets are captured Example i eth0 eth1 t maximum flow lifetime Regardless of the flow duration a flow that has been active for more that the specified maximum lifetime is considered expired and it will be emitted Further packets belonging to the same flow will be accounted on a new flow d maximum flow idle lifetime A flow is over when the last packet received is older that the maximum flow idle lifetime This means that whenever applicable e g SNMP walk UDP flows will not be accounted on 1 packet 1 flow basis but on one global flow that accounts all the traffic This has a benefit on the total number of generated flows and on the overall collector performance maximum queue timeout It specifies the maximum amount of time that a flow can be queued waiting to be exported Use this option in order to try to pack several flows into fewer packets but at the same time have an upper bound timeout for queuing flows into the probe s snaplen This flag specifies the portion of the packet also called snaplen that will be captured by nProbe By default nprobe sets the snaplen automatically according to its configuration but you can override its value using thia flag nProbe User s Guide v 6 16 p flow aggregation f p Flows can be aggregated both at collector and probe side However probe allocation is much more effective as it reduces significantly the number of emitted flows hence th
22. license otherwise the probe will operate in demo mode nProbe User s Guide v 6 16 3 3 nProbe Command Line Options nProbe allows network administrators to precisely tune the flow generation policy In particular it is possible to specify a lot of command line options Below are listed the available options and a detailed explanation of each option nprobe h Welcome to nprobe v 6 16 140318 Revision 4095 for x86_64 apple darwinl13 1 0 Copyright 2002 14 ntop org SystemID 1FE719B8 0B82 5C67 9AE6 990B5030479F WARNING Invalid nProbe license etc nprobe license License mismatch error Usage nprobe n lt host port none gt i lt interface dump file gt t lt lifetime timeout gt d lt idle timeout gt l lt queue timeout gt s lt snaplen gt p lt aggregation gt f lt filter gt a b lt level gt G O lt threads gt P lt path gt F lt dump timeout gt D lt format gt u lt in dev idx gt Q lt out dev idx gt I lt probe name gt v w lt hash size gt e lt flow delay gt B lt packet count gt z lt min flow size gt M lt max num flows gt x lt payload policy gt E lt engine gt C lt flow lock file gt m lt min flows gt R lt cmd gt S lt sample rate gt A lt AS list gt g lt PID file gt T lt flow template gt U lt flow template id gt o lt v9 templ export policy gt L lt local nets gt c r 1 lt
23. lt filename gt where lt filename gt is a file path containing the networks specified using the above format count debug only Let the probe capture only up to the specified number of packets collector port specifies the NetFlow collector port It is now possible to use the nProbe as NetFlow proxy With collector port we can se the incoming NetFlow port on which flows are received instead of sniffing packets nProbe is able to convert flows from various versions For instance nprobe collector port 2055 i 192 168 0 1 2056 V 10 converts each flow received on port 2055 to IPFIX and sends them to 192 168 0 1 2056 tunnel Let the probe decode tunneled traffic e g GTP or GRE traffic and thus extract traffic information from such traffic rather than from the external envelope no promisc With this option nProbe does not use promiscuous mode to capture packets smart udp frags Ignore UDP fragmented packets with fragment offset greater than zero and compute the fragmented packet length on the initial fragment header This flag might lead to inaccuracy in measurement but it speeds us operations with fragmented traffic ipsec auth data len Length of the authentication data of IPSec in tunnel mode If not set IPSec will not be decoded but just accounted dump stats dump some flow statistics on file Periodically dump NetFlow statistics on the specified file Note that when using nProbe over PF_RING nProb
24. lt mode gt nprobe version v flow lock C lt flow lock gt help h interpret flow packets debug json labels quick mode fake capture drop flow no plugin dont nest dump dirs performance syslog I lt probe name gt hash size w lt hash size gt no ipv6 W flow delay e lt flow delay gt count delay B lt packet count gt min flow size z lt min flow size gt max num flows M lt max num flows gt netflow engine E lt type id gt lt min flows gt min num flows m sender address q lt host port gt sample rate S as list A lt AS list gt city list lt city list gt nProbe User s Guide v 6 16 set to the last two bytes of the MAC address of the flow sender Index of the output device used in the emitted flows outgoing traffic Default value is 0 Use 1 as value to dynamically set to the last two bytes of the MAC address of the flow receiver Use vlaniId 0 for untagged traffic as interface index Mode specifies with stacked VLANs which vlanId to choose Values are inner outer single or dual inner use the most inner VLAN tag outer use the first the one close to ether single for even outer VLAN tags E where E 2 4 6 4094 ifIdx is set to IN 0 OUT E For odd outer VLAN tags 0O where 0 3 5 7 4095 ifIdx is set to IN 0 1 OUT 0 for even outer VLAN tags E where E 2 4 6
25. the DECNET destination address is host 37 nProbe User s Guide v 6 16 decnet host host True if either the DECNET source or destination address is host ip arp rarp decnet Abbreviations for ether proto p where p is one of the above protocols lat moprc mopdl Abbreviations for ether proto p where p is one of the above protocols Note that ntop does not currently know how to parse these protocols tcp udp icmp Abbreviations for ip proto p where p is one of the above protocols expr relop expr True if the relation holds where relop is one of gt lt gt lt and expr is an arithmetic expression composed of integer constants expressed in standard C syntax the normal binary operators amp I a length operator and special packet data accessors To access data inside the packet use the following syntax proto expr size Proto is one of ether fddi ip arp rarp tcp udp or icmp and indicates the protocol layer for the index operation The byte offset relative to the indicated protocol layer is given by expr Size is optional and indicates the number of bytes in the field of interest it can be either one two or four and defaults to one The length operator indicated by the keyword len gives the length of the packet For example ether 0 amp 1 0 catches all multicast traffic The expression ip 0 amp Oxf 5 catches all IP packets with options The expression ip 6 2
26. used with either names or numbers for host ehost dst net net True if the IP destination address of the packet has a network number of net which may be either an address or a name src net net True if the IP source address of the packet has a network number of net net net True if either the IP source or destination address of the packet has a network number of net dst port port True if the packet is ip tcp or ip udp and has a destination port value of port The port can be a number or a name used in etc services If a name is used both the port number and protocol are checked If a number or ambiguous name is used only the port number is checked e g dst port 513 will print both tcp login traffic and udp who traffic and port domain will print both tcp domain and udp domain traffic src port port True if the packet has a source port value of port port port 36 nProbe User s Guide v 6 16 True if either the source or destination port of the packet is port Any of the above port expressions can be prepended with the keywords tcp or udp as in tcp src port port which matches only tcp packets less length True if the packet has a length less than or equal to length This is equivalent to len lt length greater length True if the packet has a length greater than or equal to length This is equivalent to len gt length ip proto protocol True if the packet is an ip packet of protocol type protocol Protoco
27. without creating this nested directory tree log to syslog lt probe name gt nProbe logs on stdout unless the g flag see above is used If the syslog needs to be used instead of a file this flag instruments nProbe to log on it using the specified name this is useful when multiple nProbe instances are active on the same host Please note that g is ignored if I is used and this option is not available on nProbe for Win32 w size of the hash that stores the flows The default size is 131072 and it should be enough for most of networks In case flows are not emitted often and with strong traffic conditions it would be necessary to increase the hash See later in this manual for knowing more about nProbe tuning W Discard IPv6 traffic Use this flag if you want nProbe not to account IPv6 traffic e flow export delay Some collectors cannot keep up with nProbe export speed This flag allows flows to be slow down by adding a short delay specified in ms between two consecutive exports The maximum allowed delay is 1000 ms B packet count delay It specified how many flow packets need to be sent before e is applied z minimum TCP flow size Peer to peer applications attacks or misconfigured applications often generate a lot of tiny TCP flows that can cause significant load on the collector side As most collector setups often discarded those flows it is possible to instrument nProbe via the z flag not to emit such flows
28. 257 NetFlow Version 5 NFv5 9 NFv9 10 IPFIX Specify how many flow pkts are exported between template exports default 10 Specify the list of local networks whose format is lt net gt lt mask gt if multiple use comma All the IPv4 hosts outside the local network lists will be set to 0 0 0 0 L must be specified before c This reduces the load on the probe instead of discarding flows on the collector side All the traffic going towards the local networks L must also be specified before r is assumed incoming traffic all the rest is assumed outgoing see also u and Q Specify the maximum flow size NOTE This parameter has influence on m Specify the binding between interfaceId and a network see below Capture a specified number of packets and quit debug only NetFlow IPFIX sFlow collector flows port Compute flows on tunneled traffic rather than on the external envelope Capture packets in non promiscuous mode Ignore UDP fragmented packets with fragment offset greater than zero and compute the fragmented packet length on the initial fragment header Length of the authentication data of IPSec in tunnel mode If not set IPSec will not be decoded Periodically dump traffic stats into the specified file All the IPv4 hosts inside the networks black list will be discarded This reduces the load on the probe instead of discarding flows on the collector side Specify a filename containing
29. 4 7 IMAP Plugin This plugin dissects IMAP traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information element IMAP_LOGIN Mail sender The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It 25 nProbe User s Guide v 6 16 is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order imap dump dir lt dump dir gt Directory where IMAP logs will be dumped imap exec cmd lt cmd gt Command executed whenever a directory has been dumped imap peek headers Dump both emails body and headers default body only 4 8 MySQL Plugin This plugin dissects MySQL unencrypted traffic information and saves the queries log in dump files as well export the information via NetFlow IPFIX using the following information elements MYSQL_SERVER_VERSION MySQL server version MYSQL_USERNAME MySQL username MYSQL_DB MySQL database in use MYSQL_QUERY MySQL Query MYSQL_RESPONSE MySQL server response MYSQL_APPL_LATENCY_USEC MySQL request gt response latecy usec The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It is po
30. 801 57802 57803 Plugin GTPv1 57692 57693 57694 57695 57696 57697 57698 57699 57700 57701 57702 57703 57704 57814 57815 57816 57817 57705 57706 57707 57804 Plugin GTPv2 57742 57743 57744 57745 57746 57747 57748 57749 57750 57751 57752 57753 57754 57805 57652 57832 57653 57654 57655 57656 57659 57660 57833 57732 Plugin MySQL 57667 57668 57669 57670 57671 57792 IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX 35632 35632 35632 35632 35632 35632 321 35632 323 35632 35632 322 326 324 325 35632 328 35632 330 331 327 329 GTPV _REQ_MSG_TYPE sGTPV _RSP_MSG_TYPE GTPVO_TID GTPVO_APN_NAME GTPVO_END_USER_IP GTPVO_END_USER_MSISDN GTPVO_RAI_MCC GTPVO_RAI_MNC GTPVO_RAI_CELL_LAC GTPVO_RAI_CELL_RAC GTPVO_RESPONSE_CAUSE Signaling Protocol templates IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 220 221 35632 35632 222 223 224 35632 226 35632 228 229 35632 225 227 230 231 232 342 35632 35632 343 344 345 35632 233 234 35632 332 235 GTPV1_REQ_MSG_TYPE GT
31. Box coverage Release 2 1 June 2003 Added nFlow support Release 2 0 1 February 2003 Added the ability to save flows on disk P flag Release 2 0 January 2003 Added the ability to select multiple NetFlow collectors Added p flag for ignoring TCP UDP ports Added e flag for slowing down flow export speed Added u flag for identifying input NetFlow devices into emitted flows Added z flag for preventing nProbe from emitting tiny flows Added a flag for selecting the way flows are exported to several collectors if defined Added the ability to control an LCD display where the probe can report traffic statistics Enhanced TCP flags support in exported flows Release 1 3 July 2002 First public release nProbe User s Guide v 6 16 3 Using nProbe The nProbe probe has to be activated on a PC from which it is possible to see capture all the traffic you are interested in For this reason in case of switched networks it is necessary to either mirror traffic VLAN or port mirror or place the probe on a location e g by the border gateway where most of the traffic flows When activated nProbe will collect traffic data and emit NetFlow v4 v5 v9 IPFIX flows towards the specified collector A set of packets with the same src ip amp port dst ip amp port protocol is called flow note that some protocols such as ICMP have no concept of ports Every flow even a very long standing ISO CD image down
32. C_AS bgpSourceAsNumber Source BGP AS 17 DST_AS bgpDestinationAsNumber Destination BGP AS 21 LAST_SWITCHED f LowEndSysUpTime SysUptime msec of the last flow pkt 22 FIRST_SWITCHED f LowStartSysUpTime SysUptime msec of the first flow pkt 23 0UT_BYTES post0ctetDeltaCount Outgoing flow bytes dst gt src 24 OUT_PKTS S SpostPacketDeltaCount Outgoing flow packets dst gt src 27 IPV6_SRC_ADDR sourcelPv6Address IPv6 source address 28 IPV6_DST_ADDR sdestinationIPv6Address IPv6 destination address 29 IPV6_SRC_MASK sourceIPv6PrefixLength IPv6 source mask 30 IPV6_DST_MASK sdestinationIPv6PrefixLengthIPv6 destination mask 32 ICMP_TYPE S SicmpTypeCodeIPv4 ICMP Type 256 ICMP code 34 SAMPLING_INTERVAL Sampling rate 35 SAMPLING_ALGORITHM Sampling type deterministic random 36 FLOW_ACTIVE_TIMEOUT f LowActiveTimeout Activity timeout of flow cache entries 37 FLOW_INACTIVE_TIMEOUT f LowIdleTimeout Inactivity timeout of flow cache entries 38 SENGINE_TYPE Flow switching engine 39 SENGINE_ID Id of the flow switching engine 40 TOTAL_BYTES_EXP exportedO0ctetTotalCount Total bytes exported 41 TOTAL_PKTS_EXP S exportedMessageTotalCount Total flow packets exported 42 TOTAL_FLOWS_EXP exportedF LowRecordTotalCount Total number of exported flows 52 MIN_TTL sminimumTTL Min flow TTL 53 MAX_TTL smaximumTTL Max flow TTL 56 IN_SRC_MAC sourceMacAddress Source MAC Address 58 SRC_VLAN sv laniId Source VLAN 59 DST_VLAN po
33. L SsHTTP_METHOD SsHTTP_RET_CODE SsHTTP_REFERER SsHTTP_UA SsHTTP_MIME SsHTTP_HOST SsHTTP_FBOOK_CHAT SsHTTP_SITE Plugin IMAP Protocol templates sIMAP_LOGIN MYSQL_SERVER_VERSION SMYSQL_USERNAME MYSQL_DB MYSQL_QUERY MYSQL_RESPONSE MYSQL_APPL_LATENCY_USEC Plugin Oracle Protocol templates NFv9 57672 IPFIX 35632 200 ORAC NFv9 57673 IPFIX 35632 201 ORAC NFv9 57674 IPFIX 35632 202 0RAC NFv9 57675 IPFIX 35632 203 0RAC NFv9 57676 IPFIX 35632 204 0RAC Plugin POP3 Protocol templates LE_USERNAME LE_QUERY LE_RSP_CODE LE_RSP_STRING LE_QUERY_DURATION NFv9 57682 IPFIX 35632 210 POP_USER Plugin Radius Protocol templates NFv9 57712 IPFIX 35632 240 RADIUS_REQ_MSG_TYPE NFv9 57713 IPFIX 35632 241 RADIUS_RSP_MSG_TYPE NFv9 57714 IPFIX 35632 242 RADIUS_USER_NAME NFv9 57715 IPFIX 35632 243 RADIUS_CALLING_STATION_ID NFv9 57716 IPFIX 35632 244 RADIUS_CALLED_STATION_ID NFv9 57717 IPFIX 35632 245 RADIUS_NAS_IP_ADDR NFv9 57718 IPFIX 35632 246 RADIUS_NAS_IDENTIFIER NFv9 57719 IPFIX 35632 247 RADIUS_USER_IMSI NFv9 57720 IPFIX 35632 248 RADIUS_USER_IMEI NFv9 57721 IPFIX 35632 249 RADIUS_FRAMED_IP_ADDR NFv9 57722 IPFIX 35632 250 RADIUS_ACCT_SESSION_ID NFv9 57723 IPFIX 35632 251 RADIUS_ACCT_STATUS_TYPE NFv9 57724 IPFIX 35632 252 RADIUS_ACCT_IN_OCTETS NFv9 57725 IPFIX 35632 253 RADIUS_ACCT_OUT_OCTETS GTPv GTPv GTPv GTPv GTPv GT
34. Mall das Lal das ts 25 Bel UMAP PIO GIR inoen E EEEE ttm cite ttl catia t ter eit tte teins 25 o VOCS PIII inh oat A ssbal seal acetal hatched A ceshalstebaheh cecal Satchel eA databalers 26 3 9 Oracle Plugin oo cc ccccccscesesesesesescscscscsesesesesesesesesesescacscsesneneeeeseseseseseseseaes 26 3104 POPS PUG es e a ote a Het es etn ed lat ele ater less ela te 27 CAH mes 0 koa 2 010119 Ren Rae Re A RnR a AR a A A R 27 3 12 742221 9 Pee eeer a nen Seat eee ae a ee Ree ae ae ne ee ee 27 ro Re mes og 2 C0 eses eaaa a a aae AEA a AAAA ERE EAE EEEa EE 28 3AA Oe tO NU sees bate ete tet tah ah fate ta tet taht tal tet hada tat a tet ta ints 28 3 15 NetFlow Lite 21 Te g Renee ene tee ne ote es es ire Ese ne ee eee ee ee tee ee 28 A Developing NProbe PIUGINS 0 0 ccccccscssesesesesesesesesescseessuesesesesesesesesescscscsneneesesenens 29 tS 21 316 eee er ee or E ee rea E ee ree ee 34 BM MCCCONS E htt Rte tah De hes Ra R Oe Oe Ae A eek meee 34 6 Appendix A BPF Packet Filtering Expressions c cccccccsesesseeseeteeseeeeteseseeteeaeeees 35 eTA a lt 0 01 cee een RITE A ERR SERN N ATTA ESR SISOS AE AAI AMA AA IRN A ET 38 7 Appendix B Flow Information Elements c cccccscesesssseseceesesesesessesesneeseseseeneaeeeenens 40 8 Appendix C nProbe Usage MOdeS cccccessssesessesssesestesesesnssesesestessseseeeaeseseeneaeeees 44 9 Appendix D nProbe License oo eeeeceesecsesesessesesesesseseseenen
35. PV1_RSP_MSG_TYPE GTPV1_C2S_TEID_DATA GTPV1_C2S_TEID_CTRL GTPV1_S2C_TEID_DATA GTPV1_S2C_TEID_CTRL GTPV1_END_USER_IP GTPV1_END_USER_IMST GTPV1_END_USER_MSISDN GTPV1_END_USER_IMET sGTPV1_APN_NAME GTPV1_RAI_MCC sGTPV1_RAI_MNC GTPV1_RAI_LAC GTPV1_RAI_RAC GTPV1_ULI_MCC GTPV1_ULI_MNC GTPV1_ULI_CELL_LAC GTPV1_ULI_CELL_CI GTPV1_ULI_SAC sGTPV1_RESPONSE_CAUSE Signaling Protocol templates IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 35632 270 35632 271 272 273 35632 274 275 35632 276 277 35632 278 279 280 35632 282 35632 281 333 180 181 182 35632 183 184 187 35632 188 361 IPFIX 35632 260 Plugin templates 35632 IPFIX IPFIX IPFIX IPFIX IPFIX IPFIX 35632 35632 35632 195 196 35632 197 198 199 35632 320 GTPV2_REQ_MSG_TYPE GTPV2_RSP_MSG_TYPE GTPV2_C2S_S1U_GTPU_TEID GTPV2_C2S_S1U_GTPU_IP GTPV2_S2C_S1U_GTPU_TEID GTPV2_S2C_S1U_GTPU_IP GTPV2_END_USER_IMST GTPV2_END_USER_MSISDN sGTPV2_APN_NAME GTPV2_ULI_MCC GTPV2_ULI_MNC GTPV2_ULI_CELL_TAC GTPV2_ULI_CELL_ID sGTPV2_RESPONSE_CAUSE Plugin HTTP Protocol templates 35632 360 35632 SsHTTP_UR
36. Pv GTPv GTPv GTPv GTPv GTPv GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv1 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 GTPv2 HTTP HTTP HTTP HTTP HTTP HTTP HTTP HTTP HTTP Mail MySQL MySQL MySQL MySQL MySQL MySQL Oracle Oracle Oracle Oracle Oracle POP3 u RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU RADIU nProbe User s Guide v 6 16 Request Msg Type Response Msg Type Tunnel Identifier APN Name End User IP Address End User MSISDN Mobile Country Code Mobile Network Code Cell Location Area Code Cell Routing Area Code Cause of Operation Request Msg Type Response Msg Type Client gt Server Tunnelld Client gt Server Tunnelld Server gt Client TunnellId Server gt Client TunnellId Data Control Data Control End User IP Address End User IMSI End User MSISDN End User IMEI APN Name RAI Mobile Country Code RAI Mobile Network Code RAI Location Area Code RAI Routing Area Code ULI Mobile Country Code ULI Mobile Network Code ULI Cell Location Area Code ULI Cell CI ULI SAC Cause of Operation Request Msg Type Response Msg Type Client gt Svr S1U GTPU TEID Client gt Svr S1U GTPU IP Srv gt Client S1U GTPU TEID Srv gt Client S1U GTPU IP End User IMSI End User MSISDN APN Name Mobile C
37. TP_RCPT_TO Mail recipient The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order smtp dump dir lt dump dir gt Directory where SMTP logs will be dumped smtp exec cmd lt cmd gt Command executed whenever a directory has been dumped 4 15 NetFlow Lite Plugin This plugin collects NetFlow Lite flows and uses them as simulated packets as if they where received from a captured device As the plugin acts as a collector for flows sent in NF Lite format you need to specify the listening port and an optional number of sequential ports to which flows will be sent The more ports the more performance can be achieved nflite lt flow listen port low gt lt num ports gt gt I Specify NetFlow Lite listen port s max 32 28 nProbe User s Guide v 6 16 5 Developing nProbe Plugins Each nProbe plugin is implemented as shared library to be loaded at runtime by nProbe The probe comes with several plugins that can be used as example for this activity Below we list the main concepts you need to know if you plan to develop nProbe plugins Each plugin has to defined a plugin entry point as follows static PluginEntryPoint dbPlugin NPROBE_REVISION
38. _AS DST_AS SRC_MASK DST_MASK Note that the fields start with a and are separated by a space 43 nProbe User s Guide v 6 16 9 Appendix C nProbe Usage Modes nProbe can be used in three modes e Probe default e Collector Flow collection only no Probe e Proxy Receive flows via NetFlow and emit them optionally combining with captured traffic to a remote collector 1 Probe mode default NetFlow NetFlow IPFIX EUNE Flows nProbe NetFlow iPFIX gt Collector Probe Flows e g ntop eae A diniati gt DiswDB Save Flows to Probe Mode MySQL Disk optional Command nprobe i ethO n collector_ip 2055 2 Collector mode Save Flows to ae Traffic MySQL Disk Miror ae i id Collector Mode Command nprobe nf collector port 2055 3 Proxy mode NetFlow NetFlow IPFIX Probe Flows gt nProbe gt Collector e g ntop A teceeeeeeeces gt DiswDB Save Flows to MySQL Disk optional Proxy Mode Command nprobe nf collector port 2055 n collector_ip 2055 V 9 In proxy mode you can convert from to IPFIX NetFlow v5 v9 in order to smoothly upgrade to newer netflow protocol versions while capitalizing on previous protocol versions So you can for instance convert flows coming from your v5 router into IPFIX and vice versa Note that with some combinations e g from v9 to v5 you might loose some flow information 44 nProbe User s Guide v 6 16 10 Appendix D nP
39. ackets with TTL gt 224 and lt 255 OSI Source SAP OSI Traffic Only OSI Destination SAP OSI Traffic Only Src AS Src AS Src AS Src AS Src AS Src AS Src AS Src AS Src AS Src AS Dest AS Dest AS Dest AS Dest AS Dest AS Dest AS Dest AS Dest AS Dest AS Dest AS path path path path path path path path path path path path path path path path path path path path position position position position position position position position position position position position position position position position position position position position POANDUBWNP POANDUBWNEROS MAC of the DHCP client DHCP assigned client IPv4 address DHCP client name DNS query DNS query transaction Id DNS query type e g 1 A 2 NS DNS return code e g Q no error DNS of returned answers TTL of the first A record if any FTP client login FTP client password FTP client command FTP client command return code 4 Plugin GTPv Signaling Protocol templates NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 NFv9 57793 57794 57795 57798 57796 57797 57799 57800 57
40. allocated 21 free 8 ItoBeExported O0 lfrags 0 04 Jul 2007 18 16 00 nprobe c 1149 Fragment queue len 0 04 Jul 2007 18 16 00 nprobe c 1153 Num Packets 111 max bucket search 0 04 Jul 2007 18 16 00 nprobe c 1170 115 pkts rcvd 0 pkts dropped G start nprobe as a daemon Useful when starting nprobe as daemon O set the number of threads that fetch packets out of the network interface In general the more threads are available the better is the performance However it is not suggested to have too many threads as in some platforms this can slow down the probe Start with 1 and increase it if necessary We suggest to run nprobe as single threaded application and distribute the traffic across multiple probes using PF_RING e g PF_RING cluster or libzero In fact adding threads you will end up spending a lot of time on synchronization without improving the performance Please refer to this post http www ntop org nprobe 10 gbit line rate netflow traffic analysis using nprobe and dna for more information P dump flows F This path specifies the directory where flows will be dumped The dump format is text and it depends on the nProbe template specified with T It specifies the frequency at which files are dumped on disk D dump flows format nProbe User s Guide v 6 16 Flows stored on disks can be stored in two formats text with user specified format or SQLite format availability depends on the platform and
41. ate V flow export version Itis used to specify the flow version for exported flows Supported versions are 5 v5 9 v9 and 10 IPFIX 0 intra templates packet export It specifies the number of flow packets that are exported between two templates export L local networks Use this flag to specify format network mask e g 192 168 0 10 24 the list of networks that are considered local see c c track local hosts only It allows nProbe to set to 0 0 0 0 all those hosts that are considered non local see L This is useful when it is necessary to restrict the traffic analysis only to local hosts r set traffic direction nProbe User s Guide v 6 16 When this option is used L must be specified before r all the traffic that goes towards the local networks is considered incoming all the rest is outgoing This has effect on the u Q that are then forced with r if networks specify a mapping between MAC address Interface index Flags u and Q are used to specify the SNMP interface identifiers for emitted flows In mirrored environments it is possible to simulated a switched environment by playing with MAC addresses This option allows users to bind a MAC or IP address to a specified interfaceld The syntax of if networks is lt MACIIP mask gt lt interfaceld gt where multiple entries can be separated by a comma Example if networks AA BB CC DD EE FF 3 192 168 0 0 24 2 or if networks
42. ation elements GTPV2_REQ_MSG_TYPE GTPv2 Request Msg Type GTPV2_RSP_MSG_TYPE GTPv2 Response Msg Type GTPV2_C2S SIU_GTPU_TEID GTPv2 Client gt Svr SIU GTPU TEID GTPV2_C2S SIU_GTPU_IP GTPv2 Client gt Svr SIU GTPU IP GTPV2_S2C _SIU_GTPU_TEID GTPv2 Srv gt Client SIU GTPU TEID GTPV2_S2C_SIU_GTPU_IP GTPv2 Srv gt Client SIU GTPU IP GTPV2_END_USER_IMSI GTPv2 End User IMSI GTPV2_END_USER_MSISDN GTPv2 End User MSISDN GTPV2_APN_NAME GTPv2 APN Name GTPV2_ULI_MCC GTPv2 Mobile Country Code GTPV2_ULI_MNC GTPv2 Mobile Network Code GTPV2_ULI_CELL_TAC GTPv2 Tracking Area Code GTPV2_ULI_CELL_ID GTPv2 Cell Identifier GTPV2_RESPONSE_CAUSE GTPv2 Cause of Operation 24 nProbe User s Guide v 6 16 The plugin supports the following command line options that are used to specify where the optional GTP log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order gtpv2 dump dir lt dump dir gt Directory where GTP logs will be dumped gtpv2 exec cmd lt cmd gt Command executed whenever a directory has been dumped Please note that GTP U is not handled by this plugin but rather by the nProbe core when the tunnel option is used 4 6 HTTP Plugin This plugin dissects HTTP traffic information https can be decoded if the plugin is compiled with CyaSLL and the
43. avour of Registered Users NTOP di Deri Luca Licensor grants you a non exclusive license to use the Software including any updates of the Software that the Licensor may make available to you at his own discretion and any documentation files pertaining to the Software Documentation only in connection with a single hardware unit with a unique Media Access Control MAC address or unique system identifier of any kind All copyright notices in the Software and Documentation must be retained at all times The Software and Documentation shall be used only for your own personal non commercial use and not for the benefit of any other person or entity You may also make one copy of the Software in machine readable form only for back up purposes provided you properly reproduce also both any Licensor s copyright notices credits and any proprietary legends disclaimers 2 IP Ownership All the intellectual property rights pertaining to Software and Documentation shall be reserved to the Licensor at all times Provided the Section 1 above any other use of the Software by any person or entity of any kind is strictly forbidden and is a violation of this EULA All the rights not expressly granted to you herein are reserved to the Licensor You are not allowed to remove any Licensor s copyright notices credits and any proprietary legends disclaimers from any copy of Software Documentation or any component thereof 3 Restrictions Y
44. ce nprobe needs to be started with i 1 option 3 5 Licenses Installation Binary nProbe instances require a per server license that is released according to the EULA End User License Agreement as specified in the appendix Each license is perpetual i e it does not expire and it allows to install updates for one year since purchase license issue This means that a license generated on 1 1 2013 will be able to activate new versions of the software until 1 1 2014 If you want to install new versions of the software release after that date you need to purchase a new license or avoid further updating the software For source based nProbes you still have to obey to the nProbe license listed in appendix nProbe licenses are generated using the orderld and email you provided when the license has been purchased on http shop ntop org The licenses are generated at http www nmon net mklicense 3 6 Tuning nProbe Performance As nProbe can be deployed on very different environments it is necessary to tune it according to the network where is active In order to achieve a good probe setup it is necessary to understand how nProbe is working internally Each captured packet is analyzed associated to a flow and stored onto a hash Periodically the hash is analyzed and expired flows are emitted The hash size is static w flag as this allows nProbe to Allocate all the needed memory at startup this is compulsory on embedded syste
45. cifies the NetFlow collectors addresses to which nProbe will send the flows If more than one is specified they need to be separated with a comma or the n flag can be repeated several times e g n 172 22 3 4 33 172 22 3 4 34 and n 172 22 3 4 33 n 172 22 3 4 34 are equivalent When multiple collectors are defined you can control the way flows are exported using the a option see below if on a collector address the destination port is omitted flows are sent to 2055 port and whereas if all the option is not specified by default flows are sent to the loop back interface 127 0 0 1 on port 2055 If this parameter is used nProbe exports flows towards collector running at 127 0 0 1 2055 By default the UDP protocol is used but also TCP and SCTP Linux only when nProbe is compiled with SCTP support and the kernel supports it In this case you can specify the collector address as udp lt host gt lt port gt tcp lt host gt lt port gt and sctp lt host gt lt port gt interface name It specifies the interface from which packets are captured If i is not used nProbe will use the default interface if any In case a user needs to activate nProbe on two different interfaces then he she needs to activate multiple nProbe instances once per interface For debugging purposes it is possible to pass nProbe a pcap file from which packets will be read If nProbe is compiled and activated with PF_RING support you can specify multiple
46. d by nProbe are those specified in the NetFlow v9 RFC namely in square brackets it is specified the field Id as defined in the RFC As nProbe can be extended by means of plugins further information elements can be defined based on plugin presence 1 IN_BYTES S SoctetDeltaCount Incoming flow bytes src gt dst 2 IN_PKTS S SpacketDeltaCount Incoming flow packets src gt dst 3 SFLOWS Number of flows 4 PROTOCOL S Sprotocolidentifier IP protocol byte 58500 PROTOCOL_MAP IP protocol name 5 SRC_TOS ipClassOfService Type of service byte 6 TCP_FLAGS S StcpControlBits Cumulative of all flow TCP flags 7 L4_SRC_PORT sourceTransportPort IPv4 source port 58503 L4_SRC_PORT_MAP Layer 4 source port symbolic name 8 IPV4_SRC_ADDR sourcelPv4Address IPv4 source address 9 IPV4_SRC_MASK sourcelPv4PrefixLength IPv4 source subnet mask lt bits gt 10 SINPUT_SNMP ingressInterface Input interface SNMP idx 11 L4_DST_PORT sdestinationTransportPort IPv4 destination port 58507 L4_DST_PORT_MAP Layer 4 destination port symbolic name 58508 L4_SRV_PORT Layer 4 server port 58509 L4_SRV_PORT_MAP Layer 4 server port symbolic name 12 IPV4_DST_ADDR sdestinationIPv4Address IPv4 destination address 13 IPV4_DST_MASK sdestinationIPv4PrefixLengthIPv4 dest subnet mask lt bits gt 14 OUTPUT_SNMP egressInterface Output interface SNMP idx 15 IPV4_NEXT_HOP ipNextHopIPv4Address IPv4 next hop address 16 SR
47. defined it must be placed into the nProbe plugins directory so that the nProbe build process will detect and compile it 33 nProbe User s Guide v 6 16 6 References 1 Pa OO Oe Uae ESE e et N e o 6 1 Introduction to Cisco NetFlow http www cisco com warp public cc pd iosw ioft neflct tech napps_wp htm ntop http www ntop org nProbe http www ntop org nprobe html nBox http www nmon net nBox html Linux Debian http www debian org tcodump http www tcpdump org Extreme Happy Netflow Tool http ehnt sourceforge net Libpcap http www tcpdump org Winpcap http winpcap polito it PC Engines http www pcengines ch SQLite http www salite org Perf http dast nlanr net Projects Ipert Credits NetFlow is a trademark of Cisco Systems Windows is a trademark of Microsoft Corporation 34 nProbe User s Guide v 6 16 7 Appendix A BPF Packet Filtering Expressions This section has been extracted from the tcpdump man page and it describes the syntax of BPF filters you can specify using the f flag The expression consists of one or more primitives Primitives usually consist of an id name or number preceded by one or more qualifiers There are three different kinds of qualifier type qualifiers say what kind of thing the id name or number refers to Possible types are host net and port E g host foo net 128 3 port 20 If there is no type qualifier host is as
48. e work that the collector has to carry on nProbe supports various aggregation levels that can be selected specifying with the p flag The aggregation format is lt vlanid gt lt proto gt lt IP gt lt port gt lt TOS gt lt AS gt where each option can be set to O ignore or 1 take care Ignored fields are set to a null value For instance the value 0 0 1 0 0 0 is useful for creating a map of who s talking to who network conversation matrix acket capture filter This BPF filter see the appendix for further information about BPF filters allows nProbe to take into account only those packets that match the filter if specified a select flow export policy When multiple collectors are defined see n option nProbe sends them flows in round robin However it is possible to send the same flow to all collectors as a flow redirector does if the a option is used b enable verbose logging Using this flag nProbe generates verbose output that can be used to tune its performance see chapter 2 4 Zero is the lowest level little information is printed 1 displays traffic statistics 2 is really verbose Example of traffic statistics 04 Jul 2007 18 16 00 nprobe c 1129 Average traffic 1 7 pkt sec Kb sec 04 Jul 2007 18 16 00 nprobe c 1134 Current traffic 1 9 pkt seclfl Kb sec 04 Jul 2007 18 16 00 nprobe c 1140 Current flow export rate 0 9 flows sec 04 Jul 2007 18 16 00 nprobe c 1144 Buckets factive 13
49. e dumps statistics on proc net pf_ring stats lt nprobe stats file gt black list With this option you can specify a list of networks or hosts from which all the incoming packets will be discarded by the probe The accepted notation can be CIDR format or the classical network netmask format pcap file list lt file gt The specified file path contains a list of pcap files to be read in sequence by nProbe Use this option when you want nProbe to read a list of pcap files e g when generated using tcpdump nProbe User s Guide v 6 16 biflows export policy lt policy gt Bi directional flows are such when there is traffic in both direction of the flow i e source gt dest and dest gt source As mono directional flows might indicate suspicious activities this flag is used to determine the export policy e 0 Export all know i e mono and bi directional flows e 1 Export only bi directional flows discarding mono directional flows e 2 Export only mono directional flows discarding bi directional flows csv separator lt separator gt Override the default I separator in dumps with the specified one dont drop privileges Do not drop root privileges to user nobody when this option is specified See al unprivileged user later int this manual bi directional Force flows to be bi directional This option is not supported by NetFlow V5 that by nature supports only mono directional flows account I2
50. e same options otherwise specified on the command line The only difference between the command line and the configuration file is that different options need to be specified on different lines For instance nprobe n 127 0 0 1 2055 i en0 a p is the same as nprobe etc nprobe conf where etc nprobe conf contains the following lines cat etc nprobe conf n 127 0 0 1 2055 i en0 q p Note that flags with no parameter associated e g a also need to have specified Any standard NetFlow collector e g ntop can be used to analyze the flows generated by nProbe When used with ntop the nProbe can act as a remote and light traffic collector and nProbe User s Guide v 6 16 ntop as a central network monitoring console See chapter 3 for further information about this topic 3 4 nProbe on Windows nProbe is activated as service or application i e you can start it from cmd exe The nProbe installer registers the service and creates an entry on the Start menu Example E ntop Source nprobe Debug gt nprobe h Available options i nprobe options Install nprobe as service c nprobe options Run nprobe on a console t Deinstall the service Example Install nprobe as a service nprobe i i O n 192 168 0 1 2055 Remove the nprobe service nprobe r Notes 1 Type nprobe c h to see all options 2 In order to reinstall a service with new options it is necessary to first remo
51. er element will be defined e u_int8_t protoMode Set it to BOTH_IPV4_IPV6 or ONLY_IPV4 ONLY_IPV6 if this element is for both IPv4 and IPv46 flows just for IPv4 flows or just for IPv flows e const u_int8_t isOptionTemplate Set it to 0 if this is a flow template default or 1 if it used as option template e const u_int8_t useLongSnaplen Set it to 1 if this plugin requires nProbe to capture packets with long snaplen that are needed when the plugin has to perform payload analysis e const u_int32_t templateElementEnterpriseld Specify the IANA defined enterprise Id for this custom field ntop uses NTOP_ENTERPRISE_ID for the proprietary ones e const u_intl6_t templateElementid Used by nProbe leave it to 0 e u_int8_t variableFieldLength Set it fo 1 to indicate that if nProbe exports flows in IPFIX format V 10 this field will have a variable field size e U_intl6_t templateElementLen Specify the static field size V 9 or max field size V 10 e const ElementFormat elementFormat Specify the format of the element This information will be used when this data is 31 nProbe User s Guide v 6 16 printed into MySQL The supported format types are ascii_format hex_format numeric_format ipv6_address_format e const ElementDumpFormat fileDumpFormat Specify the field format when the nProbe metadata information is printed metadata The supported format types are dump_as_uint dump_as_formatted_uint dump_as_ip_port
52. er engine e g InnoDB unprivileged user lt name gt nProbe User s Guide v 6 16 When nprobe drops privileges unless dont drop privileges is used the user nobody is used It is possible to use another user by using this option disable cache nProbe implements a flow cache for merging packets belonging to the same flow In proxy collector mode nProbe can disable this feature so that incoming flows are not put in cache but immediately exported redis lt host gt lt port gt The redis database when nProbe is compiled with it is used to implement a data cache and for aggregating flow information This option specifies the host and optionally the port where redis is listening nProbe opens several connections to redis not just one in order to maximize performance ucloud This option enables the micro cloud concept Please refer to http www ntop org nprobe monitoring on the microcloud for more information show system id Shown the systemlid where nProbe is running for binary nProbe s only check license Checks if the configured license is valid for binary nProbe s only dump plugin families Dump installed plugin family names As some people prefer to have a configuration file containing the options that otherwise would be specified on the command line it is also possible to start nProbe as follows nprobe lt configuration file path gt where the configuration file contains th
53. eseseeneseseseeneaeseeteneaeeeeneneaeeees 45 J0 App ndix E EULA sissors seietan tater nea eter tert eraa tnt kraon broot broot S atte 45 nProbe User s Guide v 6 16 2 Introduction Traffic measurements are necessary to operate all types of IP networks Networks admin need a detailed view of network traffic for security accounting and management reasons The compositions of the traffic have to be analyzed accurately when estimating traffic metrics or when finding network problems All of these measurements have to be made by analyzing all the packets flowing to the central points in the network such as router and or switches The analysis could be done on the fly or by logging all the packets and than post processing them But with the increasing network capacities and traffic volumes this kind of approach is not very efficient Instead similar packets packets with a set of common properties can be grouped together composing flows As an example a flow can be composed of all flowing packets that share the same source and destination address so a flow can be derived using only some fields of a network packet This way similar types of traffic can be stored in a more compact format without loosing the information we are interested in This information can be aggregated in a flow datagram and exported to a collector able to report network metrics in a user friendly format When collected this information provides a detailed view of the network t
54. f this Section 9 Termination shall in any case survive the termination or expiration hereof 11 General Provisions Governing Law Jurisdiction This EULA shall be governed by construed and interpreted in accordance with Italian Law The licensee agrees that any dispute arising from or connected to this EULA shall be submitted to the exclusive jurisdiction of the Italian Specialized IP Courts Therefore the jurisdiction of any other court is expressly excluded This EULA shall constitute the entire agreement between the parties any waiver or modification of this EULA shall be effective only if it is in writing and signed by both parties Should any part of this EULA be found invalid or unenforceable by an Italian Specialized IP Court the remainder of this EULA shall be interpreted so as to reasonably effect the intention of the parties EACH PARTY IS HEREBY CONFIRMING ITS AGREEMENT WITH THE FOREGOING BY SIGNING AND RETURNING ONE COPY OF THIS EULA TO THE OTHER PARTY PURSUANT TO ARTICLES 1341 AND 1342 OF THE ITALIAN CIVIL CODE THE PARTIES HEREBY ACKNOWLEDGE AND EXPRESSLY APPROVE SECTIONS 1 Grant of License in Favour of Registered Users 2 IP Ownership 3 Restrictions 5 No Warranty 6 Limitation of Liability 8 High Risk Activities 9 Termination AND 10 General Provisions Governing Law Jurisdiction 47
55. if nProbe has been compiled with it Using flow SQLite format D d can significantly reduce the size of stored files although all the collectors might not support this format Text flows D t are the safest setting if you want to use a standard collector able to read flows dump on disk You can also export core flow fields D B in binary format for post processing by binary applications Note that this flag has no effect unless P is used u input device index The NetFlow specification contains a numeric index in order to identify flows coming from different interfaces of the same probe As multiple nProbe instances can be started on the same host but on different devices the collector to divide flows according to interface number can use this flag If u is not used then nprobe will use O as interface index instead of 1 is used the last two bytes of the mac address of the flow sender will be used as index Q output device index Similar to u but for the output interface vlanid as iface idx lt mode inner outer gt nProbe can use the VLAN tag as interface identifier Using this flag you enable this feature As VLAN tags can be stacked you need to specify if the inner or outer tag will be used for the interface identifier discard unknown flows lt mode 0 1 12 gt nProbe includes nDPI support for analyzing packet contents in order to detect application protocol The mode value can be used to e 0 Export all know
56. ke care Example p 1 0 1 1 1 1 ignores the protocol whereas p 0 0 1 0 0 0 ignores everything but the IP bpf filter f lt BPF filter gt BPF filter for captured packets default no filter all collectors a If several collectors are defined this option gives the ability to send all collectors all the flows If the flag is omitted collectors are selected in round robin verbose b lt level gt Verbose output 0 No verbose logging 1 Limited logging traffic statistics 2 Full verbose logging daemon mode G Start as daemon num threads O lt threads gt Number of packet fetcher threads default 1 Use 1 unless you know what you re doing dump path P lt path gt Directory where dump files will be stored exec cmd dump R lt cmd gt Execute the specified command for each file dump on disk including plugins dump frequency F lt dump timeout gt Dump files dump frequencey sec Default 60 dump format D lt format gt lt format gt flows are saved as b raw uncompressed flows B raw core flow fields 152 bytes t text flows d SQLite Example D b Note this flag has no effect without P in iface idx u lt in dev idx gt Index of the input device used in the emitted flows incoming traffic Default value is 0 Use 1 as value to dynamically out iface idx Q lt out dev idx gt vlanid as iface idx lt mode gt discard unknown flows
57. l can be a number or one of the names icmp udp nd or tcp Note that the identifiers tcp udp and icmp are also keywords and must be escaped via backslash which is in the C shell ether broadcast True if the packet is an ethernet broadcast packet The ether keyword is optional ip broadcast True if the packet is an IP broadcast packet It checks for both the all zeroes and all ones broadcast conventions and looks up the local subnet mask ether multicast True if the packet is an ethernet multicast packet The ether keyword is optional This is shorthand for ether 0 amp 1 0 ip multicast True if the packet is an IP multicast packet ether proto protocol True if the packet is of ether type protocol Protocol can be a number or a name like ip arp or rarp Note these identifiers are also keywords and must be escaped via backslash In the case of FDDI e g tddi protocol arp the protocol identification comes from the 802 2 Logical Link Control LLC header which is usually layered on top of the FDDI header ntop assumes when filtering on the protocol identifier that all FDDI packets include an LLC header and that the LLC header is in so called SNAP format decnet src host True if the DECNET source address is host which may be an address of the form 10 123 or a DECNET host name DECNET host name support is only available on Ultrix systems that are configured to run DECNET decnet dst host True if
58. lled when the plugin is loaded in memory This function is called regardless of the fact that the plugin will later be used or not e PluginTermFctn termFctn Plugin termination function called when the plugin is terminated during nProbe shutdown e PluginConf pluginFlowConf Function that returns the flow configuration see below e PluginFctn deleteFlowFctn Flow callback that is called for flows handled by this plugin whenever a flow has been exported This function is used to free memory of resources associated to the flow Set it to NULL if no function will be defined e u_int8_t call_packetFlowFctn_for_each_packet Set it to 1 to ask nProbe to call the packetFlowFctn callback for every packet belonging to this flow or 0 for calling it only for the first flow packet e PluginPacketFctn packetFlowFctn Callback called whenever nProbe has a packet belonging to the flow to be processed by the plugin e PluginGetTemplateFctn getTfemplateFctn Function used to return the template Element for the specified information element passed as parameter e PluginExportFctn pluginExportFctn Callback called whenever the flow handled by this plugin is going to be exported e PluginPrintFctn pluginPrintFctn Function that is called when nprobe P is used and that is supposed to print flow information into text files e PluginStatsFctn pluginStatsFctn Function that is called when not set to NULL whenever nProbe prints periodic information b 1 or
59. load has a limited lifetime this is because the flow collector should periodically receive flow chunks for accounting traffic precisely In the following sections we discuss all the command line options and how to efficiently configure nProbe to run on your network 3 1 Compiling nProbe Source Code The nProbe source code if you have decided to compile nProbe from source instead of using a binary package on Unix it can be compiled as follows cd lt nprobe source code directory gt autogen sh make On Windows the compilation is much more complicated as NET compiler is needed and all the code dependencies must be satisfied For this reason ntop releases a pre built nProbe binary for the windows platform Please note that the nProbe source code compiles both on Unix and Windows 3 2 Installing a Binary nProbe The windows version of nProbe comes in a standard installer package that can be installed using the wizard On Linux we pre build two packages for the two most popular platforms Ubuntu Server LTE x64 and CentOS x64 We always build binaries for the latest server versions Such packages can be installed from e http apt ntop org Ubuntu e http rom ntop org CentOS Often the above packages can be installed on sister distributions such as Debian and RedHat Fedora although we cannot guarantee that they will work or install properly Once the installation is completed it is necessary to create the nProbe
60. mple rate that reduce the egress flow rate thus lowering the load on collectors The default value is 1 1 no packet sample no flow sample A AS file Network probes are usually installed on systems where the routing information is available e g via BGP in order to specify the AS Autonomous System id of the flow peer As nProbe has no access to BGP information unless you enable the BGP plugin users need to provide this information by means of a static file whose format is lt AS gt lt network gt The file can be stored in both plain text and gzip format city list City List With this option you can enable geolocation of IP addresses at city country detail level Here you need to specify the GeolP city database e g GeoLiteCity dat Q It specifies the path where nProbe will save the process PID T flow template definition Contrary to NetFlow v5 where the flow format is fixed NetFlow V9 and IPFIX flows have a custom format that can be specified at runtime using this option as specified in appendix U flow template id NetFlow v9 and IPFIX flows format is specified in a template whose definition is sent by nProbe before to start sending flows The flow format is defined by T where U is used to set the template identifier This option should not be used unless the default template value 257 needs to be changed As based on T nProbe can define several templates this value is the one used for the first defined templ
61. ms where memory is limited and it is necessary to know at startup whether a certain application can operate with the available resources Avoid exhausting all the available memory in case of attacks that can produce several flows Selecting the hash size is a matter of trade off between efficiency an efficient hash is at least 1 3 empty and memory usage This statement does not mean that a huge hash is always the solution as the flow export process can be slower and more CPU cycles are needed as a large hash needs to be explored On the other hand the hash size is just a part of the problem In fact the hash fill percentage can be also controlled by other factors such as Reducing the flow lifetime t Reducing the maximum flow idle time d Increasing how often the hash is walked searching expired flows s nProbe allows users to ease the tuning process by printing the status of internal hashes using the b flag Users who experience severe nProbe performance problems packet loss 1 It is worth to remark that packets are captured while nProbe performs flow export i e packet capture is not stopped during flow export 2 Note that the basic hash has a static size specified by w that can grow as needed according to traffic conditions nProbe User s Guide v 6 16 or high CPU usage should start nProbe with b in order to find out whether their probe setup is optimal 3 7 Using nProbe with ntopng On
62. n gt flows intra templ o lt num gt local networks L lt nets gt local hosts only c local traffic direction r max flow size 0 lt size gt if networks 1 lt nets gt count 2 lt number gt collector port 3 lt port gt tunnel 5 no promisc 6 smart udp frags 7 ipsec auth data len 8 lt len gt dump stats 9 lt path gt black list lt networks gt pcap file list lt filename gt biflows export policy N lt pol gt csv separator lt separator gt dont drop privileges bi directional account 12 dump metadata lt file gt dump pkts lt pcap file gt max log lines lt num gt timestamp format lt mode gt ndpi proto lt proto gt account imsi traffic event log lt file gt collection filter lt filter gt imsi aggregation simulate storage zmq lt socket gt tcp lt server port gt dump bad packets lt file gt lru cache size lt size gt enable throughput stats ndpi proto ports lt file gt disable 17 protocol guess original speed dont reforge timestamps db engine lt database engine gt unprivileged user lt name gt disable cache redis lt host gt lt port gt nProbe User s Guide v 6 16 attempt to load GeoLiteCityv dat Put the PID in the specified file Specify the NFv9 IPFIX template see below Specify the NFv9 IPFIX template identifier default
63. n order to use it you need to e Edit the bgp_probe_client pl file and configure the IP address of the machine where the script is listening local_ip and its AS local_as the IP address of the router remote_ip and its AS remote_as Of course you better define a private AS for doing alll this BGP my local_ip 192 168 48 2 my local_as 65498 my remote_ip 192 168 48 1 my remote_as 2597 nProbe my nprobe_ip 127 0 0 1 my nprobe_port 4096 e Start the script and configure the router to connect to the script that acts as a server The router will initially send its BGP table and then periodically send BGP updates e Start nProbe on the same machine where the script is active with the option bgp port lt port gt where lt port gt is set to the value of nprobe_port With this plugin nProbe will emit AS information with exported flows using the information exported by the router via BGP If the plugin is not active nProbe will use information from GeolP if configured This plugin defines the following information elements used to export not just the AS to which flows belong to but also the whole AS path SRC_AS PATH_1 Src AS path position 1 SRC_AS_ PATH_2 Src AS path position 2 SRC_AS PATH_3 Src AS path position 3 SRC_AS_ PATH_4 Src AS path position 4 SRC_AS PATH_5 Src AS path position 5 SRC_AS_PATH_6 Src AS path position 6 SRC_AS PATH_7 Src AS path position 7 22 nProbe U
64. nProbe User s Guide Open Source Software and Hardware NetFlow v5 v9 Probe Version 6 16 April 2014 2002 14 nProbe User s Guide v 6 16 1 Table of Contents HHT aiigore We ter DOONAN 3 Tle i i Ce oA FEOTUCS EE 4 12 WATS NEW sooth ate ia Ae aaah E A A A Ma AD 4 DP OTe Ma aKa 1ER A E E EE E E E SEE E E E a a 6 2 1 Compiling nProbe Source Code ooo ecccccccssssesesesesesesceseseesesesesesesesesesessetenenes 6 2 2 Installing a Binary NPrObe cc ecsesescsssseesesesesecesesesesesssssscscseseseseseeeeeseeeeees 6 2 3 nProbe Command Line Options ccceccceecceeeeeeceeeeeeeeeeeeees 7 24 nProbe on VIN OWS cg cto gate Seat ca a ua end Be ses R 18 2 5 Licenses NSIGIONONaaeeresce arcane eee ee 19 2 6 TUNING nProbe Performance cecccccsccsssesesesescsesceesesesesesesesesesescscscsteeetetesesess 19 2 7 Using nProbe with MOOI G scar acess ctnectes eh setae eb iw van test rtises 20 2 8 Frequently Asked QUESTIONS 6 isicssssesct cnessstvertcsot seateeskesedekvtutadeusivendonsheensonstnes 20 AMO PIUCIS tet rene teh ae a Noth ah oth atk oe ht ot hth ot 22 Oi les DOR PID T E E Men Cesena es Be Oke testa AN 22 3 2 ONS PIUGINtetcnesok cia oe doi Akt ce at et a i 23 39m STR VO PI OMI ces state ae ee ota ai oe te Ae a ete a 23 347 GTP V I PION esl lla alll al AE A E lll halal ah lal lah lelelatahates 24 35e 21 2 9 24 lt 0 0 ake et ee eee en ee ee Se iE 24 3 6 HTTP Plugin ateenee ete tak de lake el lle
65. nProbe is available for use with no further configuration On the other end the Pro version requires a per server valid license in order to work 2 1 Main Features Some of the nProbe features include Limited memory footprint regardless of the network size and speed and CPU sawy Designed for running on environments with limited resources the nProbe binary Fully user configurable e Fully NetFlow v4 v5 v9 IPFIX compliant e High performance probe commercial probes included those embedded on routers and switches are often not able to keep up with high speeds or when able their performance decreases dramatically handling small size packets Ability to work as a NetFlow proxy Support for disk dump flow either text files or SQLite files and MySQL database server dump flow 2 2 What s New Release 6 16 April 2014 Updated nProbe with 6 16features Release 6 12 January 2014 Updated nProbe with 6 12 features Release 5 0 February 2008 Updated nBox firmware Updated nProbe with latest features Updated ntop with latest 3 3 X version Release 4 0 July 2007 Updated nBox with latest 2 6 kernel series image Updated nProbe with 4 9 version coverage Release 3 9 April 2005 Updated nBox section nProbe User s Guide v 6 16 Release 3 0 1 February 2004 Updated nBox section Release 3 0 January 2004 Added nProbe 3 0 coverage Release 2 2 October 2003 Added n
66. o the specified endpoint Example tcep 5556 or ipc flows ipc Deliver flows in JSON format to the specified server via TCP Dump bad undecodeable packets into the specified pcap file Users and protocol cache size Default 16384 Compute throughput stats that can be dumped when P is used Read custom ports definitions for nDPI see nDPI example protos txt When nDPI is enabled in case a protocol is not recognized nProbe guesses the protocol based on ports This option disables this feature and uses only strict payload dissection When using i with a pcap file instead of reading packets as fast as possible the original speed is preserved debug only Disable nProbe to reforge timestamps with i lt pcap file gt debug only Define the DB engine type example MyISAM InfiniDB This information is used by the database plugin Default MyISAM Use lt name gt instead of nobody when dropping privileges Disable flow cache for avoid merging flows This option is available only in collector proxy mode i e use i none Connected to the specified redis server nProbe User s Guide v 6 16 Example redis localhost Use a redis proxy e g https github com twitter twemproxy Enable the nProbe micro cloud Print the system identifier Checks if the license is present and valid Dump all available plugin families use redis proxy ucloud show system id check license dump plugin families n collector addresses This spe
67. or ace nprobe f host helios and hot or ace To select all IP packets between ace and any host except helios nprobe f ip host ace and not helios To select all traffic between local hosts and hosts at Berkeley nprobe f net ucb ether To select all ftp traffic through internet gateway snup note that the expression is quoted to prevent the shell from mis interpreting the parentheses nprobe f gateway snup and port ftp or ftp data To select traffic neither sourced from nor destined for local hosts if you gateway to one other net this stuff should never make it onto your local net nprobe f ip and not net localnet To select the start and end packets the SYN and FIN packets of each TCP conversation that involves a non local host nprobe f tcp 13 amp 3 0 and not src and dst net localnet To select IP packets longer than 576 bytes sent through gateway snup nprobe f gateway snup and ip 2 2 gt 576 To select IP broadcast or multicast packets that were not sent via ethernet broadcast or multicast nprobe f ether 0 amp 1 0 and ip 16 gt 224 To select all ICMP packets that are not echo requests replies i e not ping packets nprobe f icmp 0 8 and icmpl0 0 39 nProbe User s Guide v 6 16 8 Appendix B Flow Information Elements The T flag enabled users to specify the format of NetFlow v9 IPFIX flows The format options currently supporte
68. ou are not allowed to or permit assist any third party to a publish display disclose rent lease modify copy loan distribute or create derivative works based on the Software or any part thereof b reverse engineer decompile translate adapt or disassemble the Software or any part thereof c attempt to create or otherwise reproduce in any form the source code from the object code of any portion or component of the licensed Software d sublicense the Software or permit the exploitation of the Software by more than a single hardware unit with a unique MAC address or unique system identifier of any kind e attempt to disable or circumvent any technological protection measure of the Software or assist third parties to do so 4 Confidentiality You agree to maintain the confidentiality of Software and Documentation you will not disclose Software or Documentation or any information or materials related thereto to any third party without the express written consent of NTOP di Deri Luca You further agree to take all reasonable measures to limit access to Software and Documentation only to those of your employees who reasonably require such access to perform their employment obligations and who are bound by confidentiality agreements with you in order to maintain the confidentiality of Software and Documentation 5 No Warranty Software is provided as it is To the maximum extent permitted by applicable laws Licensor disclaims
69. ountry Code Mobile Network Code Tracking Area Code Cell Identifier Cause of Operation URL METHOD return code e g 200 304 Referer User Agent Mime Type Host Name Facebook Chat server without host name sender server version username database in use Query server response request gt response latecy usec Username Query Response Code Response String Query Duration msec ser login S Request Msg Type S Response Msg Type S User Name Access Only S Calling Station Id S Called Station Id S NAS IP Address S NAS Identifier S User IMSI Extension S User MSISDN Extension S Framed IP S Accounting Session Name S Accounting Status Type S Accounting Input Octets S Accounting Output Octets 42 NFv9 57726 IPFIX 35632 254 RADIUS_ACCT_IN_PKTS NFv9 57727 IPFIX 35632 255 RADIUS_ACCT_OUT_PKTS Plugin RTP Plugin templates NFv9 57622 IPFIX 35632 150 RTP_FIRST_SSRC NFv9 57623 IPFIX 35632 151 RTP_FIRST_TS NFv9 57624 IPFIX 35632 152 RTP_LAST_SSRC NFv9 57625 IPFIX 35632 153 RTP_LAST_TS NFv9 57626 IPFIX 35632 154 RTP_IN_JITTER NFv9 57627 IPFIX 35632 155 RTP_OUT_JITTER NFv9 57628 IPFIX 35632 156 RTP_IN_PKT_LOST NFv9 57629 IPFIX 35632 157 RTP_OUT_PKT_LOST NFv9 57633 IPFIX 35632 161 RTP_IN_PAYLOAD_TYPE NFv9 57630 IPFIX 35632 158 RTP_OUT_PAYLOAD_TYPE NFv9 57631 IPFIX 35632 159 RTP_IN_MAX_DELTA NFv9 57632 IPFIX 35632 160 RTP_OUT_MAX_DELTA
70. packets before the e delay default 1 Minimum TCP flow size in bytes If a TCP flow is shorter than the specified size the flow is not emitted default unlimited Limit the number of active flows This is useful if you want to limit the memory or CPU allocated to nProbe in case of non well behaved applications such as worms or DoS default 524288 Specify the engine type and id The format is engineType engineld default 0 66 where enginelId is a random number Minimum number of flows per packet unless an expired flow is queued for too long see 1 default 30 for v5 dynamic for v9 Specifies the address port of the flow sender This option is useful for hosts with multiple interfaces or if flows must be emitted from a static port IP lt pkt rate gt lt flow rate gt Packet capture sampling rate and flow sampling rate If lt pkt rate gt starts with it means that nprobe will report the specified sampling rate but will not sample itself as incoming packets are already sampled on the specified capture device at the specified rate Default 1 1 no sampling GeoIP file containing with known ASs Example GeoIPASNum dat GeoIP file containing the city IP mapping Note that nProbe will load the IPv6 file equivalent if present Example city list GeoLiteCity dat will also pid file g lt PID file gt flow templ1 T lt flow template gt flow templ id U lt templ id gt flow version V lt versio
71. private SSL key is available and configured in the plugin and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements HTTP_URL HTTP URL HTTP_RET_CODE HTTP return code e g 200 304 HTTP_REFERER HTTP Referer HTTP_UA HTTP User Agent HTTP_MIME HTTP Mime Type HTTP_HOST HTTP Host Name HTTP_FBOOK_CHAT HTTP Facebook Chat The plugin supports the following command line options that are used to specify where the optional HTTP log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order http dump dir lt dump dir gt Directory where HTTP logs will be dumped ssl config file lt path gt Configuration file for SSL certificate decoding ssl debug Enables ssl tracing highly verbose http exec cmd lt cmd gt Command executed whenever a directory has been dumped dont hash cookies Dump cookie string instead of cookie hash max http log lines Max number of lines per log file default 10000 http dump timeout After that timeout in sec the log file will be closed default 60 sec http ports List of ports used for http protocol default 80 https ports List of ports used for https protocol default 443 proxy ports List of ports used for proxy protocol default 3128 8080
72. r and not to combine primitives E g host foo and not port ftp and not port ftp data To save typing identical qualifier lists can be omitted E g tcp dst port ftp or ftp data or domain is exactly the same as icp dst port ftp or tcp dst port ffp data or tcp dst port domain Allowable primitives are dst host host True if the IP destination field of the packet is host which may be either an address or a name src host host True if the IP source field of the packet is host 35 nProbe User s Guide v 6 16 host host True if either the IP source or destination of the packet is host Any of the above host expressions can be prepended with the keywords ip arp or rarp as in ip host host which is equivalent to ether proto ip and host host If host is a name with multiple IP addresses each address will be checked for a match ether dst ehost True if the ethernet destination address is ehost Ehost may be either a name from etc ethers or a number ether src ehost True if the ethernet source address is ehost ether host ehost True if either the ethernet source or destination address is ehost gateway host True if the packet used host as a gateway l e the ethernet source or destination address was host but neither the IP source nor the IP destination was host Host must be a name and must be found in both etc hosts and etc ethers An equivalent expression is ether host ehost and not host host which can be
73. r s Guide v 6 16 4 10 POP3 Plugin This plugin dissects POP3 traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information element POP_USER POP3 user login The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order pop dump dir lt dump dir gt Directory where POP3 logs will be dumped pop exec cmd lt cmd gt Command executed whenever a directory has been dumped 4 11 Radius Plugin This plugin dissects Radius unencrypted traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements RADIUS_REQ_MSG_TYPE RADIUS Request Msg Type RADIUS_RSP_MSG_TYPE RADIUS Response Msg Type RADIUS_USER_NAME RADIUS User Name Access Only RADIUS_CALLING_STATION_ID RADIUS Calling Station Id RADIUS_CALLED_STATION_ID RADIUS Called Station Id RADIUS_NAS_IP_ADDR RADIUS NAS IP Address RADIUS_NAS_IDENTIFIER RADIUS NAS Identifier YRADIUS_USER_IMSI RADIUS User IMSI Extension RADIUS_USER_IMEI RADIUS User MSISDN Extension RADIUS_FRAMED_IP_ADDR RADIUS Framed IP RADIUS_ACCT_SESSION_ID RADIUS Accounting Session Name
74. raffic Precise network metric measurements is a challenging task so a lot of work has been done in this filed In commercial environments NetFlow is probably the de facto standard for network traffic accounting and billing NetFlow is a technology originally created by Cisco in 1996 and is now standardized as Internet Protocol Flow Information eXport IPFIX RFC 3917 NetFlow is based on the probe collector paradigm The probe usually part of network appliance such as a router or a switch is deployed on the measured network segment it sends traffic information in NetFlow format towards a central collector nProbe is a software NetFlow v5 v9 IPFIX probe able to collect analyze and export network traffic reports using the standard Cisco NetFlow v5 v9 IPFIX format It is available for most of the OSs on the market Windows BSD Linux MacOSX When installed on a PC nProbe turn it into a Network aware monitoring appliance This manual aims at describing how to use nProbe deploy it in networks and how to develop plugins for extending it functionalities nProbe User s Guide v 6 16 There are two main version of the nProbe probe e The one that is public available and distributed in both source and binary format see Appendix for the license information with only installation support e The binary nProne distributed only in binary format that comes with additional features with respect to the previous version The first version of
75. robe License nProbe is C 2002 14 by ntop di Luca Deri that is distributed under GNU GPL who owns the nProbe intellectual property and copyright beside the sFlow collector code If you wish to integrate or embed nProbe into a proprietary software or an embedded software hardware solution such as the nBox be aware that we offer alternative licenses that allow you to do that Please contact license ntop org GPL requires that any work derived from a GPL licensed work as nProbe must also be distributed under GPL As the term derivative work see http en wikipedia org wiki Derivative_work is not entirely clear we want to clarify this concept in the case of nProbe We consider a derivated work of nProbe for the purpose of this license if it does any of the following Integrates even partially nProbe source code Includes even partially nProbe copyrighted data files Integrates embeds nProbe into a binary installer application Includes the nProbe into an appliance router or similar device Links even through nProbe s plugins to a library that is not available under GPL Executes nProbe and uses the produced results usually flows either on the network disk or database Note that the list applies to both nProbe as a whole and also to portions of it The above list is not exhaustive but it s used to clarify the term derivative work with respect to nProbe This means that nothing prevents you from
76. s Seconds epoch of the last flow packet 152 FLOW_START_MILLISECONDS f LowStartMilliseconds Msec epoch of the first flow packet 153 FLOW_END_MILLISECONDS f LowEndMilliseconds Msec epoch of the last flow packet 239 BIFLOW_DIRECTION sbiflow_direction 1 initiator 2 reverseInitiator 277 OBSERVATION_POINT_TYPE Observation point type 300 OBSERVATION_POINT_ID Observation point id 302 SELECTOR_ID Selector id 304 IPFIX_SAMPLING_ALGORITHM Sampling algorithm 309 SAMPLING_SIZE Number of packets to sample 310 SAMPLING_POPULATION Sampling population 312 SFRAME_LENGTH Original L2 frame length 318 PACKETS_OBSERVED Tot number of packets seen 319 PACKETS_SELECTED Number of pkts selected for sampling 335 SELECTOR_NAME Sampler name 40 NFv9 57552 NFv9 57554 NFv9 57555 deprecated NFv9 57595 NFv9 57556 NFv9 57557 deprecated NFv9 57596 NFv9 57558 NFv9 57559 NFv9 57597 NFv9 57560 NFv9 57561 NFv9 57562 NFv9 57563 NFv9 57564 NFv9 57565 NFv9 57570 NFv9 57573 IPFIX 35632 80 F IPFIX 35632 82 C IPFIX 35632 83 C IPFIX 35632 84 S IPFIX 35632 85 S IPFIX 35632 88 N IPFIX 35632 89 N IPFIX 35632 90 N IPFIX 35632 91 N IPFIX 35632 92 N IPFIX 35632 93 N IPFIX 35632 98 C IPFIX 35632 NFv9 57819 NFv9 57818 NFv9 57806 NFv9 57807 NFv9 57808 NFv9 57809 NFv9 57810 NFv9 57811 NFv9 57812 NFv9 57813 NFv9 57821 NFv9 57822 IPFIX 35632 347 IPFIX 35632 346
77. ser s Guide v 6 16 SRC_AS_PATH_8 Src AS path position 8 SRC_AS PATH_9 Src AS path position 9 SRC_AS_PATH_10 Src AS path position 10 DST_AS PATH_1 Dest AS path position 1 DST_AS_PATH_2 Dest AS path position 2 DST_AS PATH_3 Dest AS path position 3 DST_AS PATH_4 Dest AS path position 4 DST_AS PATH_5 Dest AS path position 5 DST_AS PATH_6 Dest AS path position 6 DST_AS_PATH_7 Dest AS path position 7 DST_AS PATH_8 Dest AS path position 8 DST_AS_PATH_9 Dest AS path position 9 DST_AS PATH_10 Dest AS path position 10 4 2 DNS Plugin This plugin dissects DNS traffic and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements DNS_QUERY DNS query DNS_QUERY_ID DNS query transaction Id DNS_QUERY_TYPE DNS query type e g 1 A 2 NS DNS_RET_CODE DNS return code e g O no error DNS_NUM_ANSWERS DNS of returned answers Using dns dump dir lt dump dir gt it is possible to specify where the DNS dump files will be saved Each file is up to 1000 lines long and when is completed a new file will be created 4 3 GTPv0 Plugin This plugin dissects GTPvO signaling information GTP C and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements GTPVO_REQ_MSG_TYPE GTPvO Request Msg Type GTPVO_RSP_MSG_TYPE GTPvO Response Msg Type GTPVO_TID GTPvO0 Tunnel Identifier GTPVO_APN_NAME GTPvO APN Name GTPVO_END_USER_IP GTP
78. slow down nProbe export rate A nProbe has been for high speed networks 1Gb and above so its export rate can be high due to traffic conditions There are several solutions available 20 nProbe User s Guide v 6 16 a Specify a minimum intra flow delay e flag b Use several collectors and send them flows in round robin n flag in order to balance load among the collectors 21 nProbe User s Guide v 6 16 4 nProbe Plugins nProbe has been designed as an engine that processes packets and compute basic statistics and plugins that extend the core with additional capabilities Each plugin dissects a specific traffic e g SMTP email traffic but you can enable the use of multiple plugins simultaneously nProbe based on the template configuration T will selectively enable plugins and define as many templates as necessary Their number depends on the plugins enabled and on the fact that you might enable IPv4 and or IPv6 traffic support The following sections cover the configuration and information elements provided by each individual plugin Most plugins are available also in source format but sometimes due to license restrictions e g the plugin has been sponsored by a company that does not want others to access the source code we are unable to release all plugins in source format 4 1 BGP Plugin This plugin is used in combination with the bgp_probe_client p script for receiving BGP information and updates from a router I
79. ssible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order mysql dump dir lt dump dir gt Directory where MySQL logs will be dumped mysql exec cmd lt cmd gt Command executed whenever a directory has been dumped max mysq log lines Max number of lines per log file default 10000 4 9 Oracle Plugin This plugin dissects Oracle unencrypted traffic information and saves the queries log in dump files as well export the information via NetFlow IPFIX using the following information elements ORACLE_USERNAME Oracle Username ORACLE_QUERY Oracle Query ORACLE_RSP_CODE Oracle Response Code ORACLE_RSP_STRING Oracle Response String ORACLE_QUERY_DURATION Oracle Query Duration msec The plugin supports the following command line options that are used to specify where the optional log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order oracle dump dir lt dump dir gt Directory where Oracle logs will be dumped oracle exec cmd lt cmd gt Command executed whenever a directory has been dumped max oracle log lines Max number of lines per log file default 10000 Note that not all Oracle DB version might be supported by this plugin 26 nProbe Use
80. stVlanId Destination VLAN 60 IP_PROTOCOL_VERSION ipVersion 4 IPv4 6 IPv6 61 DIRECTION flowDirection It indicates where a sample has been taken always Q 62 IPV6_NEXT_HOP ipNextHopIPv6Address IPv6 next hop address 70 SMPLS_LABEL_1 smplsTopLabelStackSection MPLS label at position 1 71 MPLS_LABEL_2 smplsLabelStackSection2 MPLS label at position 2 72 MPLS_LABEL_3 smp lsLabelStackSection3 MPLS label at position 3 73 MPLS_LABEL_4 smplsLabelStackSection4 MPLS label at position 4 74 MPLS_LABEL_5 smplsLabelStackSection5 MPLS label at position 5 75 MPLS_LABEL_6 smplsLabelStackSection6 MPLS label at position 6 76 MPLS_LABEL_7 smplsLabelStackSection7 MPLS label at position 7 77 MPLS_LABEL_8 smplsLabelStackSection8 MPLS label at position 8 78 MPLS_LABEL_9 smplsLabelStackSection9 MPLS label at position 9 79 MPLS_LABEL_10 smplsLabelStackSection10 MPLS label at position 10 80 OUT_DST_MAC sdestinationMacAddress Destination MAC Address 95 APPLICATION_ID Ssapplication_id Cisco NBAR Application Id 102 SPACKET_SECTION_OFFSET Packet section offset 103 SAMPLED_PACKET_SIZE Sampled packet size 104 SAMPLED_PACKET_ID Sampled packet id 130 EXPORTER_IPV4_ADDRESS S exporterIPv4Address Exporter IPv4 Address 131 EXPORTER_IPV6_ADDRESS exporterIPv6Address Exporter IPv6 Address 148 SFLOW_ID f lLowId Serial Flow Identifier 150 FLOW_START_SEC f LowStartSeconds Seconds epoch of the first flow packet 151 SFLOW_END_SEC f LowEndSecond
81. sumed dir qualifiers specify a particular transfer direction to and or from id Possible directions are src dst src or dst and src and dst E g src foo dst net 128 3 src or dst port ftp data If there is no dir qualifier src or dst is assumed proto qualifiers restrict the match to a particular protocol Possible protos are ether fddi ip arp rarp decnet lat moprc mopdl tcp and udp E g ether src foo arp net 128 3 icp port 21 If there is no proto qualifier all protocols consistent with the type are assumed E g src foo means ip or arp or rarp src foo except the latter is not legal syntax net bar means ip or arp or rarp net bar and port 53 means tcp or udp port 53 fddi is actually an alias for ether the parser treats them identically as meaning the data link level used on the specified network interface FDDI headers contain Ethernet like source and destination addresses and often contain Ethernet like packet types so you can filter on these FDDI fields just as with the analogous Ethernet fields FDDI headers also contain other fields but you cannot name them explicitly in a filter expression In addition to the above there are some special primitive keywords that don t follow the pattern gateway broadcast less greater and arithmetic expressions All of these are described below More complex filter expressions are built up by using the words and o
82. the Internet there are several NetFlow collectors see Reference paragraph that can be used to handle flows generated by nProbe Among them ntopng is included This section explains how to configure ntopng to take advantage of nProbe Packet Capture j ZMQ inj m nProbe Flow Collection sFlow NetFlow IPFIX Fig 1 Using ntop with nProbe ntopng is using nProbe for converting flows from their native format to the ntop format JSON based Flows are delivered by nProbe to ntopng via ZMaQ In this case ntopng connects to nProbe via ZMQ contrary to what happens with NetFlow probes that send flows to the collector Supposing to have the probe active on host whose IP is 1 2 3 4 and ntopng on host 1 2 3 5 the configuration to be used are in case host apps are on the same host please replace the IPs with 127 0 0 1 ntopng i tcp 1 2 3 4 5556 nprobe zmq tcp 5556 i ethX n none b 2 3 8 Frequently Asked Questions 1 Q lm sending 60 bytes ping packets using ping s 60 but nProbe reports 92 bytes packets A nProbe counts the packet size at IP level An ICMP Echo Request packet with 60 bytes payload is 92 bytes long 2 Q need to capture traffic from several interfaces but nProbe allows just one interface to be used What can do A You can start several instances of nProbe each on a different network interface 3 Q nProbe is exporting flows too fast and my collector cannot keep up with it How can
83. v0 End User IP Address GTPVO_END_USER_MSISDN GTPv0O End User MSISDN GTPVO_RAI MCC GTPvO Mobile Country Code GTPVO_RAILMNC GTPvO Mobile Network Code GTPVO_RAI_CELL_LAC GTPvO Cell Location Area Code GTPVO_RAI CELL_RAC GTPv0 Cell Routing Area Code GTPVO_RESPONSE_CAUSE GTPvO Cause of Operation The plugin supports the following command line options that are used to specify where the optional GTP log file is saved As previously described for P dumps are nested in directories It is possible to instruct nProbe to execute a command when a directory not a log file if fully dumped i e nProbe has moved to the next directory in time order gtpv0 dump dir lt dump dir gt Directory where GTP logs will be dumped gtpvO exec cmd lt cmd gt Command executed whenever a directory has been dumped Please note that GTP U is not handled by this plugin but rather by the nProbe core when the tunnel option is used 23 nProbe User s Guide v 6 16 44 GTPv1 Plugin This plugin dissects GTPv1 signaling information GTP C and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements GTPV1_REQ_MSG_TYPE GTPv1 Request Msg Type GTPV1_RSP_MSG_TYPE GTPv1 Response Msg Type GTPV1_C2S_TEID_DATA GTPv1 Client gt Server Tunnelld Data GTPV1_C2S TEID_CTRL GTPv1 Client gt Server Tunnelld Control GTPV1_S2C_TEID_DATA GTPv1 Server gt Client Tunnelld Data GTPV1_S2C_TEID_CTRL GTPv1 Server gt Client
84. ve the service then add it again with the new options 3 Services are started stopped using the Services control panel item If nProbe is started on the console the c flag needs to be used e g nprobe c n 127 0 0 1 2055 If used as service the command line options need to be specified at service registration and can be modified only removing and adding the service The nProbe installer registers nProbe as a service with the default options If you need to change the nProbe setup you need to do as follows nprobe r Remove the service nprobe i lt put your options here gt Install the service with the specified options Services are started and stopped using the Services application part of the Windows administrative tools As network interfaces on Windows can have long names a numeric index is associated to the interface in order to ease the nProbe configuration The association interface name and index is shows typing the nprobe c h C ntop nprobe Debug gt nprobe exe c h Running nProbe for Win32 Welcome to nprobe v 4 9 2 for Win32 Built on 05 03 07 10 35 28 Copyright 2002 07 by Luca Deri lt deri ntop org gt Available interfaces index 0 Adapter for generic dialup and VPN capture index 1 Realtek 8139 series PCI NIC nProbe User s Guide v 6 16 For instance in the above example the index 1 is associated to the interface Realtek 8139 series PCI NIC hence in order to select this interfa
85. ware is not fault tolerant and is not designed manufactured or intended for any kind of use with on line control equipment in hazardous environments requiring fail safe performance such as in the operation of nuclear facilities aircraft navigation or communication systems air traffic control direct life support machines or weapon systems in which the failure of the Software could lead directly to death personal injury or severe physical or environmental damage all the so called High Risk Activities Accordingly Licensor specifically disclaims any express or implied warranty of fitness for High Risk Activities 9 Ethics The licensor commits itself to use the Software in compliance with all applicable local national and international laws rules and regulations including any laws regarding the transmission of technical data exported from its country of residence In no case the licensed Software can be used to track spy intercept or collect evidence of network communications to be used against individuals or organizations to prosecute individuals or organizations or to restrict their freedom 10 Termination Licensor may terminate this Agreement at any time if you violate its terms Upon termination you must immediately destroy or return to Licensor the Software and Documentation The provisions of Sections 2 IP Ownership 3 Restrictions 4 Confidentiality 5 No Warranty 6 Limitation of Liability and the provisions o
86. warranties of any kind either explicit or implied including without limitation implied warranties of merchantability and fitness for a particular purpose Licensor does not warrant that the functions contained in the Software will meet any requirements or needs you may have or that the Software will operate error free or in an uninterrupted fashion or that any defects or errors in the Software will be corrected or that the Software is compatible with any particular platform Licensor is not obligated to provide any updates to the Software 6 Limitation of Liability To the extent not prohibited by applicable laws in no event shall the Licensor be liable to you or any third party for any incidental or consequential damages including without limitation indirect special punitive or exemplary damages for loss of business loss of profits business interruption or loss of business information arising out of or related to the use of or the inability to use the Software or for any claim by any other party even if the Licensor has been advised of the possibility of such damages In no event shall Licensor s total liability to you for all damages exceed the amount of the license fee paid by you to Licensor for Software and Documentation 7 Export Restrictions You may not export Software and Documentation in violation of applicable laws and regulations 46 nProbe User s Guide v 6 16 8 High Risk Activities The licensed Soft
87. ype RTP_IN_MAX_DELTA Max delta ms 100 between consecutive pkts RTP_OUT_MAX_DELTA Max delta ms 100 between consecutive pkts RTP_SIP_CALL_ID SIP call id corresponding to this RTP stream 4 13 SIP Plugin This plugin dissects SIP traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements SIP_CALL_ID SIP call id SIP_CALLING_PARTY SIP Call initiator SIP_CALLED_PARTY SIP Called party SIP_RTP_CODECS SIP RTP codecs SIP_INVITE_TIME SIP SysUptime msec of INVITE SIP_TRYING_TIME SIP SysUptime msec of Trying SIP_RINGING_TIME SIP SysUptime msec of RINGING SIP_INVITE_OK_TIME SIP SysUptime msec of INVITE OK SIP_INVITE_FAILURE_TIME SIP SysUptime msec of INVITE FAILURE SIP_BYE_TIME SIP SysUptime msec of BYE SIP_BYE_OK_TIME SIP SysUptime msec of BYE OK SIP_CANCEL_TIME SIP SysUptime msec of CANCEL SIP_CANCEL_OK_TIME SIP SysUptime msec of CANCEL OK SIP_RTP_IPV4_SRC_ADDR SIP RTP stream source IP SIP_RTP_L4 SRC PORT SIP RTP stream source port SIP_RTP_IPV4_DST_ADDR SIP RTP stream dest IP SIP_RTP_L4_DST_PORT SIP RTP stream dest port SIP_FAILURE_CODE SIP failure response code SIP_REASON_CAUSE SIP Cancel Bye Failure reason cause 4 14 SMTP Plugin This plugin dissects IMAP traffic information and saves it in dump files as well export the information via NetFlow IPFIX using the following information elements SMTP_MAIL_FROM Mail sender SM

nProbe User`s Guide

Contents

Download Pdf Manuals

Related Search

Related Contents