Overview

The File PCAP Source is a source component aimed at efficiently reading PCAP files. It extends the functionality of a File Source component but adds additional features to deal with network traffic recorded in PCAP format. It does not support custom deserializers because it is focused on deserializing PCAP format to PCAP4J packets. 

FQCN

com.datumize.ddc.net.core.component.FilePcapSource


The only input expected is a base directory, that will be polled depending on the runtime policies that apply to the component. Usually, this source component triggers the execution of the pipeline and is configured to run scheduled or polling every number of milliseconds. The base directory should contain files in PCAP format.

Restriction: this component must be run just by 1 thread.

When the File PCAP Source component is activated, it behaves like a File Source component. However, the PCAP format is understood and network packets are retrieved in PCAP4J format. Once a file is picked up from the source directory, it creates PCAP4J packets.

Properties


Common properties that apply to all source and processor components of a pipeline.

PROPERTYIDDESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

Common
IdentifierIDComponent unique Identifier within the pipeline, read only, only useful for advanced mode.YesStringAuto

MyComponent_23423

This Identifier is automatically generated by the system and you can't change it. Might be helpful for advanced pipeline configuration.

DescriptiondescriptionA short description for the component, to be visualized inside the component representation, aimed at providing additional information to understand the pipeline at a glance.NoString

Extract customer id and loyalty number.

Short and sweet description.

TopictopicAll Source and Processor components support a topic to tag the output records. If the record does not have a tag applied, this topic will be automatically applied. The topic may be used by future components to route, group or classify records.NoString

foo

All output records will be tagged using "foo", unless they have not been tagged during the execution of the step.

All properties defined for File Source are available except a custom deserializer.

PROPERTYIDDESCRIPTIONREQUIREDTYPEDEFAULT
Default
Base directorydirectory-baseBase directory to read files from.YesPath
Network filterfilterNetwork filtering expression.NoPCAP Network Filter
Delete directory if emptydirectory-delete-emptyRemoves the directory once it is empty (all files have been processed). Please note that sufficient privileges are required.NoBooleanfalse
Filtering
File minimum agefile-age

Minimum file age for files to be retrieved.NoDuration5ms
File patternfile-pattern

A regular expressing pattern to match selected files. It accepts Unix style regular expressions.NoDuration*
File sortingfile-sort

Sorting criteria. Decide the ordering to pick up files, either by name or age.No

Constant:

NAME: Alphabetical order, A to Z, A first.

AGE: Age order, older files first.

NAME
File minimum sizemin-lengthThe minimum size for files to be retrieved.No

Capacity

0
File maximum sizemax-lengthThe maximum size for files to be retrieved.No

Capacity

0
Advanced
File suffix on successfile-suffix-on-success

The file suffix that will be appended once the file is processed without any error. A suffix can be used to avoid the file from being selected again, so you must take care to wipe the directory off eventually. If not indicated, the file is removed after processing.

No

String


File suffix on errorfile-suffix-on-errorThe file suffix that will be appended once the file is processed with any error. A suffix can be used to flag for reprocessing or discarding, so you must take care to wipe the directory off eventually. If not indicated, the file is removed after processing.No

String


Just Readjust-readTo avoid adding any suffix on success or error and just read the file without changing the state.NoBooleanfalse