Overview

Serializers are converters aimed at transforming a single object into an array of bytes.Serializers must comply to following Java interface signature:

public interface Deserializer<I> {
byte[] serialize(I data);
}

Script Serializer

Serialize an object into an array of bytes using a custom scripting language. See DDC Extensibility (SDK) for more information. This serializer allows custom extension of the serialization, although you must consider the scripting language overhead in runtime.

PROPERTY

ID

DESCRIPTION

TYPE

REQUIRED

DEFAULT

EXAMPLES

ScriptscriptScript to convert input to outputStringYes

Languagelanguage

Language compatible with the JVM in which operations are written.

Note: the language pack must be installed in the system.

Script LanguageNogroovy

javascript

Select the  scripting language to be Javascript.

ParametersparametersScript optional parameters

Parameter List

No
param1=val1,val2;param2=va3
DependenciesdependenciesPath to external jars that will be added to the classloaderNoList of Paths
/home/acme/coyote.jar
Bootstrap scriptscript-bootstrapScript to execute when the step is initialized. Useful to initialize classesNoString

Map to AVRO Serializer

Serialize a Map into an AVRO binary.

PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

NamenameObject name.NoString
Documentationdoc

Object documentation.

NoString
SchemaschemaObject schemaNoString
FieldsfieldsAvro object fields.YesList of Avro Field

The serializer supports a list of Avro Fields in Avro format. Each field has the following properties.

PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

NamenameField nameYesString
idcard
Typetype

Field data type.

YesINT | LONG | DOUBLE | FLOAT | BOOL | STRING | ARRAY
STRING
Nullableis-nullableSpecify if the field can be null or not
Booleanfalsetrue

Map to CSV Serializer

Serialize a Map into a CSV line. Each map entry will be iterated in order, the key used as the column. The separator will be used to separate columns, and at the end of the iteration, the end of line will be used.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

FieldsfieldsOrdered list of fields.YesList of String
Name, Age, Income
Column separatorseparatorSeparator character for CSV columns.No

String

,

;

Sets the separator to a semicolon.

End of lineend-of-lineEnd of line character. Dec unicode ex: 10. Or the character itself '\n'.No

Character

System line separator (depends on OS)


Empty valueempty-valueValue if field does not exist or is null.NoString(empty string)
CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8

true

The serializer expects the first line to contain the list of column names, separated using the separator.

Map to JSONL Serializer

Serialize a Map into a JSONL. There is a direct translation between keys and values.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

End of lineend-of-lineEnd of line character. Dec unicode ex: 10. Or the character itself '\n'.NoCharacterSystem line separator (depends on OS)

Map to JSON Serializer

Serialize a Map into a JSON. There is a direct translation between keys and values.

PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8

true

The serializer expects the first line to contain the list of column names, separated using the separator.

Serializerscustom-serializersRegister custom serializers where key is the full class of the type to adapt and the value is the full class of the adapterNoList of Mappings

It is possible to use Mappings to map custom serializers based on source type and a custom JSON serializer.

Echo Serializer

Serializer aimed at doing nothing but just copying input to output.

Objects to CSV Serializer

Serialize some iterable objects into a CSV line. Each object will be iterated and serialized to a String. It allows user-defined characters to deliminate each column and row.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

Column separatorseparatorSeparator character for CSV columns.No

String

,

;

Sets the separator to a semicolon.

End of lineend-of-lineEnd of line character. Dec unicode ex: 10. Or the character itself '\n'.No

Character

System line separator (depends on OS)


CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8

true

The serializer expects the first line to contain the list of column names, separated using the separator.

Object to AVRO Serializer

Serialize a Java object into an AVRO binary.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

NamenameObject nameNoString
Documentationdoc

Object documentation.

NoString
SchemaschemaObject schemaNoString
FieldsfieldsAvro object fields.YesList of Avro Field

The serializer supports a list of Avro fields in Avro format . Each field has the following properties.

PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

NamenameField nameYesString
idcard
Typetype

Field data type.

YesINT | LONG | DOUBLE | FLOAT | BOOL | STRING | ARRAY
STRING
Nullableis-nullableSpecify if the field can be null or not
Booleanfalsetrue

Object to JSONL Serializer

Serialize a Java object into a JSONL. All object properties will be serialized into a single line in JSON format.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

Date formatdate-formatDate format to represent serialized dates.NoString according to SimpleDateFormatDefault JVM date format.

yyyy.MM.dd

To represent 2014.12.31

Time zonetime-zoneTime zone used to represent serialized dates.No

String according to TimeZone

Default JVM time zone.

PST

Sets the time zone to Pacific US time.

Include nullsnullsIndicate if null values should be serialized or not.No

Boolean

false


Ignore empty objectsignore-empty-beansBy default, empty objects are ignored to make the serialization more robust. If disabled, the serialization will fail with empty objects.NoBooleantrue
CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8

true

The serializer expects the first line to contain the list of column names, separated using the separator.

Serializerscustom-serializersRegister custom serializers where key is the full class of the type to adapt and the value is the full class of the adapterNoList of Mappings

End of lineend-of-lineEnd of line character. Dec unicode ex: 10. Or the character itself '\n'.NoCharacterSystem line separator (depends on OS)

It is possible to map custom serializers based on source type and a custom JSON serializer.

Object to JSON Serializer

Serialize a Java object into a JSON. There is a direct translation between object properties and JSON keys and values.


PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

EXAMPLES

Date formatdate-formatDate format to represent serialized dates.NoString according to SimpleDateFormatDefault JVM date format.

yyyy.MM.dd

To represent 2014.12.31

Time zonetime-zoneTime zone used to represent serialized dates.No

String according to TimeZone

Default JVM time zone.

PST

Sets the time zone to Pacific US time.

Include nullsnullsIndicate if null values should be serialized or not.No

Boolean

false


Ignore empty objectsignore-empty-beansBy default, empty objects are ignored to make the serialization more robust. If disabled, the serialization will fail with empty objects.NoBooleantrue
CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8

true

The serializer expects the first line to contain the list of column names, separated using the separator.

Serializerscustom-serializersRegister custom serializers where key is the full class of the type to adapt and the value is the full class of the adapterNoList of Mappings

It is possible to map custom serializers based on source type and a custom JSON serializer.

Object to XML Serializer

Serialize a Java object into a XML. There is a direct translation between object-XML attributes and values.

PROPERTY

ID

DESCRIPTION

REQUIRED

TYPE

DEFAULT

CharsetcharsetCharset to be used to encode the serialized String retrieved from each object.No

Charset

UTF-8