esp_kdbin

Reads data from a KDB database table into an Event Stream Processor stream.

The esp_kdbin adapter reads data from a KDB database into a stream in the Event Stream Processor. You can configure the adapter to read either queried or streaming data, based on a configuration parameter.

By default, the adapter matches the field names (in a case-insensitive manner) to decide the mapping between the source KDB table and the target stream. You also have the option of explicitly specifying the mapping.

Syntax

esp_kdbin -H [kdbhost:]kdbport -p [host:]port -q source -s stream -c user[:password] [OPTION...]

Required Arguments

-c user[:password] – Passes authentication credentials to the Event Stream Processor. If you do not provide either the password or the -k option, you are prompted for the password. If the Event Stream Processor successfully authenticates with these credentials, the connection is maintained, otherwise the Event Stream Processor will immediately close the connection.
-H kdbhost:kdbport – Specifies the port number, or the host name and port number, on which KDB is listening. The default value is localhost:5001.
-p [<host>:]<port>/workspace-name/project-name – Together, the host:<port>/<workspace name>/ <project name> arguments specify the URI to connect to the ESP Server (cluster manager). For example, if you have started your ESP cluster server in the port 19011, the host name is set as localhost, and you are running a project called prj1 in the default workspace, specify -p as: -p localhost:19011/default/prj1
-q source – Specifies the KDB table when running in streaming mode. When running in non-streaming mode, specifies a valid query string.
-s stream – Specifies the target stream: the stream to which the data being read is published.

Options

-a – Uses asynchronous mode transmission; where the adapter does not wait for acknowledgment from the Event Stream Processor that it received the data. This option is necessary when using a hot spare configuration to ensure that both the primary and the hot spare receive the data. The default value is 'non async'.
-b blocksize – Specifies how many records to put in a block of data for transmission. A higher value may increase throughput but it will also increase latency. A block may contain fewer records than specified if there is not enough data available. The default value is 64.
-d – Outputs debug messages.
-e – Uses encrypted OpenSSL sockets for all communications between the adapter and the Event Stream Processor (which requires that it be started in encrypted mode). When this option is not present, no encryption occurs. The default is not encrypted.
-g gatewayhost – Uses the specified gateway host. Ignore the host name returned by the Event Stream Processor.
-h – Prints detailed help.
-k privateRsaKeyFile – Performs authentication using the RSA private key file mechanism instead of password authentication. The privateRsaKeyFile must specify the pathname of the private RSA key file.
Note: Ensure that the ESP Server has been started with the -k option specifying the directory in which to store the RSA keys.
-I interval – Specifies the number of seconds to wait before running the supplied query again when running in nonstreaming mode. A value of 0 indicates that the query should only be run once; no polling is performed. The default value is 0.
-M mapping – Specifies a mapping between the column name in the target stream and the column name in the KDB database table. The mapping is a colon-separated series of SPColumn=KDBColumn statements. If this parameter is not provided, the connector absorbs data only for those columns where the target stream column name matches the source table column name (in case-insensitive manner).
-m – If this option is not specified, the adapter connects to a KDB database and reads in streaming data. If it is specified, the adapter executes the supplied database query and feeds the result to the Event Stream Processor. The default is to use streaming mode.
-T attempts – Specifies the number of times to attempt to reconnect to the KDB database if the connection breaks during operation. The default value is 1.
-t – Uses transaction blocks. This improves performance, but causes all records in a block to be rejected when one record fails. The default is to use envelopes.
-u user:password – Passes authentication credentials to the KDB database.

Examples

Execute a basic streaming mode query that reads data from a KdbTrades table in a KDB database on the server myServer, where KDB is listening on port 9200, and writes it to the SpTrades stream in the Event Stream Processor on the local server where the Command and Control interface is on port 1190, use:

esp_kdbin -p 1190 -H myServer:9200 -q KdbTrades -s SpTrades

To execute the same query, explicitly mapping fields in the KDB database to columns in the Event Stream Processor stream, use:

esp_kdbin -p 1190 -H myServer:9200 -q KdbTrades -s SpTrades \
-M SpId=KId:SpSymbol=KSymbol:SpPrice=KPrice:SpCount=KCount

To execute a pull mode operation that issues the specified query every 5 seconds to a KDB database on the server myServer, where KDB is listening on port 9200, and writes data to the Event Stream Processor, on the server outputServer, where the Command and Control interface is on port 1221, use:

esp_kdbin -p outputServer:1221 -H myServer:9200 -q 'select Id, Symbol, Price, Count from KdbTrades' \
-s SpTrades -m -I 5