Title Rows and the Hidden Row Timestamp Column

The ReadFromCSVFile adapter enables input data to contain an optional title row, which lists the names of the columns of data.

To ensure that the number of column names in the title row matches the number of columns in the data, you must enter a placeholder name in the title row if your input data contains a hidden row timestamp column. For example, if your data contains a name column, an ID column, and a hidden row timestamp column, then the title row might look similar to the following:

Ts,Name,ID

The name "Ts" is a placeholder. The name must follow the usual rules for column names, but the name is not used in any way. The name does not appear in the stream schema, and you cannot use the name in queries. Since the hidden row timestamp column must be the first column, the placeholder name must be the first name in the title row.

Other Time-Related Properties

Rate

The Rate property allows you to specify the rate (in rows per second) at which the server reads and processes rows. If you specify it, the minimum value for Rate is 0. The maximum rate you can enter is approximately 2 billion, but the practical maximum depends upon the speed of your computer.

Furthermore, the exact time that rows are sent depends on low-level factors such as task switching, so the actual rate at which rows are sent only approximately the same as the rate that you specify.

If Rate is set, the server will act as though "Set timestamp to current time" is also set. In other words, the actual time that the row is sent overwrites the row timestamp in the input file. Setting the rate does not change the row timestamps in the rows.

File has timestamp column

If you set the "File has timestamp column" property to true, the first column will be treated as the row timestamp column.

When the Read From CSV File adapter starts running, it sends the first row in the file immediately. Subsequent rows are sent based on the difference between row timestamps. For example, if the second row's timestamp is 5 seconds after the first row's timestamp, then the second row will be sent 5 seconds after the first row unless another property, such as Rate, overrides this. If you are using the Accelerated Playback feature, the actual time that the second row is sent depends on the Accelerated Playback rate.

Note:

The Rate property and Timestamp base property override or alter the behavior of the File has timestamp column property. See the descriptions of those properties for details.

If you set Has title row to true, the first row of the input file must include a name for the row timestamp column, even though that column name is not used in any way. For example, if your stream has three columns, row timestamp, department number, and department name, and if File has title row is set to true, then the first line of your input file should look similar to:

dummyTsColName,DeptNum,DeptName

where dummyTsColName may be any string that is valid as a column name.

Timestamp format column

The "Timestamp format column" property allows you to specify the format of the row timestamp column (for example, 'YYYY-MM-DD HH24:MI:SS.FF'). Since the row timestamp column is of type TIMESTAMP, the format may be any of the formats that are valid for columns of type TIMESTAMP. (See the CCL Reference for a description of the valid formats for TIMESTAMP data type. See Daylight Savings Time and the Time Zone Database and Reading, Writing, and Converting Timestamps for more information about time zones.)

The Timestamp format column property should only be set to a value if you set the property File Has Timestamp Column to true.

If you do not set the Timestamp Format Column, the server assumes that the row timestamp is specified in microseconds since midnight January 1, 1970 GMT.