Controlling Sort Chunk Size

To enable sort-based operators in data services to handle arbitrarily large result sets, Sybase Data Federation breaks the inputs into chunks and uses the local disk as a temporary backing store to hold intermediate results. You can control the size of the chunks in the view model.

When a data service is executed, there’s a tradeoff between the amount of memory that a grid server uses for its computation and the amount of I/O that it must perform to sort or join large result sets. Data Federation can break the inputs into a large number of small chunks or into a smaller number of larger chunks.

A large number of small chunks requires less memory at any given time, but results in more disk I/O.
A small number of large chunks results in a larger peak memory usage, but less disk I/O overall.

By default, a view model operates with a chunk size of 10,000 rows. You can override this default in the view model’s JavaScript include (.jsi) file:

Right-click in the background of the view model editor and select Open JSI File. The editor creates, if necessary, and opens this model’s JSI file.
Insert a declaration of this form into the JSI file: var overrideSortChunkSize = <new_chunk_size_value>;
Right-click in the editor and select Save to save the file.

When the view model is deployed as a data service, WorkSpace Data Federation automatically includes the JSI file logic as part of the data service.

Send your feedback on this help topic to Sybase Technical Publications: pubs@sybase.com

Your comments will be sent to the technical publications staff at Sybase, Inc. For product-related issues or technical support, contact Sybase Technical Support at 1-800-8SYBASE.