How snap-to-grid and tolerance impact spatial calculations

Snap-to-grid is the action of positioning the points in a geometry so they align with intersection points on a grid. When aligning a point with the grid, the X and Y values may be shifted by a small amount - similar to rounding. In the context of spatial data, a grid is a framework of lines that is laid down over a two-dimensional representation of a spatial reference system. SAP Sybase IQ uses a square grid.

As a simplistic example of snap-to-grid, if the grid size is 0.2, then the line from Point( 14.2321, 28.3262 ) to Point( 15.3721, 27.1128 ) would be snapped to the line from Point( 14.2, 28.4 ) to Point( 15.4, 27.2 ). Grid size is typically much smaller than this simplistic example, however, so the loss of precision is much less.

By default, SAP Sybase IQ automatically sets the grid size so that 12 significant digits can be stored for every point within the X and Y bounds of a spatial reference system. For example, if the range of X values is from -180 to 180, and the range of Y values is from -90 to 90, the database server sets the grid size to 1e-9 (0.000000001). That is, the distance between both horizontal and vertical grid lines is 1e-9. The intersection points of the grid line represents all the points that can be represented in the spatial reference system. When a geometry is created or loaded, each point's X and Y coordinates are snapped to the nearest points on the grid.

Tolerance defines the distance within which two points or parts of geometries are considered equal. This can be thought of as all geometries being represented by points and lines drawn by a marker with a thick tip, where the thickness is equal to the tolerance. Any parts that touch when drawn by this thick marker are considered equal within tolerance. If two points are exactly equal to tolerance apart, they are considered not equal within tolerance.

As a simplistic example of tolerance, if the tolerance is 0.5, then Point( 14.2, 28.4 ) and Point( 14.4, 28.2 ) are considered equal. This is because the distance between the two points (in the same units as X and Y) is about 0.283, which is less than the tolerance. Tolerance is typically much smaller than this simplistic example, however.

Tolerance can cause extremely small geometries to become invalid. Lines which have length less than tolerance are invalid (because the points are equivalent), and similarly polygons where all points are equal within tolerance are considered invalid.

Snap-to-grid and tolerance are set on the spatial reference system and are always specified in same units as the X and Y (or Longitude and Latitude) coordinates. Snap-to-grid and tolerance work together to overcome issues with inexact arithmetic and imprecise data. However, you should be aware of how their behavior can impact the results of spatial operations.

Note: For planar spatial reference systems, setting grid size to 0 is never recommended as it can result in incorrect results from spatial operations. For round-Earth spatial reference systems, grid size and tolerance must be set to 0. SAP Sybase IQ uses fixed grid size and tolerance on an internal projection when performing round-Earth operations.

The following examples illustrate the impact of grid size and tolerance settings on spatial calculations.

Example 1: Snap-to-grid impacts intersection results

Two triangles (shown in black) are loaded into a spatial reference system where tolerance is set to grid size, and the grid in the diagram is based on the grid size. The red triangles represent the black triangles after the triangle vertexes are snapped to the grid. Notice how the original triangles (black) are well within tolerance of each other, whereas the snapped versions in red do not. ST_Intersects returns 0 for these two geometries. If tolerance was larger than the grid size, ST_Intersects would return 1 for these two geometries.

Graph with two black triangles and two red triangles. The black triangles represent where the geometries are located before being snapped to grid. The red triangles represent where the geometries are located after being snapped to grid.

Example 2: Tolerance impacts intersection results

In the following example, two lines lie in a spatial reference system where tolerance is set to 0. The intersection point of the two lines is snapped to the nearest vertex in the grid. Since tolerance is set to 0, a test to determine if the intersection point of the two lines intersects the diagonal line returns false.

In other words, the following expression returns 0 when tolerance is 0:

vertical_line.ST_Intersection( diagonal_line ).ST_Intersects( diagonal_line )

Graph with two black lines, one of which crosses over the other. A red dot indicates where the intersection point is after snap-to-grid is applied. However the red dot does not appear to be the actual intersection point.

Setting the tolerance to grid size (the default), however, causes the intersection point to be inside the thick diagonal line. So a test of whether the intersection point intersects the diagonal line within tolerance would pass:

Graph with two black lines, one of which crosses over the other. A red dot indicates what the intersection point is after snap-to-grid is applied. However, because tolerance is set high, the intersection point is actually located where the lines intersect.

Example 3: Tolerance and transitivity

In spatial calculations when tolerance is in use, transitivity does not necessary hold. For example, suppose you have the following three lines in a spatial reference system where the tolerance is equal to the grid size:

Example showing a black line, which appears straight, a red line, which is bowed out slightly from the black line, and a blue line, which is bowed out slightly from the red line. The illustration here is that even though the black and red lines are equivalent within tolerance, and the red and blue lines are equivalent within tolerance, it doesn't mean that the black and blue lines are equivalent, since they are outside of tolerance.

The ST_Equals method considers the black and red lines to be equivalent within tolerance, and the red and blue lines to be equivalent within tolerance but black line and the blue line are not equivalent within tolerance. ST_Equals is not transitive.

ST_OrderingEquals considers each of these lines to be different, and ST_OrderingEquals is transitive.

Example 4: Impact of grid and tolerance settings on imprecise data

Suppose you have data in a projected planar spatial reference system which is mostly accurate to within 10 centimeters, and always accurate to within 10 meters. You have three choices:

Use the default grid size and tolerance that SAP Sybase IQ selects, which is normally greater than the precision of your data. Although this provides maximum precision, predicates such as ST_Intersects, ST_Touches, and ST_Equals may give results that are different than expected for some geometries, depending on the accuracy of the geometry values. For example, two adjacent polygons that share a border with each other may not return true for ST_Intersects if the leftmost polygon has border data a few meters to the left of the rightmost polygon.
Set the grid size to be small enough to represent the most accuracy in any of your data (10 centimeters, in this case) and at least four times smaller than the tolerance, and set tolerance to represent the distance to which your data is always accurate to (10 meters, in this case). This strategy means your data is stored without losing any precision, and that predicates will give the expected result even though the data is only accurate within 10 meters.
Set grid size and tolerance to the precision of your data (10 meters, in this case). This way your data is snapped to within the precision of your data, but for data that is more accurate than 10 meters the additional accuracy is lost.

In many cases predicates will give the expected results but in some cases they will not. For example, if two points are within 10 centimeters of each other but near the midway point of the grid intersections, one point will snap one way and the other point will snap the other way, resulting in the points being about 10 meters apart. For this reason, setting grid size and tolerance to match the precision of your data is not recommended in this case.