Data Synchronization: Data Filtering Overview

When developing a mobile application the concept of data filtering should always be at the forefront of the developer’s mind during all phases of the development life-cycle. Data filtering is the term used to refer to filtering the data provided to the mobile user so that unnecessary and unneeded data is not retrieved. When the proper data is retrieved for the user and unneeded data is excluded, the application ultimately provided will be far easier for the end users, and will operate more efficiently during synchronization and client-side operations.

The need for data filtering in mobile software development is driven by two main factors. First, while mobile devices continue to become more powerful and more sophisticated, they do not have the same capabilities as a traditional personal computer or work station. Attempts to store large amounts of data on such devices can result in, at the least, poor performance of the application, and at worst the client device can become overwhelmed and not function at all.

The second factor in the need for data filtering is the end user. Many users of mobile software need only certain information concerning a particular business entity. Additional information can result in a cluttered user interface as well as confusion on the part of the end users.

For these reasons as well as others the concept of data filtering can and should be applied to all areas of the application design and development process as it relates to the data structures.

Overall there are many areas in which data filtering should be applied and there are often many options available on how to do this. Which is used will depend on the type of data and what information is available about the data in the back end system. It is important to remember that the client device is not a permanent data store for the enterprise system in use. Rather, it is both a snapshot and a subset of data from that enterprise system.

Object Data Filtering: Property Definitions

Objects contain the child definition property. A property stores a single value for the object. A given object will contain multiple properties. When designing an object and the properties it will contain, it is important to consider what data the user actually needs.

Using a database back end system as an example, where an object is defined to contain data from a given table, the developer should always consider what data the end user will need from that table. Many database tables in a back end system can contain dozens or more columns. This data is necessary for the records within the table and is likely related to multiple processes. Examples of these processes and needs can include performance reporting, accounting requirements, change tracking, and auditing. However, much of this data is not needed by the end user. It will not be displayed to them nor captured from them on the Client. Furthermore it is not needed during downstream or upstream synchronization.

Because of this, there is no need to retrieve this data from the back end system. Though a given value may be small in size for a particular object, remember that it is likely that there will be dozens or hundreds of instances of a given object stored on the Client at a given time. As a result, a single unneeded value for an object can result in significant wasted resources on the Client for storage, as well as unnecessary bandwidth and processing being consumed during synchronization. These same statements can be made about any object for any type of back end system.

Complex Table Data Filtering: Field Definitions

Furthermore, complex tables should also be designed and developed with data filtering in mind. Complex tables contain field definitions, with each record in the table containing the fields defined for the table. Like properties, a single unneeded complex table field can result in significant wasted resources on the client device. With complex tables, however, the resources wasted can be even more detrimental than with objects. While there may be hundreds of instances of a given object stored on the Client, there can be thousands of complex table records.

User-Specific Data and Data Filtering

Taking this a step further, entire object instances or complex table records may be excluded from the Client if the proper design and development considerations are applied as they relate to data filtering. Whenever possible the developer should consider what data can be user specific. For objects this tends to be the case most of the time. Objects are usually defined to encapsulate business entities in the back end system that are user specific. A work order is assigned to a specific technician. A customer is assigned to a single account executive. These are two examples of what would normally be object definitions in a mobile application.

Complex tables tend to store data that is applicable to multiple users. Complex tables may contain records of inventory items available to be ordered by any customer, or assets for the company that one of many technicians may work with. However, there are still ways to filter this data. First, in some cases the data of a complex table may be user specific. In this case the data can easily be filtered for a given user during synchronization.

In other more common situations, the data is not user specific. In these cases the developer should look for other ways to partition data. Some suggestions can include the location a user may work in can mean certain records will never be needed. If a technician works in location A, then the complex table containing assets need only contain the assets that reside in location A. Similarly inventory items customers may order can also be filtered. If an account manager services customers in a specific industry or of a certain type, there may be items within the inventory that those customers will not order. Assuming the inventory information can be cross referenced with an industry or customer type, records can be excluded from the complex table containing that data on the Client.