Data status

Concept overview and key details

Synchronizing a data type ensures that our data cache is up-to-date with the data source and means you'll be able to retrieve the synchronized company's data using our API or view the data in the Portal.

There are three ways to initiate the synchronization of data:

  • Queue a sync to refresh the data from within Codat's Portal
  • Use the API to queue a data synchronization (known as a dataset)
  • Use the Sync settings feature in the Codat Portal

By default, full synchronization of each data type marked as Fetch on first link in your data type settings will be queued automatically when a company first authorizes itself with Codat.

Dataset statuses

All datasets are created in the Queued state. For online packages, the transition to Fetching will be near instant. For on-premise packages, the synchronization will remain Queued until the on-premises connector on the end user's machine is online and ready to process the dataset.

A dataset going to the AuthError state indicates that the connection to the data source is no longer authorized and will require the company to re-authorize your connection before you're able to complete subsequent data synchronizations for that connection.

Other error states may be resolved by queueing a new synchronization after waiting some time. For example, you may see datasets go to FetchError if you attempt a synchronization during a period of scheduled maintenance for the underlying data source.

Datasets can also transition to the NotSupported state if the dataset you are trying to sync is not supported by the linked platform.

A diagram of possible dataset statuses from Queued to CompleteA diagram of possible dataset statuses from Queued to Complete

Dataset statuses from Queued through to Complete.
Click to expand.

StateDescription
FetchingThe data is currently being pulled from the integration.
MappingThe data is being converted into Codat's standard model.
ValidatingThe data is being checked for consistency and correctness.
ProcessingThe data is being stored into Codat's cache.
CompleteThe dataset has completed, and the data is available to be queried via Codat's API.
NotSupportedThe integration does not support the datatype that was requested. For example, Clearbooks does not support pulling the Profit and Loss Report.
FetchError, MapError, ValidationError, ProcessingError, InternalErrorThe dataset failed in one of the above states. These are monitored by our engineering teams, but you may contact support team for assistance as required.
AuthErrorThe authentication to the data source has expired. This usually means you will need to relink, by getting the end user to follow the data connections's linkUrl to re-enter their credentials.
QueuedThe dataset has just been requested, and will move into fetching shortly.

Examples

Data type that has never been synced

{
  "suppliers": {
    "dataType": "suppliers”
  },
  ...
}

First sync for the data type failed

{
  "suppliers": {
    "dataType": "suppliers",
    "currentStatus": "FetchError",
    "latestSyncId": "31632c48-23dc-4cb1-b3ff-0829343c8e85",
  },
  ...
}

Last sync failed although a previous sync was successful

{
  "suppliers": {
    "dataType": "suppliers",
    "lastSuccessfulSync": "2019-10-10T00:31:04.497225Z",
    "currentStatus": "FetchError",
    "latestSyncId": "31632c48-23dc-4cb1-b3ff-0829343c8e85",
    "latestSuccessfulSyncId": "9d6d3754-deeb-42b7-ad37-e10942f9e258"
  },
  ...
}

Sync was successful

{
  "suppliers": {
    "dataType": "suppliers",
    "lastSuccessfulSync": "2019-10-10T00:31:04.497225Z",
    "currentStatus": "Complete",
    "latestSyncId": "31632c48-23dc-4cb1-b3ff-0829343c8e85",
    "latestSuccessfulSyncId": "31632c48-23dc-4cb1-b3ff-0829343c8e85"
  },
  ...
}