Discussions

Ask a Question
Back to All

How to get incremental changes

Hi,

I am planning to setup a data extraction process that runs daily and pull data from _query and _export APIs.

I have done a one time full load where I have extracted complete data from all APIs. On ongoing basis I only what to pull the incremental data. As I am running the extraction process daily, I only want to only fetch the records that were modified or newly added on that day.

Can I use "modified" filter for "_query" API for this ?

$filter=modified gte '2024-03-17'

"_export" APIs also have modified timestamp columns. Can you do the same for that too?