Order of processes from: api/v2/export (get list of export processes)

API
#1

The order in which processes are returned from

api/v2/export

using limit and/or offset seems to be random, definitely not chronological i.e. not by JobId.

I have a server with 136 completed export processes. When I retrieve the list using above API endpoint, I get:

limit = 10:
JobIds: 83,84,92,94,95,108,109,119,120,130
limit = 10, offset = 10:
JobIds: 85,96,97,98,110,121,122,127,131,132

I want to use the request to determine the latest completed job (highest JobId).
If I don’t supply any limit/offset I get 52 processes, including the latest one (JobId 136). This is exactly the list shown in the Data Export tab (presumably generated by the same request).
My doubt is: what happens when I have 500 jobs in the list? Will the latest job always be among those returned?
Ideally I would request jobs with limit 1, expecting to get the latest job.
When I do this, however, I get job 94.

Does anyone have an idea:

  • what determines the order of jobs returned in this request?
  • or another way how to determine the JobId of the latest completed export process?

Thanks,

Klaus

#2

I would just like to mention why it can be important to determine the latest completed export process:

In a monitoring app for a survey with huge amounts of interviews (e.g. a census) it is important not to generate unnecessary export files, because that consumes huge amounts of time and storage space.
So for a monitoring request I want to determine if there already exists an export file for the current date.
If so, I just download it, otherwise I generate a new export file.
This way any number of monitoring requests during a day (could be hundreds if supervisors need this monitoring functionality) will generate a single export file.

#3

I haven’t dealt with this endpoint in a long time.

Does it return the count of all jobs? (For the list of assignments, /api/v1/assignments/ returns TotalCounts. From the API documentation, I don’t see any similar info being returned about export jobs.)

If so, would it make sense to compile all jobs, making a request for each page of jobs, and from that compilation identifying the latest job?

If not, this seems like a reasonable feature request:

  • Have the endpoint return the total count of export jobs
  • Return export jobs in some order.
#4

Export Job objects returned have StartDate and CompleteDate timestamps, you can inspect/sort by those to pick a suitable job.

#5

This issue is fixed an will be released in the closest future with the next version.
Sorry for inconvenience.

#6

@arthurshaw2002 No, it doesn’t return a total count. If it would I would have the latest jobID.

@zurab Yes, that would be a workaround. But I would have to get all the export processes to be sure the latest is among them. If the request without any limit/offset returns the latest jobs (as it does for the limited number of jobs I have at the moment) it would be a solution. But does it do this always?

@vitalii Thank you. In which way has it been fixed? Does it return the jobs in the order of recency now?

Thanks for your help.

Klaus

#7

@klaus There was an issue that sorting was applied only to the specific batch(page) of the results but not to the whole set. Now the sorting applies first and then slicing imposed.

#8

I see. Perfect, Thanks.