Can you use the API R package on non-survey solutions server?

Hello, I’m having difficulty using the API R package to pull data from a non-survey solutions server. Is this possible in the first place?

Here some test code that breaks and the error in R is attached.

library(SurveySolutionsAPI)

pull metadata

suso_set_key(“https://www.chaton-study.com”, “XXXX”, “XXXX”)
questlist <- suso_getQuestDetails()
t <- suso_export(questID = “64907834-3220-4cbc-84a0-9e50e48797aa”, version = 1)

API of Survey Solutions doesn’t depend where the server is or what is it called. But it may depend on the version. The exact API documentation as applicable to your version you can see here:
https://www.chaton-study.com/apidocs/index.html

As far as I can see the problem (red text) is in the code of the library you are using, not in the Survey Solutions code.

Thank you for your help! I think it’s an issue of using the correct URL when using GET function to pull the data. If I want to pull all interviews for a given questionnaire, how could I find the appropriate URL to use for the GET function? I could not find this use case in the api docs.

And does Survey Solutions provide support for the API R package?

On the first question it is not the URL for GET that you should be looking for, but URL for EXPORT that you need. Click on the link that I have included in the above, scroll to Data Export and you will see both the URL and the corresponding method (GET, POST, PATCH, etc).

PS: GET has nothing to direct connection to exporting of data. It is called GET if it doesn’t affect the state of the server. Basically all questions like “Server tell me something, but just tell, don’t change anything about yourself”. More formally here: https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

Thanks! It looks like I do have the correct URL for EXPORT (I get a status 200) but the redirect url seems to be incorrect. And the format of the content for the GET call is “application/zip”, not “application/xml” as it it in other cases where this exact code has worked. I’m following the steps from this helpful forum but they don’t address my particular issue. And when I try the different GET methods suggested, I run into the same issue. The R code I’m using is below. The last line is where I call the redirect URL that seems to be incorrect.

#Specify action start to request the export file in the corresponding format
#to be rebuilt on the server
action = “start”
#use the sprintf() function to put the query together
query <- sprintf("%s/api/v1/export/%s/%s/%s",
headquarters_address, export_type, q_id, action)
query
#The next thing we need to do is request that the server builds the file.
#We can do that using the POST function which is available in the httr package.
#POST request: requesting the re-creation of a particular export file
data <- POST(query, authenticate(login, password))
data$status_code # returns 200

Sys.sleep(30)

Creating the query to download the dataset

#GET request: inspecting the status of a particular export file
action = “”
query <- sprintf("%s/api/v1/export/%s/%s/%s",
headquarters_address, export_type, q_id, action)
data <- GET(query, authenticate(login, password)) # status code 200
redirectURL <- data$url
rawdata <- GET(redirectURL, authenticate(login, password)) # status code 200 but returns content format “application/zip” not “application/xml”

Not sure what you mean exactly, but the response to downloading the data is a compressed zip file, so the declaration “application/xml” would be misleading.

should not be used in general. the duration to process the export job may depend on the presence of other jobs in the queue.

Once the redirect is followed, the exported data should come as a response. What is the stumbling block then?

Best, Sergiy

I have a loop that checks the status set up in my actual code - I just added that for the example.

The issue is that I end up with a zip file that, when unzipped, only returns another zip file. Maybe it’s corrupt or there’s nothing in it but the main difference I see between successfully exporting the data and not is the redirect URL. When it works, the redirect URL is a very long string that includes “amazonaws”. When it produces the bad zip file, the redirect URL is simply https://www.server_name/api/v1/export/tabular/questionnaire_ID/. But confusingly, this GET call does have 200 status code. I hope that’s clear.

I am running currently queries for export already on v2 of the export API and don’t have any similar problem.

I might be a bit limited in this, but I don’t understand the

When it produces the bad zip file, the redirect URL is simply https://www. server_name /api/v1/export/tabular/ questionnaire_ID /. But confusingly, this GET call does have 200 status code.

  1. You should be following the redirect when you get a redirect HTTP response code 30X (typically 301, 302), but not 200.
  2. I read it as “sometimes it works and sometimes it doesn’t” is that the correct interpretation?

Hello, thanks much for your interest in the Survey Solutions R API package. Package author here. I have seen your question just now.

Regarding your initial question: Yes, the package should run with any server running a recent version of Survey Solutions, under any domain.

As Sergiy already pointed out, the error seems to be thrown by the package, because of a lack of the questName variable. May i ask which server version you are using?

We had recently conducted an upgrade of the API, which is not yet reflected in the package. If you are using a very recent version, then this might be the reason.

I also see, that you are running:
questlist <- suso_getQuestDetails()
Did this work? Because if yes, then it is clearly the new API Syntax, which currently primarily affects the export.

In any case, i will update the package during the next days, and let you know. Simultaneously i will also check your specific error.

One more thing: Are you using the command inside a shiny application? I have seen, that your inshinyApp=T.
If you set this outside a shiny environment, it would also throw you an error, immediately after the above one.

1 Like

Thank you both for the support. As of about 24 hours ago, exporting data using both the package and the homemade POST, GET script that I’ve used have worked! I have not changed any of my code so have there been changes to the server that addressed the issue I was having? Some responses to your questions:

No I’m not using it inside a Shiny, I was just playing around with that argument but it didn’t change anything.

Yes, that function worked even before the exporting data mechanism worked.

I’ll have to check on the exact version but I will say that we created the server ~3 weeks ago.

Previously, I never got it to work for our new, non-World Bank hosted server. I’ve been successful using the same export function for pulling data from our older, WB-hosted servers. But as I said above, it now works for all servers.