Best Practice: Automatically Testing Web APIs
I wrote a Python program that works with two different APIs to get data from two different services (CKAN and MediaWiki). In particular, there is a Resource class that requests data from the aforementioned services and processes it.
At some point, I came to the conclusion that there is a need for tests for my application. And the problem is that all the examples I found on the internet and in books are irrelevant to such cases.
For example, inside a resource class I have a method:
def load_from_ckan(self):
"""
Get the resource
specified by self.id
from config.ckan_api_url
"""
data = json.dumps({'id': self.id})
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
url = config.ckan_api_url + '/action/resource_show'
r = requests.post(url, timeout=config.ckan_request_timeout, data=data, headers=headers)
assert r.ok, r
resource = json.loads(r.content)
resource = resource["result"]
for key in resource:
setattr(self, key, resource[key])
The load_from_ckan method retrieves resource data from the CKAN API and assigns it to an object. It's simple, but ...
My question is, how can I test methods like this? OR What should I be testing here?
I thought about the possibility to sort (save) the results to HDD. Then I could load it in a test and compare against an object initialized with load_from_ckan (). But CKAN is a community-driven platform, and the behavior of such tests will be unpredictable.
If there are any books on the philosophy of automated tests (like what to test, what not to test, how to make tests meaningful, etc.) please give me a link to it.
source to share
In any testing, the key question really is what can go wrong?
In your case, this looks like three risks:
- The web API in question may stop working. But you check this already, p
assert r.ok
. - You, or someone else, may mistakenly modify code in the future (for example, silence a variable) that breaks it.
- The API might change so that it no longer returns the fields or format you want.
It looks like you could write a pretty simple test for the last two, depending on what data from that API you are actually relying on: for example, if you expect JSON to have a field called "temperature" that is a Celsius floating point number , you can write a test that calls your function and then checks that self.temperature is a "float" instance and is within a reasonable range (-30 to 50?). This should leave you confident that both the API and your function work as intended.
source to share
Usually, if you want to test any external service like this, you will need to use a mock / dummy object to spoof the api of the external service. This needs to be configured at run time, either through method arguments or through a class constructor or some other type of indirection. Another more complex option would be to use global patch globals during testing, such as "import requests, request.post = fake_post", but this can create more problems.
So, for example, your method can take an argument like this:
def load_from_ckan(self, post=requests.post):
# ...
r = post(url, timeout=config.ckan_request_timeout, data=data,
headers=headers)
# ...
Then, during testing, you will write your own post function that will return the json results you see when returning from ckan. For example:
def mock_post(url, timeout=30, data='', headers=None):
# ... probably check input arguments
class DummyResponse:
pass
r = DummyResponse()
r.ok = True
r.content = json.dumps({'result': {'attr1': 1, 'attr2': 2}})
return r
Plotting the result in your test gives you more flexibility than the pickle results and returns them because you can create error conditions or focus on certain formats that your code might not expect, but you know they might exist.
In general, you can see how difficult it is to do this, so I would just start adding this kind of testing if you are experiencing repetitive errors or other difficulties. It will just be more code that you have to maintain.
source to share