THE PROBLEM STATEMENT

It is not uncommon to migrate services written in one language to another for performance benefits. While doing so, the consumer of the service should not be affected in any way. How do you ensure that response from the old and the new service is the same irrespective of the programming language driving it?.

Let us see sample JSONs to understand the problem better.

{
    "name" : "Jane",
    "email": "jane@email.com",
    "friends" : ["John", "Mark", "Emily"]
}

{
    "email" : "jane@email.com",
    "name": "Jane",
    "friends": ["Emily", "John", "Mark"]
}

If you observe the 2 JSONs we can make the following observations.

1. the fields name, email and friends are in a different order
2. values in the friends field are also in a different order.
3. both JSONs essentially hold the same information.

Now, coming back to the migration problem, if the response from 2 services is similar to what we see in the above example, how do you verify this programmatically?.

A simple == comparison will fail because the responses are out of order. Also, comparison becomes more complex for a deeply nested response.

SOLUTION

This is where deepdiff comes in handy. Deepdiff is a powerful python library to compare 2 dictionaries. What makes it powerful is that, during the comparison, deepdiff does not consider the order in which the elements inside the dictionaries are present. Hence, solves the problem.

Let’s see deepdiff in action

from deepdiff import DeepDiff
import requests

r1 = requests.get(url="https://old.api/response") # response from old service

r2 = requests.get(url="https://new.api/response") # response from new service

diff = DeepDiff(r1, r2, ignore_order=True) # compare the dictionaries

assert not diff, f"difference in response: {diff}"

Let us understand what code is doing

1. get the response from old and new service
2. use DeepDiff to compare r1 and r2
3. if there is any difference, the diff will not be empty.

There’s a lot more you can do with Deepdiff, refer to the link to know more.