I recently had to solve a problem in a real data system with a nested dict/list combination. I worked on this for quite a while and came up with a solution, but I am very unsatisfied. I had to resort to using globals() and a named temporary global parameter.

I do not like to use globals. That's just asking for an injection vulnerability. I feel that there must be a better way to perform this task without resorting to globals.

Problem Dataset:

d = {"k":1,"stuff":"s1","l":{"m":[{"k":2,"stuff":"s2","l":None},{"k":3,"stuff":"s3","l":{"m":[{"k":4,"stuff":"s4","l":None},{"k":5,"stuff":"s5","l":{"m":[{"k":6,"stuff":"s6","l":None},]}},]}},]}
}

Desired Output:

[{'k': 1, 'stuff': 's1'},{'k': 2, 'stuff': 's2'},{'k': 3, 'stuff': 's3'},{'k': 4, 'stuff': 's4'},{'k': 5, 'stuff': 's5'},{'k': 6, 'stuff': 's6'}]

My Solution:

def _get_recursive_results(d, iter_key, get_keys):if not 'h' in globals():global hh = []h.append({k:d.get(k) for k in get_keys})d2 = d.copy()for k in iter_key:if not d2:continued2 = d2.get(k)for td in d2:d3 = td.copy()for k in iter_key:if not d3:continued3 = d3.get(k)if d3:return _get_recursive_results(td, iter_key, get_keys)h.append({k:td.get(k) for k in get_keys})else:l = [k for k in h]del globals()['h']return l

Calling my function as follows returns the desired result:

_get_recursively(d, ['l','m'], ['k','stuff'])

How would I build a better solution?

This is a slightly modified version without using globals. Set h to None as default and create a new list for the first call to _get_recursive_results(). Later provide h as an argument in the recursive calls to _get_recursive_results():

def _get_recursive_results(d, iter_key, get_keys, h=None):if h is None:h = []h.append({k:d.get(k) for k in get_keys})d2 = d.copy()for k in iter_key:if not d2:continued2 = d2.get(k)for td in d2:d3 = td.copy()for k in iter_key:if not d3:continued3 = d3.get(k)if d3:return _get_recursive_results(td, iter_key, get_keys, h)h.append({k:td.get(k) for k in get_keys})else:l = [k for k in h]return l

Now:

>>> _get_recursive_results(d, ['l','m'], ['k','stuff'])
[{'k': 1, 'stuff': 's1'},{'k': 2, 'stuff': 's2'},{'k': 3, 'stuff': 's3'},{'k': 4, 'stuff': 's4'},{'k': 5, 'stuff': 's5'},{'k': 6, 'stuff': 's6'}]

There is no need for the copying of intermediate dicts. This is a further modified version without copying:

def _get_recursive_results(d, iter_key, get_keys, h=None):if h is None:h = []h.append({k: d.get(k) for k in get_keys})for k in iter_key:if not d:continued = d.get(k)for td in d:d3 = tdfor k in iter_key:if not d3:continued3 = d3.get(k)if d3:return _get_recursive_results(td, iter_key, get_keys, h)h.append({k: td.get(k) for k in get_keys})else:return h