Listen

Comparing List Comprehensions vs. Built-In Functions in Python

Image showing Python code that squares all values in a list, using both the built-in map function and a list comprehension.

In Python, this situation often arises when a programmer needs to choose between a functional programming approach, such as the built-in functions map(), filter(), and reduce(), and the more Pythonic list comprehensions.

List Comprehensions

In Python, a list comprehension is a concise method that produces a list based on an already existing list. In simple terms, it’s essentially a one-liner of a for loop with the option to include an if condition at the end. The syntax can be broken down as follows: Syntax of a list comprehension in Python.

Let’s assume we have a list of numbers, called numbers, from which we’d like to take the even ones and square them. Now the boring, old school way would be to do it like this:

squared_numbers = []
for number in range(11):
    if number % 2 == 0:
        squared = number ** 2
        squared_numbers.append(squared)
print(squared_numbers)

With a list comprehension, however, we can do this in a single line of code:

numbers=range(11)
squared_numbers = [i**2 for i in numbers if i % 2 == 0]
print(squared_numbers)

Either way produces the same result, but the list comprehension provides a much clearer and more readable solution as its syntax is literally: “Do this for each value in this list if this condition is met”. Generally, list comprehensions are also faster than regular for loops as they don’t have to look up the list and call its append method on every iteration. Now that we have a pretty good understanding of list comprehensions, let’s move on to see how they fare compared to some of the commonly used, built-in functions such as map(), filter(), and reduce(). This is the paradox-of-choice part I was referring to earlier. Programmers tend to be aware that these various methods exist, but which one to choose? Let’s go through each built-in function at a time and compare them to their Pythonic counterpart: the list comprehension.

Map

If your goal is to apply a transformation function to each item in an iterable (such as a list), the map() function is a good place to start. The syntax is quite straightforward and only requires two input arguments: (1) a transformation function, and (2) an iterable (i.e. your input list). Let’s say we have a list of numbers corresponding to Euros and we’d like to convert them to US Dollars. This can be done as follows:

eur = [1, 2, 3, 4, 5]
usd = list(map(lambda x: x / 0.939276, eur))
print(usd)

Notice that we have to explicitly specify the list() function here, since map() natively returns an iterator — a map object. Also note that map()allows you to use these anonymous, ad-hoc lambda functions that allow you to define a function on the fly.

Comparison with List Comprehension

You may have noticed by now that the exact same task could be done with a list comprehension as well. So let’s see how they compare with respect to readability and performance. Specifically, we will look at three scenarios: (1) list comprehensions, (2) map() with a predefined input function, and (3) map()with an ad-hoc lambda function.

import timeit
# predefined conversion function
def eur_to_usd(x):
    return x / 0.939276
lst = list(range(100))

# list comprehension
print( [i / 0.939276 for i in lst])
print(timeit.timeit( "[i / 0.939276 for i in lst]", globals=globals()))
print()

# map with predefined input function
print( list(map(eur_to_usd, lst)) )
print(timeit.timeit( "list(map(eur_to_usd, lst))", globals=globals()))
print()

# map with lambda function
print(list(map(lambda x: x / 0.939276, lst)) )
print(timeit.timeit( "list(map(lambda x: x / 0.939276, lst))", globals=globals()))

In terms of simplicity and readability, the list comprehension appears to be winning the race here. The programmer’s intention is immediately apparent and there’s no need to include any extra keywords or define additional functions. However, it’s worth noting that for more complex operations, separate transformation functions may need to be defined, which would take away some of the brownie points that list comprehensions generally receive for their readability. With respect to performance, the example above clearly demonstrates that list comprehensions are fastest, followed by map() with a predefined input function, and lastly map() with a lambda function. Here’s the thing about using an ad-hoc lambda function: it gets called for each item in the input list, resulting in a computational overhead due to the creation and destruction of lambda function objects, ultimately leading to degraded performance. Predefined functions, by contrast, are optimized and stored in memory, which leads to more efficient execution.

List comprehensions clearly have the upper hand over map() when it comes to performance. In addition, their syntax is easy to read, typically perceived as more intuitive, and considered more Pythonic than the one of map(), which was derived from functional programming.

Filter

The filter() function allows you to select a subset of your iterable based on a given condition. Similar to map(), it requires two input arguments: (1) a filter function, which is often a lambda function, and (2) an iterable. An example is given below, where we filter out all odd numbers and only keep the even ones:

numbers = [1, 2, 3, 4, 5]
filtered = list(filter(lambda x: x % 2 == 0, numbers))
print(filtered)

Again, similar to map(), we have to explicitly state that we want a list returned, because filter() natively returns an iterator object.

Comparison with List Comprehension

Let’s take a look at the performance difference between the built-in filter() function, again using both a predefined input function and a lambda, and a list comprehension.

import timeit
# predefined filter function
def fil(x):
    if x % 2 == 0:
        return True
    else:
        return False
lst = list(range(100))

# list comprehension
print( [i for i in lst if i % 2 == 0] )
print(timeit.timeit( "[i for i in lst if i % 2 == 0] ", globals=globals()))

# filter with predefined filter function
print( list(filter(fil, lst)) )
print(timeit.timeit( "list(filter(fil, lst))", globals=globals()))

# filter with lambda function
print( list(filter(lambda x: x % 2 == 0, lst)))
print(timeit.timeit( "list(filter(lambda x: x % 2 == 0, lst))", globals=globals()))

In terms of readability, the same can be said as above for map(): list comprehensions are pretty easy to read and don’t require any predefined or ad-hoc functions or extra keywords. However, it has been argued that the use of the function filter() immediately gives away the programmer’s intent of, well, filtering something, perhaps more so than the list comprehension does. This is of course a highly subjective matter and will depend on the individual’s preferences and tastes. Regarding performance, we’re also seeing similar results to the ones obtained for map(). List comprehensions are fastest, followed by filter() with a predefined filter function, and filter() with an ad-hoc lambda function comes in last. Again, this is due to the overhead incurred by lambda functions requiring the creation of a new function object at runtime.

List comprehensions outperform their functional filter()counterpart — by nearly a factor of 2 — and are typically perceived to be more Pythonic as well. Readability, however, is a bit more subjective with this one. While some people prefer the intuitive and Pythonic way of list comprehensions, others prefer using the filter() function as it clearly conveys its functionality and the intention of the programmer.

Reduce

Finally, let’s take a look at reduce(). This build-in function is often used in situations that require the accumulation of a single result over multiple steps. It also takes two input arguments: (1) a reduce function, and (2) an iterable. Let’s look at an example to make its functionality more clear. In this case, we would like to compute the product of a list of integers:

from functools import reduce
integers = [1, 2, 3, 4, 5]
print(reduce(lambda x, y: x * y, integers))

Again, we use a lambda to define our reduce function, which in this case is a simple, rolling multiplication over our list of integers. This results in the following computation being performed: 1 x 2 x 3 x 4 x 5 = 120.

Comparison with List Comprehension

Achieving the same goal with a list comprehension is a little trickier this time and requires some additional steps, such as the initialization of a variable and the use of the walrus operator:

integers = [1, 2, 3, 4, 5]
product = 1
[product := product * num for num in integers]
print(product)

While it’s still possible to obtain the same result with a list comprehension, these additional steps considerably degrade the readability of our code. Furthermore, there are multiple, low-code alternatives now to both methods, such as math.prod():

from math import prod
integers = [1, 2, 3, 4, 5]
print(prod(integers))

Performance-wise, however, there doesn’t seem to be a major difference between the two:

from functools import reduce
import timeit

integers = list(range(1, 101))

# using reduce
print(reduce(lambda x, y: x * y, integers) )
print(timeit.timeit( "reduce(lambda x, y: x * y, integers)", globals=globals()))

# using math.prod
from math import prod
print(prod(integers))
print(timeit.timeit( "prod(integers)", globals=globals()))

In Python, the use of reduce() for rolling computations to value pairs in lists has been slowly declining over the years, mainly due to more efficient and intuitive alternatives such as math.prod(). Both reduce() and list comprehensions do not exactly provide a clear-cut syntax here that would provide the reader with an easy and quick understanding of the code.

Conclusion

While not as prevalent as in other languages, map(), filter(), and — occasionally — reduce() are still used and found in Python-based applications. However, list comprehensions tend to be seen as more Pythonic due to their more intuitive syntax and can, in most situations, replace map() and filter() functions, while also benefiting from a marked performance boost. By contrast, the nature of the reduce() function makes it hard and more cryptic to be replaced by list comprehensions. However, they can be replaced by low-code alternatives such as the math.prod() function, as we have seen above.