Fredrik Håård is a partner and specialist at Softhouse Consulting as well as co-founder and head developer at Visual Units, a Swedish company creating support tools for logistics. Fredrik is a DZone MVB and is not an employee of DZone and has posted 21 posts at DZone. You can read more from them at their website. View Full User Profile

How to (Effectively) Explain List Comprehensions to Programmers

02.16.2012
| 7879 views |
  • submit to reddit

For the first year or two programming Python, I never used list comprehensions (at the time, those were the only comprehensions). I read about them, I kinda figured out how they worked, and then I stuck to map() and filter(), which I understood. Looking back, I think that this has a lot to do with the fact that explanations of comprehensions are done using their origin - mathematics - rather than the domain we use them in - programming.

A quick duckduckgo search tells me this is still the case - Wikipedia asks us to consider something like this: S={2⋅x|xϵN|x>3}, and other sources also seem to start out with ‘this is how its done in math, so...’ (a notable exception is the tutorial on python.org).

When talking to programmers, I’d like to explain comprehensions differently, because not all programmers have a background in mathematics. For a programmer, a list comprehension is simply a for loops for constructing lists, using a more declarative notation than your usual for loop. For those of us used to map() and filter(), list comprehensions are both of those as well.

Consider:

def loop(my_list):
  result = []
  for x in my_list:
      if x > 3:
          result.append(x*2)
  return result


Ever written code like this? This is code that explicitly states which steps should be taken to construct your list; but you don’t have to - you can instead state what you want:

def compr(my_list):
  return [x*2 for x in my_list if x > 3]


This translates to give me value*2 for every value in my_list, but only if that value is more than three. Note also that this expression does the work of both map (multiply by two) and filter (take only values that are greater than three). The general case would be [add something to the list for each value in an iterable, optionally only if a condition is True for that value].

Comprehensions also work nested - consider this simple but ugly code:

def create_matrix_loop(size, default):
  new_matrix = []
  for y in range(size):
    row = []
    for x in range(size):
      row.append(default)
    new_matrix.append(row)
return new_matrix


Sample output:

>create_matrix_loop(3, None)
[[None, None, None],
 [None, None, None],
 [None, None, None]]


Since comprehensions can be nested, this can be replaced with:

def create_matrix_compr(size, default):
  return [[default for x in range(size)] for x in range(size)]


As an added bonus, when we don’t tell the compiler how we want to do something, but rather what we want done, it can generate better - faster - bytecode for us. The loop version of create_matrix is translated into 35 bytecode instructions, and the version using a list comprehension is only 20 (try import dis; dis.dis(func) to see what func looks like in bytecode) and in reality, you will often avoid making a function at all when using comprehensions since they’re terse enough on their own, making this difference even bigger. Timing the implementations, the difference is evident:

>timeit -n100 create_matrix_loop(1000, None)
100 loops, best of 3: 113 ms per loop

 

>timeit -n100 create_matrix_compr(1000, None)
100 loops, best of 3: 49.1 ms per loop


That's right: less code, declarative syntax, and faster execution! (Note: I used iPython when creating and timing the examples - it's awesome and you should try it)


Source: http://blaag.haard.se/Explaining-comprehensions-to-programmers/

Published at DZone with permission of Fredrik Håård, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Sc_wizard29 Foo replied on Fri, 2012/02/17 - 10:53am

I'm not a python expert, but are you sure that you were not meaning "take only values that are greater than 3" ?

Alex Martínez replied on Mon, 2012/02/20 - 5:31pm

I'm starting to learn Python and I would probably shoot myself if I find code like that. Seriously, is it really worth it to write code so complicated? Is it better to save that time/bytecode than the readability and sharpness of two nested for loops? hehe

 Thanks for the info anyway, it is always good to learn new ways to write code. 

Kookee Gacho replied on Tue, 2012/05/29 - 9:38pm

The problem is that these files are separated only physically but not logically: a Wicket component and its Html are extremely bounded together and there is no way you are not going to experience an exception stating that a wicket id declared in the html is not present in the java class or vice-versa. -Arthur van der Vant

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.