Meta-programming in Python: Higher Order Functions and Decorators part 1

Welcome to the second part of this mini series ‘meta-programming in python’. In the last part we had an introduction to meta-programming. In this part we will be looking at various ways to write function and classes that alters the result of a different piece of code from what it would have otherwise returned, so let’s dive right in!

A higher order function is a function that takes a function as an argument or returns a function. A decorator is a typical example of a higher order function you see. However, not all decorators are higher order functions.

Decorators are “wrappers”, which means that they let you execute code before and after the function they decorate without modifying the function itself.

How Exactly do Decorators differ from Higher Order functions in python

Really it’s in their design pattern. Higher Order functions and Decorators are similar in that both take a functions as arguments, however, there are some differences:

1. A higher order function is capable of taking several functions as arguments whereas decorators are designed to take only a single one
2. A decorator can take a class as an parameter/argument instead of a function which means it modifies the resulting class instance, instead of a function result. The whole premise of a higher order function on the other hand is to accept functions as parameters/arguments or return them as result.
3. A decorator can be created with a class instead of a function: a decorator class — speaking specifically for python on this one — a higher order function cannot, because it has to be a function, hence the name ‘higher order function’.
4. Finally, in python, decorators come with syntax sugar to make using them that much easier, whereas higher order functions do not.

Higher order functions are scarcely new to anyone in python, some good examples of higher order functions are the python builtin functions map, reduce and filter.

map(lambda x: return x+1, [1,2,3,4,5,6])

In the block of code above, we can see map takes in a function — in this case an unnamed function — and an iterable — in this case a list.

Creating a higher order function is as easy as creating a regular function in python.

def compose(f, g, *args, **kwargs):
   return g(f(*args, **kwargs))
# This will be called like a normal python function

Writing a Decorator in Python

Decorators are fairly easy to create in python, albeit there are a few different ways of achieving this with the most popular way of doing things being decorator functions.

# A typical example of a decorator function
def outer(func):
    def inner(*args, **kwargs):
        print("Calling", fn.__name__)
        new_value = func(*args, **kwargs)
        return new_value + '2'
    return inner

This is perhaps the simplest possible decorator you can write. It consist of an outer function — which is what takes the required function as a parameter — and an inner function which is responsible for calling the function passed, getting the result and modifying it to its new form. The outer function then returns the inner function.

One very important thing to note in the line of code above would be args and kwargs specified as arguments in the inner function. This is how the parameters of the function getting decorated get passed into the decorator such that the function can utilize it, in its absence python will throw an error of excess arguments being submitted when you try calling your decorated function.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: inner() takes 0 positional arguments but 1 was given

How to Call a Decorator

Decorators are not called the way typical functions are called, this is because it comprises of nested functions, so each of those functions have to be called individually.

def run(name):
    value = 'run' + name
    print(value)
    return value
get_func = outer(run)
runs_func = get_func('marho onothoja')
runs_func()

One key thing to note in decorators is that it stores the function in memory as a variable before it gets called later on, this is why in calling the outer you do not call the function you pass.

If this all looks rather tedious, it is because it is, luckily python has a little syntax sugar to make life easy for everyone. Using the @ symbol, and the name of the decorator, and defining the target function underneath makes it such that calling the decorated function automatically calls the decorator

# This makes it such that the decorator run by default when you call the function
@outer
def run(name):
    value = 'run' + name
    print(value)
    return value
run('marho') #this will call the decorator function as well.

One last thing to take note of. Higher order functions can also be written the same way decorators/decorator functions are written, and called as such.

#this is a higher order function but not a decorator
def compose(f, g):
    def wrapper(*args, **kwargs):
        return g(f(*args, **kwargs))
    return wrapper

The code snippet above is a higher order function, but not a decorator, hence the ‘@’ syntax will not work with it, as such you would have to call it as you would call a decorator function when you do not make use of the ‘@’ syntax

# running a higher order function which resembles but is not a decorator function
get_func = compose(func_1, func_2)
runs_func = get_func('args_1', 'args_2')
runs_func()

We have looked at decorators a great deal in this post, but there is still more to cover. In the next post we will be looking at some other very important decorator features like class decorators — functions and classes that decorate classes, hence changing their resulting instances— and decorator classes — the alternative to decorator functions, but with the use of python classes instead. See you there!

If you have any questions leave it in the comment section, if you have any topic you would like me write about reach me on twitter @marho219.