Meta-programming: Introduction

Welcome to the first part of my mini-series on meta-programming. I decided to write this series after learning one of the very intriguing ways of altering python code, carried out on the fly. I found it fascinating and decided to talk about it. Thought I might as well do a run-through of other related concepts.

This mini-series does not cover everything there is to meta-programming. However, it is a great start. So if you are looking to produce more complex better functioning code, you are on the right track.

Alright, what exactly is meta-programming?

Meta-programming is a programming technique. It gives computer programs the ability to treat other programs as data. Essentially, if used, programs can read, generate, analyze or transform other programs. It can even modify itself while running.

In oversimplified terms, it is a technique that allows one block of code to alter others(itself or others).

It is important to note that how this is carried out is largely based on what is available in the language in use.

There are several applications of meta-programming and various ways for achieving it. Different Languages often handle metaprograms… well, differently.

A couple of examples include templating in C++, macros in Rust and Annotations in Java.

Now that we have a basic knowledge of what meta-programming is and what it does. The next question would have to be:

Why should I care about meta-programming?

Well in truth, meta-programming is used everywhere in “human life”. What that means is that for the most part, software used today often has traces of it. Parsers and Compilers exist because of the concept of meta-programming. A good example of this is the python interpreter (CPython). Some of our favourite languages and tools exist because of meta-programming. Tools like ORMs, query-builders, linting tools and compilers.

One key take away is that many languages achieve metaprogramming through reflection. Decorators and metaclasses in Python all hinge on the concept of reflection. And while reflective programming is a subset of meta-programming, it is widely used. Fun fact, even Java has a place for it in its convoluted design.

Reflective Meta-programming refers to a variety of ways a program has knowledge of or can manipulate of itself

There are several ways to accomplish reflective meta-programming. I’ll be covering a handful. This isn’t a completely comprehensive list. However, if you use languages like Python and Javascript you’ll run into them often

1. First Class Functions. The gist here is First-class functions act like any other variable. They can be assigned as a value to a variable. They can be passed as arguments into functions or returned by functions. This is the primary approach to meta-programming by functional languages like Scala. Popular languages like Python and Javascript also use this technique. This approach gives rise to several different techniques: Currying, Closures, etc.
2. Decorators. They are a bit different from first-class functions. They can either be achieved using functions or classes — if the language supports it. For the most part, it’s somewhat language-specific.
3. Meta-classes: A rather dicey style. In this series, this part will be geared toward python. You will have to understand how classes and objects work in Python to best wield this tool. Don’t worry, no need to understand everything Object-Oriented programming to follow. That’s a different rabbit hole for another time.
4. Abstract Syntax Tree transformation. This is the direct alteration of the source code. Using a python AST library is the intriguing way I mentioned earlier in this post. Possibly the least common/used way of achieving meta-programming in python. With this technique, one can even write DSLs(Domain Specific Languages) with python.

As an added treat, I’ll also be dealing with how meta-programming is handled in rust (eventually). Already note though that Rust has meta-programming ingrained into it. Macro is how rust handles meta-programming. And you can scarcely write rust without making use of macros. Even in learning the language, you start by using it (println! is a macro).

Conclusion:

Meta-programming is a concept that commits great power into the hands of engineers. It is a great innovation that has birthed other innovations in the realm of software. But it is also like giving someone a gun. You might get a blown-up foot or brain(in the case of Javascript developers) if you don’t know how to handle it properly.

Because of this, I will be going through each of the methods in subsequent episodes of the series. I will be showing you how to wield this great superpower. Want to learn when and how to wield these powers? Stay tuned!

Follow me to get notified whenever I release a new article on meta-programming or any other topic.