Clean code 2: How to Interface with Python

Valentin DION -

In my previous post Clean code 1: Dependency injection explained simply, I tried to make clear why interfaces are useful.

Interfaces allow for clear boundaries between the different logical bricks of your architecture.

Interfaces allow to treat any given technical implementation as a black box, thanks to dependency injection.

That's cool, we're happy

Yet, here's still a question you may ask:

Python doesn't have Interfaces

Let me first point out that this is not a question.

That said, you have a point. Python does not natively have Interfaces.

But Python, when it comes to classes, objects, initialization, lets us implement whatever behaviour we want. Thanks Python !

What interfaces should do

Interfaces should specify the signatures of public methods to implement.
By convention, a non-public method's name starts with _
Since Python has keyword args, our interface should also specify args name.

So, a method signature is defined by:

  • Name
  • Args name
  • Args types
  • Return type

What interfaces may or may not do

We could choose one of two approaches here:

  • Testing interfaces implementation with tests (we would probably want to use mypy)
  • Enforcing interface implementation at runtime

Both have their merits, which I wont go through in this article. We'll see here how to implement the second, but the first will be explored in future articles.

Quick words on testing:

Testing is great, it enforce functional decomposition avoid esoteric bugs.

I must admit, it took me a while to use tests even in small personnal projects. When you're used to using them (also, with the best practices involved doing so) tests may only become an essential tool you're unconfortable doing without.

If you're using unit tests on a daily basis, you're amazing, I like you, and you can skip those next lines.

If you don't, it's only natural to be skeptical about using them. It takes time. More often than not, you're asking questions which answer is trivial and obvious. You could achieve the same quality of code with no tests.

So I thought before using them. Trust me on this... those are NOT fair points. And every developper on earth either would do or are doing a better and faster job with a correct use of tests.

For today, we'll only use simplistic (and a bit dirty) tests. You won't be surprised to read a future article of this series will talk about TDD. In the mean time,

Dependencies

We're going to need pytest, which unsurprisingly, runs tests.
in a console, run:

mkdir interface_python_tutorial
cd interface_python_tutorial
python3.8 -m pip install pytest
touch interface{,_test}.py

Running tests

Each time I'll talk about running test, it can be achieved by simply running: pytest in the directory we created

Implementation

Let's first write our classes without method bodies
file: interface.py

""" Interfaces for Python """
from typing import Dict


class InterfaceException(Exception):
    pass

class Interface:
    @classmethod
    def __check_implements__(cls) -> None:
        """
        :raises: InterfaceError if implemented methods don't match the interface
        :return:
        """
        
    @classmethod
    def _public_methods_signature(cls) -> Dict["str", Dict["str", type]]:
        """
        :return: A dict which keys are method names and values are __annotations__
        """

Interface methods vs implementation

It's important distinguish two things:

In the class scope of an interface, a function we define won't conceptually be the same as a function defined in an implementation.

In the first case it will represent a contract, in the second it will be an implementation of that contract.

In our Interface base class, we defined a function named _public_methods_signature. Which is a pretty self explanatory name.

Signature of a function --including args name, annotated and return type-- can be obtained thanks to the dunder __annotations__.

So, we want our method _public_methods_signature to return a dict matching the name of every members of the class which are:

  • Methods (in other terms: callable)
  • Public (By convention: which name won't start with _)

Expressed in tests those requirements look something like that
file: interface_test.py

import pytest
from interface import Interface, InterfaceError


class ITest(Interface):
    def public_method1(self): pass
    def public_method2(self, foo): pass
    def public_method3(self, foo: str): pass
    def public_method4(self, foo: str) -> int: pass
    def _protected_method(self): pass
    def __private_method(self): pass
    
def test_Interface_public_methods_signature():
    assert ITest._public_methods_signature() == {
        "public_method1": {},
        "public_method2": {},
        "public_method3": {"foo": str},
        "public_method4": {"foo": str, "return": int},
    }

Now, to pass our test (which should always be the goal of every line of code you write, but in depth TDD is a matter for a future article) this code will do:
file: interface.py

...
    @classmethod
    def _public_methods_signature(cls) -> Dict["str", Dict["str", type]]:
        """
        :return: A dict which keys are method names and values are __annotations__
        """
        return {member_name: member.__annotations__
                for member_name, member
                in cls.__dict__.items()
                if callable(member)
                and not member.__name__.startswith('_')}

Enforcing contract implementation

Now we have everything we need to make proper implementation of interfaces enforced at runtime.

What's next:

  • When an interface is declared, we'll extract keep a dict representing our contract
  • When a class implementing an interface, we'll match defined methods against that contract

at the end of our class Interface we'll happend this:
file interface.py

    ...
    def __init_subclass__(cls):
        if cls.__base__ is Interface:
            cls._interface_contract = cls._get_public_methods_signature(cls)
        else:
            cls.__check_implements__()

Here are a few tests expressing the behaviours we'd like:
file interface_test.py

...

def test_Interface():
    with pytest.raises(InterfaceError):
        class TestImplementation(ITest):
            """Nothing implemented, should raise InterfaceError```
        
    with pytest.raises(InterfaceError):
        class TestImplementation(ITest):
            """public_method3 has an arg with the wrong name"""
            def public_method1(self): pass
            def public_method2(self, foo): pass
            def public_method3(self, bar: str): pass
            def public_method4(self, foo: str) -> int: pass

    with pytest.raises(InterfaceError):
        class TestImplementation(ITest):
            """public_method3 has an arg with the wrong type"""
            def public_method1(self): pass
            def public_method2(self, foo): pass
            def public_method3(self, foo: int): pass
            def public_method4(self, foo: str) -> int: pass
            
    class TestImplementation(ITest):
        """Respects the contract
           Should not raise an InterfaceError"""
        def public_method1(self): pass
        def public_method2(self, foo): pass
        def public_method3(self, foo: str): pass
        def public_method4(self, foo: str) -> int: pass
        def _protected_method(self): pass
        def __private_method(self): pass

With what we've got at this point, the only thing we need to passe our tests is to make the method Interface.__check_implements__.

What it should do is:
for each method contractually defined, if there is no implementation with the right signature: raise InterfaceError

Here it is:
file: interface.py

...
    @classmethod
    def __check_implements__(cls):
        """
        :raises: InterfaceError if implemented methods don't match the interface
        :return:
        """
        implemented_methods = cls._public_methods_signature()
        for method_name, method_signature in cls._interface_contract.items():
            if not method_signature == implemented_methods.get(method_name):
                raise InterfaceError(
                    f"{cls.__name__} should implement the method {method_name}"
                    f" with this signature: {method_signature}"
                )
               
...

All the tests should run just fine

Limitations, pitfals, and future

This was meant to give you the very basics of how to use Interfaces in Python. A basic walkthrough of the most straightforward mechanism for that concept.

Here are some limitations, and trades-off we've made (We'll tackle them... in a future article):

Only handle exact match

Built-in python module typing should be handled, and an interface method looking like def foo(self) -> bar should be able to have for implementation def foo(self) -> subtype_of_bar

Doesn't allow for a class to implement more than one interface

Which is for instance allowed in Java

Interface contracts are represented by dicts

Any structure, even short lived or internal to a class should be wrapped into custom types.