Expanding our capacity to respond to unforeseen changes
Generating tests for interfaces in Python
In yesterday's post, I proposed a way to write and test interfaces in Python. Testing these interfaces was quite verbose. I left refactoring that testing as an exercise to the reader. Then I decided to do that exercise. Here's a neat way to generate interfaces and abstract tests given a dictionary mapping interface names to a list of method names:
# interfaces.py
def Interface(interface_name, method_names):
def interface_helper(*args, **kwargs):
raise NotImplementedError
methods = {method_name: interface_helper for method_name in method_names}
return type(interface_name, (object,), methods)
def AbstractInterfaceTest(test_name, method_names):
def abstract_interface_test_helper(method_name):
def test_method(self):
try:
getattr(self.obj, method_name)()
except NotImplementedError:
self.fail(
type(self.obj).__name__ +
' does not implement ' +
method_name
)
return test_method
methods = {
'test_' + method_name: abstract_interface_test_helper(method_name)
for method_name
in method_names
}
return type(test_name, (object,), methods)
interfaces = {
'CountFish': ['one', 'two'],
'ColorFish': ['red', 'blue']
}
for interface_name, methods in interfaces.iteritems():
interface_name += 'Interface'
globals()[interface_name] = Interface(interface_name, methods)
test_name = 'AbstractTest' + interface_name
globals()[test_name] = AbstractInterfaceTest(test_name, methods)
In order to use this with the other code from yesterday, we'd have to update tests.py as well:
# tests.py
from unittest import TestCase, main
from interfaces import CountFishInterface, ColorFishInterface,\
AbstractTestColorFishInterface, AbstractTestCountFishInterface
from models import OurFish
class TestOurFish(AbstractTestCountFishInterface,
AbstractTestColorFishInterface,
TestCase):
def setUp(self):
self.obj = OurFish()
if __name__ == '__main__':
main()
Implementing interfaces in Python
There's been a long history of proposals and disagreement over interfaces in Python. I'm going to ignore all of that and show one way to utilize interfaces.
An interface can be a class from which implementing classes inherit:
# interfaces.py:
class CountFishInterface(object):
def one(self, *args, **kwargs):
raise NotImplementedError
def two(self, *args, **kwargs):
raise NotImplementedError
class ColorFishInterface(object):
def red(self, *args, **kwargs):
raise NotImplementedError
def blue(self, *args, **kwargs):
raise NotImplementedError
# models.py:
from interfaces import CountFishInterface, ColorFishInterface
class OurFish(CountFishInterface, ColorFishInterface):
pass
Now, OurFish doesn't yet implement the interface. Before we do that, let's add some tests. Note that since multiple classes may implement our interfaces, we make abstract tests for each interface.
# tests.py:
from unittest import TestCase
from interfaces import CountFishInterface, ColorFishInterface
from models import OurFish
class AbstractTestCountFishInterface(object):
def test_one(self):
try:
self.obj.one()
except NotImplementedError:
self.fail(
str(type(self.obj)) + 'does not implement one'
)
def test_two(self):
try:
self.obj.two()
except NotImplementedError:
self.fail(
str(type(self.obj)) + 'does not implement two'
)
class AbstractTestColorFishInterface(object):
def test_red(self):
try:
self.obj.red()
except NotImplementedError:
self.fail(
str(type(self.obj)) + 'does not implement red'
)
def test_blue(self):
try:
self.obj.blue()
except NotImplementedError:
self.fail(
str(type(self.obj)) + 'does not implement blue'
)
class TestOurFish(AbstractTestCountFishInterface,
AbstractTestColorFishInterface,
TestCase):
def setUp(self):
self.obj = OurFish()
Now our tests should fail! Let's implement the interface in OurFish:
# models.py:
from interfaces import CountFishInterface, ColorFishInterface
class OurFish(CountFishInterface, ColorFishInterface):
def one(self):
return 1
def two(self):
return 2
def red(self):
return '#FF0000'
def blue(self):
return '#0000FF'
Now our tests should pass!
Declaring new interfaces and writing the tests is quite verbose. Here's a simpler way of declaring new interfaces:
# interfaces.py
def Interface(interface_name, method_names):
def interface_helper(*args, **kwargs):
raise NotImplementedError
methods = {method_name: interface_helper for method_name in method_names}
return type(interface_name, (object,), methods)
ColorFishInterface = Interface('ColorFishInterface', [
'red',
'blue'
])
That's not that pythonic looking, but here's another way to do it:[^1]
# interfaces.py
interfaces = {
'CountFishInterface': ['one', 'two'],
'ColorFishInterface': ['red', 'blue']
}
for interface_name, methods in interfaces.iteritems():
globals()[interface_name] = Interface(interface_name, methods)
Still messy, but it makes it easy to add more interfaces.
I'll leave refactoring the test cases as an exercise to the reader. Beyond moving the try-block into a helper method, the best solution I can presently come up with is code-generation.
[^1]: Update: No one noticed, but I was originally missing the call to iteritems!
An observation relating primality from number theory to databases
My fiancée is taking a databases course right now, and she pointed out that database normalization is related to prime factorization.
If we think of JOIN as a multiplication operation, and the multiplicative inverse as splitting a table into two tables, the equivalent process to prime factorization would be database normalization!
My limited background in abstract algebra and category theory makes me think this is a field, but do all fields have this property? Chemistry could possibly seen as a field with this property, where the prime factors would be the set of elements in a compound.
Snoopy swearing in Perl
With Perl you can [...] write programs that look like Snoopy swearing.
Providing more implementations for a singular dispatch method
In my earlier singular dispatch post, I suggested that the update method of the singular_dispatch class is quite useful. One place I'm using it is for an implementation of QuickCheck in Python[^1].
[^1]: I'll be posting more on this as things get implemented.
An important aspect of QuickCheck is an arbitrary function which returns a random value of a specified type. Assuming we have a random_int function, we can create an initial arbitrary function which works for int:
def random_int():
return random.randint(-sys.maxint - 1, sys.maxint)
arbitrary = singular_dispatch({
int: random_int,
})
Now, when you want to extend this behavior to new types in Haskell, one would write something like:
instance Arbitrary Bool where
arbitrary = ...
...
Back in Python land, we can add another instance to arbitrary using the update method:
arbitrary.update({
bool: lambda: arbitrary[int]() > 0
})
Calling arbitrary[cls]() isn't very pythonic. The preferred notation would be cls.arbitrary(). This mixin provides that notation for classes which inherit from it[^2]:
[^2]: This mixin does not provide the object-oriented notation for primitive types.
class ArbitraryMixin(object):
@classmethod
def arbitrary(cls):
return arbitrary[cls]()
class Natural(int, ArbitraryMixin):
## Does not actually enforce self >= 0!
pass
class NaturalPlus(int, ArbitraryMixin):
## Does not actually enforce self >= 1!
pass
arbitrary.update({
Natural: lambda: abs(arbitrary[int]())
# Only arbitrary integers >= 0.
NaturalPlus: lambda: Natural.arbitrary() + 1
# Only arbitrary positive integers, and sys.maxint + 1.
})
(Note: for simplicity, we're ignoring the sys.maxint + 1 case. It would not still be an int due to Python casting on overflow.)
This is reminiscent of deriving in Haskell, though nowhere near as powerful.
This even works for complex data structures such as Trees, as long as the structure is defined in arbitrary appropriately. To do so, we create a class combinator which returns a random arbitrary function:
def or_(*args):
return arbitrary[random.choice(args)]
class Tree(ArbitraryMixin):
pass
class Leaf(Tree, ArbitraryMixin):
def __init__(self, value):
self.value = value
class Node(Tree, ArbitraryMixin):
def __init__(self, left, right):
self.left = left
self.right = right
arbitrary.update({
Tree: lambda: or_(Leaf, Node)(),
Leaf: lambda: Leaf(Natural.arbitrary()),
Node: lambda: Node(Tree.arbitrary(), Tree.arbitrary())
})
One benefit of this is Leaf.arbitrary() is guaranteed to be a Leaf and Node.arbitrary() is guaranteed to be a Node but Tree.arbitrary() makes no such guarantees. This is a useful result of subtyping in Python.
Implementing pattern matching and singular dispatch in Python
Pattern matching and singular dispatch are useful tools not readily usable in Python. There is PEP 443, but I'm not a fan for how verbose that is.
We can use dictionaries to do basic pattern matching. Here's an absolute value function:
def abs(x):
abs_dict = {
True: lambda x: x,
False: lambda x: -x
}
return abs_dict[x >= 0](x)
We can use any expression to dispatch the abs_dict. This allows for more complex patterns than just True and False. This function takes an int or a string and returns more:
def more(x):
more_dict = {
int: lambda x: x + 1,
str: lambda x: x + 's'
}
return more_dict[type(x)](x)
more(1) == 2
more('noun') == 'nouns'
This used singular dispatch to choose which implementation to use. I’ve been using the following:
from collections import OrderedDict
class singular_dispatch(OrderedDict):
def __call__(self, *args, **kwargs):
return self[type(args[0])](*args, **kwargs)
Pretend OrderedDict is a normal dict for now. By inheriting from dict, our singular_dispatch objects can function like any other dictionary, and has all of its methods.[^1] We want singular dispatch objects to also function as a function, so we add a __call__ method. The call method uses the type of its first argument (other than self) to dispatch which function within self to use. This assumes that the values in self are functions which operate on the type for their keys.
This simplifies the definition of more:
more = singular_dispatch({
int: lambda x: x + 1,
str: lambda x: x + 's'
})
more(1) == 2
more('noun') == 'nouns'
singular_dispatch objects also allow us to specify an instance to use instead of the type of the first argument. In this case, that would raise a TypeError:
more[int](1) == 2
# more[str](1) raises TypeError
# more[int]('noun') raises TypeError
Inheriting from OrderedDict isn’t strictly necessary in this case, but it is useful for the singular_dispatch class since it remembers the order in which keys are added.
[^1]: Including update. More on that in a different post.
Implementing pattern matching and singular dispatch in Python
Pattern matching and singular dispatch are useful tools not readily usable in Python. There is PEP 443, but I'm not a fan for how verbose that is.
We can use dictionaries to do basic pattern matching. Here's an absolute value function:
def abs(x):
abs_dict = {
True: lambda x: x,
False: lambda x: -x
}
return abs_dict[x >= 0](x)
We can use any expression to dispatch the abs_dict. This allows for more complex patterns than just True and False. This function takes an int or a string and returns more:
def more(x):
more_dict = {
int: lambda x: x + 1,
str: lambda x: x + 's'
}
return more_dict[type(x)](x)
more(1) == 2
more('noun') == 'nouns'
This used singular dispatch to choose which implementation to use. I’ve been using the following:
from collections import OrderedDict
class singular_dispatch(OrderedDict):
def __call__(self, *args, **kwargs):
return self[type(args[0])](*args, **kwargs)
Pretend OrderedDict is a normal dict for now. By inheriting from dict, our singular_dispatch objects can function like any other dictionary, and has all of its methods.[^1] We want singular dispatch objects to also function as a function, so we add a __call__ method. The call method uses the type of its first argument (other than self) to dispatch which function within self to use. This assumes that the values in self are functions which operate on the type for their keys.
This simplifies the definition of more:
more = singular_dispatch({
int: lambda x: x + 1,
str: lambda x: x + 's'
})
more(1) == 2
more('noun') == 'nouns'
singular_dispatch objects also allow us to specify an instance to use instead of the type of the first argument. In this case, that would raise a TypeError:
more[int](1) == 2
# more[str](1) raises TypeError
# more[int]('noun') raises TypeError
Inheriting from OrderedDict isn’t strictly necessary in this case, but it is useful for the singular_dispatch class since it remembers the order in which keys are added.
[^1]: Including update. More on that in a different post.
Computer programs should not blame people for their own failings
Many people don't like using computers because they internalize computer problems when things go wrong. People blame themselves when computers don't behave as expected.
One attempt to fix this would be to provide better error messages. Instead of 'Operation Failed', software should report why the operation failed. Was it a data problem? Network connectivity? A timeout for the server? Invalid response from the server? [^1] By telling people what is wrong, rather than telling them that something went wrong, software can empower people to adjust the necessary conditions to fix the problem.
Another current trend which people internalize is obfuscated UI. When people cannot find a menu because it is hidden off the edge of the screen, or buttons are labeled with abstract icons, they blame themselves.
Software needs to teach people not to blame themselves for things beyond their control.
[^1]: Note: this also makes things easier to debug!
What if there were a startup to find your lost keys?
Losing keys is an ordeal, but thankfully I'm not going through it presently.
What if each set of keys had a unique identifier on it? It turns out, they nearly already do. They keyhole and the teeth of each key provide us with the information we need.
If the average person had a way of cataloging this information, they could get replacements from a locksmith. Perhaps there should be an app which takes pictures of your keys and stores this information for you. There are already methods for reconstructing keys using pictures from afar. I am assuming it would be even easier to get good photos when the photographer and the key owner are the same person.
Reconstructing existing keys may be easier than switching out all of your locks, but since the original keys are still out in the world somewhere, you may need to switch out all of your locks anyway. What if we could get our original keys back?
Perhaps there needs to be a service where you can upload information about a set of keys and your contact information. Then when someone finds your keys, they would upload images of your keys and get your contact information.
In order for someone to know to go to this service, either everyone would need to know about it and use it, or there would need to be a key tag advertising this service, and providing a bar-code identifier for the set of keys. However, a key tag could simply have the phone number or some other contact information, so this service would be unnecessary. Perhaps someone should sell low-quantity key tag printing for contact information.