Difficulties in abstraction

Deriving abstractions for a given problem description is always challenging. No matter what your experience level is, a new scenario will always be challenging to model. Sometime back I was talking to a scientist working on crystallography. In crystallography the arrangement of atoms and molecules is determined by the x-ray diffraction pattern produced. I asked the person about the technique used to analyse x-ray diffraction pattern to find out structure of the material under test. The answer startled me. He said, basically we guess what the structure should be and start our analysis with that and iterate till we find out an arrangement which produces given pattern i.e. they use trial and error. In software we saw this problem in deciphering an encrypted message. There is opacity of the structure of the system from observational perspective. Hence designing an abstraction is an inverse problem. An inverse problem in science is the process of calculating from a set of observations the causal factors that produced them.

At this point I would like to differentiate between abstraction and abstraction principle. To me abstraction is core idea of a given problem devoid of it’s representational qualities.

Abstraction as a Inverse Problem

Most of the business requirements we deal with, consists of multiple parts interacting with each other. Often the component interactions appear as property of the system rather than interactions. To give an example, air pressure of a gas is due to collision of atoms. A single atom does not have this property. So for a given business requirement, it is generally difficult to figure out its components and their interactions accurately based on observations about the system.

I thought of a nice analogy of light interference pattern. The interference pattern is a result of intensity of each source, direction in which they are moving and their relative placement. It is a dynamic system. As light sources move or change their intensity changes the interference pattern changes. In some ways this closely matches with software development. We know the end result, we possibly know the participants, but the relative arrangement of each component is always challenging.

So as Naseem Taleb said, it is difficult to reverse engineer a system compared to building one, an inverse problem. In that sense we are trying to reverse engineer the business requirement to find out components and their relative behaviour. So as in the crystallography experiment, we are always going to be tentative when we start and iterate over it. And that’s probably why R-G-R refactoring cycle is so useful.

Abstraction depends on context

When I heard the statement “A lock is a lock because of the key”, it was one of the exhilarating insights I have listened to. To put in perspective, if you happen to lose the key to the lock, the lock is nothing but a dead weight. So when we design abstractions for lock, we are also parallelly designing abstractions for key. So our prior knowledge is playing a role there. We are always designing abstraction in pair most of the time sub-consciously. The abstraction we derive is nothing but interactions between the two components of the coupling. As coupling changes the abstraction should also change. But since only a part of the coupling is exhibited in code, the other part is often left to interpretations and assumptions.

I like to end this post with a reminder that our abstraction is always incomplete. A famous example is scientists trying to recreate prairie from the description of the prairie. They could not recreate it successfully till the time they realise the missing ingredient: Fire.