Dynamically determine base classes on instantiation

Hi list,

I'm confronted with a strang problem I cannot find a clean solution
for. To me it seems like I need meta-classes. Anyway, I stucked a bit
deeper in that topic and couldn't find a proper solution neither. But,
judge for yourselve.

I want a class that determines on instantiating its base classes
dynamically. Consider the following two use cases

a = Foo(['a', 'list']) # returns an instance that behaves like a list
assert len(a) == 2
assert a[0] == 'a'
assert a == ['a', 'list']
assert isinstance(a, list) # This would be nice, but no must-have

b = Foo({'blah': 8}) # returns an instance that behaves like a dict
assert b['blah'] == 'blah'
assert b == {'blah': 8}
assert isinstance(b, dict) # again, no must-have

a.do_something() # common function to both instances as defined
b.do_something() # in the Foo class


What I'm currently doing something like the following:

class Foo(object):

def __init__(self, obj):
self._obj = obj

def __len__(self):
return len(self._obj)

def __getitem__(self, name):
return self._obj[name]

# …

def do_something(self):
# do something on self._obj
pass

Which seems ugly. Is there a way to provide the functions of `list'
and `dict' in Foo's look-up path without having to write all the
stubs myself?

Regards,
Thomas Bach.

On Wed, 15 Aug 2012 23:17:41 +0200, Thomas Bach wrote:

> Hi list,
>
> I'm confronted with a strang problem I cannot find a clean solution for.
> I want a class that determines on instantiating its base classes
> dynamically. Consider the following two use cases

Some comments:

1) What you show are not "use cases", but "examples". A use-case is a
description of an actual real-world problem that needs to be solved. A
couple of asserts is not a use-case.

2) You stated that you have a "strange problem", but you haven't told us
what that problem is, you went directly to what you think is the
solution: "a class that determines on instantiating its base classes
dynamically".

How about you tell us the problem, and we'll suggest a solution?

I'm pretty sure it isn't going to be what you asked for, because that
goes completely against the fundamentals of object-oriented design.

Consider your two examples:

a = Foo(['a', 'list'])
b = Foo({'blah': 8})

According to your design:

a is a Foo
b is a Foo
therefore a and b are the same type

So far so good: this is perfectly normal object-oriented design.

But you also have

a is a list, but not a dict
b is a dict, but not a list
therefore a and b are different types

So you contradict yourself: at the same time, a and b are both the same
and different types.

So now you see why you shouldn't do what you ask for. Now let me tell you
why you *can't* do what you ask for: Python's classes don't work like
that. You can't set the base classes of an instance individually. All
instances of a class share the same base classes.

I think that the right solution here is not inheritance, but composition
and delegation. You're already on the right track when you give your Foo
instances an attribute _obj and then operate on that, but you are wrong
to focus on inheritance.

Instead, Foo should implement only the shared operations, and everything
else should be delegated to _obj.

Automatic delegation is trivially easy (except see below):

http://code.activestate.com/recipes/52295

This is a *really old* recipe, from ancient days before you could inherit
from built-in types like list, dict etc., so the description of the
problem is no longer accurate. But the technique is still good, with
unfortunately one complication:

If you inherit from builtins, you cannot use automatic delegation on the
magic "double-underscore" (dunder) methods like __eq__, __len__, etc.

See this thread here for one possible solution:

http://www.velocityreviews.com/forum...ython-3-a.html



--
Steven

On Thu, Aug 16, 2012 at 12:16:03AM +0000, Steven D'Aprano wrote:
> Some comments:
>
> 1) What you show are not "use cases", but "examples". A use-case is a
> description of an actual real-world problem that needs to be solved. A
> couple of asserts is not a use-case.

Thanks for the clarification on that one. So, here's the use-case: I'm
querying the crunchbase API which returns JSON data and is rather
poorly documented. I want to create a data model for the companies
listed on Crunchbase in order to be able to put the queried data in a
data-base. As I am too lazy to examine all the data by hand I thought
I automatize this. I thought that it would be nice to be able to pass
a function a parsed JSON object (AFAIK these are lists, dicts,
strings, ints, floats, strs in Python) and it returns me the type of
these objects. For the simple classes (str, int, float) this is quite
trivial: F('foo') should return `str' and F(8) should return `int'.

For a compound object like dict I would like it to return the data
fields with their type. Hence, F({'foo': 8}) should return
{'foo': int}, and given that f = F({'foo': {'bar': 80}}) I would like
f to equal to {'foo': dict}, with the option to query the type of
'foo' via f.foo, where the latter should equal to {'bar': int}. So
far, this is not a complicated case. But, sometimes a data field on
returned data set is simply None. Thus, I want to extract the types from
another data set and merge the two.

So, my question (as far as I can see it, please correct me if I am
wrong) is less of the "How do I achieve this?"-kind, but more of the
"What is a clean design for this?"-kind. My intuitive thought was that
the `merge' function should be a part of the object returned from `F'.

> How about you tell us the problem, and we'll suggest a solution?

I can see your point. On the other hand, by expressing my thoughts you
can at least tell me that these are completely wrong and correct my
way of thinking this way.

> Consider your two examples:
>
> a = Foo(['a', 'list'])
> b = Foo({'blah': 8})
>
> According to your design:
>
> a is a Foo
> b is a Foo

I actually never said that. I simply wanted `a' and `b' to share the
same function (the `merge' function), I thought that the easiest way
to achieve this is by letting them share the same name-space. But, as
you show: …

> therefore a and b are the same type
>
> So far so good: this is perfectly normal object-oriented design.
>
> But you also have
>
> a is a list, but not a dict
> b is a dict, but not a listn
> therefore a and b are different types
>
> So you contradict yourself: at the same time, a and b are both the same
> and different types.

… I already made a mistake on the logical level.

> Instead, Foo should implement only the shared operations, and everything
> else should be delegated to _obj.
>
> If you inherit from builtins, you cannot use automatic delegation on the
> magic "double-underscore" (dunder) methods like __eq__, __len__, etc.
>
> See this thread here for one possible solution:
>
> http://www.velocityreviews.com/forum...ython-3-a.html
>

OK, thanks for the hint. I will see how I'm going to put all this
stuff together.

Regards,
Thomas.

On 16/08/12 14:52:30, Thomas Bach wrote:
> On Thu, Aug 16, 2012 at 12:16:03AM +0000, Steven D'Aprano wrote:
>> > Some comments:
>> >
>> > 1) What you show are not "use cases", but "examples". A use-case is a
>> > description of an actual real-world problem that needs to be solved. A
>> > couple of asserts is not a use-case.
> Thanks for the clarification on that one. So, here's the use-case: I'm
> querying the crunchbase API which returns JSON data and is rather
> poorly documented. I want to create a data model for the companies
> listed on Crunchbase in order to be able to put the queried data in a
> data-base. As I am too lazy to examine all the data by hand I thought
> I automatize this. I thought that it would be nice to be able to pass
> a function a parsed JSON object (AFAIK these are lists, dicts,
> strings, ints, floats, strs in Python) and it returns me the type of
> these objects. For the simple classes (str, int, float) this is quite
> trivial: F('foo') should return `str' and F(8) should return `int'.
>
> For a compound object like dict I would like it to return the data
> fields with their type. Hence, F({'foo': 8}) should return
> {'foo': int}, and given that f = F({'foo': {'bar': 80}}) I would like
> f to equal to {'foo': dict}, with the option to query the type of
> 'foo' via f.foo, where the latter should equal to {'bar': int}. So
> far, this is not a complicated case. But, sometimes a data field on
> returned data set is simply None. Thus, I want to extract the types from
> another data set and merge the two.
>
> So, my question (as far as I can see it, please correct me if I am
> wrong) is less of the "How do I achieve this?"-kind, but more of the
> "What is a clean design for this?"-kind. My intuitive thought was that
> the `merge' function should be a part of the object returned from `F'.

The misunderstanding is that you feel F should return an object with
a 'merge' method and a varying abse type, while Steven and others
think that F should be a function.

Maybe something like:

def F(obj):
if obj is None:
return None
tp = type(obj)
if tp in (bool, int, float, str):
return tp
elif tp is list:
return merge([F(elem) for elem in obj])
elif tp is dict:
return dict((k, F(v)) for k,v in obj.iteritems())
else:
raise ValueError("Unexpected type %s for value %s" %(tp, obj))

def merge(lst):
if None in lst:
not_nones = [elem for elem in lst if elem is not None]
if not_nones:
not_none = not_nones[0]
lst = [not_none if elem is None else elem for elem in lst]
else:
return lst # all elements are None; nothing can be done
types = {}
for elem in lst:
if type(elem) is dict:
for k,v in elem.iteritems():
if v is None:
if k in types:
elem[k] = types[k]
else:
for other in lst:
if (other is not elem
and type(other) is dict
and k in other
and other[k] is not None
):
elem[k] = types[k] = other[k]
break
return lst


The merge logic you have in mind may be different from what I just
made up, but the idea remains: F and merge can be functions.


Hope this helps,

-- HansM