一半君的总结纸

听话只听一半君

Context Aware Context Decorator inside Maya

python3的contextDecorator虽然可以backport到 python 2,可是在Maya里我们经常遇到嵌套使用context manager或者decorator的情况,此时如果能避免多次建立和销毁context,也许速度能得到提升(当然如过有可能,应该避免嵌套使用)

假设有如下 context manager

from contextlib import contextmanager

@contextmanager
def myContext(i):
    print 'build context', i
    try:
        yield
    finally:
        print 'exiting context',i

def func():
    print 'call func'

with myContext(1):
    print 'outside'
    with myContext(2):
        func()

运行结果是

build context 1
outside
build context 2
call func
exiting context 2
exiting context 1

如上所示 context被创建和销毁了两次, 希望达到的效果是,内层的context manager能知道已经存在自身的context了,不要再次创建


改进方法1

class MyContext(object):
    _instance = None

    def __new__(cls, i):
        if cls._instance is None:
            cls._instance = super(cls, MyContext).__new__(cls, i)
            cls._instance.depth = 0
        return cls._instance

    def __init__(self, i):
        self.i = i

    def __enter__(self):
        self.depth += 1
        if self.depth == 1:
            self.some_expensive_value = expensive_calculation(self.i)
        return self       # or maybe return self.some_expensive_value?

    def __exit__(self, exec_type, exec_value, traceback):
        self.depth -= 1
        if self.depth == 0:
            self.expensive_value.close()   # clean up value if necessary

但是这样觉得不是最好的使用Singleton的方法

改进方法2

from abc import ABCMeta, abstractmethod
from functools import wraps
import time

class Singleton(ABCMeta):
    _instance = None

    def __call__(cls, *args, **kwargs):
        if not hasattr(cls, "_instance") or cls._instance is None:
            cls._instance = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instance


class ContextDecorator(object):

    __metaclass__ = Singleton

    def __init__(self, *args, **kwargs):
        self.args = args
        self.__dict__.update(kwargs)
        self._built = False
        self._contextExists = False

    @abstractmethod
    def _build(self):
        pass

    @abstractmethod
    def _destroy(self):
        pass

    @classmethod
    def clear_singleton(cls):
        cls._instance = None

    def __enter__(self):
        if not self._built:
            self._build()
            self._built = True
            print 'call _build first time'
        else:
            print 'skip _build'
            self._contextExists = True

        return self

    def __exit__(self, typ, val, traceback):
        if not self._contextExists:
            self._destroy()
            self.clear_singleton()
            # self._contextExists=False
            print 'call _destroy first time'
        else:
            print 'skip _destroy'
            self._contextExists = False

    def __call__(self, f):
        self.function = f

        @wraps(f)
        def wrapper(*args, **kw):
            with self:
                try:
                    return f(*args, **kw)
                except:
                    raise
        return wrapper

测试

class CustomContext(ContextDecorator):
    def __init__(self, *args, **kwargs):
        super(CustomContext, self).__init__(*args, **kwargs)

    def _build(self):
        pass

    def _destroy(self):
        pass

print 'context manager test'

with CustomContext():
    for i in range(3):
        with CustomContext():
            time.sleep(0.01)

print '-' * 10
print 'decorator test'

@CustomContext()
@CustomContext()
def test():
    print 'inside test func'

test()

输出结果如下

context manager test
call _build first time
skip _build
skip _destroy
skip _build
skip _destroy
skip _build
skip _destroy
call _destroy first time
----------
decorator test
call _build first time
skip _build
inside test func
skip _destroy
call _destroy first time

可能的改进:

  • 加个参数控制是否要检测已有的context
  • Exception Handling
  • 稍加改进的方法二的gist在这里

  • 但是上面的使用Singleton class的解决方法有个根本性的问题,只能用于简单的context,如果想在context instance 上保存attribute的就不行了,因为用了Singleton, instance只有一份,所以只好不用Singleton了

    改进3

    #!/usr/bin/env python
    
    from abc import ABCMeta, abstractmethod
    from functools import wraps
    import time
    
    
    class ContextDecorator(object):
    
        _built = False
    
        def __init__(self, *args, **kwargs):
            self.args = args
            self.__dict__.update(kwargs)
            self._contextExists = False
            self._rebuild = kwargs.get('rebuild', False)
    
        @abstractmethod
        def _build(self):
            pass
    
        @abstractmethod
        def _destroy(self):
            pass
    
        def clear_built(self):
            ContextDecorator._built = False
    
        def __enter__(self):
    
            if not ContextDecorator._built or self._rebuild:
                try:
                    self._build()
                except:
                    self.clear_built()
                    raise
                ContextDecorator._built = True
                print 'call _build first time'
            else:
                print 'skip _build'
                self._contextExists = True
    
            return self
    
        def __exit__(self, typ, val, traceback):
            if not self._contextExists or self._rebuild:
                try:
                    self._destroy()
                except:
                    self.clear_built()
                    raise
    
                self.clear_built()
                print 'call _destroy first time'
            else:
                print 'skip _destroy'
                self._contextExists = False
    
        def __call__(self, f):
            self.function = f
    
            @wraps(f)
            def wrapper(*args, **kw):
    
                with self:
                    try:
                        return f(*args, **kw)
                    except:
                        self.clear_built()
                        raise
            return wrapper
    
    
    class CustomContext(ContextDecorator):
    
        def __init__(self, *args, **kwargs):
            super(CustomContext, self).__init__(*args, **kwargs)
    
        def _build(self):
            pass
    
        def _destroy(self):
            pass
    
    
    print 'context manager test'
    try:
        with CustomContext():
            for i in range(3):
                with CustomContext():
                    time.sleep(0.01)
    
    except:
        print 'caught exception'
        print CustomContext._built
    
    
    print '-' * 10
    print 'context manager test always rebuild'
    
    try:
        with CustomContext(rebuild=True):
            for i in range(3):
                with CustomContext(rebuild=True):
                    time.sleep(0.01)
    except:
        print 'caught exception'
        print CustomContext._built
    
    print '-' * 10
    print 'decorator test', CustomContext._built, ContextDecorator._built
    # print CustomContext._built
    
    
    @CustomContext()
    @CustomContext()
    def test():
        print 'inside test func'
    
    try:
        test()
    except:
        print 'caught exception', CustomContext._built
    
    
    print '-' * 10
    print 'decorator test rebuild=True', CustomContext._built, ContextDecorator._built
    
    
    @CustomContext(rebuild=True)
    @CustomContext(rebuild=True)
    def test1():
        print 'inside test func'
    
    try:
        test1()
    except:
        print 'caught exception', CustomContext._built
        # raise
    

    测试输出结果

    context manager test
    call _build first time
    skip _build
    skip _destroy
    skip _build
    skip _destroy
    skip _build
    skip _destroy
    call _destroy first time
    ----------
    context manager test always rebuild
    call _build first time
    call _build first time
    call _destroy first time
    call _build first time
    call _destroy first time
    call _build first time
    call _destroy first time
    call _destroy first time
    ----------
    decorator test False False
    call _build first time
    skip _build
    inside test func
    skip _destroy
    call _destroy first time
    ----------
    decorator test rebuild=True False False
    call _build first time
    call _build first time
    inside test func
    call _destroy first time
    call _destroy first time
    [Finished in 0.1s]
    
  • 
    

待续…

参考:
Creating a singleton in Python

Advertisements

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s

%d 博主赞过: