OwlCyberSecurity - MANAGER
Edit File: memcacheclient.pyc
� �t�ac @ s� d d l m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m Z e � Z y d d l Z Wn e k r� d d l Z n Xy d d l m Z m Z e Z Wn e k r� e Z d � Z n Xy d d l m Z Wn! e k r+d d l m Z n Xd d l m Z e Z d Z d Z d Z d Z d Z d d Z d e! f d � � YZ" d e" f d � � YZ# d e# f d � � YZ$ d e# f d � � YZ% y d d l& m' Z' Wn' e k r�d e( f d � � YZ' n Xd e( f d � � YZ) d e' f d � � YZ* d e* f d � � YZ+ d f d! � � YZ, d" d# � Z- d$ � Z. e/ d% k r�e0 d& � e. � e0 d' � e0 d( g g Z1 d) e j2 k r�e1 j3 e j4 j5 e j6 � d* � g � n xe1 D]�Z7 e* e7 d+ d, �Z8 d- � Z9 d. � Z: d/ f d0 � � YZ; e: d1 d2 � e: d3 d4 � e: d5 e< d, d6 >� � r�e0 d7 d8 d9 �e8 j= d5 � r{e0 d: � n e0 d; � n e0 d< d8 d9 �e0 e8 j> d1 d3 g � � e0 d= d8 d9 �e0 e9 e8 j? d> � � � e; � Z@ e: d? e@ � e0 d@ d8 d9 �e8 jA d3 d, � ZB eB dA k r+e0 d: � n e0 d; � e0 dB d8 d9 �e8 jC d3 d, � ZB eB d4 k rpe0 d: � n e0 d; � e0 dC d8 d9 �y e8 jD dD d, � ZB Wn e* jE k r�ZF e0 d: � n Xe0 d; � e0 dE d8 d9 �y e8 jD dF d, � ZB Wn e* jE k rZF e0 d: � n Xe0 d; � e0 dG d8 d9 �y e8 jD dH e dI d, � ZB Wn e* jG k rpZF e0 d: � n Xe0 d; � e0 dJ d8 d9 �y e8 jD dK d, � ZB Wn& e* jH k r�ZF e0 d: d8 d9 �n Xe0 d; d8 d9 �y# e8 jD dL e jI dM � d, � ZB Wn e0 d; d8 d9 �n Xe0 d: d8 d9 �e jJ dN � ZK y# e8 jD eK e jI dM � d, � ZB Wn e* jG k rye0 d: � n Xe0 d; � e0 dO d8 d9 �e8 jD dP dH e � ZB e8 j? dP � eL k r�e0 d: d8 d9 �n e0 d; d8 d9 �e8 jD dP dH e dQ � ZB e8 j? dP � eL k r e0 d: � n e0 d; � e0 dR d8 d9 �e8 jM � e8 jN i dH dP 6dS dT 6� ZO eO g k rxe0 d; � n e0 d: � e0 dU d8 d9 �e8 jM � e8 jP i dH dP 6dS dT 6� ZQ eQ d, k r�e0 d; � n e0 d: � q�Wn d S(V i����( t print_functionN( t Logger( t config( t compresst decompressc C s t d � � d S( NsF received compressed data but I don't support compession (import error)( t _Error( t val( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR I s ( t StringIO( t crc32s Evan Martin <martine@danga.com>s 1.44s$ Copyright (C) 2003 Danga Interactivet Pythoni� i R c B s e Z RS( ( t __name__t __module__( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR ` s t MemcacheErrorc B s e Z d Z RS( s# Memcache connection error ( R R t __doc__( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR d s t NotFoundErrorc B s e Z d Z RS( s NOT_FOUND error ( R R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR j s t TokenMismatchErrorc B s e Z d Z RS( s& Check-and-set token mismatch ( R R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR p s ( t localR c B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR z s t ClientFactoryc B s5 e Z e Z e d d e j e j d d d � � Z RS( i c C s | j r7 t | d | d | d | d | d | d | �St j j j j rw t | d | d | d | d | d | d | �Sd Sd S( Nt debugt pickleProtocolt picklert unpicklert ploadt pid( t allowTestCachet TestClientR t Memcachedt Poolst Defaultt ClientEnabledt Clientt None( t clst serversR R R R R R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt getClient� s N( R R t FalseR t classmethodt picklet Picklert UnpicklerR R" ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR ~ s R c B s3 e Z d Z d3 Z d4 Z d5 Z d6 Z d Z d e f d � � YZ d e f d � � YZ d e f d � � YZ d e f d � � YZ d e f d � � YZ d e f d � � YZ d d e j e j d7 d7 d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d d d � Z d d � Z d d � Z d d! � Z! d" � Z" d d d# � Z# d d d$ � Z$ d d d% � Z% d d d& � Z& d d d7 d' � Z' d( � Z( d d d d) � Z) d* � Z* d d7 d+ � Z+ d, � Z, d- � Z- d d. � Z. d d/ � Z/ d7 d0 � Z0 d7 d1 � Z1 d2 � Z2 RS(8 s� Object representing a pool of memcache servers. See L{memcache} for an overview. In all cases where a key is used, the key can be either: 1. A simple hashable type (string, integer, etc.). 2. A tuple of C{(hashvalue, key)}. This is useful if you want to avoid making this module calculate a hash value. You may prefer, for example, to keep all of a given user's objects on the same memcache server, so you could use the user's unique id as the hash value. @group Setup: __init__, set_servers, forget_dead_hosts, disconnect_all, debuglog @group Insertion: set, add, replace, set_multi @group Retrieval: get, get_multi @group Integers: incr, decr @group Removal: delete, delete_multi @sort: __init__, set_servers, forget_dead_hosts, disconnect_all, debuglog, set, set_multi, add, replace, get, get_multi, incr, decr, delete, delete_multi i i i i i t MemcachedKeyErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR( � s t MemcachedKeyLengthErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR) � s t MemcachedKeyCharacterErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR* � s t MemcachedKeyNoneErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR+ � s t MemcachedKeyTypeErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR, � s t MemcachedStringEncodingErrorc B s e Z RS( ( R R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR- � s c C s� t j | � | j | � | | _ i | _ | | _ | | _ | | _ | | _ | | _ t � } y% | j | d | j �} t | _ Wn t k r� t | _ n Xd S( s Create a new Client object with the given list of servers. @param servers: C{servers} is passed to L{set_servers}. @param debug: whether to display error messages when a server can't be contacted. @param pickleProtocol: number to mandate protocol used by (c)Pickle. @param pickler: optional override of default Pickler to allow subclassing. @param unpickler: optional override of default Unpickler to allow subclassing. @param pload: optional persistent_load function to call on pickle loading. Useful for cPickle since subclassing isn't allowed. @param pid: optional persistent_id function to call on pickle storing. Useful for cPickle since subclassing isn't allowed. t protocolN( R t __init__t set_serversR t statsR R R t persistent_loadt persistent_idR t Truet picklerIsKeywordt TypeErrorR# ( t selfR! R R R R R R t file( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR/ � s c C s6 g | D] } t | | j � ^ q | _ | j � d S( sd Set the pool of servers used by this client. @param servers: an array of servers. Servers can be passed in two forms: 1. Strings of the form C{"host:port"}, which implies a default weight of 1. 2. Tuples of the form C{("host:port", weight)}, where C{weight} is an integer weight value. N( t _Hostt debuglogR! t _init_buckets( R7 R! t s( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR0 � s (c C s� g } x� | j D]� } | j � s( q n | j t j k rY d | j | j | j f } n d | j | j f } | j d � i } | j | | f � | j } xN | � } | s� | j � d k r� Pn | j d d � } | d | | d <q� Wq W| S( sS Get statistics from each of the servers. @return: A list of tuples ( server_identifier, stats_dictionary ). The dictionary contains a number of name/value pairs specifying the name of the status field and the string value associated with it. The values are not converted from strings. s %s:%s (%s)s unix:%s (%s)R1 t ENDt i i ( R! t connectt familyt sockett AF_INETt ipt portt weightt addresst send_cmdt appendt readlinet stript split( R7 t dataR< t namet serverDataRI t lineR1 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt get_stats� s$ c C s6 g } x)| j D] } | j � s( q n | j t j k rY d | j | j | j f } n d | j | j f } i } | j | | f � | j d � | j } x� | � } | s� | j � d k r� Pn | j d d � } | d j d d � } | d | k ri | | d <n | d | | d | d <q� Wq W| S( Ns %s:%s (%s)s unix:%s (%s)s stats itemsR= R> i i t :( R! R? R@ RA RB RC RD RE RF RH RG RI RJ RK ( R7 RL R< RM RN RI RO t itemt slab( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt get_slabs s* "c C sD x= | j D]2 } | j � s" q n | j d � | j d � q Wd S( s2 Expire all data currently in the memcache servers.t flush_allt OKN( R! R? RG t expect( R7 R< ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRU / s c C s$ | j r t j j d | � n d S( Ns MemCached: %s ( R t syst stderrt write( R7 t str( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR: 7 s c C s6 | | j k r d | j | <n | j | c d 7<d S( Ni ( R1 ( R7 t func( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt _statlog; s c C s! x | j D] } d | _ q Wd S( sC Reset every host in the pool to an "alive" state. i N( R! t deaduntil( R7 R< ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt forget_dead_hostsA s c C sK g | _ x; | j D]0 } x' t | j � D] } | j j | � q) Wq Wd S( N( t bucketsR! t rangeRE RH ( R7 t servert _ignroe_i( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR; H s c C s� t | t � r | \ } } n t | � } xc t t j � D]R } | j | t | j � } | j � rp | | f St t | � t | � � } q: Wt j d � d S( Ns/ Memcacheclient _get_server( ) failed to connect( NN( t isinstancet tuplet serverHashFunctionRa R t _SERVER_RETRIESR` t lenR? R[ t logt errorR ( R7 t keyt serverhasht iRb ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt _get_serverN s c C s" x | j D] } | j � q Wd S( N( R! t close_socket( R7 R< ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt disconnect_all] s t c C s� | j d � | j | | � \ } } g } d } x� | j � D]� } g } | j } | d k r� xL | | D] } | d | | f � qj Wn# x | | D] } | d | � q� Wy | j d j | � � Wq> t j k r } d } t | t � r� | d } n | j | � | j | � q> Xq> Wx | D] } | | =q(Wx� | j � D]v \ } } y" x | D] } | j d � q\WWqFt j k r�} t | t � r�| d } n | j | � d } qFXqFW| S( s� Delete multiple keys in the memcache doing just one query. >>> notset_keys = mc.set_multi({'key1' : 'val1', 'key2' : 'val2'}) >>> mc.get_multi(['key1', 'key2']) == {'key1' : 'val1', 'key2' : 'val2'} 1 >>> mc.delete_multi(['key1', 'key2']) 1 >>> mc.get_multi(['key1', 'key2']) == {} 1 This method is recommended over iterated regular L{delete}s as it reduces total latency, since your app doesn't have to wait for each round-trip of L{delete} before sending the next one. @param keys: An iterable of keys to clear @param time: number of seconds any subsequent set / update commands should fail. Defaults to 0 for no delay. @param key_prefix: Optional string to prepend to each key when sending to memcache. See docs for L{get_multi} and L{set_multi}. @return: 1 if no failure in communication with any memcacheds. @rtype: int t delete_multii s delete %s %d s delete %s Rq i t DELETEDN( R] t _map_and_prefix_keyst iterkeysRH R t send_cmdst joinRA Rj Rd Re t mark_deadt iteritemsRW ( R7 t keyst timet key_prefixt server_keyst _ignore_prefixed_to_orig_keyt dead_serverst rcRb t bigcmdRZ Rk t msg( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRr a s@ c C s� t | � | j | � \ } } | s) d S| j d � | d k rU d | | f } n d | } y | j | � | j d � Wn@ t j k r� } t | t � r� | d } n | j | � d SXd S( s� Deletes a key from the memcache. @return: Nonzero on success. @param time: number of seconds any subsequent set / update commands should fail. Defaults to 0 for no delay. @rtype: int i t deletes delete %s %ds delete %sRs i N( t check_keyRn R] R RG RW RA Rj Rd Re Rx ( R7 Rk R{ Rb t cmdR� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s" c C s | j d | | � S( s4 Sends a command to the server to atomically increment the value for C{key} by C{delta}, or by 1 if C{delta} is unspecified. Returns None if C{key} doesn't exist on server, otherwise it returns the new value after incrementing. Note that the value for C{key} must already exist in the memcache, and it must be the string representation of an integer. >>> mc.set("counter", "20") # returns 1, indicating success 1 >>> mc.incr("counter") 21 >>> mc.incr("counter") 22 Overflow on server is not checked. Be aware of values approaching 2**32. See L{decr}. @param delta: Integer amount to increment by (should be zero or greater). @return: New value after incrementing. @rtype: int t incr( t _incrdecr( R7 Rk t delta( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C s | j d | | � S( sV Like L{incr}, but decrements. Unlike L{incr}, underflow is checked and new values are capped at 0. If server value is 1, a decrement of 2 returns 0, not -1. @param delta: Integer amount to decrement by (should be zero or greater). @return: New value after decrementing. @rtype: int t decr( R� ( R7 Rk R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C s� t | � | j | � \ } } | s) d S| j | � d | | | f } y' | j | � | j � } t | � SWn@ t j k r� } t | t � r� | d } n | j | � d SXd S( Ni s %s %s %di ( R� Rn R] RG RI t intRA Rj Rd Re Rx R ( R7 R� Rk R� Rb RO R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C s | j d | | | | � S( s� Add new key with value. Like L{set}, but only stores in memcache if the key doesn't already exist. @return: Nonzero on success. @rtype: int t add( t _set( R7 Rk R R{ t min_compress_len( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C s | j d | | | | � S( s� Append the value to the end of the existing key's value. Only stores in memcache if key already exists. Also see L{prepend}. @return: Nonzero on success. @rtype: int RH ( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRH s c C s | j d | | | | � S( s� Prepend the value to the beginning of the existing key's value. Only stores in memcache if key already exists. Also see L{append}. @return: Nonzero on success. @rtype: int t prepend( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� s c C s | j d | | | | � S( s� Replace existing key with value. Like L{set}, but only stores in memcache if the key already exists. The opposite of L{add}. @return: Nonzero on success. @rtype: int t replace( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� s c C s | j d | | | | d | �S( sH Unconditionally sets a key to a given value in the memcache. The C{key} can optionally be an tuple, with the first element being the server hash value and the second being the key. If you want to avoid making this module calculate a hash value. You may prefer, for example, to keep all of a given user's objects on the same memcache server, so you could use the user's unique id as the hash value. @return: Nonzero on success. @rtype: int @param time: Tells memcached the time which this value should expire, either as a delta number of seconds, or an absolute unix time-since-the-epoch value. See the memcached protocol docs section "Storage Commands" for more info on <exptime>. We default to 0 == cache forever. @param min_compress_len: The threshold length to kick in auto-compression of the value using the zlib.compress() routine. If the value being cached is a string, then the length of the string is measured, else if the value is an , then the length of the pickle result is measured. If the resulting attempt at compression yeilds a larger string than the input, then it is discarded. For backwards compatability, this parameter defaults to 0, indicating don't ever try to compress. t sett token( R� ( R7 Rk R R{ R� R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� " s c C s t | � } | r t | � n i } i } x� | D]� } t | t � r} t | d � } | j | d | | f � \ } } n% t | � } | j | | � \ } } t | d | �| s� q2 n | | k r� g | | <n | | j | � | | | <q2 W| | f S( s� Compute the mapping of server (_Host instance) -> list of keys to stuff onto that server, as well as the mapping of prefixed key -> original key. i i t key_extra_len( Rh R� Rd Re R[ Rn RH ( R7 t key_iterableR| R� R} t prefixed_to_orig_keyt orig_keyt str_orig_keyRb Rk ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRt < s& & c C s� | j d � | j | j � | � \ } } g } x� | j � D]� } g } | j } yr xU | | D]I } | j | | | | � } | d | | d | | d | d f � qa W| j d j | � � Wq> t j k r} t | t � r� | d } n | j | � | j | � q> Xq> Wx | D] } | | =q W| s@| j � Sg } x� | j � D]� \ } } yD x= | D]5 } | j � } | d k r�qiqi| j | | � qiWWqSt t j f k r�} t | t � r�| d } n | j | � qSXqSW| S( s� Sets multiple keys in the memcache doing just one query. >>> notset_keys = mc.set_multi({'key1' : 'val1', 'key2' : 'val2'}) >>> mc.get_multi(['key1', 'key2']) == {'key1' : 'val1', 'key2' : 'val2'} 1 This method is recommended over regular L{set} as it lowers the number of total packets flying around your network, reducing total latency, since your app doesn't have to wait for each round-trip of L{set} before sending the next one. @param mapping: A dict of key/value pairs to set. @param time: Tells memcached the time which this value should expire, either as a delta number of seconds, or an absolute unix time-since-the-epoch value. See the memcached protocol docs section "Storage Commands" for more info on <exptime>. We default to 0 == cache forever. @param key_prefix: Optional string to prepend to each key when sending to memcache. Allows you to efficiently stuff these keys into a pseudo-namespace in memcache: >>> notset_keys = mc.set_multi({'key1' : 'val1', 'key2' : 'val2'}, key_prefix='subspace_') >>> len(notset_keys) == 0 True >>> mc.get_multi(['subspace_key1', 'subspace_key2']) == {'subspace_key1' : 'val1', 'subspace_key2' : 'val2'} True Causes key 'subspace_key1' and 'subspace_key2' to be set. Useful in conjunction with a higher-level layer which applies namespaces to data in memcache. In this case, the return result would be the list of notset original keys, prefix not applied. @param min_compress_len: The threshold length to kick in auto-compression of the value using the zlib.compress() routine. If the value being cached is a string, then the length of the string is measured, else if the value is an object, then the length of the pickle result is measured. If the resulting attempt at compression yeilds a larger string than the input, then it is discarded. For backwards compatability, this parameter defaults to 0, indicating don't ever try to compress. @return: List of keys which failed to be stored [ memcache out of memory, etc. ]. @rtype: list t set_multis set %s %d %d %d %s i i i Rq t STORED( R] Rt Ru RH t _val_to_store_infoRv Rw RA Rj Rd Re Rx Rz Ry RI R ( R7 t mappingR{ R| R� R} R� R Rb R� RZ Rk t store_infoR� t notstoredRz RO ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� c sB ) - c C sz d } t | t � r n� t | t � rG | t j O} d | } d } n� t | t � rv | t j O} d | } d } n� | t j O} t � } | j r� | j | d | j �} n | j | | j � } | j r� | j | _ n | j | � | j � } t | � } | rQt rQ| | k rQt | � } t | � | k rQ| t j O} | } qQn t | � t k rgd S| t | � | f S( s� Transform val to a storable representation, returning a tuple of the flags, the length of the new value, and the new value itself. i s %dR. ( Rd R[ R� R t _FLAG_INTEGERt longt _FLAG_LONGt _FLAG_PICKLER R5 R R R3 t dumpt getvalueRh t _supports_compressR t _FLAG_COMPRESSEDt SERVER_MAX_VALUE_LENGTH( R7 R R� t flagsR8 R t lvt comp_val( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s: c C s� t | � | j | � \ } } | s4 t d � � n | j | � | j | | � } | s] d S| d k r� d } d | | d | | d | | d f } n( d | | | d | | d | d f } y� | j | � | j d � } | d k r� t S| d k r t | � � n | r:| d k r:t j d � t | � � n t j d d | d | �t SWn? t j k r�} t | t � r�| d } n | j | � n Xd S( Ns Memcache connection errori t cass cas %s %d %d %d %s %si i s %s %s %d %d %d %sR� t NOT_FOUNDt EXISTSs# Memcacheclient check-and-set faileds5 Memcacheclient {cmd} command failed with result ({r})R� t r( R� Rn R R] R� R RG RW R4 R Ri R R Rj R# RA Rd Re Rx ( R7 R� Rk R R{ R� R� Rb R� t fullcmdt resultR� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s: +( c C s� t | � | j | � \ } } | s4 t d � � n | j d � yY | j d | � | j | � \ } } } | sw d S| j | | | � } | j d � WnQ t t j f k r� } t | t � r� | d } n | j | � t d � � n X| S( sP Retrieves a key from the memcache. @return: The value or None. s Memcache connection errort gets get %sR= i N( R� Rn R R] RG t _expectvalueR t _recv_valueRW R RA Rj Rd Re Rx ( R7 Rk Rb t rkeyR� t rlent valueR� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� s$ c C s� t | � | j | � \ } } | s4 t d � � n | j d � y\ | j d | � | j | � \ } } } } | sz d S| j | | | � } | j d � WnQ t t j f k r� } t | t � r� | d } n | j | � t d � � n X| | f S( sP Retrieves a key from the memcache. @return: The value or None. s Memcache connection errorR� s gets %sR= i N( NN( R� Rn R R] RG t _expectvalue_casR R� RW R RA Rj Rd Re Rx ( R7 Rk Rb R� R� R� t cas_tokenR� R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt gets) s$ c C s� | j d � | j | | � \ } } g } x� | j � D]w } y" | j d d j | | � � Wq8 t j k r� } t | t � r� | d } n | j | � | j | � q8 Xq8 Wx | D] } | | =q� Wi } x� | j � D]� } y� | j � } x� | r|| d k r|| j | | � \ } } } | d k rm| j | | | � } y | | | | <Wqmt k riqmXn | j � } q� WWq� t t j f k r�} t | t � r�| d } n | j | � q� Xq� W| S( s2 Retrieves multiple keys from the memcache doing just one query. >>> success = mc.set("foo", "bar") >>> success = mc.set("baz", 42) >>> mc.get_multi(["foo", "baz", "foobar"]) == {"foo": "bar", "baz": 42} 1 >>> mc.set_multi({'k1' : 1, 'k2' : 2}, key_prefix='pfx_') == [] 1 This looks up keys 'pfx_k1', 'pfx_k2', ... . Returned dict will just have unprefixed keys 'k1', 'k2'. >>> mc.get_multi(['k1', 'k2', 'nonexist'], key_prefix='pfx_') == {'k1' : 1, 'k2' : 2} 1 get_mult [ and L{set_multi} ] can take str()-ables like ints / longs as keys too. Such as your db pri key fields. They're rotored through str() before being passed off to memcache, with or without the use of a key_prefix. In this mode, the key_prefix could be a table name, and the key itself a db primary key number. >>> mc.set_multi({42: 'douglass adams', 46 : 'and 2 just ahead of me'}, key_prefix='numkeys_') == [] 1 >>> mc.get_multi([46, 42], key_prefix='numkeys_') == {42: 'douglass adams', 46 : 'and 2 just ahead of me'} 1 This method is recommended over regular L{get} as it lowers the number of total packets flying around your network, reducing total latency, since your app doesn't have to wait for each round-trip of L{get} before sending the next one. See also L{set_multi}. @param keys: An array of keys. @param key_prefix: A string to prefix each key when we communicate with memcache. Facilitates pseudo-namespaces within memcache. Returned dictionary keys will not have this prefix. @return: A dictionary of key/value pairs that were available. If key_prefix was provided, the keys in the retured dictionary will not have it present. t get_multis get %sR> i R= N( R] Rt Ru RG Rw RA Rj Rd Re Rx RH RI R� R R� t KeyErrorR ( R7 Rz R| R} R� R Rb R� t retvalsRO R� R� R� R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� C s> & " c C s� | j d � | j | | � \ } } g } x� | j � D]w } y" | j d d j | | � � Wq8 t j k r� } t | t � r� | d } n | j | � | j | � q8 Xq8 Wx | D] } | | =q� Wi } x� | j � D]� } y� | j � } x� | r�| d k r�| j | | � \ } } } } | d k rv| j | | | � } y | | f | | | <Wqvt k rrqvXn | j � } q� WWq� t t j f k r�} t | t � r�| d } n | j | � q� Xq� W| S( s| Retrieves multiple keys from the memcache doing just one query. See also L{gets} and L{get_multi}. t gets_multis gets %sR> i R= N( R] Rt Ru RG Rw RA Rj Rd Re Rx RH RI R� R R� R� R ( R7 Rz R| R} R� R Rb R� R� RO R� R� R� R� R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s> " c C sj | s | j � } n | d d k rb | j � \ } } } } t | � } t | � } | | | f Sd Sd S( Ni t VALUE( NNN( RI RK R� R ( R7 Rb RO t _ignore_respR� R� Rh R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C sp | s | j � } n | d d k rh | j � \ } } } } } t | � } t | � } | | | | f Sd Sd S( Ni R� ( NNNN( RI RK R� R ( R7 Rb RO R� R� R� Rh t rtokenR� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s c C sv | d 7} | j | � } t | � | k rJ t d t | � | f � � n t | � | k ri | d } n | t j @r� t | � } n | d k s� | t j k r� | } n� | t j @r� t | � } n� | t j @r� t | � } n� | t j @rayC t | � } | j | � } | j r$| j | _ n | j � } Wqrt k r]} | j d | � d } qrXn | j d | � | S( Ni s# received %d bytes when expecting %di����i s Pickle error: %s s unknown flags on get: %x ( t recvRh R R R� R R� R� R� R� R� R R R2 t loadt ExceptionR: R ( R7 Rb R� R� t bufR R8 R t e( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s4 i i i i N(3 R R R R� R� R� R� Rg R� R( R) R* R+ R, R- R% R& R' R R/ R0 RP RT RU R: R] R_ R; Rn Rp Rr R� R� R� R� R� RH R� R� R� Rt R� R� R� R� R� R� R� R� R� R� ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR � s\ # D 'U .) M - R c B s e Z d Z d d e j e j d d d � Z d � Z d � Z d � Z d � Z d d d � Z d d � Z d d � Z d d � Z d d d � Z d d d � Z d d d � Z d d d � Z d d d d � Z d d d d � Z d d d � Z d � Z d � Z d d � Z d d � Z RS( s. Fake memcache client for unit tests i c C s] t j | � t t | � j | d | d | d | d | d | d | �i | _ d | _ d S( NR R R R R R i ( R R/ t superR RL R� ( R7 R! R R R R R R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR/ � s c C s t � � d S( N( t NotImplementedError( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRP s c C s t � � d S( N( R� ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRT s c C s t � � d S( N( R� ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRU s c C s t � � d S( N( R� ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR_ s Rq c C s6 | j d � x" | D] } | | } | j | =q Wd S( sf Delete multiple keys in the memcache doing just one query. >>> notset_keys = mc.set_multi({'key1' : 'val1', 'key2' : 'val2'}) >>> mc.get_multi(['key1', 'key2']) == {'key1' : 'val1', 'key2' : 'val2'} 1 >>> mc.delete_multi(['key1', 'key2']) 1 >>> mc.get_multi(['key1', 'key2']) == {} 1 Rr i ( R] RL ( R7 Rz R{ R| Rk ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRr s c C s t | � | j | =d S( s� Deletes a key from the memcache. @return: Nonzero on success. @param time: number of seconds any subsequent set / update commands should fail. Defaults to 0 for no delay. @rtype: int i ( R� RL ( R7 Rk R{ ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� + s i c C s t � � d S( N( R� ( R7 Rk R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� 6 s c C s t � � d S( N( R� ( R7 Rk R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� 9 s c C s t � � d S( N( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� < s c C s t � � d S( N( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRH ? s c C s t � � d S( N( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� B s c C s t � � d S( N( R� ( R7 Rk R R{ R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� E s c C s, | j d � | j d | | | | d | �S( NR� R� ( R] R� ( R7 Rk R R{ R� R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� H s c C sQ | j d � x= | j � D]/ \ } } | | } | j d | | | | � q Wg S( NR� R� ( R] Ry R� ( R7 R� R{ R| R� Rk R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� L s c C s� t | � | j | � t j | t j � } | d k r{ | | j k r{ | j | \ } } | | k rx t | � � qx q{ n | t | j � f | j | <| j d 7_ t S( Ni ( R� R] R% t dumpst HIGHEST_PROTOCOLR RL R R[ R� R4 ( R7 R� Rk R R{ R� R� t serializedt _ignore_stored_valt stored_token( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� S s c C sP t | � | j d � | | j k rL | j | \ } } t j | � } | Sd S( NR� ( R� R] RL R% t loadsR ( R7 Rk t stored_valt _ignore_stored_tokenR ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� d s c C sI t | � | | j k rE | j | \ } } t j | � } | | f Sd S( N( NN( R� RL R% R� R ( R7 Rk R� R� R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� n s c C sK | j d � i } x1 | D]) } | | } | j | � } | | | <q W| S( NR� ( R] R� ( R7 Rz R| t resultsRk R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� v s c C s^ | j d � i } xD | D]< } | | } | j | � } | d d k r | | | <q q W| S( NR� i ( R] R� R ( R7 Rz R| R� Rk R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s N( R R R R% R& R' R R/ RP RT RU R_ Rr R� R� R� R� RH R� R� R� R� R� R� R� R� R� ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR � s. R9 c B s� e Z d Z d Z d d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z RS( i i c C sN t | t j � r$ | \ } | _ n d | _ t j d | � } | sZ t j d | � } n | su t j d | � } n | s� t d | � � n | j � } | j d � d k r� t j | _ | d | _ nI t j | _ | d | _ t | j d d � � | _ | j | j f | _ | s&d � } n | | _ d | _ d | _ d | _ d S( Ni s ^(?P<proto>unix):(?P<path>.*)$s5 ^(?P<proto>inet):(?P<host>[^:]+)(:(?P<port>[0-9]+))?$s" ^(?P<host>[^:]+):(?P<port>[0-9]+)$s' Unable to parse connection string: "%s"t protot unixt patht hostRD i�+ c S s | S( N( ( t x( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt <lambda>� Rq i Rq ( Rd t typest TupleTypeRE t ret matcht ValueErrort groupdictR� RA t AF_UNIXR@ RF RB RC R� RD R: R^ R t buffer( R7 R� t debugfunct mt hostData( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR/ � s4 c C s/ | j r" | j t j � k r" d Sd | _ d S( Ni i ( R^ R{ ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt _check_dead� s c C s | j � r d Sd S( Ni i ( t _get_socket( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR? � s c C sN t j d d | �| j d | | f � t j � t j | _ | j � d S( Ns' Memcacheclient socket marked dead ({r})R� s MemCache: %s: %s. Marking dead.( Ri Rj R: R{ R9 t _DEAD_RETRYR^ Ro ( R7 t reason( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRx � s c C s% | j � r t j d � d S| j r- | j St j | j t j � } t | d � rg | j | j � n y | j | j � Wn� t j k r� } t j d d | �| j d | � d St j k r} t | t � r� | d } n t j d d | �| j d | � d SX| | _ d | _ | S( Ns. Memcacheclient _get_socket() found dead sockett settimeouts7 Memcacheclient _get_socket() connection timed out ({m})R� s connect: %si s3 Memcacheclient _get_socket() connection error ({m})Rq ( R� Ri Rj R RA R@ t SOCK_STREAMt hasattrR� t _SOCKET_TIMEOUTR? RF t timeoutRx Rd Re R� ( R7 R< R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� � s. c C s& | j r" | j j � d | _ n d S( N( RA t closeR ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRo � s c C s | j j | d � d S( Ns ( RA t sendall( R7 R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRG � s c C s | j j | � d S( s( cmds already has trailing 's applied N( RA R� ( R7 t cmds( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRv � s c C s� | j } | j j } x` t rw | j d � } | d k r= Pn | d � } | sj | j d t | � � Pn | | 7} q W| d k r� | | d | _ | | } n d | _ | S( Ns i i s' Connection closed while reading from %si Rq ( R� RA R� R4 t findRx t repr( R7 R� R� t indexRL ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRI � s c C s6 | j � } | | k r2 | j d | | f � n | S( Ns2 while expecting '%s', got unexpected response '%s'( RI R: ( R7 t textRO ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyRW � s c C s� | j j } | j } xZ t | � | k rq | d � } | | 7} t | � d k r t d t | � | f � q q W| | | _ | | S( Ni i s9 Read %d bytes, expecting %d, read returned 0 length bytes( RA R� R� Rh R ( R7 R� t self_socket_recvR� t foo( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� s c C se d } | j r d | j } n | j t j k rP d | j d | j d | f Sd | j | f Sd S( NRq s (dead until %d)s inet:%s:%d%si i s unix:%s%s( R^ R@ RA RB RF ( R7 t d( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt __str__ s N( R R R� R� R R/ R� R? Rx R� Ro RG Rv RI RW R� R� ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR9 � s $ i c C s� d St | t � r | d } n | s5 t j d � n t | t � rS t j d � n t | t � sq t j d � n t | t � r� t | � | t k r� t j d t � n xD | D]9 } t | � d k s� t | � d k r� t j d � q� q� Wn d S( s� Checks sanity of key. Fails if: Key length is > SERVER_MAX_KEY_LENGTH (Raises MemcachedKeyLength). Contains control characters (Raises MemcachedKeyCharacterError). Is not a string (Raises MemcachedStringEncodingError) Is an unicode string (Raises MemcachedStringEncodingError) Is not a string (Raises MemcachedKeyError) Is None (Raises MemcachedKeyError) Ni s Key is Nones` Keys must be str()'s, not unicode. Convert your unicode strings using mystring.encode(charset)!s Key must be str()'ss Key length is > %si i s Control characters not allowed( Rd Re R R+ t unicodeR- R[ R, t basestringRh t SERVER_MAX_KEY_LENGTHR) t ordR* ( Rk R� t char( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� s$ $c C sW d d l } d d l m } d g } t | d d �} i | d 6} | j | d | �S( Ni����( t memcacheclients 127.0.0.1:11211R i t mct globs( t doctestt twistedcaldavR� R t testmod( R� R� R! R� R� ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt _doctest< s t __main__s Testing docstrings...s Running tests:s 127.0.0.1:11211s --do-unixs memcached.socketR i c C s. t | t j � s& d | t | � f Sd | S( Ns %s (%s)s %s( Rd R� t StringTypest type( R ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt to_sP s c C sq t d t | � t | � f d d �t j | | � t j | � } | | k r_ t d � d St d � d Sd S( Ns Testing set/get {'%s': %s} ...t endRq RV i t FAILi ( t printR R� R� R� ( Rk R t newval( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt test_setgetU s & t FooStructc B s# e Z d � Z d � Z d � Z RS( c C s d | _ d S( Nt baz( t bar( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR/ b s c C s d S( Ns A FooStruct( ( R7 ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR� e s c C s# t | t � r | j | j k Sd S( Ni ( Rd R R ( R7 t other( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt __eq__h s ( R R R/ R� R ( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyR ` s t a_strings some random stringt an_integeri* R� i s Testing delete ...R Rq RV R s Testing get_multi ...s Testing get(unknown value) ...t unknown_valuet foostructs Testing incr ...i+ s Testing decr ...s Testing sending spaces...s this has spacess% Testing sending control characters...s thishascontrol characterss" Testing using insanely long key...t at aaaas' Testing sending a unicode-string key...u keyhereu as utf-8s V\u4f1a p0 .s> Testing using a value larger than the memcached value limit...t keyheret aaas. Testing set_multi() with no memcacheds runningt bt keytheres1 Testing delete_multi() with no memcacheds running(R t __future__R RX RA R{ t osR� R� t twext.python.logR t twistedcaldav.configR Ri t cPickleR% t ImportErrort zlibR R R4 R� R# t cStringIOR t binasciiR Rf t __author__t __version__t __copyright__t __license__R� R� R� R R R R t threadingR t objectR R R R9 R� R� R R t serverListt argvRH R� Rw t getcwdR! R� R R R R� R� R� R� t fR� R� R� R� R* R� R) R- t encodeR� R< R Rp R� t errorsRr t ret( ( ( s/ /opt/cpanel-ccs/twistedcaldav/memcacheclient.pyt <module> s - � � � d�� " ( ##