OwlCyberSecurity - MANAGER
Edit File: upgrade.pyc
� �t�ac @� s d d l m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z d d l m Z m Z m Z m Z d d l m Z d d l m Z d d l m Z d d l! m" Z" d d l# m$ Z$ d d l% m& Z& d d l' m( Z( d d l) m* Z* d d l+ m, Z, m- Z- m. Z. d d l/ m0 Z0 d d l1 m2 Z2 d d l3 m4 Z4 d d l5 m6 Z6 d d l7 m8 Z8 d d l9 m: Z: d d l; m< Z= d d l> m? Z? d d l@ mA ZA mB ZB mC ZC d d lD mE ZE d d lF mG ZG d d lF mH ZH d d lI mJ ZJ mK ZK d d lL mM ZM d d lN m< Z< eG d � ZO d ZP d ZQ eM � ZR d � ZS d! � ZT d" � ZU eA d# � � ZV eA d$ � � ZW eA d% � � ZX eA d& � � ZY eA d' � � ZZ d( � Z[ d) � Z\ d* eX f d+ eZ f g Z] eA d, � � Z^ d- e_ f d. � � YZ` eA d/ � � Za eA d0 � � Zb d d d1 � Zc d2 � Zd d3 je d4 � ef d5 � D� � Zg d6 � Zh d7 � Zi eA d8 � � Zj d9 � Zk d: � Zl eA d; � � Zm d< en f d= � � YZo d> en f d? � � YZp d S(@ i����( t with_statementN( t compress( t loadst UnpicklingError( t parset ParseErrort tostringt Element( t rmdir( t caldavxml( t ProxySqliteDB( t XMLCalendarUserProxyLoader( t" DirectoryCalendarPrincipalResource( t ResourceInfoDatabase( t ELEMENT_AUGMENTS( t Component( t readXMLt writeXMLt addSubElement( t db_basename( t LocalCalendarUser( t migrateTokensToStore( t DirectScheduler( t normalizationLookup( t Delegates( t RecordType( t element( t inlineCallbackst succeedt returnValue( t FilePath( t namedAny( t namedClass( t getRootResourcet FakeRequest( t LoggersD txdav.base.propertystore.xattr.PropertyStore.deadPropertyXattrPrefixs inboxitems.txtt trigger_resource_migrationc C� s t | S( N( t deadPropertyXattrPrefix( t n( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt xattrnameR s c C� s� d } | j rU y t j | j � j } WqU t k rQ t j d d | j �qU Xn d } | j r� y t j | j � j } Wq� t k r� t j d d | j �q� Xn | | f S( Ni����s User not found: {user}t users Group not found: {grp}t grp( t UserNamet pwdt getpwnamt pw_uidt KeyErrort logt errort GroupNameR) t getgrnamt gr_gid( t configt uidt gid( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt getCalendarServerIDsV s c C� s� | j d � d k s? | j d � d k s? | j d � d k r� xJ t r� | j d d � j d d � j d d � } | | k r� PqB | } qB W| t f S| t f Sd S( Ns \"i����s \ "s "t "s "( t findt Truet replacet False( t datat newData( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt fixBadQuotesj s * c c� s< t } t } x�t j | � D]�} | j d � r7 q n t j j | | � } t j j | � rd q n t j d d | �t } t | � �j} | j � } y8 t | � \ } } | r� t j d d | �t } n Wn5 t k r} t j d d | d | �t } w n Xy8 t | � \ } } | r?t j d d | �t } n Wn5 t k rw} t j d d | d | �t } w n Xy? t | | | � V\ } } | r�t j d d | �t } n Wn5 t k r�} t j d d | d | �t } w n XWd QX| r t | d � � } | j | � Wd QXd t j | � j � f } t j | � } y t j | t d � | � Wn1 t k r�} | j t j k r�q�n � n Xt } q q W| r.d t t j j � � f } t j | � } y t j | t d � | � Wq.t k r!} | j t j k r+q+q.� q.Xn t! | � d S( Nt .s Processing: {path}t paths Fixing bad quotes in {path}s- Error while fixing bad quotes in {path}: {ex}t exs% Removing illegal characters in {path}s7 Error while removing illegal characters in {path}: {ex}s Normalized CUAddrs in {path}s$ Error while normalizing {path}: {ex}t ws <?xml version='1.0' encoding='UTF-8'?> <getcontentmd5 xmlns='http://twistedmatrix.com/xml_namespace/dav/'>%s</getcontentmd5> sD {http:%2F%2Ftwistedmatrix.com%2Fxml_namespace%2Fdav%2F}getcontentmd5se <?xml version='1.0' encoding='UTF-8'?> <getctag xmlns='http://calendarserver.org/ns/'>%s</getctag> s. {http:%2F%2Fcalendarserver.org%2Fns%2F}getctag(" R< t ost listdirt startswithRA t joint isdirR/ t debugt opent readR? t warnR: t ExceptionR0 t removeIllegalCharacterst normalizeCUAddrst writet hashlibt md5t hexdigestt zlibR t xattrt setxattrR' t IOErrort errnot EOPNOTSUPPt strt datetimet nowR ( t calPatht directoryt cuaCachet errorOccurredt collectionUpdatedt resourcet resPatht needsRewritet resR= t fixedt et md5valuet ioet ctagValue( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgradeCalendarCollection~ s� c c� s� t } t j d d | �yJxCt j | � D]2} t j j | | � } t j j | � s_ q, n | d k r{ t | � q, n t j d d | �t | | | � Vs� t } n | d k r, yq xj t j | � j � D]S \ } } | t d � k r� t | | � V} | d k r"t j | | | � q"q� q� WWq^t k rQ} | j t j k r[q[q^� q^Xq, q, WWn/ t k r�} t j d d | d | �� n Xt | � d S( Ns Upgrading calendar home: {path}RA t notificationss Upgrading calendar: {path}t inboxs5 {urn:ietf:params:xml:ns:caldav}calendar-free-busy-sets, Failed to upgrade calendar home {path}: {ex}RB ( R< R/ RI RD RE RA RG RH R Rk R: RU t iteritemsR' t updateFreeBusySett NoneRV RW RX RY RM R0 R ( t homePathR^ R_ R` t calR] t attrt valueRi Rg ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgradeCalendarHome� s: " !c $ #� s� g � d � f d � } d d d � } d d d � } i } | j } t | � \ } } t j j | j � s� t | j d | d | �n t j j | � r�t j j | d � } t j j | � r� | | d | d | �t | � t j d d | �n t j j | d � } t j j | � r�t j j | d � } t j d d | �t j j | � r�x� t j | � D]� } t | � d k r�qhn t j j | | � } t j j | � s�qhn t j j | | d d !| d d !| � } | | | d | d | �qhWn t j | � t j | | | � x*t j d f t j d f t j d f t j d f f D]� \ } } t j j | | � } t j j | � rSx� t j | � D]� } | j | | � V} t j j | | � } | d k r t j j | | j d d !| j d d !| j � } | | | d | d | �q�t | | | | � q�Wt j | � qSqSWd } t � } t j � | j d d d } x|t j | � D]k} t | � d k r�t j j | | � } x;t j | � D]'} t | � d k r�t j j | | � } x� t j | � D]� } t j j | | � } t j j | � s4qn | d 7} t j j | d � } t j j | � rx| t j | � D]h } | j d � sut j j | | � } t j j | � } | | k r�t j | � q�| j! | � ququWqqWq�q�Wq�q�W| rdt j j | j t" � } t# | d � �) }! x | D] }" |! j$ d |" � q-WWd QXt j | | | � n | r�t j d d | d | �d }# x t j | � D]} t | � d k r�t j j | | � } x� t j | � D]� } t | � d k r�t j j | | � } x� t j | � D]� } t j j | | � } t j j | � sFqn t% | | | � Vsc| � n |# d 7}# |# d d k rt j d d |# d | �qqWq�q�Wq�q�Wt j d! � q�q�n � r�t j d" � n d S(# sQ Upconvert data from any calendar server version prior to data format 1. c � s- | d k r t j | � n � j t � d S( N( Rp R/ R0 t appendR: ( t f( R` ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt setError s i����c S� s7 d } d } t j j | j | � } t j j | � r: d St j j | j d | � } t j j | � sk d Syz t j j | j � s� t | j d | d | �n y t j | | � Wn. t k r� t j | | � t j | � n XWn2 t k r} t d | | t | � f � � n Xt j d d | d | �d S( Ns .db.calendaruserproxys proxies.sqlitet principalsR5 R6 s] Upgrade Error: unable to move the old calendar user proxy database at '%s' to '%s' due to %s.s? Moved the calendar user proxy database from '{old}' to '{new}'.t oldt new( RD RA RG t DataRoott existst DocumentRoott makeDirsUserGroupt renamet OSErrort shutilt copy2t removeRM t UpgradeErrorRZ R/ RI ( R4 R5 R6 t oldFilenamet newFilenamet newDbPatht oldDbPathRg ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt doProxyDatabaseMoveUpgrade s. c S� sj t j j | � r+ t d | | f � � n t t j j | j d � � d | d | �t j | | � d S( Nsq Upgrade Error: calendar home is in two places: %s and %s. Please remove one of them and restart calendar server.t /R5 R6 ( RD RA R} R� R t dirnamet rstripR� ( t oldHomet newHomeR5 R6 ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt moveCalendarHome? s (R5 R6 Ry s0 Removed the old principal directory at '{path}'.RA t calendarst __uids__s Moving calendar homes to {path}i i i u usersu groupsu locationsu resourcesi i< i Rm R@ RC s %s Ns+ Processing {total} calendar homes in {path}t totali s* Processed calendar home {count} of {total}t counts Done processing calendar homessK Data upgrade encountered errors but will proceed; see error.log for details(& Rp R~ R7 RD RA R} R| R RG R R/ RI RL RE t lenRH t mkdirt chownR R( t groupt CalRecordTypet locationRb t recordWithShortNameR5 t archivet sett timet MigratedInboxDaysCutoffRF t getmtimeR� t addt INBOX_ITEMSRJ RP Ru ($ R4 R^ Rx R� R� R_ t docRootR5 R6 t oldPrincipalst calRoott uidHomest homeR� R� t recordTypet dirNamet dirPatht shortNamet recordR� t inboxItemst cutoffTimestampt firstt firstPatht secondt secondPathRq t inboxPatht inboxItemt itemPatht timestampt inboxItemsFilet outt itemR� ( ( R` s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgrade_to_1 s� & ) ) "c #� s_ t j | � } t � f d � � } | j | | j � Vt | � } t | | | k f � d S( s Normalize calendar user addresses in data. Cache CUA lookup results for great speed. @param data: the calendar data to convert @type data: C{str} @param directory: the directory service to lookup CUAs with @type directory: L{DirectoryService} @param cuaCache: the dictionary to use as a cache across calls, which is updated as a side-effect @type cuaCache: C{dict} @return: tuple of (converted calendar data, boolean signaling whether there were any changes to the data) c 3� sH | � k r t � | � n t | | | � V} | � | <t | � d S( N( R R ( t cuaddrt recordFunctionR4 t result( R_ ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt lookupFunction� s N( R t fromStringR t normalizeCalendarUserAddressest recordWithCalendarUserAddressRZ R ( R= R^ R_ Rr R� R> ( ( R_ s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRO � s c #� s[ � f d � } d � � � � f d � } | � t � | � V| � rW t d � � n d S( Nc � sx d } d } t j j � j | � } t j j � j | � } t j j | � rt t j j | � rt t j | | � n d S( Ns calendaruserproxy.sqlites proxies.sqlite( RD RA RG R| R} R� ( R� R� R� R� ( R4 ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt renameProxyDB s %c S� s. t j d d | �y� x� t j | � D]� } t j j | | � } t j j | � sY q& n | d k rk q& n t j j t j j | t � � r� q& n t j j | � r& t j d d | �t j | t j j | d t j j | � � � q& q& WWn3 t k r#} t j d d | d | �t t � SXt t � S( Ns Flattening calendar home: {path}RA t dropboxs! Regular collection hidden: {path}s .collection.s, Failed to upgrade calendar home {path}: {ex}RB ( R� ( R/ RI RD RE RA RG RH R} R RL R� t basenameRM R0 R R< R: ( t calHomeRr R] Rg ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt flattenHome s" !: c � s� t } t j d � � j } t j j | � r�t j j | d � } t j j | � r�t j j | � r�t j j | d � } t j j | � r~x� t j | � D]� } t j j | | � } t j j | � s� q� n x� t j | � D]� } t j j | | � } t j j | � sq� n xY t j | � D]H } t j j | | � } t j j | � sWq$n � | � s$t } q$q$Wq� Wq� Wq~q�n | S( s` Make sure calendars inside regular collections are all moved to the top level. s Flattening calendar homesR� R� ( R< R/ RI R~ RD RA R} RG RH RE R: ( R` R� R� R� t path1t uidLevel1t path2t uidLevel2R� R� ( R4 R� ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt flattenHomesL s. $ s. Data upgrade failed, see error.log for details( t migrateAutoScheduleR� ( R4 R^ R� R� ( ( R4 R� s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgrade_to_2 s 0 c C� s� yT | j � �B } y t | � } Wn( t k rL t j d d | j �d SXWd QXWn. t t f k r� t j d d | j �d SX| j � } | j d k r� d Si d d 6d d 6d d 6} t j d � t d � } | j d | j d � � x� | D]� } | j } t d � } | j d | � x� | D]� } | j | j d � } | r*xo | D]d } t | � } | j } y t t j | � � j � } Wn t k r�n X| | _ | j | � qRWq*q*W| j | � q� W| j t | d � � d S( s� Convert the old XML format to the twext.who.xml format @param resourcesFilePath: the file to convert @type resourcesFilePath: L{FilePath} s Cannot parse {path}RA Ns Cannot read {path}t accountss short-nameR5 t guids full-namet names Converting resources.xmlR^ t realmR� t types utf-8( s short-name( R� R5 ( s full-name( RJ t parseXMLt XMLParseErrorR/ R0 RA R� RW t getroott tagt infot XMLElementR� t getRp t textRZ t uuidt UUIDt uppert ValueErrorRv t setContentt etreeToString( t resourcesFilePatht fht etreet accountsNodet tagMapt directoryNodet sourceNodeR� t destNodet sourceFieldNodet tagsR� t destFieldNodeRt ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgradeResourcesXMLt sL c C� s8 y t | j t � \ } } Wn( t k rF t j d d | j �d SXt j d � t } x� | D]� } | j d � } | d k r� | j d � } | d k r� t | d d d �} n | j d k r� d | _ n | j | � t } n | j d � } | d k ra | j | � t } qa qa W| r4t | j | � n d S( s� Convert the old augments XML auto-schedule related elements to the twext.who.xml format @param augmentsFilePath: the file to convert @type augmentsFilePath: L{FilePath} s Cannot parse {path}RA Ns Converting augments.xmls auto-schedules auto-schedule-modeR� t automatict falset nonet enable( R RA R R� R/ R0 R� R< R9 Rp R R� R� R: R ( t augmentsFilePatht _ignore_treet augments_nodet changedt recordNodet autoScheduleElementt autoScheduleModeElementt enableElement( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgradeAugmentsXML� s. i i c c� s t j j | j t � } t j j | � re y t | � VWqe t k ra } t d | f � � qe Xn | j } t j j | � s� t j d d | �d St j j | d � } d } t j j | � rBy1 t | � � } t | j � j � � } Wd QXWqBt k rt j d d | �qBt k r>t j d d | �qBXn t | � \ } } x� t D]� \ } } | | k r[t j d d | �| | | � Vt j d d | �t | d � � } | j t | � � Wd QXt j | | | � q[q[Wt j j | � rt j | � n d S( Ns5 Unable to migrate locations and resources from OD: %ss7 DocumentRoot ({path}) doesn't exist; skipping migrationRA s .calendarserver_versioni s& Cannot open {path}; skipping migrations4 Invalid version number in {path}; skipping migrations Upgrading to version {vers}t verss Upgraded to version {vers}RC ( RD RA RG t ServerRoott TRIGGER_FILER} t migrateFromODRM R� R~ R/ R� RJ t intRK t stripRW R0 R� R7 t upgradeMethodsRL RP RZ R� R� ( R4 R^ t triggerPathRg R� t versionFilePatht onDiskVersiont versionFileR5 R6 t versiont methodt verFile( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt upgradeData� sF " R� c B� s e Z d Z RS( s Generic upgrade error. ( t __name__t __module__t __doc__( ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR� s c c� s� | j d � } | d d k r, t d � n | d } | d } | j | j | � | � V} | d k r� t j d d | �t d � n | j } d | | d f } t | � d S( NR� i R� i s; Can't update free-busy href; {user} is not in the directoryR( t s /calendars/__uids__/%s/%s/i ( t splitR Rp R� t oldNameToRecordTypeR/ R0 R5 ( t hrefR^ t piecesR� R� R� R5 t newHref( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt updateFreeBusyHref s c c� ss y t j | � } Wn t j k r) n Xy t j j | � } | j } WnL t k r� y t | � } Wq� t k r� t j d � t d � q� Xn Xt � } t } xk | j D]` } t | � } t | | � V} | d k r� | j | � q� t } | d k r� | j | � q� q� W| ret j g | D] } t j | � ^ q(� } t | j � � } t | � n t d � d S( Ns Invalid free/busy property valueR ( RT t decompressR0 R t WebDAVDocumentR� t root_elementR� t unpickleR R/ R Rp R� R< t childrenRZ R R� R: R t CalendarFreeBusySett HRefR t toxml( Rt R^ t doct freeBusySett fbsett didUpdateR R t fbhreft property( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRo 1 s: % c C� s� | j d � } | d d k r, d | d <n d } xd | D]\ } | sK q9 n t j j | | � } t j j | � s9 t j | � t j | | | � q9 q9 Wd S( NR� i R ( R RD RA RG R} R� R� ( RA R5 R6 t partst part( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR Z s c C� s t j j | j d � } t j j | � s: t j | � n t j | | | � t j j | � } | } d } t j j | | � } xE t j j | � r� | d 7} d | | f } t j j | | � } q� Wy t j | | � Wn. t k r t j | | � t j | � n Xd S( s� Move srcPath into dataroot/archived, giving the destination a unique (sequentially numbered) name in the case of duplicates. t archivedi i s %s.%dN( RD RA RG R| R} R� R� R� R� R� R� R� R� ( R4 t srcPathR5 R6 t archiveDirt baseNamet newNameR� t destPath( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR� i s" R c c� s' | ] } | d k r t | � Vq d S( i i i N( i i i ( t chr( t .0t i( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pys <genexpr>� s i c C� sN t | � } | j d t � } t | � } | | k r@ | t f S| t f Sd S( s� Remove all characters below ASCII 32 except HTAB, LF and CR Return tuple with the processed data, and a boolean indicating whether the data changed. N( R� t translateRp t DELETECHARSR: R< ( R= t beforeLent afterLen( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRN � s c � s� t j d � d d l m � d d l m } d d l m � d d l m } d d l m � d | f � � � f d � � Y} | | � | � S( Ns! Migrating locations and resourcesi����( R ( t DirectoryService( t ConstantsContainer( t migrateResources( t _CSRecordTypet EnhancedDirectoryServicec � s, e Z � � j � j � j � j f � Z RS( ( R R R( R� R� Rb R� ( ( t BaseRecordTypeR1 R3 ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR4 � s ( R/ RL t twext.who.idirectoryR t twext.who.opendirectoryR0 t twext.who.utilR1 t calendarserver.tools.resourcesR2 t txdav.who.opendirectoryR3 ( R^ t OpenDirectoryServiceR2 R4 ( ( R5 R1 R3 s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR� � s "c c� sz d } i d d 6} t | | j j � } y | | j j � } Wn t j d � n X| rvg } t j j | j t j � } t j j | � rvt j d � t | j � } | j d � } x� | D]{ \ } } | d k r� | j | � V} | d k r<| j | | j | j � � V} | r d n d | _ | j | � q<q� q� W| rW| j | � Vn t j d d t | � �qvn d S( Ns, twistedcaldav.directory.augment.AugmentXMLDBt xmls Could not start augment services Migrating auto-schedule settingss+ select GUID, AUTOSCHEDULE from RESOURCEINFOR� t defaults% Migrated {len} auto-schedule settingsR� ( Rp R t AugmentServiceR� t paramsR/ R0 RD RA RG R| R t dbFilenameR} RL t _db_executet recordWithUIDt getAugmentRecordt recordTypeToOldNameR� t autoScheduleModeRv t addAugmentRecordsR� ( R4 R^ t augmentServicet serviceClasst augmentClasst augmentRecordst dbPatht resourceInfoDatabaset resultsR5 t autoScheduleR� t augmentRecord( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR� � s8 c C� s t | � } | j | � S( N( R t updateProxyDB( t xmlFilet servicet loader( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt loadDelegatesFromXMLintoProxyDB� s c C� s t | � } | j | � S( N( R t updateProxyStore( RQ t storeRS ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt loadDelegatesFromXMLintoStore� s c c� s> t j d � t d � } | j � } | j d d � } x� | j d � VD]� \ } } d | k rf qH n | j d � \ } } | j | � V} | d k r� qH n | j | � V} | d k r� qH n | d k } t j | | | | � VqH W| j � V| j � t j | j � | j d } t j j | � r:t j | � n d S( s� Migrate the old sqlite proxyDB data into the store. Remove the sqlite file afterwards. @param store: the store to migrate into @type store: L{CommonDataStore} s Migrating delegates to the stores proxies.sqlitet labelt migrateDelegatesToStores$ select GROUPNAME, MEMBER from GROUPSt #s calendar-proxy-writes -journalN( R/ RL R t directoryServicet newTransactiont queryR RB Rp R t addDelegatet committ closeRD R� t dbpathRA R} ( RV RR R^ t txnt groupNamet memberUIDt delegatorUIDt groupTypet delegatorRecordt delegateRecordt readWritet journalPath( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRY � s0 t UpgradeFileSystemFormatStepc B� s/ e Z d Z d � Z e d � � Z d � Z RS( s4 Upgrade filesystem from previous versions. c C� s | | _ | | _ d S( s) Initialize the service. N( R4 RV ( t selfR4 RV ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt __init__ s c c� sg | j j j j j } t | j j j j _ t | j | j j � � V| | j j j j _ t d � d S( s� Do the upgrade. Called by C{startService}, but a different method because C{startService} should return C{None}, not a L{Deferred}. @return: a Deferred which fires when the upgrade is complete. N( R4 t Memcachedt Poolst Defaultt ClientEnabledR< R RV R[ R Rp ( Rl t memcacheEnabled( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt doUpgrade" s c C� s | j � S( s# Execute the step. ( Rs ( Rl R� ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt stepWithResult7 s ( R R R Rm R Rs Rt ( ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRk s t PostDBImportStepc B� sD e Z d Z d � Z e d � � Z e d � � Z e d � � Z RS( so Step which runs after database import but before workers are spawned (except memcached will be running at this point) The jobs carried out here are: 1. Populating the group-membership cache 2. Processing non-implicit inbox items 3. Migrate IMIP tokens into the store 4. Migrating delegate assignments into the store c C� s | | _ | | _ | | _ d S( s) Initialize the service. N( RV R4 t doPostImport( Rl RV R4 Rv ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRm K s c c� s] | j rYt j j | j j d � } t j j t j j | j j d � � r� t | j � V| j j rt j j | j j � rt j j | � rt | � j � qnn | j j rt j j | j j � rt j j | � rt j d � t | j j | j � Vt | � j � n | j j t � | j � V| j j t � t | j j | j � Vn d S( Ns proxies-loadeds proxies.sqlites% Loading delegate assignments from XML( Rv RD RA RG R4 R| R} RY RV t ProxyLoadFromFileR t touchR/ RL RW t setMigratingR: t processInboxItemsR< R ( Rl R� t loadDoneFilePath( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRt S s, c c� s� t j j | j j t � } t j j | � r�t j d � t | j | j � } | j � } | j } t � } t | � �1 } x' | D] } | j � } | j | � q� WWd QXzdy$t | � } t | � } t � } x�t | � D]�\ } } t j d d | d d | d | �| j d d � \ } } } } | | k r_t j d d | �| j | � q� n | j | � V} | d k r�t j d d | �| j | � | j | � q� n | j | � V} | d k s�t | t � r t j d d | �| j | � | j | � q� n t | d d � } t | _ t | _ | | _ | _! t" } z]| j# | � V} | d k r�t j d d | �| j | � | j | � w� n | j$ d � V} | d k r�| j � r�| j$ | � V} | d k rm| j � rmd | | f } | | _ | j% | | � y) | j& | | | | | | | | � V} Wq�t' k ri} t j( d d | d | �q�Xq�t j d d | �n t j d d | �| j | � Wd | r�t) | d � r�| j* j+ � n Xq� WWn9 t' k r�} t j( d d | �n t j( d � n XWd | r�t | d d � �, } x" | D] } | j, d | f � q6WWd QXt j- | d | � t j d � t j( d � n t j | � t j d � Xn d S( s� When data is migrated from a non-implicit scheduling server there can be inbox items that clients have not yet processed. This method runs those inbox items through the implicit scheduling mechanism. s Starting inbox item processing.Ns+ Processing {ctr}/{total} inbox item: {item}t ctri R� R� R� i s) Ignored inbox item - uuid ignored: {item}s& Ignored inbox item - no record: {item}s) Ignored inbox item - no principal: {item}t PUTs- Ignored inbox item - no calendar home: {item}Rm s /calendars/__uids__/%s/inbox/%ss* Error processing inbox item: {item} ({ex})RB s( Ignored inbox item - no resource: {item}s% Ignored inbox item - no inbox: {item}t _newStoreTransactions, Exception during inbox item processing: {ex}s/ Unknown exception during inbox item processing.s .tmpRC s %s s% Inbox item processing did not finish.s< Restart calendar service to re-attempt inbox item processings Completed inbox item processing.(. RD RA RG R4 R| R� R} R/ R� R! RV t getDirectoryt principalCollectionR� RJ R� R� t listR� t enumeratet rsplitRI R� RB Rp t principalForRecordt isinstanceR R" R: t noAttendeeRefresht checkedSACLt authnUsert authzUserR< t calendarHomet getChildt _rememberResourcet processInboxItemRM R0 t hasattrR~ t abortRP R� ( Rl t inboxItemsListt rootR^ R� R� t inputR� t itemsToProcesst totalItemst ignoreUUIDsR| t ignoreR� t fileNameR� t principalt requestt txnCommittedR� Rm t inboxItemResourcet uriRg t output( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRz � s� # c c� s{ t j d d | �| j } | } d | f } t | | | | j � � } | j � V} | j � d k r;y& | j d � } t j d d | �Wn t k r� t d � n X| d k r� | j � d } n | j � } | j } | j | � V} t | | � } | f } t | | � } | j | | | d t d t �Vn t j d d | �| j | t | � V| j � Vt t � d S( sm Run an individual inbox item through implicit scheduling and remove the inbox item. s Processing inbox item {item}R� s urn:x-uid:%st METHODs Inbox item method is {method}R t REPLYi t internal_requestR� s" Removing invalid inbox item: {uri}R� N( R/ RI R~ R t scheduleInboxURLt componentForUsert mainTypeRp t propertyValueR� R� R t getAttendeest getOrganizerR� t principalForCalendarUserAddressR t doSchedulingViaPUTR< R: RL t storeRemoveR_ ( Rl R� R^ R� R� Rm R� R� R� Rb t ownerPrincipalt cuat ownert calendarR t originatorR� t originatorPrincipalt recipientst scheduler( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyR� s: ( R R R Rm R Rt Rz R� ( ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyRu > s 0~(q t __future__R RU RD RT RQ R[ R+ R) R� RX R� R� R t cPickleR R R t xml.etree.ElementTreeR R� R R� R R� R R� t txweb2.dav.fileopR t twistedcaldavR t) twistedcaldav.directory.calendaruserproxyR t/ twistedcaldav.directory.calendaruserproxyloaderR t! twistedcaldav.directory.principalR t$ twistedcaldav.directory.resourceinfoR t) twistedcaldav.directory.xmlaugmentsparserR t twistedcaldav.icalR t twistedcaldav.xmlutilR R R t! txdav.caldav.datastore.index_fileR t+ txdav.caldav.datastore.scheduling.cuaddressR t2 txdav.caldav.datastore.scheduling.imip.mailgatewayR t+ txdav.caldav.datastore.scheduling.schedulerR t txdav.caldav.datastore.utilR t txdav.who.delegatesR t txdav.who.idirectoryR R� t txdav.xmlR t twisted.internet.deferR R R t twisted.python.filepathR t twisted.python.reflectR R t calendarserver.tap.utilR! R" t twext.python.logR# R6 R% R� R� R/ R' R7 R? Rk Ru R� RO R� R� R� R R t RuntimeErrorR� R Ro R R� RG t xrangeR- RN R� R� RT RW RY t objectRk Ru ( ( ( s( /opt/cpanel-ccs/twistedcaldav/upgrade.pyt <module> s� " `/�+g 6 0 4 ) + -(