OwlCyberSecurity - MANAGER
Edit File: metafd.pyc
� �t�ac @ s� d Z d d l m Z d d l m Z m Z d d l m Z d d l m Z m Z m Z m Z d d l m Z d d l m Z d d l m Z d d l m Z m Z d d l m Z d d l m Z d d l m Z d d l m Z d d l m Z e � Z d e f d � � YZ! d e e f d � � YZ" d e f d � � YZ# e e � d e e f d � � Y� Z$ e e � d e e f d � � Y� Z% d e f d � � YZ& d S( s� Implementation of dispatching HTTP connections to child processes using L{twext.internet.sendfdport.InheritedSocketDispatcher}. i����( t print_function( t implementert directlyProvides( t ISSLTransport( t InheritedPortt InheritedSocketDispatchert InheritingProtocolFactoryt IStatus( t MaxAcceptTCPServer( t Logger( t HTTPFactory( t MultiServicet Service( t reactor( t FancyStrMixin( t Server( t IStatusWatcher( t MaxAcceptSocketFileServert JustEnoughLikeAPortc B s e Z d Z d Z RS( sV Fake out just enough of L{tcp.Port} to be acceptable to L{tcp.Server}... t inherited( t __name__t __module__t __doc__t _realPortNumber( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR + s t ReportingHTTPServicec B s; e Z d Z d Z e d � Z d � Z d � Z d � Z RS( s� Service which starts up an HTTP server that can report back to its parent process via L{InheritedPort}. This is instantiated in the I{worker process}. @ivar site: a txweb2 'site' object, i.e. a request factory @ivar fd: the file descriptor of a UNIX socket being used to receive connections from a master process calling accept() @type fd: C{int} @ivar contextFactory: A context factory for building SSL/TLS connections for inbound connections tagged with the string 'SSL' as their descriptive data, or None if SSL is not enabled for this server. @type contextFactory: L{twisted.internet.ssl.ContextFactory} or C{NoneType} i c C s( | | _ | | _ | | _ | | _ d S( N( t contextFactoryt sitet fdt usingSocketFile( t selfR R R R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt __init__H s c C se t j | � t | j d t �| _ t | j | j | j � } | j _ | j � | j d � d S( s6 Start reading on the inherited port. t varyt 0N( R t startServicet ReportingHTTPFactoryR t Truet reportingFactoryR R t createTransportt inheritedPortt startReadingt reportStatus( R R& ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR! U s " c C s* t j | � | j j j � | j j � S( s� Stop reading on the inherited port. @return: a Deferred which fires after the last outstanding request is complete. ( R t stopServiceR$ R& t stopReadingt allConnectionsClosed( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR) a s c C s | j d 7_ t | | | t | j t � } | d k rq | j r^ d � | _ t | t � qq | j | j � n | j � | S( sT Create a TCP transport, from a socket object passed by the parent. i t SSLc S s d S( N( t None( t _( ( s /opt/cpanel-ccs/txweb2/metafd.pyt <lambda>{ t ( t _connectionCountR R R R t getPeerCertificateR R t startTLSR R' ( R t sktt peert datat protocolt transport( ( s /opt/cpanel-ccs/txweb2/metafd.pyR% p s ( R R R R1 t FalseR R! R) R% ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR 3 s R" c B s) e Z d Z d � Z d � Z d � Z RS( s� An L{HTTPFactory} which reports its status to a L{InheritedPort<twext.internet.sendfdport.InheritedPort>}. Since this is processing application-level bytes, it is of course instantiated in the I{worker process}, as is L{InheritedPort<twext.internet.sendfdport.InheritedPort>}. @ivar inheritedPort: an L{InheritedPort} to report status (the current number of outstanding connections) to. Since this - the L{ReportingHTTPFactory} - needs to be instantiated to be passed to L{InheritedPort}'s constructor, this attribute must be set afterwards but before any connections have occurred. c C s | j j | � d S( s8 Report a status message to the parent. N( R& R( ( R t message( ( s /opt/cpanel-ccs/txweb2/metafd.pyt _report� s c C s! t j | | � | j d � d S( s� Add the connected channel, and report the current number of open channels to the listening socket in the parent process. t +N( R t addConnectedChannelR; ( R t channel( ( s /opt/cpanel-ccs/txweb2/metafd.pyR= � s c C s! t j | | � | j d � d S( s� Remove the connected channel, and report the current number of open channels to the listening socket in the parent process. t -N( R t removeConnectedChannelR; ( R R> ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR@ � s ( R R R R; R= R@ ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR" � s t WorkerStatusc B s� e Z d Z d j � Z d d d d d d d d d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z RS( s) The status of a worker process. sM acknowledged unacknowledged total started abandoned unclosed starting stoppedi i c C sL | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ d S( s� Create a L{ConnectionStatus} with a number of sent connections and a number of un-acknowledged connections. @param acknowledged: the number of connections which we know the subprocess to be presently processing; i.e. those which have been transmitted to the subprocess. @param unacknowledged: The number of connections which we have sent to the subprocess which have never received a status response (a "C{+}" status message). @param total: The total number of acknowledged connections over the lifetime of this socket. @param started: The number of times this worker has been started. @param abandoned: The number of connections which have been sent to this worker, but were not acknowledged at the moment that the worker was stopped. @param unclosed: The number of sockets which have been sent to the subprocess but not yet closed. @param starting: The process that owns this socket is starting. Do not dispatch to it until we receive the started message. @param stopped: The process that owns this socket has stopped. Do not dispatch to it. N( t acknowledgedt unacknowledgedt totalt startedt abandonedt unclosedt startingt stopped( R RB RC RD RE RF RG RH RI ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR � s ) c C s/ t g | j D] } | t | | � f ^ q � S( N( t dictt showAttributest getattr( R t attr( ( s /opt/cpanel-ccs/txweb2/metafd.pyt items� s c C s | j | j S( s- The current effective load. ( RB RC ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt effective� s c C s | j d k o | j d k S( s� Is the subprocess associated with this socket available to dispatch to. i.e, this socket is neither stopped nor starting i ( RH RI ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt active� s c C s | j d d d d � S( s� The child process for this L{WorkerStatus} is about to (re)start. Reset the status to indicate it is starting - that should prevent any new connections being dispatched. RH i RI i ( t reset( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt start� s c C s | j d | j d d d � S( s� The child process for this L{WorkerStatus} has indicated it is now available to accept connections, so reset the starting status so this socket will be available for dispatch. RE i RH i ( RQ RE ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt restarted� s c C s2 | j d d d d d | j | j d d d d � S( s� The child process for this L{WorkerStatus} has stopped. Stop the socket and clear out existing counters, but track abandoned connections. RB i RC RF RH RI i ( RQ RF RC ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt stop s c K sM xF | j � D]8 \ } } t | | � | } t | | t | d � � q W| S( sg Update the L{WorkerStatus} by adding the supplied values to the specified attributes. i ( RN RL t setattrt max( R t kwargst kt vt newval( ( s /opt/cpanel-ccs/txweb2/metafd.pyt adjust s c K s1 x* | j � D] \ } } t | | | � q W| S( sg Reset the L{WorkerStatus} by setting the supplied values in the specified attributes. ( RN RU ( R RW RX RY ( ( s /opt/cpanel-ccs/txweb2/metafd.pyRQ s ( R R R t splitRK R RN RO RP RR RS RT R[ RQ ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyRA � s$ ) t ConnectionLimiterc B s� e Z d Z d Z d Z d � Z d � Z e d � Z d d � Z d � Z d � Z d � Z d � Z d � Z e d � � Z e d � � Z RS( s� Connection limiter for use with L{InheritedSocketDispatcher}. This depends on statuses being reported by L{ReportingHTTPFactory} i c C sD t j | � g | _ t | � | _ | | _ | | _ t | _ d S( sl Create a L{ConnectionLimiter} with an associated dispatcher and list of factories. N( R R t factoriesR t dispatchert maxAcceptst maxRequestsR9 t overloaded( R R` Ra ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR 3 s c C s$ t t | � j � | j j � d S( sF Start up multiservice, then start up the dispatcher. N( t superR] R! R_ t startDispatching( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR! A s c C sE t | | � } | j j | � | | | d | d | �j | � d S( sY Add a L{MaxAcceptTCPServer} to bind a TCP port to a socket description. t interfacet backlogN( t! LimitingInheritingProtocolFactoryR^ t appendt setServiceParent( R t descriptiont portRe Rf t serverServiceMakert lipf( ( s /opt/cpanel-ccs/txweb2/metafd.pyt addPortServiceH s c C s? t | | � } | j j | � t | | d | �j | � d S( NRf ( Rg R^ Rh R Ri ( R Rj t addressRf Rm ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt addSocketFileServiceU s c C s t � S( s? The status of a new worker added to the pool. ( RA ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt initialStatus^ s c C sX | d k r | j d d � S| d k r2 | j � S| j d d d d d d d d � Sd S( so Determine a subprocess socket's status from its previous status and a status message. R? RB i����R i RC RD RG N( R[ RS ( R t previousStatusR: ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt statusFromMessaged s c C s | j } | | j d | � f S( sS Determine the number of sockets to close from the current status. RG ( RG R[ ( R t statust toClose( ( s /opt/cpanel-ccs/txweb2/metafd.pyt closeCountFromStatus| s c C s | j d d � S( sV A connection was just sent to the process, but not yet acknowledged. RC i ( R[ ( R Rr ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt newConnectionStatus� s c C s� t d � | j j D� � } | | _ t | j | j � | _ | j } | | k } t t d � | j j � � } | p | d k | _ x1 | j D]& } | j r� | j � q� | j � q� Wd S( s� The L{InheritedSocketDispatcher} is reporting that the list of connection-statuses have changed. Check to see if we are overloaded or if there are no active processes left. If so, stop the protocol factory from processing more requests until capacity is back. (The argument to this function is currently duplicated by the C{self.dispatcher.statuses} attribute, which is what C{self.outstandingRequests} uses to compute it.) c s s | ] } | j � Vq d S( N( RO ( t .0Rt ( ( s /opt/cpanel-ccs/txweb2/metafd.pys <genexpr>� s c S s | j � S( N( RP ( t x( ( s /opt/cpanel-ccs/txweb2/metafd.pyR/ � R0 i N( t sumR_ t statusest _outstandingRequestsRV t _maxOutstandingRequestsRa t lent filterRb R^ t loadAboveMaximumt loadNominal( R R{ t currentt maximumRb t availablet f( ( s /opt/cpanel-ccs/txweb2/metafd.pyt statusesChanged� s c C s | j S( N( R| ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt outstandingRequests� s c C s | j } | j | _ | S( s� Reset the max value to the current outstanding value every time the max is read. It is up to the caller to track the overall max value. ( R} R| ( R t temp( ( s /opt/cpanel-ccs/txweb2/metafd.pyt maxOutstandingRequests� s N( R R R R| R} R R! R Rn R- Rp Rq Rs Rv Rw R� t propertyR� R� ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR] ( s Rg c B sA e Z d Z d � Z d � Z d � Z d � Z e d � � Z RS( s� An L{InheritingProtocolFactory} that supports the implicit factory contract required by L{MaxAcceptTCPServer}/L{MaxAcceptTCPPort}. Since L{InheritingProtocolFactory} is instantiated in the I{master process}, so is L{LimitingInheritingProtocolFactory}. @ivar outstandingRequests: a read-only property for the number of currently active connections. @ivar maxAccepts: The maximum number of times to call 'accept()' in a single reactor loop iteration. @ivar maxRequests: The maximum number of concurrent connections to accept at once - note that this is for the I{entire server}, whereas the value in the configuration file is for only a single process. c C sJ t t | � j | j | � | | _ | j | _ | j | _ t | _ d S( N( Rc Rg R R_ t limiterR` Ra R9 t stopping( R R� Rj ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR � s c C s t t | � j � t | _ d S( s� Mark this factory as being stopped to prevent attempts to start reading on its port again when the limiter statuses change during shutdown. N( Rc Rg t stopFactoryR# R� ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR� � s c C s | j j j � d S( sM The current server load has exceeded the maximum allowable. N( t myServert myPortR* ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR� � s c C s | j s | j j j � n d S( s� The current server load is nominal; proceed with reading requests (but only if the server itself is still running). N( R� R� R� R' ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR� � s c C s | j j S( N( R� R� ( R ( ( s /opt/cpanel-ccs/txweb2/metafd.pyR� � s ( R R R R R� R� R� R� R� ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyRg � s N(' R t __future__R t zope.interfaceR R t twisted.internet.interfacesR t twext.internet.sendfdportR R R R t twext.internet.tcpR t twext.python.logR t txweb2.channel.httpR t twisted.application.serviceR R t twisted.internetR t twisted.python.utilR t twisted.internet.tcpR R t twext.internet.socketfileR t logt objectR R R" RA R] Rg ( ( ( s /opt/cpanel-ccs/txweb2/metafd.pyt <module> s, " P' } �