Monday, August 25, 2014

WLST Library: Work Manager configuration automation in weblogic 12c

Work Manager in weblogic server allows the application to optimize (or) control the execution of its work based on the requirement. Using Work Manager we can setup a dedicated thread pool via the constraints and dictate on how the application requests needs to executed.

Weblogic release 8 and before used different pre-configured Execute Queues like weblogic.kernel.default(Default Queue),weblogic.admin.HTTP/RMI(protocol based) to perform different kinds of work.The later releases of weblogic started using a single self-tuning threading model which is used to execute all kinds of work. Weblogic server will automatically prioritise the work based on the configurations and also the runtime metrics.

The user can create additional Work Manager specific for their applications and provide constraints to dictate how its work needs to be prioritised. By default the total shared capacity for all Work Managers for a server is '65536' requests and they can be configured.


Some of the more common scenarios for using Work Managers are to handle the following :
  • Throttling the number of requests to applications to avoid overloading them.Most of the legacy applications cannot handle huge overload and care should be taken to load the application with only the number of requests that it can handle.
  • Throttle the number of requests to low-priority applications(Notification Service, Logging Service, etc.) to avoid over-servicing them.These applications can be throttled to avoid overloading of servers during the peak time so that critical services can be provided with more resources
A work manager consists of the following components:
  • Request classes - They provide a scheduling guideline on how the threads needs to allocated to the application. By defining the request classes, we can dictate that high-priority applications needs to be serviced before the low-priority ones.
    • Fair-Share - Specifies relative thread usage share required to process requests. Default value is 50
    • Response-Time - The thread usage share required to process requests is determined by the reponse-time goal provided.
    • Context - Assigns request classes to requests based on context information, such as the current user or the current user's group.
  • Constraints - Dictates the minimum, maximum & total capacity threads allocated for a Work Manager to execute its requests.
    • Minimum Threads Constraint - The guaranteed number of threads allocated for the work manager.
    • Maximum Threads Constraint - The maximum number of threads that can be allocated for the work manager.
    • Capacity Constraint - Total capacity of requests that can be executed & queued for servicing. Any requests over the capacity will be rejected.
    In SOA 12c, the invoke/engine & system threads have been replaced through 'Work Manager Groups'. Each Partition will now need to be associated with a Work Manager Group and it dictates how invoke/engine & system threads are allocated to it. The below figure shows the work manager allocation for the 'default' partition. I will write a detailed article about this feature at a later date.


    In this article I will show how to automate creation of Work Managers with constraints through WLST. Lets reuse the previous the library(WlstObject) that we created in our previous article.

    The below figure shows that a new package 'wm' has been created and it contains the module 'workManagerObject.py'


    The below script will allow creation of multiple Work Managers at the same time by providing the required constraints(Request Classes still needs to be implemented).  The script is idempotent in the sense, if the Work Manager & the constraints are already available, then it will just update it otherwise will create it.

    workManagerObject.py
    ###################################################
    #    Name -   workManageObject.py                 #
    #    Author - gkrishna                            #
    #    Version - 1.0                                #
    ###################################################
    
    
    import sys
    from java.io import FileInputStream
    from java.util import Properties 
    from weblogic.descriptor import BeanAlreadyExistsException
    import wlstModule
    import os
    print os.path.abspath("../../../")
    sys.path.append(os.path.abspath("../../../"))
    from base import WlstObject
    
    
    class WorkManagerObject(WlstObject):
        
        _workManagerProperties = Properties()
        _workManagers=None
        
        ###################################################
        #    Method  -__init__                            #
        #    __init__ method for WorkManagerObject        #
        #     class                                       #
        ###################################################
        def __init__(self,commonPropertiesFile,workManagerPropertiesFile):
            try:
                super(WorkManagerObject,self).__init__(commonPropertiesFile)
                self.defaultMBeanHierarchy='/SelfTuning/'+str(self.commonProperties.get("wls.domain.name"))
                self.print_message('__init__ method in WorkManagerObject invoked - Reading property file '+workManagerPropertiesFile+' for Work Manager configurations.')
                propertyInputStream = FileInputStream(workManagerPropertiesFile)
                self._workManagerProperties.load(propertyInputStream)
                
                self.connect()
                self.action()
                self.exit()
            except:
                self.print_message("Error executing __init__ method  - " +str(sys.exc_info()[0]) +" "+ str(sys.exc_info()[1]))
        
        ###################################################
        #    Method  - action                             #
        #    Overriden method to configure the Work       #
        #    Manager.                                     #
        ###################################################
        
        def action(self):
            try:
                workManagers=self._workManagerProperties.get("WorkManagers")
                if(workManagers==''):
                    raise 'No WorkManagers available in the properties file to configure.'
                else:
                    for workManager in workManagers.split():
                        self.name=self._workManagerProperties.get(workManager+('.name'))
                        self.targets=self._workManagerProperties.get(workManager+('.targets'))
                        self.targetType=self._workManagerProperties.get(workManager+('.targetType'))
                        ignoreStuckThread=self._workManagerProperties.get(workManager+('.ignoreStuckThread'))
    
                    if(self.targets=='' or self.targetType==''):
                            raise 'Invalid Target configuration provided for the work manager - '+workManager
                    
                    self.start_edit()
                    ########## Minimum Thread Constraints ##########
                    minThreadsConstraintName=self._workManagerProperties.get(workManager+('.minThreadsConstraint.name'))
                    minThreadsConstraintCount=self._workManagerProperties.get(workManager+('.minThreadsConstraint.count'))
                    if(minThreadsConstraintName!='' and minThreadsConstraintCount!=''):
                        self.print_message('Creating MinThreadConstraint '+str(minThreadsConstraintName) + ' with count '+str(minThreadsConstraintCount))
                        minThreadsConstraint=MinThreadsContraint(minThreadsConstraintName,minThreadsConstraintCount,self.targetType,self.targets,self.defaultMBeanHierarchy)
                        minThreadsConstraint.action()
                        minThreadsConstraintExists=True
                    
                    ########## Maximum Thread Constraints ##########
                    maxThreadsConstraintName=self._workManagerProperties.get(workManager+('.minThreadsConstraint.name'))
                    maxThreadsConstraintCount=self._workManagerProperties.get(workManager+('.minThreadsConstraint.count'))
                    if(maxThreadsConstraintName!='' and maxThreadsConstraintCount!=''):
                        self.print_message('Creating MaxThreadConstraint '+maxThreadsConstraintName + ' with count '+maxThreadsConstraintCount)
                        maxThreadsConstraint=MaxThreadsContraint(maxThreadsConstraintName,maxThreadsConstraintCount,self.targetType,self.targets,self.defaultMBeanHierarchy)
                        maxThreadsConstraint.action()
                        maxThreadsConstraintExists=True
                    
                    ########## Capacity Constraints ##########
                    capacityConstraintName=self._workManagerProperties.get(workManager+('.capacityConstraint.name'))
                    capacityConstraintCount=self._workManagerProperties.get(workManager+('.capacityConstraint.count'))
                    if(capacityConstraintName!='' and capacityConstraintCount!=''):
                        self.print_message('Creating CapacityConstraint '+capacityConstraintName + ' with count '+capacityConstraintCount)
                        capacityConstraint=CapacityConstraint(capacityConstraintName,capacityConstraintCount,self.targetType,self.targets,self.defaultMBeanHierarchy)
                        capacityConstraint.action()
                        capacityConstraintExists=True
                    
                    ########## Work Manager ##########
                    wlstModule.cd(self.defaultMBeanHierarchy)
                    self.print_message('Creating WorkManager '+self.name)
                    wlstModule.cd(self.defaultMBeanHierarchy)
                    
                    try:
                        wlstModule.cmo.createWorkManager(self.name)
                    except BeanAlreadyExistsException:
                        self.print_message(" Work Manager "+self.name+" already exists")
                        pass
                    
                    wlstModule.cd(self.defaultMBeanHierarchy+'/WorkManagers/'+self.name)
                    self.set_targets()
                    if(minThreadsConstraintExists):
                        wlstModule.cmo.setMinThreadsConstraint(wlstModule.getMBean(self.defaultMBeanHierarchy+'/MinThreadsConstraints/'+minThreadsConstraintName))
                    if(maxThreadsConstraintExists):
                        wlstModule.cmo.setMaxThreadsConstraint(wlstModule.getMBean(self.defaultMBeanHierarchy+'/MaxThreadsConstraints/'+maxThreadsConstraintName))
                    if(capacityConstraintExists):
                        wlstModule.cmo.setCapacity(wlstModule.getMBean(self.defaultMBeanHierarchy+'/Capacities/'+capacityConstraintName))
                    if(ignoreStuckThread!=''):
                        wlstModule.cmo.setIgnoreStuckThreads(bool(ignoreStuckThread))
                    self.save_and_activate()
        
            except:
                self.print_message("Error executing create_workmanager method - " +str(sys.exc_info()[0]) +" "+ str(sys.exc_info()[1]))
                self.cancel_and_exit()
        
       
    class MinThreadsContraint(WlstObject):
        _count=None    
        
        ###################################################
        #    Method  -__init__                            #
        #    __init__ method for MinThreadsContraint      #
        #     class                                       #
        ###################################################
        def __init__(self,name,count,targetType,targets,defaultMBeanHierarchy):
            self.name=name
            self._count=count
            self.targetType=targetType
            self.targets=targets
            self.defaultMBeanHierarchy=defaultMBeanHierarchy
        
        ###################################################
        #    Method  - action                             #
        #    Overriden method to configure                #
        #    MinThreadsContraint for the Work             #
            #    Manager                      #
            ###################################################
        def action(self):
            wlstModule.cd(self.defaultMBeanHierarchy)
            try:
                wlstModule.cmo.createMinThreadsConstraint(self.name)
            except BeanAlreadyExistsException:
                self.print_message(" MinThreadsConstraints "+self.name+" already exists")
                pass
    
            wlstModule.cd(self.defaultMBeanHierarchy+'/MinThreadsConstraints/'+self.name)
            wlstModule.cmo.setCount(int(self._count))
            self.set_targets()
            
    class MaxThreadsContraint(WlstObject):
        _count=None
        
        ###################################################
        #    Method  -__init__                #
        #    __init__ method for MaxThreadsContraint   #
        #     class                      #
        ###################################################
        def __init__(self,name,count,targetType,targets,defaultMBeanHierarchy):
            self.name=name
            self._count=count
            self.targetType=targetType
            self.targets=targets
            self.defaultMBeanHierarchy=defaultMBeanHierarchy
    
            ###################################################
            #    Method  - action              #
            #    Overriden method to configure               #
            #    MaxThreadsContraint for the Work      #
            #    Manager                      #
            ###################################################
        def action(self):
            wlstModule.cd(self.defaultMBeanHierarchy)
            try:
                wlstModule.cmo.createMaxThreadsConstraint(self.name)
            except BeanAlreadyExistsException:
                self.print_message(" MaxThreadsConstraints "+self.name+" already exists")
                pass
    
            wlstModule.cd(self.defaultMBeanHierarchy+'/MaxThreadsConstraints/'+self.name)
            wlstModule.cmo.setCount(int(self._count))
            self.set_targets()
            
    class CapacityConstraint(WlstObject):
        _count=None
        
        ###################################################
        #    Method  -__init__                #
        #    __init__ method for CapacityContraint     #
        #     class                      #
        ###################################################
        def __init__(self,name,count,targetType,targets,defaultMBeanHierarchy):
            self.name=name
            self._count=count
            self.targetType=targetType
            self.targets=targets
            self.defaultMBeanHierarchy=defaultMBeanHierarchy
        
        ###################################################
            #    Method  - action              #
            #    Overriden method to configure               #
            #    CapacityContraint for the Work            #
            #    Manager                      #
            ###################################################
        def action(self):
            wlstModule.cd(self.defaultMBeanHierarchy)
            try:
                wlstModule.cmo.createCapacity(self.name)
            except BeanAlreadyExistsException:
                self.print_message(" CapacityConstraint "+self.name+" already exists")
                pass
    
            wlstModule.cd(self.defaultMBeanHierarchy+'/Capacities/'+self.name)
            wlstModule.cmo.setCount(int(self._count))
            self.set_targets()
            
    if __name__=="main":
        w = WorkManagerObject(sys.argv[1],sys.argv[2])
        
    Now we will see how to automate the deployment/creation of the work managers through Maven. With 12c, we have wlst goal in the maven-weblogic-plugin which allows to execute wlst scripts through Maven. In the case of 11g, we would be required to use the exec-maven-plugin to execute the python module using the weblogic.WLST class. Please note that the below code in the module has been added for simplicity to handle the base package import dependency. Ideally while integrating with Maven, we need to use the maven-dependency-plugin to bring the base package into the target folder to handle the dependency. But for now, we put in a relative path reference(hard-coded) to handle dependency.
    print os.path.abspath("../../../")
    sys.path.append(os.path.abspath("../../../"))
    
    Below is the weblogic 12c Maven example to create the Work Manager through Maven. pom.xml
    
     4.0.0
     
      techsamples.fmw
      parent
      1.0
     
     techsamples.fmw
     wm
     pom
     1.0
     wm
     
     
            /XXXXXXXX/wlst/base/dev_common.properties
            /XXXXXXXX/wlst/wm/dev_workmanager.properties
        
        
     
      
       
        
        com.oracle.weblogic
        weblogic-maven-plugin
        12.1.3-0-0
        
         /XXXXXXXXX/Oracle/bpm12c
        
        
         
          wlst-execute
          deploy
          
           wlst
          
          
           /XXXXXXXX/Oracle/bpm12c
           ${project.basedir}/workManagerObject.py
           ${common.propeties.file} ${wm.properties.file}
           true
           false
           true
           wls
          
         
        
       
    
    The script takes two property files :

    • dev_common.properties - Contains the environment related configurations and will be used to execute the control/edit wlst commands against the weblogic server.admin.target.server=localhost
    admin.target.port=7101
    admin.target.user=weblogic
    admin.target.password=weblogic1
    wls.domain.name=DefaultDomain
    
    • dev_workmanager.properties - Contains the work manager related configurations.
    ###################################################
    # Name -   dev_workmanager.properties   #
    # Author - gkrishna    #
    # Version - 1.0      #
    ###################################################
    
    WorkManagers=TestWorkManager1 TestWorkManager2
    
    ########## Work Manager ##########
    TestWorkManager1.name=TestWorkManager1
    TestWorkManager1.targets=DefaultServer
    TestWorkManager1.targetType=Server
    TestWorkManager1.ignoreStuckThread=True
    
    TestWorkManager2.name=TestWorkManager2
    TestWorkManager2.targets=DefaultServer
    TestWorkManager2.targetType=Server
    TestWorkManager2.ignoreStuckThread=True
    
    ########## Minimum Threads Constraint ##########
    TestWorkManager1.minThreadsConstraint.name=TestWorkManager1MinThreadsConstraint
    TestWorkManager1.minThreadsConstraint.count=10
    
    TestWorkManager2.minThreadsConstraint.name=TestWorkManager2MinThreadsConstraint
    TestWorkManager2.minThreadsConstraint.count=20
    
    ########## Maximum Threads Constraint ##########
    TestWorkManager1.maxThreadsConstraint.name=TestWorkManager1MaxThreadsConstraint
    TestWorkManager1.maxThreadsConstraint.count=20
    
    TestWorkManager2.maxThreadsConstraint.name=TestWorkManager2MaxThreadsConstraint
    TestWorkManager2.maxThreadsConstraint.count=30
    
    ########## Capacity Constraints ##########
    TestWorkManager1.capacityConstraint.name=TestWorkManager1CapacityConstraint
    TestWorkManager1.capacityConstraint.count=40
    
    TestWorkManager2.capacityConstraint.name=TestWorkManager2CapacityConstraint
    TestWorkManager2.capacityConstraint.count=40
    
    
    Before we execute the pom file to create the work managers, lets take a snapshot of the existing work managers to identify the difference after executing the script.
    Lets execute the pom file to create the work manager.
    mvn clean deploy
    


    Lets check in console to verify the creation of Work Manager:


    No comments:

    Post a Comment