Session replication how-to

PC clustering


In a brief:

The jar library for PC replication: exo.pc.portal-framework-VERSION.jar
There is a container component SessionReplicator.
The PortalFrameworkFilter#doFilter for send session attributes for other PC instances with current session id.

Documentation
http://wiki.exoplatform.com/xwiki/bin/view/PC/Session+replication
http://www.theserverside.com/tt/articles/article.tss?l=ClusteringTomcat
http://www.ja-sig.org/wiki/display/PLT/Clustering+JSR-168+Portlet+Applications+in+Tomcat


Developing

The replication class SessionReplicator (org.exoplatform.frameworks.portletcontainer.portalframework.replication)
which is Startable and RequestHandler.

The container configuration configuration.xml (/conf/portal)


<component>
    <type>org.exoplatform.frameworks.portletcontainer.portalframework.replication.SessionReplicator</type>\\
  </component>




Usage (ex. PortalFrameworkFilter#doFilter) 

try {
      SessionReplicator sessionReplicator = (SessionReplicator)container.getComponentInstanceOfType(SessionReplicator.class);\\
      sessionReplicator.send(httpSession.getId(), framework.getPortalName(), framework.getRenderedPortletInfos());\\
    } catch (Exception e){\\
      log.error("SessionReplicator send exception",e);\\
    }




JGroups Registered information 

'INFO' JChannel - JGroups version: 2.5.2
-------------------------------------------------------
GMS: address is 192.168.0.23:33070
-------------------------------------------------------





Session attribute names creation:

Set uniqueID for WindowID:
org.exoplatform.services.portletcontainer.pci.ExoWindowID.setUniqueID(String)
org.exoplatform.frameworks.portletcontainer.portalframework.PortalFramework.addPortletWithId(String, String, String)
org.exoplatform.frameworks.portletcontainer.portalframework.PortalFramework.addPortlet(String, String)
org.exoplatform.frameworks.portletcontainer.portalframework.PortalFramework.createOrUpdatePortletWindows()
org.exoplatform.services.portletcontainer.test.filters.PortletFilter.doFilter(ServletRequest, ServletResponse, FilterChain)

Set session attribute:
org.exoplatform.services.portletcontainer.plugins.pc.portletAPIImp.utils.PortletSessionImpUtil.encodePortletSessionAttribute(String, String, int)
org.exoplatform.services.portletcontainer.plugins.pc.portletAPIImp.PortletSessionImp.setAttribute(String, Object, int)


windowUniqueID = "" + (appName + "/" + portletName + "/" + (new Date().toString())).hashCode() + "_" + httpSession.getId()
\\where is:       httpSession.getId() = 7014D59968F36B5B67F7DDD9570EBA01.WRK_Portal2
\\result:      == -688941700_7014D59968F36B5B67F7DDD9570EBA01.WRK_Portal2
      attributeName = PORTLET_SCOPE_NAMESPACE + windowUniqueID + "?" + attributeName
\\where is:      PORTLET_SCOPE_NAMESPACE = "javax.portlet.p."      
\\result:      == "javax.portlet.p.-688941700_7014D59968F36B5B67F7DDD9570EBA01.WRK_Portal2?a"





Install and configure Tomcats

Build, configure and deploy

java/eXoProjects/portlet-container/trunk/
$mvn clean install

Configure JGroups replication configuration file (conf/portal/configuration.xml) within jar library
java/eXoProjects/portlet-container/trunk/applications/portal-framework/target/exo.pc.portal-framework-trunk.jar



java/eXoProjects/portlet-container/trunk/applications/assembly/
$deploy-tomcat6-withdelete.sh

web.xml

Modify WEB-INF/web.xml by adding the 
<distributable/>
tag. This should appear immediately after the web-app entry.


Create two tomcat instances

cd java/exo-working

mkdir 1
cp -r exo-tomcat 1/exo-tomcat

mkdir 2
cp -r exo-tomcat 2/exo-tomcat

Tomcat 1

java/exo-working/1/exo-tomcat/conf/
sudo mcedit server.xml

Change this:


<Engine name="Catalina" defaultHost="localhost">


with this:


<Engine name="Catalina" defaultHost="localhost" jvmRoute="WRK_Portal1">


Change ports 80xx with 81xx

UPDATED: Full configuration. 


=== server.xml ==============================================================
<?xml version='1.0' encoding='utf-8'?>
  <Server port="8105" shutdown="SHUTDOWN">

  
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">
  
    <Connector port="8180" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />

    <Connector port="8109" protocol="AJP/1.3" redirectPort="8443" />



    <Engine name="Standalone" defaultHost="localhost" jvmRoute="WRK_Portal1">         

    <Engine name="Catalina" defaultHost="localhost">

        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 managerClassName="org.apache.catalina.cluster.session.DeltaManager"
                 expireSessionsOnShutdown="false"
                 useDirtyFlag="true">

               <Membership       className="org.apache.catalina.cluster.mcast.McastService"
                mcastAddr="228.0.0.4"
                mcastPort="45564"
                mcastFrequency="500"
                mcastDropTime="3000"/>

 

           <Receiver className="org.apache.catalina.cluster.tcp.ReplicationListener"
                tcpListenAddress="auto"
                tcpListenPort="4001"
                tcpSelectorTimeout="100"
                tcpThreadCount="6"/>

            <Sender className="org.apache.catalina.cluster.tcp.ReplicationTransmitter" replicationMode="pooled"/>

            <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"/>

</Cluster>

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
             resourceName="UserDatabase"/>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true"
            xmlValidation="false" xmlNamespaceAware="false">
      </Host>
    </Engine>
  </Service>
</Server>
===========================================================================


Tomcat 2

java/exo-working/2/exo-tomcat/conf/
sudo mcedit server.xml

Change this: 

<Engine name="Catalina" defaultHost="localhost">

with this: 

<Engine name="Catalina" defaultHost="localhost" jvmRoute="WRK_Portal2">




Change ports 80xx with 82xx
UPDATED: Full configuration. 

=== server.xml ==============================================================
<?xml version='1.0' encoding='utf-8'?>
  <Server port="8205" shutdown="SHUTDOWN">

  
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">
  
    <Connector port="8280" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />

    <Connector port="8209" protocol="AJP/1.3" redirectPort="8543" />



    <Engine name="Standalone" defaultHost="localhost" jvmRoute="WRK_Portal2">         

    <Engine name="Catalina" defaultHost="localhost">

        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 managerClassName="org.apache.catalina.cluster.session.DeltaManager"
                 expireSessionsOnShutdown="false"
                 useDirtyFlag="true">

               <Membership       className="org.apache.catalina.cluster.mcast.McastService"
                mcastAddr="228.0.0.4"
                mcastPort="45564"
                mcastFrequency="500"
                mcastDropTime="3000"/>

 

           <Receiver className="org.apache.catalina.cluster.tcp.ReplicationListener"
                tcpListenAddress="auto"
                tcpListenPort="4002"
                tcpSelectorTimeout="100"
                tcpThreadCount="6"/>

            <Sender className="org.apache.catalina.cluster.tcp.ReplicationTransmitter" replicationMode="pooled"/>

            <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"/>

</Cluster>

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
             resourceName="UserDatabase"/>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true"
            xmlValidation="false" xmlNamespaceAware="false">
      </Host>
    </Engine>
  </Service>
</Server>
=========================================================================

Note that tcpListenPort value under Cluster serction must also be unique on all nodes.




Install and configure Apache (balancer)

sudo apt-get install apache2-mpm-worker
wget http://apache.infocom.ua/tomcat/tomcat-connectors/jk/binaries/linux/jk-1.2.27/i386/mod_jk-1.2.27-httpd-2.2.6.so
sudo wget http://apache.infocom.ua/tomcat/tomcat-connectors/jk/binaries/linux/jk-1.2.27/i386/mod_jk-1.2.27-httpd-2.2.6.so
sudo cp ./mod_jk.load /etc/apache2/mods-available/
sudo cp ./mod_jk.conf /etc/apache2/mods-available/
sudo mcedit mod_jk.conf
sudo mcedit workers.properties
sudo mcedit 000-default


workers.properties (/etc/apache2)



worker.list=wwwrouter

# ------------------------------------------------------------
# Main status worker
worker.wwwrouter.type=lb
worker.wwwrouter.balance_workers=_WRK_Portal1, WRK_Portal2
worker.wwwrouter.sticky_session=True

# ------------------------------------------------------------
# Tomcat-1 (tools1.exoplatform.org)
worker.WRK_Portal1.type=ajp13
worker.WRK_Portal1.host=127.0.0.1
worker.WRK_Portal1.port="8109"
worker.WRK_Portal1.socket_timeout=10
worker.WRK_Portal1.lbfactor=10
worker.WRK_Portal1.socket_keepalive=0
worker.WRK_Portal1.route=WRK_Portal1

# ------------------------------------------------------------
# Tomcat-2 (tools3.exoplatform.org)
worker.WRK_Portal2.type=ajp13
worker.WRK_Portal2.host=127.0.0.1
worker.WRK_Portal2.port="8209"
worker.WRK_Portal2.socket_timeout=10
worker.WRK_Portal2.lbfactor=10
worker.WRK_Portal2.socket_keepalive=0
worker.WRK_Portal2.route=WRK_Portal2



httpd.conf (/etc/apache2)

ServerName    localhost
#listen        80



mod_jk.load (/etc/apache2/mods-available)


LoadModule jk_module /usr/lib/apache2/modules/mod_jk.so




mod_jk.conf (/etc/apache2/mods-available)

<IfModule jk_module>
# ---- Where to find workers.properties
        JkWorkersFile   workers.properties

        # ----  Where to put jk shared memory
        JkShmFile       /var/log/apache2/mod_jk.shm
        JkShmSize       128k

        # ---- Where to put jk logs
        JkLogFile       /var/log/apache2/mod_jk.log

        # ---- Set the jk log level [debug/error/info]
        JkLogLevel    info
        # ---- Select the timestamp log format
        JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
        JkRequestLogFormat "%w %R %T"
        # ---- Send everything for context /examples to worker named worker1 (ajp13)
        JkMountFileReload       "0"

       JkMount /portal/*  loadbalancer                                                                                                                                             

       JkMount /portlets/* loadbalancer                                                                                                                                           

       JkMount /portlets2/* loadbalancer 
</IfModule>




Mods-enabled

alexey@alexey:/etc/apache2/mods-enabled$ ls -all | grep jk
lrwxrwxrwx 1 root root 29 2008-11-13 18:01 mod_jk.conf -> ../mods-available/mod_jk.conf
lrwxrwxrwx 1 root root 29 2008-11-13 18:00 mod_jk.load -> ../mods-available/mod_jk.load

000-default

(/etc/apache2/sites-enabled/000-default -> /etc/apache2/sites-available/default/000-default)



NameVirtualHost *

<VirtualHost *>
    ServerAdmin webmaster@localhost
#    ServerName    localhost

    JkMount    /*    wwwrouter
    DocumentRoot /var/www/
    ...






Start and log

/etc/apache2# apache2ctl start

/etc/apache2# apache2ctl status




============================================
                       Apache Server Status for localhost
   Server Version: Apache/2.2.4 (Ubuntu) mod_jk/1.2.27
==============================================

*/etc/apache2

Tags:
Created by Alexey Zavizionov on 11/19/2008
Last modified by Max Shaposhnik on 04/17/2009

Search

 

Products

generated on Fri Jul 30 19:09:17 UTC 2010

eXo Optional Modules

eXo Core Foundations

Recently Modified

queuerKib | constructiongirl | VuccereCync | effinyper
Copyright (c) 2000-2010. All Rights Reserved - eXo platform SAS
2.4.30451