Merging Traffic Manager and Gateway Profiles - WSO2 APIM

This guide includes how to configure a WSO2 API Manager 2.0.0 cluster, highlighting the specific scenario of merging Traffic Manager Profile into the Gateway. I will describe how to configure a sample API Manager setup which demonstrate Merging Traffic Manager and Gateway Profiles.

I will configure publisher, store and key manager components in a single server instance as the expectation is to illustrate merging gateway and traffic manager and starting that merged instance separately from other components.

This sample setup consists of following 3 nodes,

1. Publisher + Store + Key Manager (P_S_K)                       ; offset = 0
2. Gateway Manager/Worker + Traffic Manager (GM_T)     ; offset = 1
3. Gateway Worker + Traffic Manager (GW_T)                    ; offset = 2

• We will refer the 3 nodes as P_S_K , GM_T , GW_T for convenience.
• There is a cluster of gateways, one node is acting as the manager/worker node and other one is as a simple worker node.
• Traffic managers are configured with high availability.
• Port offset is configured as mentioned above. To set the port offsets edit the <Offset> element in <APIM_HOME>/repository/conf/carbon.xml , in each of the nodes.

Figure 1 : Simple architectural diagram of the setup

1. Configuring datasources

We can configure databases according to the APIM documentation

https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0#ClusteringAPIManager2.0.0­Installingandconfiguringthedatabases

[Please open such these documentation links in Chrome or any other browser except Firefox, as Firefox has a bug with Confluence Atlassian document links in opening the link at the expected position of the page.] 

Follow the steps in it carefully. Assume that the names of the databases we created are as follows,

       API Manager Database ­ - apimgtdb

       User Manager Database ­- userdb 

       Registry Database​ ­ ​        - regdb

In the above mentioned doc, apply all the steps defined for all three store, publisher, key manager

nodes to our P_S_K node. Because in that documentation publisher, store, key manager are existing in different nodes, but in our this setup we are using a single node which acts as all 3 components

(publisher, store and key manager).

Following is a summary of configured datasources for each node.

         P_S_K node : ​apimgtdb , userdb , regdb

         GM_T node / GW_T node​ : Not required

2. Configuring the connections among the components

You will now configure the inter-component relationships of the distributed setup by modifying their <APIM_HOME>/repository/conf/api-manager.xml files.

This section includes the following sub-topics.

1. Configuring P_S_K node
2. Configuring GM_T node & GW_T node

2.1 Configuring P_S_K node

Here we have to configure this node for all the 3 components, publisher, store, key manager related functionalities.

For this follow this steps mentioned in the given docs. (it is recommended not to open these links in Firefox browser ) In them, the setup is created as the publisher, store and key manager that are in separate nodes in a cluster. So follow the steps as per the requirement of yours, considering the port offsets too.

2.1.1 Configurations related to  publisher

https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0#ClusteringAPIManager2.0.0-ConfiguringtheAPIPublisher

Follow all the steps in the above wso2 documentation except the configurations for jndi.properties file. Configurations for that file should be changed as following.

This is for configuring fail over for publishing Custom Templates and Block conditions into the Gateway node.

In <APIM_HOME>/repository/conf/jndi.properties file, line,

connectionfactory.TopicConnectionFactory = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<Traffic-Manager-host>:5676'

topic.throttleData = throttleData

should be changed as following.

connectionfactory.TopicConnectionFactory = amqp://admin:admin@clientID/carbon?failover='roundrobin'%26cyclecount='2'%26brokerlist='tcp://<IP_Of_GM_T_Node>:5673?retries='5'%26connectdelay='50';tcp://<IP_Of_GW_T_Node>:5674?retries='5'%26connectdelay='50''

In the above config,

5673 => 5672+offset of GM_T node

5674 => 5672+offset of GW_T node

2.1.2 Configurations related to  store

Follow all the steps appropriately in the below wso2 documentation link

https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0#ClusteringAPIManager2.0.0-ConfiguringtheAPIStore

2.1.3 Configurations related to key manager

Follow all the steps appropriately in the below wso2 documentation link

https://docs.wso2.com/display/CLUSTER44x/Clustering+API+Manager+2.0.0#ClusteringAPIManager2.0.0-ConfiguringtheKeyManager

Note : In the above docs, setup is created as the publisher, store and key manager that are in separate nodes in a cluster. So follow the steps as per the requirement of yours, keeping in mind that you are configuring them into a single node considering the port offsets too.

2.2 Configuring GM_T node & GW_T node

Configurations for the two Gateway+Traffic Manager nodes are very much similar. So follow each below steps for both the nodes. I will mention the varying steps when required.

Please note that when starting these nodes you have to start them in default profiles as there is no customized profile for gateway+traffic manager.

2.2.1 Gateway component related configurations

This section involves setting up the gateway component related configurations to enable it to work with the other components in the distributed setup.

I will use G_T in short for the GM_T or GW_T node. Apply the IP address of the own node in the configurations below.

1. Open the <APIM_HOME>/repository/conf/api-manager.xml file in the GM_T/GW_T node.   
2. Modify the api-manager.xml file as follows. This configures the connection to the Key Manager component.

3. Configure key management related communication. (both nodes)

In a clustered setup if the Key Manager is fronted by a load balancer, you have to use WSClient as KeyValidatorClientType in <APIM_HOME>/repository/conf/api-manager.xml. (This should be configured in all Gateway and Key Manager components and so as per our setup configure this in GM_T and GW_T nodes)

<KeyValidatorClientType>WSClient</KeyValidatorClientType>

4. Configure throttling for the Traffic Manager component. (both nodes).  Modify the api-manager.xml file as follows

In the above configs, <connectionfactory.TopicConnectionFactory> element used to configure jms topic url where worker node uses to listen for throttling related  events. In this case, each Gateway node has to listen Topics in both Traffic managers. Because if one node gets down, throttling procedures should work without any interrupt. Even though one node gets down, the throttling related counters will now exist synced with the other node. Hence, here we have configured failover jms connection url as pointed above.

2.2.2 Clustering Gateway + Traffic Manager nodes

In our sample setup we are using two nodes.

1. Manager/Worker

2. Worker

We have followed the below steps for Traffic manager related clustering, and if you want to do the configurations for Gateway clustering with load balancer, follow the documentation https://docs.wso2.com/display/CLUSTER44x/Clustering+the+Gateway and configure host names in carbon.xml appropriately, add svn deployment synchronizer, etc.
Follow the below steps for both the nodes or Traffic manager related clustering.

Open the <AM_HOME>/repository/conf/axis2/axis2.xml file

1. Scroll down to the 'Clustering' section. To enable clustering for the node, set the value of "enable" attribute of the "clustering" element to "true", in each of 2 nodes.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent"

enable="true">

2. Change the 'membershipScheme' parameter to 'wka'.

<parameter name="membershipScheme">wka</parameter>

3. Specify the host used to communicate cluster messages. (in both nodes)

<parameter name="localMemberHost"><ip of this GM_T node></parameter>

4. Specify the port used to communicate cluster messages.

        Let’s give port 4001 in GM_T node.

              <parameter name="localMemberPort">4001</parameter>

        Let’s give port 4000 in GW_T node.

              <parameter name="localMemberPort">4001</parameter>

5. Specify the name of the cluster this node will join. (for both nodes)

<parameter name="domain">wso2.carbon.domain</parameter>

6. Change the members listed in the <members> element. This defines the WKA members. (for both nodes)

2.2.3 Traffic manager related configurations

This section involves setting up the Traffic manager component related configurations to enable it to work with the other components in the distributed setup. 

1. Delete the contents of <APIM_HOME>/repository/conf/registry.xml file and copy the contents of the <APIM_HOME>/repository/conf/registry_TM.xml file, into the registry.xml file (in both nodes)

2. Remove all the existing webapps and jaggeryapps from the <APIM_HOME>/repository/deployment/server directory.

(in both nodes)

3. High Availability configuration for traffic manager component (in both nodes)

• Open <APIM_HOME>/repository/conf/event-processor.xml and enable HA mode as below.

    <mode name="HA" enable="true">

• Set ip for event synchronization

    <eventSync>

        <hostName>ip     of this node</hostName>

        .....

    </eventSync>

2.2.4 Configuring JMS TopicConnectionFactories

Here we are configuring TopicConnectionFactories to get data from to traffic manager. So in this cluster configuration we are using 2 TopicConnectionFactories (one for each), and configure to send data to both TopicConnectionFactories. (send to own TopicConnectionFactory and other node’s TopicConnectionFactory too)

So open in <APIM_HOME>/repository/conf/jndi.properties file (in both nodes) and do th efollwoing changes in it,

connectionfactory.TopicConnectionFactory = amqp://admin:admin@clientid/carbon?brokerlist='tcp://localhost:5672'

line to following,

connectionfactory.TopicConnectionFactory1 = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<ip of GM_T>:5673

And add new line as,

connectionfactory.TopicConnectionFactory2 = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<ip of GW_T>:5674'

Finally that section would be as below.

# register some connection factories

# connectionfactory.[jndiname] = [ConnectionURL]

connectionfactory.TopicConnectionFactory1 = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<ip of GM_T>:5673'

connectionfactory.TopicConnectionFactory2 = amqp://admin:admin@clientid/carbon?brokerlist='tcp://<ip of GW_T>:5674'

5673 => 5672 + portOffset of GM_T

5674 => 5672 + portOffset of GW_T

2.2.5 Add event publishers to publish data to related JMS Topics.

(Do this for both the nodes.) We have to publish data from Traffic manager component to TopicConnectionFactory in its own and other node too. So there should be 2 jmsEventPublisher files in the <APIM_HOME>/repository/deployment/server/eventPublishers/ for that.

There is already a,<APIM_HOME>/repository/deployment/server/eventPublishers/jmsEventPublisher.xml file in the default pack. In it, update the ConnectionFactoryJNDIName as below.

And that is it. U are done :-)