On this cluster hive database is Mysql-server on node name hiverserver and node2 has oozie-server installed. To make oozie-server as high available the backend must be a database with supports concurrent connections (which mysql supports)
Here hiveserver is FQDN of a node in this cluster setup just as an example for demonstration.
Ssh into hiveserver as shown below.
A sample truncated output for for Sudo service --status-all
Finding Mysqld as shown below.
Mysql server my.cnf configuration file
1)A another installation of Mysql-server on FQDN named node2 in this cluster just for demonstration.
bashshell> SSH root@node2 (provide password when asked)
2)once entered in node2 the bashshell looks like as below
[root@node2]
3) using yum package installer install MYSQL-SERVER
[root@node2] yum install mysql-server
4)After installation is successful start mysqld in sbin folder as shown below.
5)Run the mysql_secure_installation script in /usr/bin/ folder to secure your mysql-server installation to remove any default databases, to change root password and remove any anonymous users.
This is for demo purposes only , kept all users and test database without any change, but the production and any other cluster requirements change.
6) Login to mysql cli as shown below.
7) when enabling oozie HA you need to have any other database as a persistence store for Oozie-server other than default derby. derby can only serve single connection at a time.
so when running ambari as a monitoring tool for the cluster the Ambari needs to connect to mysql instance too. Execute the below command on Ambari -server host to make the jdbc connection available and so that testing the database connection can be done
8) To enable oozie-ha deploy two Oozie-server on two different host and go to configs tab and change the properties of the following configurations to both reflect the same
a)oozie-server1 installed on hiveserver node
b)oozie-server2 installed on node2 node
go to configs tab and change the properties
c)oozie.zookeeper.connection.url = list of zookeeper nodes( values)
d)oozie,service.ext =org.apache.oozie.service.ZKLocksService,org.apache.oozie.service.ZKXLogStreamingService,
org.apache.oozie.service.ZKJobsConcurrencyService
e)oozie.base.url= http://haproxy.node2:11000/oozie
and export the base url
f) EXPORT OOZIE_BASE_URL ="http://haproxy.node2.11000/oozie
g)oozie.zookeeper.namespace = oozie
In the haproxy.cfg file we need to change few properties as shown below
frontend http( the incoming request coming from )
bind *:11000
mode http
acl url_oozie path_beg "http://haproxy.node2.11000/oozie"
use_backend oozie-backend if url_oozie
default_backend web-backend
-------------------
backend oozie-backend
balance roundrobin
mode http
server url_oozie oozie.hiveserver:11000 check
server url_oozie oozie.node2:11000 check
----------------------------
Backend
A backend is a set of servers that receives forwarded requests. Backends are defined in the backend section of the HAProxy configuration. In its most basic form, a backend can be defined by:- which load balance algorithm to use
- a list of servers and ports
---mode http specifies that layer 7 proxying will be used.Frontend
A frontend defines how requests should be forwarded to backends. Frontends are defined in the frontend section of the HAProxy configuration. Their definitions are composed of the following components:- a set of IP addresses and a port (e.g. 10.1.1.7:80, *:443, etc.)
- ACLs
- use_backend rules, which define which backends to use depending on which ACL conditions are matched, and/or a default_backend rule that handles every other case
Layer 7 Load Balancing
Another, more complex way to load balance network traffic is to use layer 7 (application layer) load balancing. Using layer 7 allows the load balancer to forward requests to different backend servers based on the content of the user's request.
