LVS is a mechanism implemented by Linux kernel to do layer-4 (transport layer) switching, mainly to achieve load distribution for high availability purposes.
LVS is managed using a command line tool: ipvmadm
I used docker containers running apache 2.4 to test the concept on a Virtual box VM.
First, I needed to install docker containers, and create docker volums to store the apache configuration and the content outside the containers:
root@fingolfin:~# docker volume create httpdcfg1
httpdcfg1
root@fingolfin:~# docker volume create httpdcfg2
httpdcfg2
root@fingolfin:~# docker volume create www2
www2
root@fingolfin:~# docker volume create www1
www1
root@fingolfin:~#
Once the volumes are created, we can start the containers:
root@fingolfin:~# docker run -d --name httpd1 -p 8180:80 -v www1:/usr/local/apache2/htdocs/ -v httpdcfg1:/usr/local/apache2/conf/ httpd:2.4
a6e11431a228498b8fc412dfcee6b0fc682ce241e79527fdf33e7ceb1945e54a
root@fingolfin:~#
root@fingolfin:~# docker run -d --name httpd2 -p 8280:80 -v www2:/usr/local/apache2/htdocs/ -v httpdcfg2:/usr/local/apache2/conf/ httpd:2.4
b40e29e187b0841d81b345ca975cd867bcce587be8b3f79e43a2ec0d1087aba8
root@fingolfin:~#
root@fingolfin:~# docker container ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a665073770ec httpd:2.4 "httpd-foreground" 41 minutes ago Up 41 minutes 0.0.0.0:8280->80/tcp httpd2
d1dc596f68a6 httpd:2.4 "httpd-foreground" 54 minutes ago Up 45 minutes 0.0.0.0:8180->80/tcp httpd1
root@fingolfin:~#
Then we need to change the index.html on the volumes www1 and www2 to show different nodes.
This can be done by directly accessing the file from the docker volume mount point:
root@fingolfin:~# docker volume inspect www1
[
{
"CreatedAt": "2020-08-23T16:59:50+02:00",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/www1/_data",
"Name": "www1",
"Options": {},
"Scope": "local"
}
]
root@fingolfin:~# cd /var/lib/docker/volumes/www1/_data/
Next step is to obtain the docker container IP address using docker container inspect:
root@fingolfin:~# docker container inspect httpd1|grep '"IPAddress":'
"IPAddress": "172.17.0.2",
"IPAddress": "172.17.0.2",
root@fingolfin:~#
root@fingolfin:~# docker container inspect httpd2|grep '"IPAddress":'
"IPAddress": "172.17.0.3",
"IPAddress": "172.17.0.3",
root@fingolfin:~#
One more thing, we need to install ping inside the containers, just to make sure that we can troubleshoot network in case things didn't work.
To do this we use docker exec:
root@fingolfin:~# docker exec -it httpd1 bash
root@a6e11431a228:/usr/local/apache2#
root@a6e11431a228:/usr/local/apache2# apt update; apt install iputils-ping -y
Next we will create a subinterface IP address or as people call it a vip (virtual ip) on the main ethernet device of the docker host, will use a 10.0.2.0 network address, but it should be equally fine to use any other IP addess.
root@fingolfin:~# ifconfig enp0s3:1 10.0.2.200 netmask 255.255.255.0 broadcast 10.0.2.255
root@fingolfin:~#root@fingolfin:~# ip addr show dev enp0s3
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 08:00:27:7e:6e:97 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp0s3
valid_lft 74725sec preferred_lft 74725sec
inet 172.17.60.200/16 brd 172.17.255.255 scope global enp0s3:0
valid_lft forever preferred_lft forever
inet 10.0.2.200/24 brd 10.0.2.255 scope global secondary enp0s3:1
valid_lft forever preferred_lft forever
inet6 fe80::a790:c580:9be5:55ef/64 scope link noprefixroute
valid_lft forever preferred_lft forever
root@fingolfin:~#
We can then ping the IP of the host, 10.0.2.15 from within the docker container, to ensure container can reach that:
root@a6e11431a228:/usr/local/apache2# ping 10.0.2.15
PING 10.0.2.15 (10.0.2.15) 56(84) bytes of data.
64 bytes from 10.0.2.15: icmp_seq=1 ttl=64 time=0.114 ms
64 bytes from 10.0.2.15: icmp_seq=2 ttl=64 time=0.127 ms
^C
--- 10.0.2.15 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 17ms
rtt min/avg/max/mdev = 0.114/0.120/0.127/0.012 ms
root@a6e11431a228:/usr/local/apache2#
Then set up the Linux LVS from the host prompt:
root@fingolfin:~# ipvsadm -A -t 172.17.60.200:80 -s rr
root@fingolfin:~# ipvsadm -a -t 172.17.60.200:80 -r 172.17.0.3:80 -m
root@fingolfin:~# ipvsadm -a -t 172.17.60.200:80 -r 172.17.0.2:80 -m
root@fingolfin:~# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.2.200:80 rr
-> 172.17.0.2:80 Masq 1 0 0
-> 172.17.0.3:80 Masq 1 0 0
root@fingolfin:~#
Then to test the setup, we use curl from the host command line:
root@fingolfin:~# curl http://10.0.2.200
It works! node 1
root@fingolfin:~# curl http://10.0.2.200
It works! node 2
root@fingolfin:~# curl http://10.0.2.200
It works! node 1
root@fingolfin:~# curl http://10.0.2.200
It works! node 2
root@fingolfin:~#
root@fingolfin:~# ipvsadm -L -n --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.0.2.200:80 4 24 16 1576 1972
-> 172.17.0.2:80 2 12 8 788 986
-> 172.17.0.3:80 2 12 8 788 986
root@fingolfin:~#
As can be seen, the requests are equally distributed between the 2 Apache containers.
Other loadbalancing algorithms can be used to suite the need of the use case using the -s (scheduler) option.
More information can be found in the Linux ipvsadm man page: https://linux.die.net/man/8/ipvsadm
Referances:
http://www.ultramonkey.org/papers/lvs_tutorial/html/
No comments:
Post a Comment