Enhancing My Proxy Server: Load Balancing with Multi-Route
Written on
Chapter 1: Introduction to ChiProxy
In this section, we'll explore the ChiProxy server, which demonstrates the unique capabilities of the Toolips framework. Toolips, unlike many other web-development frameworks in Julia and beyond, is an extensible server platform that allows for the creation of various server types. A notable example is ToolipsUDP, which enables the development of servers using a connection-less protocol. The recent major update of Toolips has further enhanced its extensibility and functionality.
ChiProxy serves as a prime example of Toolips' versatility. While most frameworks are limited to web server development, ChiProxy is designed as a proxy server using Toolips. With Toolips 0.3, we can import functions to modify our router's behavior, allowing ChiProxy to seamlessly integrate into the Toolips ecosystem, complete with its own router embedded within the established framework.
For a deeper understanding of this integration, I suggest reviewing my initial article about the project, which details the router's creation.
Section 1.1: Requirements for Implementation
Upon developing the proxy server, I identified three essential features for its deployment. First, I wanted the server to handle requests from various sources. For instance, if another web server is running within the same Julia process, we should be able to call it easily. The proxy server can serve functions directly, leveraging the full capabilities of the Toolips framework.
The second crucial feature is load balancing, which is vital for distributing incoming traffic across multiple machines. For example, if two servers are hosting the same website, load balancing allows us to share the user traffic effectively.
Lastly, SSL support is necessary to serve websites securely with HTTPS certificates.
So far, I've successfully implemented the source-loading feature. The load balancing was introduced in my previous update, but I have new and improved ideas for optimizing this aspect, which I'll explore today. This will prepare the server for the final article, where SSL implementation will be discussed.
Subsection 1.1.1: Previous Implementation
In the last article, I introduced source routes supported by a balanced abstract type:
abstract type Balanced{T <: Number} end
struct Source{T <: Any} <: AbstractSource
sourceinfo::Dict{Symbol, <:Any}
end
This design utilized methods for creating and retrieving sources, enabling our server to manage load effectively.
function source(path::String, loads::SourceRoute{<:Any, <:Any} ...)
srcinfo::Dict{Symbol, Any} = Dict{Symbol, Any}(:routes => [root for root in loads], :at => 1)
src = Source{Balanced{Integer}}(srcinfo)
SourceRoute{Connection, Balanced{Integer}}(path, src)
end
Although this implementation was functional, I aim to rework the load balancing to adopt a simpler, more effective approach based on Toolips 0.3's multi-route feature.
Section 1.2: Understanding Multi-Route
To grasp the multi-route concept, it's important to understand its operation. The server initiates routing by calling route! on a vector of abstract routes and the connection object. Toolips’ standard routing mechanism identifies the path associated with the target connection, which we adapted to create a proxy router based on hostname.
route!(c::Connection, vec::Vector{<:AbstractProxyRoute}) = begin
if Toolips.get_route(c) == "/favicon.ico"
write!(c, "no icon here, fool")
return
end
selected_route::String = get_host(c)
if selected_route in vec
route!(c, vec[selected_route])else
write!(c, "this route is not here")end
end
Next, route! is called to send the TCP connection to the appropriate application via the proxy_pass! function.
function route!(c::Toolips.AbstractConnection, pr::AbstractProxyRoute)
Toolips.proxy_pass!(c, "http://$(string(pr.ip4))")
end
Multi-route allows us to handle connections with various dispatch mechanisms. For example, the server can serve both mobile and desktop versions of a website on the same route.
module Serv
using Toolips
main = route("/") do c::Connection
write!(c, "hello!")end
mobile = route("/") do c::Toolips.MobileConnection
write!(c, "hello mobile!")end
home = route(main, mobile)
export home
end
In our case, we want the multi-route to facilitate load balancing among the various routes, selecting the optimal one for incoming requests.
Chapter 2: Implementing Balanced Multi-Routing
To enhance the load balancing feature, we'll create a new BalancedMultiRoute type.
abstract type AbstractProxyRoute <: Toolips.AbstractRoute end
abstract type ProxyMultiRoute <: AbstractProxyRoute end
mutable struct BalancedMultiRoute <: ProxyMultiRoute
path::String
sources::Vector{AbstractProxyRoute}
loads::Vector{Float64}
status::Pair{Int64, Int64}
scale::Int64
end
Next, we’ll establish a routing function that combines proxy routes into the BalancedMultiRoute.
proxy_route(path::String, routes::Vector{Int64, AbstractProxyRoute} ...; scale::Int64 = 100) = begin
end
This will act as a constructor, ensuring that the load percentages sum to 1.0.
proxy_route(path::String, routes::Pair{Float64, <:AbstractProxyRoute} ...; scale::Int64 = 100) = begin
loads::Vector{Float64} = Vector{Float64}()
new_routes::Vector{AbstractProxyRoute} = Vector{AbstractProxyRoute}()
for r in routes
push!(loads, r[1])
push!(new_routes, r[2])
end
if sum(loads) != 1.0
throw(Toolips.RouteError("balanced proxy-route", "load balances must add up to 100 (percent)."))end
BalancedMultiRoute(path, new_routes, loads, 1 => 0, scale)
end
The routing logic must determine the next source and route accordingly.
function route!(c::Toolips.AbstractConnection, pr::BalancedMultiRoute)
current_source::Int64 = pr.status[1]
current_count::Int64 = pr.status[2]
if current_count / pr.scale >= pr.loads[current_source]
current_source += 1
current_count = 0
if current_source > length(pr.sources)
current_source = 1end
end
current_count += 1
pr.status = current_source => current_count
route!(c, pr.sources[current_source])
end
Let's put this implementation to the test.
server1 = proxy_route("192.168.1.15", "127.0.0.1":8000)
server2 = proxy_route("192.168.1.15", "127.0.0.1":8001)
balances = proxy_route("192.168.1.15", .5 => server1, .5 => server2, scale = 10)
This demonstration of the proxy server will run on port 80, directing incoming clients to local IP addresses.
The first video provides insights on building a load balancer in .NET using YARP Reverse Proxy, which complements the load balancing discussion here.
The second video explores load balancing and HTTP routing with Envoy Proxy, offering additional context for our implementation.
Next Steps
With a robust load-balancing feature now in place for the proxy server, the next significant milestone is implementing SSL. Though the complexity of performing the TLS handshake may seem daunting, my research is demystifying the process. Thankfully, the availability of an OpenSSL library in Julia should facilitate this implementation.
Once SSL is integrated, I'll focus on creating a configuration file format that enables server setup without direct interaction with Julia, similar to the ChiNS project.
Conclusion
In summary, Toolips 0.3 has made remarkable progress. Although I’ve completed the project, additional testing could enhance its reliability. I’m currently awaiting the completion of ToolipsSession 0.4, as both packages are closely linked and often released together.
I look forward to advancing the proxy server and name server projects, and soon, we will begin developing the web server, bringing us closer to deploying actual websites. In a few months, my goal is to have a fully operational server system. Thank you for following along!