require 'socket'
require 'thread'
require 'openssl'
require 'wisper'
require 'socket'
require 'thread'
require 'openssl'
require 'wisper'
Proxi gives you flexible TCP/HTTP proxy servers for use during development.
This can be useful when developing against external services, to see what goes over the wire, capture responses, or simulate timeouts.
There are three main concepts in Proxi, Server
, Connection
, and Socket
.
A server listens locally for incoming requests. When it receives a request, it
establishes a Connection
, which opens a socket to the remote host, and then
acts as a bidirectional pipe between the incoming and outgoing network
sockets.
To allow for multiple ways to handle the proxying, and multiple strategies for
making remote connections, the Server
does not create Connections
directly, but instead delegates this to a "connection factory".
A Connection
in turn delegates how to open a remote socket to a "socket
factory".
Both Servers and Connections are observable, they emit events that objects can subscribe to.
To use Proxi, hook up these factories, and register event listeners, and then start the server.
module Proxi
These are provided for basic use cases, and as a starting point for more
complex uses. They return the server instance, call #call
to start the
server, or use on
or subscribe
to listen to events.
With Proxy.tcp_proxy
you get basic proxying from a local port to a remote
host and port, all bytes are simply forwarded without caring about their
contents.
For example:
Proxi.tcp_proxy(3000, 'foo.example.com', 4000).call
def self.tcp_proxy(local_port, remote_host, remote_port)
reporter = ConsoleReporter.new
connection_factory = -> in_socket do
socket_factory = TCPSocketFactory.new(remote_host, remote_port)
Connection.new(in_socket, socket_factory).subscribe(reporter)
end
Server.new(local_port, connection_factory)
end
Proxi.http_host_proxy
allows proxying to multiple remote hosts, based on
the HTTP Host:
header. To use it, gather the IP addresses that correspond
to each domain name, and provide this name-to-ip mapping to
http_host_proxy
. Now configure these domain names in /etc/hosts
to point
to the local host, so Proxi can intercept traffic intended for these
domains.
For example
Proxi.http_host_proxy(80, {'foo.example.org' => '10.10.0.1'}).call
def self.http_host_proxy(local_port, host_mapping)
reporter = ConsoleReporter.new
connection_factory = -> in_socket do
socket_factory = HTTPHostSocketFactory.new(host_mapping)
Connection
.new(in_socket, socket_factory)
.subscribe(socket_factory, on: :data_in)
.subscribe(reporter)
end
Server.new(local_port, connection_factory)
end
end
require_relative 'proxi/server'
require_relative 'proxi/connection'
require_relative 'proxi/socket_factory'
require_relative 'proxi/reporting'
require_relative 'proxi/listeners'
module Proxi
Proxi::Server
accepts TCP requests, and forwards them, by creating an
outbound connection and forwarding traffic in both directions.
The destination of the outbound connection, and the forwarding of data, is
handled by a Proxi::Connection
, created by a factory object, which can be a
lambda.
Start listening for connections by calling #call.
Proxi::Server
broadcasts the following events:
new_connection(Proxi::Connection)
dead_connection(Proxi::Connection)
class Server
include Wisper::Publisher
Public: Initialize a Server
listenport - The String or Integer of the port to listen to for incoming connections connectionfactory - Implements #call(insocket) and returns a Proxi::Connection maxconnections - The maximum amount of parallel connections to handle at once
def initialize(listen_port, connection_factory, max_connections: 5)
@listen_port = listen_port
@connection_factory = connection_factory
@max_connections = 5
@connections = []
end
def call
@server = TCPServer.new('localhost', @listen_port)
until @server.closed?
in_socket = @server.accept
connection = @connection_factory.call(in_socket)
broadcast(:new_connection, connection)
@connections.push(connection)
connection.call # spawns a new thread that handles proxying
reap_connections
while @connections.size >= @max_connections
sleep 1
reap_connections
end
end
ensure
close unless @server.closed?
end
Public: close the TCP server socket
Included for completeness, note that if the proxy server is active it will likely be blocking on TCPServer#accept, and the server port will stay open until it has accepted one final request.
def close
@server.close
end
private
def reap_connections
@connections = @connections.select do |conn|
if conn.alive?
true
else
broadcast(:dead_connection, conn)
conn.join_thread
false
end
end
end
end
end
module Proxi
A Connection
is a bidirectional pipe between two sockets.
The proxy server hands it the socket for the incoming request from, and
Connection
then initiates an outgoing request, after which it forwards all
traffic in both directions.
Creating the outgoing request is delegated to a Proxi::SocketFactory
. The
reason being that the type of socket can vary (TCPSocket
, SSLSocket
), or
there might be some logic involved to dispatch to the correct host, e.g.
based on the HTTP Host header (cfr. Proxi::HTTPHostSocketFactory
).
A socket factory can subscribe to events to make informed decision, e.g. to inspect incoming data for HTTP headers.
Proxi::Connection broadcasts the following events:
start_connection(Proxi::Connection)
end_connection(Proxi::Connection)
main_loop_error(Proxi::Connection, Exception)
data_in(Proxi::Connection, Array)
data_out(Proxi::Connection, Array)
class Connection
include Wisper::Publisher
attr_reader :in_socket, :thread
def initialize(in_socket, socket_factory, max_block_size: 4096)
@in_socket = in_socket
@socket_factory = socket_factory
@max_block_size = max_block_size
end
Connection#call
starts the connection handler thread. This is called by
the server, and spawns a new Thread that handles the forwarding of data.
def call
broadcast(:start_connection, self)
@thread = Thread.new { proxy_loop }
self
end
def alive?
thread.alive?
end
def join_thread
thread.join
end
private
def out_socket
@out_socket ||= @socket_factory.call
end
def proxy_loop
begin
handle_socket(in_socket)
loop do
begin
ready_sockets.each do |socket|
handle_socket(socket)
end
rescue EOFError
break
end
end
rescue Object => e
broadcast(:main_loop_error, self, e)
raise
ensure
in_socket.close rescue StandardError
out_socket.close rescue StandardError
broadcast(:end_connection, self)
end
end
def ready_sockets
IO.select([in_socket, out_socket]).first
end
def handle_socket(socket)
data = socket.readpartial(@max_block_size)
if socket == in_socket
broadcast(:data_in, self, data)
out_socket.write data
out_socket.flush
else
broadcast(:data_out, self, data)
in_socket.write data
in_socket.flush
end
end
end
end
module Proxi
This is the most vanilla type of socket factory.
Suitable when all requests need to be forwarded to the same host and port.
class TCPSocketFactory
def initialize(remote_host, remote_port)
@remote_host, @remote_port = remote_host, remote_port
end
def call
TCPSocket.new(@remote_host, @remote_port)
end
end
This will set up an encrypted (SSL, https) connection to the target host. This way the proxy server communicates unencrypted locally, but encrypts/decrypts communication with the remote host.
If you want to forward SSL connections as-is, use a TCPSocketFactory
, in
that case however you won't be able to inspect any data passing through,
since it will be encrypted.
class SSLSocketFactory < TCPSocketFactory
def call
OpenSSL::SSL::SSLSocket.new(super).tap(&:connect)
end
end
Dispatches HTTP traffic to multiple hosts, based on the HTTP Host:
header.
HTTPHostSocketFactory expects to receive data events from the connection, so
make sure you subscribe it to connection events. (see Proxi.http_proxy
for
an example).
To use this effectively, configure your local /etc/hosts
so the relevant
domains point to localhost. That way the proxy will be able to intercept
them.
This class is single use only! Create a new instance for each Proxi::Connection
.
class HTTPHostSocketFactory
Initialize a HTTPHostSocketFactory
host_mapping
- A Hash mapping hostnames to IP addresses, and, optionally, ports
For example:
HTTPHostSocketFactory.new(
'foo.example.com' => '10.10.10.1:8080',
'bar.example.com' => '10.10.10.2:8080'
)
def initialize(host_mapping)
@host_mapping = host_mapping
end
This is an event listener, it will be broadcast by the Connection
whenever
it gets new request data. We capture the first packet, assuming it
contains the HTTP headers.
Connection
will only request an outgoing socket from us (call #call
)
after it received the initial request payload.
def data_in(connection, data)
@first_packet ||= data
end
def call
host, port = @host_to_ip.fetch(headers["host"]).split(':')
port ||= 80
TCPSocket.new(host, port.to_i)
end
def headers
Hash[
@first_packet
.sub(/\r\n\r\n.*/m, '')
.each_line
.drop(1) # GET / HTTP/1.1
.map do |line|
k,v = line.split(':', 2)
[k.downcase, v.strip]
end
]
end
end
end
Proxi's server and connection classes don't have any logging or UI capabilities built in, but they broadcast events that we can listen to to perform these tasks.
module Proxi
This is a very basic console reporter to see what's happening
Subscribe to connection events, and you will see output that looks like this
1. +++
1. < 91
2. +++
3. +++
2. < 91
3. < 91
1. > 4096
1. > 3422
1. ---
Each connection gets a unique incremental number assigned, followed by:
class ConsoleReporter
def initialize
@count = 0
@mutex = Mutex.new
@connections = {}
end
def start_connection(conn)
@mutex.synchronize { @connections[conn] = (@count += 1) }
puts "#{@connections[conn]}. +++"
end
def end_connection(conn)
puts "#{@connections[conn]}. ---"
@connections.delete(conn)
end
def data_in(conn, data)
puts "#{@connections[conn]}. < #{data.size}"
end
def data_out(conn, data)
puts "#{@connections[conn]}. > #{data.size}"
end
def main_loop_error(conn, exc)
STDERR.puts "#{@connections[conn]}. #{exc.class} #{exc.message}"
STDERR.puts exc.backtrace
end
end
end
module Proxi
Log all incoming and outgoing traffic to files under /tmp
For example:
Proxi.tcp_server(...).subscribe(RequestResponseLogging.new).call
The in and outgoing traffic will be captured per connection in
/tmp/proxi.1.in
and /tmp/proxi.1.out
; /tmp/proxi.2.in
, etc.
class RequestResponseLogging
def initialize(dir: Dir.tmpdir, name: "proxi")
@dir = dir
@name = name
@count = 0
@mutex = Mutex.new
end
def new_connection(connection)
count = @mutex.synchronize { @count += 1 }
in_fd = File.open(log_name(count, "in"), 'w')
out_fd = File.open(log_name(count, "out"), 'w')
connection
.on(:data_in) { |_, data| in_fd.write(data) ; in_fd.flush }
.on(:data_out) { |_, data| out_fd.write(data) ; out_fd.flush }
.on(:end_connection) { in_fd.close ; out_fd.close }
end
def log_name(num, suffix)
'%s/%s.%d.%s' % [@dir, @name, num, suffix]
end
end
Wait before handing back data coming from the remote, this simulates a slow connection, and can be used to test timeouts.
class SlowDown
def initialize(wait_seconds: 5)
@wait_seconds = wait_seconds
end
def new_connection(connection)
connection.on(:data_out) { sleep @wait_seconds }
end
end
end