Polipo: Easy Transparent proxy

When your server numbers grow and you have clusters you want to upgrade all at one time, you will probably want some kind of proxy that will cache stuff for you. Upgrading cluster of hadoop with like 10 servers, can take up to 2hours just to download packages to all nodes, since Cloudera limits bandwidth to 125kB/s per download. This can be pretty painful without proxy and your downtime would increase a lot. There are several other occasions where you can benefit from proxy:

  • upgrading distributions
  • updating your servers to latest packages
  • downloading packages with yumdownloader etc.

To speed things up, we need really small and easy to setup proxy with caching ability. Introducing Polipo. Unlike Squid which is total pain to configure and set up, Polipo is quick and easy. I got it running in under 10 minutes (i wanted to read all config file), but you can simply copy my configuration and get it up straight a way.

Installing polipo

Polipo is included in Epel 6 repository. To get it, we first need to install Epel repository and then install Polipo:

rpm -Uvh http://mirror.uv.es/mirror/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
yum install polipo
Configuring polipo

Grab following config file and copy it into /etc/polipo/config Be sure to edit:

  • allowedClients to your server’s network range
  • proxyAddress (0.0.0.0 would listen to all interfaces)
  • dnsQueryIPv6 in case you use IPv6
daemonise = true
pidFile = /var/run/polipo/polipo.pid
proxyAddress = "0.0.0.0"
allowedClients ="xxx.xxx.xxx.xxx/xx"
cacheIsShared = true
chunkHighMark = 50331648
objectHighMark = 16384
dnsQueryIPv6 = no

We should simply start our web proxy now with: service polipo start

To make sure our yum starts using our proxy, we should add a line to /etc/yum.conf on all client servers we want to use with this proxy server:

proxy=http://hostname_or_ip_of_your_polipo_server:8123

To test your proxy works perfectly, try downloading some large file with yumdownloader for example: yumdownloader libreoffice-core First time it actually downloads package from repository, but if you repeat this command on another server, download should start directly from proxy server and therefore really fast.

Comments

  1. Reply

  2. Reply

Leave a Reply

help-hint.png
Purpose of the commenting system is to share your experience. I encourage you to post feedback with your own suggestions, ideas or optimizations regarding the topic of a blog post. What commenting system isn't for, is asking questions about similar issues of yours and requesting support for it. Blog post is provided as is and I am not here to solve all your problems. Please bear that in mind and try to avoid posting such comments. I do take privilege to remove comment from my blog for any reason whatsoever. Usually I do it when I sense a comment was posted only for spam/seo reasons or is out of blog post's topic. Thank you for reading this, now you may continue :)
 

Your email address will not be published. Required fields are marked *