hadoop Archive

Polipo: Easy Transparent proxy

When your server numbers grow and you have clusters you want to upgrade all at one time, you will probably want some kind of proxy that will cache stuff for you. Upgrading cluster of hadoop with like 10 servers, can take up to 2hours just to download packages to all nodes, since Cloudera limits bandwidth

Leap second, Java and NTP leads to disaster – how to setup ntpd to avoid that

Like many other companies around the globe we also had some issues with last leap second. We couldn’t figure out why is our hadoop cluster acting strangely and using almost all CPU. After a while of browsing we found out, that the real cause of this was Java and leap second. As you may know