SSH connection’s channel limit and Node.js

Recently while rebuilding Outkept, I learned something interesting. I knew there was a channel limit for each SSH connection, what I didn’t know was that the default limit was so low (usually 8 channels per connection by default in the majority of ssh daemons).

In Outkept each sensor uses a channel, if I exclusively used one channel per sensor that meant that only 8 sensors per server was allowed.

This meant that I needed to implement a queue based pool of some sort, in order to “multiplex” the available channels. Something like the gist below, just pass it a SSH2 module connection object and use the send method.

This code does not feature stream error treatment, so complement it or only use it when missing something isn’t important.

In this use case, each sensor interval is user defined, if someone defined a few sensors with intervals in the millisecond range the queue is going to quickly fill forever. I monitor this alerting the user in the dashboard for a “queued up” scenario, in this situation there is only two solutions: increase the intervals or increase the channels limit.

Rebuilding Outkept

Outkept DashboardLast November I did a talk at Codebits about a private project that ended to be Outkept, it was basically a project that was build for a  specific use case and then I generalized it.

At the time I developed it in Java, but after the talk I decided to rebuild it for more generalist use cases and start everything entirely from scratch in node.js.

Outkept allows you to control and monitor a large server cluster, without needing to manually specify which and what servers to monitor/control. Instead you just define which subnets you want to control and then, using SSH, Outkept‘s crawlers will look for servers and what do they support.

Rebuilding it in node.js was awesome, allowing me to tackle my node.js skills and dig more into node’s scene while using a lot ‘earlier adopter’ tech.

Right now Outkept v2 supports everything the old version supported and even more, things are quicker and more fluid. New dashboard connects using shoe (sockjs) and the new system relies entirely in multiple node.js processes.

I will talk more about this project later, but right now I would love some feedback. The codebase is big and mostly uncommented, in the next posts will fix this and talk a bit about it.

If you want to give it a try just follow these instructions. If you need help you can reach me at petermdias@gmail.com

Disrupting Java apprentices with node.js

I started lecturing a few years ago, for a while it was a fulltime job. I lectured mainly in two programming languages: C in introductory classes (first semester) and Java for OOP, data structures and distributed systems classes (equally distributed along the bachelor degree).

After a while I got bored and nowadays being a lecturer is not a fulltime job, my main contribution now is to bring fresh tech into the play and introduce it to seniors and other lecturers and this is where this story starts.

nodejs_logo

About one year ago I started diving into node.js. Initially I only used it for non critical stuff, still using Java as my main language.
I really liked node from the first day but I just didn’t have the time to dive into it at the time.
About 6 months ago everything changed, I’m really hooked into it and now I finally feel that I acquired enough skill to allow it to replace Java in my head.
Node ecosystem is awesome and it doesn’t have all that clutter that Java has, it is so clean and agile…

This semester I decided to bring node.js into a senior class and simultaneously a few senior projects.

Remember how I started this post?

In here almost every design/programming topics are taught Java, when I gave the first class about JavaScript and node.js everyone was like “WTF”?
A few students even looked at me thinking I was drunk because in their heads Javascript was for browsers and nothing else.

In order to sell them node.js I live coded a drag-n-drop DIV element that moved simultaneously in all browsers using socket.io. Instantly everyone on the room was curious about this, even the more sceptic ones. (if you want to disrupt someone show them something very graphic/explicit and that I did)
Code available at https://github.com/apocas/psi2013-node (it uses prototype based objects, modules, and events)

It’s true that I could have implemented that in almost any other language, but the lack effort needed to implement it in node.js is really impressive and that was what disrupted the audience (npm awesomeness helped :-D ).

Right now seniors are starting to put their hands on node.js at multiple projects (https://github.com/portugol - Portugol rewritten in node.js) so far I feel that the hardest thing for them is the asynchronous architecture.

Although one big advantage I felt was the fact that they came from a language where everything is an Object (Java), because of this they quickly understood objects in JavaScript and how can events be used for message passing in an asynchronous environment.

In my opinion this is one of the most important thing to understand in the Javascript/node.js world.

cPanel + logster + graphite

cPanel servers use one accesslog per virtualhost, which they call domlogs.

This can be problematic if you want to grab metrics from your logs. Tools like logster (metrics), beaver (log collection), … are usually designed to tap one single log file.

Some months ago I rewrote beaver, implemented daemon mode, multi-file and threading. At the time it seemed a good idea, but multi-file and daemon mode is just a resource hog when you have a server with ~300 vhosts.
From now on I follow Etsy advice which is, stay with simple and cron based log processing tools.

If you just want metrics, logster is the way to go you just have to find a way to parse all domlogs.

Below is a small bash script, that finds all the access logs and push their metrics to graphite using logster.

#!/bin/bash

for file in `find /usr/local/apache/domlogs/ -type f ! -name *.gz ! -name *-ftp_log ! -name *bytes_log ! -name *.offset* ! -name *-ssl*`
do
    aux=${file//./_}
    domain=${aux##*/}
    hostname=`/bin/hostname`
    hostname=${hostname//./_}

    logster -p servers.$hostname.$domain --output=graphite --graphite-host=XXX.XXX.XXX.XXX:2003 SampleLogster $file
done

Graphite

You just have to run this script in a cronjob, cron interval will depend on how many vhosts you have and how much delay you consider acceptable in your metrics.
In graphite this will create a folder for each server, then there will be a sub-folder for each virtualhost. All this is automatically created.

I will not cover graphite in this post, installing logster in a cPanel however may require some work.

The easiest way is to install logcheck from the epel repo, do not enable this repo globally.

The problem is, to install logcheck you need some perl dependencies, which cPanel will not allow you to install them.

exclude=apache* bind-chroot courier* dovecot* exim* filesystem httpd* mod_ssl* mydns* mysql* nsd* perl* php* proftpd* pure-ftpd* ruby* spamassassin* squirrelmail*

To install them you need to edit /etc/yum.conf and remove the “perl*” exclusion, install logtail and then enable the exclusion again.

Backbone.js and i18n


backbonei18n
A few weeks ago I had to support i18n in a backbone.js project I’m developing. Many doubts quickly appeared, when to load the i18n’ed content? how? where to define it? in which format? …

I started looking into JavaScript i18n libraries/frameworks, and decided to take a look at three: i18next, jed and jsperanto.
Jed looks really good and is truly the closest thing you can get to gettext in JavaScript.

But this time I needed something simpler which could allow me to quickly delegate translations to someone without explaining all these concepts about pluralization and interpolation since the i18n that needed to be done was really simple and small.

From my perspective jsperanto is like i18next on steroids. (edit: not that jsperanto supported more features, but it felt more complex due to the lack of documentation in comparison to i18next)

I ended up using i18next, it was clean and simple solution for a simple problem with basic i18n necessities.

Initialization

Since i18next features async initialization, it is the first thing initialized and then the main backbone.js router is initialized in its callback. Something like this:

$.i18n.init({
    lng: "en-US",
    ...
}, function(t) {
    app = new Router();
    Backbone.history.start();
});

Since the main router is initialized in i18next initialization callback, I will have i18next initialized inside the app scope allowing me to call the i18n main method.

Translation

With initialization done, the only thing left was content translation.
Which in i18next is accomplished by inner html replacement in DOM elements that contain the specified i18next attribute.

I didn’t want to blind translate everything. Instead I decided to translate each backbone.js view in its rendering final stage.
Something like this:

window.ProfileView = Backbone.View.extend({
    ...
    render: function() {
        $(this.el).html(this.template(this.model.toJSON()));

        $('#profile_container', this.el).html(new ProfileInfoView({
            model: this.model
        }).render().el);

        $('.content_sub_menu', this.el).i18n();

        return this;
    }
});
Follow

Get every new post delivered to your Inbox.

Join 135 other followers