jmendeth2019-03-20T13:30:53-04:00https://jmendeth.comAlba MendezSSH tunnels done right2016-09-08T00:00:00-04:00https://jmendeth.com/blog/ssh-tunneling<p>In case you’ve never used them, SSH tunnels are freaking useful.
They turn SSH servers into temporary <em>proxies</em> so you can connect to hosts
you can’t reach directly. Keep reading.</p>
<p>The traditional way to create tunnels is by using the <code class="highlighter-rouge">-L</code> option of <code class="highlighter-rouge">ssh</code>.
Let’s say you want to manage your home’s AP. You usually do this by browsing
to <code class="highlighter-rouge">https://192.168.1.4</code>.</p>
<p>If you’re outside home, but are able to SSH into your home’s router, you’d type
something like:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ssh <router public IP> -L 8080:192.168.1.4:443
</code></pre>
</div>
<p>When it logs in, you’d browse to <code class="highlighter-rouge">https://localhost:8080</code> and see your AP’s
administration portal.</p>
<h2 id="the-problem">The problem</h2>
<p>All seems good, but your browser has no way of knowing that
<code class="highlighter-rouge">localhost:8080</code> is actually going to <code class="highlighter-rouge">https://192.168.1.4</code>, which means that:</p>
<ul>
<li>Static assets will need to be redownloaded and cached again.</li>
<li>If the website uses absolute URLs to load its resources, it won’t work.</li>
<li>Each time the website redirects you to <code class="highlighter-rouge">https://192.168.1.4/something</code>, you’ll
have to correct the URL manually.</li>
<li>You’ll have to login again, because the session cookies for <code class="highlighter-rouge">192.168.1.4</code>
will not be in effect.</li>
<li>Any previously saved passwords won’t apply.</li>
<li>URL autocomplete won’t be useful either.</li>
<li>The AP will not receive <code class="highlighter-rouge">Host: 192.168.1.4</code> as usual, but a confusing
<code class="highlighter-rouge">Host: localhost:8080</code>, which can give problems with virtual hosts.</li>
</ul>
<p>Fortunately, there’s a way out of all this.</p>
<h2 id="the-solution">The solution</h2>
<p>Instead of using <code class="highlighter-rouge">-L</code>, use <code class="highlighter-rouge">-D <port></code>. This instructs <code class="highlighter-rouge">ssh</code> to create a SOCKS
proxy that’ll tunnel every incoming connection through your SSH session.</p>
<p>So, staying with our AP example, you’d run something like:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ssh <router public IP> -D 49000
</code></pre>
</div>
<p>And set your browser to use the SOCKS proxy at <code class="highlighter-rouge">localhost</code> port 49000.
Then, just browse to <code class="highlighter-rouge">https://192.168.1.4</code> as you’d do at home!</p>
<p>Think about it. No more bashing against the wall until you realize you typed
<code class="highlighter-rouge">-L 8080:192.168.1.4:80</code> instead of <code class="highlighter-rouge">-L 8080:192.168.1.4:443</code>. No more seing
<code class="highlighter-rouge">ERR_CONNECTION_CLOSED</code> until you type in the «https://» before «localhost».
URL completion works. Redirects work. Absolute URLs work. You can browse just
as if you were at home.</p>
<p>It’s also more natural. You can first ssh into the router, <em>then</em> think of
what you want to access. If you then want to manage another device,
no need to kill ssh and run it again changing the <code class="highlighter-rouge">-L</code> option.</p>
<p>There’s one more perk: names are resolved from your router too, so if browsing
to <code class="highlighter-rouge">https://ap.lan</code> would work from home, it’ll also work with the SOCKS proxy.</p>
<p><strong>Isn’t this the way you always wanted tunneling to work?</strong></p>
<h2 id="tunneling-other-things">Tunneling other things</h2>
<p>We can see that using <code class="highlighter-rouge">-D</code> plays well with browsers. But not all tunnels are
made to access web portals. Another frequent use for SSH tunnels is SSH itself
(i.e. to SSH into an otherwise unreachable host).</p>
<p>I made a helper program that adds proxy support to SSH. Once installed (see
below) you can do:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ssh <router public IP> -D 49000
<span class="c"># on another terminal</span>
<span class="nb">export </span><span class="nv">all_proxy</span><span class="o">=</span><span class="s2">"socks://localhost:49000"</span>
ssh root@ap.lan
</code></pre>
</div>
<p>Again, compare with:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>ssh <router public IP> -L 8080:ap.lan:22
<span class="c"># on another terminal</span>
ssh -p 8080 root@localhost
</code></pre>
</div>
<p>When you do <code class="highlighter-rouge">ssh root@ap.lan</code>, ssh won’t ask you «Are you sure you want to
continue connecting (yes/no)?» since it knows it’s connecting to your AP.
And options in your <code class="highlighter-rouge">.ssh/config</code> set for your AP will be honored. This is
impossible with <code class="highlighter-rouge">-L</code>.</p>
<h2 id="ssh-from">ssh-from</h2>
<p>To make it even easier, I made a command called <a href="https://gist.github.com/jmendeth/346f2233310d8292efe7595d60aa3659"><code class="highlighter-rouge">ssh-from</code></a>.
You invoke it just like <code class="highlighter-rouge">ssh</code>, but instead of getting a remote
shell to issue commands in, you get a local shell “tunnelled” through that
host. When you’re finished, type <code class="highlighter-rouge">exit</code> to terminate the proxy.</p>
<p>Inside this “tunnelled” shell, <code class="highlighter-rouge">ssh</code> and derivatives like <code class="highlighter-rouge">scp</code>, <code class="highlighter-rouge">sftp</code>,
<code class="highlighter-rouge">sshfs</code> all work out of the box:</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-09-08-ssh-tunneling/example-ap-ssh.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-09-08-ssh-tunneling/example-ap-ssh.jpg" alt="Using ssh-from to SSH into the AP from outside." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Using ssh-from to SSH into the AP from outside.</blockquote></div>
<p>Chromium will also work if you launch it with
<code class="highlighter-rouge">chromium-browser --temp-profile --proxy-server=$all_proxy</code>:</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-09-08-ssh-tunneling/example-ap-web.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-09-08-ssh-tunneling/example-ap-web.jpg" alt="Using ssh-from to manage the AP through its website." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Using ssh-from to manage the AP through its website.</blockquote></div>
<p><code class="highlighter-rouge">ssh-from</code> is just <code class="highlighter-rouge">ssh</code>, so it’ll also work! This makes it really easy to
create double (or triple) SSH tunnels:</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-09-08-ssh-tunneling/example-double-ssh.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-09-08-ssh-tunneling/example-double-ssh.jpg" alt="Connecting to a laptop behind two routers, by nesting ssh-from sessions." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Connecting to a laptop behind two routers, by nesting ssh-from sessions.</blockquote></div>
<p>Some commands like <code class="highlighter-rouge">curl</code> will also work out of the box, but for commands
with no proxy support, <code class="highlighter-rouge">ssh-from</code> integrates with tsocks so just prefix<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup> them
by <code class="highlighter-rouge">tsocks</code>:</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-09-08-ssh-tunneling/example-telnet.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-09-08-ssh-tunneling/example-telnet.jpg" alt="Connecting by telnet to a host through ssh-from." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Connecting by telnet to a host through ssh-from.</blockquote></div>
<h2 id="installation">Installation</h2>
<p>Want to have <code class="highlighter-rouge">ssh-from</code> too? To install, make sure you have a compiler,
python, lsof and tsocks:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install build-essential python lsof tsocks
</code></pre>
</div>
<p>Then:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo wget https://gist.github.com/jmendeth/346f2233310d8292efe7595d60aa3659/raw/ssh-from.py -O /usr/local/bin/ssh-from <span class="o">&&</span> sudo chmod a+rx /usr/local/bin/ssh-from
wget https://gist.github.com/jmendeth/9b3c50226aa82a292a452107b34aca79/raw/ssh-proxy-dialer.c <span class="o">&&</span> cc ssh-proxy-dialer.c <span class="o">&&</span> sudo install a.out /usr/local/bin/ssh-proxy-dialer <span class="o">&&</span> rm ssh-proxy-dialer.c
</code></pre>
</div>
<p>Finally, put this at the end of <code class="highlighter-rouge">/etc/ssh/ssh_config</code>:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="n">Match</span> <span class="n">exec</span> <span class="s2">"ssh-proxy-dialer test"</span>
<span class="n">ProxyCommand</span> <span class="n">ssh</span>-<span class="n">proxy</span>-<span class="n">dialer</span> <span class="n">dial</span> <span class="s1">'%h'</span> <span class="s1">'%p'</span>
<span class="n">ProxyUseFdpass</span> <span class="n">yes</span>
</code></pre>
</div>
<p>Was <code class="highlighter-rouge">ssh-from</code> useful to you? Any suggestions?
Please let me know in the comments!</p>
<!-- TODO: mac support -->
<div class="footnotes">
<ol>
<li id="fn:1">
<p>A limitation of tsocks is that name resolution happens locally, not remotely, so you should specify IPs instead of internal names if using it. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
Telegram shell bot2016-05-09T00:00:00-04:00https://jmendeth.com/blog/telegram-shell-bot<p>If you use Telegram regularly, you’ve probably heard of <a href="https://core.telegram.org/bots">bots</a>. They’re like regular accounts, you can talk to them or have them in groups and they’ll do all sorts of useful things for you.</p>
<p>In this post, I present you the <a href="https://github.com/botgram/shell-bot">shell bot</a>! Unlike other bots, this one is self-hosted: you use it by running your own instance of the bot in your server.</p>
<p>This handy bot runs commands on demand and sends the live output, allowing you to interact at any time. It can even run graphical apps!</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-05-09-telegram-shell-bot/example-1.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-05-09-telegram-shell-bot/example-1.jpg" alt="The shell bot running some simple commands." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">The shell bot running some simple commands.</blockquote></div>
<p>It was made as an example for the <a href="https://github.com/botgram/botgram">botgram</a> framework, but I’ve been using it for some time and it’s proven to be very useful. When you’re on mobile and just want to issue a few commands, talking to a bot is probably more convenient than opening an SSH app and connecting to the server.</p>
<div class="article-img"><a href="https://jmendeth.com/media/2016-05-09-telegram-shell-bot/example-2.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2016-05-09-telegram-shell-bot/example-2.jpg" alt="Graphical application example, alsamixer in this case." /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Graphical application example, alsamixer in this case.</blockquote></div>
<p>It has creative uses as well: I once was troubleshooting a server problem with some friends, so we added the bot to the group and started playing around. Everyone could see what was going on, and suggest possible causes or things to try.</p>
<p>If you want to have your own shell bot, this post will guide you through the process of creating the Telegram bot itself, and installing the necessary software.</p>
<h2 id="create-your-bot">Create your bot</h2>
<p>First of all, you need to register a Telegram bot. Don’t worry, it only takes some seconds. <a href="https://t.me/BotFather">Click here</a> to talk to the BotFather, then say <code class="highlighter-rouge">/newbot</code> and you’ll be asked some things (name, username, etc.). At the end, you’ll be given an <strong>authentication token</strong> that looks like the following:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11
</code></pre>
</div>
<h2 id="preparations">Preparations</h2>
<blockquote>
<p><strong>Edit</strong>: The bot now lives in its own repository and the installation process has been simplified.
If you followed a previous version of this post, remove it and follow these steps from the start.</p>
</blockquote>
<p>The following should be done on the computer you want to run commands on. First of all, make sure you have <a href="https://nodejs.org">Node.JS</a> installed (you can verify by running <code class="highlighter-rouge">npm -v</code>). You’ll also need a working compiler and git:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install build-essential git
</code></pre>
</div>
<p>Clone the project and install dependencies:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>git clone https://github.com/botgram/shell-bot.git
<span class="nb">cd </span>shell-bot
npm install
</code></pre>
</div>
<p>Then run the bot for the first time, to create the configuration file:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>node server
</code></pre>
</div>
<p>It will ask you a set of questions, including the auth token that you got from the BotFather,
and will ask that you talk to your bot (so it can know who his owner is).</p>
<h2 id="run-it">Run it!</h2>
<p>Now that we have everything set up, we can run the shell bot:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>node server
</code></pre>
</div>
<p>Try saying <code class="highlighter-rouge">/run uname -a</code> for example, or say <code class="highlighter-rouge">/help</code> to learn about the available commands.</p>
<h2 id="autostart">Autostart</h2>
<p>Once you’ve played around, you may want the bot to start automatically on boot, and respawn if it crashes. For that, let’s install <a href="https://github.com/foreverjs/forever">forever</a>:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo npm install -g forever
</code></pre>
</div>
<p>Then, from your <code class="highlighter-rouge">/etc/rc.local</code> or an init script, call:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>forever start /path/to/shell-bot/server.js
</code></pre>
</div>
<p>Also, it’s a good idea to talk to the BotFather and say <code class="highlighter-rouge">/setcommands</code> to define a list of commands your bot accepts. You’ll find this list in <a href="https://github.com/botgram/shell-bot/blob/master/commands.txt"><code class="highlighter-rouge">commands.txt</code></a>; just paste the contents when asked. You may also want to change the bot’s profile photo and description.</p>
<p>Did you find the bot useful? Got any suggestions? Feel free to tell me about it in the comments!</p>
Understanding Architect2015-08-13T00:00:00-04:00https://jmendeth.com/blog/understanding-architect<p>If you have worked with <a href="https://c9.io">Cloud9</a>, you may have heard about
<a href="https://github.com/c9/architect">Architect</a>:</p>
<blockquote>
<p>Architect is a simple but powerful structure for Node.js applications.</p>
</blockquote>
<p>I have to admit, after reading the whole readme for the first time, I
still wasn’t seeing the potential that such a simple system hides. In
my opinion, the best way to present the benefits of a plugin-based app
will be with a practical example. Without further ado, meet…</p>
<h1 id="the-logger">The Logger™</h1>
<p>It’s not needed at all, but I suggest you to read <a href="https://learn.adafruit.com/low-power-wifi-datalogging">Low Power WiFi
Datalogger</a>. It covers
the creation of a battery-powered Arduino, that connects to your WiFi every
few minutes, and reports some measures.</p>
<p>Here, we’ll build the other part of the system—the server that collects
and accumulates measures from a series of these dataloggers.</p>
<p>So here’s the thing: we have ten dataloggers, which will regularly open a
TCP connection to us and report their readings (currently one). Our server
has to store all readings for the last ten hours and present them in a
nice website.</p>
<h2 id="planning">Planning</h2>
<p>We can see there are three basic parts in our application:</p>
<ul>
<li>The module that listens for connections from an Arduino, identifies it
and decodes the reading. We’ll call this the <strong>backend</strong>.</li>
<li>The module that stores the readings, associated with each logger.
This will be the <strong>storage</strong>.</li>
<li>The module that presents the readings in a nice UI.
This will be the <strong>frontend</strong>.</li>
</ul>
<p>We can also include a <strong>logger</strong> module where all modules can push messages
whenever something does not work as expected. This is our dependency graph,
in glorious ASCII art:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>+---------+ +---------+ +----------+
| backend |----------->>| storage |<<-----------| frontend |
+---------+ save +---------+ query +----------+
| | |
| log | log | log
| +--------+ | |
`------>>| logger |<<----+------------------------'
+--------+
</code></pre>
</div>
<p>Alright, time to start coding!</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir the-logger <span class="o">&&</span> <span class="nb">cd </span>the-logger
<span class="nb">echo</span> <span class="s2">"Datalogger server"</span> > README.md
wget http://jmendeth.mit-license.org -O LICENSE
npm install architect
</code></pre>
</div>
<p>We’ll start with <code class="highlighter-rouge">backend</code> module, or rather, plugin. Which takes us to…</p>
<h2 id="our-first-plugin">Our first plugin</h2>
<p>“Plugin” may sound like a bold word, but fear not! A plugin is simply
a <code class="highlighter-rouge">package.json</code> with some metadata and a JavaScript file with code.
That’s why they need to be in their own folders:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir backend <span class="o">&&</span> <span class="nb">cd </span>backend
</code></pre>
</div>
<p>The <code class="highlighter-rouge">package.json</code> will say: «This is called <code class="highlighter-rouge">backend</code>, it’s a plugin, and it
wants to use things from <code class="highlighter-rouge">storage</code>»</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"backend"</span><span class="p">,</span><span class="w">
</span><span class="nt">"plugin"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nt">"consumes"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"storage"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p>As always, you could go on and include a <code class="highlighter-rouge">description</code>, <code class="highlighter-rouge">version</code>, <code class="highlighter-rouge">keywords</code>,
<code class="highlighter-rouge">author</code>, etc. but as long as it has a <code class="highlighter-rouge">name</code> and a <code class="highlighter-rouge">plugin</code> section, it’s good.</p>
<p>We’ll now create <code class="highlighter-rouge">index.js</code> next to it, with the following boilerplate:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">options</span><span class="p">,</span> <span class="nx">imports</span><span class="p">,</span> <span class="nx">register</span><span class="p">)</span> <span class="p">{</span>
<span class="p">}</span></code></pre></figure>
<p>As you can see, <code class="highlighter-rouge">index.js</code> exports a single function, the constructor, which
creates a new instance of that plugin. It takes three parameters. The
first has user-provided preferences for this plugin. The second contains
the dependencies we asked for (in this case, only <code class="highlighter-rouge">storage</code>). And instead
of returning the result, it calls <code class="highlighter-rouge">register</code> with it. Example of how our
constructor may get called:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">constructor</span><span class="p">({</span> <span class="na">port</span><span class="p">:</span> <span class="mi">1940</span><span class="p">,</span> <span class="na">verbose</span><span class="p">:</span> <span class="kc">true</span> <span class="p">},</span> <span class="p">{</span> <span class="na">storage</span><span class="p">:</span> <span class="o"><</span><span class="nx">API</span><span class="o">></span> <span class="p">},</span> <span class="nx">result_callback</span><span class="p">);</span>
<span class="kd">function</span> <span class="nx">result_callback</span><span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">result</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">"Plugin failed to initalize!"</span><span class="p">);</span>
<span class="k">else</span>
<span class="c1">// continue...</span>
<span class="p">}</span></code></pre></figure>
<p>This plugin is really simple, it just creates a server and, whenever
a measure arrives, call a method of our <code class="highlighter-rouge">storage</code> plugin to store it.</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">options</span><span class="p">,</span> <span class="nx">imports</span><span class="p">,</span> <span class="nx">register</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">net</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s2">"server"</span><span class="p">);</span>
<span class="c1">// Retrieve our storage dependency</span>
<span class="kd">var</span> <span class="nx">storage</span> <span class="o">=</span> <span class="nx">imports</span><span class="p">.</span><span class="nx">storage</span><span class="p">;</span>
<span class="c1">// Create our server</span>
<span class="nx">net</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">client</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">client</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s2">"end"</span><span class="p">,</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Parse what it sent us</span>
<span class="kd">var</span> <span class="nx">parsed</span> <span class="o">=</span> <span class="nx">client</span><span class="p">.</span><span class="nx">read</span><span class="p">().</span><span class="nx">match</span><span class="p">(</span><span class="sr">/^</span><span class="se">(\d</span><span class="sr">+</span><span class="se">)\n</span><span class="sr">$/</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">measure</span> <span class="o">=</span> <span class="nb">parseInt</span><span class="p">(</span><span class="nx">parsed</span><span class="p">[</span><span class="mi">1</span><span class="p">]);</span>
<span class="c1">// Call the storage plugin to save the measure</span>
<span class="nx">storage</span><span class="p">.</span><span class="nx">storeMeasure</span><span class="p">(</span><span class="nx">client</span><span class="p">.</span><span class="nx">remoteAddress</span><span class="p">,</span> <span class="nx">measure</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">}).</span><span class="nx">listen</span><span class="p">(</span><span class="nx">options</span><span class="p">.</span><span class="nx">port</span> <span class="o">||</span> <span class="mi">8000</span><span class="p">);</span>
<span class="c1">// Tell whoever called us we're ready</span>
<span class="nx">register</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="p">{});</span>
<span class="p">}</span></code></pre></figure>
<p>This plugin does not provide any API for other plugins to use,
that’s why <code class="highlighter-rouge">register</code> is called with an empty object.</p>
<h2 id="our-second-plugin">Our second plugin</h2>
<p>Let’s now create the storage plugin. Because we’re just testing that
the whole thing works, and don’t want to bring in any database yet,
the plugin will just store the measures locally, in memory.</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="nb">cd</span> ..
mkdir <span class="nb">local</span>-storage <span class="o">&&</span> <span class="nb">cd local</span>-storage
</code></pre>
</div>
<p>This time, the <code class="highlighter-rouge">package.json</code> looks a bit different. This doesn’t depend on
any plugin, but it provides the <code class="highlighter-rouge">storage</code> API for other plugins to use.</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"local-storage"</span><span class="p">,</span><span class="w">
</span><span class="nt">"plugin"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nt">"provides"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"storage"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p>And <code class="highlighter-rouge">index.js</code>:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">options</span><span class="p">,</span> <span class="nx">imports</span><span class="p">,</span> <span class="nx">register</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">database</span> <span class="o">=</span> <span class="p">{};</span>
<span class="c1">// This is the API object that will be passed to other plugins</span>
<span class="kd">var</span> <span class="nx">api</span> <span class="o">=</span> <span class="p">{};</span>
<span class="c1">// Plugins will call this function to store a measure</span>
<span class="nx">api</span><span class="p">.</span><span class="nx">storeMeasure</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">ip</span><span class="p">,</span> <span class="nx">measure</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="p">(</span><span class="nx">ip</span> <span class="k">in</span> <span class="nx">database</span><span class="p">))</span> <span class="nx">database</span><span class="p">[</span><span class="nx">ip</span><span class="p">]</span> <span class="o">=</span> <span class="p">[];</span>
<span class="nx">database</span><span class="p">[</span><span class="nx">ip</span><span class="p">].</span><span class="nx">push</span><span class="p">({</span><span class="na">time</span><span class="p">:</span> <span class="nb">Date</span><span class="p">.</span><span class="nx">now</span><span class="p">(),</span> <span class="na">value</span><span class="p">:</span> <span class="nx">measure</span><span class="p">});</span>
<span class="p">};</span>
<span class="c1">// Plugins will call this function to query all stored</span>
<span class="c1">// measures for a given IP, sorted by time</span>
<span class="nx">api</span><span class="p">.</span><span class="nx">queryMeasures</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">ip</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">database</span><span class="p">[</span><span class="nx">ip</span><span class="p">];</span>
<span class="p">};</span>
<span class="c1">// Plugins will call this function to query all IPs we know</span>
<span class="nx">api</span><span class="p">.</span><span class="nx">listIPs</span> <span class="o">=</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="nb">Object</span><span class="p">.</span><span class="nx">keys</span><span class="p">(</span><span class="nx">database</span><span class="p">);</span>
<span class="p">};</span>
<span class="c1">// Periodically clean old measures from the database</span>
<span class="nx">setInterval</span><span class="p">(</span><span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">now</span> <span class="o">=</span> <span class="nb">Date</span><span class="p">.</span><span class="nx">now</span><span class="p">(),</span> <span class="nx">limit</span> <span class="o">=</span> <span class="nx">options</span><span class="p">.</span><span class="nx">limit</span> <span class="o">||</span> <span class="p">(</span><span class="mi">10</span> <span class="o">*</span> <span class="mi">3600</span> <span class="o">*</span> <span class="mi">1000</span><span class="p">);</span>
<span class="nx">database</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">entry</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">entry</span><span class="p">.</span><span class="nx">filter</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">m</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="p">(</span><span class="nx">now</span> <span class="o">-</span> <span class="nx">m</span><span class="p">.</span><span class="nx">time</span><span class="p">)</span> <span class="o">></span> <span class="nx">limit</span> <span class="p">});</span>
<span class="p">});</span>
<span class="p">},</span> <span class="nx">options</span><span class="p">.</span><span class="nx">cleanInterval</span> <span class="o">||</span> <span class="p">(</span><span class="mi">600</span> <span class="o">*</span> <span class="mi">1000</span><span class="p">));</span>
<span class="c1">// Done! Export our storage API for other plugins to use</span>
<span class="nx">register</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="p">{</span> <span class="na">storage</span><span class="p">:</span> <span class="nx">api</span> <span class="p">});</span>
<span class="p">}</span></code></pre></figure>
<p>Halt. There’s something important to note here. <code class="highlighter-rouge">backend</code> is <em>not</em> depending on
<code class="highlighter-rouge">local-storage</code>, it just wants us to give him a <code class="highlighter-rouge">storage</code> API to push measures
to. And this plugin we just wrote is a candidate of providing that API (the
only one, currently). This is called a <strong>soft dependency</strong>, and is the key
advantage of plugin-based apps. More on that later.</p>
<h2 id="the-third-plugin">The third plugin</h2>
<p>Let’s use Express for the webserver, and let’s make it quick.</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="nb">cd</span> ..
mkdir web-frontend <span class="o">&&</span> <span class="nb">cd </span>web-frontend
</code></pre>
</div>
<p>The <code class="highlighter-rouge">package.json</code> (by now you should know what’s coming):</p>
<figure class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"web-frontend"</span><span class="p">,</span><span class="w">
</span><span class="nt">"plugin"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nt">"consumes"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"storage"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span></code></pre></figure>
<p>And the <code class="highlighter-rouge">index.js</code>:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">options</span><span class="p">,</span> <span class="nx">imports</span><span class="p">,</span> <span class="nx">register</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">express</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s2">"express"</span><span class="p">);</span>
<span class="c1">// Retrieve our dependency</span>
<span class="kd">var</span> <span class="nx">storage</span> <span class="o">=</span> <span class="nx">imports</span><span class="p">.</span><span class="nx">storage</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">app</span> <span class="o">=</span> <span class="nx">express</span><span class="p">();</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">set</span><span class="p">(</span><span class="s2">"views"</span><span class="p">,</span> <span class="nx">__dirname</span> <span class="o">+</span> <span class="s2">"/views"</span><span class="p">);</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">set</span><span class="p">(</span><span class="s2">"view engine"</span><span class="p">,</span> <span class="s2">"your favorite view engine"</span><span class="p">);</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">get</span><span class="p">(</span><span class="s2">"/"</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">ips</span> <span class="o">=</span> <span class="nx">storage</span><span class="p">.</span><span class="nx">listIPs</span><span class="p">();</span>
<span class="nx">res</span><span class="p">.</span><span class="nx">render</span><span class="p">(</span><span class="s2">"list"</span><span class="p">,</span> <span class="p">{</span><span class="na">ips</span><span class="p">:</span> <span class="nx">ips</span><span class="p">});</span>
<span class="p">});</span>
<span class="nx">app</span><span class="p">.</span><span class="nx">get</span><span class="p">(</span><span class="s2">"/:ip"</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">,</span> <span class="nx">next</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">measures</span> <span class="o">=</span> <span class="nx">storage</span><span class="p">.</span><span class="nx">queryMeasures</span><span class="p">(</span><span class="nx">req</span><span class="p">.</span><span class="nx">params</span><span class="p">.</span><span class="nx">ip</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">measures</span><span class="p">)</span> <span class="k">return</span> <span class="nx">next</span><span class="p">();</span>
<span class="nx">res</span><span class="p">.</span><span class="nx">render</span><span class="p">(</span><span class="s2">"measures"</span><span class="p">,</span> <span class="p">{</span><span class="na">ip</span><span class="p">:</span> <span class="nx">req</span><span class="p">.</span><span class="nx">params</span><span class="p">.</span><span class="nx">ip</span><span class="p">,</span> <span class="na">measures</span><span class="p">:</span> <span class="nx">measures</span><span class="p">});</span>
<span class="p">});</span>
<span class="c1">// This plugin doesn't export any APIs either</span>
<span class="nx">register</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="p">{});</span>
<span class="p">}</span></code></pre></figure>
<p>Writing the views and choosing the view engine is out of the scope
of this article.</p>
<h2 id="launching-the-application">Launching the application</h2>
<p>At this point we have all the plugins we need to start the first version
of The Logger™. But there’s nothing we can actually run yet; something
has to call the create an instance of each plugin and put them together.</p>
<p>For that purpose let’s create <code class="highlighter-rouge">run.js</code> on the project root, next to the readme:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="c1">// Create storage, it's needed by the other plugins</span>
<span class="nx">require</span><span class="p">(</span><span class="s2">"./local-storage"</span><span class="p">)({},</span> <span class="p">{},</span> <span class="kd">function</span><span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">result</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="nx">process</span><span class="p">.</span><span class="nx">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="c1">// Create backend</span>
<span class="nx">require</span><span class="p">(</span><span class="s2">"./backend"</span><span class="p">)({},</span> <span class="p">{</span> <span class="na">storage</span><span class="p">:</span> <span class="nx">result</span><span class="p">.</span><span class="nx">storage</span> <span class="p">},</span> <span class="kd">function</span><span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">result</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="nx">process</span><span class="p">.</span><span class="nx">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">"Backend ready"</span><span class="p">);</span>
<span class="p">});</span>
<span class="c1">// Create frontend</span>
<span class="nx">require</span><span class="p">(</span><span class="s2">"./web-frontend"</span><span class="p">)({},</span> <span class="p">{</span> <span class="na">storage</span><span class="p">:</span> <span class="nx">result</span><span class="p">.</span><span class="nx">storage</span> <span class="p">},</span> <span class="kd">function</span><span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">result</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="nx">process</span><span class="p">.</span><span class="nx">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s2">"Frontend ready"</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">});</span></code></pre></figure>
<p>And that’s it, we have our app running! But it felt a bit tedious to write
that launcher, didn’t it? <strong>That’s where Architect comes in.</strong> Since we have
all the dependencies written in each <code class="highlighter-rouge">package.json</code>, we can replace the above
code with:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="kd">var</span> <span class="nx">architect</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s2">"architect"</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">config</span> <span class="o">=</span> <span class="nx">architect</span><span class="p">.</span><span class="nx">loadConfig</span><span class="p">(</span><span class="nx">__dirname</span> <span class="o">+</span> <span class="s2">"/config.js"</span><span class="p">);</span>
<span class="nx">architect</span><span class="p">.</span><span class="nx">createApp</span><span class="p">(</span><span class="nx">config</span><span class="p">);</span></code></pre></figure>
<p>And <code class="highlighter-rouge">config.js</code> just exports a list of plugins to load:</p>
<figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">[</span>
<span class="s2">"./backend"</span><span class="p">,</span>
<span class="s2">"./local-storage"</span><span class="p">,</span>
<span class="s2">"./web-frontend"</span><span class="p">,</span>
<span class="p">]</span></code></pre></figure>
<p>Then when we run <code class="highlighter-rouge">run.js</code>, Architect will figure out the right order in which
to create each plugin, making sure to satisfy all dependencies, avoiding
dependency cycles, etc. You only need to modify <code class="highlighter-rouge">config.js</code> as needed.</p>
<p>Now for some little practice: would you be able to add in a logger plugin,
and use it throughout the app? The plugin should just log messages to the
console.</p>
<h2 id="why-do-you-do-this-to-me">Why do you do this to me</h2>
<p>Now you are probably thinking: couldn’t we have just used modules and be done?
What do we gain by structuring our application in plugins?</p>
<p><strong>Scalability.</strong></p>
<p>Okay, now you want to take this application into a production environment.
Write a <code class="highlighter-rouge">mongodb-storage</code> plugin which saves the data to a MongoDB database.</p>
<p>Then you use <code class="highlighter-rouge">local-storage</code> or <code class="highlighter-rouge">mongodb-storage</code> depending on the environment
you’re on. As long as both export the same API, nothing changes for the rest
of the code.</p>
<p>Oh, your server is not very powerful and you don’t want to run any frontend
in it? Fine, disable the plugin. Or maybe you’d prefer the frontend to be
a REST API to query it from a big, central server? Write a REST frontend.</p>
<p>And what if you’re a user and don’t maintain the code, but just want
to hook up support for your custom datalogger? No problem.</p>
<p>Some plugins can be in other modules. Heck, you could have a freaking repo for
every plugin if you wanted. Anything <code class="highlighter-rouge">require()</code> and NPM can fetch works.
Here, the team that writes the Arduino code could also mantain the <code class="highlighter-rouge">backend</code>
plugin there.</p>
<p>Oh, and that also works the other way: other people can require our backend
plugin for their own purposes; as long as they provide a storage API.</p>
<p>This is just a subset of the advantages. I don’t want to make it seem like
plugins are a magic bullet for Node.JS applications. It’s far from that,
but they have allowed me to build complex designs very fast. I totally
recommend trying them with a more real example.</p>
<p>Happy coding!</p>
<p>PS: Want to know how to pass options to plugins? Register your own error
handler when a plugin fails? Curious to see the full Architect API? This and
more, in their <a href="https://github.com/c9/architect">repository</a>.</p>
Quick boot server2014-12-14T00:00:00-05:00https://jmendeth.com/blog/quick-boot-server<p>Time for a little tutorial!</p>
<p>When fixing computers I’d often like to boot a live-CD so I can freely inspect and play around,
instead of working with the shitty Windows they usually have.</p>
<p>But what if there isn’t a pen (or writable CD) at hand? Today we’re gonna setup a quick
<strong>boot server</strong> so that computers can boot from us! Wow.</p>
<p>Advantages of this method:</p>
<ul>
<li>No need to find a pen with enough space, you just need an Ethernet cable.</li>
<li>Quick to use. Set it up, connect the cable, boot via network and you’re done.</li>
<li>Also acts as a router, so when the computer boots it can access the Internet
through your connection.</li>
<li>Portable. Start and stop serving whenever you want.</li>
</ul>
<p>Disadvantages:</p>
<ul>
<li>Can be slower than a pen sometimes.</li>
<li>Getting the BIOS to cooperate and boot from network is, in my experience, tricky.</li>
</ul>
<h2 id="preparing">Preparing</h2>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install dnsmasq syslinux-common nfs-kernel-server
</code></pre>
</div>
<p>Now we have everything we need. But before continuing, edit <code class="highlighter-rouge">/etc/default/dnsmasq</code>
and set <code class="highlighter-rouge">ENABLED</code> to <code class="highlighter-rouge">0</code> to prevent dnsmasq to start automatically. Then, stop it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo service dnsmasq stop
</code></pre>
</div>
<p>And while we’re at, you should follow <a href="http://stackoverflow.com/questions/5321380/disable-network-manager-for-a-particular-interface#6810854">these steps</a>
to make sure NetworkManager won’t mess with the wired interface!</p>
<h2 id="basic-setup">Basic setup</h2>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir myserver <span class="o">&&</span> <span class="nb">cd </span>myserver
</code></pre>
</div>
<p>Okay, the first thing is to setup a simple DHCP server with dnsmasq.
Create <code class="highlighter-rouge">dnsmasq.conf</code> with this content:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>listen-address=192.168.99.1
bind-interfaces
dhcp-range=192.168.99.100,192.168.99.199,12h
dhcp-authoritative
</code></pre>
</div>
<p>Let’s now start dnsmasq to test the config works:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo ifconfig eth0 up 192.168.99.1/24
sudo dnsmasq -C dnsmasq.conf
</code></pre>
</div>
<p>Now connect an Ethernet cable to the other computer and verify it gets an IP as expected.
<strong>Bonus!</strong> Dnsmasq already forwards DNS requests and sets the gateway so you just need to:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo iptables -t nat -A POSTROUTING -s 192.168.99.1/24 -j MASQUERADE
</code></pre>
</div>
<p>and now the computer can also access the Internet through the NAT we just created!</p>
<h2 id="adding-tftp">Adding TFTP</h2>
<p>Now on to the interesting thing: network booting.</p>
<p>TFTP is frequently used by bootloaders to download the image when booting via network.
Let’s create the directory we’ll serve over TFTP:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir tftp
</code></pre>
</div>
<p>Now take the ISO of your favorite distribution (I recommend LUbuntu) and <em>extract</em> its contents
at the directory we just created:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir /tmp/iso
sudo mount -t iso9660 /path/to/lubuntu.iso /tmp/iso
cp -r /tmp/iso/<span class="k">*</span> tftp
sudo umount /tmp/iso
</code></pre>
</div>
<p>Now let’s put in <code class="highlighter-rouge">pxelinux.0</code>, which will be the invoked by the bootloader:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>cp /usr/lib/syslinux/pxelinux.0 tftp
</code></pre>
</div>
<p>And create the configuration files to tell pxelinux where to find the Linux kernel, and the options:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir tftp/pxelinux.cfg
<span class="nb">echo</span> <span class="s2">"DEFAULT casper/vmlinuz.efi initrd=casper/initrd.lz boot=casper root=/dev/nfs netboot=nfs nfsroot=192.168.99.1:/absolute/path/to/tftp quiet splash --"</span> > tftp/pxelinux.cfg/default
</code></pre>
</div>
<p>Make sure <code class="highlighter-rouge">casper/vmlinux.efi</code> exists within <code class="highlighter-rouge">tftp</code> and if not, change it as appropiate.
Same with <code class="highlighter-rouge">casper/initrd.lz</code>. Also change <code class="highlighter-rouge">/absolute/path/to/tftp</code> to, well, you know.</p>
<p>Make sure the paths have no spaces or strange characters in them.</p>
<h2 id="start-it">Start it!</h2>
<p>Let’s export the <code class="highlighter-rouge">tftp</code> folder through NFS (so that the booted Linux can access it):</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo tee -a /etc/exports <span class="o"><<<</span> <span class="s2">"/absolute/path/to/tftp 192.168.99.0/24(ro,no_subtree_check,fsid=DDD)"</span>
sudo service nfs-kernel-server restart
</code></pre>
</div>
<p>(where DDD is a random number)</p>
<p>And finally, let’s tell dnsmasq to serve TFTP! Add this to <code class="highlighter-rouge">dnsmasq.conf</code>:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>enable-tftp
dhcp-boot=pxelinux.0
tftp-root=/absolute/path/to/tftp
</code></pre>
</div>
<p>We’re done! Restart dnsmasq:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo killall dnsmasq
sudo dnsmasq -C dnsmasq.conf
</code></pre>
</div>
<p>Now it’s time to reboot the other computer, make it boot via network, and pray it’ll work.
Note: it takes some time to boot. Be patient.</p>
<h2 id="automating-everything">Automating everything</h2>
<p>Keep in mind this is designed to be temporal: we don’t want to turn our laptop into a
permanent boot server, instead we start and stop the server when needed.</p>
<p>Starting and stopping the server is three or two commands; we can create two scripts to
handle that for us.</p>
<p>Create <code class="highlighter-rouge">startserver</code> next to <code class="highlighter-rouge">dnsmasq.conf</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c">#!/bin/bash</span>
<span class="c"># Start a DHCP, DNS and TFTP boot server, and a NAT.</span>
<span class="nb">cd</span> <span class="k">$(</span>dirname <span class="nv">$0</span><span class="k">)</span>
sudo ifconfig eth0 up 192.168.99.1/24
sudo iptables -t nat -A POSTROUTING -s 192.168.99.0/24 -j MASQUERADE
sudo dnsmasq -C dnsmasq.conf</code></pre></figure>
<p>Create <code class="highlighter-rouge">stopserver</code> too:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c">#!/bin/bash</span>
<span class="c"># Start a DHCP, DNS and TFTP boot server, and a NAT.</span>
<span class="nb">cd</span> <span class="k">$(</span>dirname <span class="nv">$0</span><span class="k">)</span>
sudo killall dnsmasq
sudo iptables -t nat -F</code></pre></figure>
<p>Make them executable:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>chmod +x startserver stopserver
</code></pre>
</div>
<p>And that’s it! Next time you need to boot Linux on a computer (or need the NAT)
just do <code class="highlighter-rouge">./startserver</code> and connect the cable, and <code class="highlighter-rouge">./stopserver</code> when done.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I find myself using scripts like these much of the time I need to fix problems on
foreign computers, especially these two, and I hope they’ll be useful to someone else.</p>
<p>Keep in mind we’re not setting any firewalling, i.e. hosts from the “outside” can access
our NAT-ed hosts. Adding firewalling is left as an excercise to the reader.</p>
The Sandbox: part I2014-11-16T00:00:00-05:00https://jmendeth.com/blog/the-sandbox-part-one<p>Now’s my first year at university.</p>
<p>We’re given credentials so we can login into the computers around the campus, which run Ubuntu 12.04 LTS and —surprise!— have <em>not</em> been updated in a whole lot of time.</p>
<p>After using them for some time, the unevitable happened: It lacked my essential set of programs (Chromium or Node.JS or KmPlot for example) and I really
missed my laptop where I have admin rights and I can just <code class="highlighter-rouge">apt-get install</code> what I need. These computers boot via PXE, so I thought about bringing a Pi
to make them boot whatever I wanted, but it’d be a questionalbly legal approach which could get me into trouble.</p>
<p>I don’t want that. (Not yet.)</p>
<p>So I’m making this series of posts explaining the steps I did to get a bit of <strong>freedom</strong> within my restricted user, in the hope they’ll be useful for someone else.<br />
<strong>Spoiler:</strong> I could finally run my own, sandboxed Debian install inside my user, with great results! This is explained in the next parts.</p>
<h2 id="first-steps">First steps</h2>
<p>Let’s start with the basics. You don’t have root access so you can’t install binaries at <code class="highlighter-rouge">/usr/bin</code>; you need a directory that is always in <code class="highlighter-rouge">PATH</code>, where you can drop programs and run them as if they were system-wide commands.</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir -p ~/.local/bin
</code></pre>
</div>
<p>In case you didn’t know, there’s a convention that <code class="highlighter-rouge">.local</code> is like the <code class="highlighter-rouge">/usr/local</code> prefix but for individual users (<code class="highlighter-rouge">/usr/local/share</code> is equivalent to <code class="highlighter-rouge">~/.local/share</code>, same with <code class="highlighter-rouge">bin</code>, <code class="highlighter-rouge">lib</code>, …). We’ll be following this convention.</p>
<p>Now, edit <code class="highlighter-rouge">~/.profile</code> and append the code:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="nb">export </span><span class="nv">PATH</span><span class="o">=</span><span class="nv">$PATH</span>:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:<span class="nv">$HOME</span>/.local/bin
</code></pre>
</div>
<p>This will add our directory to <code class="highlighter-rouge">PATH</code> as well as the <code class="highlighter-rouge">sbin</code> directories where commands like <code class="highlighter-rouge">ifconfig</code> are stored.</p>
<blockquote>
<p><strong>Important:</strong> add it to <code class="highlighter-rouge">.profile</code>, not <code class="highlighter-rouge">.bashrc</code>. The earlier is session-wide, the latter only applies to your interactive Bash shells.</p>
</blockquote>
<p>And while we’re at it, what about having our own directory for <em>libraries</em> as well? Maybe our programs depend on libraries which are not installed. So let’s do it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir -p ~/.local/lib
</code></pre>
</div>
<p>Edit <code class="highlighter-rouge">~/.profile</code> and append:</p>
<div class="highlighter-rouge"><pre class="highlight"><code><span class="nb">export </span><span class="nv">LD_LIBRARY_PATH</span><span class="o">=</span><span class="nv">$LD_LIBRARY_PATH</span>:<span class="nv">$HOME</span>/.local/lib
</code></pre>
</div>
<p>Now the linker will also load libraries from our local directory.</p>
<p><strong>Great!</strong> Logout and login again to make changes effective.</p>
<h2 id="installing-packages">Installing packages</h2>
<p>Now to something more interesting: installing <code class="highlighter-rouge">.deb</code> packages. Let’s suppose we want to install the <code class="highlighter-rouge">tree</code> utility.</p>
<p><code class="highlighter-rouge">apt-get install tree</code> won’t work of course, but we can instead do <code class="highlighter-rouge">apt-get download tree</code>, which will just fetch the <code class="highlighter-rouge">.deb</code> package from the repos.</p>
<p>Then let’s extract the contents of the package in some directory:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir <span class="nb">test
</span>dpkg-deb -x tree_xxxxx_xxxx.deb <span class="nb">test</span>
<span class="c"># repeat for every downloaded package</span>
</code></pre>
</div>
<p>It’ll create a bunch of files in <code class="highlighter-rouge">test/usr/share</code> (documentation and manual pages), and the binary that we want at <code class="highlighter-rouge">test/usr/bin/tree</code>.
So we can take that binary and drop it in <code class="highlighter-rouge">.local/bin</code> and we should be able to run <code class="highlighter-rouge">tree</code> without problems now.</p>
<h4 id="what-if-i-cant">What if I can’t?</h4>
<p>That’s probably because, in order for <code class="highlighter-rouge">tree</code> to run, you must install some dependencies too.
Try to simulate how <code class="highlighter-rouge">apt-get</code> would install that package:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>apt-get -s install tree
</code></pre>
</div>
<p>Note the dependencies <code class="highlighter-rouge">apt-get</code> installs, and repeat the procedure above for each dependency.
Remember to copy any library you see to <code class="highlighter-rouge">.local/lib</code>!</p>
<h2 id="compiling-programs">Compiling programs</h2>
<p>Now, what if we want to install a program that isn’t on the repos, or you want to install a newer version?
I’d recommend you do something very similar to what we did above.</p>
<p>Compile it as you’d do on your computer (i.e. <code class="highlighter-rouge">./configure && make</code> or whatever it needs) but don’t do the final <code class="highlighter-rouge">make install</code> since we don’t have root access.</p>
<p>Instead, create an empty directory and tell <code class="highlighter-rouge">make</code> to install the files there:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>mkdir <span class="nb">test
</span><span class="nv">DESTDIR</span><span class="o">=</span><span class="nb">test </span>make install
</code></pre>
</div>
<p>Examine the files it installed, and copy what you need to the local directories as we did in the previous section.</p>
<p>Even if the program doesn’t use <code class="highlighter-rouge">make</code> as buildsystem, you should still try to invoke it with the <code class="highlighter-rouge">DESTDIR</code> variable set, as we did before.
It’s a pretty standard convention. If it still doesn’t work, just copy the files yourself and see if it works.</p>
<h2 id="conclusion">Conclusion</h2>
<p>We’re still limited, but we can at least install our own programs now.<br />
But there must be a better way…</p>
<p>This comes at part II!</p>
SnowShoe Stamp: first impressions2014-08-25T00:00:00-04:00https://jmendeth.com/blog/snowshoestamp<p>Today I saw <a href="http://snowshoestamp.com">SnowShoe Stamp</a>.</p>
<p>SnowShoe Stamp describes itself as a great way to link physical assets to
virtual ones. I have to admit it: when I heard of them, my first thought was
“Oh no, yet another company that rebrands NFC”.</p>
<p>Later I realized I was completely wrong.</p>
<p>SnowShoe is about stamps that are, well, stamped onto a touchscreen, and then
recognized by the device.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/eoTPAoKFKDo" frameborder="0" allowfullscreen=""></iframe>
<p>The idea behind SnowShoe Stamp is simple: the stamp touches the screen and
produces touches that allow the app to detect and identify it. No RF involved,
just a touchscreen.</p>
<p>I <em>absolutely love</em> the idea.</p>
<p>So I went to <a href="https://github.com/snowshoestamp">their Github</a> to learn more
about the software side (aka SDK) of their engine. Here’s what I found out.</p>
<blockquote>
<p>Note: this post is a draft. There may be some typos or wrong assumptions,
because I didn’t spend as much time verifying the info as I usually do.</p>
</blockquote>
<blockquote>
<p><strong>Edit:</strong> they finally released a JS SDK that eliminates the need for a stamp
screen, but the other issues explained here still apply.</p>
</blockquote>
<h2 id="code">Code</h2>
<p>Here’s the repo that allows one to “host a stamp screen”: <a href="https://github.com/snowshoestamp/touchy_sdk">Touchy SDK</a></p>
<p>When the user stamps in this webpage, the data is sent to a “Callback URL”.
To use this stamp screen, you copy the files and edit the <code class="highlighter-rouge">index.html</code> file to
insert your own callback / error URLs and background.</p>
<p>The idea of having to direct your user to a specific webpage, stamp, and then
go back to your application doesn’t feel right at all, but I supposed the logic
was too complex to be embedded in another webpage.</p>
<p>But it got worse.</p>
<p>Trashy code. Poorly formatted. Hard to understand. Almost no comments. Didn’t
follow good practices. I had to carefully read it a few times to know what each
variable was used for.</p>
<p>The important code is in <code class="highlighter-rouge">index.html</code>. It uses <a href="http://touchyjs.org">Touchy.js</a>
to wait for five simultaneous touches to be active, grabs the coordinates of
each of them, puts them in an object, and sends it as Base64-encoded JSON to
the Callback URL.</p>
<p>If anything other than five touches is detected for more than two seconds, or
detected more than two times, the user is redirected to an “Error URL”.</p>
<p><strong>It could have been better. Way better.</strong></p>
<p>I understand the developers are better at Ruby or Python than at Javascript,
but the stamp screen is the core of the project and deserved a better
implementation. It totally did.</p>
<p>But the client-side logic is so simple that I could implement it myself in
minutes, so I don’t care so much about the code.</p>
<h2 id="design">Design</h2>
<p>This is the real problem. I was expecting an SDK that allowed me to match
the points to the identity. Instead, I have to pass the points to <em>their</em> API
to do the matching.</p>
<p>I can’t work offline. I can’t do it locally. No, I have to wait for the request
to go to my server, my server makes an OAuth request to their API, their API
returns the response, my server acts upon that and the client receives the
feedback.</p>
<p>I wanted a company that would give me the hardware part (the stamps) and would
leave the software part to me, as happens with NFC tags, or the good ol’
<a href="https://en.wikipedia.org/wiki/Fiducial_marker">fiducials</a>! I didn’t want to
register an API key and hand out every request to them. And what if they suddenly
decide to start charging for their API?</p>
<p>That’s not an ideal business model. SnowShoe: You’re doing it wrong.</p>
<h2 id="its-magic">It’s magic</h2>
<p>SnowShoe markets their product like magic. They literally do.</p>
<blockquote>
<p>SnowShoe provides software that once integrated into your project, can identify the stamps when touched to the screen of a multi-touch device, allowing the magic to ensue.</p>
</blockquote>
<p>Anywhere in their website is actually explained that stamps produce five touches at
different positions, and the coordinates are sent to the server for matching.
It’s magic.</p>
<p>They never talk about touch coordinates or encoded JSON, they say “pattern”,
“digital identity” and “stamp observation data”. It’s magic.</p>
<p><em>Even the “How it Works” page is magic</em>, they mention the <code class="highlighter-rouge">data</code> parameter, but
<del>never make an indication to what is actually in that data</del>.<br />
<strong>Edit:</strong> they do, they say “points” are being captured.</p>
<p>I’m a developer, not a kid.</p>
<p>But here comes the worst part: <strong>it’s secret</strong>.</p>
<blockquote>
<p>SnowShoe makes individual pieces of plastic, where each one carries a different <em>secret digital identity</em>.</p>
</blockquote>
<blockquote>
<p>They are <em>nearly impossible to spoof</em>.</p>
</blockquote>
<p>So. You’re saying five bidimensional points are impossible to spoof? Wow.
I’m pretty confident that a stamp that has no moving parts, no electricity and
no actual interaction with the smartphone is vulnerable to <a href="https://en.wikipedia.org/wiki/Replay_attack">replay
attacks</a>.</p>
<p>So yeah, it’s impossible to spoof until it’s stamped. Then its identity is no
longer secret or unique. Again, magic.</p>
<h2 id="conclusion">Conclusion</h2>
<p>These stamps are a great idea, but I was really disappointed to see how the idea
is being implemented. This isn’t good for anyone.</p>
<p>Don’t get me wrong, I love what these guys are doing. But it makes me sad that
the project is taking that direction. If I could work with the company to make
a client-side engine from scratch, I’d do it without questioning.</p>
<p>If SnowShoe opens their magic to the public; if they let the people know how the
stamps actually work, great ideas could be implemented, like:</p>
<ul>
<li>
<p>Detecting the position of the stamp, to make the user stamp in different
spots of the screen.</p>
</li>
<li>
<p>Detecting the time a stamp is in the screen. Longer stamps give different
feedback or unlock hidden features.</p>
</li>
<li>
<p>Detecting the rotation of the stamp. The user could press and rotate the
stamp as if it was a key.</p>
</li>
</ul>
<p>Innovation like this would give SnowShoe the push it needs to become a success.</p>
Reverse-engineering Flash2014-05-17T00:00:00-04:00https://jmendeth.com/blog/reverse-engineering-flash<p>Flash has always intimidated me. Websites usually use it to evade inspection
(together with minified JS) or to make use of specific features (clipboard, memory, …).</p>
<p>Turns out, in practice Flash helps in reverse-engineering. This is because there
are few Flash obfuscators and people don’t think anyone is ever going to look
inside their SWFs, so they don’t use them. Sometimes I even find additional
<strong>debug info</strong>, like the complete filename of each source file, line numbers, etc.</p>
<p>Flash is high-level assembly, like Java. You get function names, parameter names,
class names, field names and the assembly is easy to understand once you’re
accostumed to it. That, plus the fact it runs in a sandboxed environment (just
like Java applets) makes it really easy to deal with.</p>
<p>There’s open-source, high quality software out there that allows for precise
manipulation of SWFs. But before we dive in, let’s talk briefly about the SWF.</p>
<h2 id="small-web-format">Small Web Format</h2>
<p>I don’t know much about the format, but every SWF consists of a <strong>header</strong>
(indicating, among other things, Flash version and compression) and then
a series of <strong>tags</strong>. A tag can contain other tags, text, controls, multimedia,
vector paths, compiled ActionScript or arbitrary binary content, to name a few.</p>
<p>If you have never programmed in ActionScript, there’s an important thing to
note. In Flash, classes “reference” objects on the SWF if the name matches
and they extend the correct class.</p>
<p>For example, if the SWF has a button named <code class="highlighter-rouge">example.Submit</code> and the ActionScript
declares a class named <code class="highlighter-rouge">Submit</code> on package <code class="highlighter-rouge">example</code> that extends
<code class="highlighter-rouge">flash.display.Button</code>, then adding event listeners on that class will add them
onto the original button, and so on.</p>
<p>Similarly for binary tags, declaring a class named <code class="highlighter-rouge">Payload</code> that extends
<code class="highlighter-rouge">flash.utils.BinaryArray</code> allows ActionScript to access the binary content of
a binary tag with the same name, that could be a hidden resource or a compressed
asset.</p>
<h2 id="actionscript-bytecode-abc">ActionScript ByteCode (ABC)</h2>
<p>ActionScript source is compiled to bytecode, that is run by the ActionScript
Virtual Machine. I strongly recommend you to read an
<a href="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2014-05-17-reverse-engineering-flash/avm2overview.pdf" target="_blank">overview of the AVM</a>
now, to be able to understand the assembly better.</p>
<p>ActionScript bytecode is placed into a <code class="highlighter-rouge">DoABC</code> tag on the SWF. An SWF can
contain multiple <code class="highlighter-rouge">DoABC</code> tags. When such a tag is found, the player loads the
bytecode, verifies it<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup> and runs it.</p>
<h2 id="setting-up">Setting up</h2>
<p>We’re going to install the software that will allow us to see inside SWFs.</p>
<h3 id="basic-things">Basic things</h3>
<p>We need a working D compiler. Better <a href="http://dlang.org/download.html">download it from the official site</a>, since the
APT version often causes trouble. Then, install it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo dpkg -i <path to the .deb>
</code></pre>
</div>
<p>Make sure <code class="highlighter-rouge">flashplugin-installer</code> is installed (not <code class="highlighter-rouge">adobe-flashplugin</code>):</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install flashplugin-installer
</code></pre>
</div>
<p>Git, the JDK, and LZMA development files are also needed:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install git openjdk-7-jdk liblzma-dev
</code></pre>
</div>
<h3 id="rabcdasmhttpsgithubcomcybershadowrabcdasm"><a href="https://github.com/CyberShadow/RABCDAsm">RABCDAsm</a></h3>
<div class="highlighter-rouge"><pre class="highlight"><code>git clone https://github.com/CyberShadow/RABCDAsm.git <span class="o">&&</span> <span class="nb">cd </span>RABCDAsm
rdmd build_rabcdasm.d
sudo install <span class="o">{</span>abc,swfbin<span class="o">}{</span><span class="nb">export</span>,replace<span class="o">}</span> rabc<span class="o">{</span>d,<span class="o">}</span>asm swf<span class="o">{</span>7z,lzma,de<span class="o">}</span>compress /usr/local/bin
</code></pre>
</div>
<p>RABCDAsm contains utilities for:</p>
<ul>
<li>Extracting ABC blocks from an SWF file (<code class="highlighter-rouge">abcexport</code>), and replacing them
(<code class="highlighter-rouge">abcreplace</code>).</li>
<li>Disassembling the ABC blocks into a well structured assembly language
(<code class="highlighter-rouge">rabcdasm</code>) and assembling them back (<code class="highlighter-rouge">rabcasm</code>).</li>
<li>Extracting binary tags from an SWF file (<code class="highlighter-rouge">swfbinexport</code>), and replacing them
(<code class="highlighter-rouge">swfbinreplace</code>). We’ve said earlier that these tags can contain any data,
and are often used to hide resources or whole SWFs.</li>
<li>Manual compression and decompression of an SWF file. All the other utilities
can deal with compressed SWF —there’s no need to decompress them first—
but these are provided for debugging and manual inspecting of SWFs.</li>
</ul>
<p>The code also allows for programmatic parsing and manipulation of SWFs and their
tags, as well as deep parsing and manipulation of ActionScript blocks. The
disassembler can be easily tuned to modify the formatting of the disassembly.</p>
<p>RABCDAsm is fast and resistent to any obfuscations applied to the bytecode.
It’s typically used like this:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>abcexport file.swf <span class="c"># will create file-0.abc, file-1.abc, file-2.abc, etc.</span>
rabcdasm file-0.abc <span class="c"># disassemble!</span>
rabcdasm file-1.abc
rabcdasm file-2.abc
</code></pre>
</div>
<p>Which disassembles each block in the directories <code class="highlighter-rouge">file-0</code>, <code class="highlighter-rouge">file-1</code>, <code class="highlighter-rouge">file-2</code>,
… After editing, to assemble the ABC and update the SWF:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>rabcasm file-0/file-0.asasm <span class="c"># assemble!</span>
rabcasm file-1/file-1.asasm
rabcasm file-2/file-2.asasm
abcreplace file.swf 0 file-0/file-0.asasm <span class="c"># replace the blocks</span>
abcreplace file.swf 1 file-1/file-1.asasm
abcreplace file.swf 2 file-2/file-2.asasm
</code></pre>
</div>
<h3 id="redasm-abchttpsgithubcomjmendethredasm-abc"><a href="https://github.com/jmendeth/redasm-abc">redasm-abc</a></h3>
<div class="highlighter-rouge"><pre class="highlight"><code>git clone --recurse https://github.com/jmendeth/redasm-abc.git <span class="o">&&</span> <span class="nb">cd </span>redasm-abc
rdmd build_redasm.d
sudo install redasm /usr/local/bin
</code></pre>
</div>
<p>redasm-abc is a simple assistant to RABCDAsm. It aims to remove the tedious
workflow you just saw. To use redasm-abc, put the SWF in an empty directory,
then just run:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>redasm
</code></pre>
</div>
<p>And it will disassemble all the blocks at <code class="highlighter-rouge">block-0</code>, <code class="highlighter-rouge">block-1</code>, <code class="highlighter-rouge">block-2</code>, …
When you have made changes and want to update the SWF, run again:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>redasm
</code></pre>
</div>
<p>And it will reassemble the files that have changed. It will work from everywhere
inside the directory of the SWF. It also creates a backup of the SWF, just in
case.</p>
<p>redasm-abc is <a href="http://showterm.io/f8e8416a2968828a75353">especially useful in SWFs with lots of blocks</a>,
and it doesn’t create intermediate files so it’s more comfortable to use.
Sometimes though, RABCDAsm utilities need to be used directly.</p>
<h3 id="flash-player-debuggerhttpwwwadobecomsupportflashplayerdownloadshtml"><a href="http://www.adobe.com/support/flashplayer/downloads.html">Flash Player debugger</a></h3>
<div class="highlighter-rouge"><pre class="highlight"><code>wget http://fpdownload.macromedia.com/pub/flashplayer/updaters/11/flashplayer_11_plugin_debug.i386.tar.gz
tar xzOf flashplayer_11_plugin_debug.i386.tar.gz libflashplayer.so > libflashplayer-debug.so
sudo install -m 644 libflashplayer-debug.so /usr/lib/flashplugin-installer
sudo update-alternatives --install <span class="s2">"/usr/lib/mozilla/plugins/flashplugin-alternative.so"</span> <span class="s2">"mozilla-flashplugin"</span> /usr/lib/flashplugin-installer/libflashplayer-debug.so 50
<span class="c"># only if you are running 64-bit</span>
sudo apt-get install libnss3:i386 nspluginwrapper
sudo nspluginwrapper -i /usr/lib/flashplugin-installer/libflashplayer-debug.so
sudo mv /usr/lib/mozilla/plugins/npwrapper.libflashplayer-debug.so /usr/lib/flashplugin-installer/libflashplayer-debug.so
</code></pre>
</div>
<p>The Flash Player content debugger is essential if you’re going to modify your
SWF. You get a nice error box showing the error instead of the player stopping
abruptly.</p>
<p>To switch between the regular Flash player and the debugger, do:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo update-alternatives --config mozilla-flashplugin
</code></pre>
</div>
<p>And restart the web browser to use it. <strong>Edit:</strong> Chromium recently dropped support for NSAPI,
so the flash debugger won’t work in it. Use another browser instead. If someone knows a way to
debug with PepperFlash, please post a comment!</p>
<p>Visit <code class="highlighter-rouge">about:plugins</code> to verify that the correct plugin has loaded.</p>
<h3 id="vizzyhttpcodegooglecompflash-tracer"><a href="http://code.google.com/p/flash-tracer">Vizzy</a></h3>
<blockquote>
<p>To install, <a href="http://code.google.com/p/flash-tracer/downloads">download</a> the ZIP for Linux and extract it.</p>
</blockquote>
<p>Vizzy is a small tool to display the Flash Player logs. You just run the JAR
and it shows highlighted real-time logs, allowing you to filter by keywords.</p>
<p>This is handy when you want to get some values from the SWF at runtime.
To see them in the logs, just <code class="highlighter-rouge">trace()</code> them:</p>
<pre><code class="language-asasm">findproperty QName(PackageNamespace(""), "trace")
; (push the value to the stack)
callpropvoid QName(PackageNamespace(""), "trace"), 1
</code></pre>
<h3 id="swftoolshttpswftoolsorg-optional"><a href="http://swftools.org">SWFTools</a> (optional)</h3>
<div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install swftools
</code></pre>
</div>
<p>They have some interesting utilities, namely:</p>
<ul>
<li><code class="highlighter-rouge">swfdump</code> parses the SWF and outputs a dump of its structure.
You can see which tags, sprites, IDs, are there, and at which offset
they’re found.</li>
<li><code class="highlighter-rouge">swfextract</code> extracts specific assets from an SWF (images, streams or whole
frames). You need to lookup their IDs through <code class="highlighter-rouge">swfdump</code> first.</li>
<li><code class="highlighter-rouge">swfstrings</code> extracts strings out of an SWF.</li>
</ul>
<p>I won’t go into their usage, that’s out of the scope of this post.
But the dump should be minimally intuitive to read, especially if
you have worked with Flash before.</p>
<h3 id="intercepting-proxy">Intercepting proxy</h3>
<p>Requests made by Flash aren’t usually logged on the Developer Tools console (even though
they’re cached by the browser) so you’ll often need a good MITM proxy to save SWF files,
see what other SWFs are being loaded and serve the reassembled copy instead.</p>
<p>I’ve been using MITMProxy (which works with HTTPS out of the box, and with IPTables you
can do transparent proxying) together with a hand-written Node proxy server, but I find
that too low-level.</p>
<p>Fiddler also has an alpha build for Linux that looks promising, but it isn’t open-source.</p>
<h3 id="other-software">Other software</h3>
<p>There are some other open-source utilities for SWFs, but I don’t consider them
to be of much use in reverse-engineering.
The <a href="http://libming.org">Ming library</a>, <a href="http://swfmill.org/">swfmill</a>,
<code class="highlighter-rouge">swfc</code> (part of SWFTools), the <a href="http://flex.apache.org">Flex toolkit</a>,
<a href="https://github.com/jindrapetrik/jpexs-decompiler">JPEXS</a> —that one might be
useful, but I haven’t tried it against obfuscated files—
<a href="http://nowrap.de/flasm.html">Flasm</a>, <a href="http://mtasc.org">MTASC</a>.</p>
<h2 id="some-tips">Some tips</h2>
<p>Put the SWF in his own directory and add the files to a Git repository
just after disassembling it:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>redasm
git init <span class="o">&&</span> git add -A
git commit -m <span class="s2">"disassemble SWF"</span>
</code></pre>
</div>
<p>Always run these commands when getting on an SWF, even if you’re only planning
to read the assembly. You’ll thank me later.</p>
<p>Save <a href="http://www.anotherbigidea.com/javaswf/avm2/AVM2Instructions.html">this page</a>
as a reference for the AVM instructions.
Also, the syntax used in the disassembly is explained in <a href="https://github.com/CyberShadow/RABCDAsm#syntax">the README</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>While it’s a bit tedious to read the disassembly, these tools really give us
a lot of control over the SWF, and the fact they’re open-source gives you the
ability to tune them or build on top of them (like I did with redasm-abc).</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>“Verification” means the code is checked for overflows, invalid jumps or other illegal operations. At any point is the SWF checked for a signature from the publisher, which can be done in Java. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>
How it all started2014-05-15T00:00:00-04:00https://jmendeth.com/blog/how-it-all-started<p>When I was three years old, my father would take me to the computer, which ran
Windows 95. He would open Word and set a big dingbat font, that had
animals instead of regular letters. I pressed the keys, and some dinosaurs,
elephants and zebras appeared on the screen. I was <em>amazed</em>, every time!</p>
<p>I liked to play with all the stuff I could find—faxes, an old amplifier,
PCs, TVs. My grandfather was an electrician, and I learned to connect RCDs
to light bulbs and plugs. Those times when I accidentally felt the 230 Volts
through my body didn’t stop me from trying again, no matter what my father said.</p>
<div class="article-img"><a href="https://jmendeth.com/media/2014-05-15-how-it-all-started/rcd.jpg" target="_blank"><img src="https://jmendeth.com/snapshot/4d9475cfb10af8142e331551dc9b91e1217dc8c6/media/2014-05-15-how-it-all-started/rcd.jpg" alt="Some of the RCDs I used. They're gathering dust now..." width="100%" /></a>
<blockquote style="border-left: 10px solid #D5D5D5;">Some of the RCDs I used. They're gathering dust now...</blockquote></div>
<p>With eight years, my uncle asked “Do you want to learn Java?”. I had no
idea what ‘java’ could mean. My uncles don’t usually make me presents; instead
they sit with me and we have some great time together. So I agreed.</p>
<p>When I discovered that ‘java’ allowed me to take control of the computer,
I couldn’t stop. It was <em>fascinating</em>. For the first time ever, I could tell
the computer what to do. I’d no longer be a mere consumer, <em>the computer
would work for me</em>.</p>
<p>What’s more, I learned the Internet had lots of great information and
tutorials, and I quickly saw Java wasn’t the only option, so I started exploring
other languages. But I had a dream: I wanted to find a way the computer could
interact with all the electrical circuits I made… That’d have to wait.</p>
<p>My emotional life hasn’t been easy, but I feel really happy now. I want to
learn about everything, and I can’t wait for university. I started teaching
JavaScript to my brother (eight or nine years old) and he likes it a lot!</p>
<p>So in the end, I want to thank all my family (<a href="/media/2014-05-15-how-it-all-started/cadi.jpg">dog</a>
included) for all their help. And a bunch of other people, which I consider to
be part of the family. You made me possible. Thank you so much.</p>