Slownet :-( , Fasternet? Raspinet!

The problem

My internet was under performing quite severly and I didn’t know why. Occasionally for periods of time from a few minutes to hours at a time our internet would become unusable.

First thing was isolating the problem . I installed all the monitoring tools I could to find to narrow down the problem to an application or device.

I managed to find the app causing all my issues, it was Google Photo‘s but there were no throttling options in the software to stop it.

Image result for google photos logo

I was honestly so surprised, but anytime a device in our house came home to land on the WiFi, we couldn’t browse the web until it was done uploading it’s photo’s.

The worst thing about the problem is that we love Google Photo’s and couldn’t live without. From the free uploads at a great quality to the AI on the image categorization and knowing all the photos I’m in with my favourite people.

Back to the cons! No internet for quite a substantial portion of time until videos and photos were uploaded snugly and safely to the web vault, was not something we wanted to live with.

What I wanted to solve

I wanted internet all the time, I didn’t want to manually have to throttle all connections on the network, install software on each device to do the same.

Image result for slow internet meme

I wanted something to automatically take care of an issue I couldn’t solve inside the Google Photo’s app. I had an idea that I could setup a device which would automatically share the bandwidth fairly between all our users and programs fairly.

The solution

The Raspberry Pi is an awesome piece of hardware, anytime I have a IT household problem to solve it usually has just what I need. From the earliest iteration it was a phenomenal piece of hardware in a bite size format and a teeny tiny price of just $35.

A brief investigation into the Raspberry Pi 3 showed that you can run it in a Access Point mode, and with the Ethernet wired into the network, the rest would be software.

Image result

The initial problem was to setup the Pi as a WiFi router, which is quite well documented on the web, I followed this guide.

This will help you setup a Wifi Access Point, DHCP and DNS and absolutely nothing was solved yet… no QoS.

Image result for qos logo

The last piece of the puzzle. I was actually quite surprised that a piece of software called IP Tables, which is normally used for directing traffic was actually able do quality management and throttling.

The IP Tables result was actually pretty great, but was quite complicated to setup, I modified a guide I found and … we now have the best internet we ever had at home, even though the line is quite slow, it’s never felt slow for us again.

A little bit more technical stuff

Using Traffic Control (tc) command you can setup discovery queues which allow traffic to be classified into classes.

tc qdisc add dev eth0 root handle 1: htb default 15
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 80kbit ceil 80kbit prio 0

Inside the classes you can classify traffic filters, which will put your traffic into buckets.

Once you have your traffic classified into buckets you start prioritizing which buckets will get bandwidth allocation.

tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10

And you can also prioritize certain packets, normally ones that are critical but never actually use a lot of bandwidth, some like SSH which is text only for controlling servers and SYN, which is handshaking packets, giving these the highest priorities should never take away from your top download speed, but does give you much better response times on websites.

iptables -t mangle -I PREROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 0x1
iptables -t mangle -I PREROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j RETURN

I did have a few issues getting it to start on boot, which was quite a pain due to all the commands, but did eventually get it to start up in init.d

Where to from here

I could have used some expensive off the shelf hardware to fix the problem, or buy faster internt, but it was actually great being able to take hardware I had around the house and fix a problem I was having.

This actually accomplished everything I wanted, but there were a few things that would have been nice to add. Since I had the WiFi router functioning on a full linux computer, getting some metrics off it would have been great.

  • View bandwidth usage by PC, Protocol, Country, Website.
  • Setup a user login and guest system for logins and turn this into an open hotspot, with only unused capacity being shared.
  • Thresholds for certain services, like limit Netflix automatically so it only streams at 720p and not 1080p.

Back to cat memes!

Image result for cat soul starve

Related imageImage result for high speed cat meme

Toe Dabbing in ESP8266

The start of something small

I’ve always had a dream of learning electronics and I’ve had a ESP8266 sitting around on my desk for a quite a while now.

By Sparkfun Electronics - https://c1.staticflickr.com/1/494/19681470919_9a9bcd5692_z.jpg, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=57331438
By Sparkfun Electronics – https://c1.staticflickr.com/1/494/19681470919_9a9bcd5692_z.jpg, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=57331438

This board is a marvel of technology, originally it was designed as a WiFi chip, but it was made so well and flexibly, that hackers learnt that you could program this small chip to do wonderful amazing low power things via the serial port using just the AT command set, which is how they used to talk to modems, way back when.

An entire eco-system of tools formed around this and a community of hardware aficionados who wanted to build small pervasive electronics were born.

The dream

My dream was always to build small ‘throw away’ electronic devices cheaply that could form some sort of network and provide access to metrics of some sort, light levels, temperatures, air pressure and layer them throughout a room and then build displays of this data. An example that I’ve seen recently isn’t quite this but it is similar in a popular supermarket chain that shows their electricity and water usage on real time updated graphs at the entrance.

I find this sort of information which is kinda mundane being turned into something useful, fascinating.

Turn it into a reality

To find a project there first needs to be something to do… so the simplest project for the ESP is to set it up as a WiFi Access Point, pretty advanced functionality for a single chip, or you can make it connect to an existing wifi network and display a button on a website to click on and off . Once it’s connected with this example above, it should serve a website to you regardless of where you try to browse, regardless of where you try to browse to.

The board I got has a micro usb port with a serial port chip, two buttons and easy pin outs for doing additional things like adding sensors or servos’

The first steps into the new brave world.

The chip arrives blank so I had to first program the firmware using a flasher and a ROM.

This was relatively scary as I always find my heart skips a beat when it comes time to click the flash button.

Once we have the firmware, which is only actually writing some code. I installed the Arduino IDE and a few plugins to make it work with the ESP8266.

I found the online code sample for doing exactly what I wanted and the code was simple enough to copy paste while still understanding every line, so that made it perfect.

Up and running.

With the code deployed and live on the ESP, I can fire up the wireless connection wizard on my PC (and my phone) and can connect to the new WiFi network (IoT — Free WiFi).

And it displays this page when you try to browse anything!

No too exciting, but pretty good for a tiny little board.

Where to from here ?

Now that I have an ESP8266 up and running and doing something useful, what more could I do ?

There’s some hardware like the Sonoff BasicĀ 

which can let you switch electrical switches on and off using a simple programmable interface.

Getting the whole house hooked up to be controlled from a mobile phone is something that I find fascinating, control sound, appliances, moods, lighting.

You could for instance have a temperature sensor which knows what times of day to activate a heater for when you get home to a nice and cosy lounge.

You can have movement sensors that could play music which follows you around the house, but only in the room you’re in.

The world becomes yours to control and gamify. A future I look forwards too and I’m glad I have dabbled my toes, and will definitely be back to dive in, in the coming years!