pages tagged openwrtFeeding the Cloudhttps://feeding.cloud.geek.nz/tags/openwrt/Feeding the Cloudikiwiki2023-10-28T23:54:26ZRemote logging of Turris Omnia log messages using syslog-ng and rsysloghttps://feeding.cloud.geek.nz/posts/remote-logging-turris-omnia-router/
<a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>
2023-10-28T23:54:26Z2022-07-02T03:45:00Z
<p>As part of debugging an upstream connection problem I've been seeing
recently, I wanted to be able to monitor the logs from my <a href="https://www.turris.com/en/omnia/overview/">Turris
Omnia</a> router. Here's how I
configured it to send its logs to a server I already had on the local
network.</p>
<h2 id="Server_setup">Server setup</h2>
<p>The first thing I did was to open up my server's
<a href="https://www.rsyslog.com/">rsyslog</a> (Debian's default syslog server) to
remote connections since it's going to be the destination host for the
router's log messages.</p>
<p>I added the following to <code>/etc/rsyslog.d/router.conf</code>:</p>
<pre><code>module(load="imtcp")
input(type="imtcp" port="514")
if $fromhost-ip == '192.168.1.1' then {
if $syslogseverity <= 5 then {
action(type="omfile" file="/var/log/router.log")
}
stop
}
</code></pre>
<p>This is using the latest rsyslog configuration method: a handy scripting
language called
<a href="https://www.rsyslog.com/doc/v8-stable/rainerscript/index.html">RainerScript</a>.
<a href="https://wiki.archlinux.org/title/Rsyslog#Severity_levels">Severity level</a> 5
maps to "notice" which consists of <em>unusual non-error conditions</em>, and
<code>192.168.1.1</code> is of course the IP address of the router on the LAN side.
With this, I'm directing all router log messages to a separate file,
filtering out anything less important than severity 5.</p>
<p>In order for rsyslog to pick up this new configuration file, I restarted it:</p>
<pre><code>systemctl restart rsyslog.service
</code></pre>
<p>and checked that it was running correctly (e.g. no syntax errors in the new
config file) using:</p>
<pre><code>systemctl status rsyslog.service
</code></pre>
<p>Since I added a new log file, I also setup log rotation for it by putting
the following in <code>/etc/logrotate.d/router</code>:</p>
<pre><code>/var/log/router.log
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}
</code></pre>
<p>In addition, since I use
<a href="https://packages.debian.org/stable/logcheck">logcheck</a> to monitor my server
logs and email me errors, I had to add <code>/var/log/router.log</code> to
<code>/etc/logcheck/logcheck.logfiles</code>.</p>
<p>Finally I opened the rsyslog port to the router in my server's firewall by
adding the following to <code>/etc/network/iptables.up.rules</code>:</p>
<pre><code># Allow logs from the router
-A INPUT -s 192.168.1.1 -p tcp --dport 514 -j ACCEPT
</code></pre>
<p>and ran <code>iptables-apply</code>.</p>
<p>With all of this in place, it was time to get the router to send messages.</p>
<h2 id="Router_setup">Router setup</h2>
<p>As <a href="https://forum.turris.cz/t/remote-log-how-to-configure/992/3">suggested on the Turris
forum</a>, I
ssh'ed into my router and added this in <code>/etc/syslog-ng.d/remote.conf</code>:</p>
<pre><code>destination d_loghost {
network("192.168.1.200" time-zone("America/Vancouver"));
};
source dns {
file("/var/log/resolver");
};
log {
source(src);
source(net);
source(kernel);
source(dns);
destination(d_loghost);
};
</code></pre>
<p>Setting the <a href="https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.33/administration-guide/time-zone">timezone</a> to the same as my server was needed because the router
messages were otherwise sent with UTC timestamps.</p>
<p>To ensure that the destination host always gets the same IP address
(<code>192.168.1.200</code>), I went to the <a href="https://192.168.1.1/cgi-bin/luci/admin/network/dhcp">advanced DHCP configuration
page</a> and added a
<em>static lease</em> for the server's MAC address so that it always gets assigned
<code>192.168.1.200</code>. If that wasn't already the server's IP address, you'll have
to restart it for this to take effect.</p>
<p>Finally, I restarted the syslog-ng daemon on the router to pick up the new
config file:</p>
<pre><code>/etc/init.d/syslog-ng restart
</code></pre>
<h2 id="Testing">Testing</h2>
<p>In order to test this configuration, I opened three terminal windows:</p>
<ol>
<li><code>tail -f /var/log/syslog</code> on the server</li>
<li><code>tail -f /var/log/router.log</code> on the server</li>
<li><code>tail -f /var/log/messages</code> on the router</li>
</ol>
<p>I immediately started to see messages from the router in the third window
and some of these, not all because of my severity-5 filter, were flowing to
the second window as well. Also important is that none of the messages make
it to the first window, otherwise log messages from the router would be mixed
in with the server's own logs. That's the purpose of the <code>stop</code> command in
<code>/etc/rsyslog.d/router.conf</code>.</p>
<p>To force a log messages to be emitted by the router, simply ssh into it and
issue the following command:</p>
<pre><code>logger Test
</code></pre>
<p>It should show up in the second and third windows immediately if you've got
everything setup correctly</p>
<h2 id="Timezone_problems">Timezone problems</h2>
<p>If I do the following on my router:</p>
<p> /etc/init.d/syslog-ng restart
logger TestA</p>
<p>I see the following in <code>/var/log/messages</code>:</p>
<p> Aug 14 20:39:35 hostname syslog-ng[9860]: syslog-ng shutting down; version='3.37.1'
Aug 14 20:39:36 hostname syslog-ng[10024]: syslog-ng starting up; version='3.37.1'
Aug 15 03:39:49 hostname root: TestA</p>
<p>The correct timezone is the one in the first two lines. Other daemon
messages are displayed using an incorrect timezone like <code>logger</code>.</p>
<p>Thanks to a <a href="https://lists.balabit.hu/pipermail/syslog-ng/2022-August/thread.html#26509">very helpful <code>syslog-ng</code> mailing list thread</a>, I found that this is actually an <a href="https://gitlab.nic.cz/turris/os/packages/-/issues/471">upstream OpenWRT bug</a>.</p>
<p>My favourite work-around is to tell syslog-ng to simply ignore the timestamp
provided by the application and to use the time of reception (of the log
message) instead. To do this, simply change the following in
<code>/etc/syslog-ng.conf</code>:</p>
<pre><code>source src {
internal();
unix-dgram("/dev/log");
};
</code></pre>
<p>to:</p>
<pre><code>source src {
internal();
unix-dgram("/dev/log", keep-timestamp(no));
};
</code></pre>
<p>Unfortunately, I wasn't able to fix it in a way that would survive a
<code>syslog-ng</code> package update, but since this is supposedly fixed in Turris 6.0,
it shouldn't be a problem for much longer.</p>
Upgrading the Wi-Fi cards in a Turris Omnia 2020https://feeding.cloud.geek.nz/posts/upgrading-wifi-cards-turris-omnia/
<a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>
2022-04-10T07:41:52Z2022-04-10T08:30:00Z
<p>I've been <a href="https://twitter.com/fmarier/status/1204117367149645825">very
happy</a> with my
<a href="https://www.turris.com/en/omnia/overview/">Turris Omnia router</a> and decided
recently to take advantage of the fact that is is easily upgradable to
replace the original radios for <a href="https://en.wikipedia.org/wiki/IEEE_802.11ac-2013#Wave_1_vs._Wave_2">Wave
2</a>
models.</p>
<p>I didn't go for a <a href="https://en.wikipedia.org/wiki/Wi-Fi_6">Wi-Fi 6</a>-capable
card because I don't have any devices that support it at this point. There
is also an <a href="https://youtu.be/qiuLtIqjK-U?t=390">official WiFi 6 upgrade kit</a>
in the works and so I might just go with that later.</p>
<h2 id="Wi-Fi_card_selection">Wi-Fi card selection</h2>
<p>After seeing a report that <a href="https://forum.turris.cz/t/what-mini-pci-express-version-is-omnia-turris-using/11691/2?u=fmarier">someone was already using these cards on the
Omnia</a>,
I decided to look for the following:</p>
<ul>
<li><a href="https://compex.com.sg/shop/wifi-module/802-11ac-wave-2/wle1216v2-20-2/">Compex WLE1216V2-20</a></li>
<li><a href="https://compex.com.sg/shop/wifi-module/802-11ac-wave-2/wle1216v5-20-2/">Compex WLE1216V5-20</a></li>
</ul>
<p>Compex themselves don't appear to sell to consumers, but I found an American
store that would sell them to me and ship to Canada:</p>
<ul>
<li><a href="https://www.embeddedworks.net/wlan721/">https://www.embeddedworks.net/wlan721/</a></li>
<li><a href="https://www.embeddedworks.net/wlan720/">https://www.embeddedworks.net/wlan720/</a></li>
</ul>
<p>Each card uses 4 antennas, which means that I would need an additional
diplexer, an extra pigtail to SMA-RP connector, and two more antennas to
wire everything up. Thankfully, the Omnia already comes with two extra holes
drilled into the back of the router (covered by plastic caps) and so there
is no need for drilling the case.</p>
<p>I put the two cards in the middle and right-most slots (they don't seem to
go in the left-most slot because of the SIM card holder being in the way)
without worrying about antennas just yet.</p>
<h2 id="Driver_installation">Driver installation</h2>
<p>I made sure that the chipsets were supported in <a href="https://openwrt.org/releases/19.07/start">OpenWRT
19.07</a> (LTS 4.14 kernel) and found
that support for the <a href="https://www.qualcomm.com/products/technology/wi-fi/qca9984">Qualcomm
QCA9984</a> chipset
was <a href="https://github.com/torvalds/linux/commit/651b4cdcf97e75f6346784b75ca7bf3c85187143">added in the <code>ath10k</code> driver as of kernel
4.8</a>
but <a href="https://wireless.wiki.kernel.org/en/users/drivers/ath10k#supported_devices">only for two cards
apparently</a>.</p>
<p>I installed the following proprietary firmware package via the <a href="https://192.168.1.1/cgi-bin/luci/admin/system/opkg">advanced
configuration
interface</a>:</p>
<pre><code>ath10k-firmware-qca9984
</code></pre>
<p>and that automatically pulled in the free <code>ath10k</code> driver. After rebooting,
I was able to see one of the two cards in the ReForis admin page and
configure it.</p>
<p>Note that there is an <a href="https://www.candelatech.com/ath10k-10.4.php">alternative
firmware</a> available in OpenWRT
as well (look for packages ending in <code>-ct</code>), but since both firmware/driver
combinations gave me the same initial results, I decided to go with the
defaults.</p>
<h2 id="Problems">Problems</h2>
<p>The first problem I ran into is that I could only see one of the two cards
in the output of <code>lspci</code> (after <code>ssh</code>ing into the router). Looking for <code>ath</code>
or <code>wlan</code> in the <code>dmesg</code> output, it doesn't look like the second card is
being recognized at all.</p>
<p>Neither the 2.4 GHz or 5 GHz Wave 2 card worked in the right-most slot, but
either of them works fine when moved to the middle slot. The stock cards
work just fine in the right-most slot. I have no explanation for this.</p>
<p>The second problem was that I realized that the antenna holes are not all
the same. The two on each end are fully round and can accommodate the
diplexers which come with a round SMA-RP connector.</p>
<p>On the other hand, the three middle ones have a notch at the top which can
only accommodate the single antenna connectors which have a flat bit on one
side. I would have to file one of holes in order to add a third diplexer to
my setup.</p>
<h2 id="Final_working_configuration">Final working configuration</h2>
<p>Since I didn't see a way to use both new cards at once, I ended up on a
different configuration that would nevertheless still upgrade both my 2.4
GHz and 5 GHz Wi-Fi.</p>
<p>I moved the <a href="https://compex.com.sg/shop/wifi-module/802-11ac-wave-1/wle900vx-2/">original dual-band
card</a> to
the right-most slot and switched it to the 2.4 GHz band since it's more powerful
(both in dB and in throughput) than the <a href="https://compex.com.sg/shop/wifi-module/802-11n/wle200n2-2/">original half-length
card</a>.</p>
<p>Then I put the WLE1216V5-20 into the middle slot.</p>
<p>The only extra thing I had to buy were two extra <a href="https://www.amazon.ca/gp/product/B00TI1XBOE/">pigtails to SMA-RP
connectors and antennas</a>.</p>
<p>Here's what the final product looks like:</p>
<p><img alt="" src="https://feeding.cloud.geek.nz/posts/upgrading-wifi-cards-turris-omnia/omnia-inside.jpg" /></p>
<p><img alt="" src="https://feeding.cloud.geek.nz/posts/upgrading-wifi-cards-turris-omnia/omnia-outside.jpg" /></p>
Using all of the 5 GHz WiFi frequencies in a Gargoyle Routerhttps://feeding.cloud.geek.nz/posts/using-all-5ghz-wifi-frequencies-in-gargoyle-router/
<a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>
2021-06-11T20:43:57Z2017-12-11T02:00:00Z
<p>WiFi in the 2.4 GHz range is usually fairly congested in urban environments.
The 5 GHz band used to be better, but an increasing number of routers now
support it and so it has become fairly busy as well. It turns out that there
are a
<a href="https://en.wikipedia.org/wiki/List_of_WLAN_channels#5_GHz_.28802.11a.2Fh.2Fj.2Fn.2Fac.29">number of channels on that band</a>
that nobody appears to be using despite being legal in my region.</p>
<h2 id="Why_are_the_middle_channels_unused.3F">Why are the middle channels unused?</h2>
<p>I'm not entirely sure why these channels are completely empty in my area,
but I would speculate that access point manufacturers don't want to deal
with the extra complexity of the middle channels. Indeed these channels are
not entirely unlicensed. They are also used by weather radars, for example.
If you look at the regulatory rules that ship with your OS:</p>
<pre><code>$ iw reg get
global
country CA: DFS-FCC
(2402 - 2472 @ 40), (N/A, 30), (N/A)
(5170 - 5250 @ 80), (N/A, 17), (N/A), AUTO-BW
(5250 - 5330 @ 80), (N/A, 24), (0 ms), DFS, AUTO-BW
(5490 - 5600 @ 80), (N/A, 24), (0 ms), DFS
(5650 - 5730 @ 80), (N/A, 24), (0 ms), DFS
(5735 - 5835 @ 80), (N/A, 30), (N/A)
</code></pre>
<p>you will see that these channels are flagged with "DFS". That stands for
<a href="http://wifi-insider.com/wlan/dfs.htm">Dynamic Frequency Selection</a> and it
means that WiFi equipment needs to be able to detect when the frequency is
used by radars (by detecting their pulses) and automaticaly switch to a
different channel for a few minutes.</p>
<p>So an access point needs extra hardware and extra code to avoid interfering
with priority users. Additionally, different channels have
<a href="http://www.radio-electronics.com/info/wireless/wi-fi/80211-channels-number-frequencies-bandwidth.php">different bandwidth limits</a>
so that's something else to consider if you want to use 40/80 MHz at once.</p>
<h2 id="Using_all_legal_channels_in_Gargoyle">Using all legal channels in Gargoyle</h2>
<p>The first time I tried setting my access point channel to one of the middle
5 GHz channels, the SSID wouldn't show up in scans and the channel was still
empty in <a href="https://f-droid.org/packages/com.vrem.wifianalyzer/">WiFi Analyzer</a>.</p>
<p>I tried changing the channel again, but this time, I ssh'd into my router
and looked at the errors messages using this command:</p>
<pre><code>logread -f
</code></pre>
<p>I found a number of errors claiming that these channels were not authorized
for the "world" regulatory authority.</p>
<p>Because <a href="https://www.gargoyle-router.com/">Gargoyle</a> is based on
<a href="https://openwrt.org/">OpenWRT</a>, there are a lot more
<a href="https://wiki.openwrt.org/doc/uci/wireless">wireless configuration options</a>
available than what's exposed in the Web UI.</p>
<p>In this case, the solution was to explicitly <a href="https://feeding.cloud.geek.nz/posts/setting-wifi-regulatory-domain-linux-openwrt/">set my country in the wireless options</a> (where <code>CA</code> is the
<a href="https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2">country code</a> for Canada, where my
router is physically located).</p>
<p>Then I rebooted and I was able to set the channel successfully via the Web UI.</p>
<p>If you are interested, there is a lot more information about how all of this
works in the
<a href="https://wireless.wiki.kernel.org/en/developers/regulatory/processing_rules#country_definition">kernel documentation for the wireless stack</a>.</p>
Setting the wifi regulatory domain on Linux and OpenWRThttps://feeding.cloud.geek.nz/posts/setting-wifi-regulatory-domain-linux-openwrt/
<a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>
2021-06-11T20:43:57Z2015-08-01T20:20:00Z
<p>The list of <a href="https://en.wikipedia.org/wiki/List_of_WLAN_channels">available wifi channels</a>
is slightly different from country to country. To ensure access to the
right channels and transmit power settings, one needs to set the right
regulatory domain in the wifi stack.</p>
<h1 id="Linux">Linux</h1>
<p>For most Linux-based computers, you can look and change the current
regulatory domain using these commands:</p>
<pre><code>iw reg get
iw reg set CA
</code></pre>
<p>where <em><code>CA</code></em> is the <a href="https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2">two-letter country code</a>
when the device is located.</p>
<p>On Debian and Ubuntu, you can make this setting permanent by putting the
country code in <code>/etc/default/crda</code>.</p>
<p>Finally, to see the list of channels that are available in the current
config, use:</p>
<pre><code>iwlist wlan0 frequency
</code></pre>
<h1 id="OpenWRT">OpenWRT</h1>
<p>On <a href="https://openwrt.org/">OpenWRT</a>-based routers (including derivatives like
<a href="https://www.gargoyle-router.com/">Gargoyle</a>), looking and setting the
regulatory domain temporarily works the same way (i.e. the <code>iw</code> commands
above).</p>
<p>In order to persist your changes though, you need to use the
<a href="http://wiki.openwrt.org/doc/uci/wireless">uci</a> command:</p>
<pre><code>uci set wireless.radio0.country=CA
uci set wireless.radio1.country=CA
uci commit wireless
</code></pre>
<p>where <em><code>wireless.radio0</code></em> and <em><code>wireless.radio1</code></em> are the wireless devices
specific to your router. You can look them up using:</p>
<pre><code>uci show wireless
</code></pre>
<p>To test that it worked, simply reboot the router and then look at the
selected regulatory domain:</p>
<pre><code>iw reg get
</code></pre>
<h1 id="Scanning_the_local_wifi_environment">Scanning the local wifi environment</h1>
<p>Once your devices are set to the right country, you should scan the local
environment to pick the least congested wifi channel. You can use the
<a href="https://kismetwireless.net/spectools/">Kismet spectools</a>
if you have the hardware, otherwise
<a href="https://f-droid.org/repository/browse/?fdid=com.vrem.wifianalyzer">WiFi Analyzer</a>
is a good choice on Android.</p>
Debugging OpenWRT routers by shipping logs to a remote syslog serverhttps://feeding.cloud.geek.nz/posts/debugging-openwrt-routers-by-shipping/
<a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>
2022-07-02T02:45:10Z2012-01-14T08:45:00Z
<p>Trying to debug problems with consumer-grade routers is notoriously difficult due to a lack of decent debugging information. It's quite hard to know what's going on without at least a few good error messages.</p>
<p>Here is how I made my <a href="https://openwrt.org/">OpenWRT</a>-based <a href="http://gargoyle-router.org/">Gargoyle</a> router send its log messages to a network server running <a href="http://rsyslog.com/">rsyslog</a>.</p>
<h3 id="Server_Configuration">Server Configuration</h3>
<p>Given that the router (<code>192.168.1.1</code>) will be sending its log messages on UDP port 514, I started by opening that port in my firewall:</p>
<pre><code>iptables -A INPUT -s 192.168.1.1 -p udp --dport 514 -j ACCEPT
</code></pre>
<p>Then I enabled the UDP module for rsyslog and redirected messages to a separate log file (so that it doesn't fill up <code>/var/log/syslog</code>) by putting the following (a modified version of <a href="http://rsyslog.com/storing-messages-from-a-remote-system-into-a-specific-file/">these instructions</a>) in <code>/etc/rsyslog.d/10-gargoyle-router.conf</code>:</p>
<pre><code>$ModLoad imudp
$UDPServerRun 514
:fromhost-ip, isequal, "192.168.1.1" /var/log/gargoyle-router.log
& ~
</code></pre>
<p>The name of the file is important because this configuration snipet needs to be loaded before the directive which writes to <code>/var/log/syslog</code> for the discard statement (the "& ~" line) to <a href="http://lists.adiscon.net/pipermail/rsyslog/2012-January/014201.html">work correctly</a>.</p>
<h3 id="Router_Configuration">Router Configuration</h3>
<p>Finally, I followed the <a href="http://www.gargoyle-router.com/wiki/doku.php?id=remote_syslog">instructions</a> on the Gargoyle wiki to get the router to forward its log messages to my server (<code>192.168.1.2</code>).</p>
<p>After logging into the router via ssh, I ran the following commands:</p>
<pre><code>uci set system.@system[0].log_ip=192.168.1.2
uci set system.@system[0].cronloglevel=7
uci commit
</code></pre>
<p>before rebooting the router.</p>
<p>Now whenever I have to troubleshoot network problems, I can keep a terminal open on my server and get some visibility on what the router is doing:</p>
<pre><code>tail -f /var/log/gargoyle-router.log
</code></pre>