Here are some notes I took around programming my AnyTone AT-D878UV radio to operate on DMR using the CPS software that comes with it.
Note that you can always tune in to a VFO channel by hand if you haven't had time to add it to your codeplug yet.
DMR terminology
First of all, the terminology of DMR is quite different from that of the regular analog FM world.
Here are the basic terms:
- Frequency: same meaning as in the analog world
- Repeater: same meaning as in the analog world
- Timeslot: Each frequency is split into two timeslots (1 and 2) and what that means that there can be two simultaneous transmissions on each frequency.
- Color code: This is the digital equivalent of a CTCSS tone (sometimes called privacy tone) in that using the incorrect code means that you will tie up one of the timeslots on the frequency, but nobody else will hear you. These are not actually named after colors, but are instead just numerical IDs from 0 to 15.
There are two different identification mechanisms (both are required):
- Callsign: This is the same identifier issued to you by your country's amateur radio authority. Mine is VA7GPL.
- Radio ID: This is a unique numerical ID tied to your callsign which you must register for ahead of time and program into your radio. Mine is 3027260.
The following is where this digital mode becomes most interesting:
- Talkgroup: a "chat room" where everything you say will be heard by anybody listening to that talkgroup
- Network: a group of repeaters connected together over the Internet (typically) and sharing a common list of talkgroups
- Hotspot: a personal simplex device which allows you to connect to a network with your handheld and access all of the talkgroups available on that network
The most active network these days is Brandmeister, but there are several others.
- Access: This can either be Always on which means that a talkgroup will be permanently broadcasting on a timeslot and frequency, or PTT which means a talkgroup will not be broadcast until it is first "woken up" by pressing the push-to-talk button and then will broadcast for a certain amount of time before going to sleep again.
- Channel: As in the analog world, this is what you select on your radio when you want to talk to a group of people. In the digital world however, it is tied not only to a frequency (and timeslot) and tone (color code), but also to a specific talkgroup.
Ultimately what you want to do when you program your radio is to find the talkgroups you are interested in (from the list offered by your local repeater) and then assign them to specific channel numbers on your radio. More on that later.
Callsign and Radio IDs
Before we get to talkgroups, let's set your callsign and Radio ID:
Then you need to download the latest list of Radio IDs so that your radio can display people's names and callsigns instead of just their numerical IDs.
One approach is to only download the list of users who recently
talked on talkgroups you
are interested in. For example, I used to download the contacts for the
following talkgroups:
91,93,95,913,937,3026,3027,302,30271,30272,530,5301,5302,5303,5304,3100,3153,31330
but these days, what I normally do is to just download the entire worldwide
database (user.csv
) since my
radio still has enough storage (200k entries) for it.
In order for the user.csv
file to work with the AnyTone CPS, it needs to
have particular columns and use the DOS end-of-line characters (apt install
dos2unix
if you want to do it manually). I wrote a
script to
do all of the work for me.
If you use dmrconfig to program this
radio instead, then the conversion is unnecessary. The user.csv
file can
be used directly, however it will be truncated due to an incorrect limit
hard-coded in the software.
Talkgroups
Next, you need to pick the talkgroups you would like to allocate to specific channels on your radio.
Start by looking at the documentation for your local repeaters (e.g. VE7RAG and VE7NWR in the Vancouver area).
In addition to telling you the listen and transmit frequencies of the repeater (again, this works the same way as with analog FM), these will tell you which talkgroups are available and what timeslots and color codes they have been set to. It will also tell you the type of access for each of these talkgroups.
This is how I programmed a channel:
and a talkgroup on the VE7RAG repeater in my radio:
If you don't have a local repeater with DMR capability, or if you want to access talkgroups available on a different network, then you will need to get a DMR hotspot such as one that's compatible with the Pi-Star software.
This is an excerpt from the programming I created for the talkgroups I made available through my hotspot:
One of the unfortunate limitations of the CPS software for the AnyTone 878 is that talkgroup numbers are globally unique identifiers. This means that if TG1234 (hypothetical example) is Ragchew 3000 on DMR-MARC but Iceland-wide chat on Brandmeister, then you can't have two copies of it with different names. The solution I found for this was to give that talkgroup the name "TG1234" instead of "Ragchew3k" or "Iceland". I use a more memorable name for non-conflicting talkgroups, but for the problematic ones, I simply repeat the talkgroup number.
Simplex
Talkgroups are not required to operate on DMR. Just like analog FM, you can talk to another person point-to-point using a simplex channel.
The convention for all simplex channels is the following:
- Talkgroup:
99
- Color code:
1
- Timeslot:
1
- Admit criteria:
Always
- In Call Criteria:
TX
orAlways
After talking to the British Columbia Amateur Radio Coordination Council, I found that the following frequency ranges are most suitable for DMR simplex:
- 145.710-145.790 MHz (simplex digital transmissions)
- 446.000-446.975 MHz (all simplex modes)
The VECTOR list identifies two frequencies in particular:
- 446.075 MHz
- 446.500 MHz
Learn more
If you'd like to learn more about DMR, I would suggest you start with this excellent guide (also mirrored here).
I've been following Planet Linux Australia for many years and discovered many interesting FOSS blogs through it. I was sad to see that it got shut down a few weeks ago and so I decided to manually add all of the feeds to my RSS reader to avoid missing posts from people I have been indirectly following for years.
Since all feeds have been removed from the site, I recovered the list of blogs available from an old copy of the site preserved by the Internet Archive.
Here is the resulting .opml
file if you'd
like to subscribe.
Changes
Once I had the full list, I removed all blogs that are gone, empty or broken (e.g. domain not resolving, returning a 404, various database or server errors).
I updated the URLs of a few blogs which had moved but hadn't updated their feeds on the planet. I also updated the name of a blogger who was still listed under a previous last name.
Finally, I removed LA-specific tags from feeds since these are unlikely to be used again.
Work-arounds
The following LiveJournal feeds didn't work in my RSS reader but opened fine in a browser:
- https://ninjafoo.livejournal.com/data/rss
- https://certifiedwaif.livejournal.com/data/rss
- https://tau-iota-mu-c.livejournal.com/data/rss
- https://lathiat.livejournal.com/data/rss
However since none of them have them updated in the last 7 years, I just left them out.
A couple appear to be impossible to fetch over Tor, presumably due to a Cloudflare setting:
- https://www.jamver.id.au/cgi-bin/blosxom.cgi/index.rss
- https://www.rwhitby.net/feed
- https://blog.khax.net/feed/
- https://dalts.com/index.xml
- https://pento.net/feed/
Since only the last two have been updated in the last 9 years, I added these to Feedburner and added the following "proxied" URLs to my reader:
Similarly, I couldn't fetch the following over Tor for some other reasons:
- http://algorithm.com.au/blog/files/rss.xml
- http://soundadvice.id.au/blog/index.atom
- http://www.bytebot.net/blog/archives/category/databases/feed
- http://levlafayette.com/blog/feed
- http://www.rowetel.com/?feed=rss2
- https://hamishtaylor.com.au/photekgraddft-hamish-taylors-blog?format=rss
I excluded the first two which haven't been updated in 6 years and proxied the other ones:
A few years ago, the advertising industry introduced the ads.txt
project in order to defend against
widespread domain spoofing vulnerabilities in programmatic
advertising.
I decided to use this technology to opt out of having ads sold for my domains, at least through ad exchanges which perform this check, by hosting a text file containing this:
contact=ads@fmarier.org
at the following locations:
(In order to get this to work on my blog, running
Ikiwiki on
Branchable, I had to disable the txt
plugin in order to get ads.txt
to be
served as a plain text file instead of being automatically rendered as
HTML.)
Specification
The key parts of the specification for our purposes are:
[3.1] If the server response indicates the resource does not exist (HTTP Status Code 404), the advertising system can assume no declarations exist and that no advertising system is unauthorized to buy and sell ads on the website.
[3.2.1] Some publishers may choose to not authorize any advertising system by publishing an empty ads.txt file, indicating that no advertising system is authorized to buy and sell ads on the website. So that consuming systems properly read and interpret the empty file (differentiating between web servers returning error pages for the /ads.txt URL), at least one properly formatted line must be included which adheres to the format specification described above.
As you can see, the specification sadly ignores
RFC8615 and requires that the
ads.txt
file be present directly in the root of your web server, like the
venerable robots.txt
file, but unlike the
newer security.txt
standard.
If you don't want to provide an email address in your ads.txt
file, the
specification recommends using the following line verbatim:
placeholder.example.com, placeholder, DIRECT, placeholder
Validation
A number of online validators exist, but I used the following to double-check my setup:
I recently ran into a corrupted data pack in a Restic
backup on my
GnuBee. It led
to consistent failures during the prune
operation:
incomplete pack file (will be removed): b45afb51749c0778de6a54942d62d361acf87b513c02c27fd2d32b730e174f2e
incomplete pack file (will be removed): c71452fa91413b49ea67e228c1afdc8d9343164d3c989ab48f3dd868641db113
incomplete pack file (will be removed): 10bf128be565a5dc4a46fc2fc5c18b12ed2e77899e7043b28ce6604e575d1463
incomplete pack file (will be removed): df282c9e64b225c2664dc6d89d1859af94f35936e87e5941cee99b8fbefd7620
incomplete pack file (will be removed): 1de20e74aac7ac239489e6767ec29822ffe52e1f2d7f61c3ec86e64e31984919
hash does not match id: want 8fac6efe99f2a103b0c9c57293a245f25aeac4146d0e07c2ab540d91f23d3bb5, got 2818331716e8a5dd64a610d1a4f85c970fd8ae92f891d64625beaaa6072e1b84
github.com/restic/restic/internal/repository.Repack
github.com/restic/restic/internal/repository/repack.go:37
main.pruneRepository
github.com/restic/restic/cmd/restic/cmd_prune.go:242
main.runPrune
github.com/restic/restic/cmd/restic/cmd_prune.go:62
main.glob..func19
github.com/restic/restic/cmd/restic/cmd_prune.go:27
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra/command.go:838
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra/command.go:943
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra/command.go:883
main.main
github.com/restic/restic/cmd/restic/main.go:86
runtime.main
runtime/proc.go:204
runtime.goexit
runtime/asm_amd64.s:1374
Thanks to the excellent support forum, I was able to resolve this issue by dropping a single snapshot.
First, I identified the snapshot which contained the offending pack:
$ restic -r sftp:hostname.local: find --pack 8fac6efe99f2a103b0c9c57293a245f25aeac4146d0e07c2ab540d91f23d3bb5
repository b0b0516c opened successfully, password is correct
Found blob 2beffa460d4e8ca4ee6bf56df279d1a858824f5cf6edc41a394499510aa5af9e
... in file /home/francois/.local/share/akregator/Archive/http___udd.debian.org_dmd_feed_
(tree 602b373abedca01f0b007fea17aa5ad2c8f4d11f1786dd06574068bf41e32020)
... in snapshot 5535dc9d (2020-06-30 08:34:41)
Then, I could simply drop that snapshot:
$ restic -r sftp:hostname.local: forget 5535dc9d
repository b0b0516c opened successfully, password is correct
[0:00] 100.00% 1 / 1 files deleted
and run the prune
command to remove the snapshot, as well as the incomplete
packs that were also mentioned in the above output but could never be
removed due to the other error:
$ restic -r sftp:hostname.local: prune
repository b0b0516c opened successfully, password is correct
counting files in repo
building new index for repo
[20:11] 100.00% 77439 / 77439 packs
incomplete pack file (will be removed): b45afb51749c0778de6a54942d62d361acf87b513c02c27fd2d32b730e174f2e
incomplete pack file (will be removed): c71452fa91413b49ea67e228c1afdc8d9343164d3c989ab48f3dd868641db113
incomplete pack file (will be removed): 10bf128be565a5dc4a46fc2fc5c18b12ed2e77899e7043b28ce6604e575d1463
incomplete pack file (will be removed): df282c9e64b225c2664dc6d89d1859af94f35936e87e5941cee99b8fbefd7620
incomplete pack file (will be removed): 1de20e74aac7ac239489e6767ec29822ffe52e1f2d7f61c3ec86e64e31984919
repository contains 77434 packs (2384522 blobs) with 367.648 GiB
processed 2384522 blobs: 1165510 duplicate blobs, 47.331 GiB duplicate
load all snapshots
find data that is still in use for 15 snapshots
[1:11] 100.00% 15 / 15 snapshots
found 1006062 of 2384522 data blobs still in use, removing 1378460 blobs
will remove 5 invalid files
will delete 13728 packs and rewrite 15140 packs, this frees 142.285 GiB
[4:58:20] 100.00% 15140 / 15140 packs rewritten
counting files in repo
[18:58] 100.00% 50164 / 50164 packs
finding old index files
saved new indexes as [340cb68f 91ff77ef ee21a086 3e5fa853 084b5d4b 3b8d5b7a d5c385b4 5eff0be3 2cebb212 5e0d9244 29a36849 8251dcee 85db6fa2 29ed23f6 fb306aba 6ee289eb 0a74829d]
remove 190 old index files
[0:00] 100.00% 190 / 190 files deleted
remove 28868 old packs
[1:23] 100.00% 28868 / 28868 files deleted
done
I ran into a corrupt MariaDB index page the other day and had to restore my MythTV database from the automatic backups I make as part of my regular maintainance tasks.
Signs of trouble
My troubles started when my daily backup failed on this line:
mysqldump --opt mythconverg -umythtv -pPASSWORD > mythconverg-200200923T1117.sql
with this error message:
mysqldump: Error 1034: Index for table 'recordedseek' is corrupt; try to repair it when dumping table `recordedseek` at row: 4059895
Comparing the dump that was just created to the database dumps in
/var/backups/mythtv/
, it was clear that it was incomplete since it was
about 100 MB smaller.
I first tried a gentle OPTIMIZE TABLE recordedseek
as suggested in this
StackExchange
answer
but that caused the database to segfault:
mysqld[9141]: 2020-09-23 15:02:46 0 [ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace mythconverg/recordedseek page [page id: space=115871, page number=11373]. You may have to recover from a backup.
mysqld[9141]: 2020-09-23 15:02:46 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
mysqld[9141]: len 16384; hex 06177fa70000...
mysqld[9141]: C K c {\;
mysqld[9141]: InnoDB: End of page dump
mysqld[9141]: 2020-09-23 15:02:46 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 102203303, calculated checksums for field1: crc32 806650270, innodb 1139779342, page type 17855 == INDEX.none 3735928559, stored checksum in field2 102203303, calculated checksums for field2: crc32 806650270, innodb 3322209073, none 3735928559, page LSN 148 2450029404, low 4 bytes of LSN at page end 2450029404, page number (if stored to page already) 11373, space id (if created with >= MySQL-4.1.1 and stored already) 115871
mysqld[9141]: 2020-09-23 15:02:46 0 [Note] InnoDB: Page may be an index page where index id is 697207
mysqld[9141]: 2020-09-23 15:02:46 0 [Note] InnoDB: Index 697207 is `PRIMARY` in table `mythconverg`.`recordedseek`
mysqld[9141]: 2020-09-23 15:02:46 0 [Note] InnoDB: It is also possible that your operating system has corrupted its own file cache and rebooting your computer removes the error. If the corrupt page is an index page. You can also try to fix the corruption by dumping, dropping, and reimporting the corrupt table. You can use CHECK TABLE to scan your table for corruption. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
mysqld[9141]: 200923 15:02:46 2020-09-23 15:02:46 0 [ERROR] InnoDB: Failed to read file './mythconverg/recordedseek.ibd' at offset 11373: Page read from tablespace is corrupted.
mysqld[9141]: [ERROR] mysqld got signal 11 ;
mysqld[9141]: Core pattern: |/lib/systemd/systemd-coredump %P %u %g %s %t 9223372036854775808 %h ...
kernel: [820233.893658] mysqld[9186]: segfault at 90 ip 0000557a229f6d90 sp 00007f69e82e2dc0 error 4 in mysqld[557a224ef000+803000]
kernel: [820233.893665] Code: c4 20 83 bd e4 eb ff ff 44 48 89 ...
systemd[1]: mariadb.service: Main process exited, code=killed, status=11/SEGV
systemd[1]: mariadb.service: Failed with result 'signal'.
systemd-coredump[9240]: Process 9141 (mysqld) of user 107 dumped core.#012#012Stack trace of thread 9186: ...
systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 1.
mysqld[9260]: 2020-09-23 15:02:52 0 [Warning] Could not increase number of max_open_files to more than 16364 (request: 32186)
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=638234502026
...
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] InnoDB: Recovered page [page id: space=115875, page number=5363] from the doublewrite buffer.
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] InnoDB: Starting final batch to recover 2 pages from redo log.
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] InnoDB: Waiting for purge to start
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] Recovering after a crash using tc.log
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] Starting crash recovery...
mysqld[9260]: 2020-09-23 15:02:53 0 [Note] Crash recovery finished.
and so I went with the nuclear option of dropping the MythTV database and restoring from backup.
Dropping the corrupt database
First of all, I shut down MythTV as root
:
kilall mythfrontend
systemctl stop mythtv-status.service
systemctl stop mythtv-backend.service
and took a full copy of my MariaDB databases just in case:
systemctl stop mariadb.service
cd /var/lib
apack /root/var-lib-mysql-20200923T1215.tgz mysql/
systemctl start mariadb.service
before dropping the MythTV databse (mythconverg
):
$ mysql -pPASSWORD
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| mythconverg |
| performance_schema |
+--------------------+
4 rows in set (0.000 sec)
MariaDB [(none)]> drop database mythconverg;
Query OK, 114 rows affected (25.564 sec)
MariaDB [(none)]> quit
Bye
Restoring from backup
Then I re-created an empty database:
mysql -pPASSWORD < /usr/share/mythtv/sql/mc.sql
and restored the last DB dump prior to the detection of the corruption:
sudo -i -u mythtv
/usr/share/mythtv/mythconverg_restore.pl --directory /var/backups/mythtv --filename mythconverg-1350-20200923010502.sql.gz
In order to restart everything properly, I simply rebooted the machine:
systemctl reboot
Here is the process I followed when I moved my GnuBee's root partition from one flaky Kingston SSD drive to a brand new Samsung SSD.
It was relatively straightforward, but there are two key points:
- Make sure you label the root partition
GNUBEE-ROOT
. - Make sure you copy the network configuration from the SSD, not the
tmpfs
mount.
Copy the partition table
First, with both drives plugged in, I replicated the partition table of the
first drive (/dev/sda
):
# fdisk -l /dev/sda
Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Disk model: KINGSTON SA400S3
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 799CD830-526B-42CE-8EE7-8C94EF098D46
Device Start End Sectors Size Type
/dev/sda1 2048 8390655 8388608 4G Linux swap
/dev/sda2 8390656 234441614 226050959 107.8G Linux filesystem
onto the second drive (/dev/sde
):
# fdisk /dev/sde
Welcome to fdisk (util-linux 2.33.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xd011eaba.
Command (m for help): g
Created a new GPT disklabel (GUID: 83F70325-5BE0-034E-A9E1-1965FEFD8E9F).
Command (m for help): n
Partition number (1-128, default 1):
First sector (2048-488397134, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-488397134, default 488397134): +4G
Created a new partition 1 of type 'Linux filesystem' and of size 4 GiB.
Command (m for help): t
Selected partition 1
Partition type (type L to list all types): 19
Changed type of partition 'Linux filesystem' to 'Linux swap'.
Command (m for help): n
Partition number (2-128, default 2):
First sector (8390656-488397134, default 8390656):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (8390656-488397134, default 488397134): 234441614
Created a new partition 2 of type 'Linux filesystem' and of size 107.8 GiB.
Command (m for help): p
Disk /dev/sde: 232.9 GiB, 250059350016 bytes, 488397168 sectors
Disk model: Samsung SSD 860
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 83F70325-5BE0-034E-A9E1-1965FEFD8E9F
Device Start End Sectors Size Type
/dev/sde1 2048 8390655 8388608 4G Linux swap
/dev/sde2 8390656 234441614 226050959 107.8G Linux filesystem
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
I wasted a large amount of space on the second drive, but that was on purpose in case I decide to later on move to a RAID-1 root partition with the Kingston SSD.
Format the partitions
Second, I formated the new partitions:
# mkswap /dev/sde1
Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
no label, UUID=7a85fbce-2493-45c1-a548-4ec6e827ec29
# mkfs.ext4 /dev/sde2
mke2fs 1.44.5 (15-Dec-2018)
Discarding device blocks: done
Creating filesystem with 28256369 4k blocks and 7069696 inodes
Filesystem UUID: 732a76df-d369-4e7b-857a-dd55fd461bbc
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done
and labeled the root partition so that the GnuBee can pick it up as it boots up:
e2label /dev/sde2 GNUBEE-ROOT
since GNUBEE-ROOT
is what uboot
will be looking
for.
Copy the data over
Finally, I copied the data over from the original drive to the new one:
# umount /etc/network
# mkdir /mnt/root
# mount /dev/sde2 /mnt/root
# rsync -aHx --delete --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* --exclude=/mnt/* --exclude=/lost+found/* --exclude=/media/* --exclude=/rom/* /* /mnt/root/
# sync
Note that if you don't unmount /etc/network/
, you'll be copying the
override provided at boot time instead of the underlying config that's on
the root partition. The reason that this matters is that the script that
renames the network interfaces to ethblack
and
ethblue
expects specific files in order to produce a working network configuration.
If you copy the final modified config files then you end up with an
bind-mounted empty directory as /etc/network
, and the network interfaces
can't be brought up successfully.
In order to fix the following error after setting up SIP TLS in Asterisk 16.2:
asterisk[8691]: ERROR[8691]: tcptls.c:966 in __ssl_setup: TLS/SSL error loading cert file. <asterisk.pem>
I created a Let's Encrypt certificate using certbot:
apt install certbot
certbot certonly --standalone -d hostname.example.com
To enable the asterisk
user to load the certificate successfuly (it
doesn't permission to access to the certificates under /etc/letsencrypt/
),
I copied it to the right directory:
cp /etc/letsencrypt/live/hostname.example.com/privkey.pem /etc/asterisk/asterisk.key
cp /etc/letsencrypt/live/hostname.example.com/fullchain.pem /etc/asterisk/asterisk.cert
chown asterisk:asterisk /etc/asterisk/asterisk.cert /etc/asterisk/asterisk.key
chmod go-rwx /etc/asterisk/asterisk.cert /etc/asterisk/asterisk.key
Then I set the following variables in /etc/asterisk/sip.conf
:
tlscertfile=/etc/asterisk/asterisk.cert
tlsprivatekey=/etc/asterisk/asterisk.key
Automatic renewal
The machine on which I run asterisk has a tricky Apache setup:
- a webserver is running on port 80
- port 80 is restricted to the local network
This meant that the certbot domain ownership checks would get blocked by the firewall, and I couldn't open that port without exposing the private webserver to the Internet.
So I ended up disabling the built-in certbot renewal mechanism:
systemctl disable certbot.timer certbot.service
systemctl stop certbot.timer certbot.service
and then writing my own script in /etc/cron.daily/certbot-francois
:
#!/bin/bash
TEMPFILE=`mktemp`
# Stop Apache and backup firewall.
/bin/systemctl stop apache2.service
/usr/sbin/iptables-save > $TEMPFILE
# Open up port 80 to the whole world.
/usr/sbin/iptables -D INPUT -j LOGDROP
/usr/sbin/iptables -A INPUT -p tcp --dport 80 -j ACCEPT
/usr/sbin/iptables -A INPUT -j LOGDROP
# Renew all certs.
/usr/bin/certbot renew --quiet
# Restore firewall and restart Apache.
/usr/sbin/iptables -D INPUT -p tcp --dport 80 -j ACCEPT
/usr/sbin/iptables-restore < $TEMPFILE
/bin/systemctl start apache2.service
# Copy certificate into asterisk.
cp /etc/letsencrypt/live/hostname.example.com/privkey.pem /etc/asterisk/asterisk.key
cp /etc/letsencrypt/live/hostname.example.com/fullchain.pem /etc/asterisk/asterisk.cert
chown asterisk:asterisk /etc/asterisk/asterisk.cert /etc/asterisk/asterisk.key
chmod go-rwx /etc/asterisk/asterisk.cert /etc/asterisk/asterisk.key
/bin/systemctl restart asterisk.service
# Commit changes to etckeeper.
pushd /etc/ > /dev/null
/usr/bin/git add letsencrypt asterisk
DIFFSTAT="$(/usr/bin/git diff --cached --stat)"
if [ -n "$DIFFSTAT" ] ; then
/usr/bin/git commit --quiet -m "Renewed letsencrypt certs."
echo "$DIFFSTAT"
fi
popd > /dev/null
As part of the #MoreOnionsPorFavor
campaign, I decided to
follow brave.com
's lead and make
my homepage available as a Tor onion
service.
Tor daemon setup
I started by installing the Tor daemon locally:
apt install tor
and then setting the following in /etc/tor/torrc
:
SocksPort 0
SocksPolicy reject *
HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80 [2600:3c04::f03c:91ff:fe8c:61ac]:80
HiddenServicePort 443 [2600:3c04::f03c:91ff:fe8c:61ac]:443
HiddenServiceVersion 3
HiddenServiceNonAnonymousMode 1
HiddenServiceSingleHopMode 1
in order to create a version 3 onion service without actually running a Tor relay.
Note that since I am making a public website available over Tor, I do not need the location of the website to be hidden and so I used the same settings as Cloudflare in their public Tor proxy.
Also, I explicitly used the external IPv6 address of my server in the configuration in order to prevent localhost bypasses.
After restarting the Tor daemon to reload the configuration file:
systemctl restart tor.service
I looked for the address of my onion service:
$ cat /var/lib/tor/hidden_service/hostname
ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion
Apache configuration
Next, I enabled a few required Apache modules:
a2enmod mpm_event
a2enmod http2
a2enmod headers
and configured my Apache vhosts in /etc/apache2/sites-enabled/www.conf
:
<VirtualHost *:443>
ServerName fmarier.org
ServerAlias ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion
Protocols h2, http/1.1
Header set Onion-Location "http://ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion%{REQUEST_URI}s"
Header set alt-svc 'h2="ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion:443"; ma=315360000; persist=1'
Header add Strict-Transport-Security: "max-age=63072000"
Include /etc/fmarier-org/www-common.include
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/fmarier.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/fmarier.org/privkey.pem
</VirtualHost>
<VirtualHost *:80>
ServerName fmarier.org
Redirect permanent / https://fmarier.org/
</VirtualHost>
<VirtualHost *:80>
ServerName ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion
Include /etc/fmarier-org/www-common.include
</VirtualHost>
Note that /etc/fmarier-org/www-common.include
contains all of the
configuration options that are common to both the HTTP and the HTTPS sites
(e.g. document root, caching headers, aliases, etc.).
Finally, I restarted Apache:
apache2ctl configtest
systemctl restart apache2.service
Testing
In order to test that my website is correctly available at its .onion
address, I opened the following URLs in a Brave Tor
window:
- http://ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion/
- https://ixrdj3iwwhkuau5tby5jh3a536a2rdhpbdbu6ldhng43r47kim7a3lid.onion/ (a TLS certificate error is expected)
I also checked that the main URL (https://fmarier.org/) exposes a working
Onion-Location
header
which triggers the display of a button in the URL bar (recently merged
and available in Brave Nightly):
Testing that the Alt-Svc
is working required using the Tor Browser
since that's not yet supported in
Brave:
- Open https://fmarier.org.
- Wait 30 seconds.
- Reload the page.
On the server side, I saw the following:
2a0b:f4c2:2::1 - - [14/Oct/2020:02:42:20 +0000] "GET / HTTP/2.0" 200 2696 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
2600:3c04::f03c:91ff:fe8c:61ac - - [14/Oct/2020:02:42:53 +0000] "GET / HTTP/2.0" 200 2696 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
That first IP address is from a Tor exit node:
$ whois 2a0b:f4c2:2::1
...
inet6num: 2a0b:f4c2::/40
netname: MK-TOR-EXIT
remarks: -----------------------------------
remarks: This network is used for Tor Exits.
remarks: We do not have any logs at all.
remarks: For more information please visit:
remarks: https://www.torproject.org
which indicates that the first request was not using the .onion
address.
The second IP address is the one for my server:
$ dig +short -x 2600:3c04::f03c:91ff:fe8c:61ac
hafnarfjordur.fmarier.org.
which indicates that the second request to Apache came from the Tor relay
running on my server, hence using the .onion
address.
I recently upgraded from Ubuntu 18.04.5 (bionic) to 20.04.1 (focal) and it was one of the roughest Ubuntu upgrades I've gone through in a while. Here are the notes I took on avoiding or fixing the problems I ran into.
Preparation
Before going through the upgrade, I disabled the configurations which I know interfere with the process:
Enable etckeeper auto-commits before install by putting the following in
/etc/etckeeper/etckeeper.conf
:AVOID_COMMIT_BEFORE_INSTALL=0
Remount
/tmp
as exectuable:mount -o remount,exec /tmp
Another step I should have taken but didn't, was to temporarily remove safe-rm since it caused some problems related to a Perl upgrade happening at the same time:
apt remove safe-rm
Network problems
After the upgrade, my network settings weren't really working properly and so I started by switching from ifupdown to netplan.io which seems to be the preferred way of configuring the network on Ubuntu now.
Then I found out that netplan.io is
not automatically enabling the
systemd-resolved handling of .local
hostnames.
I would be able to resolve a hostname using avahi:
$ avahi-resolve --name machine.local
machine.local 192.168.1.5
but not with systemd:
$ systemd-resolve machine.local
machine.local: resolve call failed: 'machine.local' not found
$ resolvectl mdns
Global: no
Link 2 (enp4s0): no
The best solution I found involves keeping
systemd-resolved and its /etc/resolv.conf
symlink to /run/systemd/resolve/stub-resolv.conf
.
I added the following in a new /etc/NetworkManager/conf.d/mdns.conf
file:
[connection]
connection.mdns=1
which instructs NetworkManager to resolve mDNS on all network interfaces it manages but not register a hostname since that's done by avahi-daemon.
Then I enabled mDNS globally in systemd-resolved by setting the following
in /etc/systemd/resolved.conf
:
MulticastDNS=yes
before restarting both services:
systemctl restart NetworkManager.service systemd-resolved.service
With that in place, .local
hostnames are resolved properly and I can
see that mDNS is fully enabled:
$ resolvectl mdns
Global: yes
Link 2 (enp4s0): yes
Boot problems
For some reason I was able to boot with the kernel I got as part of the focal update, but a later kernel update rendered my machine unbootable.
Adding some missing RAID-related modules to
/etc/initramfs-tools/modules
:
raid1
dmraid
md-raid1
and then re-creating all initramfs:
update-initramfs -u -k all
seemed to do the trick.
I ran into filesystem corruption
(ext4) on the root partition of my
backup server
which caused it to go into read-only mode. Since it's the root partition,
it's not possible to unmount it and repair it while it's running. Normally I
would boot from an Ubuntu live CD / USB
stick, but in this case
the machine is using the
mipsel
architecture and
so that's not an option.
Repair using a USB enclosure
I had to pull the shutdown the server and then pull the SSD drive out. I then moved it to an external USB enclosure and connected it to my laptop.
I started with an automatic filesystem repair:
fsck.ext4 -pf /dev/sde2
which failed for some reason and so I moved to an interactive repair:
fsck.ext4 -f /dev/sde2
Once all of the errors were fixed, I ran a full surface scan to update the list of bad blocks:
fsck.ext4 -c /dev/sde2
Finally, I forced another check to make sure that everything was fixed at the filesystem level:
fsck.ext4 -f /dev/sde2
Fix invalid alternate GPT
The other thing I noticed is this messge in my dmesg
log:
scsi 8:0:0:0: Direct-Access KINGSTON SA400S37120 SBFK PQ: 0 ANSI: 6
sd 8:0:0:0: Attached scsi generic sg4 type 0
sd 8:0:0:0: [sde] 234441644 512-byte logical blocks: (120 GB/112 GiB)
sd 8:0:0:0: [sde] Write Protect is off
sd 8:0:0:0: [sde] Mode Sense: 31 00 00 00
sd 8:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 8:0:0:0: [sde] Optimal transfer size 33553920 bytes
Alternate GPT is invalid, using primary GPT.
sde: sde1 sde2
I therefore checked to see if the partition table looked fine and got the following:
$ fdisk -l /dev/sde
GPT PMBR size mismatch (234441643 != 234441647) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
Disk /dev/sde: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Disk model: KINGSTON SA400S3
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 799CD830-526B-42CE-8EE7-8C94EF098D46
Device Start End Sectors Size Type
/dev/sde1 2048 8390655 8388608 4G Linux swap
/dev/sde2 8390656 234441614 226050959 107.8G Linux filesystem
It turns out that all I had to do, since only the backup / alternate GPT partition table was corrupt and the primary one was fine, was to re-write the partition table:
$ fdisk /dev/sde
Welcome to fdisk (util-linux 2.33.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
GPT PMBR size mismatch (234441643 != 234441647) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
Command (m for help): w
The partition table has been altered.
Syncing disks.
Run SMART checks
Since I still didn't know what caused the filesystem corruption in the first place, I decided to do one last check: SMART errors.
I couldn't do this via the USB enclosure since the SMART commands aren't forwarded to the drive and so I popped the drive back into the backup server and booted it up.
First, I checked whether any SMART errors had been reported using smartmontools:
smartctl -a /dev/sda
That didn't show any errors and so I kicked off an extended test:
smartctl -t long /dev/sda
which ran for 30 minutes and then passed without any errors.
The mystery remains unsolved.