Nextcloud-Tools: Working with the Nextcloud Server-Side Encryption
02.12.2019 yahe administration code security
At the beginning of the year we ran into a strange problem with our server-side encrypted Nextcloud installation at work. Files got corrupted when they were moved between folders. We had found another problem with the Nextcloud Desktop client just recently and therefore thought that this was also related to synchronization problems within the Nextcloud Desktop client. Later in the year we bumped into this problem again, but this time it occured while using the web frontend of Nextcloud. Now we understood that the behaviour did not have anything to do with the client but with the server itself. Interestingly, another user had opened a Github issue about this problem at around the same time. As these corruptions lead to quite some workload for the restore I decided to dig deeper to find an adequate solution.
After I had found out how to reproduce the problem it was important for us to know whether corrupted files could still be decrypted at all. I wrote a decryption script and proved that corrupted files could in fact be decrypted when Nextcloud said that they were broken. With this in mind I tried to find out what happened during the encryption and what broke files while being moved. Doing all the research about the server-side encryption of Nextcloud, debugging the software, creating a potential bugfix and coming up with a temporary workaround took about a month of interrupted work.
Even more important than the actual bugfix (as we are currently living with the workaround) is the knowledge we gained about the server-side encryption. Based on this knowledge I developed a bunch of scripts that have been published as nextcloud-tools on GitHub. These scripts can help you to rescue your server-side encrypted files in cases when your database was corrupted or completely lost.
I also wrote an elaborate description about the inner workings of the server-side encryption and tried to get it added to the documentation. It took some time but in the end it worked! For about a week now you can find my description of the Nextcloud Server-Side Encryption Details in the official Nextcloud documentation.
twastosync: Synchronize Mastodon Toots to Twitter
Ever since I built up my own Mastodon instance and created my own account I wanted to be able to cross-post messages between the two platforms. I searched for different solutions like IFTTT but I never got them to work properly - until I found dlvr.it which worked right out of the box for me. Well, at least at the beginning...
I have to admit that I did not use Mastodon regularly for quite some time and even thought about deleting my instance, but I wanted to give it another try now and wanted to use the synchronization feature - which did not work anymore. So I checked my dlvr.it account and found out that the synchronization had silently broken when Mastodon switched from providing Atom feeds to providing RSS feeds. So I reconfigured the synchronization, just to find out that dlvr.it now has limited the free tier to three posts per day. Think about it: three. posts. per day.
Looking at the pricing of dlvr.it I just had to laugh. They want me to pay $8.29 per month for an unlimited amount of messages. In comparison, the server that my Mastodon instance is running on is cheaper than that. My next step was to look for alternatives: circleboom also allows only three posts from an RSS feed per day and for an unlimited number of posts they want to charge you $17.00 per month or $5.99 per month if you are on an annual subscription. socialoomph does not even provide RSS feed synchronization in the free tier and comes at $15.00 per month or $162.00 per year. Finally, I also looked at Hootsuite, which is the strangest of them all. In order to auto-publish posts from an RSS feed you actually have to subscribe to the paid RSS AutoPublisher app which wants to charge you $5.99 per month.
That was the point where I decided to build something on my own. Because, what is better than writing code? Writing code and saving money by doing so. I took one of my previous Twitter tools as a basis and created twastosync. The tool uses the TwitterOAuth library to communicate with the Twitter API and the simplexml_load_string() function of PHP to parse the RSS feed that you can find at the URL https://mastodon-instance-url/@username.rss. Finally, it uses my unchroot library to prevent concurrent executions.
When you installed and configured the script you can use something like CRON to call the twastosync script regularly. Thanks to the concurrency prevention you can even call it every minute. One step that might be a bit scary is to use the app registration page of Twitter to register your own bot. I did this myself and there did not pop up any problem.
There we are: With a small script we are able to synchronize our Mastodon toots to Twitter without relying on a third party platform and by doing it ourselves we are even saving real money. 🤑
.local, .home.arpa and .localzone.xyz
11.11.2019 yahe administration
35 years ago the IETF defined special IPv4 addresses in RFC 1597 to be used solely in private intranets and with that created the separation between internal networks and the internet. 5 years later they proceeded to define special-use top-level domains like .example, .invalid and .test in RFC 2606. However, the IETF did not reserve a top-level domain to be used within the private intranets that they had introduced before. This lead to confusion with administrators that still persists to the current day.
Every time an administrator was tasked to design a network structure they also had to think about naming conventions within the network. As there was a pre-defined set of "valid" top-level domains within the internet, it was rather easy to just select an "invalid" TLD and use that for the private network. At least, until the ICANN decided to delegate new TLDs to anyone who was willing to apply and pay the required fees. The ICANN even provided a paper on how to identify and mitigate name collisions to be used by professionals.
With RFC 6762 the .local TLD was approved as a special-use TLD to be used in internal networks and many administrators thought that this would solve the problems, but unfortunately, it did not. .local was designated to be used with multicast DNS, meaning that each device in a network could grab its preferred hostname. At the beginning this was not a big deal, but as more and more devices implemented Apple's Bonjour protocol (also known as zeroconf) interoperability problems started to pop up. Appendix G of said RFC even mentioned alternative TLDs that might be used, however, the ICANN did not accept any of them as special-use domain names.
Years later the new RFC 7788 defined .home as a potential local-only TLD, but it was changed to the rather unusual .home.arpa domain name with RFC 8375. This domain is safe to be used in local networks and was accepted as a special-use domain name by the ICANN. Unfortunately, due the word "home" in the domain it is not quite fitting for business environments.
There are two more TLDs that might be safe to use: .corp and .home are not officially recognized special-use domain names but the ICANN has refrained from delegating them to bidders as the risk of breaking internal networks is deemed to be too big.
As of today, most tutorials still propose to use a publicly registered domain for internal networks. This is why I registered .localzone.xyz about half a year ago. It is explicitly meant to be used locally, has public DNS records set to prevent any public CA from issuing trusted TLS certificates for that domain as well as DNS records defining all mails using that domain as SPAM and will not be used for publicly accessible services as long as it is owned by me. I am using this domain internally myself and am also trying to get the domain added to the public suffix list. This list defines domain and cookie boundaries so that e.g. example-a.com is not able to set cookies for example-b.com.
So if you are looking for a domain that you can use for internal domain names then look no further: Either use the special-use domain name .home.arpa (which might not be fitting for businesses) or use .localzone.xyz as I am. It makes clear that you are in a local environment without being too specific about whether it is a private or business network.
Yahe.sh wechselt ins Englische
Im letzten Artikel hatte ich erklärt, weshalb ich mich dazu entschieden habe, von der Domain "weizenspr.eu" hin zur Domain "yahe.sh" zu wechseln. Hintergrund ist der Fokus der Webseite, der sich in den letzten fast 10 Jahren von Alltagserfahrungen hin zu technischen Themen gewandelt hat. Mit dem Wechsel des Themenfeldes geht jedoch auch ein Wechsel des Zielpublikums einher.
In der IT ist Englisch die Sprache der Wahl, um ein möglichst großes Publikum zu erreichen. Daher habe ich mich nach langem Ringen dazu entschlossen, auch diesen Blog zukünftig auf Englisch zu führen. Meine Hoffnung ist, dass die Leute, die auch bisher schon meine technischen Inhalte gelesen haben, der englischen Sprache mächtig sind. Zumindest in meinem Bekanntenkreis stellt der Sprachwechsel kein Problem dar, eröffnet mir jedoch die Möglichkeit einer internationalen Leserschaft.
Daher bleibt mir an dieser Stelle nur noch zu sagen: Welcome to Yahe.sh!
Goodbye WeizenSpr.eu
Am 02.02.2009 habe ich die Domain "weizenspr.eu" registriert. Der Blog, der sich dahinter verbarg, durchlebte die Blütezeit des Bloggens in Deutschland. Damals brauchten Blogs unbedingt ein Motto und so entschied ich mich für "trennt Spreu vom Weizen". Ich wollte sowohl die guten Dinge ("Weizen"), als auch die schlechten Dinge ("Spreu") im Alltag thematisieren.
Der unbeschwerte Studentenalltag wich dem stressigen Berufsalltag und damit auch die alltäglichen Themen aus dem Blog und genauso wandelten sich die kurzen Wortmeldungen zu längeren Themenartikeln. Um dem gerecht zu werden, begann ich, nicht mehr passende Artikel offline zu nehmen, um eine ausgewogenere Webseite zu erhalten. Doch die kleineren Arbeiten hier und da führten eher dazu, dass die Seite immer inkonsistenter wurde. Und das ursprüngliche Spreu-vs.-Weizen-Konzept machte keinen Sinn mehr. Ein Neuanfang musste her.
Der begann damit, dass ich mich auf Github und auf Twitter umbenannte. Der Nickname, den ich mir ausgesucht habe, basiert entfernt auf meinem Nachnamen und nicht mehr auf einem thematischen Konzept.
Wenn ich schon dabei war, alles neu zu machen, wollte ich in dem Zuge auch gleich die technische Basis meiner Onlinepräsenz überarbeiten. Seit ich mit WeizenSpr.eu begonnen habe, verwende ich für alle Webseiten WordPress, was an vielen Stellen eher übertrieben ist. Ich wollte etwas kleineres, schlankeres, schnelleres haben. Ich schaute mir verschiedene dateibasierte Content-Management-Systeme an, war jedoch mit keinem wirklich zufrieden. Also begann ich, in meiner Freizeit das CMS Urlaube zu entwerfen und zu programmieren.
Als das CMS nun im Juli 2018 halbwegs Fortschritte gemacht hatte, begann ich, die Artikel von WeizenSpr.eu zu übertragen. Ich nahm mir jeden Artikel einzeln vor und überlegte, ob er zur neuen Webseite passen würde. Falls ja, migrierte ich ihn händisch von HTML zu Markdown. Dabei nahm ich mir die Zeit, Formulierungen zu verbessern, defekte Links zu reparieren und veraltete Inhalte zu aktualisieren. Von den ursprünglich 667 Artikeln haben es 111 Artikel auf die neue Webseite geschafft. All diese Artikel befinden sich in der Legacy-Kategorie.
Die nächste Zeit werde ich nun dafür nutzen, neue Artikel zu verfassen, die ich bereits ewig vor mir hergeschoben habe, weil ich unbedingt bis zum Relaunch der Seite warten wollte.
Wie euch aufgefallen sein dürfte, gibt es aktuell noch keine Kommentarfunktion. Ich bin bereits am überlegen, ob und wie diese am besten implementiere. Bis dahin habt ihr natürlich die Möglichkeit, mich per Mastodon, Twitter oder über das Mailkontaktformular zu erreichen.
