Goodbye Pushover, Hello Overpush

I have been a paid Pushover user since December 19, 2012, and have contributed to its ecosystem with projects like Lemon and pushover-to-xmpp. Even though it’s one of the last remaining non-free mobile apps that I’m using, I have been loyal to the service over all these years. Not only because I believe that its founder, Joshua, is an overall cool human doing a great job with Pushover, but also because many other platforms support Pushover right out of the box, and, if not, it’s usually fairly easy to make use of a generic HTTP trigger to push notifications to Pushover.

Back in 2022, after switching to a more privacy-respecting smartphone that lacks the support for Firebase Cloud Messaging, I contacted Pushover to ask whether they had considered offering an alternative method for Pushover to function on non-FCM devices:

Dear Pushover Team,

I’ve been using Pushover for years for all my notification needs and have always been very satisfied with the service. Recently I have switched my mobile device, though, and the new Android-based smartphone does not support Google’s Services Framework and Firebase Cloud Messaging anymore.

I have tried running the Pushover Android app, unfortunately it won’t even start due to the lack of FCM.

I’m wondering if there is any chance that Pushover would introduce a fallback-implementation into the Android app, allowing it to run as a “foreground service” under Android and use polling with the Pushover API to check for new notifications?

The following FAQ section covers this topic briefly: https://grapheneos.org/faq#notifications

Thank you and best regards, Marius

The answer that I received back in 2022 was the following:

Hi, unfortunately not. That would waste a lot of battery life, bandwidth, and server resources to be constantly polling with any reasonable timeframe for message delivery. Additionally, if the app is not installed through Google Play, we can’t reliably deliver updates to it to fix bugs or introduce new functionality.

I did understand that Pushover might not have the server bandwidth/performance in place to offer WebSockets as a fallback for non-FCM devices – even though I’d argue that the percentage of people using non-FCM devices is probably insignificant enough for performance to not become an issue.

As for the battery life, it’s true, that using WebSockets does have an impact on the battery life. However, I’d argue that anyone using a non-FCM device understands this and can decide for themselves whether an app is worth the additional toll on the battery or not.

However, the “not installed through Google Play” argument makes little to no sense at all these days and seems to be a remainder from ancient times. Not only do most Smartphone manufacturers these days have own app stores in place anyway, but with the amount of malware coming through the Google Play store I’d argue that it’s rather one of the last places you’d want to get your APKs from.

I replied to this e-mail thanking Joshua for the quick feedback but ultimately decided to let the topic rest. Since I was effectively unable to use Pushover on my phone anymore, I built pushover-to-xmpp for the time being and recently also rewrote Lemon to become usable as a desktop notifications client for Pushover.

However, I was nevertheless dissatisfied by the fact that over the years I invested a lot of time and effort into integrating Pushover for all sorts of Continuous Integration notifications and system health alerts, yet I needed to use workarounds like routing notifications through my XMPP server to get notified of things. Ultimately, by having to run additional components on some server, as well as on the desktop (Lemon) it begged the question of whether using Pushover in the first place was still a smart way forward.

With projects like UnifiedPush becoming more mature, the non-FCM ecosystem gained the ability to receive push notifications in a far smoother way than the WebSockets approach from years ago.

This year, I decided to give it another go and ask Joshua about his thoughts on using UnifiedPush for the Android Pushover client:

Hey there again Joshua,

A little over a year has passed and the GCM/FCM-free ecosystem has made big advancements, namely UnifiedPush[1].

I would kindly reiterate on the previous request from back in November ‘22 and ask whether you have plans to support this open-source initiative and implement UnifiedPush support into the official Pushover app.

Even though I have completely rewritten Lemon[2] to make it usable not only on a Raspberry Pi with Unicorn Hat, but also on a regular desktop environment running D-Bus for desktop notifications, I would still very much love to be able to receive Pushover notifications on my degoogled Android smartphone.

Let me know what you think!

Thanks and best regards,
Marius

[1]: https://unifiedpush.org
[2]: https://github.com/mrusme/lemon

Unfortunately, he never replied to this e-mail, meaning that the chances that UnifiedPush would ever come to Pushover are probably low at best. It appears as if Pushover, as a service, is being developed and run on a least-effort basis, with app updates coming in every month or two and mainly consisting of bug fixes rather than new features – which is fine. However, with my infrastructure and devices not remaining stagnant, I figured it was probably time to move along.

Let’s do UnifiedPush!

The first idea that comes to mind when hearing UnifiedPush is probably ntfy. The ntfy project consists of a server-side component, as well as a mobile client, that makes it possible to set up your personal Pushover service. In addition, ntfy offers a hosted version of their server-side for anyone who wouldn’t want to deal with hosting their infrastructure.

The ntfy Android app not only receives push notifications from the ntfy server, but it also acts as UnifiedPush distributor, meaning that every other app that supports push notifications via UnifiedPush (e.g. Element), can use the permanent connection that ntfy keeps up to tunnel their notifications through it.

However, since I’m already using a distributor app – namely Conversations – I would like all other apps, including ntfy, to use that one to tunnel UnifiedPush notifications. Unfortunately, though, this doesn’t seem possible with ntfy yet. If I was to run ntfy, my phone would hence keep two permanent connections open at all times: One for Conversations, which is connected to my XMPP server and tunnels UnifiedPush notifications through that, and the other one to the ntfy server.

That would not be ideal and lead to a noticeable battery drain. Looking around further into the UnifiedPush ecosystem, it doesn’t seem like there are other options around that would offer what ntfy offers, but that could make use of an existing distributor app.

How about PushBits?

There’s an interesting project called PushBits, that does something similar to what ntfy does: It offers a way to send push notifications to your phone, via its interface. However, unlike ntfy, PushBits doesn’t require a dedicated phone app, as it uses Matrix to deliver the notifications. It does so by logging in as a dedicated user account on Matrix and sending the notification as a conversation to any other user specified in its config. To receive the message as a push notification, the official Element/Element X client for Matrix can be used in combination with Conversations as a distributor app. This way, it’s still only Conversation that keeps a connection open in the background.

I gave PushBits a quick try but ultimately decided against using it. First of all, it doesn’t support E2EE, which means that the administrator of the Matrix home server in use can see what’s being sent as push notifications. As I do not host my own Matrix home server – yet, until either conduit or dendrite reaches 100% server-server and client-server parity – not having E2EE could be a potential security risk, depending on the content of notifications. My notifications usually do not contain sensitive information though.

Second, I encountered an issue that I haven’t been able to solve quickly enough for me to continue testing PushBits in the first place. I still don’t know what caused this issue and I didn’t dive deep enough into the Matrix API to try to figure out a solution.

Last but not least, PushBits, just like ntfy, came with a big downside which is re-writing all my services that currently perform curl requests to Pushover. And while ntfy at least appears to be an active enough project for this effort to not go down the drain once development comes to a halt, PushBits didn’t appear particularly active, to begin with. Not only would I need to change all the simple curl requests to fit PushBits’ API, but I would also eventually need to dive deep enough into what appears to be an overly complex (for what it does) code base, just to be able to iterate on things that might not work for my use case.

Then keep pushover-to-xmpp?

It appears that I would have to go back to pushover-to-xmpp nevertheless. None of the solutions I found – even the ones I haven’t explicitly listed here, like Gotify – seem to solve my particular issue in a simple enough yet elegant way for me to consider migrating my existing setup. All this try-and-error made me think though:

The only reason why pushover-to-xmpp requires the actual Pushover service is because it does not provide an HTTP API that would allow my curl requests to directly send the notifications to it. So what if I just add some Fiber into the mix and remove the Pushover integration?

Introducing: Overpush

No sooner said than done, I got down to work and can present what is the very first version of Overpush! :-)

Overpush combines the same XMPP logic used by pushover-to-xmpp, with a Fiber v3 API that emulates the Pushover /1/messages.json endpoint, to deliver push notifications via XMPP. While I have not reached 100% feature parity yet (e.g. attachments, icons), I’ll continue iterating on this to get there ASAP. However, for the most basic notifications, it works as it should already.

Overpush will accept incoming requests to its /1/messages.json endpoint and will put them into Redis using asynq. It will then reply with a successful HTTP response and continue processing the message delivery in the background. This way, in case the Overpush service should die for whatever reason, it will simply pick up on all previously submitted, undelivered notifications and try to deliver them upon restart, one by one.

Hence, the service can also be easily scaled horizontally. With asynq supporting Redis clusters as well as automatic failover, it was easy for me to add these features to Overpush. Additionally, if you’re serious about your infrastructure, asynq comes with a dedicated Web UI and metrics exporter, that allows you to view Redis performance metrics in a dedicated web interface or have them exported to your Prometheus server.

Enhancements that will soon come to Overpush contain the separation of the API from the workers delivering the actual messages over XMPP, the integration of other means of delivery apart from XMPP, and the possibility of running Overpush as Lambda/Cloud Functions. Another big thing on my to-do list for Overpush is E2EE.

At this stage, the only thing you need to set up your self-hosted version of Pushover using Overpush is a VPS running either Linux or BSD* and a Redis server. You could host both, Overpush and Redis, on the same VPS, although this would come with a risk of losing notifications in case the machine should malfunction. Instead, for a low-volume setup, you can sign up on Redis.com and run a small Redis database completely free of charge. For most people, this solution would be completely sufficient and would have the advantage that your VPS could be as small as e.g. the smallest Vultr instance available (currently $3.50/month with IPv4 or $2.50/month without).

I’m currently in the process of migrating every script and callback that is using the Pushover API to my own Overpush instance. If you’re also looking to replace Pushover with a fully open-source, self-hosted, and minimal drop-in solution, make sure to check out Overpush! While the project is still in its early stages it already does a great job delivering basic notifications sent to its API endpoint via simple curl commands. As a bonus, check out this pushover script in my dotfiles that you can use to easily submit notifications to Overpush:

$ pushover -T "Title of the notification" "This is a test notification!"

Note: Make sure to replace PUSHOVER_URL with the URL to your Overpush installation. And yes, if you’d like to use this script with Pushover, it will still work by setting PUSHOVER_URL to https://api.pushover.net/1/messages.json.