Realistically, since they just autoescalate if the incident isn't acknowledged it appears, one would think any/all notification methods would be a positive. Let the user (or their manager...) sort out what method(s) works best for them.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:01:21.000Z","created_at_i":1682262081,"id":35676816,"options":[],"parent_id":35676788,"points":null,"story_id":35675029,"text":"I'm also not sure about why multiple notification channels can't be used. If there's an alert I'd like to be notified by all means possible (depending on severity)","title":null,"type":"comment","url":null},{"author":"smachiz","children":[{"author":"mads_quist","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-23T16:24:04.000Z","created_at_i":1682267044,"id":35677600,"options":[],"parent_id":35677026,"points":null,"story_id":35675029,"text":"I think this is a better model than what I've seen setup in PD.
Typically there was a fall-back escalation, and then it would go straight up the management chain, which seemed idiotic.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:24:51.000Z","created_at_i":1682263491,"id":35677026,"options":[],"parent_id":35676853,"points":null,"story_id":35675029,"text":"Yes, it will auto-escalate to your team mates, if you don't acknowledge.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:05:51.000Z","created_at_i":1682262351,"id":35676853,"options":[],"parent_id":35676788,"points":null,"story_id":35675029,"text":"I think the design is you use the automatic escalation if the page isn't acknowledged.","title":null,"type":"comment","url":null},{"author":"mads_quist","children":[{"author":"paol","children":[],"created_at":"2023-04-23T15:51:01.000Z","created_at_i":1682265061,"id":35677265,"options":[],"parent_id":35677018,"points":null,"story_id":35675029,"text":"A call is more reliable.
Furthermore if a call can't be placed the originating system is aware of this immediately, and can proceed accordingly (e.g. move to the next person in line). This isn't the case for any other notification method, which requires the (absense of) an explicit ack by the user to infer that the notification failed.
Pagerduty handles this very well. If you have the app it will first try to notify through the app. If there is no ack within a couple of minutes it calls you.
If you persevere with this project you'll quickly realize you need phone calls.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:23:54.000Z","created_at_i":1682263434,"id":35677018,"options":[],"parent_id":35676788,"points":null,"story_id":35675029,"text":"I cannot ensure that notifications are received. But neither can you with Email or SMS. Technically a Call might be more reliable, but still, your phone could be turned off or might not even have reception for a call.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T14:58:31.000Z","created_at_i":1682261911,"id":35676788,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"> Don't lose alerts. We believe that crucial incidents need to have a dedicated context. Alerts sent out through Slack or Email can often be overlooked in those channels. That's why All Quiet notifications are app only.
Question - How does your app ensure incident notifications are received? I haven't used PagerDuty before, but for others I've used, we got a phone call for alerts and sometimes texts for warnings.
Recently, I've implemented Android notifications for an app. Even if you set "priority" : "high" (FCM level), some will still not be received right away, depending on battery level and polling frequency at device level. You may ask the user to disable battery optimizations for your app, but a call is still more reliable IMO.","title":null,"type":"comment","url":null},{"author":"leetrout","children":[],"created_at":"2023-04-23T14:58:50.000Z","created_at_i":1682261930,"id":35676791,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Thank you for this!
There is so much room for more affordable pricing in software.","title":null,"type":"comment","url":null},{"author":"madmod","children":[],"created_at":"2023-04-23T15:01:09.000Z","created_at_i":1682262069,"id":35676814,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"You should have a status page with alerts to build trust about your uptime as a new company.
I love your simple design in the app and the site.","title":null,"type":"comment","url":null},{"author":"dmattia","children":[{"author":"mads_quist","children":[{"author":"BryantD","children":[{"author":"mads_quist","children":[{"author":"zufallsheld","children":[{"author":"yawaramin","children":[{"author":"zufallsheld","children":[],"created_at":"2023-04-23T17:01:56.000Z","created_at_i":1682269316,"id":35678042,"options":[],"parent_id":35677820,"points":null,"story_id":35675029,"text":"> What are you supposed to do exactly when you are in a region with spotty internet and you get a page?
Go somewhere with better internet access. Not all incidents require that you must immediately check on it.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:41:10.000Z","created_at_i":1682268070,"id":35677820,"options":[],"parent_id":35677473,"points":null,"story_id":35675029,"text":"What are you supposed to do exactly when you are in a region with spotty internet and you get a page? Jump on an incident bridge, pull up the logs and dashboards, and start debugging? If are you going to somewhere with spotty internet, presumably it didn't just happen all of a sudden and you knew in advance, and thus could have swapped on-call duty with someone else who would have reliable internet?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:11:57.000Z","created_at_i":1682266317,"id":35677473,"options":[],"parent_id":35676999,"points":null,"story_id":35675029,"text":"I for one sometimes are in regions where internet access is spotty, but getting a call works.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:21:41.000Z","created_at_i":1682263301,"id":35676999,"options":[],"parent_id":35676934,"points":null,"story_id":35675029,"text":"Too bad. :( Would've loved to help you out. Would you mind elaborating on your usecases and why it's axactly a deal breaker?","title":null,"type":"comment","url":null},{"author":"BryantD","children":[],"created_at":"2023-04-26T18:13:56.000Z","created_at_i":1682532836,"id":35717842,"options":[],"parent_id":35676934,"points":null,"story_id":35675029,"text":"dmattia noted the most important one. It\u2019s a low probability situation but it does occur, and in some cases people reasonably object to installing an app because of a work requirement.
I also strongly prefer multiple notification channels making noise on my phone at night. A page usually wakes me up. A page plus a phone ringing always does, so far.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:14:07.000Z","created_at_i":1682262847,"id":35676934,"options":[],"parent_id":35676906,"points":null,"story_id":35675029,"text":"This a deal breaker for me. I understand your intention here and from a pure alerting perspective you're correct, but I have use cases for other forms of notification.","title":null,"type":"comment","url":null},{"author":"dmattia","children":[],"created_at":"2023-04-23T15:51:40.000Z","created_at_i":1682265100,"id":35677275,"options":[],"parent_id":35676906,"points":null,"story_id":35675029,"text":"I guess I just can't work for any company using All Quiet then, which is fair as I understand both my perspective and yours. But just adding this comment as a data point to consider, and I wish you the best.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:11:05.000Z","created_at_i":1682262665,"id":35676906,"options":[],"parent_id":35676850,"points":null,"story_id":35675029,"text":"Yes, it doesn't have SMS / call Options explicitly.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:05:15.000Z","created_at_i":1682262315,"id":35676850,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"> That's why All Quiet notifications are app only.
Does this mean that there are no text/call options? I don't have a smartphone to install apps, but haven't had problems with being on on-call rotations with some of the other tools mentioned.","title":null,"type":"comment","url":null},{"author":"steveBK123","children":[{"author":"mads_quist","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-23T15:31:51.000Z","created_at_i":1682263911,"id":35677097,"options":[],"parent_id":35676990,"points":null,"story_id":35675029,"text":"I think the problem with integration is probably a man-hours thing.\nA lot of shops have a hodgepodge of 3rd party tools which are not necessarily the latest&greatest Cloud/FAANG darlings - Geneos, Jira, ServiceNow, ControlM, Autosys, Tidal.
These tools can vary by industry & org size, so maybe finding some traction in a particular slice so that you can really serve that niche can give you more traction, I dunno.
It just has to be dead simple to integrate, and it's worth considering some sort of professional services/customer success engineering layer.
The problem I've seen is that in every (financial services) org I have worked, the teams responsible for integrations to these types of tools are really operations type teams that do a little development in their spare time. So it never gets done well, or completely.
To me, the only benefit of PagerDuty type tool is when it becomes the single point of escalation. If its just one of many, the product will not be sticky within a given org.","title":null,"type":"comment","url":null},{"author":"steveBK123","children":[],"created_at":"2023-04-23T15:42:00.000Z","created_at_i":1682264520,"id":35677191,"options":[],"parent_id":35676990,"points":null,"story_id":35675029,"text":"Re: cost & holidays/hours
On pricing - I don't think it matters for small orgs the way it does for big orgs.\nLet's say today I am in a role where I am effectively the CTO of a 5ish person startup, with only 3 technical. So whether I spend $200/year or $1500/year, I don't really care. Both are more than $0, and require me to review a contract, do a POC, have a lawyer or someone review things, etc.
My internal time to do all those things is more like $10-30k, easily.
Now, do I decide it's worth my time knowing that my devs are going to basically turn off their phones / go to do-not-disturb-mode still because it's sense of working hours is non-existent (making it worse than PagerDuty & Slack notification tuning, both of which I already despise)?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:20:37.000Z","created_at_i":1682263237,"id":35676990,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"I tried to make managing the incident escalation policies as simple as possible. It currently works with a simple drag&drop mechanism. The way PagerDuty is doing this also seemed very cumbersome to me. \nOf course All Quiet is currently missing holiday / off hours features. I'm still figuring out what's the best approach there.
Regarding your last paragraph - can you elaborate on this? I tried to make All Quiet be integratable with basically every system by Email and Webhook integrations. Do you see this as a problem?
Thanks for your feedback in general!","title":null,"type":"comment","url":null},{"author":"PragmaticPulp","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-23T15:58:16.000Z","created_at_i":1682265496,"id":35677334,"options":[],"parent_id":35677272,"points":null,"story_id":35675029,"text":"It felt like a tool that was sold to CTOs, not users.
As a recipient of pages, I abhorred PagerDuty more than Slack, Teams, email or even phone calls.
At least my phone notifications & do not disturb mode can be tuned around some of those other apps. And if a support guy or teammate calls me on holiday, I can berate them such that they remember to check the holiday calendar / out-of-office status next time. No such luck with PagerDuty.
If your org is not mature enough to put in the time to have a single pane of glass / view of the world / alert state.. you shouldn't be buying any of these tools.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:51:24.000Z","created_at_i":1682265084,"id":35677272,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"I\u2019m glad to hear I\u2019m not the only one. I constantly felt like I was missing something. A tool this mature and popular should have been a little more polished by now.","title":null,"type":"comment","url":null},{"author":"singron","children":[{"author":"steveBK123","children":[{"author":"danpalmer","children":[{"author":"thayne","children":[],"created_at":"2023-04-24T00:18:39.000Z","created_at_i":1682295519,"id":35682169,"options":[],"parent_id":35680597,"points":null,"story_id":35675029,"text":"That isn't universally true. At my medium sized company, the people on call are the ones responsible for choosing an on-call system. But we've rotated through most of the main options, because we haven't really been happy with any of them.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T21:02:57.000Z","created_at_i":1682283777,"id":35680597,"options":[],"parent_id":35677420,"points":null,"story_id":35675029,"text":"> the purchase of these tools is a top-down affair
This is absolutely it. Pager Duty don't sell on-call tools to engineers, they sell the idea of having an on-call rota to CTOs, the tools are an implementation detail.
Bottom up engineering tools take more time to start with, but almost always cost less in the long run, contribute to a good engineering culture, and build a sense of ownership.","title":null,"type":"comment","url":null},{"author":"HereBeBeasties","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-24T01:20:05.000Z","created_at_i":1682299205,"id":35682612,"options":[],"parent_id":35681993,"points":null,"story_id":35675029,"text":"My point is not that management is bad & doesn't care, but that management has different priorities & cares about different things than the people using the app.
All of us have talked on this thread about all the things PD is bad at, but few have come to the defense with a list of things its great at.
The sales pitch, I am sure, to management is more along the lines of - metrics, dashboards, reports, "accountability", response times, yada yada.. and because management write the checks, I'm sure thats an area they do deliver.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T23:55:06.000Z","created_at_i":1682294106,"id":35681993,"options":[],"parent_id":35677420,"points":null,"story_id":35675029,"text":"I am in management. I evaluated most of the on-duty tools that are available, at the cost of a couple of days of my personal time. At least they all have free trials. PageDuty, Opsgenie, Splunk On-Call (ex VictorOps), AlertOps, Squadcast, TaskCall, etc. etc.
They are all surprisingly hopeless. For example, none of them have SCIM integration to make managing your teams in them automatic. All of them have clunky calendar overrides. None of them seem to integrate well with Outlook / Google Calendar, particularly none take into account holidays. Many have no Terraform provider to manage them, and the ones that do are clunky at best, and hard to set-up/manage. OIDC is hit-and-miss. For example, for PagerDuty you need to call up their support team and get them to manually tweak something that's not in the UI settings to get OIDC for Azure AD sign-ins to work.
It's not that management is apathetic. We genuinely don't want to engineers spending their time working around vendor inadequacies and lashing this stuff together with barely-maintained scripting that they resent having to write in the first place. Why would anyone want that? Given there's seemingly no product out there that lets you avoid that, what should we do? When they're all rubbish, you either choose PagerDuty because everyone does, or Opsgenie as a protest vote, because at least both have Terraform providers and plug-ins for other things like Slack and Sentry, etc.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:06:11.000Z","created_at_i":1682265971,"id":35677420,"options":[],"parent_id":35677340,"points":null,"story_id":35675029,"text":"I think outside some FAANG firms with strong engineering management, the purchase of these tools is a top-down affair.
So as cynical as my statement may seem- " It's almost like they wanted it to be easier to just not take time off?" .. user ergonomics just doesn't enter into the conversation at all, because the users are internal devs.. who cares.
I also worked at an org that gave each of 50+ large customers a dedicated slack channel with.. wait for it.. the entire internal 200+ member engineering org in channel with them. You can imagine the mayhem as each customer figured out how to get attention (@here/@channel/@theguythatfixedthislasttime) and "hack the phone tree" as I used to call it.
This also meant the every one of our 200+ engineers was a member of 50+ customer channels which were filled with all levels of noise that 95% of the time had nothing to do with their app/team/function, and could/should be ignored.","title":null,"type":"comment","url":null},{"author":"lIl-IIIl","children":[{"author":"zamnos","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-24T01:55:13.000Z","created_at_i":1682301313,"id":35682838,"options":[],"parent_id":35682618,"points":null,"story_id":35675029,"text":"And a $3B market cap, unbelievably.
What's kind of incredible too is they actually lose money still.\nThe losses are growing faster than the revenue.\nThey've only had, what, 14 years to figure this out?\nYou wonder what their staff of 1000ish are working on.
Also, forgot this gem: "On January 21, 2023 PagerDuty CEO Tejada's layoff memo was criticized for insensitivity for inappropriately quoting Martin Luther King, announcing promotions of executives, and tone deafness." (from wiki)","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T01:21:42.000Z","created_at_i":1682299302,"id":35682618,"options":[],"parent_id":35682532,"points":null,"story_id":35675029,"text":"You're right, but for $173mm in funding, you'd think you could build a product for external users that's at least feature parity with a "cooked up" internal project.","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T01:08:08.000Z","created_at_i":1682298488,"id":35682532,"options":[],"parent_id":35677340,"points":null,"story_id":35675029,"text":"It's much much much easier to cook up a solution for internal use than to create a product for external use that you can sell and that caters to your customers whims and that reasonably easy to use without requiring direct access to the engineering team for support.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:58:43.000Z","created_at_i":1682265523,"id":35677340,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"For instance at Google, taking vacation in workday puts an OOO event on your Google calendar. Then the oncall scheduler takes that into account along with your oncall history when it schedules new rotations. You only have to get someone to cover if you are taking time off in the near future, and any surplus or deficit will be fixed by the scheduler going forward.
It's kind of wild that an expensive product with tons of funding and employees is worse than some scripts cooked up by some SREs.","title":null,"type":"comment","url":null},{"author":"BlackjackCF","children":[],"created_at":"2023-04-23T16:05:07.000Z","created_at_i":1682265907,"id":35677406,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"I\u2019m not sure why PD is so expensive, but I think why it\u2019s just known over competitors is just the sheer number of integrations it supports.","title":null,"type":"comment","url":null},{"author":"throwmeaway1212","children":[{"author":"steveBK123","children":[],"created_at":"2023-04-23T17:42:59.000Z","created_at_i":1682271779,"id":35678532,"options":[],"parent_id":35678513,"points":null,"story_id":35675029,"text":"The problem is top-down CTO purchasing of these magic bullet tools w/o budgeting/planning integration.
Instead becoming "the place to look" it becomes "the Nth place to look" for alerts. Without thoughtful planning, its entirely possible to be in worse shape for having purchased it.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:40:44.000Z","created_at_i":1682271644,"id":35678513,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"'Taking a holiday became like a 12 step program - email boss in advance, put in HR system request, cross check your PagerDuty schedule, negotiate a trade with another teammate for PagerDuty rotation, decline meetings, update your outlook calendar out-of-office, set your slack status & notifications, and re-remind everyone the week before you go. It's almost like they wanted it to be easier to just not take time off?'
Most of this has nothing to do with pager duty, but instead is around the companies process.","title":null,"type":"comment","url":null},{"author":"SergeAx","children":[],"created_at":"2023-04-24T14:20:27.000Z","created_at_i":1682346027,"id":35687994,"options":[],"parent_id":35676864,"points":null,"story_id":35675029,"text":"> decline meetings, update your outlook calendar out-of-office
When I am adding OOO to my Google Calendar, there's an option to auto-decline meetings. Isn't it like that in Outlook?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:07:05.000Z","created_at_i":1682262425,"id":35676864,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I found PagerDuty to be grossly underwhelming when our team moved to it.\nI'd focus on stupid-easy integration & some reasonable calendar management to differentiate.
PagerDuty Calendar/Holiday/Workday management and cross-regional scheduling were very poorly implemented.
The amount of manual schedule adjustment when someone actually wanted to take their 1 week vacation was insane. We ended up with overrides on top of overrides. \nI'd imagine some of the common tools like Workday should give you integrations for that.
Taking a holiday became like a 12 step program - email boss in advance, put in HR system request, cross check your PagerDuty schedule, negotiate a trade with another teammate for PagerDuty rotation, decline meetings, update your outlook calendar out-of-office, set your slack status & notifications, and re-remind everyone the week before you go.\nIt's almost like they wanted it to be easier to just not take time off?
The other problem with these tools is you only get value out of them depending on the level of systems integration you spend your time on. If even a single system is not integrated & still sends emails/slacks/only updates a dashboard .. then PagerDuty is simply yet-another-tool to monitor, rather than the single pane of glass.","title":null,"type":"comment","url":null},{"author":"mLuby","children":[{"author":"mads_quist","children":[],"created_at":"2023-04-23T15:16:04.000Z","created_at_i":1682262964,"id":35676945,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"On the pricing page I'm explaining that I think it's the fairest option. If you have a small team, you probably have "less money" than a huge team.","title":null,"type":"comment","url":null},{"author":"smachiz","children":[{"author":"LadyCailin","children":[{"author":"smachiz","children":[],"created_at":"2023-04-23T15:40:47.000Z","created_at_i":1682264447,"id":35677178,"options":[],"parent_id":35677106,"points":null,"story_id":35675029,"text":"There\u2019s a big gap between $5 and $21\u2026 for now. That I have no doubt they\u2019ll spend all their time building features so they can increase prices to close that gap.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:32:40.000Z","created_at_i":1682263960,"id":35677106,"options":[],"parent_id":35677090,"points":null,"story_id":35675029,"text":"Presumably when someone says that such a model frustrates them. But perhaps not.","title":null,"type":"comment","url":null},{"author":"iso1631","children":[{"author":"smachiz","children":[{"author":"iso1631","children":[{"author":"smachiz","children":[],"created_at":"2023-04-23T16:49:40.000Z","created_at_i":1682268580,"id":35677897,"options":[],"parent_id":35677855,"points":null,"story_id":35675029,"text":"It is interesting re: copyright, but rarely are software IP rights enforced via copyright. Patents are more likely.
Copyright for software is a very low bar to hurdle.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:45:29.000Z","created_at_i":1682268329,"id":35677855,"options":[],"parent_id":35677689,"points":null,"story_id":35675029,"text":"> landlords will exit the market until equilibrium is restored
Thus a increasing the supply of properties being sold, and thus lowering the price for those who want to buy, allowing more to buy, reducing the demand for places to rent. Win-Win.
With LVT (or higher interest rate on a variable rate mortgage) the parasitical landlord can't simply stop having a tenant, as they're still liable for the costs. They have to offer something more than the intrinsic cost of land they occupy to make it productive enough for someone to pay them.
> LVT also only applies to commodities, of which software isn\u2019t really.
There's a strong argument that copyright (specifically that used to prevent people from using it) would count as "Land" in the economic sense - it drives rentseeking and monopoly behavior - https://progressandpoverty.substack.com/p/possibility-space-...
An interesting debate, but somewhat offtopipc","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:31:19.000Z","created_at_i":1682267479,"id":35677689,"options":[],"parent_id":35677487,"points":null,"story_id":35675029,"text":"I mean, LVT or not, that is probably true. If the price increase affects the entire market, landlords will exit the market until equilibrium is restored (or renters will increase their willingness to pay a higher rate).
If market price < cost + desired margin, some sellers will exit, reducing supply and increasing market price.
LVT also only applies to commodities, of which software isn\u2019t really.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:12:46.000Z","created_at_i":1682266366,"id":35677487,"options":[],"parent_id":35677090,"points":null,"story_id":35675029,"text":"A big argument against LVT that people (often landlords, but many "socialists" too) is that rents will simply rise to reflect the extra cost to the landlord
Same with increase interest rates
It's certainly widely believed that "price = cost + %profit" rather than "price = level at which maximum profit will be achieved after factoring in market segregation"","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:30:53.000Z","created_at_i":1682263853,"id":35677090,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"Since when does anyone set or scale their price on their costs?","title":null,"type":"comment","url":null},{"author":"ratg13","children":[{"author":"skeeter2020","children":[{"author":"mlhpdx","children":[],"created_at":"2023-04-23T16:57:40.000Z","created_at_i":1682269060,"id":35677997,"options":[],"parent_id":35677444,"points":null,"story_id":35675029,"text":"There are also plenty of companies that run into trouble with excessive \u201cvalue-based\u201c pricing, aka. The Innovator\u2019s Dilemma, as Pager Duty is now.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:08:44.000Z","created_at_i":1682266124,"id":35677444,"options":[],"parent_id":35677179,"points":null,"story_id":35675029,"text":"Like most business advice this sounds good as a one-liner, but isn't super accurate. There are lots of businesses that need to evaluate price elasticity and substitution which is more than finding the right point on the demand curve.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:40:48.000Z","created_at_i":1682264448,"id":35677179,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"If you ever start a business selling anything at all, the very first thing you will need to realize is that you never base prices on how much something costs.
You base prices on what people are willing to pay.","title":null,"type":"comment","url":null},{"author":"compumike","children":[{"author":"urbandw311er","children":[{"author":"compumike","children":[],"created_at":"2023-04-23T22:52:42.000Z","created_at_i":1682290362,"id":35681524,"options":[],"parent_id":35681075,"points":null,"story_id":35675029,"text":"Shhh\u2026 that\u2019s our secret business model! Infinite revenue per user :P","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T21:56:35.000Z","created_at_i":1682286995,"id":35681075,"options":[],"parent_id":35677324,"points":null,"story_id":35675029,"text":"wait, if you remove everyone from your team it costs $32 ?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:57:01.000Z","created_at_i":1682265421,"id":35677324,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"Heii On-Call https://heiioncall.com/ has flat pricing regardless of team size.
To be pedantic, it's free for individuals / teams-of-one, so I guess you could write:
def price_per_month(team_size)\n if team_size == 1\n 0\n else\n 32\n end\n end\nEDIT: thanks to those of you signing up to try Heii On-Call! Let me know if you have any questions.","title":null,"type":"comment","url":null},{"author":"SanderNL","children":[{"author":"mLuby","children":[{"author":"jedberg","children":[],"created_at":"2023-04-23T18:47:08.000Z","created_at_i":1682275628,"id":35679289,"options":[],"parent_id":35678398,"points":null,"story_id":35675029,"text":"You've reversed the relationship. As the employee, you're the seller. You're selling your time. It seems that you've priced yourself based on cost of living, because that is what the market will bear.","title":null,"type":"comment","url":null},{"author":"SanderNL","children":[{"author":"namaria","children":[{"author":"SanderNL","children":[{"author":"namaria","children":[],"created_at":"2023-04-26T06:56:30.000Z","created_at_i":1682492190,"id":35710336,"options":[],"parent_id":35692641,"points":null,"story_id":35675029,"text":"Yes you are missing the fact that work units are not fungible and it is not trivial for a company to calculate the marginal value of employees.
The discourse of "we are paid what we are worth in the labor market" is deeply flawed due to these reasons. So when people buy into that discourse, which advances the capital owning class interests (take what we give you that's your market value) it's both sad and harmful.
Collective bargaining and regulations play a big role in compensation levels and often political capital counts for more then some abstract and impossible to calculate market value of your skill set or work output. So please disabuse yourself of the notion that you have a fair market value as a worker and that you are paid that value. This is pure political fiction.","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T20:21:04.000Z","created_at_i":1682367664,"id":35692641,"options":[],"parent_id":35690062,"points":null,"story_id":35675029,"text":"I will be replaced and when it happens it wil be for the exact reasons I laid out above and there is nothing I can do about it.
Do we disagree about something?
It\u2019s not \u201ca take\u201d, it\u2019s how markets operate. Am I missing something?","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T16:49:22.000Z","created_at_i":1682354962,"id":35690062,"options":[],"parent_id":35685024,"points":null,"story_id":35675029,"text":"This is an easy take to have on the labor market when you have bargaining power. The fact that highly paid professionals buy so readily into the moneyed class discourse disturbs me.
Here's hoping you don't get displaced by automation and suddenly discover yourself providing zero marginal gains to them. I mean, would you then sit quietly and starve?","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T08:31:24.000Z","created_at_i":1682325084,"id":35685024,"options":[],"parent_id":35678398,"points":null,"story_id":35675029,"text":"Your employer couldn't care less about your costs of living. He is a buyer and will gladly pay exactly zero dollars for your services. Unfortunately for him, that's not something you and your friends will accept. The price is hiked until both parties can live with it.
People in different situations accept different levels of compensation. Cost of living only becomes a significant factor under competitive pressure. If you were the sole programmer for the region you would be paid millions. I'm sure you personally, being an upstanding citizen, would lower your fee to match your costs of living as to maximize your clients' profitability. But you are rare. Most will accept the highest bid.
All I'm saying is that nothing is fundamentally based on "cost" unless under pressure. If it earns you say 500 bucks, you take the 20 dollar hit. It doesn't matter to you if it costs them 1 cent to produce. Until a competitor comes and drops the prices.
Also note that this "value"-thing is complex. Programmers that live close by and are sufficiently adjusted to the culture of the company have more value than cheap, capable, but remote workers that don't "get" you. Rural programmmers are indeed no less capable than devs in major cities, but it never was just about being capable.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:31:03.000Z","created_at_i":1682271063,"id":35678398,"options":[],"parent_id":35677415,"points":null,"story_id":35675029,"text":"My employer sure does. Unless you're telling me that devs in major US cities are actually 2x better than rural US coders, who in turn are multiple times better than devs in poorer countries.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:05:46.000Z","created_at_i":1682265946,"id":35677415,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"You base your salary solely on living expenses?","title":null,"type":"comment","url":null},{"author":"quickthrower2","children":[],"created_at":"2023-04-24T01:09:43.000Z","created_at_i":1682298583,"id":35682547,"options":[],"parent_id":35676877,"points":null,"story_id":35675029,"text":"Keeping the "I want it for 5 because my indie project makes no money" people happy is less lucrative than the "We are paying $1000/person in on call allowances, so what is $21 - nothing!" people.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:08:14.000Z","created_at_i":1682262494,"id":35676877,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Why is the pricing per user? Is that what your costs are most dependent on? I bet not.
I'd argue $5/user is just teaser pricing on its way to PagerDuty's $21+/user.
Regardless, cool project\u2014happy to see more competition.","title":null,"type":"comment","url":null},{"author":"lopkeny12ko","children":[{"author":"oigursh","children":[],"created_at":"2023-04-23T15:41:20.000Z","created_at_i":1682264480,"id":35677182,"options":[],"parent_id":35676940,"points":null,"story_id":35675029,"text":"I ended up hiring a US emergency dispatcher team and repurposing them - they would literally phone you up as second line - no robots.","title":null,"type":"comment","url":null},{"author":"karussell","children":[],"created_at":"2023-04-23T19:37:09.000Z","created_at_i":1682278629,"id":35679729,"options":[],"parent_id":35676940,"points":null,"story_id":35675029,"text":"I did not yet had a problem with notifications in Android but I also see app-only as a disadvantage. What if I have no internet but could be reachable via SMS/phone? (still not that rare in Germany ;))
btw: there is an helpful Android app "missed notifications reminder" in fdroid store that re-triggers app-specific notifications if you do not view them within a minute.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:15:11.000Z","created_at_i":1682262911,"id":35676940,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"> That's why All Quiet notifications are app only.
This is prone to loss. I've struggled with notification delivery reliability on Android for years. If I don't touch my phone for a few hours and then wake the screen, I get a flurry of notifications all at once from the last few hours. But I need to be aware of pages in real-time.
There should really be a call and text notification channel else I don't see this getting much traction over PD.","title":null,"type":"comment","url":null},{"author":"blantonl","children":[{"author":"biohax2015","children":[],"created_at":"2023-04-23T15:55:03.000Z","created_at_i":1682265303,"id":35677307,"options":[],"parent_id":35677269,"points":null,"story_id":35675029,"text":"What does this add to the conversation? You didn\u2019t even put in enough effort to complain about actual things\u2026","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:51:19.000Z","created_at_i":1682265079,"id":35677269,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I'm frustrated with the pricing of Pager Duty, so I'm developing my own solution and making it available here
What about a?
It won't do b, c, d, x, and y, that's a deal breaker for me.
Can I do d, f, t, m - no? Ugh, nice try good luck on your project.","title":null,"type":"comment","url":null},{"author":"mdaniel","children":[{"author":"mads_quist","children":[],"created_at":"2023-04-23T17:22:27.000Z","created_at_i":1682270547,"id":35678307,"options":[],"parent_id":35677291,"points":null,"story_id":35675029,"text":"Good Point, MongoDB supports multi document ACID transactions for more than 5 years now.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:53:43.000Z","created_at_i":1682265223,"id":35677291,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I'm surprised to see you used Mongo (https://github.com/AllQuietApp/MongoQueueing#what-about-rabb...) after all the replies in this thread citing "delivery reliability"
What did you find that made NoSQL a good fit for tracking incidents and escalations?","title":null,"type":"comment","url":null},{"author":"mcsniff","children":[{"author":"mads_quist","children":[],"created_at":"2023-04-23T17:33:26.000Z","created_at_i":1682271206,"id":35678426,"options":[],"parent_id":35677342,"points":null,"story_id":35675029,"text":"Thanks for pointing out to go through the docs section. Appreciate it.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T15:59:06.000Z","created_at_i":1682265546,"id":35677342,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I would recommend running through your docs with a native english speaker.
Congratulations on the launch! Be proud of what you've put together here, and looks to be all by yourself too? The infra for a site, backend, and polished mobile apps for both platforms on your own.
From another "I'm doing it all myself" dude, very well done!","title":null,"type":"comment","url":null},{"author":"mbaris","children":[],"created_at":"2023-04-23T16:10:08.000Z","created_at_i":1682266208,"id":35677451,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I used to work at Opsgenie, which is one of the main competitors of Pagerduty, wish you good luck
Sms is a very good fallback channel for push notifications, especially when you don't have good internet coverage for an unexpected reason. I also personally hate phone calls, but I find it more effective to wake people up at night.","title":null,"type":"comment","url":null},{"author":"matrix12","children":[],"created_at":"2023-04-23T16:18:05.000Z","created_at_i":1682266685,"id":35677550,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Ever since their IPO they've been charging minibar rates for metric data storage.","title":null,"type":"comment","url":null},{"author":"29athrowaway","children":[{"author":"mads_quist","children":[{"author":"29athrowaway","children":[],"created_at":"2023-04-24T10:04:01.000Z","created_at_i":1682330641,"id":35685587,"options":[],"parent_id":35683414,"points":null,"story_id":35675029,"text":"If it's not published or part of contract negotiation, it could be zero and it would not be a problem.","title":null,"type":"comment","url":null}],"created_at":"2023-04-24T03:40:33.000Z","created_at_i":1682307633,"id":35683414,"options":[],"parent_id":35677608,"points":null,"story_id":35675029,"text":""...requires higher high availability than your app."
How do you know the availability of All Quiet? :)","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:24:25.000Z","created_at_i":1682267065,"id":35677608,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"You can build your own Pagerduty, feature-wise. But a system like that requires higher high availability than your app.
Pagerduty's SLA for availability is 99.9% (3 nines) which is up to 43.8 minutes per month.
If you build your Pagerduty clone on AWS, your theoretical maximum availability is 99.99%.","title":null,"type":"comment","url":null},{"author":"bink","children":[{"author":"yawaramin","children":[{"author":"akavi","children":[],"created_at":"2023-04-23T19:14:30.000Z","created_at_i":1682277270,"id":35679525,"options":[],"parent_id":35677774,"points":null,"story_id":35675029,"text":"In the "old days" (2015 and before), PD used Wormly to mass-blast all engineers.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:37:45.000Z","created_at_i":1682267865,"id":35677774,"options":[],"parent_id":35677660,"points":null,"story_id":35675029,"text":"Who do PagerDuty or OpsGenie use to monitor their services?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:28:20.000Z","created_at_i":1682267300,"id":35677660,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Will you be using PagerDuty or another competitor to monitor your own service? I always wonder what these companies use... or if they rely on their own service. And if so, then if their service breaks how would they know?","title":null,"type":"comment","url":null},{"author":"ChrisInEdmonton","children":[{"author":"traceroute66","children":[{"author":"SkyPuncher","children":[],"created_at":"2023-04-23T19:01:41.000Z","created_at_i":1682276501,"id":35679426,"options":[],"parent_id":35677833,"points":null,"story_id":35675029,"text":"Sure, but contracts just establish the formal baselines for "when we have to talk".
You still want reliability that's far higher than your contract baselines.","title":null,"type":"comment","url":null},{"author":"folmar","children":[{"author":"kakwa_","children":[],"created_at":"2023-04-25T17:18:29.000Z","created_at_i":1682443109,"id":35703621,"options":[],"parent_id":35681031,"points":null,"story_id":35675029,"text":"Also, the cost of the monitored service being down far exceed (hopefully) any compensation for SLA breach.
Honestly, PD is probably the service I would the least complain about in our (large org) monitoring chain.
At least, it's reliable, which cannot be said of Newrelic for example.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T21:51:55.000Z","created_at_i":1682286715,"id":35681031,"options":[],"parent_id":35677833,"points":null,"story_id":35675029,"text":"The small print is irrelevant as the amount of money you get for SLA breach is usually less or equal to what you pay.
What matters is the execution and word of mouth. And PagerDuty is widely known to be rock solid.","title":null,"type":"comment","url":null},{"author":"jorts","children":[],"created_at":"2023-04-24T17:07:36.000Z","created_at_i":1682356056,"id":35690336,"options":[],"parent_id":35677833,"points":null,"story_id":35675029,"text":"It's actually pretty straightforward without a lot of get-outs. See Service Credits in the Service Level Agreement [1].
1: https://www.pagerduty.com/standard-service-level-agreement/","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:42:42.000Z","created_at_i":1682268162,"id":35677833,"options":[],"parent_id":35677725,"points":null,"story_id":35675029,"text":"> But you know why PagerDuty does so well? Basically bulletproof reliability. 99% uptime won't cut it. 99.9% uptime won't cut it. You need to be as close as possible to 100% uptime, no excuses.
Ah, but as always, the million dollar question is ..... What does the PagerDuty small print say ?
My guess is the small print is not "bulletproof reliability" or five-nines. I betcha their contract is full of exclusions and get-outs.
It's a bit like the famous Verizon 100% SLA. Any idiot knows its not technically feasible, but the reason you pay the Verizon-tax is so the've got some cash to pay you the inevitable SLA claims.","title":null,"type":"comment","url":null},{"author":"mads_quist","children":[{"author":"perlgeek","children":[],"created_at":"2023-04-23T18:10:44.000Z","created_at_i":1682273444,"id":35678873,"options":[],"parent_id":35678219,"points":null,"story_id":35675029,"text":"> My current stack is very robust - tried and tested in many other products that I worked in.
For uptime you also need to consider the availability of your hosting provider, you might even have to have a fallback installation at a different provider, something along these lines.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:15:14.000Z","created_at_i":1682270114,"id":35678219,"options":[],"parent_id":35677725,"points":null,"story_id":35675029,"text":"That's a very helpful comment. Thanks. I might consider adding SMS / Calls to the notification channels. I didn't look from the perspective that Folks described here.
Regarding Uptime 99.99%. That's very true and also a good hint.
My current stack is very robust - tried and tested in many other products that I worked in.
Some other HN user also suggested to include an uptime status page to create this kind of confidence.","title":null,"type":"comment","url":null},{"author":"compumike","children":[{"author":"lokar","children":[],"created_at":"2023-04-23T17:31:14.000Z","created_at_i":1682271074,"id":35678402,"options":[],"parent_id":35678274,"points":null,"story_id":35675029,"text":"Also, the system at google is not in the path of the first page (that is direct from the alert infra), the more complex system is only needed for escalation.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:20:08.000Z","created_at_i":1682270408,"id":35678274,"options":[],"parent_id":35677725,"points":null,"story_id":35675029,"text":"Anyone who has done real engineering would realize that this problem a consequence of a design flaw with PagerDuty (or other alternatives with a similar API, where alerting is only triggered directly by a webhook).
If your design requires that the alerting service can receive a one-off affirmative "something's broken" packet, then yes, you are asking an inherently unreliable distributed system (i.e. the Internet!) to reliably deliver a critical message at a time when you know something is broken. Good luck. :)
Instead, if you use something like a periodic heartbeat (also known as a dead man's switch, inbound liveness monitor, or outbound HTTP probe -- all of which we support at Heii On-Call https://heiioncall.com/ out of the box), you can tolerate some occasional lost messages, regardless of whose end they are on.
Real reliable systems (for example, embedded systems) use periodic heartbeats and watchdogs, and are usually designed to be lenient to the occasional missed heartbeat. If the system being monitored is truly down, then enough consecutive heartbeats will be missed that some threshold is reached and the on-call person can be alerted (or a watchdog timer can reboot a system, etc).","title":null,"type":"comment","url":null},{"author":"closeparen","children":[{"author":"dilyevsky","children":[],"created_at":"2023-04-23T19:56:26.000Z","created_at_i":1682279786,"id":35679887,"options":[],"parent_id":35679470,"points":null,"story_id":35675029,"text":"Yeah I\u2019ve seen a number of outages there so dunno what TP is about.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T19:07:03.000Z","created_at_i":1682276823,"id":35679470,"options":[],"parent_id":35677725,"points":null,"story_id":35675029,"text":"We use PagerDuty, but we also have an internal email/SMS based "paging backup" service and I have seen it in action 4 or 5 times in 7 years. PagerDuty isn't bulletproof.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T16:33:59.000Z","created_at_i":1682267639,"id":35677725,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I used PagerDuty for more than a decade at my previous job. I didn't care much for the UI. But you know why PagerDuty does so well? Basically bulletproof reliability. 99% uptime won't cut it. 99.9% uptime won't cut it. You need to be as close as possible to 100% uptime, no excuses. Pagerduty isn't perfect, but it was one of the most reliable services we ever used.
I sincerely wish you luck with allquiet. I just want to make very sure you are aware why people still pay for Pagerduty. To compete, you need to be looking at 99.99% uptime or better (ideally 99.999%, 5 minutes of downtime a year) where 'uptime' is defined as the ability to exercise the entire notification stack. The moment someone's site has an outage and you aren't able to deliver the notification, you lose the customer and everyone they talk to.
I also worry about in-app notifications, but that's well-covered by everyone else's comments.
Pagerduty is vulnerable. Their UI is garbage. But you need to have bullet-proof uptime to take them down. It's a tough challenge and I wish you luck!","title":null,"type":"comment","url":null},{"author":"mlhpdx","children":[],"created_at":"2023-04-23T17:02:24.000Z","created_at_i":1682269344,"id":35678046,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Is there a notification system that tries to be a little smarter about escalations? That seems a difficult area. A developer doesn\u2019t know - should this be escalated, or should it not? Is it sufficiently impactful, or not? The same questions arise in customer success who generally error on the side of raising the escalation level at the expense of rest. This seems like an area that could be vastly improved with statistics (something like Robust Decisions) or machine learning. I haven\u2019t seen that, and wonder if others have?","title":null,"type":"comment","url":null},{"author":"amluto","children":[{"author":"dmlittle","children":[{"author":"amluto","children":[],"created_at":"2023-04-24T15:16:26.000Z","created_at_i":1682349386,"id":35688678,"options":[],"parent_id":35678960,"points":null,"story_id":35675029,"text":"Sure, but this doesn\u2019t help. The \u201cPush\u201d notification type goes straight to Klaxons. I have hacked around it by delaying Push until after SMS, but that\u2019s, at best, a kludge. There should be multiple classes of push.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T18:18:15.000Z","created_at_i":1682273895,"id":35678960,"options":[],"parent_id":35678166,"points":null,"story_id":35675029,"text":"It might be plan dependent(?) but everything you\u2019re describing is configurable on your PagerDuty profile. I have it configured to send me a notification immediately followed by a call if I don\u2019t acknowledge it within a minute (yes, this is aggressive but I\u2019m in a high urgency, quick response time SLA rotation)","title":null,"type":"comment","url":null},{"author":"jq-r","children":[{"author":"viraptor","children":[{"author":"jq-r","children":[],"created_at":"2023-04-23T22:59:37.000Z","created_at_i":1682290777,"id":35681580,"options":[],"parent_id":35680280,"points":null,"story_id":35675029,"text":"Good to know, thanks. It is kind of buried and something which is difficult to find in a hurry.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T20:32:53.000Z","created_at_i":1682281973,"id":35680280,"options":[],"parent_id":35679753,"points":null,"story_id":35675029,"text":"You can manually put the service in maintenance mode. That will stop it alerting for a specified period.","title":null,"type":"comment","url":null},{"author":"mads_quist","children":[],"created_at":"2023-04-24T04:30:07.000Z","created_at_i":1682310607,"id":35683658,"options":[],"parent_id":35679753,"points":null,"story_id":35675029,"text":"With All Quiet you can acknowledge an incident. It will then stop pinging you and your colleagues. Also incidents and their triggers are idempotent, so All Quiet won't create new incidents when they're triggered several times by your monitoring solution.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T19:40:40.000Z","created_at_i":1682278840,"id":35679753,"options":[],"parent_id":35678166,"points":null,"story_id":35675029,"text":"I have absolutely no idea why PD doesn't have a big button which stops all PD notifications the moment you press it. There are very few events in life more stressful then dealing with an incident, people on the call and having your phone alerting and ringing all the frigging time. I was once told to mute myself on incident bridge because all that alerting was not only super iritating for me, but for others too. I was supposed to fix the problem but I was so distracted with alerts I wanted to snap the phone in two and punch a random PD manager in the face.
"Yes PagerDuty, I now our infra is melting, please shut the fuck up and let me do my job."
PD does have a feature IIRC "intelligent alert grouping" which will group new alerts and not display more of those. So it makes it relatively easy to snooze group of alerts for X period of time so it makes it much quieter. Unfortunately, this is locked as a feature paid extra. But still, they need a way to tell PD to stop alerting.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:10:54.000Z","created_at_i":1682269854,"id":35678166,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Can it have a calmer way to deliver notifications? The PagerDuty app goes directly from zero to klaxon, and the klaxons are horrible sounds from a bizarrely limited menu.
Ideally there would be a configurable escalation-to-the-same-user policy. I might want: vibrate-only notification, then normal notification, then critical notification, then phone call, in that order, with configurable delays. Ideally this would interact well with focus/sleep mode, so I could get a calm critical notification before full klaxons.
Other feature request: a way to tell the app to shut up already. I\u2019ve occasionally dealt with an issue causing notifications every minute or so. The last thing I need while fixing it is more notifications to ack or silence.","title":null,"type":"comment","url":null},{"author":"ginkgotree","children":[],"created_at":"2023-04-23T17:25:08.000Z","created_at_i":1682270708,"id":35678331,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Bravo. PagerDuty is long overdue to be replaced by a better service. Their pricing is exorbitant, and their sales / account team forces you into year long contracts to buy seats you don't need. We are currently being threatened with an $8k bill because they claim we didn't "cancel renewal" in time.","title":null,"type":"comment","url":null},{"author":"animitronix","children":[],"created_at":"2023-04-23T17:25:41.000Z","created_at_i":1682270741,"id":35678338,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"So cool to see someone took this up, PD is hot garbage. We'll be taking a look!!","title":null,"type":"comment","url":null},{"author":"jedberg","children":[{"author":"cylix","children":[{"author":"jedberg","children":[{"author":"icco","children":[],"created_at":"2023-04-24T03:17:47.000Z","created_at_i":1682306267,"id":35683293,"options":[],"parent_id":35680086,"points":null,"story_id":35675029,"text":"If your service needs to survive a global aws outage, you just can't run with any saas. So many of these companies are single regioned in AWS. Auth0, Okta, Datadog, many others put customers in a regional box, and if that region goes down, all of those customers go down.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T20:16:56.000Z","created_at_i":1682281016,"id":35680086,"options":[],"parent_id":35679637,"points":null,"story_id":35675029,"text":"Alerting or control plane or both? If alerting is AWS only, I'm very sad. What happens if there is a global AWS outage? Never been one yet, but never say never.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T19:27:34.000Z","created_at_i":1682278054,"id":35679637,"options":[],"parent_id":35678443,"points":null,"story_id":35675029,"text":"FYI, PagerDuty is built on top of AWS. They used to do some multi-cloud stuff, but no longer the case (too expensive, too complex, causing more issues than it solved). Source: worked there for 2 years.","title":null,"type":"comment","url":null},{"author":"mLuby","children":[{"author":"jedberg","children":[],"created_at":"2023-04-23T22:01:12.000Z","created_at_i":1682287272,"id":35681106,"options":[],"parent_id":35680652,"points":null,"story_id":35675029,"text":"We solved that by just triggering an alert five minutes after the rotation changed.
This made sure the new on-call was aware they had started their shift and had access to acknowledge the page and clear it.
And if they didn\u2019t it escalated to tier two, who would find out why tier 1 didn\u2019t clear it.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T21:09:21.000Z","created_at_i":1682284161,"id":35680652,"options":[],"parent_id":35678443,"points":null,"story_id":35675029,"text":"And yet misconfigurations still mean people don't get paged.
A simple solution is to make triggering a "fire drill" a standard part of your on-call rotation hand-off.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:34:51.000Z","created_at_i":1682271291,"id":35678443,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"The main problem that Pagerduty solved at the time was deliverability of alerts. It was hard, and still is, to solve the problem of making sure the right people actually get the alerts. It was so expensive because they have redundant hardware all over the place with different providers, and they ran it all themselves, including SMS and phone gateways.
One of their key differentiators was that they were not built on AWS, so when AWS had an outage, you still knew about it. That also made it expensive.
With Pagerduty, you're mostly paying for reliability. The peace of mind knowing that someone will get notified when there is an outage.
This looks interesting, but from your page I'm not sure how you're better than Pagerduty.
App only notifications looks like a disadvantage to me. What if push notifications are down? What if I'm on DND?
Where is your infrastructure built? Is it on a cloud provider? What happens if that provider has an outage? If you want to build a PD competitor you have to build it on your own hardware in multiple datacenters owned by different people with different interconnects. If you haven't done that how will you stay up when your customer's provider goes down so that they know about it?","title":null,"type":"comment","url":null},{"author":"siliconc0w","children":[{"author":"OJFord","children":[],"created_at":"2023-04-23T17:58:25.000Z","created_at_i":1682272705,"id":35678701,"options":[],"parent_id":35678611,"points":null,"story_id":35675029,"text":"When I used one of these (we used VictorOps^) what it was really missing was 'fairness' as you say in the 'takes/trades'. We were rarely called and paid handsomely for it, so although nobody vocalised it, there was definitely a rush to grab it (because that was the 'system') if someone announced they had to put one up for grabs (due to holiday clash or whatever).
Really I think that should have automatically de-allocated some future rotation of the taker's, and definitely would've helped if it hooked into the holiday calendaring system, so anything booked far enough in advance was already avoided.
(^Not to slam VictorOps specifically, for all I know we just didn't have it configured well and it is capable of all that.)","title":null,"type":"comment","url":null},{"author":"quickthrower2","children":[],"created_at":"2023-04-24T01:11:29.000Z","created_at_i":1682298689,"id":35682567,"options":[],"parent_id":35678611,"points":null,"story_id":35675029,"text":"What is FANNG? Facebook, Amazon, Neural Nets & Google?","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T17:50:05.000Z","created_at_i":1682272205,"id":35678611,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"At $FANNG, we have a surprisingly complex tool to handle scheduling oncall and I don't I have seen a public equivalent so there is probably an opportunity there. Ideally you to optimize around a number of factors - vacations (which can be auto populated) for one but also fairness, shift closeness (some labor laws here), holidays, or individual preferences (e.g I like to hike on the weekends). You then also need a tool for short-term trades/overrides to basically poll the team to see who can take/trade the shift to avoid the diffusion of responsibility of yelling into a slack chennel and complexity of negotiating trades. What is nice about this is that we can setup a daily oncall so if you have a re-occuring weekly obligation that is incompatible with being oncall we can set it up so you never get scheduled for that day (provided there is enough slack for vacations and so on).","title":null,"type":"comment","url":null},{"author":"snird","children":[{"author":"tky","children":[{"author":"akhayam","children":[],"created_at":"2023-04-23T19:48:52.000Z","created_at_i":1682279332,"id":35679821,"options":[],"parent_id":35679484,"points":null,"story_id":35675029,"text":"+1. I am really upset about the pricing for the value. Great that we have alternatives now","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T19:09:38.000Z","created_at_i":1682276978,"id":35679484,"options":[],"parent_id":35679154,"points":null,"story_id":35675029,"text":"They\u2019ve adopted terrible dark patterns with account management and even their service tiers. Prices keep going up; this is an excellent time for competition.","title":null,"type":"comment","url":null},{"author":"atonse","children":[],"created_at":"2023-04-23T20:21:16.000Z","created_at_i":1682281276,"id":35680138,"options":[],"parent_id":35679154,"points":null,"story_id":35675029,"text":"Yup I went from being ambivalent about PagerDuty to actively hating them. They swindle you, make it hard to cancel or downgrade, and that seems to be their way of doing business.
I\u2019ll ditch them as soon as I can.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T18:32:48.000Z","created_at_i":1682274768,"id":35679154,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Seeing this post, I went over to check how much I pay PagerDuty.
49$ per user per month!!!\nAnd I don't need the features!\nI wanted to downgrade to the 29$ immediately, at least.\nAnd then I saw this: https://support.pagerduty.com/docs/billing-invoices-payments
I have to contact their sales to downgrade.
I'm furious.\nI won't downgrade. I'll cancel completely and move to your app.","title":null,"type":"comment","url":null},{"author":"aserafini","children":[],"created_at":"2023-04-23T18:48:42.000Z","created_at_i":1682275722,"id":35679304,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"For the OP, you've got a typo in your docs.\nhttps://allquiet.app/documentation#what-is-allquiet
> by explicit excalation
should be 'escalation'.
Ps. I'm also in Berlin, and would be interested to have a chat: this product looks interesting. Email in my profile!","title":null,"type":"comment","url":null},{"author":"madmax108","children":[{"author":"motakuk","children":[],"created_at":"2023-05-04T08:16:21.000Z","created_at_i":1683188181,"id":35812631,"options":[],"parent_id":35679341,"points":null,"story_id":35675029,"text":"It also works with other alert sources like DataDog or Zabbix.","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T18:53:13.000Z","created_at_i":1682275993,"id":35679341,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Grafana OnCall is an OSS alternative (with a cloud offering) that works great out of the box if you are using Grafana/Grafana Alerting for monitoring your systems and want to have a pager-like system with phone/SMS/telegram integrations + it's own app. Best of all, it's self-hostable as well, which keeps me completely in control of my infra.
https://grafana.com/products/oncall/
https://github.com/grafana/oncall","title":null,"type":"comment","url":null},{"author":"oezi","children":[],"created_at":"2023-04-23T19:02:19.000Z","created_at_i":1682276539,"id":35679433,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"You mention price as a motivation to build allquiet. I have been happy the last couple of years with https://simplepush.io/ which is 12.49$ per year (and similarly app only).","title":null,"type":"comment","url":null},{"author":"phendrenad2","children":[],"created_at":"2023-04-23T19:16:49.000Z","created_at_i":1682277409,"id":35679544,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I worked for a company that used PagerDuty, but eventually moved to a service that provided actual physical pagers. Well, they looked like 1990s pagers, but connected to Wifi. Really cool. Not sure why more don't do this.","title":null,"type":"comment","url":null},{"author":"justinmayer","children":[],"created_at":"2023-04-23T19:21:24.000Z","created_at_i":1682277684,"id":35679581,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Frustrated by expensive per-seat pricing and unfriendly contract terms, we looked for PagerDuty alternatives and found a fantastic open-source project called Uptime Kuma: https://github.com/louislam/uptime-kuma
Uptime Kuma is one of the few open-source projects that feels like a commercial product: polished user experience, frequent release cadence, and a rich set of features including monitoring PostgreSQL servers, Docker containers, and so much more. Its list of supported notification services is so long that I don't even recognize half of the options available. Truly impressive.
Side note and shameless plug\u2026 We love Uptime Kuma so much that we made it one of the cornerstone applications provided by Fortressa, which we think of as the \u201cApp Store for Open Source\u201d: https://fortressa.com/","title":null,"type":"comment","url":null},{"author":"seangransee","children":[],"created_at":"2023-04-23T19:41:09.000Z","created_at_i":1682278869,"id":35679755,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"https://pagerduty.life","title":null,"type":"comment","url":null},{"author":"tymm","children":[],"created_at":"2023-04-23T20:16:55.000Z","created_at_i":1682281015,"id":35680085,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Cool product. Feel free to reach out at contact@simplepush.io, if you want to chat about push notifications :)","title":null,"type":"comment","url":null},{"author":"gmjosack","children":[],"created_at":"2023-04-23T22:11:47.000Z","created_at_i":1682287907,"id":35681192,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Would love to see more disruption in this space. Like others have said it's actually amazing how bad the calendar/scheduling portion of PagerDuty is. Multiple places I've worked we ended up making our own interface to managing schedules because the UI at PagerDuty is so bad. Also, people mentioning the rock solid reliability of PagerDuty likely don't have good monitoring of PagerDuty. When I worked on monitoring in the past we regularly encountered delayed notifications and ingestion failures. Pulling up their status page right now confirms they still have multiple issues a month. At a past employer we had our notification system actually check outstanding alert status and fall back to direct SMS when the ingestion latency was too high.
I worked on pygerduty, a python client library for PagerDuty, at a past job so I spent a lot of time talking to people at PagerDuty but I think they stopped inviting me to their "Founder's Club" meetings when I told them I didn't care about AI Alerts and wanted better UIs for scheduling and better reliability.","title":null,"type":"comment","url":null},{"author":"rco8786","children":[],"created_at":"2023-04-23T22:37:22.000Z","created_at_i":1682289442,"id":35681385,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I am so glad someone is doing this. I\u2019ve been using PD for 10 years and still can\u2019t figure out the UI. Things that should be dead simple (IMO) like adjusting schedules requires a separate spreadsheet to figure out.","title":null,"type":"comment","url":null},{"author":"JimXugle","children":[],"created_at":"2023-04-23T22:45:17.000Z","created_at_i":1682289917,"id":35681459,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"I can recommend GoAlert for this... It works really well!
https://github.com/target/goalert","title":null,"type":"comment","url":null},{"author":"mleonhard","children":[{"author":"mads_quist","children":[],"created_at":"2023-04-24T03:29:27.000Z","created_at_i":1682306967,"id":35683359,"options":[],"parent_id":35681680,"points":null,"story_id":35675029,"text":"Nice idea!!","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T23:12:09.000Z","created_at_i":1682291529,"id":35681680,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"Feature request: "Quiet wakeup mode" that vibrates the user's Apple Watch for a minute before making any sound. Then start the sound at the lowest volume (inaudible) and slowly increase it over 1 minute. This would be supremely useful for being oncall while sleeping near another person.
Feature request 2: "Earplug mode" that slowly increases volume and vibration from zero to max, and stays at max for 1 minute. Support alternating various tones, sounds, and voices.","title":null,"type":"comment","url":null},{"author":"winterec","children":[],"created_at":"2023-04-24T03:13:27.000Z","created_at_i":1682306007,"id":35683275,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"(This is my first post after lurking with an account for several years)
Congrats on the launch, you've done a great job. The app looks good and you've clearly and persuasively differentiated it from the competition.","title":null,"type":"comment","url":null},{"author":"armiiller","children":[],"created_at":"2023-04-27T16:44:30.000Z","created_at_i":1682613870,"id":35730838,"options":[],"parent_id":35675029,"points":null,"story_id":35675029,"text":"PagerTree (https://pagertree.com) is a great alternative to PD.
- Super simple interface, with a drag n' drop on-call scheduler.\n- Supports escalations, calendar overrides, and SSO\n- Half the price","title":null,"type":"comment","url":null}],"created_at":"2023-04-23T10:29:22.000Z","created_at_i":1682245762,"id":35675029,"options":[],"parent_id":null,"points":188,"story_id":35675029,"text":null,"title":"Show HN: I was frustrated with pricing of PagerDuty et al., so made one myself","type":"story","url":"https://allquiet.app"}