{"author":"behnamoh","children":[{"author":"bradknowles","children":[{"author":"behnamoh","children":[],"created_at":"2023-10-09T01:33:59.000Z","created_at_i":1696815239,"id":37816027,"options":[],"parent_id":37815996,"points":null,"story_id":37815938,"text":"Not really. They seem more complicated to manage (like on Azure you need to create several "resources" before you can actually run Azure Web Apps). With Runpod and Vast.ai, I can simply use TheBloke's templates in a few clicks.","title":null,"type":"comment","url":null}],"created_at":"2023-10-09T01:29:16.000Z","created_at_i":1696814956,"id":37815996,"options":[],"parent_id":37815938,"points":null,"story_id":37815938,"text":"Have you tried AWS? Or other big cloud providers?","title":null,"type":"comment","url":null},{"author":"edgoode","children":[],"created_at":"2023-10-09T01:42:37.000Z","created_at_i":1696815757,"id":37816079,"options":[],"parent_id":37815938,"points":null,"story_id":37815938,"text":"We have aggregated most cloud GPU providers into a single platform, so you have an easier time getting the machines you need with our aggregated availability. You can check live availability and prices at https://shadeform.ai

Disclaimer - I am one of the cofounders.","title":null,"type":"comment","url":null}],"created_at":"2023-10-09T01:19:31.000Z","created_at_i":1696814371,"id":37815938,"options":[],"parent_id":null,"points":3,"story_id":37815938,"text":"Runpod used to have high availability of 4090 GPUs just about a month ago, but it seems like something has happened that has made almost all of them become unavailable. Now when I want to deploy my model, I get "low availability" messages and sometimes the instance doesn't even boot up correctly.

I tried vast.ai as well but the way they do things is rather primitive and unprofessional in that there really isn't a straightforward way to get into your instance and install booga or whatever. Their SSH failed every single time as well.

I don't know what other options are available for people who want to run research on LLMs and don't have the budget to purchase a big GPU (even if it were available...).","title":"Ask HN: Why cloud GPUs have low availability?","type":"story","url":null}