Why like this?","title":null,"type":"comment","url":null},{"author":"d4rkp4ttern","children":[{"author":"detente18","children":[{"author":"d4rkp4ttern","children":[{"author":"detente18","children":[],"created_at":"2023-08-12T21:48:11.000Z","created_at_i":1691876891,"id":37104544,"options":[],"parent_id":37018764,"points":null,"story_id":36887711,"text":"You're probably calling openai from a backend server. If you're making the call because a user asked a question -> your endpoint received it -> did some processing -> called openai -> returned the response,
why would you cache the openai call instead of the endpoint that's receiving the user call?","title":null,"type":"comment","url":null}],"created_at":"2023-08-06T03:36:12.000Z","created_at_i":1691292972,"id":37018764,"options":[],"parent_id":36892872,"points":null,"story_id":36887711,"text":"Could you elaborate on this \u2014 \u201cwouldn\u2019t want to cache around the endpoint instead of GPT call\u201d. Just want to see if I\u2019m missing an important consideration here","title":null,"type":"comment","url":null}],"created_at":"2023-07-27T13:09:58.000Z","created_at_i":1690463398,"id":36892872,"options":[],"parent_id":36892443,"points":null,"story_id":36887711,"text":"You can use tenacity for retries and wouldn't you want to cache the request / response around the endpoint instead of the gpt call -> that's what we ended up doing.
Streaming output and function-calling support is interesting","title":null,"type":"comment","url":null},{"author":"ij23","children":[],"created_at":"2023-07-27T19:46:14.000Z","created_at_i":1690487174,"id":36899124,"options":[],"parent_id":36892443,"points":null,"story_id":36887711,"text":"good points, probably going to add streaming output, function calling support. As for retries tenacity does a great job already","title":null,"type":"comment","url":null}],"created_at":"2023-07-27T12:30:47.000Z","created_at_i":1690461047,"id":36892443,"options":[],"parent_id":36887711,"points":null,"story_id":36887711,"text":"Great start!\nAre you planning to add to following:
Retries w exponential backoff,\nCaching,\nStreaming output,\nFunction-calling support","title":null,"type":"comment","url":null},{"author":"kaushik92","children":[{"author":"detente18","children":[],"created_at":"2023-07-27T16:47:33.000Z","created_at_i":1690476453,"id":36896359,"options":[],"parent_id":36896206,"points":null,"story_id":36887711,"text":"Thanks!","title":null,"type":"comment","url":null},{"author":"ij23","children":[],"created_at":"2023-07-27T19:16:50.000Z","created_at_i":1690485410,"id":36898723,"options":[],"parent_id":36896206,"points":null,"story_id":36887711,"text":"Thanks!","title":null,"type":"comment","url":null}],"created_at":"2023-07-27T16:38:07.000Z","created_at_i":1690475887,"id":36896206,"options":[],"parent_id":36887711,"points":null,"story_id":36887711,"text":"This is amazing. Really needed something like this to standardize all my different AI APIs!
On a side note - I love how quickly your team is shipping! Do keep it going!","title":null,"type":"comment","url":null},{"author":"uripeled2","children":[{"author":"ij23","children":[],"created_at":"2023-07-27T22:26:01.000Z","created_at_i":1690496761,"id":36900965,"options":[],"parent_id":36900108,"points":null,"story_id":36887711,"text":"thanks for sharing, while your library looks really powerful my goal with Litellm is simplicity","title":null,"type":"comment","url":null}],"created_at":"2023-07-27T21:00:59.000Z","created_at_i":1690491659,"id":36900108,"options":[],"parent_id":36887711,"points":null,"story_id":36887711,"text":"Take a look at llm-client a similar library that also support chat, async and more llm providers\nhttps://github.com/uripeled2/llm-client-sdk","title":null,"type":"comment","url":null}],"created_at":"2023-07-27T01:31:35.000Z","created_at_i":1690421495,"id":36887711,"options":[],"parent_id":null,"points":62,"story_id":36887711,"text":"I built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. litellm has two functions - completion(), embedding()","title":"Show HN: Litellm \u2013 Simple library to standardize OpenAI, Cohere, Azure LLM I/O","type":"story","url":"https://github.com/BerriAI/litellm"}