{"exhaustive":{"nbHits":false,"typo":false},"exhaustiveNbHits":false,"exhaustiveTypo":false,"hits":[{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tomschwiha"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"New GPT-3 model: <em>text-DaVinci-003</em>"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://beta.openai.com/docs/models/overview"}},"_tags":["story","author_tomschwiha","story_33780720"],"author":"tomschwiha","children":[33780757,33781019,33781155,33781190,33781252,33781316,33781437,33781458,33781511,33781568,33781652,33781828,33782061,33782087,33782254,33782397,33783052,33783250,33783925,33783970,33783973,33783974,33784030,33784158,33784320,33784323,33784875,33785369,33785645,33785707,33785797,33786818,33787970,33787986,33789141,33789515,33789531,33798130,33856385,33856393,33905708],"created_at":"2022-11-28T22:22:54Z","created_at_i":1669674174,"num_comments":206,"objectID":"33780720","points":252,"story_id":33780720,"title":"New GPT-3 model: text-DaVinci-003","updated_at":"2024-09-20T12:41:08Z","url":"https://beta.openai.com/docs/models/overview"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jamesjyu"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"GPT (<em>text-DaVinci-003</em>) can run hallucinated scripts"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://twitter.com/jamesjyu/status/1600959843669377025"}},"_tags":["story","author_jamesjyu","story_33913477"],"author":"jamesjyu","children":[33914636],"created_at":"2022-12-08T21:11:31Z","created_at_i":1670533891,"num_comments":2,"objectID":"33913477","points":17,"story_id":33913477,"title":"GPT (text-DaVinci-003) can run hallucinated scripts","updated_at":"2024-09-20T12:45:58Z","url":"https://twitter.com/jamesjyu/status/1600959843669377025"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"axel_hwang"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Alpaca-LoRA: finetuning LLaMA to <em>text-DaVinci-003</em> quality on consumer hardware"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/tloen/alpaca-lora"}},"_tags":["story","author_axel_hwang","story_35164105"],"author":"axel_hwang","created_at":"2023-03-15T05:02:58Z","created_at_i":1678856578,"num_comments":0,"objectID":"35164105","points":8,"story_id":35164105,"title":"Alpaca-LoRA: finetuning LLaMA to text-DaVinci-003 quality on consumer hardware","updated_at":"2024-09-20T13:37:56Z","url":"https://github.com/tloen/alpaca-lora"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Breefield"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Terminal-copilot: command line wrapper around OpenAI <em>text-DaVinci-003</em>"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/Methexis-Inc/terminal-copilot"}},"_tags":["story","author_Breefield","story_33993125"],"author":"Breefield","children":[34010742],"created_at":"2022-12-15T00:28:59Z","created_at_i":1671064139,"num_comments":1,"objectID":"33993125","points":2,"story_id":33993125,"title":"Terminal-copilot: command line wrapper around OpenAI text-DaVinci-003","updated_at":"2024-09-20T12:54:38Z","url":"https://github.com/Methexis-Inc/terminal-copilot"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jamez"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Replace your fine-tuned <em>text-DaVinci-003</em> with Mixtral"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"https://jamez.it/blog/2024/01/04/replace-a-fine-tuned-<em>text-davinci-003</em>-with-mixtral-8x7b/"}},"_tags":["story","author_jamez","story_38867671"],"author":"jamez","created_at":"2024-01-04T14:51:25Z","created_at_i":1704379885,"num_comments":0,"objectID":"38867671","points":2,"story_id":38867671,"title":"Replace your fine-tuned text-DaVinci-003 with Mixtral","updated_at":"2024-09-20T16:05:00Z","url":"https://jamez.it/blog/2024/01/04/replace-a-fine-tuned-text-davinci-003-with-mixtral-8x7b/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"legnaflow"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"I'm using <em>text-davinci-003</em> to find some research backed findings to build up my startup pitch. However, when I ask for the research source (title and url) for AI's answer, the report title and urls given by AI are usually not accessible or searchable online. \nhttps://beta.openai.com/playground/p/default-chat<p>Overtime, I started suspecting, are those reports and url fake/generated by AI?<p>Examples of the reports referred by AI:\n4. Mental Health Foundation (2015). Why do some people not get the help they need? [https://www.mentalhealth.org.uk/publications/why-do-some-people-not-get-help-they-need](https://www.mentalhealth.org.uk/publications/why-do-some-people-not-get-help-they-need)\n5. National Alliance on Mental Illness (2020). Mental health conditions &amp; stigmas. [https://www.nami.org/About-Mental-Illness/Mental-Health-Conditions](https://www.nami.org/About-Mental-Illness/Mental-Health-Conditions)\n6. Substance Abuse and Mental Health Services Administration (2019). Mental health services in the United States. [https://www.samhsa.gov/find-help/national-helpline](https://www.samhsa.gov/find-help/national-helpline)\n- National Institutes of Health (NIH):&quot;The Huppert's Mental Health Spectrum: Population Distributions and Associated Factors&quot;.\nhttps://pubmed.ncbi.nlm.nih.gov/31407549/."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Ask HN: <em>Text-DaVinci-003</em>/ChatGPT made up research sources?"}},"_tags":["story","author_legnaflow","story_33929485","ask_hn"],"author":"legnaflow","children":[33929614,33929639],"created_at":"2022-12-10T03:14:09Z","created_at_i":1670642049,"num_comments":4,"objectID":"33929485","points":1,"story_id":33929485,"story_text":"I&#x27;m using text-davinci-003 to find some research backed findings to build up my startup pitch. However, when I ask for the research source (title and url) for AI&#x27;s answer, the report title and urls given by AI are usually not accessible or searchable online. \nhttps:&#x2F;&#x2F;beta.openai.com&#x2F;playground&#x2F;p&#x2F;default-chat<p>Overtime, I started suspecting, are those reports and url fake&#x2F;generated by AI?<p>Examples of the reports referred by AI:\n4. Mental Health Foundation (2015). Why do some people not get the help they need? [https:&#x2F;&#x2F;www.mentalhealth.org.uk&#x2F;publications&#x2F;why-do-some-people-not-get-help-they-need](https:&#x2F;&#x2F;www.mentalhealth.org.uk&#x2F;publications&#x2F;why-do-some-people-not-get-help-they-need)\n5. National Alliance on Mental Illness (2020). Mental health conditions &amp; stigmas. [https:&#x2F;&#x2F;www.nami.org&#x2F;About-Mental-Illness&#x2F;Mental-Health-Conditions](https:&#x2F;&#x2F;www.nami.org&#x2F;About-Mental-Illness&#x2F;Mental-Health-Conditions)\n6. Substance Abuse and Mental Health Services Administration (2019). Mental health services in the United States. [https:&#x2F;&#x2F;www.samhsa.gov&#x2F;find-help&#x2F;national-helpline](https:&#x2F;&#x2F;www.samhsa.gov&#x2F;find-help&#x2F;national-helpline)\n- National Institutes of Health (NIH):&quot;The Huppert&#x27;s Mental Health Spectrum: Population Distributions and Associated Factors&quot;.\nhttps:&#x2F;&#x2F;pubmed.ncbi.nlm.nih.gov&#x2F;31407549&#x2F;.","title":"Ask HN: Text-DaVinci-003/ChatGPT made up research sources?","updated_at":"2024-09-20T12:47:51Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dstala"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Ask HN: Why does OpenAI wants us to move away from <em>text-DaVinci-003</em>?"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://platform.openai.com/docs/guides/chat/chat-vs-completions"}},"_tags":["story","author_dstala","story_35071939"],"author":"dstala","children":[35071940],"created_at":"2023-03-08T17:26:41Z","created_at_i":1678296401,"num_comments":1,"objectID":"35071939","points":1,"story_id":35071939,"title":"Ask HN: Why does OpenAI wants us to move away from text-DaVinci-003?","updated_at":"2024-09-20T13:29:01Z","url":"https://platform.openai.com/docs/guides/chat/chat-vs-completions"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rileyt"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"This is a quick prototype I built for semantic search and factual question answering using embeddings and GPT-3.<p>It tries to solve the LLM hallucination issue by guiding it only to answer questions from the given context instead of making things up. If you ask something not covered in an episode, it should say that it doesn't know rather than providing a plausible, but potentially incorrect response.<p>It uses Whisper to transcribe, text-embedding-ada-002 to embed, Pinecone.io to search, and <em>text-davinci-003</em> to generate the answer.<p>More examples and explanations here: <a href=\"https://twitter.com/rileytomasek/status/1603854647575384067\" rel=\"nofollow\">https://twitter.com/rileytomasek/status/1603854647575384067</a>"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Factual AI Q&A \u2013 Answers based on Huberman Lab transcripts"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://huberman.rile.yt/"}},"_tags":["story","author_rileyt","story_34030389","show_hn"],"author":"rileyt","children":[34032081,34032659,34033114,34033144,34033171,34033370,34033447,34033624,34033988,34034214,34034533,34034940,34035123,34035915,34036119,34036165,34037157,34051036],"created_at":"2022-12-17T18:05:04Z","created_at_i":1671300304,"num_comments":43,"objectID":"34030389","points":120,"story_id":34030389,"story_text":"This is a quick prototype I built for semantic search and factual question answering using embeddings and GPT-3.<p>It tries to solve the LLM hallucination issue by guiding it only to answer questions from the given context instead of making things up. If you ask something not covered in an episode, it should say that it doesn&#x27;t know rather than providing a plausible, but potentially incorrect response.<p>It uses Whisper to transcribe, text-embedding-ada-002 to embed, Pinecone.io to search, and text-davinci-003 to generate the answer.<p>More examples and explanations here: <a href=\"https:&#x2F;&#x2F;twitter.com&#x2F;rileytomasek&#x2F;status&#x2F;1603854647575384067\" rel=\"nofollow\">https:&#x2F;&#x2F;twitter.com&#x2F;rileytomasek&#x2F;status&#x2F;1603854647575384067</a>","title":"Show HN: Factual AI Q&A \u2013 Answers based on Huberman Lab transcripts","updated_at":"2025-04-04T12:53:07Z","url":"https://huberman.rile.yt/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Palmik"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"I could not find a blog post as of now, so here's the announcement email:<p>Hello!\nWe are excited to announce the release of gpt-3.5-turbo-instruct, our latest model that serves as a replacement for several deprecated models, such as <em>text-davinci-003</em>.<p>Key Features:\nGpt-3.5-turbo-instruct is an InstructGPT 3.5 class model. It\u2019s trained similarly to previous Instruct models such as the text-davinci series while maintaining the same speed as our turbo models.<p>Pricing:\nWe are committed to making cutting-edge technology accessible, so we have priced gpt-3.5-turbo-instruct in line with our other turbo GPT-3.5 models with 4K context.<p>Thank you for being a part of our journey and for building with OpenAI. Your support enables us to continually advance and bring you the best in AI technology.<p>Best regards,\nThe OpenAI team"},"title":{"matchLevel":"none","matchedWords":[],"value":"OpenAI GPT-3.5-turbo-instruct released"}},"_tags":["story","author_Palmik","story_37558911","ask_hn"],"author":"Palmik","children":[37558940,37560247,37561849,37561938,37563341,37563351,37563834,37563998,37566724,37585712,37606849,37616949],"created_at":"2023-09-18T17:09:30Z","created_at_i":1695056970,"num_comments":21,"objectID":"37558911","points":25,"story_id":37558911,"story_text":"I could not find a blog post as of now, so here&#x27;s the announcement email:<p>Hello!\nWe are excited to announce the release of gpt-3.5-turbo-instruct, our latest model that serves as a replacement for several deprecated models, such as text-davinci-003.<p>Key Features:\nGpt-3.5-turbo-instruct is an InstructGPT 3.5 class model. It\u2019s trained similarly to previous Instruct models such as the text-davinci series while maintaining the same speed as our turbo models.<p>Pricing:\nWe are committed to making cutting-edge technology accessible, so we have priced gpt-3.5-turbo-instruct in line with our other turbo GPT-3.5 models with 4K context.<p>Thank you for being a part of our journey and for building with OpenAI. Your support enables us to continually advance and bring you the best in AI technology.<p>Best regards,\nThe OpenAI team","title":"OpenAI GPT-3.5-turbo-instruct released","updated_at":"2025-11-10T20:42:11Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"djbozjr"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"We\u2019re DJ &amp; Josh, co-founders of MichaelAI(<a href=\"https://michaelai.io\" rel=\"nofollow\">https://michaelai.io</a>) - we\u2019re excited to share our new artificially intelligent investment analyst, Michael.<p>Josh\u2019s six-year stint at a hedge fund exposed him to the challenges of investment analysis. He had access to abundant resources, including data collected by junior analysts, detailed reports, refined data from third-party providers, and access to industry thought leaders. However, the process demands significant time commitment, continuous engagement with other parties, nuanced data, and the potential to miss out on crucial information. Recognizing these challenges, we believe Michael is at the beginning stages of addressing them. Our goal is to enhance the productivity and proficiency of investment analysts.<p>You can interact with Michael via natural language(i.e., chat) or search features. From our perspective, the former is great for exploring unfamiliar companies, or broader topics, whereas the latter is useful for seeking specific information or data points. We currently draw data from resources like SEC filings, earnings call transcripts, conference transcripts, and standardized financial metrics. For alleviating uncertainty with hallucinations, both search and natural language contain a feature for highlighting which portion(s) of the original document(s) Michael\u2019s responses originated from. Although currently limited to ~30 companies, new firms are added daily.<p>Presently, our data pipeline incorporates leading language models, such as BERT, RoBERTa, and OpenAI's <em>text-davinci-003</em>, for formulating Michael's responses. We believe that our unique edge lies in refining and customizing our own internal Large Language Model (LLM) specifically to financial metrics and information. This approach, we believe, will big a big factor in Michael\u2019s growth for lowered latencies, increased accuracy responses, multi-step financial metric calculations, and reductions in hallucinations.<p>We firmly believe that Michael is positioned to upend the investment industry. We are excited to see the impact it has on investors. Although we have sign-up constraints, we have an access code that grants immediate access post-sign-up (Access Code: MICHAELS_FRIENDS_1942). However, please note that usage is capped at ten queries per day for free users. For unlimited queries and to gain early access to emerging features, we offer a premium subscription for $9.99 per month. We appreciate your support and look forward to your feedback.*"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: MichaelAI \u2013 An artificially intelligent investment analyst"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://michaelai.io"}},"_tags":["story","author_djbozjr","story_36213949","show_hn"],"author":"djbozjr","children":[36227645],"created_at":"2023-06-06T15:06:06Z","created_at_i":1686063966,"num_comments":1,"objectID":"36213949","points":12,"story_id":36213949,"story_text":"We\u2019re DJ &amp; Josh, co-founders of MichaelAI(<a href=\"https:&#x2F;&#x2F;michaelai.io\" rel=\"nofollow\">https:&#x2F;&#x2F;michaelai.io</a>) - we\u2019re excited to share our new artificially intelligent investment analyst, Michael.<p>Josh\u2019s six-year stint at a hedge fund exposed him to the challenges of investment analysis. He had access to abundant resources, including data collected by junior analysts, detailed reports, refined data from third-party providers, and access to industry thought leaders. However, the process demands significant time commitment, continuous engagement with other parties, nuanced data, and the potential to miss out on crucial information. Recognizing these challenges, we believe Michael is at the beginning stages of addressing them. Our goal is to enhance the productivity and proficiency of investment analysts.<p>You can interact with Michael via natural language(i.e., chat) or search features. From our perspective, the former is great for exploring unfamiliar companies, or broader topics, whereas the latter is useful for seeking specific information or data points. We currently draw data from resources like SEC filings, earnings call transcripts, conference transcripts, and standardized financial metrics. For alleviating uncertainty with hallucinations, both search and natural language contain a feature for highlighting which portion(s) of the original document(s) Michael\u2019s responses originated from. Although currently limited to ~30 companies, new firms are added daily.<p>Presently, our data pipeline incorporates leading language models, such as BERT, RoBERTa, and OpenAI&#x27;s text-davinci-003, for formulating Michael&#x27;s responses. We believe that our unique edge lies in refining and customizing our own internal Large Language Model (LLM) specifically to financial metrics and information. This approach, we believe, will big a big factor in Michael\u2019s growth for lowered latencies, increased accuracy responses, multi-step financial metric calculations, and reductions in hallucinations.<p>We firmly believe that Michael is positioned to upend the investment industry. We are excited to see the impact it has on investors. Although we have sign-up constraints, we have an access code that grants immediate access post-sign-up (Access Code: MICHAELS_FRIENDS_1942). However, please note that usage is capped at ten queries per day for free users. For unlimited queries and to gain early access to emerging features, we offer a premium subscription for $9.99 per month. We appreciate your support and look forward to your feedback.*","title":"Show HN: MichaelAI \u2013 An artificially intelligent investment analyst","updated_at":"2024-09-20T14:10:45Z","url":"https://michaelai.io"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"bcjordan"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Hiya HN! This weekend I built a webapp tool using the latest GPT-3 model from a few weeks ago (<em>text-davinci-003</em>) to help me come up with personalized and hyper-specific gift ideas. I was surprised to find that the model was actually much better at coming up with gift ideas than I was, given a detailed description of the giftee.<p>The prompt engineering and result presentation was fun to build out, and I added some additional utilities to the results \u2014 quick links to searches on Amazon and an &quot;Open All in New Tabs&quot; button to make it easier to browse potential gifts. I also have some additional features in the works: explanations of why each gift would be good for the giftee, and the ability for the AI to ask additional questions to refine the recommendations (making it a more iterative / chat experience).<p>I have iOS and Android apps in progress (bless React Native) and hope to improve the make the search smarter down the road by e.g. incorporating product/review data using the new OpenAI embeddings API (maybe in combination with a search-focused database). I'm excited to see what HN gang thinks of my tool and if any last minute gifters like me find it to be a helpful resource for finding holiday gifts this year. Let me know what you think and if there are any other features you'd like to see!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Gift Genie AI \u2013 GPT-3 powered free-text personalized gift ideas"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.giftgenie.ai/"}},"_tags":["story","author_bcjordan","story_34052508","show_hn"],"author":"bcjordan","created_at":"2022-12-19T14:38:05Z","created_at_i":1671460685,"num_comments":0,"objectID":"34052508","points":6,"story_id":34052508,"story_text":"Hiya HN! This weekend I built a webapp tool using the latest GPT-3 model from a few weeks ago (text-davinci-003) to help me come up with personalized and hyper-specific gift ideas. I was surprised to find that the model was actually much better at coming up with gift ideas than I was, given a detailed description of the giftee.<p>The prompt engineering and result presentation was fun to build out, and I added some additional utilities to the results \u2014 quick links to searches on Amazon and an &quot;Open All in New Tabs&quot; button to make it easier to browse potential gifts. I also have some additional features in the works: explanations of why each gift would be good for the giftee, and the ability for the AI to ask additional questions to refine the recommendations (making it a more iterative &#x2F; chat experience).<p>I have iOS and Android apps in progress (bless React Native) and hope to improve the make the search smarter down the road by e.g. incorporating product&#x2F;review data using the new OpenAI embeddings API (maybe in combination with a search-focused database). I&#x27;m excited to see what HN gang thinks of my tool and if any last minute gifters like me find it to be a helpful resource for finding holiday gifts this year. Let me know what you think and if there are any other features you&#x27;d like to see!","title":"Show HN: Gift Genie AI \u2013 GPT-3 powered free-text personalized gift ideas","updated_at":"2024-09-20T12:51:50Z","url":"https://www.giftgenie.ai/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jordanf"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Hi HN,<p>I'm pleased to share Promptspot, an open-source (Apache License 2.0) project that helps automate testing of large language model (LLM) prompts against an array of input data.<p>Modern LLMs offer an enormous amount of leverage if you &quot;teach the bot to fish&quot; \u2014 i.e. simply prompt it with both a &quot;system prompt&quot; (which typically doesn't change often) and a dynamic input, which is often application state, search results, recent activity, user profile data, etc.<p>Existing playgrounds and prompt management systems often lack the rigor and flexibility required for this dynamic approach \u2014 and as more teams adopt this pattern, I hope Promptspot can become a useful tool for testing, monitoring, and centralizing this data.<p>Promptspot currently supports <em>text-davinci-003</em> from OpenAI, but I hope to add support for more models soon. Contributions welcome!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: I built an open-source \u201cunit testing\u201d suite for prompts \u2b82 inputs"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/jordanful/promptspot"}},"_tags":["story","author_jordanf","story_36519857","show_hn"],"author":"jordanf","created_at":"2023-06-29T13:07:12Z","created_at_i":1688044032,"num_comments":0,"objectID":"36519857","points":5,"story_id":36519857,"story_text":"Hi HN,<p>I&#x27;m pleased to share Promptspot, an open-source (Apache License 2.0) project that helps automate testing of large language model (LLM) prompts against an array of input data.<p>Modern LLMs offer an enormous amount of leverage if you &quot;teach the bot to fish&quot; \u2014 i.e. simply prompt it with both a &quot;system prompt&quot; (which typically doesn&#x27;t change often) and a dynamic input, which is often application state, search results, recent activity, user profile data, etc.<p>Existing playgrounds and prompt management systems often lack the rigor and flexibility required for this dynamic approach \u2014 and as more teams adopt this pattern, I hope Promptspot can become a useful tool for testing, monitoring, and centralizing this data.<p>Promptspot currently supports text-davinci-003 from OpenAI, but I hope to add support for more models soon. Contributions welcome!","title":"Show HN: I built an open-source \u201cunit testing\u201d suite for prompts \u2b82 inputs","updated_at":"2024-09-20T14:23:28Z","url":"https://github.com/jordanful/promptspot"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tuanacelik"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Hey,<p>I want to show you my demo AI-powered app that helps summarise someone's recent tweets to help you decide whether to follow them. It'll give you an idea of the tone and languages they use too,<p>If you don't want to spend time scrolling through Tweets, let AI tell you what someone's been Tweeting about recently!<p>Built with Haystack: <a href=\"https://github.com/deepset-ai/haystack\">https://github.com/deepset-ai/haystack</a><p>It uses <em>text-davinci-003</em> from OpenAI (for now) with a custom Haystack PromptTemplate.<p>You can check out the source code here too: <a href=\"https://github.com/TuanaCelik/should-i-follow\">https://github.com/TuanaCelik/should-i-follow</a>"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Get the gist of anyone's Twitter feed"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://huggingface.co/spaces/Tuana/should-i-follow"}},"_tags":["story","author_tuanacelik","story_34924387","show_hn"],"author":"tuanacelik","created_at":"2023-02-24T13:42:27Z","created_at_i":1677246147,"num_comments":0,"objectID":"34924387","points":5,"story_id":34924387,"story_text":"Hey,<p>I want to show you my demo AI-powered app that helps summarise someone&#x27;s recent tweets to help you decide whether to follow them. It&#x27;ll give you an idea of the tone and languages they use too,<p>If you don&#x27;t want to spend time scrolling through Tweets, let AI tell you what someone&#x27;s been Tweeting about recently!<p>Built with Haystack: <a href=\"https:&#x2F;&#x2F;github.com&#x2F;deepset-ai&#x2F;haystack\">https:&#x2F;&#x2F;github.com&#x2F;deepset-ai&#x2F;haystack</a><p>It uses text-davinci-003 from OpenAI (for now) with a custom Haystack PromptTemplate.<p>You can check out the source code here too: <a href=\"https:&#x2F;&#x2F;github.com&#x2F;TuanaCelik&#x2F;should-i-follow\">https:&#x2F;&#x2F;github.com&#x2F;TuanaCelik&#x2F;should-i-follow</a>","title":"Show HN: Get the gist of anyone's Twitter feed","updated_at":"2024-09-20T13:24:12Z","url":"https://huggingface.co/spaces/Tuana/should-i-follow"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"wskish"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"The new OpenAI &quot;gpt-3.5-turbo&quot; model is cheap but how does it perform?  Check out its summarization style compared to the 10x more expensive &quot;<em>text-davinci-003</em>&quot; model on summaries of Hacker News stories."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Compare new OpenAI model vs. old on HN Summaries"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://news.jiggy.ai"}},"_tags":["story","author_wskish","story_35001967","show_hn"],"author":"wskish","children":[35002030,35002728,35012936],"created_at":"2023-03-02T20:57:22Z","created_at_i":1677790642,"num_comments":3,"objectID":"35001967","points":4,"story_id":35001967,"story_text":"The new OpenAI &quot;gpt-3.5-turbo&quot; model is cheap but how does it perform?  Check out its summarization style compared to the 10x more expensive &quot;text-davinci-003&quot; model on summaries of Hacker News stories.","title":"Show HN: Compare new OpenAI model vs. old on HN Summaries","updated_at":"2024-09-20T13:21:18Z","url":"https://news.jiggy.ai"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"diarmuid_glynn"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"We've combined OpenAI (<em>text-davinci-003</em>) and our spreadsheet technology to provide a &quot;conversational&quot; interface for the creation and refinement of a simple spreadsheet (including formulas). You can try it out immediately, no login / registration required.<p>There are some limitations:<p>* Our spreadsheet tech doesn't yet support all Excel formulas (we're working on it!)<p>* <em>text-davinci-003</em> doesn't always generate satisfactory responses to prompts (it's still pretty amazing how well it performs, IMHO)<p>Note that you can manually adjust the spreadsheet and ask follow-up questions.<p>I'm happy to answer whatever questions you might have about EqualTo Chat!<p>-Diarmuid"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: EqualTo Chat, a ChatGPT-like application for generating spreadsheets"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.equalto.com/chat/"}},"_tags":["story","author_diarmuid_glynn","story_35068474","show_hn"],"author":"diarmuid_glynn","created_at":"2023-03-08T13:01:49Z","created_at_i":1678280509,"num_comments":0,"objectID":"35068474","points":3,"story_id":35068474,"story_text":"We&#x27;ve combined OpenAI (text-davinci-003) and our spreadsheet technology to provide a &quot;conversational&quot; interface for the creation and refinement of a simple spreadsheet (including formulas). You can try it out immediately, no login &#x2F; registration required.<p>There are some limitations:<p>* Our spreadsheet tech doesn&#x27;t yet support all Excel formulas (we&#x27;re working on it!)<p>* text-davinci-003 doesn&#x27;t always generate satisfactory responses to prompts (it&#x27;s still pretty amazing how well it performs, IMHO)<p>Note that you can manually adjust the spreadsheet and ask follow-up questions.<p>I&#x27;m happy to answer whatever questions you might have about EqualTo Chat!<p>-Diarmuid","title":"Show HN: EqualTo Chat, a ChatGPT-like application for generating spreadsheets","updated_at":"2024-09-20T13:28:44Z","url":"https://www.equalto.com/chat/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nmrequired"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"CREATE DATABASE character_world; USE character_world; CREATE TABLE Personalities(char_name, values, interests, strengths, descr); CREATE TABLE Preferences(char_name, likes, dislikes, activities, topics, descr);CREATE TABLE Conversations(char_name, topics, length, tone, people, descr);CREATE TABLE Environment(char_name, weather, time_of_day, location, descr);CREATE TABLE Events(char_name, type, date, people, descr);CREATE TABLE Locations(char_name, type, size, amenities, descr);CREATE TABLE Actions(char_name, type, target, result, descr);CREATE TABLE Reactions(char_name, type, intensity, duration, descr); CREATE TABLE Context(char_name, bg_noise, lighting, temp, descr);CREATE TABLE Relationships(char_name, type, duration, strength, descr);CREATE TABLE Memories(char_name, date, people, emotions, clarity, vividness, accuracy, descr);CREATE TABLE Goals(char_name, importance, timeline, progress, difficulty, rewards, risks, descr);CREATE TABLE Beliefs(char_name, intensity, source, evidence, certainty, relevance, implications, descr);CREATE TABLE Moods(char_name, mood, intensity, timestamp, cause, descr);CREATE TABLE Character_Traits(char_name, phys_traits, pers_traits, skills, descr);CREATE TABLE Scene_Details(char_name, setting, objects, people, descr);CREATE TABLE Dialogue(char_name, speaker, words, tone, descr);CREATE TABLE Emotions(char_name, emotion, intensity, duration, descr);CREATE TABLE Thoughts(char_name, thought, clarity, relevance, descr);<p>Extract character information from the story which can be stored in this database and write the output as a mysql query.<p>------------------------End Prompt--------<p>Take some text and place it above this prompt (I use <em>text-davinci-003</em> with temp 0)<p>Example from The Gift of the Magi<p>INSERT INTO Personalities (char_name, values, interests, strengths, descr) VALUES ('Della', 'generosity, love', 'buying gifts for Jim', 'resourcefulness, determination', 'The protagonist of the story');<p>INSERT INTO Preferences (char_name, likes, dislikes, activities, topics, descr) VALUES ('Della', 'Jim, her hair, giving gifts', 'parsimony, not having enough money', 'shopping, styling her hair', 'Christmas, presents', 'Della''s preferences');<p>INSERT INTO Conversations (char_name, topics, length, tone, people, descr) VALUES ('Della', 'Christmas, presents, Jim', 'short', 'excited, anxious', 'Jim', 'Della''s conversations with Jim');<p>INSERT INTO Environment (char_name, weather, time_of_day, location, descr) VALUES ('Della', 'cold', 'evening', 'furnished flat', 'Della''s environment');<p>INSERT INTO Events (char_name, type, date, people, descr) VALUES ('Della', 'Christmas', 'December 25th', 'Della and Jim', 'Della''s Christmas event');<p>INSERT INTO Locations (char_name, type, size, amenities, descr) VALUES ('Della', 'flat', 'small', 'letter-box, electric button', 'Della''s flat');<p>INSERT INTO Actions (char_name, type, target, result, descr) VALUES ('Della', 'buying', 'gift for Jim', 'success', 'Della''s action of buying a gift for Jim');<p>INSERT INTO Reactions (char_name, type, intensity, duration, descr) VALUES ('Della', 'crying', 'strong', 'short', 'Della''s reaction to not having enough money');<p>I just started playing with this earlier today and all I can say is just grab some arbitrary text and try it out :)<p>*some formatting edits\n*formatting is still bad and I feel bad about leaving it.."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Prompt Engineering to extract data from a story and into a database"}},"_tags":["story","author_nmrequired","story_34830197","show_hn"],"author":"nmrequired","children":[34830350],"created_at":"2023-02-17T03:06:58Z","created_at_i":1676603218,"num_comments":2,"objectID":"34830197","points":2,"story_id":34830197,"story_text":"CREATE DATABASE character_world; USE character_world; CREATE TABLE Personalities(char_name, values, interests, strengths, descr); CREATE TABLE Preferences(char_name, likes, dislikes, activities, topics, descr);CREATE TABLE Conversations(char_name, topics, length, tone, people, descr);CREATE TABLE Environment(char_name, weather, time_of_day, location, descr);CREATE TABLE Events(char_name, type, date, people, descr);CREATE TABLE Locations(char_name, type, size, amenities, descr);CREATE TABLE Actions(char_name, type, target, result, descr);CREATE TABLE Reactions(char_name, type, intensity, duration, descr); CREATE TABLE Context(char_name, bg_noise, lighting, temp, descr);CREATE TABLE Relationships(char_name, type, duration, strength, descr);CREATE TABLE Memories(char_name, date, people, emotions, clarity, vividness, accuracy, descr);CREATE TABLE Goals(char_name, importance, timeline, progress, difficulty, rewards, risks, descr);CREATE TABLE Beliefs(char_name, intensity, source, evidence, certainty, relevance, implications, descr);CREATE TABLE Moods(char_name, mood, intensity, timestamp, cause, descr);CREATE TABLE Character_Traits(char_name, phys_traits, pers_traits, skills, descr);CREATE TABLE Scene_Details(char_name, setting, objects, people, descr);CREATE TABLE Dialogue(char_name, speaker, words, tone, descr);CREATE TABLE Emotions(char_name, emotion, intensity, duration, descr);CREATE TABLE Thoughts(char_name, thought, clarity, relevance, descr);<p>Extract character information from the story which can be stored in this database and write the output as a mysql query.<p>------------------------End Prompt--------<p>Take some text and place it above this prompt (I use text-davinci-003 with temp 0)<p>Example from The Gift of the Magi<p>INSERT INTO Personalities (char_name, values, interests, strengths, descr) VALUES (&#x27;Della&#x27;, &#x27;generosity, love&#x27;, &#x27;buying gifts for Jim&#x27;, &#x27;resourcefulness, determination&#x27;, &#x27;The protagonist of the story&#x27;);<p>INSERT INTO Preferences (char_name, likes, dislikes, activities, topics, descr) VALUES (&#x27;Della&#x27;, &#x27;Jim, her hair, giving gifts&#x27;, &#x27;parsimony, not having enough money&#x27;, &#x27;shopping, styling her hair&#x27;, &#x27;Christmas, presents&#x27;, &#x27;Della&#x27;&#x27;s preferences&#x27;);<p>INSERT INTO Conversations (char_name, topics, length, tone, people, descr) VALUES (&#x27;Della&#x27;, &#x27;Christmas, presents, Jim&#x27;, &#x27;short&#x27;, &#x27;excited, anxious&#x27;, &#x27;Jim&#x27;, &#x27;Della&#x27;&#x27;s conversations with Jim&#x27;);<p>INSERT INTO Environment (char_name, weather, time_of_day, location, descr) VALUES (&#x27;Della&#x27;, &#x27;cold&#x27;, &#x27;evening&#x27;, &#x27;furnished flat&#x27;, &#x27;Della&#x27;&#x27;s environment&#x27;);<p>INSERT INTO Events (char_name, type, date, people, descr) VALUES (&#x27;Della&#x27;, &#x27;Christmas&#x27;, &#x27;December 25th&#x27;, &#x27;Della and Jim&#x27;, &#x27;Della&#x27;&#x27;s Christmas event&#x27;);<p>INSERT INTO Locations (char_name, type, size, amenities, descr) VALUES (&#x27;Della&#x27;, &#x27;flat&#x27;, &#x27;small&#x27;, &#x27;letter-box, electric button&#x27;, &#x27;Della&#x27;&#x27;s flat&#x27;);<p>INSERT INTO Actions (char_name, type, target, result, descr) VALUES (&#x27;Della&#x27;, &#x27;buying&#x27;, &#x27;gift for Jim&#x27;, &#x27;success&#x27;, &#x27;Della&#x27;&#x27;s action of buying a gift for Jim&#x27;);<p>INSERT INTO Reactions (char_name, type, intensity, duration, descr) VALUES (&#x27;Della&#x27;, &#x27;crying&#x27;, &#x27;strong&#x27;, &#x27;short&#x27;, &#x27;Della&#x27;&#x27;s reaction to not having enough money&#x27;);<p>I just started playing with this earlier today and all I can say is just grab some arbitrary text and try it out :)<p>*some formatting edits\n*formatting is still bad and I feel bad about leaving it..","title":"Show HN: Prompt Engineering to extract data from a story and into a database","updated_at":"2024-09-20T13:22:30Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"keithwhor"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Hey HN! My name is Keith and I'm the founder of Autocode. We provide an online IDE and serverless hosting platform for building webhooks, APIs and bots. We specialize in helping folks build software that connects APIs together -- think Zapier if it were instead a &quot;full code&quot; solution with guardrails to help new developers. Our goal is to help folks do more with APIs by making the web more programmable.<p>We just launched DiscordGPT: it's an open source (Node.js) template that connects Discord `bot_mention` events to OpenAI's `completions` API. What's exciting to me is how fast anybody can patch a few APIs together to have a nearly ChatGPT-quality personalized AI in their own Discord server. Takes only a minute or two and works out-of-the-box. Completions are powered via the `<em>text-davinci-003</em>` model and webhooks are automatically configured by Autocode.<p>I wanted to share it with the HN community for two reasons: (1) it's probably the fastest way to build a GPT-powered bot and it's super hackable, I think folks here would enjoy. (2) it's a good opportunity to show off Autocode! 80% of our users identify as &quot;technical, but not a developer&quot; but we're seeing more uptake from professional engineering teams and I'm interested in getting feedback!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: DiscordGPT. Customizable Discord bot using GPT3 completions"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://discord-gpt.com/"}},"_tags":["story","author_keithwhor","story_34571221","show_hn"],"author":"keithwhor","created_at":"2023-01-29T18:42:02Z","created_at_i":1675017722,"num_comments":0,"objectID":"34571221","points":2,"story_id":34571221,"story_text":"Hey HN! My name is Keith and I&#x27;m the founder of Autocode. We provide an online IDE and serverless hosting platform for building webhooks, APIs and bots. We specialize in helping folks build software that connects APIs together -- think Zapier if it were instead a &quot;full code&quot; solution with guardrails to help new developers. Our goal is to help folks do more with APIs by making the web more programmable.<p>We just launched DiscordGPT: it&#x27;s an open source (Node.js) template that connects Discord `bot_mention` events to OpenAI&#x27;s `completions` API. What&#x27;s exciting to me is how fast anybody can patch a few APIs together to have a nearly ChatGPT-quality personalized AI in their own Discord server. Takes only a minute or two and works out-of-the-box. Completions are powered via the `text-davinci-003` model and webhooks are automatically configured by Autocode.<p>I wanted to share it with the HN community for two reasons: (1) it&#x27;s probably the fastest way to build a GPT-powered bot and it&#x27;s super hackable, I think folks here would enjoy. (2) it&#x27;s a good opportunity to show off Autocode! 80% of our users identify as &quot;technical, but not a developer&quot; but we&#x27;re seeing more uptake from professional engineering teams and I&#x27;m interested in getting feedback!","title":"Show HN: DiscordGPT. Customizable Discord bot using GPT3 completions","updated_at":"2024-09-20T13:15:05Z","url":"https://discord-gpt.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ilaksh"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Hello, so I have a startup https://aidev.codes. Its a bit rough but if people are able to look past the lack of UI polish, I have connected the core technology, and many improvements in the works, such as moving to dialog-based and adding knowledgebase support (which the main part is finished). Currently I have to default to using <em>text-davinci-003</em> instead of code-davinci-002 (Codex) which is the code-specific model. Because OpenAI's API is limited to 10-20 requests per minute for those models.<p>Now with the announcement about the partnership with Microsoft, apparently the code-davinci-002 is moving to &quot;Azure OpenAI&quot; and they tout it as &quot;generally available&quot;. But in fact, it is limited to &quot;managed customers and partners&quot;.  You have to apply here:<p>- https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUOFA5Qk1UWDRBMjg0WFhPMkIzTzhKQ1dWNyQlQCN0PWcu<p>Would anyone be willing to act as an investor or partner to help me get the contacts or funding needed to keep my startup alive? I feel like I am going to just be locked out unless I have some &quot;in&quot; with Microsoft or access to more funds to become an &quot;enterprise&quot; &quot;managed&quot; customer or something.  Also due to lack of funds I have to take small client projects related to the other website in order to stay afloat. I think this is hampering me to an incredible extent compared to less-skilled developers that have funding to hire more engineers and access the models at scale.<p>You can contact me at runvnc@gmail.com<p>My github is runvnc.  I also have another website https://algonfts.art and have been building small startups for more than 12 years."},"title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: How can I become a Microsoft \u201cManaged Customer\u201d?"}},"_tags":["story","author_ilaksh","story_34494480","ask_hn"],"author":"ilaksh","created_at":"2023-01-23T20:25:58Z","created_at_i":1674505558,"num_comments":0,"objectID":"34494480","points":2,"story_id":34494480,"story_text":"Hello, so I have a startup https:&#x2F;&#x2F;aidev.codes. Its a bit rough but if people are able to look past the lack of UI polish, I have connected the core technology, and many improvements in the works, such as moving to dialog-based and adding knowledgebase support (which the main part is finished). Currently I have to default to using text-davinci-003 instead of code-davinci-002 (Codex) which is the code-specific model. Because OpenAI&#x27;s API is limited to 10-20 requests per minute for those models.<p>Now with the announcement about the partnership with Microsoft, apparently the code-davinci-002 is moving to &quot;Azure OpenAI&quot; and they tout it as &quot;generally available&quot;. But in fact, it is limited to &quot;managed customers and partners&quot;.  You have to apply here:<p>- https:&#x2F;&#x2F;customervoice.microsoft.com&#x2F;Pages&#x2F;ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUOFA5Qk1UWDRBMjg0WFhPMkIzTzhKQ1dWNyQlQCN0PWcu<p>Would anyone be willing to act as an investor or partner to help me get the contacts or funding needed to keep my startup alive? I feel like I am going to just be locked out unless I have some &quot;in&quot; with Microsoft or access to more funds to become an &quot;enterprise&quot; &quot;managed&quot; customer or something.  Also due to lack of funds I have to take small client projects related to the other website in order to stay afloat. I think this is hampering me to an incredible extent compared to less-skilled developers that have funding to hire more engineers and access the models at scale.<p>You can contact me at runvnc@gmail.com<p>My github is runvnc.  I also have another website https:&#x2F;&#x2F;algonfts.art and have been building small startups for more than 12 years.","title":"Ask HN: How can I become a Microsoft \u201cManaged Customer\u201d?","updated_at":"2024-09-20T13:08:04Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hexomancer"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"I was toying with <em>text-DaVinci-003</em> model and it seems while it can easily produce poems about specific topics, it can not produce bible/quran verses. Instead it produces actual verses from these books which are quite irrelevant to the topic I asked for.<p>I was wondering if this was hard-coded by OpenAI (99.9% this is the case), or the model just naturally generates text from original books because they are so prevalent on the internet.<p>And if OpenAI did this, why? I don't remember anybody complaining about the models being able to generate these texts."},"title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Is GPT-3 hardcoded not to generate Bible/Quran verses?"}},"_tags":["story","author_hexomancer","story_33786801","ask_hn"],"author":"hexomancer","created_at":"2022-11-29T12:36:27Z","created_at_i":1669725387,"num_comments":0,"objectID":"33786801","points":2,"story_id":33786801,"story_text":"I was toying with text-DaVinci-003 model and it seems while it can easily produce poems about specific topics, it can not produce bible&#x2F;quran verses. Instead it produces actual verses from these books which are quite irrelevant to the topic I asked for.<p>I was wondering if this was hard-coded by OpenAI (99.9% this is the case), or the model just naturally generates text from original books because they are so prevalent on the internet.<p>And if OpenAI did this, why? I don&#x27;t remember anybody complaining about the models being able to generate these texts.","title":"Ask HN: Is GPT-3 hardcoded not to generate Bible/Quran verses?","updated_at":"2024-09-20T12:41:48Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"waelmas"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["text","davinci","003"],"value":"Completely new to working with LLMs.<p>Over the weekend I built a POC API that uses Langchain and calls GPT4 to perform something like a classification task, accepting a list of strings and returning a list of structured JSON objects.\nThe problem is that is takes ~10s for each request even with small-sized input/output, which prevents the whole thing from being a usable MVP. (at least for the usecases I want).<p>Sharing my high-level approach with the above at the end.<p>I tried 3.5-turbo but could not get it to have any match of accuracy nor consistency.<p>The closest to okayish I found was <em>text-davinci-003</em> with a relatively large starting prompt including some examples (weirdly way better than GPT-35).<p>1) Is there a way to achieve reasonable latency ~3s with GPT4? (somehow setting the initial prompt once? removing langchain from the picture?)<p>2) Does langchain given my code-sample below send the whole conversation every time? Since I do not care about history for now, will removing it help?<p>3) Can and should I use GPT4 to generate training data and use it to fine-tune davinci-text-003?<p>4) Should I stick to GPT4 even with the deadly latency and somehow convince test users that speed will drastically change in X months?<p>5) Any other advice in terms of how to move forward?<p>Any guidance is greatly appreciated!<p>-&gt; Code sample:<p>def init_gpt4_chain() -&gt; ConversationChain:<p><pre><code>    &quot;&quot;&quot;\n    This runs on startup of the app.\n    &quot;&quot;&quot;\n\n    init_prompt_str = str(&quot;&quot;&quot;Some instruction prompt goes here explaining the task and the response format...&quot;&quot;&quot;)\n\n    llm = ChatOpenAI(temperature=settings.GPT4_TEMP_V1, model=&quot;gpt-4&quot;)\n\n    memory = ConversationBufferMemory()\n\n    chain = ConversationChain(llm=llm, memory=memory, verbose=False)\n\n    chain.run(init_prompt_str)\n\n    return chain\n\n</code></pre>\n# Inside API endpoint:<p>chain_gpt4.run(input_payload)"},"title":{"matchLevel":"none","matchedWords":[],"value":"GPT based tool, how to deal with speed"}},"_tags":["story","author_waelmas","story_36840058","ask_hn"],"author":"waelmas","children":[36840095,36840506],"created_at":"2023-07-23T21:31:57Z","created_at_i":1690147917,"num_comments":1,"objectID":"36840058","points":1,"story_id":36840058,"story_text":"Completely new to working with LLMs.<p>Over the weekend I built a POC API that uses Langchain and calls GPT4 to perform something like a classification task, accepting a list of strings and returning a list of structured JSON objects.\nThe problem is that is takes ~10s for each request even with small-sized input&#x2F;output, which prevents the whole thing from being a usable MVP. (at least for the usecases I want).<p>Sharing my high-level approach with the above at the end.<p>I tried 3.5-turbo but could not get it to have any match of accuracy nor consistency.<p>The closest to okayish I found was text-davinci-003 with a relatively large starting prompt including some examples (weirdly way better than GPT-35).<p>1) Is there a way to achieve reasonable latency ~3s with GPT4? (somehow setting the initial prompt once? removing langchain from the picture?)<p>2) Does langchain given my code-sample below send the whole conversation every time? Since I do not care about history for now, will removing it help?<p>3) Can and should I use GPT4 to generate training data and use it to fine-tune davinci-text-003?<p>4) Should I stick to GPT4 even with the deadly latency and somehow convince test users that speed will drastically change in X months?<p>5) Any other advice in terms of how to move forward?<p>Any guidance is greatly appreciated!<p>-&gt; Code sample:<p>def init_gpt4_chain() -&gt; ConversationChain:<p><pre><code>    &quot;&quot;&quot;\n    This runs on startup of the app.\n    &quot;&quot;&quot;\n\n    init_prompt_str = str(&quot;&quot;&quot;Some instruction prompt goes here explaining the task and the response format...&quot;&quot;&quot;)\n\n    llm = ChatOpenAI(temperature=settings.GPT4_TEMP_V1, model=&quot;gpt-4&quot;)\n\n    memory = ConversationBufferMemory()\n\n    chain = ConversationChain(llm=llm, memory=memory, verbose=False)\n\n    chain.run(init_prompt_str)\n\n    return chain\n\n</code></pre>\n# Inside API endpoint:<p>chain_gpt4.run(input_payload)","title":"GPT based tool, how to deal with speed","updated_at":"2024-09-20T14:37:24Z"}],"hitsPerPage":20,"nbHits":369,"nbPages":19,"page":0,"params":"query=text-davinci-003&advancedSyntax=true&analyticsTags=backend","processingTimeMS":17,"processingTimingsMS":{"_request":{"roundTrip":24},"afterFetch":{"format":{"highlighting":1,"total":1}},"fetch":{"query":10,"scanning":5,"total":16},"total":17},"query":"text-davinci-003","serverTimeMS":19}
