https://github.com/zylon-ai/private-gpt","title":null,"type":"comment","url":null},{"author":"bsima","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:14:43.000Z","created_at_i":1712225683,"id":39928494,"options":[],"parent_id":39925991,"points":null,"story_id":39925316,"text":"I just DM'ed you.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T02:40:30.000Z","created_at_i":1712198430,"id":39925991,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I\u2019m literally working on it right now, dm on X if you wanna pair or something","title":null,"type":"comment","url":null},{"author":"brotchie","children":[{"author":"jdthedisciple","children":[],"created_at":"2024-04-04T05:04:46.000Z","created_at_i":1712207086,"id":39926735,"options":[],"parent_id":39926174,"points":null,"story_id":39925316,"text":"Interesting, been waiting for some free time to do this myself.
Mind sharing how you set up your RAG pipeline and which (presumabely FOSS) components you incorporated?","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T03:12:05.000Z","created_at_i":1712200325,"id":39926174,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Played around with fine tuning, but ended up just experimenting with RAG.
One thing I haven\u2019t worked out yet is the agent reliably understanding if it should do a \u201cpoint retrieval query\u201d or an \u201caggregation query.\u201d
Point query: embed and do vector lookup with some max N and distance threshold. For example:\n\u201cWho prepared my 2023 taxes?\u201d
Aggregation query: select a larger collection of documents (1k+) that possibly don\u2019t fit in the context window and reason over the collection. \u201cSummarize all of the correspondence I\u2019ve had with tax preparation agencies over the past 10 years\u201d
The latter may be solved with just a larger max N and larger context window.
Almost like it\u2019s a search lookup vs. a map reduce.","title":null,"type":"comment","url":null},{"author":"dleslie","children":[{"author":"TuringNYC","children":[],"created_at":"2024-04-04T05:17:20.000Z","created_at_i":1712207840,"id":39926814,"options":[],"parent_id":39926214,"points":null,"story_id":39925316,"text":"This is basically what all the headshot generator apps do. It is pretty simple to achieve if you can spin up GPU instances (https://huggingface.co/docs/diffusers/v0.13.0/en/training/te....)
However, I find it challenging to achieve on Macbooks, despite all the neural core horsepower I have. If anyone has achieved this with non-NVIDIA setups i'd love to hear!","title":null,"type":"comment","url":null},{"author":"em-bee","children":[],"created_at":"2024-04-04T05:43:40.000Z","created_at_i":1712209420,"id":39926945,"options":[],"parent_id":39926214,"points":null,"story_id":39925316,"text":"interesting idea.
completely unrelated i just read a scifi story where a technology was developed that could revive dead bodies for a short while in order to pose for family photos that they hadn't created before the person passed away.
https://clarkesworldmagazine.com/liu_03_23/
obviously going way overboard for something AI can do today, but probably the author wrote the story before that.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T03:21:03.000Z","created_at_i":1712200863,"id":39926214,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I want this for my photos.
I'm not a good photographer, but I have taken tens of thousands of photos of my family. I would love to provide a prompt for a specific day and persons and have it create a photo that I never was able to take. I don't mind that it's not "real" because I find photography to be philosophically unreal as it is. I want it to look good, and inspire my mind to recreate the day however it can imagine.
And I want to do it locally, without giving away my family's data and identity.","title":null,"type":"comment","url":null},{"author":"Zambyte","children":[{"author":"bshaughn","children":[],"created_at":"2024-04-04T05:33:02.000Z","created_at_i":1712208782,"id":39926889,"options":[],"parent_id":39926236,"points":null,"story_id":39925316,"text":"Not exactly what you're looking for but I a few months ago I spent a day building a llama-index pipeline against my markdown notes with a really privative note crawling implementation, and had surprisingly good results for question answering.
I don't use an org-roam note system but I've been working on a similar and highly opinionated note system that I'm always making tools for. And I'm always interested in seeing people's ideal note systems.
my crude WIP Obsidian / Markdown note RAG tool: https://github.com/bs7280/markdown-embeddings-search","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T03:26:56.000Z","created_at_i":1712201216,"id":39926236,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I have a large org-roam note system. I would like to create a pipeline where I can ask natural language questions, and it will build SQLite quries to efficiently crawl through the database and find what I want. I haven't gotten around to it though.","title":null,"type":"comment","url":null},{"author":"dumbmrblah","children":[{"author":"yyytolys","children":[],"created_at":"2024-04-05T03:42:02.000Z","created_at_i":1712288522,"id":39938475,"options":[],"parent_id":39926342,"points":null,"story_id":39925316,"text":"This idea is also what I am thinking about. At present, I am also trying to do applied research in this aspect. If it can achieve substantial help, it will be very meaningful.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T03:47:35.000Z","created_at_i":1712202455,"id":39926342,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I\u2019ve tried several different systems, nothing really stands out.
That being said, I\u2019m trying to document as much as my life in anticipation of such programs existing in the near future. I\u2019m not going overboard, but for example, I wouldn\u2019t really keep a personal diary, but now I try to jot down something every day, write down my thought processes on things, what actions were done and why.
I\u2019m looking forward to a day where I have an AI assistant (locally hosted and under my control of course) who can help me with decision-making based on my previous actions. Would be neat to compare/contrast how I do things now, compared to the future me.","title":null,"type":"comment","url":null},{"author":"ein0p","children":[],"created_at":"2024-04-04T03:55:38.000Z","created_at_i":1712202938,"id":39926369,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Somewhat related, for those of us who don\u2019t take extensive notes: are there nicely packaged plugins for RAG in email, especially for eg Outlook or Apple Mail?","title":null,"type":"comment","url":null},{"author":"KMnO4","children":[{"author":"kid64","children":[{"author":"mmahemoff","children":[{"author":"aerhardt","children":[],"created_at":"2024-04-04T09:50:59.000Z","created_at_i":1712224259,"id":39928354,"options":[],"parent_id":39927076,"points":null,"story_id":39925316,"text":"I don\u2019t know about the legality of it, but as a comical skit it\u2019d be hilarious: a patient gets in deep shit with their doctor for violating patient-doctor confidentiality. Sounds straight out of Curb Your Enthusiasm!
Edit: It seems it's straight out of Curb, because it is!\nhttps://youtu.be/YH55dFlF_Rg?si=kOLC5rGq5fi8tke2","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T06:10:00.000Z","created_at_i":1712211000,"id":39927076,"options":[],"parent_id":39926419,"points":null,"story_id":39925316,"text":"Might still be a violation if they're the patient? Unless therapist and their employer's consent is given and ofc dependent on the relevant jurisdictions (IANAL).","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:05:06.000Z","created_at_i":1712203506,"id":39926419,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"I hope you're the patient in this scenario, otherwise this is an egregious HIPAA violation.","title":null,"type":"comment","url":null},{"author":"rhcom2","children":[],"created_at":"2024-04-04T04:15:55.000Z","created_at_i":1712204155,"id":39926464,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"I'm sincerely happy you're finding value in this and it's a very impressive workflow. The idea of sending my therapy sessions to OpenAI sounds terrifying though.","title":null,"type":"comment","url":null},{"author":"fullautomation","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:32:44.000Z","created_at_i":1712226764,"id":39928584,"options":[],"parent_id":39926521,"points":null,"story_id":39925316,"text":"This is where having our own LLMs and stacks running locally will save and empower us IMHO","title":null,"type":"comment","url":null},{"author":"tqwhite","children":[],"created_at":"2024-04-04T14:08:19.000Z","created_at_i":1712239699,"id":39930601,"options":[],"parent_id":39926521,"points":null,"story_id":39925316,"text":"OpenAI's privacy claims are fine. I wouldn't worry about this any more than I worry about my email provider.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:27:33.000Z","created_at_i":1712204853,"id":39926521,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"This is really interesting, are your comfortable with OpenAI having your personal details in this case?","title":null,"type":"comment","url":null},{"author":"psychopomp","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:02:28.000Z","created_at_i":1712224948,"id":39928426,"options":[],"parent_id":39926532,"points":null,"story_id":39925316,"text":"I've created an AI Meeting Bot API to do just that [1].
At the moment it runs on AWS, and we're thinking of open-sourcing so you could also run it locally to maintain 100% privacy of such conversations.
You'd get speaker diarization, names on top of the recording [2].
[1] https://aimeetingbot.com\n[2] https://spoke-1.gitbook.io/ai-meeting-bot
Happy to get in touch and have you run it","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:29:52.000Z","created_at_i":1712204992,"id":39926532,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"I\u2019d love to replicate your workflow. Any luck with speaker diarization using whisper? I\u2019ve tried WhisperX several but it didn\u2019t work.","title":null,"type":"comment","url":null},{"author":"iamcreasy","children":[],"created_at":"2024-04-04T04:35:27.000Z","created_at_i":1712205327,"id":39926565,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"Did you discover anything interesting by being able to review all the therapy sessions?","title":null,"type":"comment","url":null},{"author":"Erazal","children":[],"created_at":"2024-04-04T10:00:07.000Z","created_at_i":1712224807,"id":39928406,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"wow - really cool.
I'm actually the founder of an AI Meeting Bot company - and we're thinking of open-sourcing so you could run exactly this set-up locally with perfect diarization / recording while also maintaining privacy [1].
I'm currently creating code examples, and just finished the "chat with each session". Would love to know how you implemented it.
[1] https://aimeetingbot.com","title":null,"type":"comment","url":null},{"author":"NicoJuicy","children":[],"created_at":"2024-04-04T13:42:28.000Z","created_at_i":1712238148,"id":39930312,"options":[],"parent_id":39926395,"points":null,"story_id":39925316,"text":"Curious on the code. ( a friend is a psychiatrist and she noticed difficulty with multiple languages and device translations).
This flow could help me improve fluency in her sessions ( eg. she has a hardware translation device (expensive) which has significant issues auto translating ), since it's missing context a lot.
Eg. When grieving is incorrectly translated between dutch-polish, it defeats a bit of the purpose of being fluent in your native language.
Reducing the error rate would help a lot.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:00:00.000Z","created_at_i":1712203200,"id":39926395,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Maybe not exactly what you\u2019re asking, but I started doing talk therapy last year. It\u2019s done virtually and I record the session with OBS. As soon as the recording finishes, the following happens:
- The audio is preprocessed (chunked) and sent to Whisper to generate a transcript
- The transcript is sent to GPT-4 to generate a summary, action items, concepts introduced with additional information
- The next meeting\u2019s date/time is added to my calendar
- A chatbot is created that allows me to chat with each session, including playing the role as the therapist and continuing the conversation (with the entire context of what I actually talked about)
It\u2019s been exceedingly helpful to be able to review all my therapy sessions this way.","title":null,"type":"comment","url":null},{"author":"jondwillis","children":[],"created_at":"2024-04-04T04:18:11.000Z","created_at_i":1712204291,"id":39926473,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I started fine-tuning GPT-3.5 on a decently large corpus of my text messages and emails, and it pretty much generated schizophrenic output. I don\u2019t think I did a very good job of curating the text that ended up fine-tuning, and I want to try again.","title":null,"type":"comment","url":null},{"author":"CSMastermind","children":[{"author":"jondwillis","children":[{"author":"CSMastermind","children":[],"created_at":"2024-04-04T18:18:32.000Z","created_at_i":1712254712,"id":39934046,"options":[],"parent_id":39926506,"points":null,"story_id":39925316,"text":"I was not, that's relatively new, though if I'm not mistaken it might make the process easier.
I mean calling the embeddings API and then having software locally that finds and appends documents to your queries.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:24:48.000Z","created_at_i":1712204688,"id":39926506,"options":[],"parent_id":39926486,"points":null,"story_id":39925316,"text":"Are you referring to the "Assistants" API?","title":null,"type":"comment","url":null},{"author":"jdthedisciple","children":[],"created_at":"2024-04-04T05:09:45.000Z","created_at_i":1712207385,"id":39926774,"options":[],"parent_id":39926486,"points":null,"story_id":39925316,"text":"I have recently briefly looked into Assistants API (if that's what you're referring to) but it seems relatively if I'm not mistaken.","title":null,"type":"comment","url":null},{"author":"Erazal","children":[],"created_at":"2024-04-04T10:07:45.000Z","created_at_i":1712225265,"id":39928459,"options":[],"parent_id":39926486,"points":null,"story_id":39925316,"text":"I'm currently looking to implement RAG locally, using QDrant [1] for instance.
Just playing around for now, but it makes sense to have a runnable example for our users too :) [2].
[1]. https://qdrant.tech/\n[2] https://aimeetingbot.com","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:21:28.000Z","created_at_i":1712204488,"id":39926486,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"It sounds like you want RAG instead of training or even fine tuning a model.
Have you looked into the OpenAI APIs? They make it relatively easy to do assuming you have some limited programming knowledge.","title":null,"type":"comment","url":null},{"author":"Brajeshwar","children":[{"author":"sabaimran","children":[],"created_at":"2024-04-09T12:23:48.000Z","created_at_i":1712665428,"id":39978652,"options":[],"parent_id":39926557,"points":null,"story_id":39925316,"text":"Hi! Founder of Khoj, happy to add more context.
Khoj will allow you to plug in your Obsidian vault or any plaintext files on your machine or Notion workspace. After you share the relevant data, it creates embeddings and uses it for RAG, so you get appropriately contextual responses with your LLM.
This is the best place to start for self-hosting:\nhttps://docs.khoj.dev/get-started/setup","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:34:16.000Z","created_at_i":1712205256,"id":39926557,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Neither mine nor endorsing, and I haven't played besides the initial installation but Khoj has an open source offering for this. Check it out https://khoj.dev","title":null,"type":"comment","url":null},{"author":"cjbprime","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:05:01.000Z","created_at_i":1712225101,"id":39928436,"options":[],"parent_id":39926635,"points":null,"story_id":39925316,"text":"I think you're right","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:47:14.000Z","created_at_i":1712206034,"id":39926635,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Going to wait for longer context local models. Fine tuning/training is lossy compression of your notes into the model weights -- there isn't much value in a vaguely-remembered copy of some of my notes. This is why other comments are pointing you towards Retrieval Augmented Generation instead, where the relevant notes are losslessly added to the prompt.","title":null,"type":"comment","url":null},{"author":"prashantsengar","children":[],"created_at":"2024-04-04T04:49:51.000Z","created_at_i":1712206191,"id":39926651,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Not on my notes, but I have used GPT4All to chat with the documents of dapr. \nI downloaded the md files from the docs GitHub repo and loaded the directory in GPT4All.
It is not "training" a model but works pretty great.","title":null,"type":"comment","url":null},{"author":"pb060","children":[{"author":"woodpanel","children":[{"author":"pb060","children":[],"created_at":"2024-04-09T05:44:02.000Z","created_at_i":1712641442,"id":39976456,"options":[],"parent_id":39951957,"points":null,"story_id":39925316,"text":"Yes I guess that \u201call writings\u201d (sorry I meant all of his writings - not an English native) means everything he wrote and also his speeches, taking into account the times when he changed his mind. Knowing how meticulous he is I would be surprised if this wasn\u2019t the case. Worth considering that he was recently diagnosed a mild form of autism himself.","title":null,"type":"comment","url":null}],"created_at":"2024-04-06T12:20:39.000Z","created_at_i":1712406039,"id":39951957,"options":[],"parent_id":39926678,"points":null,"story_id":39925316,"text":"Very interesting.
Since presumably "all writings" refers to all his writings during his lifetime, I'd hope it can account for those times in his life at which he changend his mind on certain topics?","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T04:55:02.000Z","created_at_i":1712206502,"id":39926678,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Gianluca Nicoletti, an Italian journalist, writer and radio speaker, is training a LLM with all writings as a support for his autistic child for when he won\u2019t be here anymore. The software will speak with his voice.
https://www.lospessore.com/13/07/2023/una-chatbot-per-contin...","title":null,"type":"comment","url":null},{"author":"bredren","children":[{"author":"joenot443","children":[],"created_at":"2024-04-04T12:12:56.000Z","created_at_i":1712232776,"id":39929389,"options":[],"parent_id":39927219,"points":null,"story_id":39925316,"text":"Came here to ask the same question. I've played around with a few implementations, but nothing with results that were close to useful.
Anyone had success RAG-ing a chat history??","title":null,"type":"comment","url":null},{"author":"bshaughn","children":[{"author":"bredren","children":[],"created_at":"2024-04-04T18:04:09.000Z","created_at_i":1712253849,"id":39933884,"options":[],"parent_id":39931721,"points":null,"story_id":39925316,"text":"Yes, I'd gladly trial something like this.
It must run locally / require no network requests. I can run on an M2 w 24GB or M3 with 36GB.
My email is in my profile here.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T15:32:30.000Z","created_at_i":1712244750,"id":39931721,"options":[],"parent_id":39927219,"points":null,"story_id":39925316,"text":"I've implemented a POC on exactly this and am working on something more sophisticated right now. Can I reach out to discuss more?","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T06:37:18.000Z","created_at_i":1712212638,"id":39927219,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Has anyone seen or used something that can train on a complete imessage history?
Presumably, I have more than enough messages from me along with responses from others to chat with a version of myself that bears an incredible likeness to how I speak and think. In some cases, I'd expect to be able to chat with an LLM of a given contact to see how they'd respond to various questions as well.","title":null,"type":"comment","url":null},{"author":"alexmolas","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:13:13.000Z","created_at_i":1712225593,"id":39928486,"options":[],"parent_id":39927514,"points":null,"story_id":39925316,"text":"hadn't seen your repo yet [1] - adding it to my list right now.
Your blog post is really neat on top - thanks for sharing
https://github.com/eugeneyan/obsidian-copilot","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T07:24:25.000Z","created_at_i":1712215465,"id":39927514,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"https://eugeneyan.com/writing/obsidian-copilot/","title":null,"type":"comment","url":null},{"author":"mitjam","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:56:57.000Z","created_at_i":1712228217,"id":39928725,"options":[],"parent_id":39927672,"points":null,"story_id":39925316,"text":"have never found it powerful enough - or I'm just setting too high a bar?
I have a ton of databases on Notion (with all my teams conversation transcripts, meeting to-dos, etc.) and global AI search just isn't there.
I haven't found a way there (but have elsewhere using open source) to create a kick-ass search.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T07:55:55.000Z","created_at_i":1712217355,"id":39927672,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I have heard good things about the Notion AI addon although I haven\u2019t tried it myself.","title":null,"type":"comment","url":null},{"author":"appstorelottery","children":[{"author":"injuly","children":[{"author":"appstorelottery","children":[],"created_at":"2024-04-04T14:48:38.000Z","created_at_i":1712242118,"id":39931123,"options":[],"parent_id":39928081,"points":null,"story_id":39925316,"text":"Yes, completely digital - in markdown format. I use IA writer on the Mac.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T09:05:55.000Z","created_at_i":1712221555,"id":39928081,"options":[],"parent_id":39927817,"points":null,"story_id":39925316,"text":"Is the diary digital?\nI prefer writing on paper, and I'd like to try this. Wonder if there's any decent OCR app that'll help me do it.","title":null,"type":"comment","url":null},{"author":"resource_waste","children":[{"author":"appstorelottery","children":[],"created_at":"2024-04-04T14:53:31.000Z","created_at_i":1712242411,"id":39931188,"options":[],"parent_id":39928137,"points":null,"story_id":39925316,"text":">I also imagine, you arent mentioning the times it was completely incorrect because you used a negative in the sentence.
I haven't yet observed it being completely incorrect - I keep the queries simple without negation.
>This might seem impressive because of the subjectiveness.
It's surprising how it can summarise my relationship with another person, for example - if I ask "who is X?" it will deliver quite a succinct summary of the relationship - using my own words at times.
>I'm not really sure what you are getting out of it.
Mostly it's useful for self-reflection, it's helped me to see challenges I was facing from a more generalised perspective - particularly in my relationships with others. I'm also terribly impressed by the technology - being able to natural-language query and receive a sensible, and often insightful response - it feels like the future to me.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T09:14:28.000Z","created_at_i":1712222068,"id":39928137,"options":[],"parent_id":39927817,"points":null,"story_id":39925316,"text":"This might seem impressive because of the subjectiveness. I also imagine, you arent mentioning the times it was completely incorrect because you used a negative in the sentence.
This is regular embeddings + LLM.
At the end of the day, you are basically just adding a preprompt to a search. Not to mention, the Mistral models are barely useful for logic.
I'm not really sure what you are getting out of it. I'm wondering if you are reading some mostly generic Mistral output with a few words from your pre-prompt/embedding.","title":null,"type":"comment","url":null},{"author":"NKosmatos","children":[{"author":"appstorelottery","children":[],"created_at":"2024-04-04T14:48:09.000Z","created_at_i":1712242089,"id":39931115,"options":[],"parent_id":39929595,"points":null,"story_id":39925316,"text":"Yes, check out advanced settings for the plug-in after it's installed.","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T12:34:04.000Z","created_at_i":1712234044,"id":39929595,"options":[],"parent_id":39927817,"points":null,"story_id":39925316,"text":"Nice one, I'm going to give it a try.
When you say Sbert you mean the GPT4All LocalDocs plugin?","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T08:25:25.000Z","created_at_i":1712219125,"id":39927817,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"I've been disciplined (perhaps obsessive at times) with keeping a daily diary for many years and I was interested in being able to query my diary locally via AI. I found a solution that works surprisingly well using GPT4ALL.
I found GPT4ALL (https://gpt4all.io) to have a nice-enough GUI, and it runs reasonably quickly on my M1 MacBook Air with 8Gb of ram, and it can be setup to be a completely local solution - not sending your data to the Goliaths.
GPT4ALL has an option to access local documents, via the Sbert text embedding model (RAG).
My specific results have been as follows; using the Nous Hermes 2 Mistral DPO and Sbert - I indexed 153 days of my daily writing (most days I write between 2 and 3 thousand words).
Asking a simple question like "what are the challenges faced by the author?" provides remarkable, almost spooky results (which I won't share here) - which in my opinion are spot-on regarding my own challenges over that the period - and Sbert provides references to which documents it used to generate the answer. Options are available to reference an arbitrary number of documents, however the default is 10. Ideally I'd like to have it reference all 153 documents in the query - I'm not sure if it's a ram or a token issue, however increasing the value of documents referenced has resulted in machine lock-ups.
Anyhow - that's my experience - hope it's helpful to someone.","title":null,"type":"comment","url":null},{"author":"bhupesh","children":[{"author":"Erazal","children":[],"created_at":"2024-04-04T10:09:39.000Z","created_at_i":1712225379,"id":39928469,"options":[],"parent_id":39927982,"points":null,"story_id":39925316,"text":"oh damn, I'm using gitbook too actually - for our users [1]. \nNot open-source but will definitely try it out ASAP.
[1] https://spoke-1.gitbook.io/ai-meeting-bot","title":null,"type":"comment","url":null},{"author":"phantomathkg","children":[],"created_at":"2024-04-04T14:22:33.000Z","created_at_i":1712240553,"id":39930777,"options":[],"parent_id":39927982,"points":null,"story_id":39925316,"text":"Do you have a Pro plan?","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T08:48:39.000Z","created_at_i":1712220519,"id":39927982,"options":[],"parent_id":39925316,"points":null,"story_id":39925316,"text":"Folks at gitbook are kind enough to give me a LLM over my notes https://til.bhupesh.me","title":null,"type":"comment","url":null}],"created_at":"2024-04-04T01:00:06.000Z","created_at_i":1712192406,"id":39925316,"options":[],"parent_id":null,"points":138,"story_id":39925316,"text":"Some here comment on notes they've taken for their entire life, often using powerful note-taking systems.
Of course, there's a ton of note-taking systems out there. Org-Mode [1], Obsidian [2], plain .txt, ...
And it's become quite simple to integrate such systems with LLMs.
Whether to add that data to the LLM [3], using LLM formatting, or to visualize and use it as a personal partner. For the latter - there's also a ton of open-source UIs such as Chatbot-ui[4] and Reor[5].
And that's just the tip of the iceberg.
Personally, I haven't been consistent enough through the years in note-taking.
So, I'm really curious to learn more about those of you who were and implemented such pipelines.
I'm sure there's a ton of cool interaction experiences.
[1] https://orgmode.org/\n[2] https://obsidian.md/\n[3] https://ollama.com/\n[4] https://github.com/mckaywrigley/chatbot-ui\n[5] https://github.com/reorproject/reor","title":"Ask HN: Has Anyone Trained a personal LLM using their personal notes?","type":"story","url":null}