Code: qa_web_haystack.py

That took way less time than I thought it would.

Came up with the idea two nights ago, and now I have a POC/MVP (it’s nothing new, but it’s not what chatGPT gives you for free). This isn’t free either technically, serpdev requires an api key, they give you 5000 queries for free, and then $50 for 50k after that (10 cents a query). I imagine this was done in 1 query

But the answers are state of the art. I’m using BERT, but I imagine one could input any model at the end, but the Retriever itself is key, and that is done upstream by whatever search providers haystack partners with. In other words, there doesn’t appear to be any free web retriever that is as succint as this. Else you would likely need to engineer your own DPR from google queries using sellenium, or seeing if google has a direct API.

I just upped my game with Dense Passage Retrieval using web results.

By the end of this weekend, I will have figured out how to augment my own personal QA GPT based chatbot with web results, stay tuned.