I’m building a document QA chatbot using Flowise and Vectara and want to have the chatbot always return a url for the original document so the user can verify the information. What’s the best way to index the original link so it’s returned whenever a search result is returned? Should I put the link as a metadata or add it as part of the document?
Hello, thanks for the question!
Yes, there is a way to add these links via metadata. I’m currently testing this capability in Flowise and will update you once I can confirm that it is working and live in Flowise.
In the meantime, could you let me know how you are ingesting the data?
- Are you using the Vectara API or a Flowise module?
- Does all the content you’re ingesting live online as URLs?
I currently have over 100 links from a website that I wrote a custom script to scrape and clean up. Then for each page I scrapped I produce a JSON file that I then upload via Vectara’s API.
Currently I found that when I attach the url as a metadata to the section, it does get returned, but I couldn’t get the chain in Flowise to use this information in its answer no matter how I prompt it using system prompt. I think even though the meta data are retrieved by flowise, the information isn’t fed to the chain for answering, which is something I’m not sure how to do yet.
This capability should be added to Flowise now. Are you able to view the links in Flowise using the “Return Source Documents” feature?