If I index a large “section” when indexing a document will it break it up into multiple subsections? If it doesn’t is there a good way to take a large markdown blob an break it down into subsections?
If you use the file upload API, Vectara will automatically attempt to handle basically everything for you. It will attempt to extract the text and metadata and to directly answer your question, yes, it will also attempt to break documents into appropriate subsections. Here’s an example document I uploaded to the console, where you can see Vectara has sectioned it automatically and the first result here is from section 2:
Just for completeness sake, the “standard indexing API” on the other hand, assumes that you will section documents as needed before sending them to Vectara. If you’re uploading markdown documents directly, you’ll be using the file upload API instead of the standard indexing API, so feel free to ignore this paragraph until you do some application development on non-files.