Indexing title with coreindex API

Vectara Platform Q&A
,
1

We are using coreindex API for vectara, because we were already chunking the documents ourselves and indexing API was rechunking the document. I realized that there is no way to index titles in the /coreindex API as per the API structure, only one field is indexabable which is text. Even at the document level, we can’t provide a title field.

However, in the index field, it preserves the title field both at the document and section level. We essentially need a hybrid of both of these indexing, we want to be able to index titles and corresponding text at the same time without disrupting our chunking algorithm applied. What’s the best way to handle this? During query time, we want the search results to be combination of both text and titles, which isn’t currently not working with coreindex. Using indexing API isn’t an option because the chunking its applying to document completely breaks the logic on our end

Also, it will be great to have a upsert/update API, because I just realized that titles aren’t being indexed and not showing up on query time, I now have to programmatically delete 100+ documents and re-index from scratch.

2

A good way to do this would be to make the title a separate docpart.

An even better way would be to not only make it a separate docpart, but also add metadata for that section/docpart indicating that this is a title. You can add fields like "is_title" : true to the metdata of this part. If you are feeling even more adventurous, I would suggest adding a title_level to the metadata as well indicating if it is a document level title or a doc part level title. A simple numeric value for title_level should work.

If you format it this way, you will be able to filter your results on is_title field.