I have a Python script that takes a set of Vectara-formatted JSON documents and indexes them into one or more existing corpora. My script uses a rip-and-replace approach: for a given corpus it first calls https://api.vectara.io/v1/reset-corpus to clear all documents. Then it uses asyncio and aiohttp to make multiple calls to https://api.vectara.io/v1/index to reload the corpus. However, every call to /index returns a response with status.code: INSUFFICIENT_STORAGE. Furthermore, I’ve checked the corpora in the Vectara UI and they are all still empty. No documents were added by any of the /index calls. I’ve confirmed via the Vectara UI that the corpora are indeed reset: they have no data in them. And to be clear, prior to running the script, they did contain data, so the reset calls have freed up space, if anything. Fyi, I work for a company that has a corporate Vectara account and we have confirmed with Vectara (on a prior occasion when we encountered this same status code issue) that if we exceed our usage quota, then our capacity is automatically expanded as needed and we are just charged for the excess usage. So I do not understand why I’m getting this particular status code in the response, nor why the corpora remain empty. If it is helpful, there are at total of about 660 JSON documents totaling about 21MB. Thanks for any help you can offer. Best,
David
Hi @David_Schonberger
Can you please share your customerId? And the corpusId/corpusKey in which you are trying to ingest data.
That will help us figure out the exact reason.
Hello Aamir.
The customer ID I’m using is 2338743051, although I’m part of a corporate account with SonoSim, so I’m not sure if that is the appropriate customer ID. It is the ID I use in the API calls I make and have been making for months.The issue occurs across several corpora with ids: crp_50, crp_55, crp_56, crp_62. Although in recent weeks I have seen the same status code returned with at least one other corpus. So I’m not sure the issue is corpus-specific, and may instead be related to our account. In any case, thanks for looking into this. Please let me know if you have other questions.
It appears that the storage quota for your company account had exhausted. We have increased it now. Can you please try now and let us know if it works?
I tried to drag and drop some json documents into one of our corpora, Case_Studies_Library_BLUE_55, and it did not work. I received the same message as before: “Error:400 unexpected error”. I tried a single document, I tried several documents; I tried the Retry Failed File option. Unfortunately, nothing worked.
To be clear, these are JSON documents properly formatted for Vectara. We have indexed these very same documents into corpora many times before via the v1 API /index endpoint, so hopefully there is no issue with the documents themselves. There seems to be some other issue. Just so you know, I forwarded your response to our CTO who has a direct line to your support team and may open a ticket. Thanks for your help. Let me know if you need additional information. I hope we can find a solution soon.
Quick update. I tried a small run using the API and that actually worked. I do not know why the Drag and Drop failed. But in any case, I’m going to try a full pipeline run which will index about 660 documents ino the four corpora. I’ll let you know if there are any issues. If you do not hear back from me on this matter, then it workled and all is well for now. Thanks.