Uploading integers/floats not working in filters

Vectara Platform Q&A
1

I can’t seem to figure out how to index data that get’s saved as an integer or float. I would like to do filtering on integers and floats, but this always seems to fail. When I look at the indexed data it looks like the values are always converted to strings on upload.

Create the index and set filter attribute:

def create_vectara_index(personal_api_key, customer_id):
url = “https://api.vectara.io/v1/create-corpus”

headers = {
    'Content-Type': 'application/json',
    'Accept': 'application/json',
    'x-api-key': personal_api_key
    }  

payload = json.dumps({
"corpus": {
    "name": "count_index",
    "description": "some index",
    "filterAttributes": [
        {
            "name": "some_count",
            "indexed": True,
            "type": "FILTER_ATTRIBUTE_TYPE__INTEGER",
            "level": "FILTER_ATTRIBUTE_LEVEL__DOCUMENT_PART"
        }
    ]
}
})

return requests.request("POST", url, headers=headers, data=payload)

create_vectara_index(personal_api_key, customer_id)

Upload record:

url = “https://api.vectara.io/v1/index”

payload = json.dumps({
“customerId”: customer_id,
“corpusId”: 10,
“document”: {
“documentId”: “1”,
“title”: “some title”,
“description”: “some document description”,
“metadataJson”: json.dumps({“some_count”: 10}),
“section”: [
{
“id”: “1”,
“title”: “some title”,
“text”: “a whole lot of text”
}
]
}
})
headers = {
‘Content-Type’: ‘application/json’,
‘Accept’: ‘application/json’,
‘x-api-key’: personal_api_key
}

response = requests.request(“POST”, url, headers=headers, data=payload)

View of record in Vectara platform:
image

Can you please tell me what I am missing? It looks like the integer data is being converted to the wrong type, which is causing the failure. I couldn’t find documentation to force field types during indexing or for a schema at corpus creation time.

2

Hi,

Sorry for the confusion here. While it appears that we convert things to strings, we do not actually do that in the data store. We are going to correct that API mistake in the future.

In this particular case, the filter attribute is specified to be a document part attribute. However you specified the the metadataJson at the document level, meaning that filter attribute should be at the document level. That is if you change your example from FILTER_ATTRIBUTE_LEVEL__DOCUMENT_PART to FILTER_ATTRIBUTE_LEVEL__DOCUMENT then this should work.